Welcome!

Agile Computing Authors: Liz McMillan, Zakia Bouachraoui, Elizabeth White, Pat Romanski, Maria C. Horton

Related Topics: Agile Computing, Linux Containers, Open Source Cloud, Eclipse, Machine Learning , Apache

Agile Computing: Article

TraceView’s Oboe Symfony No. 1

The adoption of Symfony has already let it pull in some beautiful pieces of architecture

Dvořák’s Symphony No. 9 requires two oboes. Instrumenting Symfony only required one liboboe!

Here at AppNeta we’re still recovering from an immensely productive DrupalCon Portland 2013. Our partnership with Acquia and our continuing support for the contrib TraceView Drupal module made sponsoring the event a natural decision. We’re glad we made the long flight out West, as there was a lot of interest in TraceView. In fact, on the plane ride back to Boston I brought back not only three Voodoo Dozens but a stack of business cards two inches high. (I’ll leave it to you to guess which one ran out first!)

But for as much fun as I had exploring Portland’s unique culture, my favorite part of the con took place well before it. I spent a lot of time coding up improvements for TraceView’s Drupal support based on customer feedback. My personal goal was to submit enough patches to the TraceView Drupal module to warrant a version bump before DrupalCon, and thanks to the hard work of module maintainer Peter Drake, we were able to get there with room to spare. If you’re running Drupal 7, check out the changelog, and then go install it on your site!

The TraceView Drupal module now supports full-stack application tracing using drupal_http_request, including via the Apache Solr module.

I was able to code up more than just improvements in the months leading up to DrupalCon, though. If you stopped by our booth, you’ll have heard all about this, but we have a second important release to announce: we have a beta Symfony bundle and a (very!) preliminary port of our module to Drupal 8! While I had initially planned to just work on Drupal 6 and 7 improvements for TraceView, the announcement that Drupal 8′s SCOTCH and WSCII initiatives had resulted in the incorporation of Symfony components into Drupal 8 prompted me to take another look at what I’d lumped into ‘yet another PHP MVC framework’. It quickly became clear that I’d horribly misjudged Symfony. Drupal 8 hasn’t quite hit code freeze yet, but the adoption of Symfony has already let it pull in some beautiful pieces of architecture.

My favorite architectural decision in Symfony is its robust sub-request framework because of the degree of support that it lends to one of my favorite technologies, Edge Side Includes (ESI). Many large-scale sites leverage CDNs to deliver cached pages and static assets, but they don’t cope with logged-in users well because the content served to them is so heavily customized. ESI allows you to offload mostly cacheable pages to a CDN and only request the uncacheable portions of the page from the origin server, but it’s currently underutilized because most web frameworks can’t render just a portion of a page. In Symfony, though, any component of a page (such as a block or a sidebar) can be built as a sub-request for little additional developer effort. A year from now, large-scale Drupal 8 sites may entirely bypass common performance pain points by taking an ‘ESI Everywhere’ approach and leaving the CDN servers to do the work of assembling a dynamic page from a static cache entry.

Symfony had caught my interest, and my first instinct towards interesting software is to hack on it. That meant I had to create a Symfony site, so naturally, I chose Drupal 8 as a starting point. The codebase is still in flux, but it’s stable enough to install and it already makes use of several core Symfony components. All that I had to do to get an environment running was spin up an Amazon EC2 server, set up a LAMP stack and install TraceView (by far the fastest part of the process). The Drupal 8 installation required installing the trunk version of Drush, but after that I was only a `drush qd` away from a functioning Drupal 8 site.

I hit a wall fairly quickly, since like most Drupal developers I’d never tried writing Symfony code or using Composer to manage packages. Instead of getting frustrated, I took a step back and began to read articles about both Symfony in its own right and how it’s being leveraged to rewrite Drupal. I made heavy use of the Drupal core issue queue and the Symfony API documentation, but I also turned to blogs written by Drupal and Symfony developers. Of particular note, chx wrote an introduction to Symfony in two articles as well as an article that helped me fix my first Drupal 8 performance issue:

Every request to the Drupal 8 server triggered well over a hundred `file_get_contents` calls.

Before writing any integration code, I pulled up some Symfony traces because I was curious what they would look like. Even without code modification, TraceView picked up on Drupal 8 making a huge number of `file_get_contents` calls on each request. While individually fast, the calls often added up to more than 100ms; worse yet, that many disk reads has severe negative implications for scalability. At my last job, a sudden increase in traffic took down our entire distributed filesystem just by repeatedly fetching a single JavaScript file!

When I examined the `file_get_contents` calls in TraceView and realized they were all fetching module-related files, I immediately made the connection to Symfony’s dependency injection container (DIC) (one of its major design paradigms) thanks to another post by chx about how the DIC is stored on the filesystem. The filesystem calls were due to Drupal not finding the DIC on the filesystem and instead recompiling it using a dependency-checking mechanism similar to a package manager but made more complicated by the ability of classes to define additional compiler passes and other overrides. Like most problems I run into on Linux, I was able to trace it back to incorrect permissions:  because I had run Drush as a normal user, I had changed the permissions on the relevant directory and prevented the webserver from writing the compiled DIC so that it could be loaded on future requests. One `chown` later and my testbed site was running more than five times faster.

I kept an open mind to Symfony as I continued to read, but coming from the perspective of a Drupal developer, I was quite surprised to hear that the familiar hook-based event system has already been partially replaced with Symfony’s event listeners (bringing it more in line with some other PHP frameworks that we support). It’s new territory for me, but I can understand the rationale as there are some serious advantages to the switch. Unlike hooks, events don’t need to check every module on the site for an implementation; they can also have their propagation stopped if no further work is necessary. This means that developers will have much finer control over how modules respond to events, rather than having to use unusual hooks simply because they get called at the ‘right’ point in page loading (I’m looking at you, `hook_footer`!). Interestingly, a total switchover has been postponed until Drupal 9, which means that these overlapping systems will both remain in place for the foreseeable future.

The differences between Drupal hooks and Symfony event listeners were quite relevant to building out TraceView support, so I spent a lot of time reading about them. It’s there that I stumbled across another excellent addition to Drupal 8: the `terminate` kernel event! This import from Symfony is the replacement for the recently removed `hook_exit`, with the key difference being that it can be run after a response is returned to a browser. This lets expensive cache setting operations be performed in a way that won’t harm end user performance, emails be fired off only when the request is complete, or disk writes take place after an upload confirmation screen has been shown to the user.

It took about a month to get up to speed on Drupal 8 and Symfony to a degree where I felt ready to stop reading and start writing. That left me with about a month to go before DrupalCon, and in that time I was able to turn out not only the Drupal 7 module improvements but a really awesome DrupalCon booth demo!

The top tier of Drupalception included both our beta TraceView Drupal 8 module and our TraceView Symfony bundle.

I couldn’t think of any better way to show that we’re serious about our Drupal support than to demonstrate every version of Drupal we support at once, and so I built Drupalception: a Drupal 8 + Symfony AWS instance using cURL (via Guzzle) to call into our Drupal 7 demo site, which in turn uses drupal_http_request to call into our Drupal 6 demo site. TraceView catches the entire stack as a single trace even when blocking calls are nested fifteen layers deep!

But wait, aren’t I skipping a step – how did I get from reading a bunch of blog articles about Drupal 8 in Symfony to a functioning Drupal 8 module and Symfony bundle? Well, that’s a story in its own right – and for now, one that you’ll have to wait for another blog post for! Until next time, sign up for our 14 day trial and try out firsthand our support for Drupal 6, 7, or even 8 (or any other stack we support, for that matter).

Related Articles

Drupal Performance Profiling With Tracelytics

Headnet Improves Drupal Performance and Reliability With TraceView

Case Study: Penton Media Improves its Drupal Application Performance

More Stories By James Meickle

James started as a hobbyist web developer, even though his academic background is in social psychology and political science. Lately his interests as a professional Drupal developer have migrated towards performance, security, and automation. His favorite languages is Python, his favorite editor is Sublime, and his favorite game is Dwarf Fortress.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


IoT & Smart Cities Stories
Moroccanoil®, the global leader in oil-infused beauty, is thrilled to announce the NEW Moroccanoil Color Depositing Masks, a collection of dual-benefit hair masks that deposit pure pigments while providing the treatment benefits of a deep conditioning mask. The collection consists of seven curated shades for commitment-free, beautifully-colored hair that looks and feels healthy.
The textured-hair category is inarguably the hottest in the haircare space today. This has been driven by the proliferation of founder brands started by curly and coily consumers and savvy consumers who increasingly want products specifically for their texture type. This trend is underscored by the latest insights from NaturallyCurly's 2018 TextureTrends report, released today. According to the 2018 TextureTrends Report, more than 80 percent of women with curly and coily hair say they purcha...
The textured-hair category is inarguably the hottest in the haircare space today. This has been driven by the proliferation of founder brands started by curly and coily consumers and savvy consumers who increasingly want products specifically for their texture type. This trend is underscored by the latest insights from NaturallyCurly's 2018 TextureTrends report, released today. According to the 2018 TextureTrends Report, more than 80 percent of women with curly and coily hair say they purcha...
We all love the many benefits of natural plant oils, used as a deap treatment before shampooing, at home or at the beach, but is there an all-in-one solution for everyday intensive nutrition and modern styling?I am passionate about the benefits of natural extracts with tried-and-tested results, which I have used to develop my own brand (lemon for its acid ph, wheat germ for its fortifying action…). I wanted a product which combined caring and styling effects, and which could be used after shampo...
The platform combines the strengths of Singtel's extensive, intelligent network capabilities with Microsoft's cloud expertise to create a unique solution that sets new standards for IoT applications," said Mr Diomedes Kastanis, Head of IoT at Singtel. "Our solution provides speed, transparency and flexibility, paving the way for a more pervasive use of IoT to accelerate enterprises' digitalisation efforts. AI-powered intelligent connectivity over Microsoft Azure will be the fastest connected pat...
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Druva is the global leader in Cloud Data Protection and Management, delivering the industry's first data management-as-a-service solution that aggregates data from endpoints, servers and cloud applications and leverages the public cloud to offer a single pane of glass to enable data protection, governance and intelligence-dramatically increasing the availability and visibility of business critical information, while reducing the risk, cost and complexity of managing and protecting it. Druva's...
BMC has unmatched experience in IT management, supporting 92 of the Forbes Global 100, and earning recognition as an ITSM Gartner Magic Quadrant Leader for five years running. Our solutions offer speed, agility, and efficiency to tackle business challenges in the areas of service management, automation, operations, and the mainframe.