Welcome!

Agile Computing Authors: Liz McMillan, Pat Romanski, Elizabeth White, Dana Gardner, Andy Thurai

Related Topics: @DevOpsSummit, Agile Computing, @DXWorldExpo

@DevOpsSummit: Blog Feed Post

Transaction Tracing for Performance Bottleneck Detection | @DevOpsSummit #DevOps

Transaction Tracing refers to end-to-end request code execution

Transaction Tracing for Performance Bottleneck Detection
by Mick Emmett

When you’re building a monitoring solution or evaluate existing ones, what do you look for? Probably these four core aspects of functionality:

  1. Collection and display of metrics
  2. Alerting based on metric values and anomalies
  3. Collection and display of server and application logs and other types of events
  4. Alerting based on log patterns and metrics extracted from logs

But there is really one more juicy piece of functionality one should look for:

  • Distributed Transaction Tracing

What’s more, as a nice side-effect of Transaction Tracing in SPM you will also get:

  • Your app’s Request Throughput, Response Latency, plus Error & Exception Rates
  • AppMap, which which how various components in your infrastructure communicate with each other

What is Distributed Transaction Tracing?
[Note: If you know what Transaction Tracing is and what the benefits are, you can jump right to the bottom of this post to see how to enable it.]

When we say transaction in this context we are not referring to database transactions.  In the context of Application Performance Monitoring, Transaction Tracing refers to end-to-end request code execution that can span multiple applications running and multiple servers.

For example, when you shop on Amazon.com and search for a product there, your search action starts a request transaction that can be traced though various layers of Amazon.com’s application and server infrastructure, all the way down to the database and back.  That is what SPM’s Transaction Tracing can do, too, for your own applications and servers.

Where Transaction Tracing fits in
Let’s start with a simple diagram. The little “SPM agent” box is the embedded agent responsible for tracing all transactions and capturing information about them.  Everything in orange and all red arrows would be traced by SPM’s agent, even as transactions go across networks and hop from server to server – hence Distributed Transaction Tracing.

TT_2

Now, let’s quickly look at a slightly more complex diagram:

TT_3

Here we also see an external HTTP service (e.g. third party REST API) our application talks to via HTTP.  That, too, is captured, along with all orange components and red arrows.  We can also see Service1 has multiple internal classes and methods.  SPM traces execution deep inside your code, too, so it can tell you which methods in which classes are slow.

How / Why is Transaction Tracing useful?
In short, Transaction Tracing is great for:

  • Pinpointing root causes of poor application performance, so you can fix ‘em and make your users happy and your mom and dad proud!
  • Finding the slowest parts of your application, so you can speed up troubleshooting and get the most return for your development time

Have a look at the following chart and you’ll be able to relate.  It shows the response time for one of our real components deployed in production.  You can see how response time improved at one point, right?

We enabled Transaction Tracing to find and fix a bottleneck in our own code.  In short, we went from the 99th percentile response time for one of the components being 1-3 seconds to about 0.15-0.30 seconds.  That’s about 10x faster!

TT_5

Where & How to enable Transaction Tracing
If you’ve been using SPM you are likely to use one of its integrations for monitoring your backend infrastructure – Apache Spark, Solr, Storm, Elasticsearch, Hadoop, Cassandra, Kafka, that sort of backend stuff. As you can see from the above diagrams, transaction tracing starts closer to the front-end — in the tier that runs client applications which talk to backend and remote services, as well as data stores like Solr, Elasticsearch, Cassandra, HBase, MySQL, and data processing frameworks like Apache Spark or Kafka, or even Hadoop.

In other words, to get the full benefit of Transaction Tracing you need to do the following:

  • If your webapps run in Tomcat create a new SPM App for Tomcat.  If you use Jetty, Glassfish, JBoss, or some other Java container simply create an SPM App for JVM.
  • Add SPM Agent to your webapp tier (e.g. to your webapps running in Tomcat, Jetty, Glassfish, JBoss, and similar Java containers).  Use the Embedded (aka javaagent), not Standalone mode.
  • Enable Transaction Tracing in SPM Agent configs in all your tiers

And that’s it.  Once you have Transaction Tracing enabled look under the new Transactions tab in SPM:

TT_1a

In the coming days, we’ll be publishing more information about Transaction Tracing in SPM, including how-tos, additional functionality derived from Transaction Tracing, etc.

In the meantime, here are the key points:

  • Transaction Tracing does not require you to modify any source code – the instrumentation is done automatically, at the JVM bytecode level
  • Transaction Tracing is currently available for Java and Scala apps running inside the JVM
  • Transaction Tracing is exposed via the new “Transactions” report (tab) on the left side in the UI.  That said, the key bit to understand is that Transaction Tracing is really means for finding bottlenecks in your applications and how they talk to backend services, frameworks, and external services, not in the backend applications your applications connect to. See the diagram further above.
  • We support deep insight into specific technologies listed in SPM Transaction Tracing documentation
  • You’ll want to grab the latest version of the SPM client (it has some optimizations, too!)
  • You’ll need to use the SPM monitor in the embedded (aka javaagent) mode, not standalone
  • To add Transaction Tracing to your own custom apps you can easily create a custom pointcut

Not using SPM yet, but would like to trace your apps? Just register here.  It doesn’t hurt, doesn’t require signing your life or kids away, there is no commitment, and you can leave your credit card in your wallet.  You get 30 days Free for new SPM Apps.  You can invite your colleagues, so you can collaboratively monitor your app and find those pesky bottlenecks!  Oh, and if you are a young startup, a small or non-profit organization, or an educational institution, ask us for a discount (see special pricing)!

Filed under: Monitoring Tagged: performance monitoring, spm, Transaction tracing

Read the original blog entry...

More Stories By Sematext Blog

Sematext is a globally distributed organization that builds innovative Cloud and On Premises solutions for performance monitoring, alerting and anomaly detection (SPM), log management and analytics (Logsene), and search analytics (SSA). We also provide Search and Big Data consulting services and offer 24/7 production support for Solr and Elasticsearch.

IoT & Smart Cities Stories
The hierarchical architecture that distributes "compute" within the network specially at the edge can enable new services by harnessing emerging technologies. But Edge-Compute comes at increased cost that needs to be managed and potentially augmented by creative architecture solutions as there will always a catching-up with the capacity demands. Processing power in smartphones has enhanced YoY and there is increasingly spare compute capacity that can be potentially pooled. Uber has successfully ...
Cloud computing delivers on-demand resources that provide businesses with flexibility and cost-savings. The challenge in moving workloads to the cloud has been the cost and complexity of ensuring the initial and ongoing security and regulatory (PCI, HIPAA, FFIEC) compliance across private and public clouds. Manual security compliance is slow, prone to human error, and represents over 50% of the cost of managing cloud applications. Determining how to automate cloud security compliance is critical...
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities - ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups.
Disruption, Innovation, Artificial Intelligence and Machine Learning, Leadership and Management hear these words all day every day... lofty goals but how do we make it real? Add to that, that simply put, people don't like change. But what if we could implement and utilize these enterprise tools in a fast and "Non-Disruptive" way, enabling us to glean insights about our business, identify and reduce exposure, risk and liability, and secure business continuity?
The Internet of Things is clearly many things: data collection and analytics, wearables, Smart Grids and Smart Cities, the Industrial Internet, and more. Cool platforms like Arduino, Raspberry Pi, Intel's Galileo and Edison, and a diverse world of sensors are making the IoT a great toy box for developers in all these areas. In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists discussed what things are the most important, which will have the most profound e...
Chris Matthieu is the President & CEO of Computes, inc. He brings 30 years of experience in development and launches of disruptive technologies to create new market opportunities as well as enhance enterprise product portfolios with emerging technologies. His most recent venture was Octoblu, a cross-protocol Internet of Things (IoT) mesh network platform, acquired by Citrix. Prior to co-founding Octoblu, Chris was founder of Nodester, an open-source Node.JS PaaS which was acquired by AppFog and ...
In today's enterprise, digital transformation represents organizational change even more so than technology change, as customer preferences and behavior drive end-to-end transformation across lines of business as well as IT. To capitalize on the ubiquitous disruption driving this transformation, companies must be able to innovate at an increasingly rapid pace.
Predicting the future has never been more challenging - not because of the lack of data but because of the flood of ungoverned and risk laden information. Microsoft states that 2.5 exabytes of data are created every day. Expectations and reliance on data are being pushed to the limits, as demands around hybrid options continue to grow.
"MobiDev is a Ukraine-based software development company. We do mobile development, and we're specialists in that. But we do full stack software development for entrepreneurs, for emerging companies, and for enterprise ventures," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
The explosion of new web/cloud/IoT-based applications and the data they generate are transforming our world right before our eyes. In this rush to adopt these new technologies, organizations are often ignoring fundamental questions concerning who owns the data and failing to ask for permission to conduct invasive surveillance of their customers. Organizations that are not transparent about how their systems gather data telemetry without offering shared data ownership risk product rejection, regu...