Agile Computing Authors: Liz McMillan, Zakia Bouachraoui, Elizabeth White, Pat Romanski, Maria C. Horton

Related Topics: Agile Computing, Containers Expo Blog, @CloudExpo

Agile Computing: Blog Feed Post

Scale Up or Scale Out?

Lots of Little Virtual Web Applications Scale Out Better than Scaling Up

Lots of little virtual Web applications scale out better than scaling up. Surprised? I was, but I shouldn’t have been.

capacity While working on other topics I ran across an interesting slide in a presentation given by Microsoft at TechEd Europe 2009 on virtualization and Exchange. Specifically the presenter called out the average 12% overhead incurred from the hypervisor on systems in internal testing. Intuitively it seems obvious that a hypervisor will incur overhead; it is, after all, an application that is executing and thus requires CPU, I/O, and RAM to perform its tasks. That led to me to wonder if there was more data on the overhead from other virtualization vendors.

I ended up reading an enlightening white paper from VMware on consolidation of web applications and virtualization in which it observes that multi-virtual configurations actually outperformed in terms of capacity and performance a server configured with a similar number of CPUs. Note that this is specifically for web applications, though I suspect that any TCP-heavy application would likely exhibit similar performance characteristics.

quotesAlthough virtualization overhead varies depending on the workload, the observed 16 percent performance degradation is an expected result when running the highly I/O‐intensive SPECweb2005 workload. But when we added the second processor, the performance difference between the two‐CPU native configuration and the virtual configuration that consisted of two virtual machines running in parallel quickly diminished to 9 percent. As we further increased the number of processors, the configuration using multiple virtual machines did not exhibit the scalability bottlenecks observed on the single native node, and the cumulative performance of the configuration with multiple virtual machines well exceeded the performance of a single native node.

-- “Consolidating Web Applications Using VMware Infrastructure” [PDF, VMware]

We know there’s overhead associated with the hypervisor. Fact. But what’s interesting here is that the overhead turns out to be irrelevant – at least in the case of web applications. What’s important is the initial degradation of performance and its subsequent improvement as additional virtual instances are added. We need to understand why that’s the case, because it has – or should have – an impact on our overall architectural strategy.















So why would multi-instances of a web server – virtual no less - scale better for performance than simply scaling out, i.e. adding more CPUs? If we look at typical performance patterns from really any TCP-connection oriented device or application, we see very similar behavior. Capacity of the device or application tends to have a steep growth curve that plateaus rather quickly and then remains somewhat constant. The associated performance pattern of such devices and applications tends to begin with very low latency and good response times, but gradually increases as capacity plateaus at or near capacity.

This pattern, with few aberrations, should be fairly recognizable to anyone who’s performed any kind of load or performance testing on a connection-oriented (TCP-based) solution. In fact, an obvious deviation from the pattern often indicates some sort of problem in the network or solution that needs to be addressed. Garbage collection processes in JavaEE application servers, for example, have traditionally been seen as regular inverse spikes in the overall number of TCP connections and CPU utilization on the host server coupled with an increase in response time as the CPU is completely utilized for a matter of microseconds while the process completes. The reason this is consistent across connection-oriented devices and applications is because they are connection oriented. Connections must be tracked, i.e. stored in memory, and subsequently accessed as messages flow across the connection. This requires RAM and, in some cases, I/O resources. As the number of connections grows, the “table” in which they are stored grows, thus increasing the amount of time necessary to “find” the connection as well as the associated resources. Too, the more connections the more serialization and locking that occurs and it is the serialization that is another primary bottleneck for the web server.

Hence, the more connections made to a given solution, the more its performance tends to degrade.

Virtualization appears to actually address this issue by limiting connection capacity by limiting available resources. On the other hand, adding more CPU and RAM will lead to higher connection capacity and thus larger connection tables which leads to a higher degradation in performance due to the increase in serialization. Rather than simply adding CPUs it would be, from a performance standpoint, probably a better option to add another virtual instance – and another, as CPUs increase – to maintain consistent capacity and a predictable performance pattern.

You need to scale up the hardware capacity, but should scale out at the virtual and application layers to optimize efficiency of the resources and maintain the end-user experience. By load balancing across multiple, smaller, homogeneous server instances you also make capacity planning much simpler because you know exactly what the capacity for a given instance will be and can use that information to prepare in advance a plan for increasing capacity on-demand. Scaling up does not offer the same consistency because capacity will be highly dependent upon the CPU and RAM provisioned as well as load.

Read the original blog entry...

More Stories By Lori MacVittie

Lori MacVittie is responsible for education and evangelism of application services available across F5’s entire product suite. Her role includes authorship of technical materials and participation in a number of community-based forums and industry standards organizations, among other efforts. MacVittie has extensive programming experience as an application architect, as well as network and systems development and administration expertise. Prior to joining F5, MacVittie was an award-winning Senior Technology Editor at Network Computing Magazine, where she conducted product research and evaluation focused on integration with application and network architectures, and authored articles on a variety of topics aimed at IT professionals. Her most recent area of focus included SOA-related products and architectures. She holds a B.S. in Information and Computing Science from the University of Wisconsin at Green Bay, and an M.S. in Computer Science from Nova Southeastern University.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

IoT & Smart Cities Stories
Moroccanoil®, the global leader in oil-infused beauty, is thrilled to announce the NEW Moroccanoil Color Depositing Masks, a collection of dual-benefit hair masks that deposit pure pigments while providing the treatment benefits of a deep conditioning mask. The collection consists of seven curated shades for commitment-free, beautifully-colored hair that looks and feels healthy.
The textured-hair category is inarguably the hottest in the haircare space today. This has been driven by the proliferation of founder brands started by curly and coily consumers and savvy consumers who increasingly want products specifically for their texture type. This trend is underscored by the latest insights from NaturallyCurly's 2018 TextureTrends report, released today. According to the 2018 TextureTrends Report, more than 80 percent of women with curly and coily hair say they purcha...
The textured-hair category is inarguably the hottest in the haircare space today. This has been driven by the proliferation of founder brands started by curly and coily consumers and savvy consumers who increasingly want products specifically for their texture type. This trend is underscored by the latest insights from NaturallyCurly's 2018 TextureTrends report, released today. According to the 2018 TextureTrends Report, more than 80 percent of women with curly and coily hair say they purcha...
We all love the many benefits of natural plant oils, used as a deap treatment before shampooing, at home or at the beach, but is there an all-in-one solution for everyday intensive nutrition and modern styling?I am passionate about the benefits of natural extracts with tried-and-tested results, which I have used to develop my own brand (lemon for its acid ph, wheat germ for its fortifying action…). I wanted a product which combined caring and styling effects, and which could be used after shampo...
The platform combines the strengths of Singtel's extensive, intelligent network capabilities with Microsoft's cloud expertise to create a unique solution that sets new standards for IoT applications," said Mr Diomedes Kastanis, Head of IoT at Singtel. "Our solution provides speed, transparency and flexibility, paving the way for a more pervasive use of IoT to accelerate enterprises' digitalisation efforts. AI-powered intelligent connectivity over Microsoft Azure will be the fastest connected pat...
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Druva is the global leader in Cloud Data Protection and Management, delivering the industry's first data management-as-a-service solution that aggregates data from endpoints, servers and cloud applications and leverages the public cloud to offer a single pane of glass to enable data protection, governance and intelligence-dramatically increasing the availability and visibility of business critical information, while reducing the risk, cost and complexity of managing and protecting it. Druva's...
BMC has unmatched experience in IT management, supporting 92 of the Forbes Global 100, and earning recognition as an ITSM Gartner Magic Quadrant Leader for five years running. Our solutions offer speed, agility, and efficiency to tackle business challenges in the areas of service management, automation, operations, and the mainframe.