Welcome!

Agile Computing Authors: Liz McMillan, Pat Romanski, Elizabeth White, John Basso, Kevin Benedict

Related Topics: Microservices Expo, Java IoT, Agile Computing, Cloud Security, Government Cloud

Microservices Expo: Article

Why Obama Administration Should Have Paid More Attention to Load Testing

What needs to be understood here is that it’s important to test early and often

October 1, 2013, was the most anticipated date for the Obama administration since his re-election. It was to be the day every American would have access to health care on one centralized website. However, according to at least one report only six people enrolled in Obamacare on the first day. Then shortly after, the entire website crashed along with its infrastructure.

The massive crash happened because within the first 10 days of launch HealthCare.gov had over 14.6 million unique views. Something the Obama administration was not prepared for, nor the testers.

The website should have been able to handle tens of thousands of people at once, but in a trial test before the launch a mere 500 users caused the website to crash. In testimony before U.S. Congress, the contractors responsible for HealthCare.gov said they didn't have enough time to fully test the website. The inability to properly load test the website well before the launch date of October 1st led to one of the worst federal website debacles of all time.

What Went Wrong
The HealthCare.gov website was designed to provide Americans with a simple solution as a one-stop-shop for health care insurance, but as we all know it wasn't that simple.

The site was built by 55 contractors and is considered one of the most complex software projects ever undertaken for the federal government, which might be where their problems all started.

According to Louis Woodhill, a contributor to Forbes magazine, the Obamacare website is comparable to the Soviet Union. "In their effort to build an IT system to implement Obamacare, the U.S Department of Health and Human Services was trying to do the same thing as the USSR's Gosplan agency: elicit coordinated, purposeful action from a collection of entities that don't know each other, don't trust each other, have conflicting objectives, and face diverging incentives."

Mixing contractors wasn't their only issue, the Obama administration continued to make a series of rookie mistakes that led to the demise of the website.

Incorrectly Assessing User Behavior. First, the administrators in charge of the website decided in late September to exclude the feature that would let people shop for health plans before registering for an online account. This lead to a bottleneck in the process because more people than expected had to go through the registration process before they could even browse through plans.

Broken Systems Integration. Second, the registration process was flawed. The consumer was supposed to enter basic account information, a security question and so on, but the communication between the systems responsible for storing this information wasn't working properly. This resulted in thousands of users who were unable to successfully create an account.

Rebuilding Components from Scratch When Proven Systems Were Available. Last, the Data Services Hub, which is a proven identity service available to the government for consumer applications, was surprisingly not used to its full extent. Instead, the website builders created new software systems meant to do exactly the same thing. In an article by Mashable the author emphasizes the fact that if the HealthCare.gov site had in fact fully leveraged the Data Hub, then it wouldn't have been such a mess.

With all of these missteps and rookie mistakes under consideration, what is known is the fact that HealthCare.gov was overwhelmed with the amount of visitors to one site.

Why the Government Should Have Made Load Testing a Priority
It seems like those responsible for deploying the site didn't really appreciate the importance of load testing, which is especially surprising when you consider that the website had in fact failed a pre-launch load test miserably. Of course, politics came into play as the deadline for the website was non-negotiable. But with all the red flags warning of failure, load testing should have played a much more critical role and here's why:

Prioritization of Problems and Fixes
A big issue with HealthCare.gov was that the contractors claimed they didn't have enough time and felt extreme pressure to roll out the website before it was properly tested. If load testing occurred earlier in the website development phase, testers would have been able to identify the parts of the website that were not working properly.

The major pain point in the entire HealthCare.gov website was the registration process that millions of Americans attempted to fill out. Had they load tested the website months out from the launch, the team would have been able to identify the root causes of performance issues and determine whether they were in application code or the app servers and infrastructure components.

Earlier Identification of Issues

 

This chart illustrates how much it costs the paying client to fix a bug according to the stage of development. At the operation stage, a bug can cost clients more than 150 times as much as a bug caught in the requirement stage.

Had the testers broken down their tests into smaller test cases, over time the administration might have taken the time to listen and understand that these little bugs needed to be fixed prior to the public launch.

Decisions Made from Intelligence on the Ground
We know the tension between testers and business owners can be pretty intense. The funders of the website want it up and running right away, but testers want to properly identify errors and have enough time to fix the issues that arise.

The administration decided to completely ignore the classic project management triangle.

The only way to increase the scope of a project without changing the due date would be to add more resources. Since the administration was rigid on all three sides of the triangle, the quality of the website suffered.

It's no wonder this website failed. The dynamics between the testers and heads of HealthCare.gov were strained, and it appeared the Obama administration chose to ignore testers who knew the website was not ready.

HealthCare.gov Today
The HealthCare.gov website isn't through the woods just yet. According to The Washington Post, the website has been flagged by over 22,000 people trying to correct errors the system made when they were signing up for a new federally-mandated health care plan.

Apparently, federal workers aren't able to access consumer data manually. "An unknown number of customers who are trying to get help through less formal means - by calling the health care marketplace directly - are told that HealthCare.gov's computer system isn't yet allowing federal workers to go into enrollment records and change them."

What needs to be understood here is that it's important to test early and often. If tests would have been conducted throughout the entire website development, the Obama administration would have avoided such an embarrassing and reputation-tarnishing event.

More Stories By Tim Hinds

Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices.

Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
Ask someone to architect an Internet of Things (IoT) solution and you are guaranteed to see a reference to the cloud. This would lead you to believe that IoT requires the cloud to exist. However, there are many IoT use cases where the cloud is not feasible or desirable. In his session at @ThingsExpo, Dave McCarthy, Director of Products at Bsquare Corporation, will discuss the strategies that exist to extend intelligence directly to IoT devices and sensors, freeing them from the constraints of ...
You think you know what’s in your data. But do you? Most organizations are now aware of the business intelligence represented by their data. Data science stands to take this to a level you never thought of – literally. The techniques of data science, when used with the capabilities of Big Data technologies, can make connections you had not yet imagined, helping you discover new insights and ask new questions of your data. In his session at @ThingsExpo, Sarbjit Sarkaria, data science team lead ...
Extracting business value from Internet of Things (IoT) data doesn’t happen overnight. There are several requirements that must be satisfied, including IoT device enablement, data analysis, real-time detection of complex events and automated orchestration of actions. Unfortunately, too many companies fall short in achieving their business goals by implementing incomplete solutions or not focusing on tangible use cases. In his general session at @ThingsExpo, Dave McCarthy, Director of Products...
Traditional IT, great for stable systems of record, is struggling to cope with newer, agile systems of engagement requirements coming straight from the business. In his session at 18th Cloud Expo, William Morrish, General Manager of Product Sales at Interoute, outlined ways of exploiting new architectures to enable both systems and building them to support your existing platforms, with an eye for the future. Technologies such as Docker and the hyper-convergence of computing, networking and sto...
WebRTC is bringing significant change to the communications landscape that will bridge the worlds of web and telephony, making the Internet the new standard for communications. Cloud9 took the road less traveled and used WebRTC to create a downloadable enterprise-grade communications platform that is changing the communication dynamic in the financial sector. In his session at @ThingsExpo, Leo Papadopoulos, CTO of Cloud9, discussed the importance of WebRTC and how it enables companies to focus...
With an estimated 50 billion devices connected to the Internet by 2020, several industries will begin to expand their capabilities for retaining end point data at the edge to better utilize the range of data types and sheer volume of M2M data generated by the Internet of Things. In his session at @ThingsExpo, Don DeLoach, CEO and President of Infobright, discussed the infrastructures businesses will need to implement to handle this explosion of data by providing specific use cases for filterin...
IoT generates lots of temporal data. But how do you unlock its value? You need to discover patterns that are repeatable in vast quantities of data, understand their meaning, and implement scalable monitoring across multiple data streams in order to monetize the discoveries and insights. Motif discovery and deep learning platforms are emerging to visualize sensor data, to search for patterns and to build application that can monitor real time streams efficiently. In his session at @ThingsExpo, ...
Early adopters of IoT viewed it mainly as a different term for machine-to-machine connectivity or M2M. This is understandable since a prerequisite for any IoT solution is the ability to collect and aggregate device data, which is most often presented in a dashboard. The problem is that viewing data in a dashboard requires a human to interpret the results and take manual action, which doesn’t scale to the needs of IoT.
Internet of @ThingsExpo has announced today that Chris Matthieu has been named tech chair of Internet of @ThingsExpo 2016 Silicon Valley. The 6thInternet of @ThingsExpo will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Much of IT terminology is often misused and misapplied. Modernization and transformation are two such terms. They are often used interchangeably even though they mean different things and have very different connotations. Indeed, it is somewhat safe to assume that in IT any transformative effort is likely to also have a modernizing effect, and thus, we can see these as levels of improvement efforts. However, many businesses are being led to believe if they don’t transform now they risk becoming ...
CenturyLink has announced that application server solutions from GENBAND are now available as part of CenturyLink’s Networx contracts. The General Services Administration (GSA)’s Networx program includes the largest telecommunications contract vehicles ever awarded by the federal government. CenturyLink recently secured an extension through spring 2020 of its offerings available to federal government agencies via GSA’s Networx Universal and Enterprise contracts. GENBAND’s EXPERiUS™ Application...
What does it look like when you have access to cloud infrastructure and platform under the same roof? Let’s talk about the different layers of Technology as a Service: who cares, what runs where, and how does it all fit together. In his session at 18th Cloud Expo, Phil Jackson, Lead Technology Evangelist at SoftLayer, an IBM company, spoke about the picture being painted by IBM Cloud and how the tools being crafted can help fill the gaps in your IT infrastructure.
SYS-CON Events announced today the Enterprise IoT Bootcamp, being held November 1-2, 2016, in conjunction with 19th Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA. Combined with real-world scenarios and use cases, the Enterprise IoT Bootcamp is not just based on presentations but with hands-on demos and detailed walkthroughs. We will introduce you to a variety of real world use cases prototyped using Arduino, Raspberry Pi, BeagleBone, Spark, and Intel Edison. Y...
SYS-CON Events announced today that LeaseWeb USA, a cloud Infrastructure-as-a-Service (IaaS) provider, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. LeaseWeb is one of the world's largest hosting brands. The company helps customers define, develop and deploy IT infrastructure tailored to their exact business needs, by combining various kinds cloud solutions.
The best-practices for building IoT applications with Go Code that attendees can use to build their own IoT applications. In his session at @ThingsExpo, Indraneel Mitra, Senior Solutions Architect & Technology Evangelist at Cognizant, provided valuable information and resources for both novice and experienced developers on how to get started with IoT and Golang in a day. He also provided information on how to use Intel Arduino Kit, Go Robotics API and AWS IoT stack to build an application tha...
It’s 2016: buildings are smart, connected and the IoT is fundamentally altering how control and operating systems work and speak to each other. Platforms across the enterprise are networked via inexpensive sensors to collect massive amounts of data for analytics, information management, and insights that can be used to continuously improve operations. In his session at @ThingsExpo, Brian Chemel, Co-Founder and CTO of Digital Lumens, will explore: The benefits sensor-networked systems bring to ...
Whether your IoT service is connecting cars, homes, appliances, wearable, cameras or other devices, one question hangs in the balance – how do you actually make money from this service? The ability to turn your IoT service into profit requires the ability to create a monetization strategy that is flexible, scalable and working for you in real-time. It must be a transparent, smoothly implemented strategy that all stakeholders – from customers to the board – will be able to understand and comprehe...
Identity is in everything and customers are looking to their providers to ensure the security of their identities, transactions and data. With the increased reliance on cloud-based services, service providers must build security and trust into their offerings, adding value to customers and improving the user experience. Making identity, security and privacy easy for customers provides a unique advantage over the competition.
SYS-CON Events announced today that Venafi, the Immune System for the Internet™ and the leading provider of Next Generation Trust Protection, will exhibit at @DevOpsSummit at 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Venafi is the Immune System for the Internet™ that protects the foundation of all cybersecurity – cryptographic keys and digital certificates – so they can’t be misused by bad guys in attacks...
"Tintri was started in 2008 with the express purpose of building a storage appliance that is ideal for virtualized environments. We support a lot of different hypervisor platforms from VMware to OpenStack to Hyper-V," explained Dan Florea, Director of Product Management at Tintri, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.