Click here to close now.



Welcome!

Agile Computing Authors: Nicholas Lee, Ian Khan, Jayaram Krishnaswamy, Elizabeth White, Cloud Best Practices Network

Related Topics: @CloudExpo, Java IoT, Microservices Expo, Agile Computing, @BigDataExpo, SDN Journal

@CloudExpo: Article

The Outside-In Battle for the Soul of the Cloud

The clouds that can best adapt to the demands of the workloads they are supporting will be best positioned for success

Whether they admit it or not, the emergence of public cloud providers has dramatically altered the playing field for hardware vendors of every type. Amazon Web Services (AWS) and its competitors opened Pandora's box by introducing the world to a completely programmatic, scalable, evolving, and pay-as-you-go way to procure and utilize network, compute and storage resources on a global scale. They have disrupted many layers of the technology industry from the applications being written to the way companies interact with the infrastructure being used to support those applications.

Nowhere is this disruption easier to see than in the virtualization ecosystem. For the better part of the last decade, hypervisor companies like VMware, Citrix, Microsoft and Red Hat worked hand-in-hand with hardware manufacturers like Cisco, NetApp, EMC, HP and Dell to define both the infrastructure foundation as well as the virtualized abstraction layer that sat underneath the entirety of the client/server era. These companies provided a direct link between the enterprise applications, the hypervisor and the hardware. They owned the traditional datacenter construct.

It's that construct, since rebranded as "private cloud," that is directly under attack by public cloud providers. I predict that this will be the battlefield for the heart and soul of enterprise IT for the next decade.

The response to the public cloud threat has been varied, and often reflects the ability of traditional companies to pivot and meet the challenge. Interestingly, erstwhile competitors Microsoft and VMware reacted similarly. This is because they were both uniquely positioned to create a software-defined solution to the problem.

For both companies, the response started with existing enterprise workloads. One of the largest challenges of the AWS public cloud is the fact that getting workloads, and especially data, into and out of an enterprise environment can be both technically challenging and expensive. Most workloads running on an enterprise-virtualized platform today can't be easily ported into AWS and this increases the cost and risk of any migration. As companies with extensive and hard-won experience running mission-critical enterprise workloads, Microsoft and VMware came to much the same conclusion: build a public cloud using their existing platform and allow customers and developers to leverage all of the investment they've made in their own data centers as they selectively move workloads outside of their own data centers. Thus, Microsoft Azure and VMware vCHS were born. Both are clouds that customers can move workloads to without the need to rewrite or re-architect them. They can also be licensed using existing agreements and can be managed by existing staff and tools.

Unfortunately, the traditional data center infrastructure is now the weak link in this new software-defined world. In each of the public clouds referenced, the focus has been on the abstraction layer and how it interacts with the end users. What's missing is how the abstraction layer and the applications and tools that sit on top of it interact with the infrastructure directly.

There have been attempts at hardware-based offloading, especially with regards to storage. VAAI is a good example of VMware trying to create a way to let enterprise storage arrays handle the tasks they are good at without requiring the direct involvement of the hypervisor. But even there it's a rudimentary exchange at best: the hypervisor asks "can you do this task instead of me?" and the array responds. If the answer is yes, the hypervisor waits for the task to complete; if the answer is no, the hypervisor does the task itself. This relationship isn't dynamic, and is ignorant of the reason for and context behind the task in the first place.

In summary, we have an outside force, AWS and public cloud, being the primary catalyst driving change into the enterprise, yet very little of that change is happening below the cloud management or hypervisor layer. Why is that? Why is it important that the infrastructure layer become more of an asset to the rest of the stack? What would that look like? Let's dig in.

The question of why is actually pretty simple: it's really, really hard to take legacy hardware architecture and retrofit it into something agile and programmatic. In some cases, it's just a new concept that requires a hardware refresh (like Cisco UCS and its take on XML-defined BIOS policies), but in many cases, especially around storage, it requires a complete reimagining of the platform. It's no coincidence that most of the innovation in this agile infrastructure space is being done by startups who have no legacy customers, technical debt or margins to deal with.

Why is it important? While the best hardware is boring hardware, it's still a critical part to providing a flexible, reliable and high-performance foundation to handle applications that matter to enterprises. There are times where the best way to handle the demands of an application or, more important, multiple applications at once is in hardware. This is true at the network layer, where the manipulation of packets benefits from proximity to processing resources; the compute layer, where apps can benefit from having specialized GPU resources to handle unique requirements; and most especially at the storage layer.

Storage services can have the most dramatic impact on workload performance, yet are often implemented in such a way that they have no direct relationship with those workloads. Services like compression, deduplication and quality-of-service are usually "on or off" features when it comes to storage arrays. Best case, a storage administrator will create a volume or LUN, choose the features that need to be enabled, and then a virtualization admin will map that volume to a data store. Perhaps the virtualization team will create manual storage profiles that define the features offered by that data store, but placing and migrating VMs remains a manual process, and they will not have the ability to map application policy equally across the hypervisor and hardware layers. (Of course, it's not impossible to create programmatic, hypervisor-aware infrastructure, but it is pretty hard.)

Enterprises have come to expect some fundamental features from the public cloud space: simple architecture, linear scaling, API availability and granular application of services. These features allow an infrastructure to respond to the increased requirements of a workload natively, without the overhead of a bolt-on orchestration engine. They provide the ability for the hypervisor to be both a northbound and southbound policy enforcer. They enable the Next-Generation Data Center, one in which the hardware, the hypervisor and the application all play an integrated, coordinated role in providing the performance and availability demanded by the enterprise.

No matter where your workloads run, the rise of public cloud has ushered in an era of computing defined by a seamless, programmatic experience. The old, monolithic infrastructure of yesterday's client/server wave is giving way to a more agile, more responsive, more services-rich and more scalable cloud-based model. The battle for the enterprise soul is beginning and, inside or outside the firewall, the clouds that can best adapt to the demands of the workloads they are supporting will be best positioned for success.

More Stories By Jeramiah Dooley

Jeramiah Dooley joined the SolidFire team as a Cloud Architect on the Technology Solutions team. Prior to SolidFire he was most recently at VCE and before that Peak 10. You can check out his Virtualization for Service Providers blog or follow him on twitter @jdooley_clt.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
As enterprises work to take advantage of Big Data technologies, they frequently become distracted by product-level decisions. In most new Big Data builds this approach is completely counter-productive: it presupposes tools that may not be a fit for development teams, forces IT to take on the burden of evaluating and maintaining unfamiliar technology, and represents a major up-front expense. In his session at @BigDataExpo at @ThingsExpo, Andrew Warfield, CTO and Co-Founder of Coho Data, will dis...
SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry's single source for the cloud. Fusion's advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including clou...
With the Apple Watch making its way onto wrists all over the world, it’s only a matter of time before it becomes a staple in the workplace. In fact, Forrester reported that 68 percent of technology and business decision-makers characterize wearables as a top priority for 2015. Recognizing their business value early on, FinancialForce.com was the first to bring ERP to wearables, helping streamline communication across front and back office functions. In his session at @ThingsExpo, Kevin Roberts...
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management...
SYS-CON Events announced today that Alert Logic, Inc., the leading provider of Security-as-a-Service solutions for the cloud, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Alert Logic, Inc., provides Security-as-a-Service for on-premises, cloud, and hybrid infrastructures, delivering deep security insight and continuous protection for customers at a lower cost than traditional security solutions. Ful...
SYS-CON Events announced today that VAI, a leading ERP software provider, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. VAI (Vormittag Associates, Inc.) is a leading independent mid-market ERP software developer renowned for its flexible solutions and ability to automate critical business functions for the distribution, manufacturing, specialty retail and service sectors. An IBM Premier Business Part...
The cloud promises new levels of agility and cost-savings for Big Data, data warehousing and analytics. But it’s challenging to understand all the options – from IaaS and PaaS to newer services like HaaS (Hadoop as a Service) and BDaaS (Big Data as a Service). In her session at @BigDataExpo at @ThingsExpo, Hannah Smalltree, a director at Cazena, will provide an educational overview of emerging “as-a-service” options for Big Data in the cloud. This is critical background for IT and data profes...
With an estimated 50 billion devices connected to the Internet by 2020, several industries will begin to expand their capabilities for retaining end point data at the edge to better utilize the range of data types and sheer volume of M2M data generated by the Internet of Things. In his session at @ThingsExpo, Don DeLoach, CEO and President of Infobright, will discuss the infrastructures businesses will need to implement to handle this explosion of data by providing specific use cases for filte...
Fortunately, meaningful and tangible business cases for IoT are plentiful in a broad array of industries and vertical markets. These range from simple warranty cost reduction for capital intensive assets, to minimizing downtime for vital business tools, to creating feedback loops improving product design, to improving and enhancing enterprise customer experiences. All of these business cases, which will be briefly explored in this session, hinge on cost effectively extracting relevant data from ...
SYS-CON Events announced today that Interoute, owner-operator of one of Europe's largest networks and a global cloud services platform, has been named “Bronze Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2015 at the Javits Center in New York, New York. Interoute is the owner-operator of one of Europe's largest networks and a global cloud services platform which encompasses 12 data centers, 14 virtual data centers and 31 colocation centers, with connections to 195 ad...
Most people haven’t heard the word, “gamification,” even though they probably, and perhaps unwittingly, participate in it every day. Gamification is “the process of adding games or game-like elements to something (as a task) so as to encourage participation.” Further, gamification is about bringing game mechanics – rules, constructs, processes, and methods – into the real world in an effort to engage people. In his session at @ThingsExpo, Robert Endo, owner and engagement manager of Intrepid D...
Eighty percent of a data scientist’s time is spent gathering and cleaning up data, and 80% of all data is unstructured and almost never analyzed. Cognitive computing, in combination with Big Data, is changing the equation by creating data reservoirs and using natural language processing to enable analysis of unstructured data sources. This is impacting every aspect of the analytics profession from how data is mined (and by whom) to how it is delivered. This is not some futuristic vision: it's ha...
WebRTC has had a real tough three or four years, and so have those working with it. Only a few short years ago, the development world were excited about WebRTC and proclaiming how awesome it was. You might have played with the technology a couple of years ago, only to find the extra infrastructure requirements were painful to implement and poorly documented. This probably left a bitter taste in your mouth, especially when things went wrong.
Learn how IoT, cloud, social networks and last but not least, humans, can be integrated into a seamless integration of cooperative organisms both cybernetic and biological. This has been enabled by recent advances in IoT device capabilities, messaging frameworks, presence and collaboration services, where devices can share information and make independent and human assisted decisions based upon social status from other entities. In his session at @ThingsExpo, Michael Heydt, founder of Seamless...
The IoT's basic concept of collecting data from as many sources possible to drive better decision making, create process innovation and realize additional revenue has been in use at large enterprises with deep pockets for decades. So what has changed? In his session at @ThingsExpo, Prasanna Sivaramakrishnan, Solutions Architect at Red Hat, discussed the impact commodity hardware, ubiquitous connectivity, and innovations in open source software are having on the connected universe of people, thi...
WebRTC: together these advances have created a perfect storm of technologies that are disrupting and transforming classic communications models and ecosystems. In his session at WebRTC Summit, Cary Bran, VP of Innovation and New Ventures at Plantronics and PLT Labs, provided an overview of this technological shift, including associated business and consumer communications impacts, and opportunities it may enable, complement or entirely transform.
There are so many tools and techniques for data analytics that even for a data scientist the choices, possible systems, and even the types of data can be daunting. In his session at @ThingsExpo, Chris Harrold, Global CTO for Big Data Solutions for EMC Corporation, showed how to perform a simple, but meaningful analysis of social sentiment data using freely available tools that take only minutes to download and install. Participants received the download information, scripts, and complete end-t...
For manufacturers, the Internet of Things (IoT) represents a jumping-off point for innovation, jobs, and revenue creation. But to adequately seize the opportunity, manufacturers must design devices that are interconnected, can continually sense their environment and process huge amounts of data. As a first step, manufacturers must embrace a new product development ecosystem in order to support these products.
Manufacturing connected IoT versions of traditional products requires more than multiple deep technology skills. It also requires a shift in mindset, to realize that connected, sensor-enabled “things” act more like services than what we usually think of as products. In his session at @ThingsExpo, David Friedman, CEO and co-founder of Ayla Networks, discussed how when sensors start generating detailed real-world data about products and how they’re being used, smart manufacturers can use the dat...
When it comes to IoT in the enterprise, namely the commercial building and hospitality markets, a benefit not getting the attention it deserves is energy efficiency, and IoT’s direct impact on a cleaner, greener environment when installed in smart buildings. Until now clean technology was offered piecemeal and led with point solutions that require significant systems integration to orchestrate and deploy. There didn't exist a 'top down' approach that can manage and monitor the way a Smart Buildi...