Welcome!

Agile Computing Authors: ManageEngine IT Matters, Craig Lowell, AppNeta Blog, Liz McMillan, Jonathan Fries

Related Topics: @CloudExpo, Java IoT, Microservices Expo, Agile Computing, @BigDataExpo, SDN Journal

@CloudExpo: Article

The Outside-In Battle for the Soul of the Cloud

The clouds that can best adapt to the demands of the workloads they are supporting will be best positioned for success

Whether they admit it or not, the emergence of public cloud providers has dramatically altered the playing field for hardware vendors of every type. Amazon Web Services (AWS) and its competitors opened Pandora's box by introducing the world to a completely programmatic, scalable, evolving, and pay-as-you-go way to procure and utilize network, compute and storage resources on a global scale. They have disrupted many layers of the technology industry from the applications being written to the way companies interact with the infrastructure being used to support those applications.

Nowhere is this disruption easier to see than in the virtualization ecosystem. For the better part of the last decade, hypervisor companies like VMware, Citrix, Microsoft and Red Hat worked hand-in-hand with hardware manufacturers like Cisco, NetApp, EMC, HP and Dell to define both the infrastructure foundation as well as the virtualized abstraction layer that sat underneath the entirety of the client/server era. These companies provided a direct link between the enterprise applications, the hypervisor and the hardware. They owned the traditional datacenter construct.

It's that construct, since rebranded as "private cloud," that is directly under attack by public cloud providers. I predict that this will be the battlefield for the heart and soul of enterprise IT for the next decade.

The response to the public cloud threat has been varied, and often reflects the ability of traditional companies to pivot and meet the challenge. Interestingly, erstwhile competitors Microsoft and VMware reacted similarly. This is because they were both uniquely positioned to create a software-defined solution to the problem.

For both companies, the response started with existing enterprise workloads. One of the largest challenges of the AWS public cloud is the fact that getting workloads, and especially data, into and out of an enterprise environment can be both technically challenging and expensive. Most workloads running on an enterprise-virtualized platform today can't be easily ported into AWS and this increases the cost and risk of any migration. As companies with extensive and hard-won experience running mission-critical enterprise workloads, Microsoft and VMware came to much the same conclusion: build a public cloud using their existing platform and allow customers and developers to leverage all of the investment they've made in their own data centers as they selectively move workloads outside of their own data centers. Thus, Microsoft Azure and VMware vCHS were born. Both are clouds that customers can move workloads to without the need to rewrite or re-architect them. They can also be licensed using existing agreements and can be managed by existing staff and tools.

Unfortunately, the traditional data center infrastructure is now the weak link in this new software-defined world. In each of the public clouds referenced, the focus has been on the abstraction layer and how it interacts with the end users. What's missing is how the abstraction layer and the applications and tools that sit on top of it interact with the infrastructure directly.

There have been attempts at hardware-based offloading, especially with regards to storage. VAAI is a good example of VMware trying to create a way to let enterprise storage arrays handle the tasks they are good at without requiring the direct involvement of the hypervisor. But even there it's a rudimentary exchange at best: the hypervisor asks "can you do this task instead of me?" and the array responds. If the answer is yes, the hypervisor waits for the task to complete; if the answer is no, the hypervisor does the task itself. This relationship isn't dynamic, and is ignorant of the reason for and context behind the task in the first place.

In summary, we have an outside force, AWS and public cloud, being the primary catalyst driving change into the enterprise, yet very little of that change is happening below the cloud management or hypervisor layer. Why is that? Why is it important that the infrastructure layer become more of an asset to the rest of the stack? What would that look like? Let's dig in.

The question of why is actually pretty simple: it's really, really hard to take legacy hardware architecture and retrofit it into something agile and programmatic. In some cases, it's just a new concept that requires a hardware refresh (like Cisco UCS and its take on XML-defined BIOS policies), but in many cases, especially around storage, it requires a complete reimagining of the platform. It's no coincidence that most of the innovation in this agile infrastructure space is being done by startups who have no legacy customers, technical debt or margins to deal with.

Why is it important? While the best hardware is boring hardware, it's still a critical part to providing a flexible, reliable and high-performance foundation to handle applications that matter to enterprises. There are times where the best way to handle the demands of an application or, more important, multiple applications at once is in hardware. This is true at the network layer, where the manipulation of packets benefits from proximity to processing resources; the compute layer, where apps can benefit from having specialized GPU resources to handle unique requirements; and most especially at the storage layer.

Storage services can have the most dramatic impact on workload performance, yet are often implemented in such a way that they have no direct relationship with those workloads. Services like compression, deduplication and quality-of-service are usually "on or off" features when it comes to storage arrays. Best case, a storage administrator will create a volume or LUN, choose the features that need to be enabled, and then a virtualization admin will map that volume to a data store. Perhaps the virtualization team will create manual storage profiles that define the features offered by that data store, but placing and migrating VMs remains a manual process, and they will not have the ability to map application policy equally across the hypervisor and hardware layers. (Of course, it's not impossible to create programmatic, hypervisor-aware infrastructure, but it is pretty hard.)

Enterprises have come to expect some fundamental features from the public cloud space: simple architecture, linear scaling, API availability and granular application of services. These features allow an infrastructure to respond to the increased requirements of a workload natively, without the overhead of a bolt-on orchestration engine. They provide the ability for the hypervisor to be both a northbound and southbound policy enforcer. They enable the Next-Generation Data Center, one in which the hardware, the hypervisor and the application all play an integrated, coordinated role in providing the performance and availability demanded by the enterprise.

No matter where your workloads run, the rise of public cloud has ushered in an era of computing defined by a seamless, programmatic experience. The old, monolithic infrastructure of yesterday's client/server wave is giving way to a more agile, more responsive, more services-rich and more scalable cloud-based model. The battle for the enterprise soul is beginning and, inside or outside the firewall, the clouds that can best adapt to the demands of the workloads they are supporting will be best positioned for success.

More Stories By Jeramiah Dooley

Jeramiah Dooley joined the SolidFire team as a Cloud Architect on the Technology Solutions team. Prior to SolidFire he was most recently at VCE and before that Peak 10. You can check out his Virtualization for Service Providers blog or follow him on twitter @jdooley_clt.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
"We're a cybersecurity firm that specializes in engineering security solutions both at the software and hardware level. Security cannot be an after-the-fact afterthought, which is what it's become," stated Richard Blech, Chief Executive Officer at Secure Channels, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
In addition to all the benefits, IoT is also bringing new kind of customer experience challenges - cars that unlock themselves, thermostats turning houses into saunas and baby video monitors broadcasting over the internet. This list can only increase because while IoT services should be intuitive and simple to use, the delivery ecosystem is a myriad of potential problems as IoT explodes complexity. So finding a performance issue is like finding the proverbial needle in the haystack.
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life sett...
The WebRTC Summit New York, to be held June 6-8, 2017, at the Javits Center in New York City, NY, announces that its Call for Papers is now open. Topics include all aspects of improving IT delivery by eliminating waste through automated business models leveraging cloud technologies. WebRTC Summit is co-located with 20th International Cloud Expo and @ThingsExpo. WebRTC is the future of browser-to-browser communications, and continues to make inroads into the traditional, difficult, plug-in web ...
20th Cloud Expo, taking place June 6-8, 2017, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy.
Internet-of-Things discussions can end up either going down the consumer gadget rabbit hole or focused on the sort of data logging that industrial manufacturers have been doing forever. However, in fact, companies today are already using IoT data both to optimize their operational technology and to improve the experience of customer interactions in novel ways. In his session at @ThingsExpo, Gordon Haff, Red Hat Technology Evangelist, will share examples from a wide range of industries – includin...
WebRTC is the future of browser-to-browser communications, and continues to make inroads into the traditional, difficult, plug-in web communications world. The 6th WebRTC Summit continues our tradition of delivering the latest and greatest presentations within the world of WebRTC. Topics include voice calling, video chat, P2P file sharing, and use cases that have already leveraged the power and convenience of WebRTC.
"We build IoT infrastructure products - when you have to integrate different devices, different systems and cloud you have to build an application to do that but we eliminate the need to build an application. Our products can integrate any device, any system, any cloud regardless of protocol," explained Peter Jung, Chief Product Officer at Pulzze Systems, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at 20th Cloud Expo, Ed Featherston, director/senior enterprise architect at Collaborative Consulting, will discuss the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
"Once customers get a year into their IoT deployments, they start to realize that they may have been shortsighted in the ways they built out their deployment and the key thing I see a lot of people looking at is - how can I take equipment data, pull it back in an IoT solution and show it in a dashboard," stated Dave McCarthy, Director of Products at Bsquare Corporation, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
IoT is rapidly changing the way enterprises are using data to improve business decision-making. In order to derive business value, organizations must unlock insights from the data gathered and then act on these. In their session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, and Peter Shashkin, Head of Development Department at EastBanc Technologies, discussed how one organization leveraged IoT, cloud technology and data analysis to improve customer experiences and effici...
Fact is, enterprises have significant legacy voice infrastructure that’s costly to replace with pure IP solutions. How can we bring this analog infrastructure into our shiny new cloud applications? There are proven methods to bind both legacy voice applications and traditional PSTN audio into cloud-based applications and services at a carrier scale. Some of the most successful implementations leverage WebRTC, WebSockets, SIP and other open source technologies. In his session at @ThingsExpo, Da...
"IoT is going to be a huge industry with a lot of value for end users, for industries, for consumers, for manufacturers. How can we use cloud to effectively manage IoT applications," stated Ian Khan, Innovation & Marketing Manager at Solgeniakhela, in this SYS-CON.tv interview at @ThingsExpo, held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA.
As data explodes in quantity, importance and from new sources, the need for managing and protecting data residing across physical, virtual, and cloud environments grow with it. Managing data includes protecting it, indexing and classifying it for true, long-term management, compliance and E-Discovery. Commvault can ensure this with a single pane of glass solution – whether in a private cloud, a Service Provider delivered public cloud or a hybrid cloud environment – across the heterogeneous enter...
The cloud promises new levels of agility and cost-savings for Big Data, data warehousing and analytics. But it’s challenging to understand all the options – from IaaS and PaaS to newer services like HaaS (Hadoop as a Service) and BDaaS (Big Data as a Service). In her session at @BigDataExpo at @ThingsExpo, Hannah Smalltree, a director at Cazena, provided an educational overview of emerging “as-a-service” options for Big Data in the cloud. This is critical background for IT and data professionals...
@GonzalezCarmen has been ranked the Number One Influencer and @ThingsExpo has been named the Number One Brand in the “M2M 2016: Top 100 Influencers and Brands” by Onalytica. Onalytica analyzed tweets over the last 6 months mentioning the keywords M2M OR “Machine to Machine.” They then identified the top 100 most influential brands and individuals leading the discussion on Twitter.
Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like “How is my application doing” but no id...
What happens when the different parts of a vehicle become smarter than the vehicle itself? As we move toward the era of smart everything, hundreds of entities in a vehicle that communicate with each other, the vehicle and external systems create a need for identity orchestration so that all entities work as a conglomerate. Much like an orchestra without a conductor, without the ability to secure, control, and connect the link between a vehicle’s head unit, devices, and systems and to manage the ...
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smar...
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and G...