Welcome!

Agile Computing Authors: Yeshim Deniz, Carmen Gonzalez, Elizabeth White, Pat Romanski, Liz McMillan

Blog Feed Post

EMC Enters The Market With “Me Too” Flash Products

Yesterday EMC finally revealed the details on their not-so-secret Lightning and Thunder flash projects.  Fortunately this launch event didn’t include cramming small people into minis or firing Chad Sakac out of a cannon, but was more focused on the market and products EMC are bringing to it.  There was also a large amount of Twitter activity; look back at the #vfvache hashtag – this being the final product name of the Lightning product.  So what exactly did EMC announce?

 

Lightning AKA VFCache

 

VFCache (Very Fast Cache) is the final product name for the project that was called Lightning.  It turns out that this offering is nothing more than a PCIe SSD card for servers (not all servers mind you, but at this stage quite a few).  The initial offering combines with software to act as a very fast read cache to the host.  In Windows terms, this is implemented as a filter driver that sits above the STORPORT driver in the I/O stack, with similar implementations on Open Systems platforms.  The software component of VFCache tracks I/O and caches reads in order to speed up future I/O requests without needing to go to external disk.  Writes to disk are not cached by VFCache and EMC tried to make a virtue of the fact their product acts as a “write-through” cache, meaning I/O writes have to be committed to physical disk before the cache acknowledges them to the host.  Rather than being a benefit, write through mode in this instance is more likely to make the cache less effective by polluting the cache with writes that can’t be released until confirmed externally.  When there’s a difference in I/O if microseconds versus milliseconds, then this difference really matters.  However, I don’t think this is a design flaw, merely a placeholder for the future, as I’ll discuss later.

Disappointingly for EMC, VFCache 1.0 really is a 1.0 version in terms of feature support.  Within VMware ESXi for instance, the card installs with a device driver that only allows the cache benefits to be used when the filter driver is deployed into each ESXi guest, so it’s not simply a case of insert the card and off you go.  Moreover, the VFCache appears as a DAS device within VMware and so can’t be used in conjunction with vMotion.  For many organisations this is a huge omission as there’s a big correlation between high performance and high availability; the lack of vMotion isn’t acceptable.

 

The Competition

 

We can’t go any further on the VFCache discussion without mentioning the competition and in one of the presented slides, EMC paid homage to the market leaders, Fusion-IO.  Their ioCache product already accelerates VMware ESXi and Windows 2008 environments, using a similar hypervisor plugin approach.  ioCache already offers double the capacity of VFCache and it’s likely Fusion-IO have larger capacity cards in the pipeline as they already offer a range of SLC and MLC flash devices.

 

Thunder Follows Lightning

 

Surely Hitachi must be enjoying the irony of EMC choosing product code names based on already defunct HDS hardware (Thunder and Lightning were the mid-range and Enterprise products respectively that preceeded AMS and USP).  The next product announcement moves the flash-in-server story forward and explains how this technology is limited in terms of availability.  The move to centralised SAN environments was done precisely to fix the issues that occur with server-side SSD today.  Data is locked into the server, is difficult to expand (requiring downtime and physical intervention) and is isolated from access if a physical failure of the server should occur.  So, step up Project Thunder, EMC’s purpose build all-flash array.  This device allows multiple servers to share data across what EMC are calling the “server-area network”.  What they mean is a physically local, high-speed interconnect (such as Infiniband or Rapid-IO) between the server and a fast all-flash storage array.

The interconnect technology between devices already exists today (as already mentioned) but placing it into the server and using this for shared storage presents more of a challenge.  Where Lightning was a simple Filter Drive, Thunder will require deeper integration in order to manage consistency across all connected servers.  This isn’t something that comes new to EMC – think of how VMAX nodes interconnect and you have the model already there.  However, the implementation may require closer co-operation with server vendors than EMC can achieve, those same companies they are already in competition with for storage – IBM, HP and Dell.  This could mean Thunder becomes a VCE only product or is severely restricted when deployed in other manufacturers hardware.  We will have to wait and see. (Side Note: This also means that other all-SSD array manufacturers could become more attractive to HP, IBM & Dell as acquisition targets – check out here and here).

 

The Competition

 

Let’s not forget that dedicated all-flash arrays are already out there.  Recently I’ve discussed Pure Storage and SolidFire and there’s also Violin Memory, who have been going at this market for quite some time.  They already have the SSD array technology to a mature level including support for Infiniband; all that’s needed is a software driver to bring clustering to their products.

 

Futures

 

What can we expect going forward?  There are lots of gaps the product releases we’ve seen today.  Lack of vMotion support and no write-back cache to name only two.  The question we should be asking is what could be delivered in the future.  EMC have access to every piece of the I/O stack, from the hypervisor, the multi-path driver, through to the array.  Using PowerPath, EMC can develop more intelligent algorithms that choose whether to cache I/O locally in the server/hypervisor, destage to the array, leverage pre-fetching from disk and other clever ways to squeeze the best level of performance out of the hardware stack.

 

Conclusions

 

EMC have ratified both the PCIe SSD and dedicated flash array markets with their announcements of Lightning and Thunder.  At this stage they are bringing only “me too” products to the market, with other vendors out there having already delivered more advanced technology than being announced today.  However EMC have two big advantages; (a) they are a huge organisation, with access to the majority of customers in the market and a great marketing team.  They have the ability to place their products into customer environments and use price as the main differentiator (b) they have a huge R&D budget and never stand still on product development.  Today’s 1.0 releases will be superseded within months and address some of the shortcomings we can see today.  The future battle will not be over the hardware, but the software that integrates I/O in the server to I/O on the array, delivering the benefits of local flash with the safety of external storage.  The eventual winner will be the vendor who gets that software and hardware integration right.

Related Articles

You may be interested in the following related articles from this and other sites.

 

 

Read the original blog entry...

IoT & Smart Cities Stories
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Atmosera delivers modern cloud services that maximize the advantages of cloud-based infrastructures. Offering private, hybrid, and public cloud solutions, Atmosera works closely with customers to engineer, deploy, and operate cloud architectures with advanced services that deliver strategic business outcomes. Atmosera's expertise simplifies the process of cloud transformation and our 20+ years of experience managing complex IT environments provides our customers with the confidence and trust tha...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
AI and machine learning disruption for Enterprises started happening in the areas such as IT operations management (ITOPs) and Cloud management and SaaS apps. In 2019 CIOs will see disruptive solutions for Cloud & Devops, AI/ML driven IT Ops and Cloud Ops. Customers want AI-driven multi-cloud operations for monitoring, detection, prevention of disruptions. Disruptions cause revenue loss, unhappy users, impacts brand reputation etc.
The Japan External Trade Organization (JETRO) is a non-profit organization that provides business support services to companies expanding to Japan. With the support of JETRO's dedicated staff, clients can incorporate their business; receive visa, immigration, and HR support; find dedicated office space; identify local government subsidies; get tailored market studies; and more.
As you know, enterprise IT conversation over the past year have often centered upon the open-source Kubernetes container orchestration system. In fact, Kubernetes has emerged as the key technology -- and even primary platform -- of cloud migrations for a wide variety of organizations. Kubernetes is critical to forward-looking enterprises that continue to push their IT infrastructures toward maximum functionality, scalability, and flexibility. As they do so, IT professionals are also embr...
As you know, enterprise IT conversation over the past year have often centered upon the open-source Kubernetes container orchestration system. In fact, Kubernetes has emerged as the key technology -- and even primary platform -- of cloud migrations for a wide variety of organizations. Kubernetes is critical to forward-looking enterprises that continue to push their IT infrastructures toward maximum functionality, scalability, and flexibility.
Today's workforce is trading their cubicles and corporate desktops in favor of an any-location, any-device work style. And as digital natives make up more and more of the modern workforce, the appetite for user-friendly, cloud-based services grows. The center of work is shifting to the user and to the cloud. But managing a proliferation of SaaS, web, and mobile apps running on any number of clouds and devices is unwieldy and increases security risks. Steve Wilson, Citrix Vice President of Cloud,...
When Enterprises started adopting Hadoop-based Big Data environments over the last ten years, they were mainly on-premise deployments. Organizations would spin up and manage large Hadoop clusters, where they would funnel exabytes or petabytes of unstructured data.However, over the last few years the economics of maintaining this enormous infrastructure compared with the elastic scalability of viable cloud options has changed this equation. The growth of cloud storage, cloud-managed big data e...