Welcome!

Agile Computing Authors: Liz McMillan, Zakia Bouachraoui, Elizabeth White, Pat Romanski, Maria C. Horton

Related Topics: @CloudExpo

@CloudExpo: Article

Cloud Database Design, Scale Out Using Shared Nothing Pattern

Virtual storage and database partitions

Abstraction Vs Flexbility :
It is evident that storage plays a major part in the data center and for cloud services. The storage virtualization plays a key part in the dynamic infrastructure attribute of Cloud Computing. Which means the storage is provisioned and de-allocated on demand and usage needs. The good part is that this complex stuff is hidden from the cloud consumer.

However while the storage allocation is abstracted it also brings in performance concerns in a multi tenant cloud environment where by most of the cloud consumers are geographically dispersed and a good amount of data retrieval and manipulation at stake.

Currently most cloud storage virtualization platforms rely on the hardware options like fiber channel, Ethernet switch, network attached storage (NAS) and iSCSI as the implementation medium.

Currently Cloud platforms have very little support for database design related virtualization enhancements. But in future designing databases specific for Cloud especially for private clouds in large enterprises is a sure possibility. In this context the following design principles are important when you design database applications which needs to be delivered using Cloud platform.

Shared Nothing Vs Shared Every Thing
Shared resource and shared nothing represent forms of data access architectures. In a shared (every thing) resource model various processes in the DBMS have access to all system resources, including the data. In the shared-nothing environment, separate DBMS resources divide up the workload, rach responsible for its own data, memory locations, and other resources.

Most popular databases support a shared-nothing model using different strategies. For example,

  • Oracle : Utilizes Range & Hash portioning of tables towards shared nothing . Here it is a table and table space level segregation of data based either on range of key values or a hashing algorithm. (note : Oracle Real Application clusters at the instance level adopt a Shared Every Thing Model)
  • DB2 UDB follows a shared nothing model. Data is partitioned according to a partition key. Rows are assigned to a partition, and each partition has total control of that row. If another partition wants to read or update a row, it must send the request to the owning partition. The owning partition then executes the command on behalf of the requestor.
  • Sql Server : Utilizes Distributed Partitioned views to implement Shared nothing architecture.

Shared nothing model greatly simplifies things like resource contention, including memory, locks and processors . Implemented properly , it offers unlimited scalability. As new rows or data sources are added, more partitions can be added. The workload on any individual partition remains the same.

Storage Virtualization & Shared Nothing Meet Each Other

While the dynamic infrastructure tenant of cloud computing is satisfied by the current classic model as seen in the picture , but the load balancing and scalability needs are fully satisfied by hardware at this time. However implementing the shared nothing data access architecture with a proper access key will highly improve the scalability and performance of cloud databases which in turn increases the QoS of the Cloud platform and make the XaaS a real success providing the best of cost efficiency and improved performance.

View Of a Shared Nothing Cloud Database

As evident implementing a Shared Nothing Cloud Database Model does not replace Storage Virtualization, but compliments the scalability further by splitting the data within multiple virtual storage drives. As seen in the diagram,

Load balancing algorithms implemented at the shared nothing cloud databases further augment the Load balancing options provided by the Cloud Platform.

Shared Nothing Cloud Database Design Considerations

  • Implement Hashing or Ranging Keys as part of the tables which logically separate the data based on the Cloud platform needs. For example if in a SaaS platform multi tenancy is the need then having the CUSTOMER ID or CUSTOMER KEY as a Hashing key for the shared nothing cloud databases
  • Increase the scalability further by implementing a Geography or Time based partition which scales up well depending on the situations
  • Ensure that the queries utilize the logical partitions established as part of the shared nothing architecture. For example if you can query a customer by SSN in a cloud database which has implemented a shared nothing architecture using Geography, then utilize the geography also part of the query filter conditions
  • Have an option to rebalance the data within the virtualized storage if they started becoming skewed due to the way the hashing algorithm is built in

What is expected from Cloud Platform Providers & Database Vendors

  • Expecting the database vendors provide shared nothing algorithms tailored for virtual storage, especially on the rebalancing based on the disk usage
  • New cloud aware partitioning techniques
  • Complimenting load balancing algorithm to augment what is currently available at the hardware
  • Ability to PIN SQLs to a virtual machine based on dynamically queried resource usage

Over all Private Clouds at Enterprise need to satisfy still more SLA or QoS as part of their service and full support from database engines is key to the success. Share Nothing Architecture is one important thing to achieve these non functional needs of the Cloud Platform.

More Stories By Srinivasan Sundara Rajan

Highly passionate about utilizing Digital Technologies to enable next generation enterprise. Believes in enterprise transformation through the Natives (Cloud Native & Mobile Native).

IoT & Smart Cities Stories
The platform combines the strengths of Singtel's extensive, intelligent network capabilities with Microsoft's cloud expertise to create a unique solution that sets new standards for IoT applications," said Mr Diomedes Kastanis, Head of IoT at Singtel. "Our solution provides speed, transparency and flexibility, paving the way for a more pervasive use of IoT to accelerate enterprises' digitalisation efforts. AI-powered intelligent connectivity over Microsoft Azure will be the fastest connected pat...
There are many examples of disruption in consumer space – Uber disrupting the cab industry, Airbnb disrupting the hospitality industry and so on; but have you wondered who is disrupting support and operations? AISERA helps make businesses and customers successful by offering consumer-like user experience for support and operations. We have built the world’s first AI-driven IT / HR / Cloud / Customer Support and Operations solution.
Codete accelerates their clients growth through technological expertise and experience. Codite team works with organizations to meet the challenges that digitalization presents. Their clients include digital start-ups as well as established enterprises in the IT industry. To stay competitive in a highly innovative IT industry, strong R&D departments and bold spin-off initiatives is a must. Codete Data Science and Software Architects teams help corporate clients to stay up to date with the mod...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...
Druva is the global leader in Cloud Data Protection and Management, delivering the industry's first data management-as-a-service solution that aggregates data from endpoints, servers and cloud applications and leverages the public cloud to offer a single pane of glass to enable data protection, governance and intelligence-dramatically increasing the availability and visibility of business critical information, while reducing the risk, cost and complexity of managing and protecting it. Druva's...
BMC has unmatched experience in IT management, supporting 92 of the Forbes Global 100, and earning recognition as an ITSM Gartner Magic Quadrant Leader for five years running. Our solutions offer speed, agility, and efficiency to tackle business challenges in the areas of service management, automation, operations, and the mainframe.
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, we provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading...
DSR is a supplier of project management, consultancy services and IT solutions that increase effectiveness of a company's operations in the production sector. The company combines in-depth knowledge of international companies with expert knowledge utilising IT tools that support manufacturing and distribution processes. DSR ensures optimization and integration of internal processes which is necessary for companies to grow rapidly. The rapid growth is possible thanks, to specialized services an...
At CloudEXPO Silicon Valley, June 24-26, 2019, Digital Transformation (DX) is a major focus with expanded DevOpsSUMMIT and FinTechEXPO programs within the DXWorldEXPO agenda. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throug...