Cloud Storage Orchestration, the automation technology for agility and flexibility

Why is cloud storage important?

Automating technology for servers, networks, and storage is essential for creating a cloud system with agility and flexibility.
In particular, storage is largely divided into block storage, file storage, and object storage according to data access methods. It is very challenging to automate storage because each vendor controls it with different systems. In general, cloud providers either designate and use one vendor storage exclusively or develop and use their own cloud storage based on their situations. Samsung SDS also developed its own technology called Storage Orchestration for storage automation of Samsung Cloud Platform.

What is storage orchestration?

Storage Orchestration is a technology that integrates, controls, and operates the storage of multiple vendors and device types, and it collects and analyzes storage data to provide the optimal storage volume to users.
In the past, we used to store data on individual servers or manage it using separate storage. Today, much more data can be stored and managed in the cloud. Therefore, the cloud system should be able to configure the system using multiple storage solutions rather than a single storage solution. Multiple storage products are provided by many vendors, such as NetApp, Dell EMC, and HPE, and there are various storage solutions available depending on the methods and specifications.

Limitations of using individual storage products

In each storage product, different storage technologies are embedded, and we can see how rapid technological changes replace old storage technologies and how new technologies appear. Accordingly, vendors are releasing various storage solutions using the latest technologies. Thus, cloud providers are committed to improving performance and reliability using the new technologies. However, it is required to understand the characteristics of the unique storage vendor system before applying it to the cloud no matter how advanced it is. Moreover, as engineers have to work on volume creation, modification, and deletion whenever it is needed, this may harm the agility of the cloud service. Now, let us suppose you created a cloud system with an automation solution provided by a vendor. In that case, you are likely to be locked into the vendor's products, and even if new storage solutions are provided at lower prices, you hardly give up the locked-in product, making it difficult to have competitive cloud system.

[Figure 1] Change of data storage method

The main characteristics of storage orchestration

Storage Orchestration is a solution that controls the storage of multiple vendors and device types upon a single request and enables storage control management automation by integrating with applications. In addition, it applies autonomous volume allocation technology to analyze storage data, select optimal storage, and create volumes to use multiple storage devices more efficiently, regardless of the storage vendors and device types. With the solution, you can also solve the aforementioned lock-in problems and create a more advanced cloud storage system. Moreover, storage orchestration allows not only the cloud, but any applications with the storage of many vendors and device types to configure a system, and a storage system can be easily configured.

[Figure 2] Storage Orchestration that connects applications and storage

Storage orchestration container storage interface (CSI)

We developed Storage Orchestration CSI to make it easier to use storage in applications running in the Kubernetes environment. Kubernetes uses objects called persistent volume (PV) and storage class (SC) to create and manage storage volume. But you cannot create and change the volume in storage just by using PV and SC. Storage Orchestration CSI is a solution that enables integrated control over storage in the Kubernetes environment based on a standard library called container storage interface (CSI) provided by Kubernetes, which was made to address such issues. With Storage Orchestration CSI, you can create and change volumes in storage simply by creating PVs after SC configuration.

In addition, storage orchestration volume (SOV) was developed based on the Kubernetes custom resource definition (CRD), which makes storage volume management easier by registering and managing each volume's characteristics and unique information.

Autonomous volume allocation technology

Autonomous volume allocation uses a group called a storage service pool to classify storage by type (based on storage services such as performance, function, etc.). It analyzes metric data from each storage solution to determine which solution is the most affordable and underloaded to create volumes in the storage. This minimizes capacity and performance variations across storage, and allows the storage scaling by simply registering with a storage service pool. This function ultimately makes it easier to run multiple storage solutions more efficiently.

Technical elements of storage orchestration

The followings are the components of storage orchestration:

• · REST API: A system that provides integrated control of storage and storage services via the storage orchestration REST API
• · Storage orchestration CSI: A system that provides storage volumes to Pods through the storage orchestration volume (SOV) in a Kubernetes environment
• · Storage service management: A system that configures storage by type, such as gold, silver, or bronze, and provides storage volume according to the users’ needs
• · Autonomous volume allocation: A system that provides optimal storage volume to users based on the storage status analysis data using storage data
• · Storage data analytics: A system that collects data from each storage device and analyzes them to examine and monitor storage status and issues
• · Storage & volume management: A system for managing physical storage and its internal componen
• · Storage orchestration driver: A module that connects the physical storage of each vendor and storage orchestration

Systems using storage orchestration

Storage orchestration is a foundational technology for cloud services and applications that provides storage volume to many users. It is currently used for block storage and file storage services of Samsung Cloud Platform (SCP), which is the Samsung SDS cloud service for enterprise, storage part of Infra Manager, and the private/group volume service of x.Cloud(Kubernetes) in the R&D center. It can also be used in systems that use multiple storage devices or systems with storage vendor lock-in issues, such as HPCs that use a lot of petabyte-level storage solutions and cloud big data that requires massive data processing.

[Figure 3] Storage Orchestration concept and technology elements

What differentiates Samsung SDS technology?

It is easy to install and use the storage orchestration technology developed by Samsung SDS. Since it was developed by Samsung SDS without relying on any other vendors, it works faster and offers various storage services by allocating the volume autonomously according to the storage data and optimizing backup.

Storage vendors previously tried to develop a solution for integrated storage control, but they failed to do so because they were only focused on the product of a leading vendor due to their interests in the process of designing and developing a solution. Some businesses have tried to develop similar solutions, but since their products were made based on storage vendor solutions, everything from installation to actual usage was too complicated. To resolve such problems, we came up with our own solution, storage orchestration. Using the solution, we were able to build a faster and more efficient cloud storage service, which has become the basis for providing seamless services to our clients.

Let's take a deeper look into the Samsung Cloud Platform where storage orchestration technology developed by our own technologies was applied for storage automation?
Go to Samsung Cloud Platform (SCP) >