Welcome to the first blog post of our blog series exploring the synergy between OpenStack and Lightbits software-defined storage solutions.
Today, we’re talking all about the seamless integration of Lightbits NVMe® over TCP block storage with OpenStack thanks to its handy Cinder driver that makes management a breeze. Next time, we’ll get into the nitty-gritty of performance, showing you just how well Lightbits scales with sustained latency, even under load. Stay tuned to see how Lightbits can seriously improve performance, scalability and durability in your OpenStack environment.
Since the inception of OpenStack in 2010 by Rackspace and NASA, it has been and continues to be the technology of choice for many private and public clouds with over 45 million compute cores deployed. This trend is supported by large organizations such as cloud service providers, telecommunication providers, and large enterprises who are looking to deploy an OpenStack private cloud for their business users.
OpenStack is an open-source platform that enables organizations to build and manage their own public or private clouds using virtual resources. Under the hood, OpenStack is made up of a series of scripts, which are bundled into packages called projects. These projects handle core cloud-computing resources such as Compute (Nova), Networking (Neutron), Storage (Cinder/Swift), Identity (Keystone), Image (Glance), and many more.
Initially, OpenStack deployments were slow due to the complexities of deployment and management which require specialized skills across multiple domains like networking, storage, and virtualization. Lack of commercial support was also an important factor prohibiting enterprises from adopting it.
Today, OpenStack has grown significantly in maturity with improvements in scalability, performance, and stability. Large enterprises and service providers like AT&T, Walmart, Workday, CERN, VEXXHOST, just to name a few, are using OpenStack in their data centers. A complete list of OpenStack customers can be found here.
Figure 1 below shows the OpenStack landscape with all the services working together.
Figure 1: OpenStack Landscape
Lightbits Disaggregated Software-Defined Storage Overview
Lightbits represents a paradigm shift in how storage resources are managed and utilized in modern data centers. Lightbits’ unique disaggregated, software-defined, NVMe® over TCP, block storage architecture – combined with Intelligent Flash Management and enterprise-rich data services – makes it the only complete cloud data platform that’s easy to provision and simple to manage but also highly scalable, performant, and cost-efficient.
At its core, it leverages NVMe over TCP (NVMe/TCP) storage protocol technology, which was invented by Lightbits and then ratified and given to the open source community, to deliver low-latency, high-performance block storage over a regular Ethernet network. This not only reduces the cost and complexity compared to other technologies like Fibre Channel or Infiniband but also democratizes access to high-speed NVMe storage.
The software-defined nature of Lightbits’ solution means it can be deployed on commodity servers and scaled across a distributed architecture without the need for specialized networking equipment or proprietary hardware. This flexibility combined with advanced features such as Intelligent Flash Management, ElasticRAID, in-line compression, thin provisioning, snapshots, and clones produces a solution that enhances data durability, optimizes storage efficiency, and reduces overall operating costs. Lightbits software can be used not only on-premises but also in public clouds like AWS and Azure.
Why Lightbits for OpenStack Storage?
Although there are many solutions for OpenStack storage today, not all of them are created equal. Most backend storage solutions provide enough performance for small to medium workloads but do not have the necessary performance for all workloads. In a large OpenStack deployment, organizations cannot afford to have application timeouts due to low throughput with high tail latency. Storage scalability is also a big concern. This is very crucial in an OpenStack deployment because the purpose of OpenStack is to provide vast amounts of virtual resources to many applications running in parallel. To meet these high SLA requirements, both OpenStack infrastructure and its storage solutions must have enough resources and have the ability to scale to meet the demands of users.
Lightbits software-defined storage offers such a solution for OpenStack. Due to its unique architecture and features that directly address the core challenges of scalability, performance, and reliability, Lightbits offers a compelling backend storage solution for OpenStack. Below are some advantages of using Lightbits as backend storage for OpenStack:
- Disaggregation: enable large scale storage clusters for OpenStack needing high performance block storage.
- Scalability, Simplicity, Flexibility & Agility: scale up by adding SSDs and scale out by adding nodes/clusters. Lightbits enables users to add NVMe devices to existing Lightbits servers with available drive slots or add more servers with NVMe devices to an existing cluster without downtime.
- High Performance: leverages NVMe/TCP to provide high IOPS/Throughput with consistent low average and tail latencies.
- Cloud-Native Compatibility: support for modern workloads that require agile and flexible storage solutions.
- Multi-Tenancy: provides tenant isolation when running on the same cluster.
- Data Durability: provides data durability that combines several key technologies such as data replication, erasure coding, cyclic redundancy checks, fine-grained failure-domains, snapshots (redirect on write) and clones.
Figure 2 below shows Lightbits as a backend storage solution for OpenStack.
Figure 2: Lightbits for OpenStack
OpenStack with Lightbits Backend Storage
Like many other storage solutions, Lightbits also provides a dedicated Cinder driver for OpenStack Block Storage service. This integration allows for easy management and provisioning of block storage across the OpenStack environment. It enables users to leverage advanced storage features directly from the OpenStack interface. The OpenStack Cinder driver is a mechanism designed to act as an interface between the Cinder Block Storage Service and the storage hardware. It supports various storage operations, including volume creation, snapshot management, and data protection, thus providing a robust and reliable storage solution for enterprise-scale deployments.
Below is a sample of the OpenStack Cinder configuration for the Lightbits storage solution. To enable the Lightbits Cinder driver, just edit the “cinder.conf” file and include the following Lightbits cluster information, and restart the “Cinder” service. You can create multiple Lightbits Cinder configurations to meet specific requirements such as volume replication factor, enable or disable compression, or have multiple Lightbits clusters providing block storage for your OpenStack environment.
[lightos]
volume_driver = cinder.volume.drivers.lightos.LightOSVolumeDriver
volume_backend_name = lightos
lightos_api_address = x.x.x.x
lightos_api_port = 443
lightos_jwt = eyJhbGciOiJSUzI1NiIsImtpZCI6InN5c3RlbTpyb290IiwidHlwIjoiSldUIn0…….
lightos_default_num_replicas = 3
lightos_default_compression_enabled = True
lightos_api_service_timeout = 30
After restarting the Cinder service, you should see Lightbits in the volume backend pool list command. It should look like this.
The next step is to create a volume type in OpenStack to include Lightbits. It should look like this after creating the volume type.
After you have successfully created a volume type, you can proceed to create Lightbits volumes and attach them to the VMs. These Lightbits block volumes can be formatted with a filesystem such as XFS or ext, depending on your application requirements. The command below shows a sample of how to create a 100GB Lightbits volume in OpenStack.
Now this volume can be attached to a VM with the name “vm-1” using the following command:
stack@controller:~$ openstack server add volume vm-1 lbvol-rf3
Conclusion
As we conclude our initial look at integrating Lightbits with OpenStack deployments, it’s evident that combining OpenStack’s comprehensive cloud management features with the advanced storage technology of Lightbits offers significant advantages. The seamless Cinder driver integration enhances any OpenStack deployment, providing substantial performance and scalability benefits. Stay tuned for our next article, where we will delve into detailed performance metrics to illustrate why Lightbits is the right choice for your cloud-native applications that require high performance to avoid having any application timeouts. Thank you for following along as we explore these exciting technologies together.
