The ramp up of Scale-Out Flash Storage (SOFS) is challenging the storage deployment status quo, but how?
Lightbits Labs’ VP of Product Marketing Josh Goldenhar, recently sat down for a webinar with colleagues from Weka.IO, Excelero and G2M Research; Shailesh Manjrekar, Tom Leyden and Mike Heumann, respectively. The purpose was to discuss this shift and how today’s new storage solutions enable data center teams to get the right capacity, performance, and data service availability for their applications.
During the webinar, this panel of experts discussed how the ramp up of Scale-Out Flash Storage (SOFS) is allowing for new architectures and capabilities.
What is Scale-Out Flash Storage?
SOFS takes software packages and servers containing NVMe flash and molds it into an entire network storage solution. SOFS uses a network fabric to connect storage and compute nodes, allowing a flexible topology with remote storage performance that approaches the reliability and low latency of local storage.
What are the leading use cases for Scale-out Flash Storage?
a. Shailesh Manjrekar, Weka.IO – We see three clear use cases across three verticals that all need performance at scale. They include:
i. AI/Machine learning
ii. Life sciences and genomics
iii. Fraud analysis
ii. Life sciences and genomics
iii. Fraud analysis
b. Josh Goldenhar, Lightbits Labs – We’re seeing different set of use cases, including:
i. Databases, both traditional ones and scale out, including distributed open source like Cassandra, MongoDB, CockroachDB and more.
ii. Smaller public cloud providers that have a special service (like HPC) so they can service customers that want Amazon EBS-like (elastic based storage) functionality out of the box
iii. The third area is private cloud for companies that want the features of public cloud, with the cost benefits of private cloud, as well as those enterprises that want to migrate to Kubernetes or OpenStack.
ii. Smaller public cloud providers that have a special service (like HPC) so they can service customers that want Amazon EBS-like (elastic based storage) functionality out of the box
iii. The third area is private cloud for companies that want the features of public cloud, with the cost benefits of private cloud, as well as those enterprises that want to migrate to Kubernetes or OpenStack.
c. Tom Leyden, Excelero – We see many use cases, including:
i. Post-production
ii. Analytics for finance and medical, government use cases and enterprise databases and data warehouses
iii. Scale out databases, artificial intelligence
ii. Analytics for finance and medical, government use cases and enterprise databases and data warehouses
iii. Scale out databases, artificial intelligence
Hybrid cloud and private cloud architectures are becoming commonplace in enterprises today. What advantages do SOFS solutions have over array-based storage solutions in this regard?
a. Tom Leyden, Excelero – Our products are built with cloud in mind with large cloud partners and have share-nothing architecture that allows organizations to scale as needed. Data path separation and software only enable the ability to design storage applications and offers choices of NVMe drives, networking, etc.
b. Shailesh Manjrekar, Weka – The key value proposition is to leverage either private or public cloud and work with a number of object storage partners and ecosystems. We exchange object file storage and are able to keep metadata. We can amortize the cost of flash to bring out hybrid cloud benefits. We can take a snapshot of metadata and data and have a complete view of the file system and can move to public cloud-based file system and have a pure SAS offering.
c. Josh Goldenhar, Lightbits – For most organizations, the cloud is undeniable and the ability to run virtual machines (VMs) or containers is paramount. Companies are looking for storage solutions that take block and standardize it; with CSI in Kubernetes, making the workload portable for public or private clouds. Our customers can run LightOS in the cloud with NVMe/TCP, have thin provisioning and compression to get more out of provisioned cloud storage and have an even better experience on-prem. As companies move to containers and Kubernetes, having a simple plug-in solution and standardized environments will be a game changer and will be more cost-effective.
What best practices would you suggest to a potential customer using a SAN or NAS storage solution today who is looking to migrate to an SOFS solution?
a. Josh Goldenhar, Lightbits – When migrating, you need to identify your applications that are highly transactional and require low bandwidth. SOFS might not be the right solution for everything. Moving to SOFS is going to require changes to your networking process, especially with RDMA, so it’s a little more labor intensive and needs a bit of teamwork. Organizations should try different solutions and shop around for ease of use and performance and look for out-of-the-box solutions that use standard networking equipment and protocols.
b. Tom Leyden, Excelero – Cost is important, but should never be the main driver. Organizations should think in terms of ROI and how this new storage solution will impact the organization. Check if new solutions will disrupt existing workflows and make sure the solution provider provides all the flexibility. As data storage capacity grows, you need to think ahead and how granularly you have to scale, and whether you’ll be locked into these choices.
c. Shailesh Manjrekar, Weka – Organizations should start with business goals and what they are ultimately trying to achieve. Start small and get a feel for the solution you’re buying and determine how you are going to operationalize it. As you think about the pipeline, you need to break silos to see what users need and how they will work with the IT department. It’s important to look for a solution that can cater to the entire pipeline and look for a hybrid solution when possible.
Final Thoughts
As SOFS solutions offer numerous benefits, we believe they will overtake tradition storage solutions over time.
It’s human nature to use what’s easiest to get a great result and hence we believe NVMe/TCP is the most powerful NVMe-oF technology, providing high performance with reduced deployment costs and design complexity. It extends NVMe across the entire data center using simple and efficient TCP/IP fabric.
To learn more, watch the webinar on demand or visit our LightOS solutions page.
