Storage For Machine Learning

Quobyte’s fast and efficient storage software solution helps speed up all stages of the machine learning workflow.

High Performance to get the most out of GPUs

For compute-intensive stages, Quobyte throughput performance scales linearly with the number of servers, with no bottleneck or limits to the number of clients. Data rushes to each client at up to 100 Gb/s, pushing maximum performance from the underlying hardware. And to help fully leverage the value of investment in GPUs, use our GPU-optimizing TensorFlow plugin to reduce latency and CPU loading, thereby conserving memory bandwidth.

Infinite Scalability

Grow your storage in terms of throughput and capacity when you need it. As ML project requirements change – oftentimes more quickly than you anticipate – your Quobyte installation will adapt. Just add disks or servers when you need more capacity or performance without any interruption to applications or services. Quobyte can handle your biggest and growing data sets to drive higher accuracy of results.

Give Your TensorFlow Workloads a Boost

As the first distributed file system, Quobyte offers a TensorFlow plugin, that increases throughput performance by up to 30%. The plugin allows TensorFlow applications to talk directly to Quobyte without going through the kernel.

Better performance

Quobyte reduces kernel mode transitions and lowers CPU usage. This increases utilization of the GPU to speed up model training and the inference stage of the ML workflow. Hence, Quobyte’s performance and scalability help you train faster across larger data sets for higher accuracy results.

More Flexibility

Quobyte’s TensorFlow filesystem plugin can be used with most any Linux system; even older versions can be used because it bypasses the kernel. It can also be used with Google Cloud Platform (GCP) for model training: train models locally on sample data sets and use GCP for training at scale.

One Storage System for all Phases in ML: From Data Wrangling to Training and Evaluating Results

Broad Platform Support for Deployment Flexibility

Quobyte unified storage makes spanning Machine Learning workloads from data center to cloud to edge easy. With support for S3, Linux, NFS, Windows, and Hadoop, Quobyte offers broad platform support so you can quickly and easily ingest data from existing systems. Containerized pipeline stages are supported with a Kubernetes CSI plugin and operator for persistent storage that easily scales. OpenStack environments also benefit from Quobyte with support for Cinder, Manila, and S3 interfaces. Public cloud platforms GCP and AWS are supported. Quobyte is unmatched in this level of platform support which gives you unmatched deployment flexibility.

Simply Efficient

The same Quobyte system that provides high performance for GPUs is the same system that can store multi-petabyte data sets cost-effectively because Quobyte can be used with SSDs and HDDs. No need for tiering to an archive system and all data stays readily available as “hot” data. With Quobyte storage at each stage, all data stays within the same file system which eliminates the perpetuation of data silos, prevents needlessly copying data between stages, and makes all data readily available for subsequent iterations of learning.

Don’t Forget Security

Often overlooked, but if your Machine Learning infrastructure is serving multiple customers, or is processing sensitive data, you need security. Our multi-tenancy lets you define isolated namespaces and physical separation of data/workloads inside the same cluster. Administrators can further isolate tenants by controlling which physical hardware they have access to.