Hybrid cloud storage vs on-premise: What data goes where?

What are the benefits of cloud storage?

The main benefits of cloud storage are well-documented and include flexibility and the ability to scale, as well as avoid the need for capital spending.

Nor is cloud now a second-rate option when it comes to performance. The big three hyperscalers and others offer options for higher read and write speeds and IOPS. Cloud, too, allows organisations to move data between different storage tiers, to optimise combinations of performance, capacity and cost as workloads change.

Cloud is the primary way to deploy object storage, and so allows a single data store to serve multiple applications. For cloud-based applications such as advanced analytics and AI, locating data close to compute makes sense.

To this, CIOs might add lower infrastructure management overheads – especially by removing the need to operate datacentres – and speed of deployment. The “all you need for cloud is a credit card” line is overplayed. But sourcing, purchasing and installing hardware takes time, and can easily add six to 18 months to an infrastructure project.

What are the benefits of on-site data storage?

On-site storage, however, retains some advantages. First and foremost is regulatory compliance and security. Although cloud storage security is now good, there are verticals and organisations that still prefer the full control that comes with local, physical assets. On-premise systems also avoid questions about data sovereignty.

“On-site data storage, or local storage, has advantages that include direct control over data security and privacy, faster access speeds, and physical control over the storage,” says Kausik Chaudhuri, chief innovation officer at Lemongrass, a service provider focused on SAP implementations.

For performance, on-premise hardware will typically outperform the cloud for a given workload. Firms have years, even decades, of experience in fine-tuning their storage and compute to work together.

And while high-performance flash storage is an option locally and in the cloud, wide-area network connectivity always adds latency. Even where low latency is not critical, performance is more predictable on-premise.

Costs can be more predictable on site. Cloud bill shock is a real issue. Organisations are often good at spinning up new capacity, but less good at scaling down what is no longer needed.

That, and egress costs incurred downloading or repatriating data, can make cloud more expensive than it first appears. Local storage costs, even with depreciation and maintenance, will be more predictable, and organisations have already invested in storage arrays, so it makes sense to work them as hard as possible.

Which data goes in the cloud and which data goes on-prem?

There is no set rulebook for which data goes to cloud and which stays on-premise. IT data management teams will have their own criteria, including the costs and capacity of on-premise storage, the contract they have with a cloud provider, and security and regulatory considerations.

For organisations without a policy, a good way to start is to look at the workload and follow the data.

A latency-critical engineering, manufacturing or financial workload will have different needs to customer records in long-term retention. The former is more likely to suit local flash storage, the latter large-scale cloud object options.

Application requirements are another factor to consider. If an application needs rapid, low latency access to data, then keeping storage and compute close makes sense. However, this could be local storage or cloud storage. It depends where the application resides.

Data sources are another important consideration. Highly instrumented systems that ingest data frequently – such as in manufacturing – benefit from local or even edge storage. If users are widely distributed across the business or data is captured directly from consumers, then cloud storage might be optimal, especially object storage with its single global namespace.

Any storage infrastructure project should ideally start with a data discovery and cataloguing exercise.

What does a hybrid cloud storage strategy look like?

The ideal data storage strategy is one that meets the demands of the business for its data, integrates with its current applications and application roadmap, and meets financial and operational requirements.

These will include cost, including management and other overheads, and the ability to scale. Factors such as resilience and business continuity will also play a part. For these reasons, most firms opt for a hybrid approach.

Gartner, for example, believes that 60% of firms will implement hybrid cloud file deployments by 2026, a significant increase from the 20% that did so in 2023.

However, there is no one single definition of hybrid storage. It can include multicloud infrastructure as well as local and cloud storage volumes. Firms with sensitive or high-value data will also want to consider private cloud options.

At its simplest, hybrid storage should allow firms to hold data locally, or on private or public clouds, with applications able to access data regardless of location. A data fabric is a key part of this, as is good understanding of which applications use which datasets and how information moves across the organisation.

Firms need to work with vendors that support hybrid storage. Most now do, but generally the easier route is to add cloud capacity to support on-premise architecture. This could be to add features such as archiving, burstable capacity or cloud backups.

Working backwards from the cloud to add on-premise storage is less common, although possible through technology such as AWS Outposts.

Either way, the best storage strategy is one that plays to the advantages of cloud and local platforms, but can change where data goes as the business changes.