Having gone through the pain of this decision both in-house and with customers - choosing between cloud and on-prem infrastructure is a common dilemma. The journey to simply evaluate the options can feel as challenging as executing the program itself! In my experience, the number of systems and stakeholders involved can add layers of complexity before reaching a decision. It’s not uncommon to first navigate enterprise agreements with favored cloud providers or preferred data-centers (in-house or partners) and coordinate (read as haggle) with teams of enterprise architects just to gain access to a decision-making platform. In a startup or small business setting, the process can be more straightforward—often just a hunt for a budget-friendly option to get off the ground. That too can derail if you don't have the right set of resources to get the ball rolling!

But, what fundamentally drives the choice between on-prem and cloud?

More often than not - this decision often hinges on specific use cases, budget constraints, available resources, and the lifecycle stage of the product or service. Over the past decade, with the ries of the hyperscalers - cloud services offer scalability, reliability, and cost-effective flexibility, essential for rapid growth or projects with variable workloads. On the other hand, on-prem infrastructure can provide enhanced control, security, and sometimes significant long-term savings, especially for high-performance or steady, predictable workloads.

Comparing on-premises and cloud infrastructure, an important consideration is resource management. Specialized roles like DevOps, security, and architecture are pre-requisite particularly if the cloud provider doesn’t manage these directly (PaaS setup's like Acquia, Pantheon etc.). These roles come with a mature DevOps teams, BizDevOps practices and nowadays DevSecOps led teams. Too much jargon.

On-premises (depending on scale) demands fewer dedicated resources for day-to-day management. Infrastructure can be set up, optimized, and governed within a more contained environment if you're going with an established provider (or a DC near you!). But as workloads and processes evolve, hybrid tends to be the prevelant approach, leading to dedicated teams at some point in the journey.

Workloads returning on-prem?

I've been hearing of businesses exploring decision to bring workloads back on-prem after initially using cloud services, also known as “cloud repatriation”. Software company 37signals (creators of Basecamp and HEY), which made headlines by projecting savings of up to $7 million over five years by moving their infrastructure back on-prem. As a company focused on predictable usage patterns, they found that owning and managing their hardware could deliver both financial savings and greater performance control than what the cloud offered for their needs.

BP’s subsidiary experienced challenges in managing cloud costs for high-performance computing (HPC) tasks related to seismic processing. For these intensive calculations, on-prem infrastructure turned out to be more cost-effective than the cloud due to the predictably high computing requirements. They achieved the required performance levels without the recurring expenses that came with their cloud setup.

Counter cases - businesses moving to the cloud

Capital One made the decision to migrate its data centers to the cloud after struggling with the complexities of maintaining in-house infrastructure. With its extensive data needs, they found it challenging to efficiently scale and manage on-premises resources while ensuring high levels of security, compliance, and data integrity.

Another example is Netflix, which transitioned its operations to Amazon Web Services (AWS) due to challenges with on-premises scalability. As Netflix’s user base rapidly expanded, its on-prem infrastructure was not able to keep up with the spikes in demand and high availability requirements. Moving to the cloud allowed Netflix to leverage global data centers, on-demand scaling, and robust failover mechanisms — all of which are more challenging to maintain and costly to implement in an on-prem setting.

Considerations

Having been part of DevOps teams where we had to deliver projects of this nature - when evaluating cloud providers, it's important to consider the potential challenges of switching providers in the future, especially as your workloads expand or evolve. A lift-and-shift migration — where workloads are moved as-is to the cloud without refactoring — can initially simplify cloud adoption. Over time - workloads often evolve in ways that are tightly coupled to the chosen provider’s tools, APIs, and infrastructure. This dependency can lead to cloud lock-in, where shifting to a new provider later becomes highly complex and costly, due to incompatible architectures, data transfer fees, and the need for additional expertise to adapt systems to a new environment.

The lock-in drama isn’t limited to cloud solutions, on-premises infrastructure providers can also create dependencies, particularly when resources require specialized hardware or software unique to the provider. Good luck trying to keep up with the evolving fine print as new partnerships are forged, or device manufacturers go under or merge. Scaling or upgrading these on-prem solutions may incur rising costs over time - best to negotiate MSA's with gusto as proprietary solutions and vendor-specific maintenance agreements may cause grief. These challenges can effectively hold on-prem resources “hostage” to escalating fees associated with managing and scaling resources.

Future outlook

As cloud providers push forward with innovations in AI, machine learning, and edge computing, they offer capabilities that can be hard to match with in-house solutions. Cloud economies of scale, the flexibility to scale resources dynamically, and rapid advancements make the cloud appealing for many projects. For businesses with specific performance, security, or cost considerations, on-prem can provide a viable and sometimes preferable solution—especially with the rise of hybrid and multi-cloud models that allow a mix of cloud and on-prem resources.

Good luck navigating through the myriad of options the infrastructure world has to offer, stay safe, and within budget!