The future of AI: To cloud or not to cloud?

Do you remember the good old days of software development before AWS? Many of us are still a bit nostalgic about CGI, the Apache stack, and deployments via FTP. Nevertheless, the shift to AWS and the broader modern cloud computing felt like magic when it began in the early 2010s. Having experienced the “before and after,” it might seem like no one would ever want to return to owning their workloads when life can be so much easier in the cloud. But has the tech community sentiment started to shift?  

The recent CloudFest survey [1] shows that a change in attitude, in particular with the corporate IT teams, might be underway. The share of people who said that the opportunities created by cloud adoption outweighed the risk has fallen from 61% to 48%. Interestingly, most optimism now comes from cloud vendors, while customers tend to shift to neutrality instead of earlier positiveness. This is reflected in the revenue growth of cloud providers that, despite a recent spike due to AI workloads, was consistently slowing down.

Multiple explanations for this process have been put forward, ranging from saturation to overspending during the COVID-19 pandemic.

I consider saturation to be a likely explanation. Since its advent 10-15 years ago, cloud computing has enabled many businesses to start and scale. It made enterprise software cheaper to build and operate while reducing the barriers to startups entering the market. Popularity came with a lot of “tech evangelism,” where people migrated software that didn’t necessarily have to be moved into the public cloud. The usual terminology you’d encounter at conferences in 2018-2019 was “cloud-lagging,” often used synonymously with “bad tech” or “a legacy system no one ever should touch unless intending to rebuild from scratch.” 

Nevertheless, even a hardline cloud evangelist would struggle to justify that every software built from 2010 onwards must either be entirely based in a public cloud or considered dead at birth. A natural conclusion is that the systems that could benefit from being deployed in the cloud have already, to some extent, been moved there. Call it “approaching the asymptote” if you’d like.

So why do businesses consciously choose to keep some of their workloads internal?

1. Costs.
   By now, we’ve already established that, despite vendor promises, the cloud is not always cheaper than owning your software or infrastructure. In particular, businesses with a stable and consistent stream of requests often find hosting in a private cloud or on-premise more affordable.
2. Security.
   According to Checkpoint, cloud infrastructure often includes multiple vendors, each one with its own specific nuances [2]. Misconfigurations were the leading source of data leaks and other breaches, usually stemming from misspecified access privileges or incorrect use of the security tools specific to the vendor.
   It seems like a problem you might also have with your own platform. It’s a yes and no. While a properly configured public cloud would eliminate many risks, it is still inherently riskier than your own infrastructure since every non-air-gapped asset is an extra attack surface to be exploited.
3. Flexibility.
   The larger you are, the more custom your use cases tend to be. You may have legacy systems that would cost billions to rebuild. You may have use cases highly specific to your industry or country. As the Nvidia & Supermicro paper noted [3], smaller custom models address most enterprise AI tasks better than large API-provided generalist solutions. The same is true for many other domains: a cloud solution is built to work for everyone but not necessarily for you.

Overall, migration to the cloud was often justified by getting access to better tools and software rather than fundamental advantages of the cloud computing model for a particular use case. So where will we be in 10 years?

As the recent KPMG survey showed, the growth of hybrid cloud-specific solutions has outpaced the growth of cloud-native systems in recent years [4]. The hybrid model is a healthy mix of private and public clouds – you shift inconsistent or scaling workloads into public clouds while keeping sensitive data and stable workloads internal. We will likely witness more and more software developed to support this trend – better developer tools and execution environments that will make private cloud development as pleasant and fast as public cloud-based engineering. In the AI sector, many tools already exist to help developers build internally: from granular open-source projects like Ollama, Llama.cpp, and LangChain built to tackle narrowly defined problems to entire containerized development environments that orchestrate and manage a state-of-the-art framework ecosystem like Byne does. 

Public cloud is not going away anytime soon (and certainly will continue growing at an astonishing rate). As we learn and explore this fairly new model of computation (often the hard way by observing soaring infrastructure bills), we see more of its limitations. Cloud evangelism will likely be slowly replaced by a more granular, workload-specific approach where organizations learn to integrate private cloud, public cloud, and bare metal to build highly cost-optimal and safe systems. There will be more software enabling them to do so. And it’s not too early to start now – we will support your GenAI hybrid cloud journey by providing tools to integrate public and internal systems seamlessly, creating a granular split of workloads per their characteristics like heaviness or execution frequency. 

[1] https://www.cloudfest.com/wp-content/uploads/2024/03/CloudFest-State-of-the-Cloud-Report-2024.pdf

[2] https://pages.checkpoint.com/2023-cloud-security-report.html

[3] https://www.supermicro.com/thought-leadership/Critical-AI-Challenges-30105-Nvidia-Supermicro.pdf

[4] https://assets.kpmg.com/content/dam/kpmg/uk/pdf/2024/04/architecting-the-future.pdf