Generative AI models require significant computing power and hardware support. As generative AI applications gain popularity, their computing needs grow exponentially. Large tech companies such as Google, Microsoft, and Amazon lead the way in software and data centre requirements while Nvidia dominates the GPU market. However, Nvidia has struggled to keep pace with the increasing demand for GPUs, resulting in supply shortages. In response, companies are adopting a vertical integration approach and exploring in-house chip design to address the GPU shortage. While this approach can diversify the supply chain away from a single firm, it may also hinder entry for smaller firms. Innovative solutions like GPU renting and venture capital offerings have emerged in this evolving landscape; however, scalable industry solutions, such as competitive pricing models for cloud services, have yet to emerge.
Generative artificial intelligence (AI) models demand significant programming inputs and computing power. [1] This includes efficient algorithms and training datasets, crucial for enhancing model accuracy, along with supporting hardware like data centres and graphics processing units (GPUs), semiconductor chips that power different stages of model development, training, and inference. [2] Typically, different firms are present at different stages of the product supply chain.[3] For example, OpenAI’s ChatGPT 3.0 is trained on a dataset with 175 billion parameters developed in-house,[4] while the infrastructure is powered by Microsoft’s cloud-based supercomputer with 10,000 Nvidia GPUs.[5] However, there's evidence suggesting that as generative AI models' computational needs grow, tech firms are seizing control over multiple supply chain stages, potentially erecting barriers to entry for smaller firms.
With the growing popularity of generative AI applications, tech companies are competing to improve their model’s predictive capabilities. This has led to an exponential increase in their computing needs as well. The amount of computing power needed to train AI algorithms is growing at an estimated 3.4-month doubling speed, according to OpenAI.[6]
Large tech companies such as Google, Microsoft and Apple can easily fulfil software demands by refining existing algorithms and furnishing supplementary training data, at minimal costs. In terms of infrastructural requirements, most of these firms have their own data centres.[7] However, the majority of them are dependent on chip manufacturers to fulfil their GPU requirements. Currently, this dependency is concentrated towards a single firm, Nvidia, which is the world leader in the GPU market, with an estimated 80 percent share as of 2024.[8] Nvidia’s GPUs are world renowned for utilising a parallel processing method which significantly reduces the computing time needed to train generative AI models.[9]
However, Nvidia has struggled to keep up with the rapid growth in demand for GPUs, leading to a supply shortage in 2023.[10] According to industry estimates, the demand for Nvidia’s chips is at least 50 percent higher than its supply capabilities.[11] Moreover, Nvidia's supply chain is reliant on the Taiwan Semiconductor Manufacturing Company (TSMC) for production, exposing it to shocks from events like the U.S.-China trade war, including export controls on semiconductor chips, further complicating GPU supply for firms.
To mitigate risks arising from GPU shortage, tech companies are planning to design their own AI chips. For example, in 2023, OpenAI was reportedly exploring options to make AI chips in-house, citing supply shortage and the existing cost of GPUs as two major concerns.[12] Microsoft is also taking a similar approach, with a “silicon to service” strategy, and has already built its custom-designed chips and integrated systems to accompany its AI accelerator. In 2017, Google also launched its first AI chip called Tensor Processing Units (TPUs) to train neural networks. Since then, the chip design has been updated multiple times, with the latest edition released in 2023.[13]
These examples illustrate how traditional boundaries between software and hardware companies are becoming increasingly blurred in the AI industry. Large tech firms are taking a vertical integration approach and establishing their presence across all stages of the supply chain. While this could diversify chip dependence across multiple firms, it might also create barriers to entry for smaller firms and startups in the AI domain. Semiconductor chip design and manufacturing entail high technical and R&D investments,[14] making it challenging for small companies to meet GPU demands without relying on larger platforms. [15]
Some companies have found innovative solutions to this issue. For instance, Index Venture, a venture capital (VC) fund is purchasing GPUs and offering them as part of their portfolio to companies.[16] Founders are forming research groups to pool resources and buy clusters of GPUs for dynamic sharing. GPU-renting companies have also sprung up, for example, the San Francisco Compute Group rents out Nvidia chips to interested parties.[17]
While these solutions are note-worthy, a scalable resolution is likely to emerge from tech companies in the form of subscription-based cloud services. Microsoft reportedly plans to offer subscription software as part of its Azure cloud computing service.[18] However, competitive pricing will only materialise if multiple firms embrace this subscription-based strategy, fostering healthy competition and driving innovation. Whether tech companies will monetize this new revenue stream competitively or opt for a more self-serving approach remains to be seen, casting a spotlight on the evolving dynamics of the AI industry.
—
[1] Sastry et al (2024)
[2] Openai.com (n.d.)
[3] Nasdaq (2023)
[4] Openai.com (2023)
[5] Reuters (2023); Harvard (2023)
[6] Openai.com (n.d.)
[7] Google.com (n.d.); Microsoft.com (n.d.)
[8] Ibid. Reuters.
[9] The Economist (2024)
[10] Wired (2023)
[11] The Guardian (2023)
[12] Reuters (2023).
[13] Google (n.d.)
[14] BCG and SIA (2021). “Strengthening the Global Semiconductor Value Chain”. Available at: https://www.semiconductors.org/wp-content/uploads/2021/05/BCG-x-SIA-Strengthening-the-Global-Semiconductor-Value-Chain-April-2021_1.pdf
[15] New York Times (2023)
[16] Ibid. New York Times.
[17] sfcompute.com (n.d.)
[18] hfsresearch.com (n.d.)