As data becomes the foundation of our economy, we must find ways to diversify the use of computing resources that harness its potential, and reduce our dependence on just a handful of models and manufacturers. That will prove to be one of the best ways to support the AI revolution as competition for GPUs increases, and will ensure that we support innovation rather than relying on just a few players controlling the market. We’re already seeing new approaches that allow your gaming PC to power new AI models.
Market concentration in the chip industry has led to shortages of most of the GPUs that train these models, raising concerns among CTOs and government policymakers alike. And while Big Tech has the resources and power to secure what is available, smaller companies often delay projects as they struggle to leverage their models. That’s because GPUs are difficult to contract if companies don’t buy in bulk, and they’re expensive, costing between $600,000 and $1 million to train even the most basic large language models (LLMs) – an prohibitive price tag for many. Driving towards a more diversified chip usage landscape is not just a pragmatic response to current challenges; it is a proactive attitude to future-proof our technological evolution and ensure the continued vitality of the AI ecosystem.
Founder of Akash Network and CEO of Overclock Labs.
Misplaced solution
The popular solution to the supply shortage seems to be to increase production of ultra-advanced chips – especially the powerful A100s and H100s – and for other tech giants to produce similar components. While that’s good news for the largest AI companies, it does little to reduce market concentration and lower prices. And most importantly, it fails to make AI acceleration hardware more accessible to smaller players. The massive orders for high-end GPUs reduce availability for other organizations looking to gain a foothold in AI training. It also allows the big tech companies to maintain their pricing power and softens incentives that could otherwise drive crucial innovations in the sector. And as more and more powerful GPUs are built, there’s an attitude emerging where a company’s ability to acquire the biggest, baddest, and newest models becomes a competitive advantage.
That thinking is misleading – or at least under-exposed – because existing technologies and new techniques offer a way to diversify the use of chips and enable startups to secure their computing. Over the next three to five years, we’ll see AI companies working with a wider range of GPUs – from the highly advanced to the less powerful – which will free up the market and unleash a new wave of innovation. This strategic pivot holds the promise to free the market from the grip of high-end exclusivity, ushering in a more inclusive, dynamic and resilient AI ecosystem, primed for sustainable growth and creativity.
Adult space
The maturation of the AI space will drive much of this change, as we will see more language models tailored to specific niches, rather than the one-size-fits-all LLMs like ChatGPT and Claude. This diversification not only meets the unique demands of different industries and applications, but also marks a break from the homogeneity that has characterized the AI landscape to date. And developers will continue to refine their models with less powerful chips, motivating them to seek access to consumer-grade GPUs that offer efficiency and accessibility. This departure from reliance on high-end components democratizes access to computing resources and drives innovation by challenging the industry-wide assumption that only the most advanced chips can facilitate breakthrough advances in AI.
To some extent this is already happening, as developers use efficient techniques such as low-rank adaptation (LoRA) that reduce the number of training variables in language models. They also parallelize the workload, deploying clusters of, say, 100,000 smaller chips to do the work of 10,000 H100s. These solutions could unleash a wave of innovation away from the “bigger is better” arms race in the chip market – one characterized by a focus on efficiency, collaboration and inventive problem solving.
Meanwhile, existing technologies, including Kubernetes and open source cloud infrastructure, will provide access to these less powerful chips. Individuals and organizations that own GPUs will be able to sell or rent their capacity on these networks, as we are already starting to see on some projects. The intersection of technology and community-driven initiatives presents an opportunity to break down barriers, both economic and technological, and foster an environment where computing power is not limited to a select few, but widely distributed across a wide range of contributors.
Second wave
In the not-too-distant future, this market has the potential to expand even further, with owners of consumer-grade GPUs making unused capacity available to AI companies – especially as major players boost consumer GPUs at nearly a third of the cost of high-end models that allow AI to run on everyday PCs or laptops. The use of gaming GPUs has the potential to accelerate innovation cycles as they are updated annually; This could allow AI training to leverage new architectural improvements more quickly than specialized enterprise hardware, which evolves more slowly.
Since many everyday objects have GPUs, this opens up a world of opportunities for people to monetize unused computing power. Think of blockchain miners pointing their GPUs at cloud markets when their projects go to proof of stake, or students doing the same with gaming PCs when they’re not playing. Additionally, smaller and more efficient AI models can run on personal devices. We’re already seeing Gemini Nano able to work offline on Google’s Pixel 8 devices; This makes locally hosted AI models on mobile devices real.
These developments could provide new sources of revenue for providers and additional GPU offerings for startups. To be fair, none of this replaces the need for top quality GPUs. But it will reduce market concentration, making companies less dependent on a single company (or country) producing the chips they need. We will have a mature, complex market where GPUs of different speeds and quality play a crucial role in a range of AI projects. That will usher in a second wave of AI innovation that will benefit everyone. As the AI landscape evolves, the merger of consumer-grade GPUs with AI capabilities will unleash unprecedented opportunities for innovation and collaboration across industries, and have profound economic impacts by spreading capability across a broader segment of society.
We’ve highlighted the best AI writers.
This article was produced as part of Ny BreakingPro’s Expert Insights channel, where we profile the best and brightest minds in today’s technology industry. The views expressed here are those of the author and are not necessarily those of Ny BreakingPro or Future plc. If you are interested in contributing, you can read more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro