OpenAI has spent $80 million to $100 million training GPT-4; Chinese company claims it trained its rival AI model for $3 million using just 2,000 GPUs
- 01.ai trained an AI model for $3 million using 2000 unnamed GPUs
- “Efficient engineering” allows 01.ai to compete globally, the company claims
- 01.ai reduced inference costs to 10 cents per million tokens
Technology companies in China are facing a number of challenges due to the US export ban, which limits access to advanced hardware from US manufacturers.
This includes Nvidia’s cutting-edge GPUs, which are critical for training large-scale AI models, forcing Chinese companies to rely on older or less efficient alternatives, making it difficult to compete globally in the rapidly evolving AI industry.
However, as we have seen time and time again, these seemingly insurmountable challenges are increasingly being overcome through innovative solutions and Chinese ingenuity. Kai-Fu Lee, founder and CEO of 01.ai, recently revealed that his team successfully trained the high-performance model Yi-Lightning with a budget of just $3 million and 2,000 GPUs. By comparison, OpenAI reportedly spent $80-$100 million training GPT-4 and is rumored to have allocated up to $1 billion for GPT-5.
Also draw conclusions quickly
“What shocks my friends in Silicon Valley is not just our performance, but that we trained the model with just $3 million,” Lee said (via @tsarnick).
“We believe in scaling the law, but if you do excellent detailed engineering, it’s not like you have to spend a billion dollars to train a great model. As a company in China, we have limited access in the first place to GPUs because of US regulations, and secondly, Chinese companies are not valued the way US companies are. So when we have less money and struggle to get GPUs, I truly believe necessity is the mother of invention.”
Lee explained that the company’s innovations include reducing computational bottlenecks, developing multi-layer caching and designing a specialized inference engine. According to him, these improvements result in more efficient memory use and optimized training processes.
“If we only have 2,000 GPUs, the team has to figure out how to use them,” Kai-Fu Lee said, without disclosing the type of GPUs. “I, as CEO, have to figure out how to prioritize it, and then not only do we have to make the training fast, we have to make inference fast… The bottom line is that our inference cost is 10 cents per million tokens. .”
For context, that’s about 1/30th of the typical rate charged by comparable models, highlighting the efficiency of 01.ai’s approach.
Some people may be skeptical of the claim that you can train an AI model with limited resources and “excellent engineering,” but according to UC Berkeley’s LMSIS, Yi-Lightning ranks sixth globally in performance, suggesting that no matter what succeeded, 01 .ai has indeed found a way to be competitive on a minuscule budget and limited GPU access.