An AI chip built using old Samsung technology is claimed to be as fast as the Nvidia A100 GPU – the prototype is smaller and much more power efficient, but is it just too good to be true?
Scientists at the Korea Advanced Institute of Science and Technology (KAIST) have unveiled an AI chip that they claim can match the speed of Nvidia’s A100 GPU, but with a smaller size and significantly lower power consumption. The chip was developed using Samsung’s 28-nanometer manufacturing process, a technology considered relatively old in the rapidly changing world of semiconductors.
The team, led by Professor Yoo Hoi-jun of KAIST’s processing-in-memory research center, has developed what it claims is the world’s first Complementary-Transformer (C-Transformer) AI chip. This neuromorphic computing system mimics the structure and functioning of the human brain, using a deep learning model commonly used in visual data processing.
“Neuromorphic computing is a technology that even companies like IBM and Intel have not been able to implement, and we are proud to be the first in the world to perform the LLM with a low-power neuromorphic accelerator,” said Yoo.
Questions remain
This technology learns context and meaning by tracking relationships within data, such as words in a sentence, which is a key technology for generative AI services like ChatGPT.
During a demonstration at the ICT Ministry headquarters, team member Kim Sang-yeob demonstrated the chip’s capabilities. On a laptop equipped with the chip, he performed tasks such as question and answer sessions, sentence additions and translations using OpenAI’s LLM, GPT-2. The tasks were completed at least three times faster and in some cases up to nine times faster than when running GPT-2 on an Internet-connected laptop.
Implementing LLMs in generative AI tasks typically requires numerous GPUs and 250 watts of power, but the team claims their semiconductor uses only 1/625 the power of Nvidia’s GPU for the same tasks. Furthermore, because it also covers 1/41 of the size and measures just 4.5mm by 4.5mm, it could eventually be used in devices such as mobile phones.
However, whether the chip can deliver on its promises in real-world applications remains to be seen. If Tom’s hardware reports: “While we have been told that the KAIST C-Transformer chip can perform the same LLM processing tasks as one of Nvidia’s high-performance A100 GPUs, we have not provided any direct comparative performance data in any of the press or conference materials. That’s an important statistic, notable by its absence, and cynics would probably suspect that a performance comparison doesn’t give the C-Transformer any advantage.”