Tiny corp., a startup that operates the Tinygrad neural network framework, has unveiled a new AI system that uses six AMD Radeon RX 7900 XTX graphics cards. The tinybox, which costs $15,000, is intended to democratize PetaFLOPS-class artificial intelligence performance.
The tinybox system is a compact powerhouse, packing six of AMD’s fastest gaming GPUs into a 12U rack enclosure. The GPUs are connected via full fabric PCIe 4.0 x16 links to ensure maximum bandwidth. Small company. says it chose the Radeon RX 7900 XTX over Nvidia’s GeForce RTX 4090 because the GPUs fully support the peer-to-peer interconnections needed for LLMs.
Tom’s hardware reports that “the tinybox can deliver up to 738 FP16 TFLOPS (0.738 FP16 TFLOPS) of performance, along with 96 GB of GDDR6 memory that delivers 21 TB/s of aggregated memory bandwidth. To put the numbers in context: one tinybox offers 37% of the Nvidia H100 compute performance (FP16), but slightly more memory (96 GB instead of 80 GB) and a significantly higher peak memory bandwidth (21 TB/s instead of 3 .35 TB/s).
Shipping soon
The tinybox contains five 1TB Western Digital SN850X SSDs (4 in RAID, 1 for boot) and also features an empty 16x OCP 3.0 slot for networking. It ships with Ubuntu 22.04 and the tinygrad framework, although tinycorp is happy to allow buyers to run other machine learning frameworks like PyTorch and JAX on AMD hardware. As the company puts it, “your hardware is your hardware… you are welcome to buy it for any purpose.”
However, not everyone is enthusiastic about the idea of the tiny box. Like a Tom’s hardware reader commented: “Oh cool, another startup that will cause gaming GPU prices to rise.”
The BOM cost of the tinybox machine is reportedly around $10,000, but the company will sell it for $15,000, which is significantly lower than Nvidia’s H100, which can sell for more than $40,000. So far, the company says it has received 583 pre-orders and plans to start shipping in April. It is possible Pre-order the Tinybox for $100.
The company says about the future of its creation: “We will push the boundaries of what is possible with this box. The hardware is solid, but the software is a huge journey. I look forward to going on that journey with all of you Tinybox owners soon. To the metal. We will squeeze every last FLOP out of this thing.”