Russian tech giant Yandex has released a new open source tool to help AI companies save money and resources when training new models.
In his announcementAccording to Yandex, the new YaFSDP tool can provide faster and more efficient large language model (LLM) training and help save up to 20% of GPU resources, saving money and time.
Yandex also published resources for trying out YaFSDP GitHub for everyone to view and experiment with.
How to train your LLM
Yandex lays out many technical details in its announcement post, saying that the new open-source tool is “the most effective, publicly available tool for improving GPU communications and reducing memory usage in LLM training,” and up to 26% higher speeds than FSDP, the version before YaFSDP.
“LLM training is a time-consuming and labor-intensive process,” says Yandex. “Machine learning engineers and companies developing their own LLMs invest a lot of time and GPU resources (which equals money) in training these models. The larger the model, the greater the time and cost involved in training connected.”
The company estimates that using YaFSDP to train a model with 70 billion parameters can save the resources of approximately 150 GPUs, which translates to approximately $0.5 to $1.5 million (depending on the virtual GPU provider or the platform) in monthly savings, if all goes well.
Using Meta’s Llama models, which are praised for being industry-leading and embracing open source, Yandex says the “final speedup” is 21% and 26% on Llama 2 70B and Llama 3 70B, respectively.
YaFSDP isn’t Yandex’s first foray into AI tools, having released CatBoost, YTsaurus, AQLM and Petals, each covering a different niche within AI development.
Yandex’s move comes after it had to divest itself following Russia’s invasion of Ukraine. to sell its domestic operations to its previous management in the country.
Many major tech companies are also making AI the foundation of their products, most recently Apple releasing its Apple Intelligence services as part of the upcoming iOS 18 update.