>
A new text-generating language model, which combines Google’s proprietary PaLM model with a technique known as Reinforcement Learning with Human Feedback to create a open-source tool that can, in theory, do everything OpenAI’s ChatGPT can.
For most, however, this will remain a theory. Unlike ChatGPT, Philip Wang’s AI developer PaLM + RLHF (opens in new tab) does not come trained on text data needed for the model to learn from. Users must build their own data sets and use their own hardware to train the model and process requests.
Text generation models that respond to human input, such as ChatGPT and PaLM + RLHF, are the latest craze in artificial intelligence. Simply put, they predict the right words after learning semantic patterns from an existing dataset, which can include anything from ebooks to internet flame wars.
Creating accessible artificial intelligence
Despite PaLM + RLHF being pre-trained, the Reinforcement Learning with Human Feedback technique is designed to produce a more intuitive user experience.
As explained by TechCrunch (opens in new tab), RLHF trains a language model by producing a wide variety of responses to a human prompt, which are then ranked by human volunteers. Those rankings are then used to train a “reward model,” which sorts the answers by order of preference.
This is not a cheap process, which will prevent all but the wealthiest AI enthusiasts from training the model. PaLM has 540 billion language model components (or parameters) to be trained on data, and a 2020 study (opens in new tab) revealed that training just a model with 1.6 billion parameters would cost anywhere from $80,000 to $1.6 million.
At this point, it seems like we’re relying on a wealthy benefactor to join in and train the model and release it to the public. Such dependencies did not end well before (opens in new tab)but there are existing efforts by other companies that want to replicate ChatGPT’s capabilities and release it as free software.
Research groups CarperAI and EleutherAI are teaming up with startups Scale AI and Hugging Face to release the first language model trained with human feedback and ready to go out of the box.
And while it’s not quite done yet, LAION, the company that provided the training dataset for the “machine learning, text-to-image” model Stable spread (opens in new tab)have made a similar project on GitHub (opens in new tab) which aims to surpass OpenAI by allowing it to use APIs, curate its own research, and enable user personalization, all while optimizing it for consumer hardware.