In an effort to increase the usefulness of generative AI tools for developers, OpenAI has introduced CriticGPT, a new model that the company says can help identify bugs in ChatGPT code output.
Based on GPT-4, OpenAI claims that CriticGPT outperforms unassisted code in 60% of cases. This shows that CriticGPT can improve human performance on code review tasks rather than replacing human workers.
OpenAI’s initiative aims to refine the ‘Reinforcement Learning from Human Feedback’ (RLHF) process to ensure higher quality and greater reliability in AI systems.
OpenAI launches new code review model
OpenAI’s latest GPT-4 series, which powers publicly available versions of ChatGPT, relies heavily on RLHF to ensure that results are both reliable and interactive. Until now, this process has been manual, relying on the human power of AI trainers, who would score ChatGPT responses to improve the model’s performance.
With the launch of CriticGPT, OpenAI can now autonomously critique ChatGPT’s responses, addressing concerns that the AI chatbot is becoming too sophisticated for many human trainers.
CriticGPT was trained by trainers who provided feedback after intentional errors were introduced into the code generated by ChatGPT. The results were promising, with CriticGPT’s critique being preferred by trainers about two-thirds (63%) of the time, thanks to the tool’s ability to reduce nitpicking and hallucinations.
However, the project is not without limitations, and collaboration between AI and humans continues to prove more effective than AI alone.
In his announcementOpenAI summarized: “CriticGPT’s suggestions are not always correct, but we find that they can help trainers solve many more problems with model-written answers than without AI help.”
The company also acknowledged that “errors can spread across many parts of an answer,” making it more complex for an AI tool to identify the root cause.
Looking ahead, OpenAI has confirmed plans to scale and deploy its work on CriticGPT.