The objective of this model is to bridge the divide between language models and human-like reasoning abilities, according to researchers.
The Evolution of AI
When engaging in introspection, an AI appears to embark on a journey of self-improvement.
A collaboration between Stanford University and a group called “Notbad AI” resulted in the development of an AI system that refrains from immediately providing solutions. Instead, it reveals its thought process, solicits feedback on the accuracy of its responses, and does so in an unpublished research paper.
The creators of Quiet Self-Taught Reasoner (Quiet-STAR) aimed for their model not only to enhance its reasoning capabilities, a milestone they accomplished in 2022 with the original Self-Taught Reasoner algorithm, but also to do so discreetly before presenting conclusions. This approach mirrors the internal dialogue of a human mind that precedes verbal communication.
An enthusiastic Eric Zelikman from Stanford shared the excitement surrounding this innovative model in a recent Twitter thread, highlighting how self-directed learning from various online texts significantly enhances reasoning abilities!
The Foundation of Progress
Quiet-STAR was constructed based on Mistral 7B, an open-source large language model (LLM) deemed more efficient than Meta’s latest Llama unit.
Essentially, Quiet-STAR was designed to elucidate its reasoning process while providing answers, allowing users to discern the most accurate response. This methodology, as outlined in the research paper, resulted in a 47.2 percent accuracy rate—an improvement from the initial 36.3 percent achieved without the supplementary reasoning training.
Initially, the pre-trained Quiet-STAR only answered 5.9 percent of questions correctly. Through training, its proficiency in mathematics doubled, albeit still performing poorly, with only a 10.9 percent accuracy rate.
While these results are not groundbreaking, they are compelling due to the historical struggle of chatbots like OpenAI’s ChatGPT and Google’s Gemini in common-sense reasoning tasks. The researchers suggest that Quiet-STAR could pave the way for advancements that narrow the gap between language models and human-like reasoning capabilities.
Could OpenAI’s enigmatic Q* model, with its striking resemblance to Quiet-STAR, hold the key to this mystery? Only time will reveal the answer.
