OpenAI, the company behind the well-known artificial intelligence (AI) language model ChatGPT, is developing a novel approach to enhancing its AI models.
The project, code-named “Strawberry,” represents a significant effort to improve the advanced reasoning capabilities of AI.
This initiative is crucial as the company aims to demonstrate that its models can go beyond generating answers to queries and start planning ahead, going through the internet autonomously, and performing deep research.
Strawberry
The details of the Strawberry project have been kept under wraps until now.
However, according to a person familiar with the matter and the internal documentation reviewed, teams inside OpenAI are actively working on Project Strawberry.
A recent internal document seen by a leading media company in May outlines how OpenAI intends to leverage Strawberry for research purposes. Although the exact date of the document is unknown, it describes a plan that is still a work in progress.
Although, it remains unclear when Strawberry will be publicly available.
A Secretive Project
Strawberry’s workings are a closely guarded secret within OpenAI.
The project aims to enable AI models to generate answers and plan and navigate the internet autonomously, conducting what OpenAI terms “deep research.” According to interviews with over a dozen AI researchers, this level of capability has been elusive for AI models so far.
Enhancing AI Reasoning
An OpenAI spokesperson commented on Strawberry, stating,
“We want our AI models to see and understand the world more like we do. Continuous research into new AI capabilities is a common practice in the industry, with a shared belief that these systems will improve in reasoning over time.”
This indicates OpenAI’s commitment to advancing the reasoning abilities of its AI models.
Previously known as Q*, the Strawberry project has been seen within OpenAI as a breakthrough.
Earlier this year, two sources described seeing Q* demos that could answer complex science and math questions beyond the reach of today’s commercially available models.
In a recent internal all-hands meeting, OpenAI demonstrated a research project showcasing new human-like reasoning skills. Although it is not confirmed if this project was Strawberry, it indicates the company’s progress in this domain.
The Importance of Reasoning in AI
AI researchers generally agree that reasoning is crucial for AI to achieve human or super-human-level intelligence. While large language models can summarize dense texts and compose elegant prose efficiently, they often struggle with common-sense problems and logical fallacies, sometimes generating incorrect information.
Thus, improving reasoning is seen as the key to enabling AI models to make significant scientific discoveries and develop new software applications.
Sam Altman, CEO of OpenAI, emphasized earlier this year that progress in AI will heavily focus on enhancing reasoning abilities. Other tech giants like Google, Meta, and Microsoft are also exploring different techniques to improve reasoning in AI models.
However, there is a debate among researchers about whether large language models can incorporate long-term planning and human-like reasoning.
Yann LeCun of Meta, a pioneer in modern AI, has frequently expressed skepticism about the ability of LLMs to achieve human-like reasoning.
Overcoming AI Challenges with Project Strawberry
Strawberry is a crucial element of OpenAI’s strategy to address the challenges associated with advancing AI reasoning capabilities, according to a source familiar with the matter.
While the document reviewed outlines Strawberry’s goals, it does not discuss the specifics of how these goals will be achieved.
In recent months, OpenAI has been privately indicating to developers and other external parties that it is on the verge of launching technology with significantly enhanced reasoning capabilities.
This information comes from four individuals who have heard the company’s presentations but chose to remain anonymous as they are not authorized to discuss private matters publicly.
Strawberry involves a specialized method known as “post-training” for refining OpenAI’s generative AI models. This technique adapts base models to improve their performance in specific ways after they have already been trained on large datasets of generalized information.
One of the sources explained that this post-training phase includes methods like “fine-tuning,” where human feedback is used to adjust the model’s responses, providing examples of both good and bad answers.
Strawberry shares similarities with a method called “Self-Taught Reasoner” (STaR), developed at Stanford in 2022.
According to a source with knowledge of the matter, STaR allows AI models to iteratively create their own training data, potentially enabling them to surpass human-level intelligence.
Noah Goodman, a Stanford professor and one of the creators of STaR, described this potential as both exciting and terrifying, noting that if AI continues in this direction, humanity will have serious considerations to address. Goodman is not affiliated with OpenAI and is not familiar with Strawberry.
OpenAI aims for Strawberry to perform long-horizon tasks (LHT), which require the model to plan ahead and execute a series of actions over an extended period. To achieve this, OpenAI is developing, training, and evaluating the models using what the company refers to as a “deep-research” dataset.
However, the specifics of this dataset and the duration implied by “extended period” remain unclear.
Additionally, OpenAI intends for its models to use these capabilities to conduct autonomous web research with the help of a “Computer-Using Agent” (CUA). This agent can take actions based on the information it gathers.
Moreover, OpenAI plans to test Strawberry’s abilities in performing tasks typically handled by software and machine learning engineers.
The Last Bit, OpenAI’s Strawberry project represents a crucial step towards advancing the reasoning capabilities of AI models.
By enabling these models to plan ahead, navigate the internet autonomously, and conduct deep research, OpenAI aims to push the boundaries of what AI can achieve.
As the field continues to evolve, the pursuit of enhanced reasoning abilities will remain a central focus for researchers and companies alike, driving further innovation and progress in artificial intelligence.
