New research from OpenAI highlights the results of their reasoning models (o-series models) and how LLMs have evolved from amateur competitive programmers to competing with the world’s best.
OpenAI’s latest AI model, o3, earned an impressive 2724 rating on CodeForces, placing it in the 99.8th percentile. It also secured a gold medal-level score at the 2024 International Olympiad in Informatics (IOI).
According to the research, o3 outperforms the o1-ioi model, which is specifically fine-tuned for IOI. This proves that reinforcement learning is more effective than hand-crafted approaches.
At IOI 2024, o3 competed under standard conditions and crossed the gold medal threshold. On CodeForces, it ranked among the top 200 programmers globally, competing with elite human coders.
“General-purpose reasoning capabilities developed through reinforcement learning are now outperforming carefully hand-crafted, domain-specific solutions,” said Ethan Mollick, associate professor at The Wharton School. “Rather than building specialised systems for specific tasks, large, general-purpose models can achieve superior results through better reasoning abilities.”
The research is part of OpenAI’s ongoing efforts to assess its models’ performance in competitive programming and broader software engineering.
Anthropic, the company behind the Claude model series, also released a report on Monday which highlighted AI’s influence on the workplace.
The findings revealed that approximately 36% of all occupations incorporate AI for at least a quarter of their tasks. Moreover, 57% of AI applications enhance human capabilities, while 43% focus on automation. However, only 4% of occupations rely on AI for at least 75% of their tasks.
The study identified software development and technical writing as the primary areas where AI is utilised. In contrast, AI plays a minimal role in tasks that involve physical interaction with the environment.
