In an impressive breakthrough, researchers have successfully created a low-cost alternative to OpenAI’s o1 “reasoning” model, named s1.
This new model was developed by a team from DeepSeek and other AI labs, who aimed to replicate the performance of OpenAI’s o1 using more efficient and cost-effective methods.
By focusing on “test-time scaling” (giving the model more time to reason before answering), the team found success with their approach, which contrasts with the more expensive reinforcement learning methods typically used.
S1 was built using a relatively small dataset—only 1,000 carefully curated questions—and fine-tuned using supervised fine-tuning (SFT), a cheaper method compared to large-scale reinforcement learning.
This approach allowed the model to be trained in under 30 minutes with 16 Nvidia H100 GPUs at a cost of just $20 in compute time.
The researchers employed a unique strategy to enhance the model’s reasoning by incorporating the instruction to “wait” during the reasoning process.
This simple adjustment helped improve the accuracy of the model’s responses. The project utilized a free, off-the-shelf AI model from Alibaba-owned Chinese lab Qwen, which helped keep the costs down.
While s1 shows strong performance on certain AI benchmarks, it demonstrates that distillation can be a practical method for creating competitive AI models without the enormous costs associated with training models like those developed by major tech companies.
However, it’s important to note that this method doesn’t yet push the boundaries of AI innovation beyond current models.
Leave a comment