Summary
Overview
This episode features an in-depth technical discussion about the state of AI in 2025-2026 with machine learning researchers Sebastian Raschka and Nathan Lambert. The conversation covers the competitive landscape of frontier AI models, the DeepSeek moment, scaling laws across pre-training and post-training, open vs. closed models, reinforcement learning breakthroughs, the future of programming with AI, and predictions for AGI timelines. The discussion emphasizes practical developments in language models, the economics of AI development, and the human implications of rapid AI progress.
The DeepSeek Moment and International AI Competition
The discussion begins with the January 2025 release of DeepSeek R1, which surprised the AI community with near state-of-the-art performance at allegedly much lower cost. This sparked intense competition between Chinese and American AI companies. The conversation explores whether there's a clear winner in the international race, with both guests arguing against a winner-takes-all scenario due to the free flow of researchers and ideas across organizations.
- DeepSeek R1 released in January 2025 surprised everyone with near state-of-the-art performance with allegedly much less compute
- Ideas in AI are not proprietary anymore because researchers frequently change jobs and rotate between labs
- The differentiating factor will be budget and hardware constraints, not access to ideas
- Chinese companies are releasing strong open-weight models that may be taking DeepSeek's crown
" I don't think nowadays, 2026, that there will be any company who is, let's say, having access to a technology that no other company has access to. And that is mainly because researchers are frequently changing jobs, changing labs. "
Open Weight Models and Chinese Innovation
The conversation explores the explosion of open-weight models from Chinese companies including DeepSeek, Kimi, Minimax, and Z.AI. Chinese companies are incentivized to release open models because US companies won't subscribe to their APIs for security reasons, so open-weight releases allow them to influence the global AI market. The guests discuss why these models remain open and what this means for the competitive landscape.
- Chinese companies see open-weight models as a way to influence and capture part of the AI expenditure market in the US
- DeepSeek is losing its crown as preeminent open model maker in China to companies like Z.AI, Minimax, and Kimi
- Chinese model licenses are more permissive than Llama or Gemma, with no user limits or financial reporting requirements
- Nathan predicts consolidation will happen but expects more open model builders in 2026 than 2025
" A lot of US tech companies and other IT companies won't pay for a API subscription to Chinese companies for security concerns. And the people at these companies then see open weight models as an ability to influence and take part of a huge growing AI expenditure market in the U.S. "
📚 10 more sections below
Sign up to unlock the complete summary with all insights, key points, and quotes
Get this summary + all future Lex Fridman Podcast episodes in your inbox
100% Free • Unsubscribe Anytime
Sign up now and we'll send you the complete summary of this episode, plus get notified when new Lex Fridman Podcast episodes are released—delivered straight to your inbox within minutes.