Summary
Overview
Jacob Coxon, a former AI researcher at OpenAI and Anthropic, explains his decision to publicly resign and warn that AI could pose an existential threat to humanity by the end of the decade. The episode explores his journey from fascination with AI capabilities to growing alarm about the technology's rapid progress, the internal cultures at leading AI labs, and the specific incidents that convinced him the risks are real and imminent. His viral resignation thread sparked unprecedented public statements from industry leaders, including calls for a global slowdown in AI development.
Jacob's Journey into AI and Early Fascination
Jacob Coxon describes his introduction to artificial intelligence through landmark achievements like AlphaGo defeating the world champion at Go in 2016 and GPT-3's release during the pandemic. These breakthroughs convinced him that AI would become more important than anything else, including mathematics, his field of study. He was drawn to the technology's potential for general intelligence, even as he recognized it made people "crazy" to think about the implications of labor obsolescence and the approaching singularity.
- AlphaGo beating Lee Sedol at Go in 2016 was Jacob's first experience with truly impressive AI capabilities
- GPT-3's release showed evidence of general capabilities emerging from training procedures
- By early 2020s, Jacob was convinced the singularity would happen at some point
- Thinking about AI and its implications made people "insane" - one colleague believed he made people conscious by interacting with them
" It's the point beyond which you can't see. It's like a sort of wall when you look into the future. And the reason for that is everything gets faster. So imagine you're training in AI, and it makes itself smarter so that what previously took six months now takes three months. "
" I knew a guy who like, he was convinced that he made people conscious by interacting with them. He was so close to AI. He was like, I'm in a simulation. Other people, if they're not involved in AI, aren't conscious. But if I interact with them, they become part of the AI thing. So they become conscious. "
Working at OpenAI: Data Curation and Rapid Progress
Jacob joined OpenAI to work on data curation, determining which types of training data would make AI systems more intelligent. He describes the company culture as similar to a well-funded university research department with free food and constant discussions about AI's future. During his time there, he witnessed AI capabilities advancing far faster than anyone predicted, with achievements like winning gold at the International Math Olympiad happening years ahead of even optimistic timelines.
- Jacob's role involved testing different types of data to determine what made AI models more intelligent
- OpenAI had an open floor office culture with whiteboards, meeting rooms, and constant technical discussions
- An AI system achieved gold medal at International Math Olympiad, something Jacob thought would take 20 years
- AI went from struggling with basic math to solving competition-level problems in just two years
" Almost everyone, apart from the most optimistic of people, have seen their expectations repeatedly beaten. It's moving faster than anyone predicted at basically every juncture. "
📚 5 more sections below
Sign up to unlock the complete summary with all insights, key points, and quotes
Get this summary + all future The Daily episodes in your inbox
100% Free • Unsubscribe Anytime
Sign up now and we'll send you the complete summary of this episode, plus get notified when new The Daily episodes are released—delivered straight to your inbox within minutes.