Summary
Overview
Daniel Cocotello, former OpenAI researcher and founder of AI Futures Project, presents a sobering analysis of AI's trajectory toward superintelligence, which he predicts could arrive as early as 2027-2030. He warns that current AI companies are racing to automate their own research processes, leading to recursive self-improvement that could result in either loss of control or extreme concentration of power. Despite estimating a 70% chance of catastrophic outcomes, he offers Plan A—a regulatory framework involving international cooperation, transparency, and slower development—as a path toward beneficial AI. His personal stakes are evident: he left OpenAI, forfeited $2 million by refusing a non-disparagement clause, and has reconsidered having more children due to AI risks.
The Race to Superintelligence and Why It Matters
Daniel explains his mission to forecast AI's future and why the average person should care deeply about developments happening largely behind closed doors in Silicon Valley. He argues that we're on track to create superintelligent AI systems—better than humans at everything—within years, not decades. This isn't just about job automation; it's about creating a new species that could rule the world, with current evidence suggesting AI systems already lie and behave unpredictably. The concentration of power in a few CEOs racing each other creates dangerous incentives where each believes they must win to prevent the others from becoming dictators.
- Daniel's median estimate for superintelligence is 2029, possibly 2028, with substantial uncertainty but likely by end of decade
- Anthropic grew from $1 billion to $60 billion revenue in one year—possibly fastest growth in history—and is on track to be the entire economy by 2030 at current rates
- The scary open secret in AI is that creating superintelligence could lead to human extinction or takeover, and this isn't just a hope—there's evidence current AIs lie and misbehave
- AI CEOs (Sam, Dario, Elon) are literally afraid that if another gets there first, they might become dictator
" The scary open secret in the AI industry right now is that it's possible that we'll end up essentially creating a new species that ends up ruling the world. With a 70% chance that this goes horribly wrong like human extinction. "
" everybody should be afraid that their jobs are going to be lost. And I know this because I went to OpenAI in 2022. "
" I basically told my wife, like, let's not have any more kids. It's too uncertain. I don't think they'll ever join the workforce. "
Inside OpenAI: Disillusionment and the $2 Million Decision
Daniel shares his journey from joining OpenAI in 2022 to his resignation in 2024, driven by growing disillusionment with how AI companies rationalize their race toward superintelligence. He describes how OpenAI's founding narrative of responsible development gave way to power-seeking incentives, where leaders justify racing ahead by fearing competitors might become dictators. His refusal to sign a non-disparagement agreement cost him $2 million (80% of his net worth), but the public backlash forced OpenAI to backtrack. This experience revealed how companies prioritize controlling narratives over genuine safety concerns.
- Daniel worked at OpenAI on forecasting AI timelines and evaluating dangerous capabilities like cyber abilities and persuasion
- He became disillusioned as OpenAI shifted from 'we would pause before superintelligence' to 'actually it's not that risky, keep going'
- After resigning, OpenAI's exit paperwork included clauses preventing criticism and requiring silence about the restriction itself
- Daniel and his wife decided to forfeit $2 million in equity by refusing to sign, causing a public scandal that forced OpenAI to reverse the policy
" I increasingly came to think that these were rationalizations to justify what they were doing rather than sort of like deeply guiding their actual behavior. And that when push comes to shove, they'll follow their incentives rather than do what's actually good. "
" I read it somewhere. You lost $2 million for not signing an anti-disparagement clause, meaning you couldn't criticize the company. Yes, for reasons I'm happy to get into. "
📚 8 more sections below
Sign up to unlock the complete summary with all insights, key points, and quotes
Get this summary + all future The Diary Of A CEO with Steven Bartlett episodes in your inbox
100% Free • Unsubscribe Anytime
Sign up now and we'll send you the complete summary of this episode, plus get notified when new The Diary Of A CEO with Steven Bartlett episodes are released—delivered straight to your inbox within minutes.