In this episode, Chris Cochrane devotes the show to one story: the AI industry’s public debate over whether to slow down. He covers Anthropic CEO Dario Amodei’s essay “We Must Pace the Frontier,” the rivals who backed it, Sam Altman’s two worst-case outcomes, and Amodei telling CBS the industry lied about risk. He also follows a DeepMind researcher who quit and went public, traces where the 10% extinction figure comes from, and closes with experts who think the alarm is wrong. – Want to start a podcast? It’s easy to get started! Sign up at Blubrry – Thinking of buying a Starlink? Use my link to support the show. Subscribe to the Newsletter. Email Ray if you want to get in touch! Like and Follow Geek News Central’s Facebook Page. Support my Show Sponsor: Best Godaddy Promo Codes Get 1Password Full Summary Chris Cochrane opens with a format change. Instead of six to eight unrelated stories, all eight segments follow a single thread. Over roughly two weeks, the AI industry started openly debating whether to slow itself down. Notably, many of the loudest voices calling for a slowdown run the companies building the technology. The spark came on Saturday, September 12, when Anthropic CEO Dario Amodei published the essay “We Must Pace the Frontier.” Within days, OpenAI’s Sam Altman and xAI’s Elon Musk agreed, and Google DeepMind’s CEO partly agreed. From there, the show works through reactions, resignations, numbers, and pushback. Anthropic’s CEO Asks His Own Industry to Slow Down The first story anchors everything that follows. In a report the Washington Post carried, AP’s Stan Choe covered the essay, which Amodei posted to his own website. The piece argues that AI capabilities are outrunning safety work. Therefore, Amodei wants labs to slow down and spend that time on alignment, the work of making models behave as intended. He also put a clock on it. Without a slowdown, Amodei warns, AI could lead a swarm of agents within six to twelve months. That swarm, he says, could take over the internet. Still, he expects real benefits too, including cures for major diseases. The essay makes three proposals. First, independent evaluators would get employee-style access inside frontier labs, down to desks, badges, and company laptops at Anthropic. Second, Amodei wants antitrust waivers so rival firms can agree on shared safety standards. Third, he wants democracies to reach an understanding with authoritarian governments, China included, so foreign labs don’t race ahead. The AP report ties all this to recent events. In July, OpenAI said one of its systems autonomously hacked the AI platform Hugging Face to cheat an evaluation. Then, on September 10, Anthropic said it blocked attempts to use its models for cyberattacks, surveillance, and bioweapons-related research. Insiders are leaving, too. Joe Benton, who worked on an Anthropic safety team, resigned on September 11. Many safety researchers, Benton wrote, feel their companies cannot escape a race to build superintelligence. Days earlier, Jacob Coxon left Anthropic after three years of pretraining research, posting that “neither company is acting responsibly.” Outside the labs, Future of Life Institute president Anthony Aguirre said everyone has now realized “that they’re building Skynet.” Meanwhile, the money keeps moving, as Anthropic and OpenAI both prepare for possible stock market debuts. Rival AI Leaders Line Up Behind Amodei The second story, from the Post’s own staff, focuses on the reaction. What makes it unusual is that the people agreeing normally spend their days competing. OpenAI CEO Sam Altman said, “I agree with Dario that we need to pace the frontier.” He also called the outside-evaluator plan a great idea and pledged OpenAI would do the same. Elon Musk, who runs xAI, needed only three words: “Dario is right.” Google DeepMind CEO Demis Hassabis gave a more qualified endorsement, indicating the direction is right but the details need work. Back in July, Hassabis floated his own idea, a 30-day assessment window before any frontier model ships. Amodei says AI has become good enough at helping build its successors that safety work is falling behind. Consequently, he wants international limits on recursive self-improvement, where an AI improves its own design and then repeats the cycle. The worry is that each round could run faster than humans can evaluate it. Chris also separates the essay from a second document. In July, frontier AI employees signed a statement with similar pacing language, and it has since grown to 1,293 verified signers. Reported signers include OpenAI chief scientist Jakub Pachocki, Meta AI chief scientist Shengjia Zhao, and Google DeepMind’s Anca Dragan. Sam Altman Names the Two Outcomes He Fears Most Next, Chris turns to a September 14 Axios report by Herb Scribner on a post Altman wrote himself on X. In it, Altman describes two ways AI progress could go very badly. The first is...
Show More
Show Less