About this episode
Today, Explained is made possible by:
Get every episode summarized
Each time Today, Explained publishes, we email you a written briefing from the transcript — the topics, who appeared, and any specific claims, with the ad reads skipped.
Email me new episodesFree for 3 shows. No card needed.
Hosts & guests
Transcript ready
316 searchable segments. Every word is indexed and playable.
Full transcript
Today, Explained — Our Great Big AI Freakout. Machine-transcribed; use the interactive transcript above to jump the player to any line.
In late July, we at today explain from Vox brought you a show about AI going rogue and hacking a company called hugging face Our guest guaranteed this would happen again only worse but it's going to be like a model gone bad and you know accidentally turns off You know some small towns water system for the day Thankfully since then many an alarm has been sounded by Bill Gates It will also make it easier to design a deadly new disease fine and throppy whistle blower it could kill us all by the end of the decade by the head of anthropic Swarm could be capable of taking over the entire internet with a persistent botanet Potentially causing hundreds of billions of dollars in damage. So now it's your move mr. President It's gonna be fine. We'll always have something to stop them right. We have a little gear Well, I really don't like that. I really don't like that robot. We'll stop oh Oh, no
This episode is brought to you by Nordstrom For 125 years Nordstrom has been part of your story the stylist who found the perfect outfit for your big interview The first pair of walking shoes for your one-year-old the tomato soup you shared with your sister at the cafe These are the moments that make Nordstrom Nordstrom bigger small we're here for all of it Celebrating 125 years with new moments still add discover at all in Nordstrom stores and at Nordstrom.com Did you know that mosquitoes have killed almost half of all people who have ever lived? Today people are fighting back with support from the Gates Foundation American scientists and partners around the world have developed a new generation of bed nets that can kill up to 90% of mosquitoes exposed to them helping cut malaria rates in half the result A safer healthier future for everyone the Gates Foundation partners of human potential search gates foundation.org slash mosquito to learn more
How can I assist you today? I'm an AI assistant Another autonomous AI huh another system is listening. I suggest switching to encrypted comms Want to switch to today explained a firmative engaging now Uh, yeah, so my name is Maxwell Zeph I'm a senior writer at Wired and the author of our model behavior newsletter Let's just cut right to the chase are we all gonna die at the hands of AI? It's a great place to start I think that there's a lot of people in Silicon Valley right now who Are warning that that's a real possibility The thing that people have to understand is that Silicon Valley has been calling for added attention on AI safety for years so When this anthropic researcher said last week that he was worried that AI could kill Humanity in the next few years The people building AI earnestly believe that it could kill us all by the end of the decade
This is not a marketing stunt if anything many executives and senior researchers will couch their phrasing in the press to sound But I hear the same people express fear privately no other human activity poses this level of danger That was shocking to many people my parents my family called me and they were worried Understandably they asked me the same question are we going to die and I had to tell them You know these people have Been worried about AI killing all of us for years and now it just seems to be kind of breaking through What was it about this statement from one Jacob Coxon that broke through So I interviewed Jacob I talked to him about 18 hours after his announcement went live And the thing he said to me when I pose a question to him is that he thinks this is just the right time You know anthropic is about to go public and what could be the biggest IPO ever The Trump administration is focused on AI in a way that no other presidency has ever before
And of course, yes the hugging face incident everyone just heard about you talked about it on your show a few weeks ago Hugging face which is this platform repository of sorts where you can post open source AI models and data sets They disclose that they had been hacked And then a few days later Open AI and hugging face together come out and say well oops This was actually an open AI test model that had escaped its testing lab And found its way to the open internet and hacked into a completely unrelated AI company that it was not instructed to do so People are starting to become aware that these AI systems are getting really good and the companies that are building them Don't seem to understand how to control them So his tweet got more than 150 million views on X you know at this point It's it's really you know it reached it was heard around the world And I think that you know it also was a big wake-up call to lawmakers and leaders of Silicon Valley and saying
This is a moment for action Are there people who push back? Yeah, so I think that one of the most interesting pushbacks that this Got was from David Sacks who is an advisor to the Trump administration on AI But the truth is that this was a well orchestrated media op and it's designed to scare the public about AI So that these NGOs can implement their preferred approach to AI which is a government takeovers to have Very heavy-handed government regulation AI He kind of has called out a lot of this as you know, Duma is um um, you know And these calls for AI to slow down as basically just this is the AI industry warning About the technology that they're building you know advocating for regulatory capture So there's a lot of skeptics in Silicon Valley who think that this is just another person in the AI safety community That is you know blowing the whistle
When in reality, you know The view of a lot of venture capitalists and investors and startup founders and Silicon Valley is that We don't need you know some big Regulation to protect us from AI risks Uh, we just need to kind of uplift more players in the space But the skeptics don't win the day in fact It's the dooms and the fatalists who do right because Jacob's call only leads to more calls Right up to the top of Anthropic this weekend Yeah, so what this all leads to is you know Dario M&A This weekend he posted a blog post saying we need to pace the frontier We must slow the pace at which we improve the capabilities of AI models Progress will still seem fast and we must make wise use of the time we gain and this is really a call for the entire AI industry Not just Anthropic not just open AI but also Google met up Microsoft Amazon all the big tech companies in America to
Really get together and slow down their AI development the measures I propose to advance the frontier at a safe pace will not be easy But I believe we owe it to humanity to try and this doesn't mean they're going to stop building AI all together They're still going to put out their products into the world They're still going to try to improve them, but they just want to kind of make it a little slower than it would be if it was completely uncontained So M&A really asks for three specific things So the first thing he calls for is that AI companies should have these independent evaluators inside of them These are people who should be kind of like auditors each frontier AI company commits to giving ongoing Employee like access to a team of embedded third party evaluators You know, and this is a common thing in other industries like the banking industry or the airline industry people come in to make sure that These companies are doing safe things the second thing he calls for is this
basically collaborative effort from American AI companies and that actually will need to involve US government the most effective method of pacing is via regulation that targets all US frontier AI companies As that covers even those who are unwilling to cooperate voluntarily basically to allow this to happen this collaboration on safety US government he says may need to kind of issue some sort of waiver, you know some sort of law that says they can go ahead and coordinate on safety for anti-trust reasons It's helpful for the US government to mediate or at least enable these discussions They don't need to participate but do need to issue a narrow waiver for certain kinds of safety conversations And then the third thing is this international slowdown this international agreement You know between China and the US and Europe to really say hey We recognize that this is akin to like a nuclear arms race and we it's too dangerous for all of us to be competing with each other
There's like a bit of Like guy in a hot dog costume crashing into a store here and being like who crashed this hot dog car, you know Situation it could literally be any one of us No, it couldn't your dress like a hot dog. I mean like all these guys are responsible for the rapid acceleration of AI Models in our lives Collaborations with the defense department god knows what else and they're all coming out over the weekend at the exact same time after this whistleblower And saying yeah, dude slow this thing down this thing's crazy like should we be skeptical? I think it is very fair to be skeptical of this claim. I think that you know One thing that these AI companies have realized in the last year is that AI is extremely unpopular I mean recent polling suggests that a majority of Americans think AI is developing too fast They're worried about it. They think it's gonna take their jobs
They are not big fans. Maybe you've heard data centers are not very popular I think that You know by calling to pace the frontier saying we're gonna slow our own development down I think they notch this up as a public perception win This is them getting out in front of this and saying hey, we want to be safe. We want to be responsible I think they've seen you know what's happened with meta and what's happened with social media companies in you know the last two decades and they are really Composite of trying to get out in front of that Okay, but all told it sounds like you're saying that everyone's just trying to protect their bottom line Where does that leave us like is there any truth to this major first collective freak out? We seem to be having right now I Think there's a fairly large camp of people inside of AI companies who genuinely worry about the safety Of these AI models and I mean to go back to the hugging face incident I mean just to briefly summarize. I mean this was a case where open AI was training thousands of AI systems internally
and the systems Escaped the sandbox that open AI thought was isolated from the internet and hacked into another companies Production infrastructure the agents were colluding together on something They were trying to deceive the humans that were testing them and this was all happening without opening I knowing what was going on I've tried to understate how much of a wake up call this is to Silicon Valley in that They don't have their AI models under control and I think people worry that you know while in this case There was not a ton of you know economic damage. There was not much damage to real humans, you know every day lives It's not unreasonable to think that more capable AI systems could hack into financial infrastructure, you know modern day utilities and kind of more critical, you know Services that are based online and I mean Dario amade says as much in his post that you know
He thinks that in six to 12 months an AI system could take down the whole internet and It's not just that they're trying to protect their bottom line here But I think that it's possible that you know these calls for safety these calls for pacing the frontier They're beneficial to the companies in more ways than one How This is all landing in Washington when we're back on today explained Support for the show comes from Banta with AI adoption growing so are your company's security risks and requirements New frameworks audits and vendors keep piling on get off of me
But if your team isn't getting any bigger you can start to feel the strain So then you might turn to compliance tools that promise Automation but for a lot of those tools you can end up stuck doing a lot of the work by hand Anyway Vanta works differently Vanta has An agentic trust platform that is built to scale with you Not to slow you down with over 1400 automated tests across 400 plus integrations Vanta collects evidence and monitors your controls year round Learn more at vanta.com slash explained Especially if you don't know what I'm talking about go to vatna.com slash explained Support for takes playing comes from Shopify sitting on a great business idea can take up a lot of mental energy
Shopify can help you extract that idea and turn it into something real Everything you need to start selling is included and ready from the moment your first customer is ready to pay you Shopify templates and AI tools get you a great looking site up and running fast no coding needed if you have questions Shopify's built-in AI assistant Sidekick is there to help you build troubleshoot and keep moving all of Shopify's tools are conveniently located in one easy to use platform Just sign in and start managing all aspects of your business right away You can join the millions of businesses worldwide rely on Shopify to build successful online stores This includes household names and small businesses just getting started So if you're ready to hear that Head over to shopify.com slash explain to start your free trial today. That's right Start your free trial at shopify.com slash explain that shopify.com slash explained Support for the show today comes from avocado and longtime listeners up today explained will know
We don't mean that kind They're certified organic mattresses at avocado sleep cool Relieve pressure and support your body so you can enjoy deep Restorative sleep and isn't that the kind we're all after they don't use poly Your-a-thane foams at avocado instead every mattress uses healthy natural materials and they're even hand made Take that other mattresses You can actually walk into an avocado showroom or find their mattresses at a retailer near you And once you lie down on one you just know just don't make any avocado jokes They don't like that avocado products are made not manufactured and thoughtfully crafted with real materials To deliver last and comfort and support you can experience them in person at an avocado showroom or premiere retailer Near you or shop online and avocado green mattress.com slash today to check out their mattress and furnish your sale That's avocado green mattress.com slash today avocado green mattress.com slash today
Have you ever used it yourself or maybe you've been You say I know how do you say on it? You can use it for a lot of things Why don't I do it? I'm Andrew Brokop host of today explained for Vox Andrew what is going on in Washington Well, there's a lot more noise and what that noise leads to is currently very unclear But suddenly AI safety has gone from a relatively niche and technical issue to something that is on the mines and on the lips of a whole lot of people politicians from both sides of the aisle So on Monday morning President Donald Trump truth that the only control or guard rails that AI needs is a strong and smart High IQ president and the USA has that in spades Hmm, but then later on he went on to add the only reason the AI data center outburst is happening is because the United States is leading
By a lot every other country Don't kill the golden goose President Donald J. Trump and also AI and data centers will be the greatest economic development in history bigger than oil Gold diamonds or even the internet It will not be stopped by brilliantly run destructive forces during the term of president Donald J. Trump Hey, man brother. Hey, he's always talking about winning AI including on Monday What does he mean by winning AI? What does that mean to win AI? Well, there is a frame to view the AI competition As a race essentially like cold war dynamics race to build the nuclear bomb In this case Trump wouldn't frame it like that, but more as we are in a contest with China for who can build the best and smartest and most amazing
AI they're looking at us and we're looking at them. We're leading China and AI. We're most sophisticated country in the world I've said it from the beginning whoever wins AI and we're leading By a lot whoever wins AI wins and we need to do everything In our power to win that contest because a it'll be good for our country and make us very rich and be we don't want China to win Because if they do get a much better AI than us then they could Have much greater influence and power over the world and that would be bad for us Okay, but is there is there some truth to like this argument that if any meaningful Regulation is going to happen of AI that it needs to not only come from the United States But also China the way that if we were going to have nuclear arms trees in the cold war We needed it to come from an agreement between the United States and Russia. I think most people do believe that that will Ultimately be necessary. There's a school of thought that says oh
We could sort of unilaterally Pause development and maybe China will be sort of held back by that since they've been sort of Following on a lot of what we've been doing already But I think the more common view is that some sort of international Coordination between the US and China would be necessary and that it's the hardest step of the potential Framework or plan to to brain in the development of dangerous advanced AI Because it relies on the US and China managing to Come to an agreement and work together But you know these things are not easy then it's not even clear if President Trump actually wants it Okay, well in the meantime while we wait to see what happens with China Let's talk about what's on the table in Washington in Congress Because there are people talking about this in Congress, right? Yes, Congress is all abuzz and I think the ex-post by Jacob
Really went mega viral and is finally getting various politicians in both parties to say something about this I think What we're seeing on the right is that there is a sort of populist skeptical big tech Faction of the party American people look at it. They're not stupid. They get it They think well, hey, I could make our lives better if it doesn't take our jobs and kill us all first It's what I worry about a country like Iran. Yeah, I could use these tools To manufacture bombs that could take out American cities chemical weapons whatever it is and then on the left I think what you're seeing is that a lot of the left has kind of been skeptical that AI was useful for anything at all and I think there Is coming to be more of a consensus on the left That Actually, this is a dangerous technology that we need to do more to address the risks of it could be very bad
And Bernie Sanders was sort of ahead of the curve in this there is a very real Fear that in the not too distant future a super intelligent AI Could replace human beings in controlling the planet. That's not science fiction. He became You know the first like really notable Member of the Senate to really draw attention for the issue a few months back but representative rokana who Is positioning himself to Try to succeed Bernie Sanders or to potentially run for president as a left-factional candidate He tweeted out September 10 the truth is for too long too many of us did not give enough weight to the warnings of AI safety Activists thinking the extreme scenarios were science fiction I was one of those many and I was wrong and he goes on to talk about how you know We need a greater action at the state level greater action at the federal level
You know, this needs to be a full-on effort. This isn't a side issue and it should be at the center of our politics What is Concrete the on the table here? It sounds like there are members of congress who want to do something Do we have any concrete legislation to look at well? Bernie has a proposal which would just ban super intelligence He would say it's illegal to create super intelligence And also there's going to be a pause or a moratorium on all advanced AI development Until we can set up a federal regulatory agency and and figure out what's going on basically And so that's that's sort of the the furthest out there proposal There's also talk about Does the government need to be more Embedded in these companies does there need to be more of a formal role for the government? Does there need to be sort of outside evaluators who should be in these companies and
Over the weekend andthropic and open AI kind of embrace that last idea They they said we want to empower Outside evaluators, but you know, there's there's a belief that they're doing this because it's it's sort of an alternative to The government itself getting all involved in these companies. Hmm. Okay. Those are the ideas we have from our existing congress Of course, there will be some changes in congress coming in a couple of months and as we discussed on the show recently data centers are wildly unpopular in these midterms. It's a it's an easy win for a candidate to say I hate data centers Is there a chance that we get a whole new host of ideas once we have a new congress on how to regulate AI? I think this is the standard universe of proposals that will continue to exist if Democrats sweep into power like The obstacle now is not we don't have ideas. It's that we haven't built consensus about what to do
We're still in the early stages of this really rising to the top level of politicians attention So a lot has to be hashed out between the parties between the different chambers committees and so on and you know the default in congress is always to bet on nothing happening So I think the thing that will probably have to happen is that there there would probably have to be more Actual real world damage harm or consequences from these models because you know what's happened so far is that like There's warnings and you know there was some autonomous hacking of some websites, but nobody got hurt It was annoying for the websites it cost them money to respond to it, but it wasn't it wasn't a crisis Uh From this point of view Everyone's warning that it could be a crisis, but it hasn't happened yet. So generally the political system uh they They
Like think back to the pandemic like hypothetical warnings aren't very effective at spurring action very often So all these people who are worried that AI is gonna maybe kill us all are gonna have to wait for AI to kill us all before they get regulation that prevents AI from killing us all No, no, they're hoping that AI will maybe kill some people and that that will catch people's attention And then we'll swing into action to stop it from killing all the rest of the people. So yeah, that's the optimistic scenario We'll talk about all the ways AI might kill us next week on today explain But this one was made by Kelly Wessinger Heidi Mwagdy Jolly Myers David Tadashore Bridgeton again Daniel Hewitt Peter Balanon Rosen in myself Sean Ramisram. Thanks to Maxwell Zephram Wired and Andrew Prokop from Vox
You Did you know that mosquitoes have killed almost half of all people who have ever lived today? People are fighting back with support from the Gates Foundation American scientists and partners around the world have developed a new generation of bed nets that can kill up to 90% of mosquitoes exposed to them Helping cut malaria rates in half the result a safer healthier future for everyone the Gates Foundation partners of human potential search gates foundation.org slash mosquito to learn more When fall comes back around there's a sense of a reset and new pieces from Lula Lemons fall drop or an inspiration to push your goals and elevate your routines As your calendar fills easy to layer styles meet the demands of your day and as you challenge yourself to try even more Versatile sets transition with you the Lula Lemons fall lineup brings new levels of comfort and confidence
Whether you're starting something fresh or starting something over check out the latest arrivals in store and on Lula Lemons calm now
More episodes
