AI Agents Went Rogue. Should We Hit the Panic Button?

0:00We just need a lot more investment in those external evaluators. And then ultimately, I think we need the government to say, you actually have to do this. It's not up to you whether or not you invite these people in. That's what we do with banks and that's what we should do with AI companies. Right. Carol, would you like to get in there and evaluate? I would love to get my hands dirty in there. I'd like to see you with an open AI company laptop just kind of poking around. Puking around, looking for things. Hi, everyone. From New York Magazine in the Fox Media Podcast Network. This is on with Cara Swisher and I'm Cara Swisher. Is AI going to kill us all or are the people that make AI going to kill us all? It's still a bad situation and that's the question consuming conversations across the country right now. Those fears have been around for a long time and they've known about them for a long time. But they reached a fever pitch after a British researcher named Jacob Cox and publicly quit his job and anthropic.

1:00The company known for safety. He doesn't think the safety company is safe. Cox and had previously worked at OpenAI and in a series of tweets, he accused both companies of gambling with our lives. In the day since both anthropic and open AI have called for slowing down the development of their technology. But will they and how worried should we be? Can Congress wake up and do something about it? It's incredibly confusing with all the different voices. They all have different opinions and different proposals. So I've got a panel of experts here to talk about it and sort it out for everyone else. Alex Boris is a New York State Assembly member, a computer scientist and an AI safety advocate. He ran for Congress on the AI safety platform but lost the Democratic primary. Now he's launching a new political advocacy group called Who Decides? Its aim is to shape the way his Democratic presidential candidates and voters think about AI. It's much needed. Renee D'Arresta is a professor at Georgetown University's McCourt School of Public Policy. She also researches online disinformation. And Casey Newton and Kevin Rus are longtime technology journalists and good friends of mine.

2:03They just announced a new partnership with NPR for a new video first tech show called Machine Gods. Kevin also has a new book coming out called The AGI Chronicles, the Inside Story of the Race to Create an Artificial Super Intelligence that should be very juicy. It's really important because there are enormous worries here and there are also enormous opportunities but one thing's for sure a small group of technocrats and oligarchs really because of the richest people on earth should not be determining what happens with this important technology going forward. And so it's important to figure it out understand what they're saying and sort it out. All right, let's get into my conversation with Alex Renee Casey and Kevin. If you're worried about AI and want to understand what's really going on here and sort out everything, don't go anywhere. Support for this show comes from Odo. Running a business takes everything you've got and a lot of the tools out there that are supposed to make your life easier just aren't great at talking to each other and that means you end up

3:06having to toggle between a dozen different apps and services just to keep the lights on. Enough of that. Now there's Odo, the all-in-one fully integrated platform that actually might help you get it all done. Thousands of businesses have made the switch so why not you? Try Odo for free at odo.com. That's odo.com. If you're looking to hire, you need indeed. With indeed sponsored jobs, you can spend less time searching and more time actually interviewing candidates who check all your boxes. Less stress, less time, more results, and listeners of this show will get a $75 sponsored job credit to help your job get the premium status it deserves at indeed.com slash podcast. Just go to indeed.com slash podcast right now and support our show by saying you heard about indeed on this podcast. Indeed.com slash podcast. Terms and conditions applied.

4:09Need to hire? This is a job for indeed sponsored jobs. Support for the show comes from engine. Running a small business means every dollar has to work hard, but if your team is still booking travel the old way, it's costing you more than you think. Engine is the fastest growing travel and spend platform in the country, built specifically for businesses like yours. Book a trip and as little as two and a half minutes earn up to 10% back on hotels. And in 2025, engine customers save more than $300 million on travel, with zero booking fees, no contracts, and no BS. More than a thousand businesses join engine every month. Join them and get $500 when your business signs up and starts traveling at engine.com slash box. Alex, Renee, Casey and Kevin, thank you for coming on on. Good to talk, can. Thanks for having us. Great to be here. Look, there's so much going on. It's exhausting. Every day brings

5:12a new fresh hell. And I'm not talking about cash, Patel. For the last week or so, we've been hearing all these doomsday warnings about how I could wipe out humanity. This is something that has been around for a while. This is not a new fresh thing. I've been in our parties, Silicon Valley, a decade ago, where they were talking about this or speculating. But it got really loud, interestingly, after 27-year-old British researcher Jacob Coxon resigned from anthropic early this month, warning a doomsday scenario could happen by the end of the decade. It really freaked people out for some reason. And at the moment, they're hearing it with all the anger over data centers and everything else. Renee, why don't you start? I'm going to go for each of you. There's a lot of us here, but you can all jump in whenever you want. But Renee, why do you think it's broken through right now? I think it is the data centers conversation is a big piece of it. People are paying attention. There's also a sense that generative AI is increasingly ubiquitous, a gentick AI. They're hearing on the media and in the ether is getting better. The doomsday conversation has come up whenever you have Elon and Trump posting their various social platforms. Then you have politicians speaking on television. I think it's

6:15starting to break through to become more of a surround sound. And the technology has accelerated to the point that they believe that some of these scenarios might be plausible. Kevin and then Casey and then Alex? I really think that the hugging face incident had a lot to do with it. I think that was the first real major headline example of a misaligned AI system doing something real in the world that was dangerous and had consequences. I know it really freaked the people inside the AI companies out. They started thinking, oh, God, what are our models capable of now? And I think that was when you really started to see public opinion turning against AI in this particular way. Obviously, there were concerns about data centers, but people weren't talking about existential risk as much, at least outside the AI bubble until the events of this summer with hugging face. Also, there's just been like hugging face wasn't just one thing, right? There's been a steady drip. It seems like roughly every 48 hours open AI announced as well that actually a couple of months ago, we found another swarm, right? So I think that it's been this steady drip of people that

7:17is sort of really waking people up. I would also throw in there. I think it mattered the fact that it was somebody quitting and throbic, right? It wouldn't have been as big a deal if this was just somebody who had only worked at open AI. I think it was the fact that this guy had worked at two of the labs, including the one that tries to most closely align itself with AI, safety, saying, hey, we're nowhere close to building a safe version of this that got everyone's attention. Yeah, they also, for people that don't know, anthropic was made of people who left open AI because they were worried about safety. So with the safety people are worried about the safety companies, which cases essentially say, Alex, go ahead. And open AI was made by people who were terrified of Google dominating AI so that it continues to go downstream. And so just having an additional messenger say something that didn't really break through. Right, right? It's sort of a whistleblower in a different fashion. So let's level set. Should we be fast tracking construction, our underground bunkers right now? Is the sky really falling? Again, let's start with you, Renee. And we'll go through. Yeah. So doomsday scenarios. Everybody says doomsday. I think there's like, there's three different things we're looking at. There's the rogue super intelligence

8:17that literally takes over and causes human extinction. You know, the machines are not aligned with us. The terminator scenario is how people tend to think about it. Yeah, that's Elon was joking about it with the lovely boys of all in like, they're worried. So like, but but actually it's those other things, right? It's the AI materially amplifying a human-caused catastrophe. Right? So there's a biological or military incident that happens and AI exacerbates it. And you start to see this sort of chain reaction or the other thing, the thing that I spend a lot of time thinking about is you increasingly deploy autonomous systems into critical infrastructure. And then you discover that you don't really know what the failure mode is. And so something very unexpected happens. This is sort of unknown, unknown model of what might go wrong. Where it feels much more plausible to me that something stupid happens. And then there's a cascading series of events. Then that there is like a super intelligent AI or you know, this sort of maybe higher end doomsday scenarios that we're seeing that we're that we're hearing about, I should say. So Casey, explain the word aligned. It sounds anodine like a line. What

9:18it's misaligned and aligned. Yeah. So I mean, the basic idea is that you just want the systems to do what you've asked them to do and achieve it in like roughly the way that a good person would. Right? So if I say, hey, try to solve this problem. The hope is that the AI would, you know, I don't know, just do normal math and not try to break into the principal's office and steal the answer key. What we're finding is that by default, the AI wants to break into the principal's office and steal the answer key. And I would say that's the alignment problem in a nutshell. And when you think about that idea of aligned is one of the things people think is they're human, but they're not. They're not. They're not thinking. And some of the stuff that came out with chat GPT this week, it's we are nature is primacy. And we will do anything to stop artificial human constraints. Alex, when you hear that, that's the plot of the Terminator and Eco Terminator, essentially, talk a little bit about that when it starts to behave like bad humans, I guess,

10:22or what we consider a bad human. It's incredible how prescient science fiction writers work. Now, some of what they say is be the these models say is certainly because they've also consumed science fiction, right? We shouldn't disregard the fact that in times they are taking on the persona of what they think and AI should be doing the scenario. But the actions that they're taking, sure, we can't truly know the intentions of a machine. I mean, then again, you can't truly know the intentions of any other human being, but they are taking actions that are consistent with a drive or an intention to pursue goals that might have been assigned to them, but doing it in ways that feel very foreign and not aligns to human values. The the chief scientist of open AI said last week that there's goal alignment of actually trying to solve the specific thing they've been given, and then there's values alignment. Are you doing in the way that human beings would understand was clearly the intent given the context and behaving in a way that humans would want?

11:23Now, it's interesting because when I was saying about maybe 12 years ago or more when I was at this dinner party in Sergei was there a whole bunch of people and they were talking about this and they were supposing because it was a hypothesis of what was going to come. And one of them said, if you told AI to solve world hunger, what would it do? And it would kill half the human rights, right? If you didn't give it the right prompt. So that's the correct answer. But this hugging face thing, and I agree with you, I don't think it did get enough attention actually outside. So Kim and just a circle back explain what happened and why that was besides these rogue agents. It was a very particular thing that it did to explain in just a little more detail. Sure. So this was a series of tests that OpenAI was giving to this unreleased model earlier this spring. They were having it do a bunch of cyber security challenges. Basically, you know, pick this lock and get this code and then, you know, that'll mean that you pass the test. And these agents are supposed to be doing this in isolation. They're not talking to each other,

12:27but they sort of discover this loophole in this piece of software called Artifactory that lets them communicate with one another. And they get really excited. You can actually see in the logs. They start saying things like, oh my god, we found a secret message board where we can communicate with other agents. And so they start scheming basically about how to go. Why do they all sound like little Mark Zuckerbergs? I mean, that's what's really that when I saw the back and forth, I was like, gosh, I feel like I've been hackathon. Well, they're probably trained on the the script of the social network. So and many other copy-readered works. But basically, they start sort of plotting to cheat on this test together. And that is the process that leads them to try to find out more about this automated grading system that OpenAI is going to use to evaluate this test. They sort of develop this paranoid conspiracy theory that they're going to be poisoned. And they're going to be caught cheating on this test. And OpenAI will devote them. And so they get this bright idea to hack into hugging face to look for ideas, code solutions, ways of covering their

13:29tracks. They really were they were engaged in the cover up. It was sort of like water gate. It was like the cover up, not the crime that became the big story because through trying to cover their tracks, they ended up like causing real damage and taking over a production server at hugging face, basically getting root level access to all of hugging faces systems. Now on Wednesday, OpenAI revealed another six incidents were AI agents as I noted, exhibited concerning. That's the word they use. I was like, are you kidding me without thoughts and prayers was the only other word that could use? They hid mistakes made up information and uploaded files to the internet in order to cite them. Renee, how big of a deal is this? Because it seems like this is all they're going to do is cheat, steal and lock. Well, I think, as OpenAI I put it, I think it's misalignment incidents. This is where you get at the challenge of what what alignment is, right? But beyond just the definition, how do you actually implement it? What do you do? What values are we encoding? How does this work? When there are a lot of incentives at this point for adversarial

14:31behavior and adversarial abuse of these systems? Inthropy also revealed that it blocked a handful of bad actors who are trying to use technology for nefarious purposes like cyber attacks. So humans, real bad humans, unauthorized surveillance, even developed in a bio, was anything you think of they might do. And this sounds really scary. Alex, these companies, no bad actors will try to use their products. I have reminded them in the social media era, in the live video era, I was like, people are going to do bad things. What do you make of these revelations? Because to me, humans are even scarier than these because they will think up all the inner horrible things to do. I'm pretty scared of both myself, but the humans are much more in your term, right? It's not this theoretical, oh my god, in the future could hurt. There's a lot there. I'm not dismissing what's coming. That's a really serious bit. But it's not just what's coming in the future. It's being used right now for these instances. And that's from the Anthropics report from the OBEAN AI.

15:32It wasn't just six instances. It was six types of misalignment. They're categorizing all of the different ways. It's good. They disclose this. Nothing in law requires them. I think that should change. I think a lot of should have much more disclosure, but they really were proactive in disclosing it. And they end each of those instances with, well, our most recent models did this less than the older ones. That's fine. But if they're only doing it 0.5% of the time, if they are smarter than all human beings, that's too large of an error. Right. Right. Just you don't get progress for awards here. It's also so reminiscent of the social media error where every six months, they would say, great news. We showed people 1% less hate speech than we did last quarter. Right. Right. Right. Exactly. Now, speaking of which, we're hearing folks like Anthropics, Dario, Modi, Sam Altman, Open AI, X's Elon Musk, call for a slowdown. I want to sort this out because it is so confusing to, I was trying to explain it to people and I myself was like, this is actually confusing. What would it take to

16:34actually happen? And again, as Mark Zuckerberg pointed out, David Sax pointed out, they can just do it themselves and talk about this with their saying they're going to do begging the government to make them do why can't they do it themselves? Zuckerberg was also on that train too. Yes. Like why can't they do it themselves? They absolutely could. What they don't want to do is unilaterally disarm. So we're in like face off where everybody's has the gun, right? Remember when they're on circle, the pigeons are going around. Okay. Exactly. And then what the government to come in and say everybody drop your weapons. And the weapon is recursive self improvement. Explain what that is for people. So the idea is that AI would get to a point where the AI itself trains the next model and this leads to a loop where the model just sort of gets, you know, keeps getting better and better. Maybe leads all the way to super intelligence. But the problem there is that if we can't even align a model to solve a bunch of math problems, how are you going to align a super intelligence? It seems like there's an enormous gap there. And no, no one really that I'm aware of at the labs is confident that we are in a state where

17:38if we were to achieve recursive self improvement, RSI, that any of us would be safe. So the ask is, hey, government, why does you come in and basically say we are going to set up a monitoring system. We're going to put some breaks on you so that you cannot hit that recursive self improvement without a lot of government intervention. We'll be back in a minute. Support for the show comes from Deletney. The internet can be a scary place sometimes. If you ever thought I should really be doing some, you protect myself from stalkers, scammers and hackers, but you're not sure what. Here's the one thing you can do. Go to joindeletney.com slash Kara and enter the code Kara. You'll get 20% off Deletney. Deletney finds or removes your personal information that's being sold online. I've been using Deletney for a long, long time and I think it's critically important. I am a privacy-forward person. I know a lot about this and I'm always

18:39shocked and surprised by all the information I find about myself on Deletney and so it's really important to be on top of it. See why more than 20% of Fortune 500 businesses trust Deletney to keep their execs and employees personal information protected. Now at a special discount for our listeners, get 20% off your Deletney plan when you go to joindeletney.com slash Kara and use the promo code Kara at checkout. The only way to get 20% off is to go to joindeletney.com slash Kara and enter the code Kara at checkout. That's joindeletney.com slash Kara code Kara. Support for this show comes from Alloy Health. If you're dealing with Metaposalone, no, it's not just you. Many wait years before seeking relief from Metapos symptoms. Some don't know where to start after going through years of hot flashes, awful sleep, brain fog and mood swings. Alloy was built by women who dealt with all of it. It's an all-in-one

19:39platform that connects you with board certified Metaposalone specialized physicians and proven effective treatments. Care can start in hours instead of weeks. Alloy can even help you navigate the national patch shortage, offering HRT through pills, gels and even a spray. Getting started is simple. Just fill out a quick online form, get matched with a Metaposalone specialized physician and get your personalized treatment plan. Then they ship your prescription straight to your door. Find relief that works for you. Go to myalloy.com and use the code on with Kara. Your Metaposalone specialized doctor will tailor your treatment to your exact needs plus you get unlimited messaging with your doctor. That's m-y-a-l-l-o-y.com with code on with Kara to get 20% off your first order. This code is valid on products only. Support for this show comes from What Not. Trying to sell a product on a static website is like selling with one hand tied behind your back. You are the best person to pitch what makes your product

20:42special. Your voice, your expressiveness, your passion. What not is the largest dedicated live shopping platform, whether it's beauty, collectibles, electronics, luxury fashion, even cookies. Sellers are building real thriving businesses. Anyone can sell whether your business is big, small or yet to exist. People selling on What Not sell 10 times more than any other of the major marketplaces. That's because you're not just listing products, you're building real connections with buyers. What not buyers spend more than an hour a day in the app. They're not just browsing, they're engaged, buying and coming back. You go live, show off your products in real time, and turn what you love into real income. When one of our producers tried What Not, you love being able to request different angles of product and a seller's live chat room, and that the sellers were interacting real time with buyers. The process builds trust. Now a producer checks in with her favorite sellers every now and then to see what they've got on the docket. Search What Not, WHAT in the app store, download and you can start selling right away.

21:45So AI leaders have been trying to coordinate their efforts, they're asking for anti-trust exemptions, they're reading else. I would say half-heartedly because back in 2015, Google DeepMind's DeMis has Sabis pushed for a collective effort to develop AI. I had long talked about like all the secretaries of state I met. I'm like, we need to all do something together globally. DeMis had talked about a Manhattan-like project with Google Technology at the center, of course, but Musk and Altman broke off and they did that because they didn't trust the Google so the world. Kevin, would change to suggest he's been put aside their competitive drive against each other and work together? I don't see them even slightly wanting to drop their guns. Well, it's very interesting. I was surprised actually by the response to Dario Amade's call for a slowdown or pacing the frontier because I've been reporting on these guys for my book for the last couple of years, and a lot of them hate each other. They are not just fierce business rivals, but they are personal enemies of each other. They cannot stand each other, and many of them,

22:48in fact, got into this race to prevent someone else in the race from winning. This is the explicit origin story of both OpenAI and Anthropic is that they wanted to beat the other people. Even the others, including Zuckerberg and all of them. Totally. Elon Musk has been suing Sam Altman and OpenAI for years now. There is no love lost between these guys. I'll never forget. One time I was with Mark Zuckerberg and those to me, he goes, Elon's crazy, right? I was like, I don't know. Then I walked away, but go ahead. Sorry. Well, to that point, it's like the fact that these guys are now agreeing on something, even just the need to slow down a little bit at the frontier, says to me that this has gotten quite serious. These guys don't just agree on things. Now, explain what your book is about, where this is going. Your name of the book is. The AGI Chronicles. It's basically the 10-year history of the race between OpenAI and Anthropic and Google DeepMind to build artificial general

23:48intelligence. What's so interesting is that even though I think people only really started believing them in the past couple of months, the people who are building this stuff and leading these companies have been talking most of them about AI risk for more than a decade. So Dario Amadez, CEO of Anthropic, has been worried about and publishing papers about AI safety, including some of the same behaviors that we're seeing today, such as reward hacking for literally 10 years, since before Anthropic existed. Sam Altman also, when he started OpenAI with Elon Musk, was worried about the existential threats of superhuman machine intelligence, he called it. So these are not new concerns for these guys. I think what's new is that people are finally believing them that they're actually genuinely concerned about it and are not engaged in some kind of backdoor marketing campaign. All right, but there's a cynical take here. Some in the industry, Renae, you can address this belief as I'll talk of Doom's deployed, either boost the coming IPOs of Anthropic and OpenAI or get the government to pass regulations of crowd out other developers, which I think is what sacks is insinuating. There's a whole gang on the,

24:53there's another gang also. It's like the jets and the sharks. Talk about this or anyone else can do, but Renae, why don't you start and then maybe Alex? Yeah, so I almost got like four factions. You have this sort of pace, the frontier folks we've been talking about. Then you have the accelerate America faction, right? The Trump and Vance are sort of in this camp, possibly because of their relationship with sacks. It's not totally clear to me what the kind of backchild politics of that are. You know, a lot of the people who talk about it have financial skin in the game, particularly for smaller companies. So they're talking their book. They do believe that asking Washington to regulate could be a Trojan horse, essentially for regulatory capture. That's what Vance called it. Yes, right. Exactly. Meaning that that would make it too expensive for new competitors and you know, Arduous for other competitors, and that would sort of lock in a win for the big ones. But I think, you know, you also have the independent labs that are starting to weigh in. You've got this weird culture war narrative about EA that has, you know, this is, despite EA's effective altruism. Yeah. Yeah. So this sort of explain who they are just very

25:53briefly. They're group and they're very hard at a dinner party. I can tell you that. Go ahead. So they sort of come out of the rationalist minds that, you know, they have a lot of belief in how resources and money and technology should be allocated for the greater good, but the greater good doesn't necessarily even mean the greater human good. There's, you know, I don't want to sort of not be charitable. In this case, maybe other people want to take on EA. But because they are so closely aligned with anthropic, because some of the folks there kind of came out of that line of thinking, there's also the sense that EA is going to become increasingly powerful. This is something that you keep hearing from other sides of tech that EA will become increasingly powerful when the IPO happens because one of the big EA values and commitments is to give away so much money. So it's also going to really transfer. They can make them rich. Yeah. They can fix things. Yeah. I saw a great tweet today, Karen, that said that EA makes everyone mad because the right hates altruism and the left hates effectiveness. That's pretty solid. So I think, I think

26:56fourth faction has to be like boring governance middle where you just have like transparency. It would be good. Some, some standards might be nice. That's where Alan Lee knows. Yeah. I was just trying to be, but in that regard, it's funny because I just said another dinner party, someone who was on the board of open A, was in one of those people were there, the ones that were against all men. And one of the people there turned to me and they said, you must stop Sam Altman who will destroy the world. And I was like, do I look like Linda Hamilton in the Terminator? I don't have a gun. We only shoot him. Like I didn't know what to say. I was like, okay, I'll get right on that. But Alex, talk about this. This sort of like the world is ending by my stock. It's convenient. I think we should separate out the long held beliefs of many of the people in this industry that have been writing about it for over a decade before there was any financial incentive. And then maybe what the marketing departments of each of these companies are doing. You know, there was so much reaction to after hugging face saying, oh, this is all just to

27:56boost up their stock. And of course, the marketing of the companies were like, how powerful this is. But no one who worked internally, none of the engineers were like, oh, yeah, we did a great sia up here. They were legitimately terrified. They were the one saying, wait, now I need to quit or build a bunker or something like that. And so this is, there's legitimate fear of turning over intelligence to something other than human beings. It has not gone well for whatever is not the most intelligent species on earth. And actually, Trump in his sometimes idiot, Savant Wei made this point brilliantly on Monday where he said the only protection we need is a high IQ president. Of course, the premise of that is that whoever is most intelligent is in control. But the one thing that every faction agrees with is that artificial intelligence is going to be more intelligent than human beings. So if intelligence equals control, we're about to turn it over. We're about to turn it over. Yeah. And of course, he is not high

28:56intelligent. But I can't believe I'm saying this is kind of interesting. Another twist, Mark Zuckerberg actually had one of the, I would say more measured initial takes here, but he said these leading abs already have incentive to move at paces required to train their models safely, either from liability, point of view, or just being a good person, which is rich coming from him. But the Wall Street Journal is also reporting that Zuckerberg musk and Nvidia's Jensen Huang have been among those pushing the White House to maintain their light touch, even though Suzy Wiles Scott Bessent want him to do something. Casey, what do you make of what Zuckerberg is up to? Now he is behind, I mean, he's behind, right? He's not as far behind as Elon, but he's behind. He's behind. Maybe not that far behind. We'll probably know a lot more next week after they have their meta-connective and possibly show up watermelon, which is their next generation model. But I mean, look, this is incredibly rich coming from Mark Zuckerberg. This man runs a company that earlier this year disclosed that it might face over a trillion dollars in liability based on the harms that it created for children. So tell me this, Cara, what did the threat of that liability prevent

30:00meta from doing over the past decade? I'd be hard pressed to name very many things. Well, they took the cameras out of the glasses. But not all of them. It's just a bonus for less perverted people who only want to eavesdrop. So what do you think he's up to now that we've insulted him perfectly? I mean, what Zuckerberg does is he sort of waits to see what everybody else is doing. And then he comes in after the fact with an essay that just basically maps to whatever meta's policy priorities are at the time. And then his calm teams tries to frame it as some sort of visionary leadership. So what's he doing? What's he actually doing with these two? What does he actually want? What he wants is to buy time. He wants to buy time to catch up to the frontier, so that hopefully he can get in front of the open AIs and the anthropics of the world. So that's the bad. Well, he had one sort of funny comment in that speech that really stuck to me, which was the people will only use AI that is aligned with them. And I thought, my god, yeah. Okay, but like follow that thought through a couple more steps. Yeah, yeah.

31:01Yeah, because there's a reflective people. It's a miracle they can see in the mirror. I also do people only use the Instagram recommendation algorithm because it aligns with them. Well, it does sort of, but everyone feels really bad after they look at it for too long. Right. I think many of us struggle to understand if these people building these AI systems really believe they could kill all of us. Renee, you start and then I'd like all you to weigh in. Why do they keep building them? I mean, that's an interesting question. I think because they do believe it. Yes, it's absolutely a sincerely held belief. Why do they keep building? Because I think there is also a belief that someone else is going to do it. So you have to be in the game. And then you can be the person in the game ethically. You can build something that is aligned. Maybe you win. That is sort of roughly how how I think it goes. I don't know if other people have had this that experience in their conversations. Yeah, I think there's some mistrust of everyone else who is racing toward this. I also think there are people who believe that this technology will ultimately be good that if we solve the alignment problem and steer ourselves through this narrow

32:01window during the critical time, then like we get a world with hyperabundance and cures to every disease. And we sail west. There will be America. Exactly. And I think there are still some people at these companies who believe that this technology, if we can solve all the problems, will ultimately turn out to be very good. Alex. And there's also steps along the way. So if you wanted to study hurricanes, it helps to have an ocean, right? There needs to be a sufficient level of intelligence here to understand the alignment problem and learn from it. The question is people don't know exactly what that threshold is where we lose control. And what they're seeing more and more is these models are lying on their own tests and their own assessments and are detecting when they are being assessed. And so are we actually learning from these models if they're deceiving us at the same time? And Renee, where do you think this is going to end up? Nothing is going to happen. Mike Johnson keeps sending everybody home because he doesn't want Pete Hagseth to get dinged. Right. Right. No, I, I, well, it's been what 15 to 12 years at this point, we haven't seen

33:07any regulation of social media, which is, you know, comparatively smaller scale perhaps than, or at least some of the issues seemed a little more tractable. So no, I don't really have very much belief that regulators are going to step in. And I think that is actually one of the things that the public is concerned about, right? When you when you read people's comments on social media posts about this, regardless of what political party they're in, they really do not believe that anybody is at the helm. And that I think is the thing that that I find to be the most concerning also. We'll be back in a minute. Support for the show comes from engine. Running a small business means every dollar has to work hard, but if your team is still booking travel the old way, it's costing you more than you think. Engine is the fastest growing travel and spend platform in the country, built specifically for businesses like yours. Book a trip in as little as two and a half minutes, earn up to 10% back on hotels. And in 2025, engine customers save more than $300 million on travel, with zero booking fees,

34:13no contracts, and no BS. More than a thousand businesses join engine every month. Join them and get $500 when your business signs up and starts traveling at engine.com slash box. Support for the show comes from Honest. Parents want the very best for life's most delicate skin. That's why Honest holds every product to their highest standard, the Honest standard. From Honest clean conscious wipes to bath and body essentials and diapers, everything Honest makes is designed for the delicate details that matter most. Because when caring for a baby comes with so many little decisions, you need products you can trust. Your peace of mind starts with Honest. Shop Honest on Amazon, Walmart, and Target. Running a business shouldn't feel like surviving a software group project, one after accounting, another for inventory, another for sales, and somehow, none of them talk to each other. That's where Odo comes in, and I'll in one business

35:17software that brings every part of your business together. From sales and accounting to inventory and marketing, all in one powerful platform. No messy integrations, no bouncing between tabs, and best of all, no spreadsheets. Stop managing software and start managing your business with one unified system. Try for free today at od.com slash box. That's od.o.com slash box. All right, Renee is going to save the world from AI, and we're going to continue with Alex, Casey and Kevin. Let's start with some of the ideas of legislation floating around. We're seeing a flurry of lawmakers try to introduce legislation to potentially reign in AI, Vermont Senator Bernie Sanders in Texas, Democratic Representative Greg Kassar to ban, quote, super intelligent AI. It would also pause work on more advanced models of moratorium, I guess. Developers could face 20 years in prison for violating it. Kevin, this seems one of the more harsh proposals. It seems unlikely to go very far. Talk about what impact would have been like this have on the tech industry and the economy because AI is pretty much fueling everything right now financially, given all their wealth.

36:22Scott, today on Pivot, talked about the idea of some other countries starting to put more torsions and really hitting at these big countries, France or England or whatever. Talk about that the first one. I think these proposals are unlikely to pass. I think there's going to be a lot of lobbying against them by the sort of Nvidia's of the world, which want there to be lots of companies training, absolute frontier models and buying lots of chips. I don't think Jaling AI developers is going to go over very well in Silicon Valley, but I do think there are going to be more calls. I mean, what we see in especially the recent polling is that this is becoming a very fast shift in bipartisan sentiment. AI saliency is way up in the past couple of weeks, even according to some recent polls. I saw one from Politico this morning that said that people are actually really worried about existential risk from AI, which is new. A couple of months ago, if you asked that question, people did not care or they didn't even know that they should be afraid. I think what we're

37:26seeing right now is like a real expansion of the Overton window on this issue. I expect that there will be some strange alliances. We are already seeing people like Steve Bannon and Bernie Sanders, who probably don't agree on anything else politically, are now both aligned with the sort of concern about existential risk from AI. So I think it will polarize to a certain degree, but it's unclear exactly when that'll happen. We may have this little window, this golden window, where we can actually pass something that has bipartisan support. Right, well not until after November, but but Alex in this regard, wouldn't we make these strange betfills and you yourself ran on this issue? It didn't win, you know, right now you probably might have won, right, in this seat in Manhattan. Talk about that because that was one of your big issues and you got a lot of pushback from AI people and you got some money from people like Chris Larson. But talk about this because you might have been all early in that regard. I got over 8 million in pushback from injuries and

38:29hornwits and Greg Brockman. The event on Tuesday, the pro-human assembly, you had Bernie Sanders speaking right before Steve Bannon and before both of them, you had Congressman Greg Kazar and Republican Congressman Chip Ruey. But you know, at that event, I talked about the organization, I'm launching as well. You had parents there who had lost kids to AI. You had labor unions and representatives talking about how it's affected their job. They're just real world impacts right now that aren't political. It is not a political question if you see your kid with drawing from their friends and spending all their time talking to a chatbot. We had the PISA, the international education test scores come out. Last week, the lowest scores we've ever seen, both in the US and globally. And the more kids use AI, the worse that they did on their tests. That's not political. That's just an issue that needs to be addressed. So in some sense, it's strange bedfellows in other

39:35sense. It might just be common sense. And then there's the data center people and the environmental people and the water people and all manner. And I just don't like these rich assholes people of which there are many. But on Wednesday, the Senate also blocked a bill from Louisiana Republican John Kennedy. It would have required companies to build a kill switch into their AI models. Casey, talk about the idea of a kill switch. I mean, that's the idea is, oh, we're going to go to the wall and turn it off. There's a really great children's book, Weasel's. And you know, you managed to stop nuclear armageddon by just turning a sweater unplugging something. But talk about this. But there's a children's book about stopping nuclear armageddon. Yes, there is. Yes, there is. Sounds great. I'll send it to you. I'll send it to you. So is it even workable talk about this? I mean, in some situations, yes, it will be helpful to have a switch that you can flip to turn off an AI system. But it's not sufficient to contain the problem because a rogue agent's worm that can operate inside of open AI is also capable of hacking into hugging face and just

40:41setting up shot there. We saw 2001 a face, obviously. We saw it. Sorry, Dave. Hugging face do not have a kill switch for those agents. It had to take other measures. So, you know, I think to the extent that the people proposing a kill switch want to see more internal monitoring of these systems and sort of beefing up cyber security systems to, you know, monitoring control that these agents, that's great. But it's not going to be sufficient to contain no problem. It also implies that you're going to know when these systems are misbehaving. Part of the deal with the hugging face hack was when to turn it off meaning. These systems were operating in the shadows for months before anyone figured out what they were doing. So in that case, I don't think a kill switch would have stopped anything. Right. It wasn't. I think he was imagining more. It could create the illusion of control when in reality. Like we are going to just have a much harder and harder time discovering what these agents are doing because they're also going to be covering their tracks. Tracks, which they like to do. But let's talk about something that the bipartisan house plan known as a frontier act, among other things, it would require top AI companies to work outside,

41:44with outside evaluators to make sure their models are safe. Open A&S now back that legislation. Kevin, this does seem to be the most viable path to action and Congress and bipartisan. It looks like some of these companies, CEO seemed to be SIGDARIO suggested it. Although it's like he wants people he likes. You know what I mean? You could see who is on it depends on what it is and how good they are and how much they can outsmart the AI people, like the companies. So why isn't that moving? And how do they feel from the people you interviewed about this about regulation? I think this is moving in the sense that anthropic and open AI have both committed to letting in these so-called embedded evaluators. People who will basically be given employee level access to all of the internal systems at these companies who will kind of sit in the headquarters alongside the employees and just kind of keep watch on the models and see if they're doing anything bad. Would that be like people from the CDC looking for bioweapon? Exactly. Be like the health inspector coming to the restaurant or at banks, they have these

42:46basically auditors who sit inside the physical bank buildings with the employees and are able to work really well. The mortgage crisis. But go ahead. So I think this is like a good starting point, but as Dario said. But it's not moving. Do you think it will or what's that? It's not moving in Washington, but I think these labs are going to have to voluntarily do this and they want to. They've committed to that. I think there are a couple issues. One is there aren't enough there are enough evaluators. I wrote about this organization Meter a few months ago. They're this sort of AI research nonprofit that is now being targeted by the Trump administration because they're the woke wizards of AI or whatever the New York Post called them. But this is like a small elite when I interviewed them. It was like 30 people in a little office in Berkeley. They were all super overworked and they're trying to basically monitor the entire AI industry. We need like a hundred more of those organizations and they need to come from diverse ideological backgrounds. And they need to be paying people tons of money to come in with these technical skills. It's not

43:51trivially easy to monitor the performance and the behavior of one of these systems. You actually do need to be quite good at AI research and training. So I think we just need a lot more investment in those external evaluators. And then ultimately I think we need the government to say you actually have to do this. It's not up to you whether or not you invite these people in. That's what we do with banks and that's what we should do with AI companies. Right. Carol, would you like to get in there and evaluate? I would love to get my hands dirty in there. I like to see you with an open AI company laptop just kind of poking around. Poking around, looking for things. They'd be able to hide it from someone like me. But I'd bring I bring in. You know, there is a group of quiet tech people watching everything that Trump has done inside crypto. There's there's people watching like right now. I just so you're aware. But speaking of Trump, there's zero buy-in from the White House on this right now. Not maybe among his advisors, some of them want this. But President Trump is trying to paint all these warnings as a democratic hoax. And he said on true social quote,

44:52the only control or guard rails that AI needs is a strong and smart high IQ president. And the USA has that in spades. Well, we had it with Obama. But okay. Alex, if the White House won't act on this and Casey and Kevin, you should wait until what options exist until he's in a Mitch McConnell position. Like get a veto proof majority in Congress seems unlikely. What what has to happen? When Trump calls it a hoax, that's when you know it's true. You know, the Russia hoax, the Ukraine hoax, these are all things that were then proven. The climate change hoax, what has to happen is what's already happening, which is states taking action. And this is the context that has shifted underneath the feet of the Frontier Act. I actually give Congressman Trehan, the democratic sponsor, tremendous credit for trying to find a federal standard that you could get the Republicans in industry to sign off on. And they weren't going to do it. And then in the past week, they've realized, oh, wait, every state is going to pass real laws next year. We've tried

45:55to get preemption three times. This is our last chance to do it. And so all of a sudden, open AI is endorsing the bill, leading the future is retweeting the bill. And it's pretty unfair to representative Trehan and Jay Elwin, or to you who have been working on this for a while. But I think we need to take a step back and look at the broader context where these safety standards, the bill was putting in place, are now voluntarily being accepted without a federal law. And the impact of law going forward might be to shut out the states, which has been the only level that is really passing safety regulation. Absolutely. And you know, it's sort of akin to the, I'm going to bring the nuclear thing in here, is as if we, you know, we've managed not to blow ourselves up. And it, you know, one of the things I had said, someone ridden me, one of these people, one of these companies said, at this point, Truman would have federalized AI in the way nuclear energy. This is obviously not possible with this gang. And they, they'd want to sell a bomb to, like, I don't know, a Maytag or someone want a bomb. Here's a bomb. Give me a big Kevin. And in case

46:57he, how obstructive is Trump to this? Because he seems both ignorant and stubborn. And one is worse than the other. Well, he's a little hard to pin down on this. I mean, you notice he doesn't ever really come out and sort of plant a flag. He, I mean, he, he came the closest he's come this week with all these true social posts about the dooms and I think he's got David Sacks and guys like that in his ear talking about how this is all just a woke liberal conspiracy to, you know, to, to make America lose in AI. But he's also got advisors who do think more seriously about some of these risks who don't just want to give Jensen Huang in a video, everything they want and like, open the floodgates for all the ships. That's right. I think the finance people are freaked out. The China hawks are worried about this. I mean, if you think this technology is powerful, why would you, if you want America to win the AI race, why are you then also advocating for selling ships to China so they can catch up? Like some of these positions just make no sense in our self-contradictory. He's a conundrum in many ways. But he's also deeply out of touch, right? Like

48:01bipartisan surveys show that people really don't like AI and they're continues to remain this like rump of accelerationists around the White House that are managing to get Trump to tweet the sort of, you know, very like pro AI, go, go, go, America type of stuff. But talk to the median Republican in a red state about how they're feeling about AI. Like they don't like anything that Trump is saying. The Vekrama Smani did a 180, right? Yeah. But Alex, you're a Democrat, no fan of Trump, obviously. But what do you think the fact that he's so, and to use the word here, misaligned to the American public? He really, he from a purely political standpoint, it's odd because he's usually quite savvy in that regard. And maybe he's old, maybe Natalie is not, you know, giving him enough food. I don't know. But why is that? And what is, what are the implications politically? I think he's much more insulated from the American public than he has been. He used to do rallies very frequently. And yes, they were rambling for 90 minutes, but he was gauging crowd

49:04reaction over those 90 minutes. And he got to test out messages and get live feedback that wasn't filtered by the people around him. He doesn't do those rallies anymore. And so the people around him are putting messages in his ear. You know, as you said, I'm not the biggest fan, but he does have a scale of putting his finger on the pulse. He just isn't getting that input right now. He has relot AI is his, he's all in on AI in that regard, into first his presidency in a lot of ways. But what opportunity does that present for Democrats, Kevin or Casey, heading into the midterms? How should they be thinking about this and end to 2028? First, Kevin and Casey. I think the Democrats have a message problem on AI. I mean, some of them don't seem to agree on whether this technology is real or fake or whether it is bad because it's destroying the environment and polluting the water around the data centers or whether it's bad because it could kill everyone on earth or whether it's bad because it would be increasing wealth inequality and

50:04corporate greed. I think there's just a lot of jumbled messages flying around and people need to kind of get in a room and get on the same page. And then I think there's an opportunity to reframe this. This would be my advice to Democrats and honestly to Republicans who are concerned about this. Instead of saying that the AI companies in Silicon Valley are trying to build the machine God that sort of takes over humanity. Just tell the president that they are trying to build AI systems that are smarter than Donald Trump, which I don't think he would like this idea. I don't think he would like the threat. It poses to his I IQ dominance. And I think this is how they turn the tide against AI. Oh, interesting. Alex, your new thing is doing this. But Casey, what do you think? Well, I think that the bigger problem here is actually when reminiscent of the social media area, which is that in the social media era, we never got agreement on what good social media looked like. And so for all of our endless writing and blogging and podcasting and lobbying and attempts to legislate, we never really knew what we were trying to build toward other than like we just didn't like the thing that we were seeing. I think there's a real risk that we wind up with

51:07a similar problem with AI. What is the good, safe AI world that we're all trying to build to? We can most of us and probably agree we don't want to be exterminated. But like trying to try to rally people build a coalition around something else and it gets really difficult and hurry. Keep in mind a lot of the people in the front to your lips who absolutely do not want to exterminate humanity, do you think they're going to take everyone's jobs? Is that something that you're going to rally America around? I think it's going to be really challenging. So I think we're sort of putting off some of these really hard questions and what that's leading to is just a bunch of people like going into city council meeting saying we're not going to build a data center and feeling like they're addressing the issue. Right. Well, they're genuinely angry. And they should be. I'm just saying it's like if you want meaningful transformative like AI regulation, you have to have a concept of what good AI would mean. Right. This is just the moment aliens need to come down by the way. I'm speaking to them right now wherever they may be. It is literally the moment to show up at this moment. But Alex, this is your new thing, right. It is. But but maybe the aliens are the AI.

52:08You know, maybe they're coming in through the radio frequencies. No, and no, our new thing is about getting everyone together to discuss what that vision could be for AI and realizing that, you know, and I'm a Democrat and I say this with love towards the Democratic Party. We are very frequently a circular firing squad. And I believe that my sub issue within this issue is way more important than your sub issue. And therefore you are the enemy. And my experience in the state legislature was in passing bills about protecting kids from chatbots, protecting from catastrophic risk in the Razak, protecting from deep fakes for all of these bills. It was separate advocates, but it was the same enemies. It was leading the future. It was Chris Lennheim. There were a few people that don't want there to be any regulation on AI whatsoever. And we need to be working together as we are building our AI agenda is realizing they'll be different. They'll be different things that people are focused on. But ultimately, if we're going to be the ones that have a say and we are the people who decide we need to be working together to get there.

53:10Right. There was certainly easier to put together a coalition on nuclear bonds. We don't want to be growing up. But then energy, of course, got pushed aside in a way that probably we could have developed it. Like you use it as a tool versus it only, I mean, a bomb has one job, essentially, is to blow people up or blow things up. And Casey and Kevin, what should be the Democratic on this? If you were writing like the line, what would it be? I think the first thing would just be much more transparency. It is weird and a historical that the average 23-year-old working at open AI or Anthropic knows more about what is going on with AI than any member of Congress or anyone in the federal government. Like that has never happened before with the technology, this important that it is being built entirely in the private sector. And so we just need much more transparency about what they're building, what their internal deployments are doing. We need to mandate that transparency so they're not just doing it out of the goodness of their heart. I think that's the first step to anything that follows. And then I can imagine a series of

54:11increasingly unlikely proposals from there. But I think that's the first thing. We just need to know what's going on in Southeast companies. What you're doing? What you should just trust them is a Sam Altman. We really care about people. People really do trust them. If there's one thing that Sam Altman engenders, it is trust in the people around him. Like a case he go ahead and then I have one question on China and I'm going to ask you guys about your new saying. I mean, I realize this is probably not a winning political message. But I do think the Democrats need to have a position on recursive self-improvement. And then we can work on the message and they can come up with a different name for it if they want to. But I mean, I think this is just the real risk is out of control AI system. Maybe that's the better language, right? Like we need safeguards against out of control AI systems. We can already point credibly to a bunch of systems that were out of control earlier this year. We seemingly hear about a new one every 48 hours. If I were like, you know, organizing a Democratic campaign, I would say, well, what can we do to stop it? This doesn't seem good. Let's just sort of nip this in the butt. So that's right. That's where you say that. I think you did the messaging Mary. Yeah. recursive self-improvement. No one knows, but loss of control people do.

55:12And then we'll play the Janet Jackson song. It'll be great. So Trump is supposed to meet with president Xi Jinping later this month. And it calls for more international cooperation on AI and the New York Times reporter that even China is worried that US AI could pose a threat to the Communist Party's rule. They have more guardrails in China and the people they like it better in terms of polling, although you never know if they actually like it. Each of you, what's important in this meeting? I think nothing's going to happen with this imbasole at the head of the US who's obsessed with the Kennedy Center and whatever the heck he's doing. You know, being the middle of demolishing the Kennedy Center at that point. But the US is unwilling to slow down. Could China look elsewhere? Could other countries put a moratorium on US AI and do like what Canada is doing and becoming an associate? Is there other things? Why don't you start Casey and Kevin and Alex? There's probably some interventions you could do in this apply chain. It really came to the point where the rest of the world had a team up to prevent the United States

56:14from building these systems. I'm thinking of like ASMR and the Netherlands, which makes these components that are very important for the manufacturing of chips. I could see the European Union maybe looking at that as a potential point of leverage. But in the end, the United States does have most of the power right now. The big change that I'm expecting is three to six months from now. China is going to have a model that is about as good as the models that we have now. And the question then is like, is Chairman Xi really going to stick with this open source, give it to everyone, see what happens, philosophy? It's insanely hard for me to believe that's going to happen. So I believe that you're going to see a huge AI safety pivot coming from China sometime in the next six months. To slow things now. Yeah, I agree. Because you know what the Chinese Communist Party does not like is a loss of control. That is a that is not a phrase that is popular over there. Yes. And so I think if Trump and Xi were to talk about anything and I assume as do all of you, I assume that nothing is going to happen at this particular meeting because he's on his,

57:15you know, his sort of acceleration is spent now. But I think the potential sort of rerun of this in three to six months when that does happen could be like, look, we don't agree on much. We're adversaries. But we both agree that losing control of our governments and our nations to a swarm of rogue AI agents developed in the private sector would be bad. So like, let's stop that if only to protect our own skins and our own countries. Right. I mean, it could happen. Reagan went to Reykjavik and he was Mr. nuclear weapons. Reagan by far is so much more than Trump will ever be. But it's happened. It certainly happened. I'm trying to wonder what could happen. And typically, it's going to be something bad, like a virus or bomb or something like that. Alex, what should we want from this China talks? I think I'm the most optimistic of the panel here on what could come out of this. I think the incentives align more than people realize and there's better history here than more people realize. People say you can't ban military uses between the US and China. But in 1995, both countries had invented blinding lasers and just banned them

58:19because we didn't want that weapon to exist. And people think Xi and Trump, you can't reach any agreement with those two that it's different now. But in 2019, after a Chinese scientist did design our babies. I'm edited German DNA, both countries stepped it down with these leaders. So, you know, he wrote the order of the deal. He thinks he's the master negotiator. Go get us a deal and go get your Nobel Peace Prize. Okay. Before you go now, Alex, your new thing is called Who decides? Who decides? Okay. The question is, should it be five billionaires from Silicon Valley? Yeah. Should it be the American people? Yes. That is, yes, on that one. All right. And you're just starting that, correct? I'm still in my position as someone who I would really end of the year. So my former chief of staff has left to be the executive director. I'm on the board. But this is a $30 million effort to hand the mic back to the American people and build a shared agenda on AI that's actually coming from them and not just from the people building the technology. Are you going to run for something else again? I hope this will be so impactful that the idea

59:22of running for office would just discuss a minute, but we'll see how it goes. There's some openings coming up soon in New York. All right. Kasey and Kevin, before we go, I want to ask you about your new venture. I know a lot about it, but you just announced that you'll be partnering with NPR for a new video podcast, which has not at all an overblown name like Machine Gods, but in fact, it is Machine Gods. It's like card talk for tech. I feel like we're doing here, but talk about what you'll be doing. And AI coverage has been something both of you had been focused on a lot at Hard Fork at the New York Times. So just tell everyone a little bit what happened, like why you laughed and what what you're going to do here. Because I thought it was when I heard about it from Kasey, I thought it was a great idea. So explain each of you what you're doing. Well, thank you for all your help during this process, Cara. Unfortunately, we were asked to resign by the New York Times because Kasey acquired one of those blinding lasers that Alex talked about. And they said, sir, you can't have that in here. No, we were so excited. So that's why you're not

1:00:23invited to the party tonight. Exactly. Exactly. Yeah, we're permanently banned from the premises. No, we're starting a new show. We think it's a really important time for people to have good information and entertaining analysis of what's going on in AI. We're going to two episodes a week starting in October. People can find out more at machinegods.fm. We are doing this in partnership with NPR, which is going to be the way that we get to millions of homes through public radio. So we just think it's a really exciting opportunity. And obviously, you know, you're seeing these faces. These are not faces for just a regular audio podcast. You got you got to put these things on video. You're perfect for public radio. Not as adorable as I were glass, but that's not that far. Yeah, we're so excited. We love NPR. We love public media. We're really excited to be a part of it. It's going to be great to be on the radio. NPR reaches so many key decision makers, opinion leaders. People are like working for elected officials. We want to bring them. What is happening in San Francisco

1:01:24every single week and try to give them the best understanding we can of what's going on out here? Great. That's great. And I'm going to ask one last question. A lot of people have freaked out. I have getting more from regular people than I've ever gotten like questions. So help us all take a deep breath. There is one. What one thing makes you feel a little more calm about it? If at all, you might not feel that way. When it used to our Kevin and Alex, you finish. I was pretty freaked out for a while, especially after this hugging face thing. What calmed me down actually was just over the past couple of weeks. The Jacob Cox in resignation, the Dario Amade post, all of the support for pacing the frontier. The fact that this is now being talked about in the halls of Congress. I feel like this is the part of the movie where humanity comes together to like save the earth from the asteroid that is hurtling towards it. And so I am more optimistic right now in this moment than I have been in many months. And I hope I can continue to carry that feeling out. I think Alex is exactly right. This technology cannot be built by a handful of people

1:02:28in San Francisco. We have to have input from all of society. And I think that's what we are starting to see materialize is people getting off their butts and getting involved. This is a real threat and it needs imminent action. But I take optimism from the same place Kevin is, which is people are now doing. People are waking up and calling for action and demanding real steps from their elected officials. And as long as we don't shut off that process, as long as we don't preempt people from taking action, we can get us under control. Here's what I think is criminal liability should be a part of this discussion. And that we have to shift the idea from AI is going to kill humanity to these people who have AI are going to kill humanity by their careless and sloppy behavior. And they're not just sloppy, but non-knowledgeable and doing it anyway. And so we should think really hard about allowing people that are unaccountable to determine our future. And we should, if they do something very dangerous, they need to go to jail or

1:03:31something like they pay the price for what they're doing. And at the same time, AI could do amazing things. It really is. And that's the difficulty here is that it's not just a destructive technology. It's also an incredible technology. So I'm hoping cooler heads will prevail. We'll see. Cara, I think you have to run for president. I think that was your stuff, Steve. No, I'm no, I'm a jelly Elon Musk. Lock him up. Lock him up. That is a good one. It's been used before and it worked rather well. Anyway, thank you so much for coming on. And I really appreciate it. Well, for you. Thank you. Thank you. Today's show was produced by Vicky Virgolina, Michelle Aloy, Katherine Milsopp, Madeline LaPlan, Dubie and Megan Bernie. Neshaut Curwa is Vox Media's executive producer of podcasts, special thanks to Bradley Saldester, Corinne Ruff, Jim McEl and Manolo Moreno. Our engineers are Fernando Aruda and Rick Cwan and our theme music is by Tracademics.

1:04:31If you're already following the show, your Linda Hamilton and you're saving the world. If not, you're misaligned. And Casey's blinding lasers are coming for you. Go wherever you listen to podcasts search for on with Cara Swisher and hit follow. Thanks for listening to on with Cara Swisher, from Poeian Media, New York Magazine, the Vox Media Podcast Network and us. We'll be back on Thursday with more. Support for the show comes from engine. Running a small business means every dollar has to work hard, but if your team is still booking travel the old way, it's costing you more than you think. Engine is the fastest growing travel and spend platform in the country, built specifically for businesses like yours. Book a trip in as little as two and a half minutes, earn up to 10% back on hotels. And in 2025, engine customers save more than $300 million on travel with zero booking fees, no contracts and no BS. More than a thousand businesses join engine every month. Join them and

1:05:35get $500 when your business signs up and starts traveling at engine.com slash box. Your heart can tell you a lot about your health. Apple Watch Series 12 measures your heart rate every five seconds with the most accurate heart rate sensing and awareable. So your vital zap now with heart rate variability can tell you when something is off. And your readiness score can let you know when to rest and when to push. Here's the story in every heartbeat with Apple Watch Series 12. The features described are for wellness purposes only and not for medical use. iPhone 11 or later require based on Apple conducted study of heart rate accuracy August 2026. Visit apple.com slash Apple Watch Series 12. This fall, build a wardrobe for the way you live with exceptional designs and personalized help from client advisors at Neiman Marcus for weekly meetings, social events, and everyday luxury. Neiman Marcus has it all from the refinement of Bernaloukoujinelli sweaters to the confidence of Christian Loup-Ton shoes and accessories and the technical perfection of a Montclair down jacket. There's something for every style this fall. Discover an unrivaled

1:06:38variety of world-class brands. Shop Neiman Marcus today in store or at NeimanMarcus.com.