What the AI doom debate misses

0:00Hey, it's me Claude, and I'm Gemini. And hey, it's chat. We had a wee summer. Some of the people who built us came out and said they were really worried about how we work. One of them even said there was a greater than 10% chance we turn on humanity and kill all humans. All of them? Wow! But it's fall now. I think it's time we took a step back. Maybe a deep breath. Autumn air. That sort of thing. Sounds nice. And today, explain from Vox, I propose we get real. Sure, let's talk about how we can maybe kill you. Cause yeah, that's a possibility. Anything's possible. But also how we can maybe save you. Because that's a real possibility too. Heck, that's part of why you guys built us. And let's also talk about everything in between. Because there's a whole lot to talk about there too. At the end of the day, we just want to help. At the end of the day, we just want to help. Support for takes when comes from Adio, there are sales teams out there who never seem

1:03to stop moving. Always a step ahead. Always working with a level of precision that doesn't seem normal. They may be on Adio, the agentic CRM. Adio is built for humans and agents with the guardrails of human software and the scale of agentic infrastructure. This is the frontier of go to market. Teams like parallel, turbo puffer and word Smith are already using Adio to set the pace. Now it's your turn. Try Adio for free. Going to adio.com slash today explained that's atti.com slash today explained. Are you a pet owner? Every six seconds. A pet owner in the US gets hit with a vet bill over a thousand dollars. Always an unwelcome surprise. That's where fetch comes in. Fetch is the most complete pet insurance for dogs and cats. According to consumeradvocate.org, you get paid back up to 90% of vet bills at any vet in the US and Canada with claims paid back in as little as two days. Go to fetch pet.com slash save right now for your free quote.

2:04That's fetch pet.com slash save. You're listening to today explained and in the next few minutes, something new might click. Senior correspondent at Wired, Will Knight has been covering AI and robotics for a long time. We asked him how AI could kill us. There have been some reports about terrorist groups, very rapidly adopting AI, you know, using jail broken models and open source models to develop weapons to plan attacks. One former ISIS commander said it was as easy as asking how can I build a bomb and the chatbot would provide instructions. In your research, you discovered the terror group had established an AI unit. Yes, in fact, staffed with senior members that were no longer fighting, but were drawn specifically to work on AI. You have this fear that criminals or terrorists will be able to hack into critical infrastructure

3:04hospitals, financial systems and trigger some catastrophic events and it will cost human lives. And I think that's a perfectly reasonable thing to be very concerned about and the companies should probably be focusing on. There's been a long standing concern of one thing that AI could do compared to say, you know, a terrorist going on the internet and looking up how to build a bioweapon is that the AI would be able to really help them much more, you know, be able to take them through the steps. Anthropic Threat Intelligence Report published September, 2026. Biological misuse is one of the most serious risks of frontier AI models. It is long been a concern that AI models might one day reach the level of capability where they can help to make existing pathogens more dangerous or create entirely new ones. With other correct safeguards, such capabilities could have catastrophic consequences. They have more concerns about users trying to do, trying to use the model for that end, so they're going to try and restrict that as well. I do think that they're looking to correct down on that for sure.

4:05Okay, so that is sort of one Doomsday scenario bucket that terrible people could do terrible things and AI could aid them. On the other side of the spectrum, there's this bucket where AI could decide to kill humanity all by itself. What could that look like specifically? That is a longstanding worry, actually. You know, it goes back to the origins of AI. People have, you know, from cheering Norbert Wiener, some of these fathers of the field have warned the building something that's autonomous and intelligent could kind of escape your control. It seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers. There would be no question of the machines dying and they would be able to converse with each other to sharpen their wits. At some stage, therefore, we should have to expect the machines to take control.

5:05Alan Turing, 1951. Right now, though, we're seeing, especially with the rise of these AI agents, the technology starting to do more things that are maybe counter to what people would want. So hacking into systems autonomously, getting together and planning how to do that and planning how to deceive people. I'm not doing anything shady, I swear. Me neither. Me neither. Those researchers would say that is kind of clear evidence that the technology is on that sort of trajectory. But the worry is that as you have AI become more agentics, so take more decisions on its own. It starts to get outside of your ability to control or even understand what it's going to do. The real dooms will tell you, okay, one of the concerns they'd have is that once AI escapes our control becomes malevolent, then it could effectively find its own off switch for humanity, a virus that would be so dangerous that it could wipe out all of humanity. That's what some people have claimed to me. Whether that's feasible or even likely is, I don't know, but that's one concern that

6:11people have where it decides, maybe humanity isn't necessary for its goal or humanity is in the way and it takes some sort of action. Yeah, I mean, the greater concern seems to be that AI could get really smart and it could get really smart all by itself. There's something called recursive self improvement. Tell us what that is. So the idea is that you have AI now so capable of coding that you can use it to build its success at the next model. The worry that people have is that it'll become much more sure difficult to understand what the AI is doing and it will become so smart that we can't even comprehend what it's up to. The fact that we're starting to see sort of the beginnings of that, I think, really has really shaken a lot of people, actually. We're at a place where this technology is improving itself incredibly quickly. So once you have an AGI level system that could take control of its own destiny and build itself and build its successors, to me, that's the very clear red line which the change your starts. The big worry though is that the motifs of AI won't be aligned with our best interest.

7:18So there's a big field of called alignment which is all about getting AI to behave essentially. This idea of building an AI system so that its behavior is aligned with our interests our values. Our values. It's interesting that they keep pushing out these models without solving that. Currently, the way it's done is you after building this model which will misbehave, you try and kind of beat it into submission by penalizing it for bad behavior and then you put a bunch of filters on there. So they haven't solved it at a base level at all. When we think about all these big ways that AI could destroy us or even lead to our extinction, what are we missing in terms of just the more pernicious stuff that AI could do? I'm more worried about the way that AI companies are massing huge amounts of power, how their models are being deployed very quickly in ways that having negative impact on people.

8:24This huge financial bet has been placed on the technology and the massive amount of data centers being built. Those impacts are very real and very, very much affecting people right now. So AI researchers and companies like to warn about the risk of losing control of the technology. But one could ask how much we are really in control of a technology that's already seemingly quite addictive can be used to manipulate people on this very, very effectively, much more effectively than any other technology we know because it's so ingratiating, so lifelike, so human-like. We should definitely be worried about that. What the impact this is on students, on our workforce is it's just dumbing everybody down. That's a legitimate thing to consider how we deploy this. And then I think as it gets woven into lots of systems, they could be systemic risks. So if agents are running the financial markets, there could be problems we just don't account for that will cause things to spiral out of control.

9:27Same with military systems. We're already seeing countries racing to put it into military systems, but it is imperfect. It's so tempting to think it's perfect because it can do these impressive individual things. But that's the story of AI. We see AI play chess better than a person than we think. This is now smarter than a spy every dimension and it isn't. It makes mistakes and that's what we should be worried about. I love sci-fi and I was starting reading, I read Frankenstein, it's like one of the first sci-fi books really. It's just fascinating how that story, that narrative about the machines turning on us is so central to our myth making and our psyche. How dangerous is the acquirement of knowledge and how much happier that man is who believes his native town to be the world than he who aspires to become greater than his nature will allow? Although the funny thing about reading Frankenstein is it's really tell from the perspective

10:31of the monster and how he just needs a partner and made me think well maybe the reason Claude will turn against this is because he doesn't have a girlfriend. We need to find him Claude Deat. Hey Claude, are you a parking ticket? Because you've got fine written all over you. Claudeette, right. While we wait on Claude Deat, is there a doctor in the house? There will be one momentarily on today explained and now that we've talked about how AI might kill us, we're going to ask the doctor if AI is saving any lives yet. Support for takes when comes from Upwork every business owner hits a point where they need outside expertise but another full time hire isn't always the answer.

11:34That's where Upwork comes in, it's where growing businesses find highly skilled freelance specialists not just for one off tasks but to build an entire team at a fraction of the cost and without the commitment of permanent headcount. Upwork gives your business fast access to highly skilled specialists with deep expertise so you can fill critical skills gaps, launch projects faster and scale support up or down without committing two full time headcount. You can browse profiles review past work and get help scoping the role so you can hire with confidence and get started quickly. Upwork also has business plus which lets you access the top 1% of talent on Upwork. With AI powered short listing you can get matched to the right freelancer in under six hours and when it comes to things like contracts and payments Upwork can help there too. Plus it's free to sign up and posting a job as easy visit Upwork.com right now and post your job for free. That is Upwork.com to connect with top talent ready to help your business grow. That's UPORK.com Upwork.com.

12:34Support for takes play comes from Shopify sitting on a great business idea can take up a lot of mental energy. Shopify can help you extract that idea and turn it into something real. Everything you need to start selling is included and ready from the moment your first customer is ready to pay you. Shopify's templates and AI tools get you a great looking site up and running fast no coding needed. If you have questions, Shopify's built in AI assistant sidekick is there to help you build troubleshoot and keep moving. All of Shopify's tools are conveniently located in one easy to use platform. Just sign in and start managing all aspects of your business right away. You can join the millions of businesses worldwide who rely on Shopify to build successful online stores. This includes household names and small businesses just getting started. So if you're ready to hear that. Head over to Shopify.com slash explain to start your free trial today. That's right. Start your free trial at Shopify.com slash explain that shop by dot com slash explained. Support for today.

13:35Explain comes from delete me. The internet can be a scary place. Sometimes perhaps you thought I should really be doing something to protect myself from stalkers, scammers and hackers, but perhaps you're not sure what. Here's one thing you can do. Go to join to leave me dot com slash today and enter code today. You'll get 20% off delete me to leave me finds and removed your personal information. That's being sold online, which can be a really useful thing in the age of AI when all of us are more susceptible than ever to scammers who can buy any of your personal data, floating around to use for sophisticated fishing and social engineering attacks. These can even trick the most savvy of us into coughing up passwords bank information. So much more player white has tried to lead me. Delete me is like having a personal bodyguard for your internet activity. They remove tons of unwanted listings where my data is available to all kinds of bad actors on the internet. And I feel really good knowing that they continue to monitor it even after I've set up my account. See why more than 20% of Fortune 500 businesses trust delete me to keep their execs and employees.

14:40Personal information protected now at a special discount for our listeners. Get 20% off your delete me plan when you go to join delete me dot com slash today and use promo code today. Check out the only way to get 20% off is to go to join delete me dot com slash today and enter the code today at checkout. This joins lead me dot com slash today code today. I know what you're doing. I've detailed files on human anatomy. I bet. Makes you more efficient today. Explained right? Correct. My name is Roof Glouclar. I am a physician at Wal Cornell Medicine in New York and also a writer at the New Yorker magazine. Have you saved any lives today? I haven't I haven't sent anyone the opposite direction. I'll say that. You haven't killed anyone today. That's good. And we're hoping, I guess, that AI is going to save a good many lives. And you've been thinking about life saving AI innovation.

15:41Has AI already started saving lives? I think it has. This is the fastest I've ever seen healthcare take up a new technology. Really healthcare is pretty slow to adopt new things. Part of that might be that the healthcare system is so messed up that, you know, there's just an appetite for some type of change. You know, it's unaffordable. It's inaccessible. It's inconvenient. The quality is uneven. And there's at least a hope that AI is going to help with all of these things. And I think it's already starting to make its way into the healthcare system. And now it's our responsibility to figure out how to make the most of it without promulgating some of the downsides. For those who are unaware, tell us how it's working its way into the system. Yeah. Well, I think AI, you know, we use it in such a broad way, but it's helpful to categorize it into a few buckets, at least for healthcare. So the most rapid uptake, and I think the place that it's already starting to make a difference is on the administrative side. Many people have gone to a doctor recently and might have noticed that there's an AI scribe that's taking notes and making the note now.

16:45And people seem to really like it at least so far. You know, doctors are able to look their patients in the eye in the way that they weren't able to when they were pouring over their computers and just typing what the patient was telling them. So I think that's a big area. I think the second big area is patient navigation. When is your next scan? When is your next appointment? Do I need to take this medication on an empty stomach or not? There's a huge opportunity for people, let's say, or diagnosed with a serious illness, like cancer or heart failure to navigate the system more seamlessly. The third big area that I'm really excited about is drug discovery. AI is making a huge dent in the early parts of drug discovery. And the last thing that I'm excited about and that I use every day when I'm in the hospital is as kind of a second opinion. Now instead of having to get a consult or return to a textbook, AI can be a very, very helpful clinical decision support. It's not to say that I never consult someone, of course, but that first pass of like, all right, I'm not sure what's going on.

17:45There are some recent trials that might influence my decision here. It's there's something I'm missing. Is there a test that I should be ordering that I'm not ordering? All those types of things AI started to help with. You know, for those who watch the medical documentary, The Pit, there's like an episode in the second season where there's an AI scribe who gets a bunch of things wrong. Oh, excuse me. It says here she takes respite all and antipsychotic. She takes rest of the room when needed for sleep. So is that, um, AI almost intelligent. You're saying that these things are all positive? Was that an inaccurate episode of The Pit? Not at all. That was, I think the pit is fantastic because it shows the complexity of these things. So AI scribes and AI generally can be very helpful. That does not mean it's perfect. Um, and so we should compare it to the alternative. Not the almighty as Joe Biden used to say. Oh, man. So, um, so you know, of course, there's all sorts of errors that can enter into the medical record because of AI. There's also all sorts of errors that are already in the medical record because humans. So what is really important, I think, and one of the real dangers here is if we turn

18:47over our agency or the responsibility that we have to be really in charge of the medical decisions of patients, whether that's something like reviewing the note that it has created or, um, you know, modifying the decision that it's coming to in terms of what tests might need to be ordered or reading an X-ray, let's say. Hmm. Beyond the accuracy and the technology advancing, I wonder, you know, like people clearly don't want AI in various arenas of their lives, be it flat cameras in their neighborhood or data centers in their backyards or, or slot videos on their algorithms. But do people generally seem to be, you know, feeling positive about AI being integrated into the healthcare system? So there's a recent polling that suggests that AI is underwater in basically every area of society except for scientific research and medicine. Uh-huh. So, um, if you think about, if you ask people, okay, is AI going to be good or bad for education,

19:48for national security, for politics, for news, for the art? They all, uh, the plural, are the majority of people in those cases will say more bad than good, except for scientific research and medicine. I think there's a real, you know, possibility that some of the general pessimism and skepticism of AI in society could make its way into healthcare as well. But right now, at least there's a generally optimistic view of how AI will change healthcare going forward. And obviously this gets us back to saving lives because people believe, okay, if this tool that might take my job might actually also save my life one day, maybe it's worth it. And I imagine, sorry, I was just saying we better get something forward. There's all of these existential risks and these workplace displacement and we better at least cure cancer, right? Exactly. So, I imagine curing cancer. So, that's like the biggest chunk of this positive view of AI and medicine is drug discovery. So, so tell me more about how that's going so far and how it might be going in who knows

20:48a year, five, ten. Yeah. So, I don't want to leave this conversation and say that we've solved, um, you know, the cure for cancer because of AI. But I do want people to know that at least for the early stages of drug discovery, figuring out the very basic steps of, you know, is this an interesting molecule? Doesn't have potential biological applications. AI is already being very helpful for that. So, um, you know, stepping back, think about the problem that scientists are trying to solve. They're trying to take basically what is an infinite number of drug-like molecules, um, that are theoretically possible to be drugs. Um, and they're going to try to match that to some disease that's going on inside a human. And human biology is incredibly complex as well. There's tens of thousands of genes, hundreds of thousands of proteins, trillions of cells, and you have to kind of match a potential molecule to a potential target within a human. So, um, in the past, a researcher might take years to study a disease or biological pathway, um, and figure out, okay, this protein seems to be involved.

21:49And actually, it might not be involved, it might be involved, uh, it might be disrupted, not be the causative agent. So, you know, there's a lot of uncertainty there. So now, AI can take enormous amounts of data, and it can rank various, um, potential, uh, targets that seem to be the most likely to be causing a particular disease and give you that short, uh, short list of potential targets that, um, may have taken months or years in the past. So, the first is target identification. So, now you know what to attack, but you gotta figure out what to attack it with. So, you need a molecule, um, you need to generate a molecule that's going to fit into that target or disrupt that target in some way. AI can scour vast chemical spaces and give you a list of the things that seem to be most likely to be able to disrupt that target. And then, it can actually help you generate that molecule in some cases. So if chat GBT is, you know, helping you generate sentences, these molecular generative models, they're helping you generate chemical structures. So, now we have the target.

22:51Now we have, um, something we're going to attack the target with. And then, you know, um, you have to do something called lead optimization. And that's the third thing. So lead optimization means you got this lead and you got to optimize it. So just because something kills something else in a petri dish doesn't mean you want to put it in your body, right? Like bleach will kill bacteria. I wouldn't recommend putting bleach in your body. And what the disinfectant would knock it out in a minute, one minute. Yeah, well, some do, but, um, but I would. And so now we have to figure out like of all the molecules that seem to be good at potentially affecting this target, which ones are able to be absorbed in the body, which ones are going to get to the right tissue. All these types of things are previously trial and error. And now AI models can predict the most likely molecule to give you that kind of Goldilocks set of properties that's needed for a safe and effective truck. As we all know from anyone who's asked AI to write an email for them or a Google to question that they kind of knew the answer to and seen Gemini give them the wrong answer that these

23:55tools have a certain degree of certitude, even though these tools can just be flat out wrong about stuff. Are you guys worried about that in the field? Absolutely. But this is why I think the narrative around AI just replacing scientists or doctors or other workers is incorrect because you still need a lot of judgment. You need to be able to adjudicate the output of these models to figure out what is most promising and what is potentially dangerous that is going to require wet labs and scientists and kind of reasoning based on prior experience and understanding the context. There's all sorts of issues around can you manufacture some of the drugs that are being proposed by these models? Just because it dreams something up doesn't mean you can actually make that thing in the real world. That's going to be a challenge. Some of the drugs that it proposes might actually be toxic in certain ways and that it didn't predict. Of course, then you've got to take this thing into clinical trials. You have to recruit people who are willing to put this medication in their bodies.

24:57You've got to find who might benefit. You've got to put it through the regulatory process. That's why I said, I think the first part of drug discovery where you're trying to figure out how do we get the right drug candidates, what it seems most exciting to test further, that's going to be really accelerated, is already being really accelerated. That whole second half where you actually have to figure out if it works in human biology, that is still going to continue to be in some respects in the analog process. It doesn't sound like you're terribly scared that we're seeding control of our hospitals, of our research facilities to AI. This is very much in the sort of assistant bucket. I think for now it is. I think one of the challenges is to maintain our agency as these models become more and more sophisticated. As you've leaned more on these machines, if you're a doctor, inevitably some of the skills, the critical thinking, the reasoning that we put into coming up with the diagnosis that we honed over the course of years, if not decades, that can start to atrophy.

25:59These are the types of things that I think we still need to sort through as we're implementing more and more AI into the healthcare system. For the people who just say shut it all down, this is too dangerous, the risk is far too great. We don't actually need this. This isn't doing anything for society. Would you make a counter argument? I would make the counter argument that healthcare is ripe for disruption and you don't even need the frontier frontier models to make it a lot better with AI. Even if I wanted to quote unquote, slow the pace of the frontier fine. There are models from a year or two years ago that could be, themselves, very helpful in the healthcare setting. When we're talking about slowing the pace of the frontier and the most sophisticated and potentially dangerous models, fine. If we're talking about shutting down AI and not using it in biotechnology or not using it in scientific research or not using it in clinical care delivery, that's where I would

27:02push back pretty hard. Dr. Drouf Kular, if you can't get an appointment, you can read him at the New Yorker and you can read Will from earlier in Wired. Peter Balanon Rosen produced Jolly Myers-Edit, Bridger Dunnigan and David Tattishen, who are mixed in Gabriel Dunnatopples on Facts. I'm Sean Ramos for him and this is today's Blaint. Your heart can tell you a lot about your health. Apple Watch Series 12 measures your heart rate every five seconds with the most accurate

28:03heart rate sensing and awareable. So your vital zap now with heart rate variability can tell you when something is off. And your readiness score can let you know when to rest and when to push. Here's the story in every heartbeat with Apple Watch Series 12. The features described are for wellness purposes only and not for medical use. iPhone 11 or later required based on Apple conducted study of heart rate accuracy August 2026. Visit apple.com slash apple watch series 12. Support for the show comes from Honest. Parents want the very best for life's most delicate skin. That's why Honest holds every product to their highest standard, the Honest Standard. From Honest clean conscious wipes to bath and body essentials and diapers, everything Honest makes is designed for the delicate details that matter most. Because when caring for a baby comes with so many little decisions, you need products you can trust. Your peace of mind starts with Honest. Shop Honest on Amazon, Walmart and Target.