The Anthropic-Pentagon Standoff

0:00It's the Brian Lara Show on WNYC Good Morning Everyone. So let's talk about a standoff in the war that's not between the U.S. and Iran. You may have heard this, you may have not heard this, it's between the U.S. and the Artificial Intelligence Company and Thropic. Now, and Thropic has certain ethical guidelines that it tries to abide by and it's AI product known as Claude. They want to adhere to these ethical standards, even though they are a defense contractor voluntarily doing business with the Pentagon. This is a war-specific dispute, yes, but it also raises questions for so much of the growing use of Artificial Intelligence in so many parts of society and so many aspects of our lives. But on this specific story, here's where the drama began. Last July, and Thropic became the first AI company to receive a contract with the Department

1:05of War or still Department of Defense, if you prefer. In this contract, and Thropic was granted national security clearances, which allowed Claude to be used on the military's classified networks, making it the first AI model ever to operate at that level. But picked into that contract was a catch. And Thropic's own terms of service, which prohibit Claude from being used in autonomous weapon systems, we'll explain what that means, or for the mass surveillance of American citizens, you probably know what that means. The Claude is now at the center of a standoff that has put the company's entire relationship with the federal government at risk and is even raising new questions about possible abuse of power by Trump and Secretary Pete Hexeth. Let's discuss. With me now is Stephen Levy, editor at large for Wired. He wrote an article called AI Safety and the War Machine just before the U.S. launched

2:06this war. Now it's doubly relevant. Stephen, thanks for joining us. Welcome to WNMC. Thanks, my pleasure to be here. And to get us into this story, I think it's worth hearing a couple of sound bites of Hal and Thropic CEO, Darryl Modi, has framed what his company is actually fighting for. So here he is speaking with CBS reporter, Jolene Kent, last week. I believe that we have to defend our country. I believe we have to defend our country from autocratic adversaries like China and like Russia. So that's on the one hand, but at the same time. I have always believed that as we defend ourselves against our autocratic adversaries, we have to do so in ways that defend our democratic values and preserve our democratic values. So Stephen, let's start there. How did a self-described AI company with these values end up in this deeper relationship with the U.S. military in the first place? Well, that's a great question. I think that, you know, Anthropic was built on the premise that they can do AI safety

3:12better than other companies, and that other companies would emulate them and build safety just as strongly into their products. They call this the race to the top. As a matter of fact, Anthropic was founded by people who left OpenAI, which was a leading startup in the AI field, because they felt that OpenAI wasn't being safe enough. So they also do believe, as Dario said, that it's important to use these technologies to advance our national interests, as a matter of fact, Anthropic has been more hawkish than other companies in saying that we have to deny our best chips and technology to China, whereas the Trump administration sort of backed down on its original stance of doing that, and licensed chips to China, the Nvidia chips, and Anthropic as an outlier saying we shouldn't

4:13do that. So they put themselves in this position by signing, as you mentioned, the contract saying, OK, we can use this art technology, and they have among the best AI technology to protect our nation, to use it in the national security context. But here's what we won't allow, and key one thing is autonomous weapons in terms of drones that can pick their own targets and exercise lethal force, and Anthropic believes that we're not ready for that. Technology is not good enough to do that reliably, and they don't want their technology being used to mess up in targeting, and down the road, there's a specter. If you have autonomous drones going around there that make the citizens on their own, then this stuff can get out of control. You could set swarms of drones that decide on their own, who to kill, that's an alarming

5:17concept. Yeah. Well, here, in fact, hang on, I want you to talk about HEXF, but let me play another clip of a mode in his own words, making the argument against fully autonomous AI weapons. He doesn't want his AI model used that way. This again was on CBS. The AI systems of today are nowhere near reliable enough to make fully autonomous weapons. Anyone who's worked with AI models understands that there's a basic unpredictability to them that in a purely technical way we have not solved. There's an oversight question, too. If you have a large army of drones or robots that can operate without any human oversight, whether there aren't human soldiers to make the decisions about who to target, who to shoot at, that presents concerns, and we need to have a conversation about how that's overseen. Well, that kind of sounds like common sense.

6:18What's the argument against it from Pete Hexf? He basically is taking a stance that in the heat of war, we should use whatever is available to us in the way that it's going to be maximally efficient and lethal as we need it. He basically decided about a month ago earlier this year that Amphropic couldn't be trusted to provide the technology if they weren't going to be on board for anything that Pentagon wanted to use it for and wanted to change the terms of the contract. That's where we wound up with his impasse. You can imagine if listeners you've run into AI mistakes or what they call hallucinations in little ways in your life, what the implications might be if AI models are picking out their own targets without human supervision. I'm thinking of a little thing, Stephen, where I was looking up sort of wars going on around

7:20the world and the AI model in my Google search was the first thing that came up and one of the ones that it listed for Latin America was Armenia versus Azerbaijan. Well, guess where is not in Latin America, but if it can do that, then I don't know, maybe that girl school in Iran where 175 people reportedly were killed already in this war, most of them children, somebody made a mistake, we don't know if it was AI that made a mistake, but you don't want AI to be able to potentially make that mistake without human supervision, right? That's right. Oversight is so important when you use AI, whether you're doing research and in the process of putting that research into a final product, you've got to check everything very carefully. Even more carefully, as you suggest, if you're targeting something with lethal

8:22force, you really want to be sure that you've got it right, otherwise you will hit a girl's school instead of a military target. This is what greatly concerns Anthropic and I think down the road, I think it should concern all of us in terms of how AI is used in general. The big question is there's no ceiling for the capabilities of AI as it's being developed. We're trying to figure this out and companies like Anthropic and OpenAI and Google and the rest of them are developing this technology and trying to make it safe, but they're in a race between each other who's going to get to artificial general intelligence first, and that's more important to them, I think, in the priorities than doing it safely. We see through the military lens, how dangerous this can be. And again, the two ethics clauses in Anthropics terms of service

9:27are prohibiting the AI model plot from being used in autonomous weapon systems. That's the one we've been talking about. The other one is for mass surveillance of American citizens. Here's CEO Amodate on that. I am concerned that AI may be uniquely well suited to autocracy and to deepening the repression that we see in autocracies. We already see it in the kind of surveillance state that is possible with today's technology. But if you think of the extent to which AI can make individualized propaganda, can break into any computer system in the world, can surveil everyone in a population, detect, dissent, everywhere, and suppress it, make a huge army of drones that could go after each individual person. It's really scary. So Stephen is Amodate describing anything that the Pentagon was specifically asking for,

10:32or is that a more hypothetical concern at this point? Well, I think it really isn't a remote possibility. If you see the nature of the argument, and you can go way down into the weeds about what in the contract with Anthropic, the Pentagon wanted to do and what they've forged with a contract that they did execute with Open AI, which Russian did take his place, you want to watch out for what they won't do. What they seem reluctant to grant the AI companies is the promise that they won't use all kinds of information that's available in the public sector or from data brokers and then use that for how they treat citizens. It turns out what's legal to do could be much more powerful if you use AI

11:34and much scarier in terms of a surveillance state. I think the AI companies, particularly Anthropic, are very sensitive about that. They don't want to be the trigger to make things 1984 dystopia and they're trying to say, don't use us for that. So now we get to the retaliation piece if retaliation isn't too strong the word. How did the Pentagon react to Amodine's company Anthropic wanting these two ethics clauses in their contract with the Pentagon? Well, this is the really striking part of the story in terms of how the administration is acting. You would think that they would say, you know what, Anthropic, you've got these red lines, we can't accept these. We're going to go to another company and use their technology. We'll have a transition period where we'll phase out your clause AI model and do something else and we'll

12:39go our separate ways. Instead, what the Pentagon did and the administration has embraced is saying that we're going to label you what's called a supply chain risk and this is something used against typically foreign companies which you've seen as our enemies. They've gone even farther and said that any company that does business with Anthropic, the government will not do business with and that's a potentially crippling blow to a corporation. Are they going through with it? Well, we've seen several agencies, Treasury, House of Human Services say that we're not going to use Anthropic and we're not going to do business with other companies that do this. Now, so far, Donald Trump has written this. It's only in the tweet. They actually haven't formally invoked this but the agencies are saying they're doing it anyway and Anthropic is indicated they're going to sue to fight that. The Financial Times reports that as of this morning, talks between Anthropic and the Pentagon have quietly reopened. Is there a version of this

13:44that ends well from an ethical standpoint or does it seem like Amode will need to relinquish his ideals for the sake of his company's profits or even existence if the retaliation is too strong? Yeah, Amode, he put out an internal memo that was very strong. He said the reason why they're being punished is because they don't cater to Donald Trump. They don't do what he called dictator like obedience to Trump. They don't give money like one of the top executives of OpenAI gave $25 billion to Trump PAC and he said that's what's behind all this. So it's a little surprising that they're going back to the table now. But I suspect if that's happening, it's because the Pentagon's realizing it's maybe tougher than they thought to disengage clawed from their activities, especially during a war and they need Anthropics help. But to your larger question,

14:46there actually isn't a way to spin this or something we're all as happy. A couple of years ago, we were in a position where all the AI companies, most of the people in the government and Congress were saying, you know what, we have to come across with some sort of regulation, some sort of plan to figure out how we're going to deal with AI's very scary powers. This great opportunity is but also very scary powers. They talked about regulation, they talked about international bodies that might eliminate AI from certain kinds of warfare. That's all gone now. We're all about competition with China and AI. The companies roll about competition with each other. So I see a rather bleak outcome that this flap is a symbol of. I said in the intro that this standoff between AI and the Pentagon, it's obviously specific to that company and that branch of government.

15:49But it might have overtones for the safe deployment of AI in all kinds of sectors of American life, world life and society. Do you see that? Absolutely. I mean, I think, you know, it's, yeah, because I just said, this is an indicator that when it comes to what we once thought very recently was, you know, an important effort to, you know, go internationally even to figure out how this technology could not be used to the detriment of human kind. That's gone by the wayside and now it's, you know, accelerate, accelerate, accelerate until AI is, you know, beyond the point we can control it. They do in the show. Listeners, we're going to talk about risks and benefits according to a doctor of using AI in any way with respect to your medical life or your health. That'll come up in about an hour. For now, we thank Stephen Levy, editor at large for wired among his recent articles,

16:55AI Safety and the War Machine. Thank you very much for joining and explaining this to our listeners. I think this one's flying a little below the radar with the actual military strikes that are going on, but it's so important. Thank you. Thank you, bro. Since WNYC's first broadcast in 1924, we've been dedicated to creating the kind of content we know the world needs. Since then, New York Public Radio's rigorous journalism has gone on to win a Peabody Award and a Dupont Columbia Award among others. In addition to this award-winning reporting, your sponsorship also supports inspiring storytelling and extraordinary music that is free and accessible to all. To get in touch and find out more, visit sponsorship.wnyc.org.