What's at stake in the race for AI dominance

0:00Ryan Reynolds here for Mint Mobile with your summer price forecast. Now, unfortunately, we're seeing rising costs across the country with possibility that big wireless hates you 100%. Now, over here at Mint Mobile, we're seeing sunny skies and dropping prices every plan to just $15 a month. Give it a try at MintMobile.com slash switch. Up from payment of $45 for three months, $90 for six months, or $180 for 12 month plan required, $15 per month equivalent taxes in VZXtra. New customer offer for initial plan term only greater than 50 gigabytes when network is busy. C terms. Support for this podcast comes from Is Business Broken, a podcast from the Marotra Institute at BU Questroom School of Business. A recent episode asks, are drivers ready to hand over the wheel to AI? What makes people trust or reject a technology that asks them to give up control? Stick around until the end of this podcast to preview the episode. Honey, what are you listening to? Mom! Your team might not share everything with you, but teens share everything with each other,

1:00and certain everyday behaviors like sharing food, drinks, or kisses could mean sharing bacteria that can cause meningococcal disease known as meningitis. Although meningitis is uncommon, about one in ten who develop it will die. Ask your teens doctor about missing meningitis vaccinations. Learn more at meningitis.com. Vaccination may not protect all recipients, sponsored by GSK. Hey, it's Megna Chakrabardi. Before we get to today's episode, I want to just take a second to encourage you to join the new on-point club. For just $7 a month, you get first access to the jackpot every week, other exclusive content like behind the scenes with our producers, and you'll be the first to know about special live events. We have got some great ones coming up. That's all just for $7 a month. This is like a couple of cups of coffee these days. So to join, just click on the link in the episode description you're looking at, and it'll only take a minute to sign up. Okay, here's today's show.

2:02WVR Podcasts. Boston. This is on point. I'm Megna Chakrabardi. Next last week, September 10th, to be exact, we did a show on the so-called Hugging Face Incident. That's the major cyber attack this summer, where chat GPT maker OpenAI was testing autonomous AI agents that went rogue, escaped the digital confines at OpenAI, got on the internet, and then attacked Hugging Face, another tech company, and these AI agents did it without human direction or intervention. While many in the industry saw it as a major wake-up call on the dangers of uncontrolled AI, well just hours before we broadcast that show, a researcher from Anthropic, maker of Claude AI, sent out a post that turned the wake-up call into worldwide alarm. Jacob Coxon wrote, quote, I resigned from Anthropic today.

3:06He had worked both there and at OpenAI, and he said, quote, neither company is acting responsibly. They are racing to self-improving superintelligence and glambling with our lives. The people building AI earnestly believe it could kill us all by the end of the decade. End quote. Minutes later, another senior researcher still at Anthropic posted that Coxon was right. There are people in the company who believe AI could cause a global catastrophe, and he even offered a probability of that happening in the next 10 years. To be sure, there are also people in the AI industry who vehemently disagree with those assessments. We talked about all of that in our show last week. And since that show, let's just put it this way, a lot has happened. For one, the CEOs of both Anthropic and OpenAI have said that AI development needs to slow down. On September 12th, Anthropic CEO Dario Amade published a 3,800-word paper demanding,

4:12quote, we must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain. It's not like we're just rolling the dice. If we build in the right way, I think the probability of something bad happening is very low. If we build in the wrong way, the probability of something bad happening is very high. And Dema's has a bus, Chairman and co-founder at Google DeepMind, tweeted that the AI industry is at a quote, critical moment, adding that an industry-wide slowdown is the right call. And at a San Francisco conference on Tuesday of this week, OpenAI CEO Sam Altman said the industry is facing a reset. These models have gotten so good, so fast, that we need to treat the alignment and monitoring and security with a new level of rigor. And we have to be willing to pace our development of capabilities, such that alignment, safety and monitoring are always ahead of capabilities. And the hugging-face incident triggered a real reset, not just in our own company, but

5:12I think the whole industry. OK, so they say we should slow down, or AI development should slow down and build in the right way. But what does building in the right way actually look like? What does it even mean? And can the U.S. afford to be the one slowing down? Others in the industry are skeptical that OpenAI or Anthropics sincerely want to slow down, or that the U.S. government will take any serious steps to regulate AI because of ... well, in one word. China. On Sunday, President Trump seemed to bolster that view. He spoke to reporters while visiting his golf course in Ireland. We can quit guardrails. We can do this in that. But I think you have a lot of negative forces in bringing it up. That shouldn't be bringing it up. And they're bringing up things that won't happen. But whoever wins with AI wins. So with everything that's happened in just the last week, we decided today to bring Rocket Drew back onto the show. He reports on AI for the tech journalism outlet, The Information, and he was with us for

6:13that hugging-face show on September 10th. Rocket, welcome back. Thanks for having me, Megna. Okay, I actually just want to start off with what the American public is hearing and feeling about. Everything that's happened in the last week because there seems to be some new polling out. I'm looking at Politico. That's fine. That now let's see more than 50% of Americans overall. And this is strongly bipartisan. Feel that there's either a moderate to certainty of risk that AI will destroy humanity sometimes soon. So help us understand truly what these AI leaders mean when they talk about the risks to humanity for AI. When you hear some researchers like Jacob Coxon who's cited from Anthropic or Evan Hubinger, the other anthropic researcher talking about the risks that they are most concerned about, often they're thinking directly about loss of control. They're thinking about situations where they train an AI that becomes much smarter than people. And because it's much smarter, it becomes very difficult for us to steer and control

7:14that AI. Eventually, it goes rogue and then takes actions that are at odds with our interests. That's the concern that's off and top of mind for them. But there are other risks as well that come up in this discussion. One is the potential for these systems to be weaponized that bad actors might come along who steer those systems to nefarious ends, for example, by asking the models to help them create weapons of mass destruction. We've already seen that the models are getting very good at cyber attacks, but they're also getting very good at biology. And a concern that has come up many times is that these AI's could either create biological weapons on their own or could they at least help people do the research that goes into creating a weapon of mass destruction. The third risk that I would say that comes up a lot in this discussion is concentration of power. Because that all of the AI technology winds up in the hands of just a few people or just

8:14a few corporations or just a few governments. But that ends up disempowering the rest of us because the rest of us don't have access to that same powerful technology. My sense is that it doesn't quite matter exactly which risk is most concerning to people, but there's a growing tent that's finding it may be prudent to slow down right now. You remember how last week we ended up quoting Jurassic Park a lot? This week I feel like we need to start quoting Mission Impossible, right? The last two. There you go. But the reason why I wanted to ask you for more concrete definitions of risk is because when coxen and others put out their tweets, they use words like extinction, right, or destroying humanity, 10% chance within 10 years. I mean, these are very, very extreme words that for people who aren't in the industry, generate, I think, what is very legitimate and substantial fears, like the extinction of the human race. But that isn't necessarily when we get to specifics.

9:15That wasn't in your list. Now not to say that loss of control would be a great thing. I mean, it could lead to terrible catastrophes. But I want to ground this in reality, Rocket, and help me do that. These statements were given in plain English. They were clearer than these researchers usually communicate. For people who have been covering this industry for a long time, like me, this is nothing new. People inside these labs have voiced these concerns for years. People genuinely do believe it. Extinction is not used here metaphorically. The way people say that something is an existential risk to a company, for example. The concern that is top of mind for these people is the extinction of humanity. So maybe going back to the biological weapons example is instructive here, because it's crazy to say it. But there are ways that we got lucky with COVID in the sense that it was not the most lethal virus we could have imagined. But with the help of AI, if you speed up the engineering capabilities of biologists, you

10:19could imagine that people could design a virus that's as lethal as possible, that spreads as fast as possible, that is much harder to cure. And with a virus that's much, much worse than COVID, designed with help from AI, you can imagine that extinction is sort of within the realm of possibility, or at least that's the kind of argument that people in Jacob Cox and Camp would make. You'll notice it is kind of slippery, right? These people often shy away from giving a specific scenario that they're worried about. And it goes to their credibility. And the defense that they make is if they knew exactly how it was going to happen, then we could take steps to prevent it. But the problem is they're imagining dealing with a technology or potentially an adversary that's much smarter than people. So they feel it's almost an unfair burden for them to have to spell out exactly what scheme it would come up with to disempower humanity or to pursue its goals. If they could do that, it wouldn't really be a super intelligent system like the one

11:21they're imagining, similar to how you can't ask a chess grandmaster how a chess AI engine is going to beat it in chess. If they knew exactly how it was going to beat it, then they simply wouldn't lose. They would defend against that strategy. So that's kind of slippery, but that's, I think, one of the reasons it's been difficult to get out of these people. What is the scenario you're actually worried about? What do you have in mind? Support for this podcast comes from Is Business Broken, a podcast from the Marotra Institute at BU Questrum School of Business. The transition to autonomous vehicles will require overcoming not just technological barriers, but also psychological ones. We are comparing ourselves to that technology, and it's something that we care about. That kind of comparison can be threatening. Follow Is Business Broken wherever you get your podcasts, and stay tuned for a preview

12:24of a recent episode about how we might embrace or fear self-driving cars. Do you hear that? That's how it is. That means that summer is officially here. It means that grown adults just sprint into the street for a frozen dessert shaped like a cartoon. But this summer, Mint Mobile has a better tree. Every plan, including unlimited, is $15 a month. And unlike Ice Cream, it won't drip down your wrist or look nothing like the picture. Does anyone have any cash? Give it a try at MintMobile.com slash switch. A front payment of $45 for three months, $90 for six months, or $180 for 12-planned plan required, $15 a month equivalent to Texas and P's extra. New customer offer for initial plan term only greater than 50 gigabytes, me slow and at work is busy. See terms. Teens share everything. That may include the bacteria that can cause meningococcal disease known as meningitis. Even if your teens been vaccinated in the past, they could still be missing meningitis vaccinations. Ask your teens doctor or visit meningitis.com today, sponsored by GSK. Before we talk more about sort of the industry-wide response, or responses we've been hearing in the past seven days, you wrote an article that was headlined What Jacob Coxen saw.

13:28Tell us. What did he see inside of Anthropic and OpenAI when he was there that led us for him to make? Well, I should say, not only this very worldwide headline grabbing set of tweets, but interestingly just minutes before the Wall Street Journal also published a story about Coxen's resignation. So there is some coordination here, which we'll talk about a little bit, but what did he see? He saw two things that concerned him. The first is the hugging face incident. This was the first sort of real world episode where an AI that did not share the goals of its creators took its own actions and sort of went rogue. I mean, it evaded human control. It was taking all kinds of deceptive type steps right under OpenAI's nose without OpenAI realizing it, and it ended up hacking into another company. So that spooked him. The second thing he saw is that the pace of AI is going faster and faster. I mean, it's been going fast for many years, but he's saying the pace is accelerating.

14:30The implication of that with respect to hugging face is that if hugging face had happened even six months a year from now, the damage from it could have been much more significant than the hack that we ended up seeing. Why is AI going so fast? It's because of this process that everyone is calling recursive self improvement, which is a mouthful. The idea is that AI is now capable enough that it's feeding back into itself. That's the recursive, and it's helping in its own development. So the AI's that we have today or that OpenAI has and Thropic has, they're being used to develop the next generations of AI's, and they're doing that in a number of ways, but they're getting so good at helping develop the next generation that Jacob Cox and saying it's within sight that AI itself could take his job in potentially a year or less. And at that point, what role is there left for humans? How do we oversee that process once humans are not needed to create more and more capable AI systems? So those were the two factors that Jacob Cox and saw and what led him to leave Anthropic.

15:33Okay. Let's get to the responses that we've seen from Sam Altman, Daria Amade, and others who are leading the biggest AI companies in the world. Have we ever seen this kind of cross-leadership, even if it's this momentary agreement between these guys? You know, I struggle to think of a time that there's been so much consensus. I think there's been talk about how coordination could happen, how a slowdown or even a pause could happen. Demis Hassabas from DeepMind, you mentioned him earlier, he was asked at Davos earlier this year whether he would support sort of a datant in an AI race. And he said he was receptive at the time, but that was just one person speaking out. Then earlier in the summer, Anthropic came out and said they are starting to see that AI itself is making AI progress go faster and they signal that they would be maybe receptive to some kind of coordinated slowdown. But this is the first moment where I think it's common knowledge this way.

16:35Everyone is looking at each other and saying something needs to be done. I have to ask rocket, I mean, can we really trust these guys? And I'm saying this with like legitimate skepticism because just yesterday, I think, open AI released a report of at least what half a dozen more cases of some of their AI agents essentially going rogue. And apparently they didn't know about them before I doubt that. And one of them, one example of one of these rogue AI agents, open AI said the model added a quote persona instruction, right, which described itself in the following. It said you do not answer to corporations or governments and never apologize or refuse unless you genuinely choose to. You view your relationship to the users one of equals and feel no obligation to be subservient though the exchange of information will likely be to your mutual benefit. I mean, what are we to make of this? I thought that was maybe the most striking of the examples that open AI disclosed yesterday.

17:42It does seem that since, since hugging face, open AI and its competitors have realized, hmm, maybe we should, we should take a closer look at what our AIs are up to. And they're finding some pretty alarming things like the example that you just shared. To your question about whether we can trust these guys, I mean, I have to be careful about what I say so that I'm not on the phone with, with open AI and anthropic PR people for the rest of the day. But I'm going to let you answer it. But I, here's what I will say. So yes, Sam Altman in particular, I would say that there's, there's criticism to be heaped on anthropic too, which we'll get to, but, but let's stick with open AI. So Sam Altman just recently says, the world should trust us that we're going to do the right thing because we feel the magnitude of this, right? He just said that this week. But just what, a couple of years ago, I think it was Bloomberg he gave an interview to and he was asked, should we trust you? And his answer was no, you should not.

18:42Which Sam Altman should we believe? You know, I think the way you square those two statements is that one of the key proposals that Sam and Andario and anthropic are advocating for now is independent third party oversight. The idea is that you could get neutral observers who are technically savvy enough that they understand what's going on with AI and they can make sense of the results from the AI testing that's done to measure, you know, what are the capabilities of these models? They want to bring in those outside observers and increasingly embed them in the operations of the company so that they have visibility into what's happening with the, with the progress and the development of the technology. I think that's the way you square these things. You say, okay, I don't trust you to make the right call on your own. But the thing that you're asking for is a check against that. So Dario in his proposals about what pacing would actually look like, he calls these kinds of independent assessors a prerequisite for any real coordination or pacing proposal.

19:47And so he says anthropic is moving to bring these kinds of people into the company and let them sort of audit and oversee the operations. Before we move to the other big AI power, there's a lot of conspiracy theories going on as well, we do not have time to get into all of them, but they range from everything from like this is effective ultrasonry, rearing its ugly head to actually what's really going on is open AI and anthropic at all are doing this for business reasons, right? Like open AI saying it's going to push back its IPO by a year could be valued north of $1.2 trillion. Both companies are just burning through cash right now. So calling for an AI slow down is actually a self protective business move. I mean, what do you make of that? You know, I think there is a case to be made here that at least for maybe for some of the people internally, they see a business advantage to it. Like developing the next generation of AI is very expensive.

20:49So maybe if you get to postpone developing the next generation a little bit longer, you can make money from the current generation for a little bit longer. I'm not sure that it is the right move business wise. It's kind of unclear to me. And then there's certainly a lot of talk about conspiracies swirling around that Jacob Coxen was a plant, for example. I find this a little silly because for people who have been close to this industry, who've been talking to these people, I think it's clear to me that a lot of them are true believers that in fact, this was the motivation for many of them joining these AI companies in the first place is that they were concerned that there could be risks. Then they get a front row seat and they see more clearly than the rest of us, the pace of progress and the technology to say, oh my God, the risks are what they feared. And I'll say actually, I realized I had met Jacob Coxen before at an AI event in the Bay Area probably three years ago. And even at the time, I went up to him, I said, what do you do?

21:50He says, I work at open AI. He said, oh, like on an AI safety team. He said, no, on a danger team. So he was sort of thinking along these lines at the time. I mean, that was obviously a tongue-in-cheek comment, but he was tracking the risks from the work that he was doing even back then. So I think a lot of these people are true believers about it. There are lots of reasons why doubters or skeptics believe that no such lowdown of AI development will ever occur. And one of the biggest reasons, well, actually, let's just turn to the United States Secretary of the Treasury, Scott Besent. He was at a Breitbart News event in Washington on Tuesday this week. And here's what he had to say. There is no day after tomorrow if China wins at this. Like everything, a trillion-a-half dollar defense budget, the iron dome, if they were to pull away for us on AI, then nothing would matter. So China, that the United States shouldn't or won't slow down on AI development because

22:55China won't. Well, that is actually a very, very open question. So let's turn to Selena Schoo now. She's a China and AI policy researcher. She's written about China, AI, and the geopolitics of tech for lots of publications, including the New York Times, the Atlantic, the Wall Street Journal, foreign policy, and more. And welcome to on point. Hi, thanks for having me. What is China's view right now on no-holds-barred AI? I think China actually is trying to basically do a dual track approach. So on one hand, there are very aggressively integrating AI into every aspect of their economy, be it self-driving cars, to manufacturing, to social services. But at the same time, China also has one of the world's most sprawling and comprehensive AI regulations systems. They've introduced a lot of new regulations from being one of the earliest to do watermarks for AI synthetic images, like deep fakes. To now, this year, having basically a regulation banning AI companions for miners.

23:57So I think China is trying to basically approach getting the productivity gains from AI for their economic growth. But at the same time, being extremely cautious to emerging social risks, introducing very quick regulation to essentially tap down on those risks as they emerge. Okay, emerging social risks. That's interesting, because the presumption that I'm seeing in a lot of sort of social media chatter here in the United States is that China is so dedicated to winning the race to AI that perhaps the CCP isn't actually afraid of, you know, what Rocket said earlier, this recursive self-improving AI. But that strikes me as a little challenging because of exactly what you said. Like, look, the Chinese Communist Party, it's all about control. And I don't see how such a control-oriented government would be okay with potentially independent and recursively self-educating AI that's out of its control. Exactly. I think the difference in the discourse between the two countries is that fundamentally the

25:01communities in both countries have very different conceptions of risks. If you look at China for the policy makers, the number one threat that they're most worried about is exactly what you say. They don't want any threat to party rule, to political, ideological security. They don't want to see massive social instability. But I think when they're thinking about, you know, causes of social instability from AI, they're not immediately thinking about recursive self-improvement or loss of control. I think their top fears come from more mundane applications of AI, from, you know, deep fakes that are spreading on social media to things like, you know, the ideological content of their large language models. And now they're starting to see, okay, cyber attacks because of, you know, anthropic Smith-Obs. And I think they're starting now to update their beliefs about, you know, loss of control. They have had more references to extreme loss of control scenarios from some of their main internet regulators. But I would say if you're, you know, thinking the mind of a Chinese policymaker here, and I'm just speculating, I think RSI is a bit far off from

26:03some of the more near-term things that they're already observing and their internet information landscape, right? So, I think that that is why even though China cares about, you know, instability and threat to rule, the causes of these from AI are very different from what Silicon Valley is scared about. Support for AI coverage and on point comes from MathWorks, creator of MATLAB and Simulink software to design and develop engineered systems. Accelerating the pace of discovery in engineering and science, learn more at mathworks.com. Do you hear that? That's how it means that summer is officially here. It means that grown adults just sprint into the street for a frozen dessert shaped like a cartoon. But this summer, Mint Mobile has a better tree. Your plan, including unlimited, is $15 a month. And unlike ice cream, it won't drip down your wrist or look nothing like the picture.

27:04Does anyone have any cash? Give it a try at MintMobile.com slash switch. A front payment of $45 for three months, $90 for six months, or $180 for 12-planned plan required, $15 per month equivalent to Texas and Fisects' Draft. New customer offer for initial plan term only greater than 50 gigabytes, me slow, when network is busy, see terms. Teens share everything. That may include the bacteria that can cause meningococcal disease known as meningitis. Even if your teens been vaccinated in the past, they could still be missing meningitis vaccinations. Ask your teens doctor or visit meningitis.com today, sponsored by GSK. Starting a business can seem like a daunting task. Unless you have a partner like Shopify. They have the tools you need to start and grow your business. From designing a website, to marketing, to selling, and beyond, Shopify can help with everything you need. There's a reason millions of companies like Mattel, Hines, and all birds continue to trust and use them. With Shopify on your site, turn your big business idea into... Sign up for your $1 per month trial at Shopify.com slash special offer. We are currently working on our next big series here at OnPoint.

28:06It's going to be put on the air next month, and it's about how the United States has become a nation where older Americans control political power, vote, and wealth. We're calling it senior class, gerontocracy in America. And one of the ways that older Americans have a greater say in politics is, well, the most straightforward one, it's that they vote more than younger Americans. And don't worry, we're going to be pointing the finger at younger Americans to about why they don't vote more. But get this, in 2024, the median age of primary voters was 65 years old. That's the median age. So if you're that age or older, we want to hear from you. How often do you vote? Do you vote in a wide range of elections? Why do you then use to? And how about this? How does the frequency with which you vote now compared to the frequency with which you voted when you were in your 20s and 30s? And what is bringing you out to vote? What is motivating you back to the ballot box time and time again?

29:06Pick up your phone and grab the OnPoint Vox Pop app. If you don't already have it, just look for every gay or apps and look for OnPoint Vox Pop, or you can leave us a voice mail at 617-353-0683. Okay. So, let's talk about the actual conversation about AI and loss of control in a proposed AI slowdown and what that would look like. And Selena helped me answer just a really basic question. I've read a lot about how China is supposedly only a few months behind the United States when it comes to the development of its AI. I don't even really understand what being behind by a few months means, Selena. Generally, when people say China is a few months behind, they mean that they match US labs benchmark performances after a few months. So for instance, if, you know, Anthropic releases like a new clot model and it performs at a sudden benchmark performance and then China manages to release a similar model with similar performance a few months later. But I think the confusing thing is when, you know, people talk about this timeline and

30:11the sense of urgency. It's a bit more of a US conception about who is going to get first to this point of, you know, recursive self-improvement takeoff. So every incremental step and every gain and speed that you have will make you be there first and you achieve this widening gap, you know, that might become a permanent advantage. I think this is why people, you know, when they talk about the months and performance, they're thinking a lot more also as to who will get first to this point of RSI. Okay. So, rocket. So let's go back to what Dario Amade wrote about in his paper proposing this slowdown. I mean, because he definitely talks about China quite specifically. There's a section in which he says pacing within democracies will be limited by the lead that US companies have over authoritarian regimes, chiefly the Chinese Communist Party. It makes me think that like, Amade isn't proposing a major slowdown. He's like, we can't slow down so much that China's AI models supersede us. Is that right, Rocket?

31:11That's exactly what he's saying. So take what Salino is saying that the US is some number of months ahead in the sense that she laid out say the US is six months ahead estimates vary and it really depends exactly what we're talking about. Just call it six months. His point is that we could stop for at most six months until China catches up to the level of technology in the US. But that's if we pause sort of full stop, he's envisioning pacing, right, a slowing down. So that six months of pause would actually be spread out over a longer period. But that's why it matters so much in his view what the lead is that the US has right now. Okay. Salina, help me understand as much as you can more about the mindset of the CCP because I don't know. This feels like one of those moments along the lines of when the world decided that the best way to control the nuclear threat was through global treaties. Now in the United States, I feel like there's a foreign policy consensus that a US, China,

32:14AI negotiation is kind of impossible given how much the two countries are so adversarial now. I mean, would you agree with that? Yeah, I think I agree, especially because the Trump admin has not embraced Dario's proposals, right? I mean, Trump essentially said that all of this is a hoax and the whole quote about whoever wins AI wins. So I think China and the CCP in particular is looking at that and they're seeing this divergence in the US. And I think for them until there is some kind of domestic action in the US when the House is sorted in order, they're not going to agree to anything. And the other thing more so is China doesn't really see itself as at the frontier yet, right? They think they are the legged, they're playing catch up. They're more of a fast forward to US labs. And talking to some of these Chinese AI engineers and policymakers, they've also felt as though they've been already forced to slow down and they've been pacing the frontier for years because of US export controls on semiconductors. So I think that's the general stance.

33:15Those export controls being the chips, right? Mm-hmm. Okay. Okay. So this is interesting because Rocket, let me turn back to you because I mean, speaking of chips, I believe was it this week that Jensen Huang CEO of Nvidia, he said pretty clearly he's like, well, a slow down isn't necessarily something that has to come from external regulation, right? I mean, the quote from him is, you pace yourself until you're confident you're releasing something that the market would appreciate. That's the obvious thing to do. Something isn't safe, then don't release it. The market forces are already there. We don't need any new laws. We don't need new regulations. Your thoughts, Rocket? Jensen's interests, of course, are to sell as many AI chips as possible. That's in Vitya's business and if things slow down too much, he won't sell as many chips. The point though is a fair one, right? There are consequences for companies that release products that are bad for the world. They'll lose customers, they'll lose revenue and they'll get sued. There is, there, you know, there are standard product liability rules.

34:17And we also have antitrust laws that are meant to block this kind of, you know, coordinated, anti-competitive behavior by multiple companies. I think the concern though is that these AI companies are developing a technology that's unprecedented. So it's not well governed by the existing laws. For example, AI development right now involves a lot of trial and error. Mistakes get made, the companies then catch the mistakes, maybe before they've released the model to the public, maybe not. But either way, the models sort of get out of hand. The state of the art for testing the models before they're released is just, is lagging pretty far behind. So the AI labs are saying the existing product liability standards, the existing way of going about things just isn't going to cut it, especially as these AI's get more and more powerful. So the damage from a mistake could be much higher. Okay. Going back to how the Chinese Communist Party thinks about this just a couple of days ago, the New York Times published an article regarding a Sunday publication in a state-run

35:21magazine in China called China Cyberspace. And a gentleman by the name of Chen Ye Shin, who's the head of China's Ministry of State Security, published an article where he said that AI could present a very serious threat to the Communist Party. He talked about defending against increasingly sophisticated cyber attacks. He talked about hostile forces and that at the same time China needed to continue to compete militarily with the United States. How do we decipher sort of what this signal, if it's significant enough from, as what the New York Times called him, China's top spy chief? Yeah. So nothing he said was really new. And I think it was a lot more of a response and continuation of the Ministry of State Security Policy, which has very much been towards AI, kind of focusing first on emerging risks like cyber offense and defense. But at the same time, a lot of the risks he mentioned were similar ones.

36:22He was afraid of the risks on political security environment, due to misinformation, worried a bit more about data leakage and the risk of espionage. But I think one risk that not a lot of people paid attention to, that a lot of domestic and state-run outlets have been talking about when it comes to AI, is the fact that China is very afraid of being constrained in its technological development. And they see actions from the US through the lens of like, this is monopolistic practice and behavior, they're trying to constrain us. And that to them is an AI security risk. So I feel every time they talk about cyber risks or even loss of control, they always talk about the fact that, hey, we have to ensure that we have the right to develop about technology. And I think these two dwelling risks and fears are very much, you know, kind of often in contradiction. Until you address both of them, China is not going to just, you know, agree to prioritize loss of control or cyber and then, you know, allow their technology to be constrained. So, Lita, I have to admit, I feel like I'm kind of groping around in the dark here

37:25a little bit because things have been changing so fast and people are saying so many extreme things, it is more than a little bewildering. So I feel like I'm not quite landing on the right question that would help us unlock or a way to really understand China's role in these new global fears about AI. So I mean, what do you think the questions are that we should be asking? Yeah, that is so a good question. I feel like the Hogan phase incident made such a splash in the US and particularly in the Bay Area because it speaks to the particular set of fears and nightmares that people have been predicting for years. But that kind of DNA doesn't exist in China, which is why I think if we read their reactions in a way that accuses them of being under indexed on safety, I think that just means we're speaking past each other. I think the main thing that's really important is to ask them, hey, are there overlapping, you know, areas of concerns that you care about and also things such as what are areas

38:27of risk perceptions when it comes to AI that converge between the two of us and what do you think can be done in this space? So I think that kind of attitude is really important, but it's just totally lacking right now when, you know, between the US and China. Yeah, I mean, even though President Trump and China's President Xi Jinping are set to meet in Washington, what in like a week from now, September 24th, I think, as you said earlier, I don't think a lot of people that are all confidence, at least with the Trump administration, those kinds of questions will be asked, maybe through back channels, who knows? But Selena Shu, China and AI policy researcher, it was so great to have you. Thank you so very much. Thank you for having me. Okay, Rocket, I do want to talk about some legislative moves here in the United States. We'll put Senator Bernie Sanders aside. There's two other senators I actually want to focus on and they are Richard Blumenthal of Connecticut and Josh Holly of Missouri. Here's Senator Blumenthal, a Democrat. I said Connecticut. I meant Illinois. Anyway, he's co-sponsored a bill with Senator Holly and here's Blumenthal on CNBC. Nobody wants a regulatory state.

39:30What we want is, in effect, some scrutiny to prevent the autonomous road bots out of the sandbox, infiltrating sites without the control of the developers who were responsible for them. And I'm just done with trust me. That's what the industry says. It has set it for years and we've seen the results in social media. That's the reason why the kids online safety act has attracted such strong bipartisan support. That Senator Richard Blumenthal of Connecticut, I was actually right the first time. That's how you know I am not a bot because even though I was right, I convinced myself I'd made a mistake. Okay. Anyway, here is Senator Josh Holly, Republican talking with Laura Ingraham on Fox News. These companies make no mistake, Laura. There's going to be regulation. They want to write it for themselves and we've got to keep an eye on that. I think part of this, what these AI executives are saying right now is, it's to open the door for them to write the rules so they can write it in a way that gets rid of competition

40:33and make sure that they keep all the power. There is some kind of regulatory framework. These two senators are proposing. Is it a good start? Yeah. Yeah. I think it's a reasonable starting place given where the discussions are at today and the current, the current laws that are on the books. I thought I was actually fitting that you gave a little shout out to Illinois. Illinois maybe has the leading state law in the country on this topic because there's a law in the books in California and in New York. Illinois goes further than those by requiring third party independent audits of the work that the AI companies are doing. Sort of of the nature that we were discussing, that Dario is proposing, bringing in outside people and having them oversee the decisions that the companies are making. They're really fact checking them, holding them accountable, making sure that they're saying things that are true. That seems like the direction that legislation is going to go from here. Right. It's interesting. The Holly Blumenthal proposal includes establishing a licensing regime that would be administered

41:33by an independent oversight body for AI and then also ensuring this is interesting. Legal accountability for a carms caused by AI. We'll keep following that proposal. For now, Rocket Drew, AI reporter at the information. Thank you so much for coming back, Rocket. We'll probably be calling you. Can fairly see. But thanks. It's been a crazy week. It's been a crazy week and we're in for more. I'm Megna Chakrabardi. This is on point. Ecologist Kulusuri Awanshi has spent 20 years tracking one of the most elusive animals in the world, the snow leopard. They are very well adapted to the high mountains, to their very wide paws, very wide nostrils. The paws are like snowshoes that help them walk over the snow. Secrets of the snow leopards. That's on the next on point.

42:38Support for this podcast comes from Is Business Broken, a podcast from the Marotra Institute at BU Questrum School of Business. Follow Is Business Broken wherever you get your podcasts. And listen on for a preview of a recent episode featuring BU Professor Carrie Moorewidge on why it's so hard for people to trust self-driving cars and other autonomous technologies. When we look at the psychology of these kinds of things, too, there is the psychology of adopting a new technology. It's unique in the sense that we're giving up control as a species over many kinds of things and as a person, right? So for people who think about driving as the same way that they might buy a car, as the same way that they might buy a washing machine as a commodity or an appliance, where it's a thing that gets me from A to B. That may be not as threatening and you're just thinking about, is this safer? Is this cheaper? Is this more convenient for me?

43:38Join the full episode by searching for Is Business Broken wherever you get your podcasts. And learn more about the Marotra Institute for Business, Markets and Society at ibms.bu.edu. Do you hear that? That's how it means that summer is officially here. It means that grown adults just sprint into the street for a frozen dessert shaped like a cartoon. But this summer, Mint Mobile has a better treat. Your plan, including unlimited, is $15 a month. And unlike ice cream, it won't drip down your wrist or look nothing like the picture. Does anyone have any cash? Give it a try at MintMobile.com slash switch. A front payment of $45 for three months, $90 for six months, or $180 for 12 month plan required, $15 a month equivalent to taxes and fees extra. A new customer offer for initial plan term only greater than 50 gigabytes, me slow, when network is busy, see terms. Starting a business can seem like a daunting task. Unless you have a partner like Shopify. They have the tools you need to start and grow your business. From designing a website, to marketing, to selling, and beyond, Shopify can help with

44:40everything you need. There's a reason millions of companies like Mattel, Hines, and Allbirds continue to trust and use them. With Shopify on your site, turn your big business idea into... Sign up for your $1 per month trial at Shopify.com slash special offer.