AI Could Kill Us by 2030. But That’s Not the Real Problem

0:00A former anthropic researcher recently went on CNN and made a very wild claim. There is a real possibility that AI, artificial intelligence, could lead to human extinction within the next few years. I am not an AEI researcher and I don't know if that prediction is right. I have no idea of knowing. And I also don't really have the background to analyze it properly and come up with a reasonably accurate opinion of my own. So that's not what I'm here to do. However, there is a much, much older, much more human problem with everything that's been going on with AI recently and I am here to talk to you about that. Let's be clear that this is not a new problem. This is a modern version of a very, very old problem. In this episode, I'll be reacting to the CNN interview with this AI researcher as well

1:01as reacting to this very fear-provoking claim. So with our further ado, let's get into it. Jacob Coxen has left anthropic sounding the alarm about the dangers of artificial intelligence on his way out the door. The 27-year-old outlined his concerns in a series of social media posts when he quit writing in part, the people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible, but I hear the same people express fear privately. No other human activity poses this level of danger. 2030 is just a little over three years away. That's very, very soon. But that's actually not what scares me. Maybe it happens in 2030. Maybe it happens in 2031. Maybe it happens in 2050. In some sense, it doesn't really matter because that's not the purpose of what I'm going to talk about here. Now I don't want to be insensitive to those people who find something like this particularly

2:03triggering the end of the world and everybody's freaking out. I don't want to be insensitive to that, but I'm not looking at this in terms of today or tomorrow. I'm looking at this throughout the broad spectrum of human history, which is quite large. So three years, if we all die in three years, that's going to be terrible. However, for the purpose of this analysis, I'm really looking at this in terms of the broad expanse of human existence. However, what does interest me about this claim is that it's really a recurring theme in human history. Human beings have always seemed to develop power before we developed the wisdom to wield it. Let's look at some examples. Fire, weapons, industrialization, nuclear weapons, social media, genetic engineering, and now, of course, AI. We don't become wise and then get power. We get power. We develop power and then we make mistakes or misuse that power and then we discover what

3:04responsibility comes with that power that we've gained. Nuclear weapons are a great example to dig into here. The United States was racing to develop an atomic bomb during a World War driven in part by the fear that Nazi Germany might develop one first. Even while some of the scientists involved were beginning to wrestle with the implications of what they were creating, development continued. It didn't stop them. It didn't slow them down. They kept going. Now eventually, of course, the bomb was developed. It was used. The war ended, but the problem did not end with it. Other nations developed their own nuclear weapons and suddenly humanity had to figure out how to live with a power capable of destroying all of civilization, all of the world as we know it. We developed the power first and then we had to figure out what it meant in terms of responsibility. How should we use it? And if you look at history, there are situations in which nuclear annihilation came frighteningly

4:10close to happening. That is a terrifying reality. One of the most famous examples happened during the Cuban Missile Crisis. A Soviet submarine carrying a nuclear torpedo was being pursued by American ships that were dropping practice depth charges around it trying to force it to surface. The Soviet crew had limited communication with Moscow and didn't know whether war had already begun or not. The captain reportedly wanted to launch the nuclear torpedo. Ultimately, they did not. Vasiliy Arkipov, one of the senior officers aboard, played an important role in preventing that decision from being carried out. But think about how insane that is. Humanity had developed weapons capable of fundamentally changing civilization, and within a couple of decades we found ourselves in situations where confusion, fear, and the decisions of a handful of people could have helped trigger nuclear escalation. We had acquired the power.

5:12We were still trying to figure out how to live responsibly with it. In some, there's a gap between acquiring power and learning the responsibility it takes to wield it. And in that gap, there is usually pain. Let me give you a personal example that you may be able to resonate with. When you first acquire the power to go out past nine o'clock as a teenager, you don't necessarily foresee the pain. Or sleep schedule, dangers in the night, and so on, you have to learn that responsibility through experience and sometimes through pain. But the real question here is what happens when the gap between acquiring power and acquiring wisdom becomes too small for us to survive the lesson? What happens when the pain can literally mean our doom? That gap was almost too big with nuclear weapons. But what about AI? Well, let's listen a bit more. You know, a lot of these CEOs of these companies will go to Congress and say, hey, we would

6:16love regulation and thropic in their new statement right now says the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models. You know, the argument, it's an arms race. If the US company doesn't achieve the best AI Chinese company or some bad actors out there might. This is a very scary argument. It's very similar to the same argument that existed during the development of nuclear weapons. The argument is if we don't do it, someone else will. And so it's better to have the power in our own hands than in the hands of bad actors to put it simply if I don't take the power, someone else will. And this is where things get really complicated because what if they're right? What if one company slows down AI development and another doesn't? What if one country decides to restrain itself while another country races ahead? Everyone involved could theoretically recognize that slowing down might be safer while simultaneously

7:18believing that slowing down individually would be really foolish. And that's the scary part. Nobody has to actually believe that racing ahead is wise. They only have to believe that somebody else won't stop. And that fear is enough to keep the arms race going. The problem with the argument if I don't take power, someone else will is that it tells me something about what my competitors might do. It doesn't necessarily tell me what I ought to do. What is the right thing to do? There's still a moral judgment that I have to make. And philosophically, that's extremely important. You don't get to completely outsource your morality to the behavior of your enemy or your competitor. The answer still might be that you have to develop the technology. Maybe there are legitimate national security concerns. Maybe allowing a bad actor to develop something this powerful first creates an even greater

8:19danger. Now those are very difficult questions to answer and we're not going to answer them. We're trying to solve them here. But why you are doing something is extremely important. Are you acting from wisdom? Are you acting from fear? From greed? The pursuit of power simply for its own sake. If I don't take power, someone else will cannot answer those questions for us. Now one of the more interesting logical flaws we tend to make here is equating intelligence with wisdom. It's a mistake that so many of us make all of the time. We do this rather casually as humans. Just because we have greater intelligence does not mean that we have better judgment about what is right. There are many brilliant people throughout human history who have been evil. Nazi Germany was filled with many of them. And there are many people who may not be particularly intelligent in the conventional sense, but who are deeply moral people.

9:20Intelligence and wisdom are not the same thing. With AI, we potentially have a very interesting combination, extraordinary intellectual ability without necessarily possessing wisdom, virtue, responsibility, or even concern for human flourishing. And this is part of the reason why this person is speaking out in the first place. But there's a very uncomfortable question that's underneath all of this and that this person is indirectly speaking to. Are the creators of this technology any different? Not necessarily because they are evil, but because intelligence has never guaranteed wisdom in us as human beings. So why would we assume if we develop AI and have it be the superpower of intelligence that it would necessarily create morality within itself? It certainly has not been the case for human beings. Now I want to be clear that from a moral perspective, AI does not need to be evil for it to destroy humanity. This isn't the Terminator movie.

10:20Things does not need to hate you in order to destroy you. Influenza does not hate you. It doesn't have any intentions at all, but it can still kill you. AI could theoretically be no different. The frightening scenario doesn't require some machine waking up one day, deciding that it hates humanity and it needs to destroy us. Many corporations have destroyed communities, not because they hate the community, but because they're chasing profit. Now that does not mean that what they are doing is moral. Adistrophic failure, AI destroying humanity does not require malice. And perhaps just maybe indifference combined with extraordinary power is more dangerous than hatred. Now this leads to a very interesting question, what should we let AI do? Which human capacities should we refuse to surrender? Not because AI can't do them or is incapable of doing them, but perhaps because it isn't

11:21right for us to give them away. It isn't wise for us to give them away. Forget about AI destroying humanity for a moment. Should we let AI do our thinking, our writing, our remembering? Should we let it become our companion and fill that psychological need for companionship? Sure it can. AI can do that and it has proven that already, but should we let it? Is it the wise thing to do? And maybe the question isn't whether AI should help us think right, remember or make decisions. It already does that. The more interesting question is at what point does assistance become abdication? You can outsource a task. You cannot outsource responsibility for that task. And we already see that tendency in human beings today. Have you heard someone say, well, AI told me to do it or AI told me that's true or AI typed up that draft email for me just because AI does something for you that does not mean

12:24that you do not have the responsibility, the moral responsibility of that thing. So the question isn't simply is AI good or bad. The question is with AI here, what should remain ours? The answer stoically is judgment, character, responsibility, choice. Those still have to belong to us. I don't know if AI is going to kill all of humanity, nor does anyone else to be completely fair. Now is there a chance that that will happen? Yes, sure, of course. But that's not the important question for me here. The important question for me is, are we taking the underlying problem seriously? We don't need certainty about the future for surely there is no certainty to act appropriately in the present. Human beings have a long history of acquiring power first and learning the responsibility that comes with it second. And usually somewhere in between there is quite a bit of pain and suffering.

13:24Maybe the challenge with AI is whether we can finally reverse that order. Then we chase power with humility. Imagine if we could figure out how to be wise without first going through the pain of our mistakes, without attaining the power, without having it almost destroy us, and then somehow miraculously surviving. Can we develop the wisdom to wield something before we have to learn through catastrophic failure, what happens when we don't? Because maybe that's the real danger here. Not that artificial intelligence becomes too intelligent, but that our power finally develops faster than our wisdom can catch up. Thanks for listening. Hit the like button, hit the subscribe button, check out the StrongStort podcast, wherever you listen to your podcast. My writing is available on Substack. And let me know in the comments below. Do you think I am being a little bit too harsh on this AI guy? Do you think that there may be a need for some wisdom here? Or are you not worried about AI at all? Let me know in the comments below. Until next time, stay strong, stay stoic.