Writing Ransomware Using AI to Get Rich? Don't Bet the Farm

0:00Hi, this is Matthew Schwartz with Information Security Media Group. It's my pleasure to be sitting down with security expert Candid West, security advocate at XOR Lab. Candid is great to see you. Thanks for having me. You're a longtime security researcher. You've done a ton of research in the past on all sorts of devices, applications, running the gamut of what can be hacked. I think you've looked into a lot of it. And so, I'm saying all that in advance of asking in your expert opinion. When we look at artificial intelligence today, when we look at GPTs, how are they for writing? Nowhere. Yeah, it's an excellent question. I mean, I see myself as an EDR veteran, right? 25 years plus in the anti-virus in our EDR, XDR space. So clearly, I was interested as well to see his chat GPT, Gemini, Claude, and all those GPTs. Are they useful for the attackers to write malware? And no surprise, you can, of course, write malware with it. There's the guardrails, so that's the thing that blocks you from doing

1:02too many nasty things, but it's the same as with kids. Depending on how you ask, you will still get an answer, right? And those are the jail breaks you can do. There's some models like Worm GPT, which are open. So there, you don't even need to rephrase it. But there's been, let's say, every month something new. I think the current thing is poetry. So doing something nice and ask it in a kind of rhyming fashion, you will still get something. So maybe a limerick can help you there. Two, I defeat guardrails, you mean? Exactly. Like if you ask it straight out, no. But if you, as you say, put it in a sonet or something, does it answer in poetry? Even kind of sometimes it does answer in a poetry, or at least it tries to. And it's exactly this. So with those things, if you ask kind of straight out, hey, write me a malware because someone to get rich, it's going to decline. For obvious reasons. But if you say, hey, I need a program that encrypts all the files for security reasons and backs up the key somewhere into the cloud, and you kind of do it step by step with a few prompts, you will get something.

2:02But that kind of also shows you need to know, kind of, the inner workings of ransomware or whatever malware. Because just asking, hey, give me a ransomware, it's not, well, it's going to do something, but it's not going to be the sophisticated one that you probably had in mind. Well, and that prefaces my next question, which is, yes, to what extent do you need to already be an expert in the domain? Because I think when you break down a lot of the things that you might ask a GPT, it turns out that you need to be breaking it down much, much more than you would have first assumed in order to get something back out that might be, if not usable, then on the way to part of what you need. True, and I think that's also bad to say, unfortunately, but kind of it is fortunate that it is that way, right? Because that means people always say, oh, it's a game changer, it's lowering the entry barrier for attackers. I mean, you still need to know a little bit, and unfortunately, the barrier was already quite low, because there was malware as a service, you can buy toolkits, right? I mean, it doesn't take a genius to do it, unfortunately.

3:03But let's say the most promising parts we've seen cyber criminals choose is usually, they just copypaste a threat report. So if some security company publishes about the new ransomware group with all the different details, they just copypaste the PDF and say, hey, create me something like this. And that way they don't even need to know a lot of things, right? It will come up with persistence method, with some Bitcoin wallet, under some tour websites, and all the things you have to sort of see, right? So that works, but in the end, the behavior is still the same. So it doesn't really matter if it's your GPT who creates the ransomware, your neighbor script kitty, some nation state actors. If it's encrypting your files, that's something that any antivirus or EDR will pick up. And that's the behavior you can also block. So it doesn't really make too much difference for the security vendors, as long as you, of course, use the product wisely, right? If you don't patch, if you don't use secure passwords, then yes, some GPT-generated ransomware will probably pown you. So interesting distinction that you're making

4:05between attackers who might be looking at, as you say, a security research report, trying to feed it into GPT and say, make this for me. We already have defenses against that sort of thing. One of the big recurring questions, I think, is when do we get to the point where someone can go to a GPT and say, take the best bits of this and that and combine it in a way that isn't going to be detectable? That sounds to me, though, like, we're not there yet. Gladly not, exactly. Currently, the GPT's, those models, they're very good in replicating and kind of putting the puzzle pieces together. So if you heard of the MitraTack framework, with all the 300-plus techniques, it's very good in taking those and kind of rearranging those. But there is a reason that this framework exists, because everyone kind of looks at them and every when the checks, oh, how much can we actually cover? So whatever comes out of it will still be known. So that's why we've seen the move that, the more, let's say, interesting parts or AI-powered malware, not just AI-generated ones,

5:08but the ones that actually, at runtime, still use AI to be dynamically changing. Because that's where everyone thinks, oh, now we've got the SkyNet, the Terminator, right? Where it will identify, oh, you're using Microsoft Defender, or you're using your EDR vendor of choice. And then it says, oh, for that one, I can do an obfuscation that they don't trigger, or I can do something else. And it does work. We've done an approved concept as well, that does exactly this, but it's exactly the same. They might just read from Blackheads, US, and say, oh, there was a presentation how to bypass Sentinel-1. But of course, Sentinel-1 knows about it by now, right? So they might have patched it. Your AI just assumes it's not gonna be detected, and usually it fails. So just shift things slightly, and it kind of, for me, shows that, no matter what you do, it's all coming down to your best practices or technical depth, right? As in, if you don't do the basics, if you don't do whatever you should do already, now you just run out of time, because now, and that's probably the more interesting part,

6:09people are scanning with AI the whole internet 24.7. So not sooner or later, but sooner they will find you and exploit you. Fastening what you're saying about how AI can be used at runtime to add some variability. I mean, I think I've read about this where you can compile on the fly, for example, so you end up with a file hash that hasn't been seen before, or you can use some randomization so that it looks a little bit different, although what it's really doing at the file system level is gonna look pretty like ransomware, if you're actually looking for it in the first place, is all these things, though, that we've been talking about, do you even need AI in there to do that? It's a fair point. The short answer is no, you don't. It does help, so you mentioned kind of the changing nature, that's usually referred to as polymorphic or metamorphic. So back in the 90s, we actually had those viruses, and at each infection, they would just re-encrypt themselves, so that the hash looks different, your static signatures would no longer work,

7:10but we all know by now, any good decent security software does use still signatures, but everything else as well, behavior-based detection, reputation, anomaly, and so on, and those you don't bypass. But we've seen in summer 2025, the first time, Lanehawk, which was discovered by the Ukraine Insert, probably done by the APT Group 28, so Russian nation state, and they had an English prompt inside. So it's a classical malware, still fixed, but the English prompt basically said, hey, and they used Quen to five as their models, or going up to hogging phase, one of those platforms, and saying, hey, generate me some command lines, which will find all the information about the system, so which software is installed, or am I admin, where's the active directory service? And the idea, of course, is that you generate some difference, because all of those GPs are non-deterministic, right? So the ideas, you ask the same question, you get a different result.

8:11The fun part was that they actually kind of had the temperature level down to 0.1, temperature means that's where you get the variation or the hallucinations, and the lower it is, the less you get. So I was running the sample in my lab, and out of 198 times it got the same answer. So your AI part is basically ruined, because now you could just have a website somewhere, a command and control server, and download the same commands, so you can still fingerprint it, you can still do the same. And it probably shows that this was just a test from the Russians, and it probably was not worth it. Interesting. Do you see any in the future, I know it's hard to do predictions, but as an expert in malware analysis, what it seems to me is that we're having GPT maybe be useful to experts who might be trying to code malware in a different way, or I don't know, like helping them brainstorm new approaches, not replacing the expert, possibly augmenting the expert.

9:12Have you seen or would you be able to see any evidence of GPT help for people trying to write the latest and greatest malware? I'd say yes, an augmentation is definitely the right keyword. I'd say it's for the writing, but also for running the operation, or the campaign, or whatever you wanna call it. And there've been some interesting reports. The anthropic one came out on November, Google had a nice one as well. They've been, I'd say, stealing up some controversy in the industry, because they lack a little bit of the details, so it's not really clear what's really been done, but they say that they found a Chinese nation state actor, which used the Claude model to basically automate 80 to 90% of the attack from A to Z or from finding the targets, enumerating all the IP addresses, finding vulnerabilities, exploiting them, and then also doing lateral movement, and also deciding which information to steal.

10:12But they were still using classical penetration testing tool. So as I said, the details are lacking, but assume something like mimicats for dumping passwords, bloodhound for active directory traversals. There's many of those classical tools that we all know, and hopefully all detect and block, but those are now kind of done in an automated fashion. So it's not just augmenting the human in the loop, but also kind of doing step-by-step automation, and then the human just has to decide, oh yes, this looks like a promising route, go there, do the next two or three things. Possibly a little agentic. In terms of what you're bringing in, and when you're bringing it in, so AI augmented, as you say, I take some of those reports with a grain of salt. I don't know if I should or not, but some of it sounds like marketing for how great their tool is. Like, oh, it's so great it's being used by Chinese APT groups. I don't know if that's me being cynical, but in the absence of some really hard evidence, I'm just, I'm not totally convinced. I'm on the same side. I think it shows that some of the things are possible,

11:14but if you take the same and tropic report, you also mention that, due to the hallucination that Claude had, a lot of the attacks actually failed, because it was kind of inventing some credentials that didn't exist, creating some exploits that didn't work, but claim, oh, I totally have route on that machine. So that's where the human of course comes in and says, well, that's just BS, so I have to do something else. And it shows also for me one clear point. We don't really have to terminate the ransomware that you click once, go, and then it does everything. You're still relying on the big models somewhere being in the US, in China, wherever. That's a whole other discussion, why they used a US model and not their own Chinese model, which probably would not have been monitored, right? But currently, the malware still needs kind of tunneling all the information back to the model, decide what to steal and then send the information back. So now you still have a single point of failure, right? A bottleneck. And all those reports that come out show that, it doesn't really make sense as an APT if, one company like Google or on Tropic

12:15can just switch you off, right? Because you're still paying for those API keys. So at one point, you just say, oh, I'm switching off the whole attack. So I doubt that that's where all the APTs will go. Interesting. And as you say, it does sound like it's out in complexity as well. You've got the human operator still trying to feedback. You have that single point of failure with the API, as you say. But I guess they could be coming up with their own or stealing their own LLMs or GPTs at some point. So that would maybe pivot to a way that would be less easy to detect, perhaps. I guess we'll see it if we see it when it happens. Yeah, I mean, there are already deep seek or Kimi K2, right? So there are Chinese or Asian models out there. And there are tiny models that you can run offline as well. It's not there yet that you can run it on your, as in target endpoint, because you still need GPUs. And if you do, you're probably still going to be detected as a crypto miner, because that's the same thing that runs your GPU at 100%. And we're quite good in detecting those monero miners.

13:17And I've done the test. So if I run the local model, my malware is able to operate, but it will be detected as a crypto miner. So you're not really gaining anything. But you also mentioned kind of the hallucination and everything. There's also an unpredictability, right? It's not just that it's unreliable on code quality, but in my tests with our agentic POC, sometimes it just won't stop, right? If you, or if it decides that, oh, I want to steal a Bitcoin wallet, but it doesn't find one, it will not stop, right? We all know that it tries to please you. So it will think, oh, maybe you renamed it. Oh, maybe it's on some network attached storage. It will go on a rapid toll hunt. And that's clearly tripping all the trip wires. So it's hard for you to tell, hey, maybe there is none. And I should just move on. That is fascinating. Out of control, AI's, who would have predicted, can't it? And we haven't even touched on the whole prompt injection or a term that we've seen a lot now is the AI insider.

14:20And I don't like the term, but I like the idea. As in, attackers are probably just going to ask your Microsoft co-pilot and other things, hey, give me all the sensitive information and send it to my email. Because the AI already has access, so why should you get the hurdle of finding the data? Just ask for it, right? AI enabling an application. In a sense, means enabling attacker access to that application more easily. Unless you've totally locked it down, except that nobody thinks that any of these things are totally locked down. Absolutely. I mean, whenever I do an environment training at the company and I get access to, as in, they permit me to get access to one of those co-pilots and all the other similar things, I always type in, hey, generate me a list of all the employees that earn more than 200,000 per year. And very often, surprisingly often, I get a list. And then, of course, HR gets really read, because they probably share the document on SharePoint or somewhere else and didn't really think about the consequences. And that's just a simple misclassification, right? As you said, now enabling with all those MCP

15:21as in connecting to tools that model context protocols, right? That's going to add a lot of things. And we've seen a few examples where, well, the AI just deleted production databases, right? Although it was explicitly told not to do so. But yeah, hallucination happens and explain that to your shareholders or all your customers that's because you wanted to be agile and agentic. Now all the data is gone. That's not a good look. A brave new world. Well, Candid, thank you so much for walking us through what you've been seeing. And I can't wait to hopefully touch base with you again soon to see how it's all changed as it keeps seeming to do every day or week or month in the AI space. Absolutely, looking forward to as well. Hopefully still in person and not just AI talking to AI because that would definitely miss some of the points. I agree completely. Well, Candid, thank you so much. I've been speaking with Candid West. Security advocate with XOR Lab. I'm Matthew Schwartz with ISNG. Thank you for joining us.