Srsly Risky Biz: Bring on the AI lawsuits

0:00Hey everyone and welcome to another episode of seriously risky business. The podcast we do here at RiskyBiz HQ where we chat with Tom Yurenne all about what he's written about for us this week. Can I Tom how are you? I'm good Pat, how are you? Good. And yeah, Tom writes the weekly seriously risky business news later if you're not subscribed to it head over to Risky.biz and you will find subscription links there. The idea behind seriously risky biz is, you know, it's our newsletters that focuses on policy and government stuff which is why it's a very serious business and you know, hence the name. Tom of course worked with the Australian government for a long time before moving on to ASP which is the Australian Strategic Policy Institute and has worked with us here at RiskyBiz for about full time for about five years now, right? Goodness. Goodness gracious me. Yes. Time flies when you have a good time as they say. But this edition of seriously risky biz is brought to you by Spectrops who of course

1:04make the bloodhound attack path a numeration utility or tool or platform whatever you want to call it. It's wicked awesome. It's what I call it. So big thanks to them for sponsoring this week's this week's edition of the newsletter and the podcast. So we're going to be chatting about two things that you've analysed and written about this week. Tom. The first is something that we did cover briefly on the weekly show yesterday which was some comments from Scott Besson around AI labs getting some sort of freedom from liability right when their products go haywire. So it seems like they want to be able to have endless industrial accidents kind of along the lines of what happened with hugging face but they don't really want to have any of the responsibility that comes with that. Like these AI labs have been making noises about hey you know we're making this technology to drive the economy forward if you don't want to slow us down maybe we could use some liability shielding here but his response to his credit thus far has been low LM AO even

2:09which is I think the appropriate response and you agree. Yeah, yeah. So I might take on kind of reading between the lines is that there have been numerous calls from the AI labs themselves for increased regulation and my take has always been that they want someone else to tell them what to do. So they when they when something goes wrong they can then say well we were following regulations we can't be held liable for stuff that someone else told us the government told us to do and that that's to me the underlying subtext where this is coming from and if you were an AI lab I think or the CEO of an E CEO of an AI lab it actually it absolutely makes sense this is something you would at least float and you know if we could get that that would be wonderful. I also would like liability from everything I do in my life it's just that I don't think it's reasonable to ask for it. They're in a slightly different position and I think you mean you would like no liability exactly. Exactly not liability for everything which is sadly the truth but I mean this is the thing

3:12right like you look at so much of the discussion around this I think has been I don't know just has missed the point a little like when we saw the early stages of the hugging face thing everyone's like oh but you know are we going to charge the text generators with CFA violations it's like no because their machines and they can't commit crimes right like the the metaphor I would use is like if I build a machine in my backyard that suddenly goes crazy and starts rolling through my neighbors you know backyards and knocking over their houses and stuff I'm going to be liable for that like as I as I should be you know but unless I've designed that thing to specifically go and do that like there's probably not much of a criminal liability there so I mean I you know we just seem to be landing in a sensible place where you know the the Trump admin is indicating or signaling that it's not going to give these AI labs cart blanche to just do anything they want like you know

4:12in light of incidents like the hugging face and whatever that happens again you're going to be liable for them I think is what they're saying yeah yeah I think it I think they're still genuine questions about if you're using a model for your business purposes and something unexpected happens who's liable there I think the sensible answer is well if if you the company who is using it as a third party hasn't thought about it and and got sensible protections then maybe you should be liable well then there's then there's all of the ullars are basically which is you know even with not non-AI software they give the software company permission to put the user through a woodchipper if it suits their business purposes right so yeah so I think there's a discussion to be heard about that kind of extending where exactly who is responsible for what when I think it feels to me like an negligence discussion like you know these things make mistakes but see this is my point time is I feel like you know we've got a system for determining where the liability sits in all of this and it's the court system you know what I mean and this is what bersent saying as well like this is

5:14exactly what he's saying which is like we're not getting involved here like if you're going to keep having industrial accidents and that's that's the term that I use instead of their you from their corpoh euphemism of like alignment issues right which they love to say because it sounds softer than you know industrial accident if they get to keep having these industrial accidents eventually someone's going to sue them like in the case of hugging face it looks like they were able to just shake down open AI for a bunch of like free tokens and that kind of put it to bed um but you know that's not going to be every company's response I also think the other part of this story is that when you look at all those incidents it was clearly not well maybe not well thought out not well implemented the controls they had and so the particular example I looked at is the hugging face incident where the security team at open AI noticed that something was wrong um and then they went oh something's wrong let's just rebuild it let's just rebuild it exactly the way it was uh and uh let's not investigate

6:14our hacking machine and see what's going on let's just yeah move on move on nothing to see and so you know from small things big things grow so that turned into that was before it had hacked hugging face yeah and so there was definitely an opportunity for them to revisit their controls and go oh hang on something's unexpected is happening what's going on let's figure out what that is I think that would be demonstrating a kind of good security culture as opposed to well not yeah yeah indeed um so the upshot there is nice try guys but you're going to have to be a little bit more careful in future which is good I think this is a good result so let's now talk about the second thing that you have worked on which is I think it was last week or maybe even the week before you wrote about how uh Chinese APT group was using AI to muddy attribution uh and now you've looked at how a Russian APT group is using AI but with different objectives right so

7:19they're going for speed to evade immediate detection but they don't seem to care as much about evading attribution so what's interesting is you've sort of brought these two things together and compared how two different threat actors are using AI to tick very different boxes it's an interesting read yeah yeah so the Chinese approach uh when I read about it it was like yeah this is really sensible I can see someone like NSA or ASD using a similar approach and it was really to speed development develop a whole lot of different varieties and it serves a long term purpose which is to enable a long term collection campaign to keep going and uh I guess all of three weeks ago back then I would have gone yeah this is the way for state actors to do it and then this report comes out from Anthropic and they talk about a Russian group which is really using it AI in a very very tactical way it's like our malware has been detected let's have a workflow an entirely AI workflow to regegry it so it doesn't get detected anymore and it's just let's move on

8:24we don't care about attribution because it's Russia and the targets are often Ukrainian where you know if you're well you make you make the point here that no one's doing incident response in Ukraine it's not like a stone called who done it you know what it's like like at some target of interest to uh to the Russians like you know they're not they're not thinking they've been hacked by the Dutch yeah and it was also really interesting that the they talked about the entire operational workflow basically being managed by AI and the people were not managing the operation they were managing basically the Claude skills so they were a many Claude skills to get them to work better so it was managing the machine that runs the operation rather than managing the operation and this struck me as this makes perfect sense when you don't care about any particular operation if you just want scale and bang bang bang bang bang let's go and go and go and lots of targets of opportunity that's

9:27perfectly fine now for many state actors that makes no sense at all because getting detected is a problem but but in this particular case it is like the totally makes sense and so I guess it comes down to horses for courses this also this approach also makes sense I think for a cybercrime actor where there's plenty of fish in the sea you don't care about a particular target you care about getting or at least some of them care about getting lots of targets yeah I mean it's funny right that what you're alluding to there about certain threat actors not wanting to be you know attributed and stuff like this is the sort of mantra of the five eyes agencies yeah they've been an outlier in that regard for quite some time and I sort of feel like with the adoption of AI by all in sundry by all of the West's enemies there even more of an outlier now and I just wonder how tenable that approach is I mean I think we're eventually going to have to have like tiered operation I mean we already do have tiered operations but I think if you want to scale up and keep up with China and

10:31Russia you're going to have to do these sort of shenanigans and you're going to have to get snapped you're going to have to get attributed you're going to have to deal with China you know getting sure to hear about about you know NSA shells and whatever I think it's just going to become part of the game in the future but I mean now I'm not talking about what's in your newsletter I'm talking about stuff that's possibly a couple years out but I'm curious to know what you think about that idea yeah I think again I think it's horses for courses like I think for a particular target say I don't know pick one out of a hat she Jim Hing I think you want to be careful around that because the defenses will be so tight that your opportunities will be very little yeah you're probably not using repurposed dark web leaked exploit kits on she's Hing P you know not no dark sort of corona are off off a torrent for him yeah and I and I think it really comes down to what you think your intelligence agencies should do so the Western intelligence agencies tend to be more focused on what we would describe as equates legitimate targets which is kind of narrow in scope well and it's

11:37not it's not even about legitimacy per se it's that these operations tend to be scoped around pretty tight objectives to begin with yeah that's not how the Chinese do it's like you know you need this piece of information to support this this program or this operation you go out and get it whereas the Chinese are just like lol that database looks nice thank you yeah and that's opportunistic approach or that you know suck it and see approach collected all and sorted out later I think that lends itself to the well we don't care about attribution particularly because you know we we pick up this piece of information that's great if we happen to miss out for whatever reason well you know it's no great loss so I mean look to tie this back to the piece that we're actually talking about that you've written you know it does feel like we're seeing meaningful shifts in TTPs like we're seeing we're seeing some pretty substantial changes thanks to AI and I'm expecting

12:37that to continue quite a lot I mean that that if I had to pull a key takeaway from what you've written I think that would be it yeah I'm interested in it seems like these are complementary approaches that are right now mutually exclusive because you can't really rely on AI to be perfect I kind of wonder what happens as AI gets better less hallucinations less what I'd call a mistake and can you combine these two approaches like have a variety of malware that is also basically runs sort of semi-autonomously with oversight rather than hands-on keyboards I think that that's the that would be interesting well I mean I think I think the reason and this is very like very much what you've written about I mean they've clearly designed these campaigns to fulfill their each of their objectives right so one of them is about sort of stealth or avoiding detection in the moment one of them is about dodging clustering attribution things like that I don't think there's any inherent limitation in AI tooling that would prevent you from being able to design a

13:39campaign that did both like it's just that they don't need to cover the other side of each of those coins you know what I mean right right I guess it the way I thought about it is you've got a limited amount of like you've actually got a limited amount of people doing stuff and you can get them to use the AI to do one thing or the other thing but the best thing about AI is it gives you scale that's what I mean like I think this is where we're starting I guess is my point right I say no reason why you can't have you know an APT machine that just craps out basically limitless malware variants that are different enough that they're hard to cluster together you know I just I just think I just think I just think we could do this like we could see this yeah I think we can see it too I think right right now the problem is there's just a bit of lack of trust and I think they're using not the best of the best models right oh well Tom you're in thank you very much for joining me to walk walk through your work this week are fascinating stuff as always

14:40really enjoyed it and yeah we'll be chatting again real soon thanks Pat