AI poses biosecurity risk, and the ice that doesn't shatter

0:00I'm Jake Stauk, co-founder and CEO of Serval. We built Serval to automate the IT work that slows companies down. Onboarding, password resets, access to applications, my laptop stopped working, while employees wait for help their real work is put on hold. IT desperately wants to automate this work, and that's why they need Serval. You just tell Serval what you want to automate in plain English, and it's built. No drag and drop workflows, no expensive consultants. These get unblocked and IT teams go from drowning in tickets to building what actually matters. With Serval IT becomes the AI engine powering the entire company. This is a new way to run IT. We guarantee you'll automate 50% of all tickets, and we'll prove it to you in a free four-week pilot. Go to Serval.com slash tickets. That's S-E-R-V-A-L dot com slash tickets. All engine running.

1:03Get this. Welcome. Welcome. Welcome. Welcome. This is the show where we bring science. What that essentially means is discovery of the most advanced research technology unbelievable. Without further ado, this is the Naked Scientist. Hello, welcome to the Naked Scientist podcast. The show that brings you the biggest breakthroughs and talks to the major movers and shakers in the world of science, technology and medicine. With me, Chris Smith and coming up, high risk or hyperbole, lawmakers clamber to prevent AI from being used to develop biological weapons. We hear from scientists who've brought a second world war building concept in from the cold, why ice might be the construction material of tomorrow they're saying, and how researchers are tracking weather patterns on far away planets to look for life. Lawmakers worldwide are considering emergency legislation to address concerns about AI's rapid

2:10advancement. Now, this comes as tech giant open AI said it will support a raft of proposed bills in the United States that could help to prevent artificial intelligence models from supercharging the threat posed by biological weapons and synthetic viruses. The company told the new site Politico that AI poses a grave threat to biological security with senior figures at rival AI firm and thropic also recently warning that AI could soon be able to engineer dangerous pathogens. Gemma Bouscher is an expert on global health security at the Centre for Conflict and Health Research at King's College London. What most people are concerned about are the use of these technologies to make pathogens so viruses or bacteria or even fungi more dangerous to humans. And that can happen in a number of ways. Either it can make them more easy to spread so you can take a fairly benign pathogen but make it more infective so infects more people from sort of bank for buck, you might say.

3:11It could also take a pathogen and make it more lethal. So it might be a question of taking a single pathogen that infects an individual but makes it kill them more quickly or make the symptoms or signs that they experience more significant and more dangerous. And the questions that we're asking now around many of these technologies is how the technology actually can be targeted and focus to create these conditions on demand as it were so that you can essentially pick up your menu of attributes in a virus or a bacteria and say I want it to do this and I want it to be more powerful, more potent, more lethal, more transmissible. What does AI bring to the table that we can't do already? Because people are already manipulating organisms 15 years ago, researchers reckoned that they could turn flu into a pretty nasty flu that would transmit very readily. They even published how to do it, although it did get some reductions a bit later on for safety reasons. So we already sort of know how to do some of these things. So where does AI come into it?

4:11So first is really around reducing the level of expertise required to do these kinds of manipulation. So if we think about some of the chatbot style models that lots of us use on a daily basis, the concern isn't so much that they hold any particular secret information. It's more that they make it more accessible for anyone on the street perhaps to be walked and talk through the process of manipulating pathogens. Right? The second is speed. We already have the grand designer of virulent pathogens and that's nature. Nature has been doing this for thousands of years. But the concern now is that these tools can take all of the learning and the published literature that we already have and move at speed to focus in on the more dangerous attributes that may cause issues of concern in the biosecurity space. Is there evidence that this is happening? This is really where the question around some of the tech giants becoming involved as it has come up because we can't really document any specific real threat in this space. But certainly these actors are concerned because they've run their own tests.

5:15So what these tests did show is that some of the models did trigger safeguards that have been introduced. So we can see that the risks that people are talking about are potentially possible and that the internal tests of the systems having some ways been failed in a sense that they did trigger issues. Now we've not seen AI engineered pathogens of concern anywhere in any attributable event. It remains a kind of a speculative risk. I said we have seen these companies say that they have put the kibosh on some inquiries coming through from certain sources which their network has caught and they've stopped it. I mean obviously that begs the question, well what ones didn't they catch or that's a group of companies being transparent. What about the ones that are not being transparent? It's the fact that it does appear to be happening then that's got people worried. And I think that's the question with any of these novel and evolving technologies. And I think there's two perspectives you can have on this.

6:15I think the first is that yes, in some ways we can see this as a very positive step in transparency from these AI companies. You know, they've set up these benchmarks, they've set up these protocols, this process of red teaming, it's called as we test these models in this way and they've published it right. And it shows that some of these disgust risks are real and that's important because as we've seen we live in a world where all of the complexity of biological events has caught the attention of the media because of the pandemics and continuous outbreaks that we continue to live through. I mean there's also another cynical perspective which is of my own which is that we can measure these risks in biological space in the context of AI tools. In fact, there's a lot of discussion at the moment around the regulation of these AI companies and perhaps they might see biological risks as a safer territory to engage in some of these conversations in because very specific proposals for regulation can be put forward, rather than perhaps some of the more diffuse catastrophic global risks that are being discussed around

7:16really the legitimacy of the technology on a wholesale basis. Suppose one of the tricky things though is that very often scientists need to do this kind of thing for all the right reasons. So if we think about flu, you might want to ask very legitimate questions about what could flu evolve into to become much more virulent or to make people much iller and what would I have to do to counter that? Now they're perfectly legitimate medical questions, medical inquiries and the kind of thing we'd like a virologist to be worrying about to make sure that we're not going to get taken down by another pandemic tomorrow. But at the same time if we block the sort of nefarious work that could involve many of the same approaches, we prevent that good work in the course of blocking the bad. Absolutely and this I think is one of the fundamental challenges in biological research and in its kind of partner area of biosecurity because at its core this kind of work is dual use. We refer to it as dual use. It is the balance between the promise and the peril.

8:19They're a completely legitimate and important types of research that we could be doing. You mentioned the work from 15 years ago. We saw the construction of a essentially a flu virus from secret state to show that it was possible. That's a scientific advance of significance both to humanity and to the medical sciences and many of the advances that we rely on in cancer, in infectious diseases, in vaccine technologies, rely on working with very dangerous pathogens and understanding what makes them more potent and what makes them less. If we don't do that kind of work, we will not be able to benefit human health of course. The petrile concern is around misuse and malign actors. That could be at the level of an individual and that's really where some of this AI work might be a problem where individuals are able to do more and more and more with less and less and less or at the level of states. We know that there are states that have had historically well established biological weapons programs and there is a whole convention that deals with this issue.

9:21One of the areas that I am concerned about is how we match AI and the regulatory powers of the convention and new regimes that might apply to manage research in a way that balances promise and peril. Jim Abousha at the Centre for Conflict and Health Research on the debate about the bio-weapon risk posed by emerging AI technologies. To cancer now and a new study has found that chemotherapy in childhood can leave lasting genetic scars on the healthy tissues of kids treated for cancer with certain platinum-based chemo drugs. These genetic changes effectively age the affected organs by the equivalent of many years making new diseases and especially new cancers that much more likely in the future. The findings which have been published in the journal Science could help to explain some of the long term health problems that children who survive cancer face later in life. Foward Rahani at Kings College Hospital and the Francis Crick Institute was one of the

10:22senior authors on the study. You've known for a long time through clinical experience that children who have had previous cancers who then go on and survive can have medical problems further down the line many years later. The secondary cancers for example that arise or diseases related to the liver. And so the question really we were trying to ask with this work was can we somehow understand what happens at the time of the initial treatment which may in some way explain what then happens years down the line. Are you sort of saying that the treatment itself carries a risk? Yes we've known for a long time that chemotherapy which is the backbone of many treatments for a range of cancers we already know that chemotherapy causes a level of damage to background healthy tissues around the body but we didn't really have a nighter of quite how extensive this damage was up until now.

11:22What's the mechanism of that damage? Is it that it's just flogging lots of cells and making them grow very fast to repair damage that we're doing with the chemotherapy or is it actually damaging the cells genetically like it is the cancer? With this work specifically we were focusing on the liver and the way that the chemotherapy that we focused on platinum containing chemotherapy works is exactly that it damages DNA and that's really how it's effective against cancers but what we've shown with this paper and what we sought to clarify with our work was whether the type of damage that we were seeing in the background tissue was exactly the same genetic changes that the chemotherapy is good at against the cancer. How have you done it? We had an experimental design where we took liver tissue from children who had a rare type of liver cancer called a hepatoblastoma and we were taking lots of different biopsy and little samples of the liver, of a number of children and then applying some quite

12:25sensitive DNA sequencing technologies to this. What are you doing comparing the genetic sequence of the liver tissue before and after the treatment for the cancer? In a lot of these cases we only have the sequence of the DNA what for the tissue comes out but what we're comparing this to was for example other tissue such as blood from the same patient from the same child. What you're asking have we caused any new damage to the DNA in these individuals in the course of trying to treat their cancer? Exactly. So one of the techniques that we apply in these sort of scenarios is to try and detect patterns of changes within DNA. Mutations, the change of the DNA and mutational signatures are these patterns. You know from other work that lots of people have done that certain types of damage such as YouTube tobacco from smoking or ultraviolet light causes characteristic patterns of mutation.

13:26It's not totally random. The changes in the DNA are characteristic and hence by looking at the DNA in the changes we can then link to a potential cause. And so what we were looking at with the tissue that we were taking was to see whether we could see a pattern which was consistent with the platinum chemotherapy that the children had been treated with. The field already knew of signatures or these patterns of DNA changes associated with platinum. So we not only saw those but the interesting thing was we saw a completely different type of signature which was only present in the children who had had been exposed to platinum chemotherapy for one and secondly the new signature that we'd found associated with platinum was only seen in liver cells and not in other tissues. And what are the implications of that then? Have we effectively artificially aged these individuals in the sense that what we've done is is add damage that is like a notch further towards a disease in weighting in these

14:29individuals? Yes, we know from lots of other work that DNA damage or mutations are associated with age and they arise over time. And so what we were able to do in this study was to compare the amount of mutations or DNA changes in the livers of these children and compare that to adults. And to our surprise we saw that the ones who had a chemotherapy, the damage that actually aged the livers of the children to approximately the same as that of an adult. Does this happen in adults as well then? So if we use the same drugs in say ovarian cancer cases or something to those people succumb to the same accelerated aging so an adult ends up with a liver of a really elderly person. It's a very good question that wasn't something we looked at in this study but I think it does follow the agents which can damage DNA such as chemotherapy would do this in an adult as well as the child. And what are the implications of that then? How you found that in the liver and shown that these children appear to have much older

15:33livers than they would have based on their chronological age, what does that mean for them? What we really uncovered with this work is something which was there all along but was invisible before some of the latest sort of sequencing technologies. And so really our work has highlighted that these patients do need continue follow up that may also be opportunities to perhaps design different types of chemotherapies for these sorts of patients. An example perhaps of the pill sometimes being as bad as the ill, fascinating findings there. Third Rihane at King's College Hospital and the Francis Crick Institute. Frustrated with spending hours on AI outputs that don't feel like your brand, active intelligence 2.8 from active campaign keeps you from sounding like everyone else. Describe your tone, your rules and your brand once. An active intelligence builds every email and campaign for you with one brand images and

16:33content that sounds like you wrote it. Marketers are saving 10 hours every week and getting campaigns they're proud to put their name on. Import your brand in seconds at activecampaign.com slash AI. The naked scientist podcast is produced in association with Spitfire. Most effective voice internet and IP engineering services for UK businesses. Find out how Spitfire can empower your company at Spitfire.co.uk. This is the naked scientist podcast with me Chris Smith. Now we're halfway through today's show which gives me an opportunity to tell you about our sister program. Ask the naked scientist each week we endeavor to answer your questions, the things that have been keeping you up at night and you can find that program on our website or wherever you get your podcast. Just search up, ask the naked scientist no two programs at every like. Still to come on this program has scientists attracting weather patterns on distant planets.

17:38Before that though, ice is plentiful, inexpensive and surprisingly strong. In fact it can make an excellent building material under certain situations. But for one problem, placed under significant load cracks can readily spread through it causing it to suddenly shatter and fail with obvious consequences. Historically engineers sought to fix this by mixing substances like wood pulp that contains cellulose into the forming ice and that has the effect of binding the crystals together and interrupting the propagation of cracks, making ice that is incredibly strong in fact half way towards the resilience of concrete. But back in the day, this was dubbed Pichrete and in World War II, a plan was even hatched in London to use it to build a submarine resistant aircraft carrier to patrol the Atlantic. Now those plans were shelved at the time but Pichrete technology remains on the table as an attractive, environmentally friendly building option for cold places.

18:41And now scientists in Israel have discovered how to make it even harder and even stronger with a recipe called BioPichrete. This uses naturally occurring proteins to stitch the wood pulp material much more tightly to the ice crystals. It's the brainchild of Hebrew University of Jerusalem's Ido Prislavski. So what we have is a mixture of ice and cellulose which is basically chopped wood in a sense but on a small crystals. And what we add to that is a special double-side glue of two proteins, basically one stick to ice and one stick to a wood or cellulose. We investigated ice binding proteins for many years and we know the ability to stick to ice and hold it firmly. And we know about this mixture of ice and cellulose. So we thought maybe there's some application for that once we have a stronger mixture.

19:43I mean, flippantly, is this stronger house is rescue moes or is there more to hit than that? Well, it's more into that because this is not make of a snow which is packed and this is stronger material. Basically it's called a Pichrete. It's a program that was used many years ago in the Second World War with a sort that can use this kind of material to build ships. So there's some history for this material of mixture of ice and wood. So this kind of material is really much stronger than what you can use for igloo or so. So you can really build things from that. One of the inherent problems with ice is it has a sort of brittleness to it. If you push it too hard, it will actually propagate cracks and then just break apart. So does what you've got here, some mount that problem then because you've got these different things integrated into it that then bind it together?

20:46Exactly. So ice is quite strong but if you push it hard, it will shatter. And it was known that if you add to that additive such as wood, it will become stronger. So it's a kind of a ductile material and the energy to break it is much, much higher than the usual ice. So eventually what we got with all the composition is 70 times more energy to break that than regular ice. And it's just by how much you can push on that. It's 10 times stronger than ice. How do those proteins do that? So I'm envisaging a giant ice cube for one of the better phrase. I've got ice crystals. They've got wood material. That's the cellulose in there. And then these proteins that bind onto the ice, bind onto each other and bind onto the wood. But how are those proteins doing that? And how does that turn into this interesting materials characteristic that you've got with the extra strength? So we take two proteins from nature.

21:47One is from fish which live in cold environment. And there the proteins attach to small tiny ice, very firmly and not let them grow. So they're doing that by organizing water on their surface in this particular way which is very similar to ice. And then all of this kind of complex stick to ice very firmly. This is one side. The other side is a protein that is a part of a bacteria complex that the bacteria use it in order to hook to cellulose and then to digest it. So this actually comes from bacteria that like heat. So they are very stable protein. And there again they have some structure which have a very flat area with a lot of connection that can make to the connection to the cellulose itself. So now once we put two proteins together, one of them stick to ice, the other stick to

22:49the cellulose, now we have a kind of double side duct tape that can glue these two together. So why does that particular configuration, protein stuck to ice, protein then glued like the duct tape to the other protein, protein stuck to wood or wood component, why does that make the ice really strong? How does it stop the inherent physical problem you have with ice shattering for example and make it into this much stronger material? What's the physics of that? So there are two steps for that. The first step is that if you take cellulose and freeze cellulose solution and freeze it in a directional way, so you're cooling it for one way and then the ice growing one direction, it creates kind of a net of cellulose which make the ice between this kind of net. And now if some crack running instead of running without stopping, it's stopping this net.

23:51So this is without the protein, so it's already much stronger. But once you add this kind of extra glue, it's double the strength of the material. So to break further the crack to go further, it's a need more energy to move on. So altogether, so we have the structure of the cellulose ice and then on top of that, you make the connection between the cellulose and the ice stronger. And is that how you make it? You have a giant ice cube tray where you mix all the ingredients together and then freeze it. And it forms an ice cube with these things in. Or is there a special way you have to layer this to get the right three dimensional structure? So that works. What we use is a directional freezing. That means that instead of just putting the whole cube into the freezer, we are cooling from the bottom. So that means that we have a directionality for the growing of the ice. So it's grow from bottom to top.

24:53So that means that once the ice is growing, instead of slowing down, it's keep growing in a constant velocity, which keep the size of the crystals quite uniform. And then you have the whole structure quite uniform and organized. So it looks like a sheet of ice that grow up from the bottom to the top. What can you do with this then? I mean, I flippantly said, are we going to build better igloos for Esquimos earlier? But I mean, is there a realistic prospect that you could use this as a building material? I mean, you obviously got to think about temperature, but is that the aspiration with this? So the big advantage is in a remote cold place. And this could be an antarticle, arctic regime, which it's expensive to bring a concrete. And also a concrete to in order to harden, you need to keep it warm for a long time. So if you go to some places which you have a water in a cold, so this is basically free.

25:59So all what you need is to bring only about 3% of the cellulose and about 0.1 or 0.15% of protein. So it's tiny fraction of the mass you need to bring. And this is a huge advantage. And now you can build with that different constructions that you need. It could be part of a housing or road or bridge. For example, a frozen road, which you make it stronger. So this kind of construction abilities is feasible when the advantage is when it's remote and cold. It's an ingenious idea, isn't it? Edo Breslabsky there. Up into space now. And scientists say they can now track where the patterns found on far away planets, starting with a nearby brown dwarf. Now this is basically a massive body that's not quite big enough to have become a star and it's located about 20 light years away.

27:03Being relatively close and a heat source makes it a convenient study subject. And the team at Trinity College Dublin have been able to watch the patterns of thermal radiation coming from the object as it rotates and map those onto models of how we think atmosphere's clouds and storms work in general. It is an important step towards being able to read the weather on remote worlds, helping us to home in eventually on the ones that might be habitable. Murloshrada. We now on our vaguely what drives the weather. So what we've found is what drives weather on planets outside of our own solar system that aren't as resolved as our solar system planets. So we can take pictures. So we have to come up with new tools to kind of disentangle the weather and that's what we're presenting here. And where are you looking at specifically? Outside of our solar system but well within our galaxy. So what we call brown dwarfs are worlds that form like stars but they don't shine like stars because they're not massive enough.

28:06So they cool down throughout their lifetimes and they develop atmosphere is very similar to plants that we have in our solar system and they're easier to study the next appliance because they're not hidden beside these bright stars. Because planets form around stars but these brown dwarfs don't. So we use them as laboratories to study weather on these worlds outside of our solar system and understand what drives the weather and what shapes their atmospheres. How are you looking at them? How do you spot them? We use the James Webb Space Telescope and we look at them in the infrared range. So we essentially observe the heat that is emitted from these objects. And as they rotate in the sky we see the different sides of the object and we figured out a few years ago that as they rotate the amount of light and the color that we see changes and that's because the different sides of the objects have different cloud phenomena, different temperatures, different storms going on. So we can see this change in light and we can try and relate it back

29:06to the weather on these objects. How do you know that's actually the weather in inverted commas that you're seeing, that those changes in light and dark and therefore the density of what's around the object. How do you know that's the weather and clouds and so on? So like I mentioned brown dwarfs form like stars so they create all of this mass and then that leaves them kind of in this hot state where they emit energy and this energy is emitted in the form of heat. And then as this light or heat reaches the James Webb Space Telescope certain colors of the light are missing or brighter and that's because it has to pass whatever is between where it is emitted in the center of these objects all the way to us and what it passes through is clouds and storms and you know all of these things that happen inside an atmosphere but what we actually observe is one pixels spread across color space. So we call that a spectrum where we have the redder colors and the bluer colors and depending on what's happening in the atmosphere we see more of the redder, more of

30:09the blue and we can relate that back to the weather and the reason that we know that there are like key clouds and storms and changes in chemistry is because we can set up experiments in the laboratories and earth and mimic what we expect these clouds to look like and what the heat from the objects is like naturally and we can see how the light changes in our lab and compare it to what we see in space. For this specific object and the reason why we chose it is because it's relatively close to us and my relatively close I mean it's so far away that it took the light from the object when it was emitted 20 years to reach earth. So it was emitted back when I was born. It's an interesting way of looking at it indeed. What can we learn from doing this though? Because you're looking at a dot in the sky which is 20 light years away. Why is that going to help us in order to in our quest to understand more about how the universe works? I mean the ultimate goal is always to try and figure out if we're alone in the universe at least

31:14for the exoplanet and brown dwarf people. I think the far away goal is that and I don't think that technique that I've presented here is necessarily the way to do this but what all scientists do all of the time is develop more and more tools to use what we can observe and what we can know in more efficient ways and to learn more from the data that we have and are able to get. And so these tools that we're developing are really setting out to understand these atmospheres to an extent where we can understand what drives the weather, what shapes it. We know on our own are part of the reason why life is stuck around for so long is because we have this atmosphere that protects us and Mars has a weaker atmosphere and Mars has lost water and we know that those are likely related facts. So in studying these atmospheres we're able to learn about the conditions on these objects which ultimately will be important when looking for habitable targets. Mönis Rada at Trinity College Dublin. She's just published that work in the Journal Astronomy

32:17and Astrophysics. That is it for today. We are of course back on Tuesday though when we're going to be examining all things shroom and asking how fungi can be used to clean up oil spills and even to treat mental health disorders and I promise not too many mushroom related puns that would be in sport taste. Do leave us a review on whatever podcasting platform you use to get the program. These rankings really help with our visibility and the show to grow and thank you in advance to those of you who've already done that and thanks also to those of you who very kindly made donations to support the program and on the off chance that you haven't done so and you'd like to help us nakedscientist.com forward slash donate is the place to do that. I'm Chris Smith you can get in touch with me with any thoughts comments or feedback on the program at Chris at thenakedscientist.com meanwhile from me and from the rest of the team here at the Nakedscientist. Thanks for listening and until next time goodbye.

33:32Who says Halloween only lasts one day with Wayfair? One night of Halloween becomes a whole season at home from larger than life yard decor and moody lighting to festive accents for every room in the house Wayfair has what you need to make Halloween feel like magic all in one place because the best Halloween memories aren't made in just one night they're made all season long shop Halloween decor now at wayfair.com that's w-a-y-f-a-i-r.com Wayfair every style every home