IMEC Says Today’s AI Will Look Ancient in 10 Years

0:00Welcome to Source3. The potential of photonics. The technology to interconnect these GPUs. Global, payroll and HR platform. 8 billion operational data points. The scale-ups over startups. You'd rather have a million of something and one of anything. Building the intelligence layer for the fans. There isn't enough power. There's not enough memory chips. We're at the end of physics of what we can still do. OK, amazing. So we're on our final talk for today. Today we have Stephen and we have Adam. Stephen, you come from IMEK. Adam, you're from Strike. So we're going to talk a little bit about what comes after the GPU. So let's start with you. Could you explain what IMEK is? Sure. I think we're a little bit the most unknown hidden gem

1:00in semiconductors, at least unknown for the average show. Definitely not unknown for the semiconductor world itself. We're the worldwide leader in chips, but in actually the early stages of building a chip. So hardware is not like software. It's not that you prompted then suddenly 30 seconds later. Something comes out of it. No, it's a process of five to 10 years. And we're at a very early stage of that process where we have a unique clean room facility that nobody else in the world has. And so all the biggest chip providers in the world actually first come to us, first in Belgium, to kind of design that next generation chip that you'll see maybe in seven, eight to 10 years from now. And going through a whole phase of designing that chip, we handed over them to them to further commercialize it. But in that sense, there's almost no chip in the world today that has not been touched by IMAC in some way. We already exist for 40 years. And we've been driven more or less that chip roadmap for the last 40 years. So I should ask you the secret question,

2:02because you know all the secrets on chips. And some of them I'm allowed to answer. What can you answer? Well, I think we're definitely going to talk more about that. But indeed, that we're actually seeing a huge amount of disruptions going on. I think we're now kind of plateauing in what we call Moore's Law. What does that mean? Chips have been becoming bigger and bigger or faster and faster over the last four years. We always did that through scaling. So what does that mean? It means that we made a transistor smaller and smaller. And we're at the level that a transistor has about the depth of one nanometer, what is one nanometer? It's about 100,000 times smaller than a human hair. That means that we're at the end of physics of what we can still do. And I think we're going to come into an era where what we do in hardware is going to be, needs to be way more inventive than what we've been doing in the last 10 to 20 years to keep on scaling that roadmap of Moore to make sure that chips become faster and faster as they already did the last four decades, basically.

3:04Adam, as you look into the chip industry, how do you make sense of it? How do you map it out? Yeah, I think right now, GPU is a very good at the training part. And we're now looking at the bottleneck that sits in the inference layer. And the bottlenecks that I look at the most and try and look at companies that attack those bottlenecks, sit in power and energy, memory, calling, and data transfer. So when we look at the inference side of things, memory, the memory shortage in general, and increasing the bandwidth and lowering the latency of getting the useful memory to a GPU as quick as possible is a really interesting and exciting place that needs to be builded within. Cooling as well and where that's going, I think bringing the liquid that takes heat away from the chip closer and closer to the GPU itself is how it's going to predict progress over the next five to 10 years. And I think a lot of the value will go to the manufacturing side

4:06of that. And yeah, I mean, power, everyone knows how much of a constraint and bottleneck that is and increasing the capacity there is huge. I recently interviewed Tony Kim of BlackRock at the Raise AI Summit. It's actually in Paris. And we're talking about hardware. It's like the hardware era, Renaissance, everything shifting to hardware. Software is getting a bit of a beating. But the margins, people don't understand. The margins for chips are fantastic. And for hardware versus software, which is getting eaten by AI. And again, like the token spend and that kind of thing. So how do you make sense of the shift in value and how the pyramid kind of flipped? Well, I think the demand is just so high for the thing that is less fruitful out there. So there isn't enough power. There's not enough memory chips. The shortage in memory.

5:06I mean, if we look at, there's only three companies in the world that produce memory, which is 95% of the market, micron, SK high-nix and Samsung, two of them are in Korea. And a lot of their shift has been filled out by hyperscalers taking over their whole inventory. So you've seen prices rise by 700% this year just for the memory part of the chip. And that will happen across the layer because it's the thing that's in the most demand. The training side and the software side of things is continually getting better. But the physical side is the thing that's harder to get better overnight. Yeah, to be honest, I don't really think it's necessarily flipped. I think it's more the immaturity of the software market right now. Might seem weird that I say immaturity of the software market, but I think what we've been doing at AI the last four or five years has, of course, and software been tremendous. But if you look at it from, OK, how complex it is

6:09to actually, in terms of computations, do whatever, I mean, large language models can do. It's still very much a brute force approach. And as a result, the value that you can get out of it is pretty limited. I think if you're going to get further, we're going to see new types of software evolutions as well, which are tied way more closer to hardware. And because of that, I think the value will increase way bigger. And I think so we're kind of filling the base of that pyramid already in hardware of creating a lot of value there. I think software will follow in that sense. We're seeing more and more chip companies founded every year, probably than ever before. And venture dollars are definitely helping fund that. Is there a capacity to how many chips can exist in the market? I think right now, there's kind of endless demand for that. And I think that's still going to be the question of how that can continue. I think it's rather the bottleneck is manufacturing. Manufacturing is the right type of components in the different regions.

7:10And I think, for example, what Timah Foundry's is doing there is super important. But having the right type of components for that is I think the real bottleneck, not necessarily the number of companies that do this because every company can come with a completely different unique design. And in that sense, can also kind of revolutionize what we're going to have as well. And probably in the software world, we're going to see way more different types of algorithms. And as a result, we will also need more diversification in the type of hardware, so more than just GPUs as well. As models get bigger and bigger, what breaks first on the chip layer? Well, yeah, I think for sure right now, memory. Memory is absolutely the number one component that is already broken to today. But I don't really think that models will become necessarily bigger and bigger. If you just think about it, the latest open source models like Kimi K3D already have more parameters than what the human brain has.

8:12But on the other hand, that K3 chip, or the chip that we need, to fuel that K3 model is about a million times more or less energy efficient than what the human brain is. And so I think we're going to see a different type of paradigm. We kind of went into an era that last five years of building bigger and bigger models. We went from 175 billion power emitters to kind of 10 trillion parameters right now. That's going to stop. And there's a lot of companies right now taking that shift as we speak, abandoning what we call the scaling hypothesis of building bigger and bigger models. And therefore trying to solve those memory bottlenecks as well. Copper has kind of met its days. I'm curious, how do you think about photonics and how that changes the layer? Yeah. I mean, for fit to. Yeah, I mean, I think there are so many benefits to using photonics

9:14instead of copper. And copper is easy because it's all in electrical, it's all digital. And the bottleneck with photonics is changing between optical and the electrical layer. But transferring data more quickly and more efficiently and not having that latent heat aspect that electrons in copper wires do will only continue to scale the efficiency of data transfer. And then I think we'll see the architecture continually shift for where photonics will be within data centers. At the moment, it's between the clusters and we're using linear plugable optics. But I think in the next five years, we will see these optical components get closer and closer to the actual GPU. And we will go towards a state of co-packaged optics. So for me, I mean, every day what I do actually at IMAG is thinking about what could be AI in the future. So how is AI going to evolve in like five to 10 years from now? And I think we're seeing kind of from a software perspective

10:17that very much diversifying path of a lot of different approaches, whether it's rolled models or more reinforcement learning based approaches. But what is 100% clear is that whatever scenario in the end will become the dominating model of the future, moving data as fast as possible in the quickest way with a huge amounts of data that we need to shift is always going to be what we need. Because again, towards the analogy of the human brain, that's what our human brain is also unbelievably good at. We have an unbelievable 3D structure that is able to do this. And so photonics will play, I think for me, especially in the next years, it's going to be a nuclear technology to enable the chips of the future. I think also on photonics, it's attacking many parts of the bottlenecks that I touched on at the start. It's attacking energy because it can transfer data quicker. It's attacking cooling because you don't need as much cooling around these GPUs if you're transferring through light.

11:20And then it's attacking data transfer at the core. So I think having spanning most of the bottlenecks that we're seeing in the AIM for build out right now, it's like the key technology that is likely to win. As we close out, and this is the last talk of the day, so thank you all very much. What is your hottest take right now? I think we're going to five to 10 years from now. We're going to laugh at how old-fashioned AI was today. That's a bit my hottest take in the sense that, of course, we went to this explosion of what AI was bringing us. But I fundamentally don't think that today's technology, that this is what we're going to be talking about in 10 years from now. I think an actual also in software revolution is coming. And for those who are all enough, we still might remember the internet days of a dial-up connection where we had all these bleeping sounds to make connections to the internet. And we kind of laugh at how that's how we did it in back those days. I think in 10 years from now, we're going to talk about large

12:24language models, which are very much a brute force approach in exactly the same way. I might be a little bit biased on this one, but I would say it's a very good idea to recruit or invest in people that have had a academic and technical background in Europe and had the well-withal and business sense in the US and combining those two mindsets. A lot of the best founders that I've backed and I've seen have that dichotomy between the two. So I'd say both recruiting and investing those two. Amazing. Well, Stephen, Adam, thank you very much. I think Louis is going to come up next to say something. Hey, it's Molly. If you enjoy our interviews, check out our newsletter sorcery.bc, where we deliver a once a week top deals and tech headlines email and also go deeper on our

13:26podcast interviews. Subscribe to sorcery today and don't forget to subscribe to the podcast on YouTube, Spotify, Apple or wherever you listen. Link in description to sign up.