Memory chip mania isn’t going away, with Dan Kim

Chips are one of those products that everyone cares about.

Consumers care about them because they're in every electronic device we own.

Tech companies care about them because they are crucial to the AI build out.

Politicians and economists care about them because they are central to

due economic tensions between the US and China.

In the past, drama in chips has been focused on logic chips.

Those are the ones that process data.

But recently, another less glamorous corner of the chip market has been going crazy.

Memory chips are red hot right now.

AI data centers are hoovering them up, pushing up prices and crowding out customers elsewhere.

It is a wild time to be in the chips business.

And today, I'm going to try to make sense of it.

What are the causes and consequences of this mayhem?

And could China play a role in bringing this mad cat market into balance?

This is the economics show with Simeon Keynes.

I'm joined today by Dan Kim.

Dan is the Chief Strategy Officer at Semiconductor Analysis firm Tech Insights

and was previously the Chief Economist at the CHIP's Program Office

at the US Department of Commerce.

Dan, welcome to the show.

Thank you for having me.

OK, so first question, on a scale of 1 to 10.

How crazy is the memory chip market right now?

Right now, it's an 8.

Next quarter, it'll be a 9.

And the quarter after that, it'll be a 10.

It'll stay at a 10 until the end of 2027, at least.

And then we'll see how long it stays at 10.

But it'll be a 10 until at least next year.

OK, that is remarkably precise, which I appreciate.

Could you put some slightly more serious numbers to that craziness?

What a price is doing right now.

Well, on a year to year basis, the price is for DRAM,

which is one specific kind of memory that is being used for data centers,

mobile phones, pretty much everywhere.

There's electronics.

From last year to this year, we're anticipating a over 200% price and increase of DRAM.

That is historically unprecedented.

You do see swings in price cycles in the DRAM industry and the NAND industry,

which is the other part of memory.

But this amount of price swing is historic.

And we actually don't quite know when this upcycle will end,

because there is no new supply coming online in significant volume until at least 2028.

And because of AI demand, demand is increasing, but supply is not increasing.

And so the only thing that could adjust right now is pricing.

And that's what's happening.

Okay, so pricing is high, and presumably there are customers elsewhere

who are just not able to get the chips that they want.

Yeah, so, you know, interesting thing about the memory market is that

because the goods are standardized, it acts like a commodity market.

There's actually a spot price on a daily basis for DRAM, for example.

Now, most of the chips are bought and sold on a contract basis,

but there is a spot market as well.

So you can track what the prices are doing.

So in a market like this, you would presume that if you're willing to pay the price,

you could get the good.

But we're heading into a territory in which the price signals are actually not working as well,

meaning the price signals are so high that even when you're willing to pay the price,

you might actually get a reply bat from the manufacturer saying,

we could only give you x percent of what you're looking for,

even if you're willing to pay the price we're quoting you.

And so we've been hearing that from not only the high end of the market,

the data center AI companies,

but we've been hearing that more and more from the lower end of the market,

meaning the legacy demanders,

automotive, medical device manufacturers, and others,

that this is an issue across the board where the prices are going up,

but because there's no new supply coming online and demand is increasing,

there is now an increasing allocation happening in the market.

Do you have a rough sense of what share of these chips that are actually being supplied

are already being sewn up in contracts and what share are being traded on the spot market?

Yeah, and so effectively, there is no supply out there that hasn't already been spoken for.

So practically, most of the supply has been sold out.

This has been the case for high bandwidth memory,

which is the specific type of DRAM that is paired with AI accelerators to make AI training trips.

But it is also the case for most of the memory products out there for this year,

if not for next year.

And so if you're looking around,

hoping that there is some memory products out there that are on the shelf

that has not been spoken for,

it's going to be very difficult to find right now.

To what extent are companies trying to lock in supplies,

contracts for these things,

even more than they need,

because they're worried about future shortages.

Is there that kind of hoarding dynamic?

There is some stockpiling.

There is some overbying.

That is partly what explains the memory cycle.

In the previous cycles,

hyperscalers, for example, have overbought.

And then the year after that turns out they don't need as much.

And so the orders crash and so the price crashes.

This is the usual cycle.

This time, we're not seeing too much of that.

We are seeing effectively, if you have memory, you're using it.

There's not some stockpile out there.

We are in a situation in which,

if memory is being offered,

companies would buy it,

even if they don't think they would need it.

And so there is some of that going on, I am sure,

but it is an extraordinary tight market at the moment.

Do you have a rough sense of how important the cost of memory chips is

for some of those, you know,

high profile electronic goods like, say, the iPhone?

Well, memory is already among the highest part of in a bill of materials

for any electronic good.

So for example, if you open up a knife phone

or if you open up any smartphone,

the three highest value components in there

would be the application processor.

This is a leading edge logic chip that is usually designed by the firm

and then produced at TSMC or Samsung.

And then the two other components that are highest value is the DRAM,

which is the memory and then the NAN, the storage.

So those are the three highest values.

Now, the two of those chips prices are going up significantly.

On an average phone,

the memory component could be somewhere between 15 and 25%.

Now it has a potential to be somewhere between 30 and 40 and even 50%.

Depending on the pricing.

Okay, so you said that basically there was no new supply coming online.

Why?

So we actually have to go back a little bit to explain why we're in the situation

that we're in.

It's not long ago, actually in 2023,

that we had a down cycle in memory,

where the prices were depressed and the memory companies were in negative margin territories

because there was too much supply.

Coincidentally, this was the same year that chat GPT was really taking off,

but there was too much memory in the market.

And so there was a decision, a rational decision by memory makers to cut their production

and to delay their build out.

At the same time, the demand was skyrocketing for AI.

And there was some uncertainty about how much demand that would be.

And so it was treated like business as usual, but then the demand tsunami came.

And so because of the delays that were happening, the supply is not being introduced today.

And so at the moment, the memory companies are all of them,

all memory companies throughout the world.

The Koreans, the Japanese, the Americans and the Chinese are racing as quickly as humanly possible

to build new supply, but they're not coming online until late next year

or early 2028.

So how long does it take between executives saying,

OK, we need some more.

And OK, there are chips being sent to customers.

Well, if you're starting from scratch, you have to find a site.

You have to develop the site.

You have to build the infrastructure around it.

Then you build the shell of the fab, then you equip it, and then you qualify it.

All of that could take somewhere between two to six years,

depending on the site readiness and depending on how fast you could build.

And depending on whether you can have the equipment ready to go.

And this is why you usually have some plans to build in case of new demand coming online.

And so for pretty much every memory company, they have sites where they could build another fab

on top of it in case needed.

But the demand is extraordinary.

And so the industry really was just cut off guard.

So there is a site in South Korea, for example, two sites in South Korea, for example,

in which both Samsung and SK HiNX are building just as quickly as possible.

There's a site in Idaho that micron is building that is trying to build as quickly as possible.

The same thing is happening in China by Chinese companies.

And so the supply is coming, and they're coming in large numbers.

They're just not coming yet.

Could you talk a bit about the structure of this market?

Because it's very highly concentrated, right?

There are only a few firms and entry.

Isn't that easy?

That's correct.

In fact, it used to be that there were many memory firms, and then there was a wave of consolidation

that happened in the memory market over the last two decades.

And the reason is because the cycle on the market is really brutal.

And so unless you actually have the right volume and the right strategy to write out those waves,

it's not a business you can really survive in.

In terms of DRAM, there's only three companies left for mainstream DRAM production.

It's Samsung, Escaix, and Micron.

So it has become a three-four-and-market, at least for DRAM.

There's a couple more in NAND.

And then there are the entrants in terms of the Chinese companies, YMTC for NAND and CXMT for DRAM.

You asked about whether it is possible for new companies to enter the market.

The answer to that is it is close to impossible for a new company to come and become competitive

in the memory industry for two reasons.

One is the simple scale and volume you need to compete.

This is a scale and cost business, because it is standardized, because it is commoditized.

Unless you have the economies of scale to actually survive in the business,

it's almost impossible to compete.

So that's economics is the biggest mode.

The other mode, of course, which is considerable is technology.

It is very difficult to make these now.

And so there are two types of firms in the semiconductor industry that actually uses

EUV lithography.

One is advanced logic, TSMC, and Intel, and Samsung, the other is DRAM.

And so you are really at the highest end of sophistication when it comes to manufacturing.

And so you have the economics mode and you have the technology mode, which makes it almost

impossible for a new entrant to come in and become competitive.

The only exception to that are Chinese firms.

And that's because there is almost no end to the state support that they're receiving

to surviving the business.

So the Chinese state is clearly interested in Chinese companies becoming more dominant

in this memory market.

What is the current trajectory of the firms that it is supporting?

So I'll take it one by one, because the two markets are quite different within memory.

So I'll take NAND first, and then I'll go to DRAM.

On the NAND, NAND is the storage part of your memory.

This is the memory that stores your pictures, your videos.

And there is a Chinese firm called YMTC that is actually quite competitive

in the market today.

So depending on how you count, it's either the second or third largest provider of NAND

into the market today.

They're not as sophisticated as their Western counterparts are.

So their ability to stack the 3D NAND chips is not as good.

And so they are still challenged.

And so they wave back and forth between being profitable and not.

And not right now, they're very profitable as all memory companies are.

But they are a global provider of NAND at the moment.

NAND from YMTC is primarily found today in Chinese products that are being sold into the Chinese

market.

And so very rarely would consumers in America or Europe actually find and use YMTC-enable

devices in their businesses or in their homes.

So YMTC is already there.

This is already an entity listed company, meaning that no Western equipment can be sold to

them.

And so they're not that they're functioning quite well and making progress.

And then to DRAM, there's a company called CXMT that is more behind relative to YMTC

to the Western competitors.

But they're starting to introduce more and more of their DRAM devices into the market.

We at TRTEC insights are finding more CXMT memory products within flagship phones like Huawei

and other Chinese pancets.

So they're more behind, but they are challenged by the fact that they do not have access to

UV lithography, which is absolutely necessary now to make advanced DRAM products.

So they are trying to find ways to get around that and also introduce large volumes of DRAM

into the market.

Do you have a sense of what their market share could be like in the next few years?

I mean, obviously Chinese buyers are a pretty big chunk of the global market.

Yeah, that's a good question.

So already in the NAND space, YMTC is producing something between 10 and 15 percent of the

bits into the market.

And so their market share is growing.

But if you sort of take a step back and look at the historical trends of market shares in

the semiconductor industry, the market shares among the companies stay remarkably stable.

As I explained before, it is very difficult to gain entry.

It's also very difficult to gain market share.

So you really have to offer a differentiated technology or something else has to come along

to change market share significantly.

We've seen some of that through, for example, high bandwidth memory that SK high necks

has really been the leader of in the past couple of years.

But of course, the other firms are catching up and providing competing products of their

own.

So it is an interesting question to ask, how much can the Chinese actually provide into

the market?

Yes, they have a little bit of a captive market that they can sell into in China and

China does consume significant amounts of memory.

But the US is actually the largest memory market in the world period.

And Chinese are a little bit far behind.

So we are estimating that in the next few years, the DRAM market, the US will be somewhere

above 60 percent of the DRAM market.

And that is primarily because there's so much AI build out happening in the US and because

the DRAM for hyperscalers and DRAM for AI, the value of that is so much higher that US

has become by far the most important market.

So there is a little bit to the limit of how much Chinese memory producers can gain market

share as long as they don't have access to the United States.

But to your point, they can capture more of the Chinese market.

And I'm sure they would have more favorable access to it.

So they can definitely gain more market share over time.

And I think the test will be when the next down cycle happens.

Because it is a cyclical market where it definitely in a prolonged up cycle, when the

next down cycle happens, how much counter cyclical investments can the Chinese make at that

point to capture more market share?

That would really be the telling sign where a few years off from that, but I see the Chinese

getting ready for that.

And that is a very predictable playbook in the memory market because it is effectively

how the Japanese memory makers planned to disrupt the American memory producers back

in the 80s and the same thing happened with the Koreans and the Japanese.

And so the Chinese, I think, are hoping that they're next.

Oh, that's interesting.

So today, you hear concerns about potential feature over capacity in Chinese chipmaking,

but maybe they were just copying a playbook that was played out previously.

So there is a significant difference between how the Japanese and then respectively Korean

companies have competed in the memory market to disrupt the incumbents, then what the Chinese

companies are doing today.

The level of state support in China is simply unprecedented.

There has been trade disputes before, famously in the 1990s, where trade remedy cases were

brought against the Korean companies.

But if you were to ask memory companies today, how to compare the disputes then to the

amount of state support the Chinese companies are receiving today, they would all tell you,

it's night and day, not even comparable.

So just thinking about what the Chinese state is doing here.

I mean, for a long time, they have spoken about wanting to reduce their reliance on other

countries.

And so you could see this domestic build out of chips as part of that.

To what extent, though, is this supply crunch reinforcing the idea that these memory chips

are a sort of strategic asset, right?

And maybe actually the South Koreans or the Japanese have power because they could restrict

access to those chips.

Or I suppose we've already seen the US restricting access to particular chips.

Well, in this case, for memory chips and other advanced chips, the idea that it is a strategic

good to have state support around it in some way, shape or form, China is not the only one.

There just been earlier, in a sense.

If you go to the US, I think they would see now that the memory is a very strategic asset

that they need to develop.

And there is state support going into that.

If you go to Korea, they would say absolutely it's a matter of critical importance to the

country to have memory competitiveness.

I think if you go to Japan, they still have memory production and it is a priority for

them.

And the same could be said with China.

It's just that they were in our current era earlier to recognizing that memory is something

they wanted to develop and support and have been at it longer.

I mean, you worked on implementing the CHIPS Act.

Was memory a big part of your job then?

Or was it really become more important over the last few years?

So when we wrote the strategy for implementing CHIPS, we did say that memory was important

and that we were in particular looking to support DRAM manufacturing in the United States.

Part of that was because there was already a project or two that had the capacity to make

DRAM at the time.

But this was 2023 in which there was a glut of memory into the market and there was an

overarching concern about the concentration of advanced semiconductors of logic and

foundries coming from Taiwan.

And so the feeling at the time was that we needed to hedge more against the logic.

But we were also recognized that we were playing the long game here and that we needed a little

bit of everything and that memory was an important thing to focus on.

What I think to be fair, we did not anticipate is how dramatically the economics of memory

would shift due to AI.

I'm not sure if anyone really fully appreciated that.

Of course, if we did, there would be more supply today.

But we missed that and I think the entire market missed it.

But memory was not a forgotten topic.

It just was a little bit less emphasized.

Does this memory production represent a choke point in the same way that high end logic

CHIPS do?

Right?

I mean, it is the concern that China might get huge market share and then cut off other

countries or is it something a bit different?

Like what's the thing that everyone is worried about happening if you weren't into

the end?

Well, right now the concern is there's not enough supply.

Right.

But that's not really a geoeconomic thing, right?

That's just like we need chips.

Yeah.

So let me try to connect it out here.

Right now the concern is there's simply not enough memory chips to go around.

So the temptation is for non-Chinese companies to be sourcing from Chinese companies today,

which opens the door for them to grab more market share down the line.

So if you're a careful observer of this market, that is a concern if you believe that more

market share, a concession is coming down the road on the next down cycle.

But I'm not sure if anyone's thinking that much far ahead.

It is difficult to gain market share in the memory market because the Korean American

and Japanese producers have been so good and so disciplined and efficient in making

them.

The Chinese plan in about 10 years ago was actually to acquire technologies and failed at

that, meaning they actually try to take over certain companies back in 2015, which did

not receive regulatory approval.

I think they recognize that it's just a very difficult business to get into with nothing

that they need to acquire technologies and to know how and the capacity to do that.

But that road was blocked to them.

And I think you have to give a lot of credit to the Korean American and Japanese producers

of keeping the market so well disciplined over time to be able to keep the Chinese out.

But it is a very brutal competitive business with very difficult time to entry.

But I think you're starting to get a little bit more sense now that this is a focus for

China and that they're not going to give up.

And they are looming in the background even as the market has never been more successful

than before.

OK, so the Chinese government is clearly giving a lot of support to its actors they want

to get a bigger market share.

How are governments elsewhere responding?

I mean, you said that the Chinese were just early.

Is it the case that the US government, the Koreans, the Japanese, are they now increasing

their support to increase investments or actually is the market signal so powerful that

they don't need to?

I haven't really seen a memory strategy from either the US or any of the partner countries

of the United States that is very deliberate except for South Korea.

They have seen and they recognize that memory is one of the most important geopolitical

assets they have.

And the South Korean president, along with industry leaders, announced a development of a new production

cluster in the Southwest region of the country.

I think that is a signal that says, OK, there's a lot more build out of memory here that's

going to happen around the world.

We want to make sure that we're part of that build out as well to keep our competitiveness.

And so in some ways, the Chinese have been recognizing the importance of memory for a long

time.

The Koreans are clearly recognizing that.

And then now I think it's the Americans and others because of the shortages that will

soon wake up and to think about what they want to do.

OK.

I want to go to a break now.

And when we get back, I want to talk about the wider chip markets, and not just memory,

but logic.

We are back from the break.

So as you said, the Chinese are massively expanding in memory chips.

What other types of chip have the Chinese been showing success in?

Well, let's be clear.

They're showing progress in a lot of different places in the semiconductor market.

So two ones that I will highlight.

One is just legacy foundry capacity.

And foundries can make logic, can make analog, can make different types of chips.

And so they are really copying Taiwan's foundry model in which they have fabulous or design

only companies in China utilizing domestic foundry capabilities.

So there's a big build out on that.

In fact, most of the new capacity that we see from say 2022 to projecting out to 2027,

most of the new foundry capacity in those older nodes are mainly in China, not so much elsewhere.

So they're becoming more self-sufficient there.

The other place I would note is that they are making great strides in what we call compound

semiconductors.

This is semiconductors that are not based solely on silicon, but on compounded materials

like silicon carbide, gallium nitride, and other more exotic materials that are much

better at power semiconductors, so they can handle higher voltages, for example.

So one example of that is a silicon carbide based power inverter for electric vehicles.

So how do you enable high voltage charging, which enables faster charging in the electric

vehicle vehicles market?

So this is a very difficult material science challenge for the industry, and China is

making great strides in that field as well.

Are there any areas where the Chinese have really been trying to expand their own chip

capacity, but they haven't managed it?

It is actually memory.

When I started covering the semiconductor industry for the US government more than 10 years

ago, it was widely assumed that that was the most obvious place where they would go and

take over first.

Standardized good, scale economics, it sort of really fell into Chinese willhouse, but they

had a really difficult time breaking into the market for a long time, particularly because

the Korean manufacturers and American and Japanese manufacturers really kept them at bay.

It is very difficult to catch up with the scale and the pace in which the memory market

moves.

It's only recently that they're making really great strides now that they're opposing

more of a threat.

What about the advanced logic?

Right.

So these are the teeny tiny two nanometer, three nanometer chips that TSMC excels in where

production has mostly been in Taiwan.

There was some US efforts to try and pull some of that production away to the US.

What's been happening there?

So there is two challenges when it comes to high end logic.

One is economics and one is technical.

So on the economics, if you do not have access to the most advanced lithography tools, then

you're bound to have more defects on the wafer.

And so that poses a technical challenge as well as an economic one.

So for example, the latest node that we see from Chinese Foundry's SMIC is what they

would call it N plus three node, which is a six nanometer node, which has already been

introduced by TSMC and others several years ago.

The technical challenge actually poses an economic challenge here as well, which is if you

have too many defects in a wafer, then you can't actually make larger size chips, which

actually limits your market opportunity to some degree.

AI chips require larger size chips, larger dies.

And if you can't yield those, then you don't really have market access to a degree that

you need.

And so they are a challenge because they don't have access to these tools.

OK, so the Chinese are struggling because they don't have access to the highest end lithography

machines.

There was that story about how they may have managed to cobble together an EUV machine using

old parts.

The US administration certainly seems concerned that they've managed to do that.

What did you make of that story?

We haven't seen evidence yet that they have a domestic EUV capability.

But I think what we are seeing is that because they are challenged in this particular technology

path, they are looking for other technology path to make up for it in ways that you would

do not need to do outside of China.

So for example, if you only have access to smaller, less performing chips, then you have

to figure out a way to package them more efficiently in an advanced way to cobble together

the performance.

So we do see more advancements in advanced packaging in China.

And we also see that systems level integration advancements are also present in China.

So how to string together chips, how to string together systems together, and then share

volume of power flowing through the system to get the compute performance out.

So what they lack for in efficiency and economics because of the expert controls, they are trying

to make up for it in different technical solutions, but also to share volume and share systems

level integration and power to make up for the compute power lack.

What do you say to the claims that because China can wriggle around the controls because it can

adjust along these other margins, actually there's no point in applying the export restrictions

in the US to just let China have what it wants.

So let me try not to make a policy point on expert controls because that's really not my place

to make them.

But let me make a market observation.

If you were to allow every type of equipment that is currently under restriction to be sold into

China today, they would purchase a lot more.

And they would advance more quickly.

There's no question about that.

So I think you have differences of opinion as to the effectiveness of expert controls.

Some prominent government officials in the US have called them a speed bump, which is not a

permanent solution and they need to run faster.

But these are very difficult challenges for policymakers to grapple with for sure.

What's the latest on US progress in trying to move production of the highest and

logic chips away from Taiwan?

This is obviously an objective as part of the Chips Act, school your own progress.

Yeah, and to be clear, the objective of the Chips Act wasn't just to hedge Taiwan risks.

It was to build more semiconductors in the United States because it was a strategic choice

that the Congress has made.

So there is significant progress.

The Arizona facilities at TSMC is yielding quite well in Fab 1 and they are now at Fab 2.

Intel is producing 18A chips that has already been introduced to the market.

And Samsung is continuing its build in Texas.

And so there are several facilities that are going up and expanding for advanced logic in a way that

hasn't. But it is also the case that capacity for advanced logic is being built faster outside

of the US than inside the US. But compared to where we were a few years ago when we pretty much had

no advanced logic capabilities for foundries, it is a significant progress.

Thinking about what the Trump administration is doing right now, they've clearly been applying

considering trade restrictions. If you were back in government, what would be your advice to

senior decision makers? What would be your priority?

What I would say is businesses need predictability in the policy environment that they live in.

And so providing as clear guidance and consistent guidance as possible is necessary

for very large investments to happen and to execute, which is what the semiconductor industry needs.

I would also note that there is enormous amounts of human capital involved, particularly because

foreign companies are such an important part of the equation here for them to make foreign

investments into the United States, having a predictable policy environment on both laborer

and capital and trade are so important to have.

Dan, thank you so much for joining me. And also thank you for your help when I was writing my

book, How to Win a Trade War. I'm so glad that I got you to share your insights on the podcast as well.

Thank you.

That is all for this week. You have been listening to the economics show with Simea Keynes.

If you enjoy the show, then I would be eternally grateful if you could rate and review us

wherever you listen. This episode was produced by Misha Frankel-Deval with original music and sound

design from Green Turner. Our executive producer is Manuel Esaragosa. I'm Simea Keynes. Thanks for listening.