Memory and storage

Why AI needs so much memory

An AI accelerator is only as fast as the memory that feeds it. Training and serving large models means moving enormous amounts of data between memory and compute, so each GPU sits beside stacks of high-bandwidth memory (HBM): DRAM chips stacked vertically and packaged right next to the processor. Every generation of accelerator carries more of it.

HBM is hard to make. Stacking dies, connecting them and packaging them beside the processor uses more wafer area per bit than ordinary DRAM and yields less, so making more HBM takes capacity away from conventional memory. That is how AI demand tightened the whole memory market, not just the high-end part of it.

The layer also includes the chips that manage memory. Rambus designs the interface chips that sit on server memory modules and connect processors to DRAM; it grew revenue +17% over its latest four quarters at a 79% gross margin, a chip-company margin earned on every server that ships with more and faster memory.

The numbers: a cycle like no other

Micron, the only US-listed maker of DRAM and HBM, shows the change most clearly. Its DRAM revenue over its latest four quarters was $101B, against $28.6B the year before. In its latest quarter alone DRAM brought in $39.8B, against $9.0B a year earlier. Company-wide, Micron’s gross margin over the latest four quarters was 81% and its operating margin 75%: margins the industry has rarely if ever reached.

Storage rode the same wave. Sandisk, spun out of Western Digital, grew revenue +175% at a 71% gross margin, as AI data centers bought solid-state drives for training data and inference. Western Digital and Seagate, which make the hard drives that store the bulk of the data AI is trained on, grew +43% and +44%. Across the layer, of the $136B of revenue added over two years, $126B was kept as gross profit.

Memory is capital-hungry. Micron spent $30.7B on capex over its latest four quarters against $89.7B of operating cash flow, and reports $134B of remaining performance obligations, a sign that customers are signing longer commitments to secure supply. Sandisk reports $59.8B.

Storage: the other half of the layer

Memory feeds the processor while it works; storage holds everything else. AI data centers store training data, model checkpoints saved during training runs, and a growing volume of user prompts, documents and generated output. Fast storage sits on solid-state drives built from NAND flash, which is Sandisk’s business. Bulk storage, where cost per terabyte matters more than speed, still sits largely on hard drives from Western Digital and Seagate.

Both have benefited, in different ways. Flash is a commodity like DRAM, so its prices and margins swing with supply; AI demand for high-capacity data-center drives tightened that supply at the same time as HBM tightened DRAM. Hard drives are a consolidated industry with two main makers, and the shift to very high-capacity drives has let them grow revenue without adding much capacity. Western Digital’s gross margin over its latest four quarters was 53% and Seagate’s 50%, levels that would have looked unusual for the business a few years ago.

For investors the storage names are a slightly different bet from the memory makers. Their demand depends on how much data AI generates and keeps, not on how many accelerators are shipped, so they may hold up later into a slowdown in accelerator purchases, and recover later too.

How to read a memory company’s filings

Three lines matter more than revenue. Gross margin tells you where pricing is in the cycle, because memory makers’ costs move slowly while their prices move fast. Inventory tells you whether supply is outrunning demand: rising inventory at a memory maker is usually the first sign that prices are about to fall. And capex tells you what supply will look like in two years, since new memory capacity takes that long to come online.

Micron also discloses revenue by product (DRAM, NAND and other) and remaining performance obligations, which show how much of its future sales customers have committed to. Longer, larger commitments are the structural-shift case in numbers. A memory company whose backlog shrinks while its capex rises is setting up the old cycle again.

Structural shift or peak of the cycle?

Memory has a long history of booms ending in busts. High prices draw new capacity, the capacity arrives all at once, and prices collapse. Every investor in the layer should start from that history. The question is whether HBM breaks the pattern.

There are reasons it might. HBM is sold under longer contracts with a few large customers, designed in with each accelerator generation, and technically harder to make, which limits how quickly any supplier can flood the market. The larger backlogs memory makers now report point the same way. There are also reasons for caution: every maker is adding capacity, conventional DRAM and flash remain commodities, and the accelerator makers have every incentive to qualify more suppliers.

The practical test is margins when AI spending is still rising. If memory margins hold while new capacity comes on, the layer has changed. If they fall even as AI demand grows, the cycle is reasserting itself, and the stocks will likely react long before revenue does.

The companies, side by side

Latest four reported quarters for US filers; latest fiscal year for 20-F filers (marked). Underlined names have their own company page; the others link to the company’s filings on SEC EDGAR. Margins are left blank where a company doesn’t report gross profit or cost of revenue.

Where the money comes from and goes

This layer is paid by servers and chips. Its own spending flows to foundry & tools. For the dollar amounts and who keeps them, see following the money.

What to watch in the next filings

• Micron’s gross margin and DRAM revenue each quarter, and any change in its HBM commentary.
• Memory makers’ capex: a sharp rise across the industry is the classic sign of a coming glut.
• Remaining performance obligations: longer customer commitments support the structural-shift case.
• Accelerator makers qualifying additional HBM suppliers.
• Conventional DRAM and NAND pricing, which AI demand has lifted but doesn’t control.