Memory chips: the quiet winners of AI’s next chapter

• Nvidia’s latest message suggests AI demand is shifting from training models to running them constantly in real time.

• That shift makes memory more strategic, because inference needs speed, bandwidth and power efficiency, not just raw compute.

• Micron’s results matter as a read-through for the wider memory cycle, not only for one stock.

At Nvidia’s GTC conference on 16 March 2026, Chief Executive Jensen Huang said “the inference inflection has arrived” and raised the company’s view of the AI chip revenue opportunity to at least USD 1 trillion through 2027. That is a sharp step up from the USD 500 billion opportunity it had discussed through 2026. In simple terms, the industry is moving from building AI models to using them, again and again, at scale.

The brain still matters, but memory is the bloodstream

Training an AI model is expensive and spectacular, which is partly why it gets so much attention. Inference is less glamorous but arguably more commercial. It is what happens when millions of users ask a model to search, summarise, code, recommend or act. That means low latency, high throughput and efficient power use. Those are not just compute problems. They are also memory problems.

High-bandwidth memory, or HBM, is the premium product here. It sits close to the processor and helps feed it data very quickly. If the processor is the engine, HBM is the fuel line. A giant engine without enough fuel flow is just an expensive sculpture.

That is why Micron’s update this week matters. On 16 March 2026, the company said its HBM4 36GB product had entered high-volume production for Nvidia’s Vera Rubin platform, with more than 2.8 terabytes per second of bandwidth and 20% better power efficiency. Micron also highlighted a new memory module that it says can deliver 2.3 times faster “time to first token” for long-context large language model inference. That is industry language for one very simple thing: answers arrive faster.

Samsung is worth a brief mention here too. At GTC, it showcased Nvidia’s new Groq LP30 inference chip made on Samsung’s 4-nanometre process. That matters because it widens the memory and AI hardware story beyond Micron alone. Samsung is also pushing ahead in advanced memory, having started HBM4 shipments and flagged HBM4E samples for later in 2026. In other words, inference is not just lifting one memory name. It is raising the strategic value of the whole high-end memory stack.

Why Micron results matters beyond Micron

Micron’s earnings on 18 March 2026 are not just a company event. They are a temperature check for the whole memory market. This industry has a long history of being cyclical, messy and prone to mood swings. One year it looks like a commodity swamp, the next it looks like a licence to print money. The difference this time is that AI may be making parts of memory less ordinary.

Micron, Samsung and SK Hynix are all struggling to keep up with AI-driven demand, while Applied Materials has partnered with Micron and SK Hynix on next-generation DRAM and HBM development. Samsung sees an “unprecedented supercycle” in chips, with memory shortages still supporting demand, even as higher prices start to pressure personal computer and smartphone markets.