원문 보기
The key to finding great long-term investments is understanding a company's products. Huge growth happens when companies have perfect products for hot new markets and telling the winners from the losers in those markets is the best way to get rich without getting lucky. Well, Nvidia just announced something that will change which companies win and lose over the entire AI era.
And of course, which AI stocks will win big as a result. Your time is valuable, so let's get right into it. First things first, this video is a little different. It gets pretty technical and I share some of my personal opinions on key design choices, which may not be for everyone. That said, I did my best to explain things as I go, keep things grounded in investing, and separate my opinions from the data, the specs, and the facts.
I'm making this video now because Nvidia announced their new Vera Rubin platform at CES a couple weeks ago, and I'm genuinely surprised by Wall Street's reaction, or really the lack of one. Investors clearly understand that Reuben is a big performance jump over Blackwell. But what everyone seems to be missing is how Nvidia achieved that jump.
And that's important because they didn't just make Reuben much faster than Blackwell. They effectively erased AMD's memory advantage, seriously undercut Google's integration advantage, and even threatened Broadcom's edge in AI networking, all in one fell swoop. Obviously, I can't cover all those things in a single video, or else it would be a couple hours long.
So, let's just tackle the big one, Nvidia versus AMD, and what their different design choices imply about the future of their data center ecosystems, which are major money makers for both companies. Just to be clear, I think AMD is in big trouble here, and I'm going to clearly lay out why. But before I can do that, we need to understand why Nvidia went with such an extreme approach to how they designed Vera Rubin in the first place.
So, let me lay out everything I learned from Jensen's keynote presentation, the exclusive Q&A session with Jensen that I went to after, and my interviews with multiple Nvidia executives, which I'll also release as full podcast episodes in the next few weeks. The big thing investors need to understand is that artificial intelligence is running into six hard limits today, and there's a lot of money to be made by solving them, both for companies and for their shareholders.
Here's my best attempt at describing all six limits without any industry jargon. The first one is model size. Frontier models are growing by around 10 times per year in terms of parameter count. So each new generation of large language models, multimodal models, and AI agents needs up to a,000% more compute power to train.
But Moore's law is currently only making chips around 30% more powerful per year. So chip makers can't just keep up by cramming more transistors onto a chip. The second one is token count. Reasoning models are using far more tokens per prompt today than they were a few years ago. In fact, studies show that reasoning tokens can outnumber output tokens 10 to one.
That means that reasoning models are spending 10 times more tokens, breaking complex problems into steps, thinking through each step, and checking their answers versus the length of the answer itself. Three, memory bandwidth, which is one of the big focuses of this video. Today's models are limited by how fast they can pull the right data from memory and feed it to the GPU, not the actual power of the GPU itself.
Expensive GPUs sit idle whenever they wait for the data to come from memory, which is obviously bad for revenues and profit margins. Four, memory bandwidth. Another bottleneck is how fast you can move data between tens of thousands of GPUs across many different racks. The whole system is only as fast as the time it takes to get the last answer from the last GPU working on a problem.
AMD and Nvidia have very different approaches to this one, which I'll also cover in this video. Five, latency or response time. It's not enough to generate good answers. Users want them fast, especially for real-time applications like search, coding, investing, video games, or robotics. AI models feel broken or unsafe if outputs don't show up in fractions of a second, which is a pretty tall order when users also expect models to think longer and provide better answers via reasoning.
And the sixth one might be the biggest bottleneck of all, power, cooling, and the grid. AI racks require hundreds of kilowatts each, and data center power demand is projected to grow by 20 to 30% per year because of AI, while power and cooling capacity is lagging far behind. As a result, AI systems need to generate way more tokens per watt and tokens per rack, not just tokens per second.
I'm not kidding when I say that there's a lot of money to be made by solving these challenges. For example, I made video after video covering Micron technology last year, ticker symbol MU, because their high bandwidth memory is directly addressing some of these challenges. And as a result, they're already sold out for over a year in advance.
Micron stock is up by 250% in the last 6 months and up almost 40% just since my last video on them from right before Christmas. I'm not trying to show off here. I'm trying to show you the direct connection between figuring out what a high growth market needs, like the solutions to the six AI bottlenecks I just covered, and making money by investing in those solutions.
And it's pretty funny how everyone's looking for the next big thing while the current market winners are still literally printing money just like Micron. But there's something else on the market that you need to know about, and that's your private data. There are hundreds of online data brokers making big money by collecting and selling your personal information.
That's why I've been using this video sponsor, Delete Me, for more than a year now, and I can't recommend them enough. Delete Me is a hands-free subscription service that will remove your personal information from those online data brokers. They give you a quarterly privacy report showing everything they've done, and they've reviewed almost 55,000 listings for me so far.
But what really surprised me is these data brokers had way more than just my private data. They had my wife's and my entire families, too. That's another reason I really like Delete Me. They have a family plan, so we can all have more control over our personal data. So, if you care about your data and your family's privacy, you can get 20% off any consumer plan with my code symbol 20 by going to joindeleteme.com/syol20 or with my link in the description.
And a big thank you to delete me and to you for supporting the channel. All right, so there are six major challenges for AI today. The insane growth in model sizes and token counts, the big bottlenecks in latency, memory, and network bandwidth, and increasingly scarce access to power from the grid. The special thing about Vera Rubin is it's the first time that Nvidia co-designed six different AI chips with all of these challenges in mind.
When I was at CES, I got to interview Joe Delair, Nvidia's product lead of AI infrastructure. And I straight up asked him how much more performance Reuben actually delivers because of the six chip approach. Now, here's the big question, right? From Blackwell to Reuben, talk to me about the performance gains at the rack level.
10x 10x the 10x more tokens per second per megawatt or per watt um and that's going to be a rack level kind of uh performance metric uh between Vera Reuben and Blackwell it's about 70% more transistors in terms of all the different chips that we we co-design but we're getting the 10x more performance per watt
so if you were just Morris law it would only be a 70% jump not a 1000% jump from yeah not a 10x. So so this kind of a all these different chips being designed together working together to maximize that performance that's the most amazing thing about this generation and the future generations. Now, normally Reuben being 10 times faster and more power efficient than Blackwell would be worth its own video, but as Joe walked me through each chip, I noticed a clear pattern,
but is also doing u other kinds of things like database analytics and those types of functions that are more CPU friendly. uh it's able to accelerate those types of effort. Oh, so really the whole idea is kind of like you let the GPUs do what they're the best at. Then you have the CPU to do things obviously that CPUs are much better at GPUs at so that you can sort of spread out the work over the right chip for the job.
Right. That's correct. Nvidia got these performance gains by actively taking work away from the GPUs so that they're only doing the most important work possible. And this is where the real trouble starts for AMD. Let me show you why. Nvidia gave Reuben a ton of memory capacity and bandwidth. Each GPU has 288 GB of HBM4 memory and around 22 terabytes per second of memory bandwidth.
I know that sounds like alphabet soup. So, let me put it this way. If you could fit Netflix's entire video library in memory, a single Reuben GPU could scan through every single frame of every single Netflix video in just over 2 minutes. And that's limited only by this memory bandwidth, not the power of the GPU.
Reuben has 50% more memory and almost three times the memory bandwidth per GPU versus Blackwell. That's already pretty bad for AMD since one of their big selling points is having more memory and bandwidth than Nvidia's Hopper and Blackwell chips. In fact, AMD's whole philosophy is cramming the biggest, densest AI models on as few GPUs as possible.
Their big idea is if you can fit a Frontier model on a smaller number of GPUs, you can save big on networking costs and you can write simpler software to serve it to end users. And AMD's new MI455X Helios platform doubles down on this idea by giving each GPU a whopping 50% more memory than Reuben, albeit at slightly lower bandwidths.
The problem is that this idea doesn't scale. Context windows will keep growing, but AMD can't just keep adding more and more high bandwidth memory forever. High bandwidth memory is expensive. It's power hungry, and there's only so much supply. So, at some point, adding more to every GPU stops being a profitable option.
But Nvidia took a fundamentally different approach. Reuben adds a second layer of memory specifically for inference context. But this memory pool is at the rack level instead of in every single GPU. This inference context memory storage or ICMS stores all the static stuff like past tokens, chat histories, static reference documents, things that don't change during a conversation with an LLM.
So now every GPU in the rack can pull from this separate shared memory pool instead of having to recomputee or hold everything in the GPU's much more expensive high bandwidth memory. This does a few big things that nobody is really talking about. First, it frees up multiple terabytes of high bandwidth memory per rack, which is like 10 to 20 Reuben GPUs worth of memory.
Remember, there are only 72 GPUs in a rack. So, this is a straightup memory gain of 15 to 30%. Second, it's around five times more power efficient because it moves all this data to cheaper low power hardware. Remember, AMD is storing everything in the GPU's high bandwidth memory, which is the most expensive place to put it.
But Nvidia's rack level memory system uses solidstate drives managed by DPUs, data processing units. They're not just cheaper, they also use much less power. And third, it ends up being around five times faster because all the context is stored in one shared memory pool. So separate GPUs don't have to repeatedly pull or recomputee the same data over and over and they spend much less time sitting idle and waiting for that data to arrive.
So this separate inference context memory storage layer orms is Nvidia's solution to the high bandwidth memory problem. Just like with the GPUs, they're using HBM only where it's needed and they're offloading everything else they can to cheaper, lower power shared memory. And just to be clear, it would take AMD years to copy this idea, if they can do it at all.
Nvidia can only do this because they control every chip and piece of software involved in the chain. They control the Reuben GPUs, the Vera CPUs, the Bluefield DPUs, the Spectrum X switches, and the massive CUDA software stack that sits on top of them. AMD definitely has strong GPUs and CPUs, but they rely on separate partners for their switches.
They don't have any equivalent to the blue field DPU and Rockom is nowhere near as tightly integrated into their ecosystem as CUDA is in Nvidia's. So if AI model context windows keep exploding in size like they have been, AMD will need to find another way to scale their memory while Nvidia's is already in full production.
If you made it this far into the video, you're probably a long-term investor who likes learning about advanced technology, thinking about the future, and investing in the companies making it happen. So, let me point out a few things that are probably already obvious to you, but now you have a lot more context.
First, hit that like button and subscribe to the channel if you haven't already. That way, you'll see more content like this, and so will more investors like you. Thanks. That really helps the channel out. Second, this will take some time to play out just like everything else I cover on this channel.
But let me be specific on what I predict happens next. And remember, this is just my opinion. Nvidia's Vera Rubin will be commercially deployed in the second half of 2026, and AMD's Helios racks will come out in the third quarter. Those designs are already locked in, and I'm sure both companies will see big commercial success with those systems.
However, compute requirements for model training are growing by around 5x per year, and newer reasoning models are pushing five to 10 times more tokens per prompt versus older ones. If those trends continue, and I do expect them to continue, AMD has one, maybe two product cycles before, their strategy of simply adding more HBM runs into hard economic and physical limits.
Realistically, they might be able to add 50% more high bandwidth memory to their GPUs one or two more times before their chips start colliding with power, cost, and packaging constraints. GPU architectures typically take 3 to 5 years to go from concept to production. Memory road maps and advanced packaging often need to be co-designed at least one generation in advance, which works out to about 2 and 1/2 years of lead time.
And rack level systems can take another 2 to 3 years to go from concept to validation and deployment. So if AMD needs a new rack level memory solution by 2030, they probably want their silicon in test labs by 2028, which means the architecture and packaging concept have to be locked in in 2027. And that means they have to start working on this problem right now.
I'm excited to see what they come up with, but don't say I didn't warn you. And if you want to see more science behind the stocks like this, check out this video .
Comments 0