저자: Nick·Kasey · 외국인
요약 보기
브로드컴은 AI 인프라 행사에서 맞춤형 실리콘(XPU) 사업을 강조했으며, 구글·메타 등 3개 대형 고객과 협업 중이다. 네트워킹 부문 매출은 33억 달러이며 FY 성장률 가이던스를 35%로 상향했다. 엔비디아는 AI 학습 시장 중심, 브로드컴은 추론·맞춤형 칩 중심으로 두 시장이 다르다고 설명한다. 데이터센터 전력 소비 증가가 투자자 관심사로 제시됐다.
다듬은 글
Nick: Broadcom is working really hard to eventually help its customers scale up its own XPU clusters for inference. To make a custom chip with scaling those up to possibly 1, 000, 000 or more XPUs. Kasey: Welcome back to Chip Stock Investor. As promised, we are covering Broadcom today. Broadcom and Nvidia have emerged as two secular growth leaders for this new bull market, each with its own take on combining chip design and hardware design with a software stack running on top of it.
But what exactly is driving Broadcom right now? We're going to break this into two parts, given the broad business that Broadcom has these days. Semiconductors and infrastructure software. For all of our new viewers, full disclosure here, Nick and I have owned Broadcom in our portfolio for many years, long before AI or generative AI or accelerated computing, whatever you want to call it, entered the mainstream consciousness.
We've got numerous videos explaining what Broadcom does, and we'll link those up here as well as in our video description so you can get a little background on it. Nick: So to kick this off, we should probably start with the semiconductor segment. Broadcom's historic bread and Butter. CEO, Hock Tan and company tried to steal some of NVIDIA's thunder last week during the GTC event.
On March 20th, 2024 Broadcom had its own AI event called Enabling AI infrastructure. And in typical Broadcom fashion, it looks like cost control was very much part of the presentation. But low budget video production aside, we're in that boat too, no judging. Let's talk about the enabling AI infrastructure.
As Kasey promised before we delve into that, take a look at our last Broadcom quarterly review video link here. We explained the high level financials and some of the goings on at Broadcom alongside baby Broadcom, as we like to call Marvell Technology group just a few weeks ago. But high level overview, here is the last quarter that we have on hand for Broadcom Q1 fiscal 2024, which ended in January of 2024.
And that big 34 percent growth rate in overall revenue driven, of course, by the VMware acquisition, which is now lumped into infrastructure software. But again, we'll start with the semiconductor segment there on top. The two big segments at Broadcom are wireless, which is mostly Apple, but as respects to AI, most of it is housed within this networking segment, which we have highlighted.
$3. 3 billion in sales last quarter. And their full year growth rate, they significantly upgraded their expectations for this part of their chip design business to 35 percent full year growth. AI infrastructure build out is the reason why they have this expectation. Late last year, ahead of the VMware merger, we said Broadcom was going to be close to a 50 50 split between semiconductors and infrastructure software.
Well, thanks to that AI growth in the networking segment, it's more like 60 40 again, 60 percent semiconductors. 40 percent software sales. So high level overview. Let's talk about some of the details from that enabling AI infrastructure event. Kasey: The big highlight from the Enabling AI Infrastructure event was Broadcom's custom silicon.
Now, in the real world, there's a very fuzzy line between merchant and custom silicon, but for Broadcom's purposes here, and for our discussion, custom silicon is in reference to these multi billion dollar, multi year projects for very specific customer use case. Only a small handful of these companies have the resources available to take on such a project.
And Broadcom has 3 of those, that they talked about at this conference. It's been confirmed that Google is one of Broadcom's largest customers, and we believe that the second customer is probably Meta that they're speaking of. Meta, of course, includes WhatsApp, Instagram, Facebook. And then that third customer that they just announced is a new one.
It could be a few different companies, there's not many companies that would have the resources to do this to do custom silicon, but possibly Amazon, Tencent, Microsoft, or Open AI. Those are all possibilities for this 3rd customer. Nick: Now, what we do know for sure is, all three of these customers, including the third one, can be categorized as a consumer internet or consumer AI business.
So why are they called consumer internet or consumer AI businesses? Well, Google, of course, used by billions of people every single day to do internet search. Meta for much the same reason. We call it social media, but essentially it's just the Internet and it's social search, video search to an increasing extent.
And as these search requests, these data queries from consumers, from users, billions of them every day. Many of these requests are growing in complexity. For example, for Meta, we have like the Reels business, the videos, as Meta continues to roll those out and push that content as it tries to compete with TikTok, obviously video, a much more data intensive, Workload computing workload than just, a simple, social interaction, like a text or a chat.
These consumer AI and consumer internet companies, there's only a handful of them out there that have the resources available to dump money and resources into research and development. They provide the data and the workload architecture that they're working on in their data center to Broadcom, kick in a couple billion dollars in cash, and then together, they go to work designing this custom silicon.
In this case, a custom accelerator, an XPU, which is really a GPU, like what NVIDIA makes, a graphics processing unit, or just a computing accelerator, these days, let's just call it that. An XPU is essentially just that, but customized for a very specific computing workload, a very specific design.
Kasey: Now, Nick, in our comments here on YouTube, and we've had some questions over on our Discord community as well about power consumption. Nvidia and Broadcom both talk about the total cost of ownership and power consumption. Can you explain a little bit more to us about the two different products that they have available?
And their power consumption, because this is obviously a big topic. Morning Brew just called out the electricity usage by data centers, from a report in the Washington Post just today. So it's obviously on the minds of a lot of investors. Nick: Absolutely. And for good reason. We actually had a conversation with Nvidia, old video at this point from a year and a half ago, where this, we asked the same question.
It was brought up at the time that, data center consumption of electricity is headed towards currently probably a mid single digit percentage of total energy usage to possibly 10 percent or higher in the coming years. So it's a big topic. I think as far as total cost of ownership goes, it's important to remember, we're actually talking about two different markets and we'll hammer this home again at the end of this segment.
We have the brand new AI training data center market or the AI factories as Jensen Huang likes to call it. A lot of that is just merchant Silicon. NVIDIA designing these big AI systems and selling them as is. It's a much simpler data center to build and operate. If you're just purely going to train the AI.
But after the AI is trained, you have to now move that algorithm over to an existing data center. Let's say that it's oftentimes connected to, if not the Internet, a public network, then maybe a private network for use by the business itself. Either way, some sort of network connection to that data center where the new AI software resides, and then users start asking it to do things.
And this is where the inference market is headed. It's possibly going to be as big or bigger than AI training. And this is where a lot of these bottlenecks are taking place. And this is where Broadcom is doing custom silicon design to solve for some of these problems. So it's important to, I think, separate the two, because your total cost of ownership for an AI training is going to be different from your total cost of ownership for your inference.
Kasey: Broadcom's XPU is the heart of its custom work. It's a custom accelerated computing chip for a very specific workload. Broadcom provided this diagram on how an XPU works in a very complex cloud environment, which is part of a compute offload branch of the data center where certain traffic is rerouted and accelerated.
You can think of this as maybe Google search or Meta's expanding video recommendation system, or maybe a product search on Amazon's e commerce marketplace. The key is embedding the compute directly in the network itself. And Jensen Huang actually talked about this at the GTC as well, and this helps prevent those bottlenecks from happening when massive amounts of information are needed to be computed.
Sometimes what happens is the network can't fuel a dedicated compute accelerator with enough data and so GPUs remain idle. And that increases the inefficiency of the customer's investment. If the workload is accelerated to the network itself, it can prevent those bottlenecks and keep the data center working closer to 24 7.
This is illustrated differently in this chart. You can see those multiple spokes coming off of the spine in order to accelerate that compute and requests made from the user. Nick: That's right. We did our own little mock up here of Broadcom's chart, illustrating how this is happening as these big consumer AI and consumer Internet companies begin to delve more and more into more complex software provided to their users.
So maybe you have this bulk of compute requests entering a data center, again from the Internet or from a private network. And you have this point in the spine of the data center, which holds all of those racks of servers. You have this point in the spine that decides, okay, we have this batch of new AI powered inference work.
We're going to split that off and send it to this new part of the AI infrastructure part of the data center with the XPUs in it. And then the old, let's call it the old cloud compute workload continues on to the, the main part of the data center or the existing part of the data center before, so that it doesn't get clogged up.
And then that new AI infrastructure takes care of the more complex workload with the XPUs. This is how a compute offload system works. And this is that primary customer. That first customer Broadcom has had for years now. it's the Google TPU or tensor processing unit. Kasey: To be clear, this is very different than merchant silicon.
Broadcom is highly complimentary to NVIDIA as it helps customers build out data centers for AI training and later on that inference of those AI systems. NVIDIA has InfiniBand, which was acquired via Mellanox. Broadcom and its peers are hard at work using Ethernet. We've talked about the Ultra Ethernet Consortium before on our videos.
This is what Broadcom continues to rely on. Which architecture a customer chooses will depend on the needs. And the total cost of ownership of running those merchant silicon chips is based on their workload expectations. As you can see from the pictures taken, the XPU doesn't look a ton different than NVIDIA's Blackwell system, but Broadcom isn't going to show too much given that these are proprietary to their custom silicon customers.
They did say they crammed in more HBM or high bandwidth memory into the XPU system. That doesn't necessarily mean superior to Blackwell, it simply means that a customer needed more memory next to the network accelerator for its specific workload application. Nick: As you've probably heard by now, NVIDIA is talking about the biggest GPU data center clusters, especially for AI training, which is again, largely merchant silicon off the shelf, can have as many as 32, 000 GPUs networked together.
Picture of that here again from NVIDIA's GTC event showing the new Blackwell platform using NVIDIA's proprietary InfiniBand. Again, on the custom silicon side, using XPUs, XPU clusters within the network itself, broadcom is working really hard to eventually help its customers scale up its own XPU clusters for inference.
AI inference and custom architecture specifically for these workloads that they have very large volumes of and have the resources to make a custom chip with scaling those up to possibly 1, 000, 000 or more XPUs. So big scaling of these data center systems coming in the next 5 to 10 years. Kasey: We'll go back to a chart that we shared earlier and you can see that much of Broadcom's revenue is housed in that big networking segment and a large portion of that is from merchant silicon.
But custom silicon is growing and that's what's aiding this massive growth that Broadcom is expecting this year as these big tech customers dump billions of dollars into R& D into custom silicon solutions. We don't need to mix this with NVIDIA's merchant Silicon GPU platform because these are different businesses addressing different parts of the fast expanding and very fast evolving generative AI industry.
Nick: Yeah, I promised we'd emphasize this again one more time. And so let's break it down because there are two parts to this market. First is that brand new AI training market that NVIDIA dominates. More than 90 percent market share. Much of it is NVIDIA's own merchant silicon. It's purchased off the shelf, usually as big systems packaged up with NVIDIA's proprietary software.
In the next few years, that market is expected to be as much as 500 billion in total value, and eventually perhaps 1 trillion in total value. These are the big AI factories that you hear NVIDIA and Jensen Huang talk about. The second market is after the AI has been trained, and it could eventually be even larger than the AI training market that is now being built.
What's happening here is this is the current existing data center. It's currently worth roughly 1 trillion in global data center, installed base that's out there in existence. And in order to handle all of this new AI software that is now being trained and really just beginning to be deployed, a lot of these big internet companies now need to add on or revamp their existing data center fleet with new AI infrastructure that can accelerate those workloads, some of it might be merchant silicon that they can buy off the shelf from NVIDIA and from Broadcom and from AMD and whoever.
Some of it, though, if the workload is big enough, and the company has enough resources that it makes sense for them to do it, some of it will be custom silicon. That the likes of Google, Meta, whoever decides to dump the money into and build a custom silicon accelerator, custom XPU to handle some of these very specific workloads.
So again, two different markets, AI training. AI inference. and the AI inference is going to include the existing data center market, that's already worth 1 trillion globally. Kasey: All right, we'll say that's a wrap on this video regarding Broadcom, but we have another one coming up. In our next video, we will focus on Broadcom's software portion of the business, especially after that acquisition of VMware.
And we'll talk about that in light of our cybersecurity industry flow chart. So make sure you check that out. That'll be our next video. And if you have not already, please subscribe to the channel so that you do not miss a video. It helps us out a lot when you just hit that subscribe button and, if you would like to have more access to more content, please join our membership here on YouTube or over on our Ko fi page and check out our manuals there as well in our Ko fi shop.
We have manuals and video notes for purchase a la carte. We'll see you again soon at Chip Stock Investor.
Comments 0