Meta and Google announce new in-house AI chips, creating a ‘trillion-dollar question’ for Nvidia fortune.com Apr. 11th, 2024 Hardware is emerging as a key AI growth area. For Big Tech companies with the money and talent to do so, developing in-house chips helps reduce dependence on outside designers such as Nvidia and Intel while also allowing firms to tailor their hardware specifically to their own AI models, boosting performance and saving on energy costs. These in-house AI chips that Google and Meta just announced pose one of the first real challenges to Nvidia’s dominant position in the AI hardware market. Nvidia controls more than 90% of the AI chips market, and demand for its industry-leading semiconductors is only increasing. But if Nvidia’s biggest customers start making their own chips instead, its soaring share price, up 87% since the start of the year, could suffer. “From Meta’s point of view … it gives them a bargaining tool with Nvidia,” Edward Wilford, an analyst at tech consultancy Omdia, told Fortune. “It lets Nvidia know that they’re not exclusive, [and] that they have other options. It’s hardware optimized for the AI that they are developing.” Why does AI need new chips? AI models require massive amounts of computing power because of the huge amount of data required to train the large language models behind them. Conventional computer chips simply aren’t capable of processing the trillions of data points AI models are built upon, which has spawned a market for AI-specific computer chips, often called “cutting-edge” chips because they’re the most powerful devices on the market. Semiconductor giant Nvidia has dominated this nascent market: The wait list for Nvidia’s $30,000 flagship AI chip is months long, and demand has pushed the firm’s share price up almost 90% in the past six months. And rival chipmaker Intel is fighting to stay competitive. It just released its Gaudi 3 AI chip to compete directly with Nvidia. AI developers—from Google and Microsoft down to small startups—are all competing for scarce AI chips, limited by manufacturing capacity. Why are tech companies starting to make their own chips? Both Nvidia and Intel can produce only a limited number of chips because they and the rest of the industry rely on Taiwanese manufacturer TSMC to actually assemble their chip designs. With only one manufacturer solidly in the game, the manufacturing lead time for these cutting-edge chips is multiple months. That’s a key factor that led major players in the AI space, such as Google and Meta, to resort to designing their own chips. Alvin Nguyen, a senior analyst at consulting firm Forrester, told Fortune that chips designed by the likes of Google, Meta, and Amazon won’t be as powerful as Nvidia’s top-of-the-line offerings—but that could benefit the companies in terms of speed. They’ll be able to produce them on less specialized assembly lines with shorter wait times, he said. “If you have something that’s 10% less powerful but you can get it now, I’m buying that every day,” Nguyen said. Even if the native AI chips Meta and Google are developing are less powerful than Nvidia’s cutting-edge AI chips, they could be better tailored to the company’s specific AI platforms. Nguyen said that in-house chips designed for a company’s own AI platform could be more efficient and save on costs by eliminating unnecessary functions. “It’s like buying a car. Okay, you need an automatic transmission. But do you need the leather seats, or the heated massage seats?” Nguyen said. “The benefit for us is that we can build a chip that can handle our specific workloads more efficiently,” Melanie Roe, a Meta spokesperson, wrote in an email to Fortune. Nvidia’s top-of-the-line chips sell for about $25,000 apiece. They’re extremely powerful tools, and they’re designed to be good at a wide range of applications, from training AI chatbots to generating images to developing recommendation algorithms such as the ones on TikTok and Instagram. That means a slightly less powerful, but more tailored chip could be a better fit for a company such as Meta, for example—which has invested in AI primarily for its recommendation algorithms, not consumer-facing chatbots. “The Nvidia GPUs are excellent in AI data centers, but they are general purpose,” Brian Colello, equity research lead at Morningstar, told Fortune. “There are likely certain workloads and certain models where a custom chip might be even better.” The trillion-dollar question Nguyen said that more specialized in-house chips could have added benefits by virtue of their ability to integrate into existing data centers. Nvidia chips consume a lot of power, and they give off a lot of heat and noise—so much so that tech companies may be forced to redesign or move their data centers to integrate soundproofing and liquid cooling. Less powerful native chips, which consume less energy and release less heat, could solve that problem. AI chips developed by Meta and Google are long-term bets. Nguyen estimated that these chips took roughly a year and a half to develop, and it’ll likely be months before they’re implemented at a large scale. For the foreseeable future, the entire AI world will continue to depend heavily on Nvidia (and, to a lesser extent, Intel) for its computing hardware needs. Indeed, Mark Zuckerberg recently announced that Meta was on track to own 350,000 Nvidia chips by the end of this year (the company’s set to spend around $18 billion on chips by then). But movement away from outsourcing computing power and toward native chip design could loosen Nvidia’s chokehold on the market. “The trillion-dollar question for Nvidia’s valuation is the threat of these in-house chips,” Colello said. “If these in-house chips significantly reduce the reliance on Nvidia, there’s probably downside to Nvidia’s stock from here. This development is not surprising, but the execution of it over the next few years is the key valuation question in our mind.” A Labs project from your friends at
Competition is going to get fiercely insane .....Intel as well has a new chip that is very competitive to the market that is due for release. ..It won't take much to have new AI companies trying out new chips instead of waiting and paying $40k for the new 200 nvda chip Nvda has 80% or so of the market ...if they lose just 5-10% of this market in the next 12 months earnings would be impacted significantly .... Amd this could also push nvda chip prices lower!
I wish the people who wrote articles would at least reach out to someone who knows what the bleep they're talking about rather than making up meaningless drivel like this. That level of laziness makes one question everything else they touch.
Maybe if you were an academic or a company insider. But as an investor or trader, what matters is not the facts but just a smidgeon of assurance. That would be enough to take the stock higher or bring it down to its knees.
Nvidea is overvalued because of its sheer monopoly potential, or so people once thought. Now that you have other contenders, albeit less talented than Nvidea, the future doesn't look so rosy for Nvidea.
It might take years for Nvidia's competitors to come on board. Soon there will be a handful of AI chip makers. Then the weaker AI chip makers will close shop. Look at the US Memory chip companies. Decades ago, there were more than 10 companies. Now there is only one in the US for the past decade (and the next decade).
Grok have the fastest AI chip and will go from 14nm to 4nm when Samsung open the fab in Austin. Not grok.ai.
LLMs are a black hole. But the hyperscalers are capable of lavishing the huge profits from their monopoly businesses on vendor financing for AI firms, designing AI chips and other such things for years to come, funding a whole downstream industrial base. It seems to be a feature of the current year that there’s far too much capital in the world, so there always tends to be some “sink” soaking up the excess. The last decade had the Vision Fund, this decade’s looks set to be AI botshit.
All the demand for AI chip is making TSM very happy indeed. Hardware wise, there is no shortage of candidate e.g. ARM, RISC-V etc. Nvidia AI chip is mainly for the data dude such at Google, Amazon etc. It's too good for the retail folks.