Who can “dump” Nvidia first?

Original source: Alphabet List

Author: Bi Andi

Image source: Generated by Unbounded AI

**OpenAI is also riding a donkey to find a horse, and wants to get rid of its dependence on Nvidia as soon as possible. **

According to Reuters, OpenAI has been discussing various solutions since at least last year, hoping to solve the problem of expensive and scarce chips. Among them, self-developed chips are one of the options, and this option has not yet been completely rejected.

Another option is to acquire a chip company directly. People familiar with the matter said OpenAI already has potential acquisition targets and has considered conducting due diligence on them. However, the report failed to identify the specific chip company.

**Coincidentally, another news comes out along with it-Microsoft will launch its first chip "Athena" designed for AI at its annual developers conference next month. **

According to The Information, citing people familiar with the matter, Athena will be used in data center servers, designed for training large language models, etc., while supporting inference, and can provide power for all AI software behind ChatGPT.

The cloud has become an important battlefield for large models, and Microsoft's two competitors in this field, Google and Amazon, already have their own AI chips. The launch of Athena will allow Microsoft to fill in its shortcomings.

The progress of Microsoft and OpenAI on chip issues is quite representative: in terms of roles, it was the three-party collaboration of Microsoft, OpenAI and NVIDIA that made ChatGPT a reality, which in turn triggered a new wave of global AIGC; in terms of time, next This month is exactly one year since ChatGPT was launched.

**The next focus of the large model competition seems to be "who can 'dump' NVIDIA first." NVIDIA, which has dominance in the chip field, has become a shackles that urgently needs to be freed. **

In 2016, OpenAI, which was only one year old, welcomed a distinguished guest, Nvidia CEO Jensen Huang. He personally gave the first lightweight small supercomputer DGX-1 to OpenAI. OpenAI can complete a year's calculations in one month with DGX-1.

Nowadays, people who are belatedly look back at Huang Renxun's signature on DGX-1 "for the future of computing and mankind" and exclaim the vicious eyes of the "leather-clad leader".

By 2019, Microsoft joined hands with OpenAI to build a supercomputer using tens of thousands of NVIDIA A100 GPUs. In this way, OpenAI contributed efforts, Microsoft contributed money, and NVIDIA provided infrastructure, using amazing computing power to support the research and development of OpenAI's large model, and finally worked hard to achieve miracles. ChatGPT was launched in November 2022, stunning the world.

OpenAI has become a star company, Microsoft is fighting fiercely with Google and others with its AI strategy, and NVIDIA's market value has soared from more than 300 billion US dollars in November last year to over one trillion US dollars today. There is a craze for big models around the world. As a "seller", NVIDIA has no worries about selling chips.

In July this year, Citi research analyst Christopher Danely pointed out in a report that Nvidia will occupy "at least 90%" of the AI chip market.

**However, in this "three-win" game, perhaps only Huang Renxun is completely happy. For "water buyers" represented by Microsoft and OpenAI, relying on Nvidia's chips has at least two problems. **

The first problem is that it is expensive. As for the supercomputer built for OpenAI, according to Bloomberg, Microsoft spent hundreds of millions of dollars on the project. Stacy Rasgon, an analyst at Bernstein Research, analyzed that ChatGPT costs about 4 cents per query. If ChatGPT's query volume grew to one-tenth the size of Google searches, it would require approximately $48.1 billion in GPUs and an additional $16 billion in chips per year to keep running.

The second problem is scarcity. Just in June this year, OpenAI CEO Sam Altman said at a conference that the shortage of chips has hindered the development of ChatGPT. Faced with customer complaints about API reliability and speed, Altman explained that most problems are caused by chip shortages.

The newly released Nvidia H100 this year is currently the most popular AI chip, but it can only meet half of the market demand. Nvidia H100 and A100 are both produced by TSMC. TSMC Chairman Liu Deyin explained last month that supply constraints are not due to a lack of physical chips, but limited capacity in advanced chip packaging services (CoWos), which is a key step in the manufacturing process.

**Liu Deyin also predicts that technical production capacity will be sufficient to meet customer demand in one and a half years, which means that the tight supply of AI chips may be alleviated by the end of 2024. **

While Athena may not launch until this year, Microsoft has been preparing for it for years. In 2019, when hundreds of millions of dollars were spent to build a supercomputer for OpenAI, Microsoft's Athena project has been launched. According to the news, Athena will be built using TSMC’s 5nm process, directly benchmarking Nvidia A100, and is expected to reduce the cost per chip by one-third.

**For Nvidia, the selfishness of Microsoft and OpenAI is a red signal. **

Microsoft is one of NVIDIA's largest customers, and there has even been news of "rounding up" H100's full-year production capacity. OpenAI is the most important weathervane in the AIGC field. The two companies' ambition to develop self-developed chips is a dark cloud over Nvidia's head.

Google was the first company to purchase GPUs on a large scale for AI computing, but later developed its own AI-specific chips. The first generation TPU (Tensor Processing Unit) was released in 2016, and was subsequently launched as the Google Cloud infrastructure Google TPU in 2017. Google has continued to iterate over the years. In April this year, it announced the details of TPU v4, saying it was 1.7 times stronger than Nvidia’s A100.

Although Google is still purchasing Nvidia GPUs in bulk, its cloud services already use its own TPUs. In this AIGC battle, the AI mapping company Midjourney and the unicorn company Anthropic, which has a ChatGPT competitor Cloude, did not purchase chips from Nvidia to build supercomputer like OpenAI, but used Google's computing power.

Another technology giant, Amazon, also acted quite early. It acquired Israeli chip startup Annapurna Labs in 2015 to develop customized chips for its cloud infrastructure. Three years later, it launched the Arm-based server chip Graviton. Later, Amazon launched Inferentia, Trainium, a chip focused on artificial intelligence.

**Last month, it was reported that Amazon would invest $4 billion in Anthropic. As part of the deal, Anthropic will use AWS Trainium and Inferentia chips to build, train and deploy its future basic models. **

In addition, other competitors of Nvidia are also launching attacks in the field of AI chips. AMD, Intel, IBM, etc. are successively launching AI chips in an attempt to compete with Nvidia's products. In June this year, AMD released Instinct MI300, which directly benchmarks NVIDIA H100 and is an accelerator specifically for AIGC. The number of integrated transistors reaches 153 billion, which is higher than the 80 billion of H100. It is AMD's largest chip since it was put into production. AMD even uses the strategy of being compatible with NVIDIA's CUDA to lower the migration threshold for customers.

It is undeniable that Nvidia still has almost a monopoly on the AI chip market. No competitor can shake its position, and no technology giant can completely get rid of its dependence on it.

But "reducing Nvidia's control" seems to have become a consensus, and external challenges come one after another. The news that Microsoft and OpenAI are developing self-developed chips is a new wave. Can Nvidia stand firm?

References:

1. Heart of the Machine: "Amazon just invested 4 billion US dollars, Google and others will invest another 2 billion, and Anthropic's valuation is soaring"

2. Sina Technology: "The shortage of AI chips is dragging down the revenue of technology companies. Nvidia H100 shipments are said to be at least tripled next year."

3. CSDN: "Having spent hundreds of millions of dollars and tens of thousands of Nvidia GPUs, Microsoft reveals the supercomputer past behind the construction of ChatGPT!" 》

4. Wall Street Insights: "Put down your pride!" How Microsoft is betting big on OpenAI》

5. Jiemian News: "Microsoft's self-developed AI chip "Athena" has surfaced, aiming to break Nvidia's computing power monopoly"

6. Yuanchuan Research Institute: "A Crack in the NVIDIA Empire"