The Singularity Moment, the Last Carnival of the Internet

Original source: AI Whale Selection Agency

Image source: Generated by Unbounded AI‌

In 2023, news that the well-known investment institution Tiger Fund failed to raise funds quietly spread throughout the Internet.

In the past 10 years, when I got used to starting a business in the wind, it seems that the "investor winter" has appeared for the first time. This is closely related to various unfavorable factors, such as new consumption, live streaming, and Metaverse. The exit channels such as mergers and acquisitions and Chinese concept stocks are half-closed. The venture capital market seems to be really deserted.

It is difficult for start-up companies to raise funds, and it is difficult for the boss to find a direction for his second venture. Wang Huiwen, who retired from Meituan, has studied Web3 and Metaverse for a long time. Wang Xiaochuan, who left the company after being acquired by Tencent, tested the waters of AI medical care. But everything changed at the end of 2022, when ChatGPT 3.5 was released, which quickly formed a consensus in the market. The era of AGI (General Artificial Intelligence) came, and the whole industry began to run into large-scale models.

It is understood that Wang Xiaochuan, who was starting a low-key business at the time, had already established a company to make smart hardware. Intended to help hundreds of millions of people with sleep disorders, create a smart pillow to treat snoring. When the upsurge of large-scale models emerged in March, Wang Xiaochuan spent two weeks making a decision to put down this entrepreneurial project and lay out large-scale models.

Wang Xiaochuan recruited former Sogou CTO Yang Hongtao to help take over the medical project. The former Sogou COO Ru Liyun's shares in this company were also exchanged to Yang Hongtao, following Wang Xiaochuan's large-scale business model. Wang Xiaochuan spent a total of 50 million U.S. dollars to establish "Baichuan Smart", and invited Soul's technical talents to be the person in charge of the algorithm to speed up the production of large models. And the story of Wang Huiwen, everyone is very familiar with it. Hero recruitment posts were posted on the wine table, and a company light years away was established to make a large model.

In the big Internet companies, the big model has also brought earth-shaking influence. The person in charge of the project with a large model proposed to resign a few years ago because of the problem of failing to be promoted. Three months later, the CEO of the group became the general manager of the large model, using the company's strength to All in the large model.

No one wants to miss this wave of the AGI era. Everyone believes that after three ups and downs in the development of AI, the singularity of general artificial intelligence is coming. After all, under the AI upsurge, dozens of companies like ChatGPT and Midjourney have created a valuation of about US$4 billion. The total market value of the "Big Seven" in the US stock market has soared to US$11 trillion a year, a surge of 60%. These exciting stories of explosive growth have once again stirred up the domestic technology business market.

**Among the major Internet companies in China, Li Yanhong, Zhang Yong, Zhang Yiming, Wang Xing and other big names have already personally taken command. It can be said that except for Pinduoduo, all of them have entered the big model. **As on July 19, the market value of Microsoft and Nvidia increased by $175 billion, Musk marveled when evaluating related tweets: "Crazy times."

The technology and business market that is about to be silent suddenly ushered in the stimulant of AI, which also brought the Internet into its last carnival.

1. A New Dawn in the Low Valley

Li Ming is the CEO of a start-up company with a team size of more than 100 people. 2023 is the year he is most worried about financing.

The process of starting a business was very smooth at the beginning, and got angel and A-round financing from well-known angel investment institutions in the early stage. "At that time, the Industrial Internet was still a popular track, and it was not as exaggerated as many AI projects." Li Ming told AI Jingxuan, but in the middle of 2023, in the new round of financing he launched, he slowly discovered that the market was not right. .

Investment institutions not only look at data and stories, but also look at revenue. Li Ming, who was obsessed with productization before, has not realized the change of investment wind direction at all. According to the speech of Wu Shichun, the founding partner of Meihua Venture Capital, the current investment projects "need not only (technology), but also (data) and (revenue)". No way, he started to find FA institutions to help with financing, and the financing rounds also regressed, asking for an A++.

"FA helped find more than 30 investment institutions, but nothing happened." The failure of financing made Li Ming a little discouraged. But in June, he felt the power of the big model, so he launched an industrialized business based on ChatGPT internally. "We haven't raised any funds yet, but investors will take the initiative to communicate with them, and the other party is obviously interested."

For Yuan Jinhui's first-class technology, the large model is also a life-saving straw. In 2022, this company, which makes AI deep learning framework, has reached the point where financing is not smooth and has to lay off employees to survive. The company was on the verge of breaking its capital chain three times before, and they all borrowed money from Su Hua, an angel investor who was also the CEO of Kuaishou at the time.

"What we do is similar to Baidu's Flying Paddle and Huawei's Shengsi. The most important thing is that the business of large-scale model training in the market has not yet started." A top-notch technology employee told AI Whale Selection Agency that the company belongs to the time when it is rich (2021) ) has no business, and when there is business (2023), there will be no money.

Just when Yuan Jinhui felt that the future was hopeless, the company also ushered in an acquisition opportunity in 2023. In April 2023, in the first-class technology company in Tsinghua Science and Technology Park, a distinguished guest was welcomed. He was Wang Huiwen, the co-founder of Meituan, who had just announced his entry into the large-scale model.

The final purchase price was not bad. A first-class technology employee who was laid off told AI Whale Selection Agency, "It is comparable to the valuation of the last round of Hillhouse Capital investment, and my own options have also been found."

And Yuan Jinhui, who became the co-founder of Light Years Away, finally no longer has to worry about financing. Wang Huiwen's financing ability is second to none in the current venture capital circle. According to the subsequent acquisition agreement of Meituan, Light Years Away raised 2 billion yuan without large-scale model products.

Of course, the investors who deployed earlier in this wave of actions have successfully hunted unicorns.

Minimax was established in November 2021, received angel round investment in January 2022, and the company's valuation reached unicorn level in early 2023. Among the first four investment institutions, there is also the Shanghai game company Mihayou. It is reported that the two founding executives have family ties. According to Jingxuan News, Zhipu has also recently raised funds at a valuation of 10 billion yuan.

**These two companies have been established for less than 2 years, but they have both become unicorns, and the development speed of the large-scale model track is amazing. **

And the AGI boom is also a salvation for those old AI companies. Previously, the story of going out to ask about the IoT listing has gone through several times without success. With the release of the story of the large-scale model "Serial Monkey" and four AIGC products, although the large-scale model is still careful not to be publicly evaluated, it also allows Momenwen to finally have a new story to tell, and the application for listing on the Hong Kong stock market has been submitted.

More large-scale models and AIGC entrepreneurs are on the road. Even in a startup camp, 60% of the projects are related to AI. With the advantages of light assets, high barriers, and high ceilings, AGI has completely become the hottest competition at the moment. road.

**2.Put the dream of AGI to its peak

If 2023 is the "first year" of large-scale model entrepreneurship. Then the "source year" when the Internet giants first entered the big model can be traced back to 2019.

Ali began to lay out the large model in September 2019, and released the PLUG large model in April 2021. Before the launch of ChaTGPT 3.0, there were already many large models with trillions of parameters in China. They are the M6 of Bodhidharma Academy, the Pangu model of Huawei Cloud, and the Enlightenment 2.0 of Zhiyuan. Compared with ChaTGPT, although the model parameters are surpassed, the data abundance is not the same, and the effect cannot be compared. According to Zhang Cong of Dharma Academy, the most important thing is that the domestic large models are not done two things.

The first thing is that no alignment is done. At that time, Ali had many large and small models, and mainly did not align the training results. "You can see that ChatGPT can compose poems and chat, which is very similar to human intelligence. In fact, it is aligned with human values." Zhang Cong said, all of these require artificial adjustments to the reasoning results, rather than machine logic. .

Second, there is no high-quality data set. ChatGPT used university professors in the Philippines for data labeling in the early days, while domestic technical secondary school students were used for labeling. The problem of corpus also greatly affected the results. In Zhang Cong's view, the fine-tuned Chat model of Llama 2 released on July 19 was trained on 1 million human-labeled data, and the total number of training tokens increased by 40%. Compared with Llama, the improvement is all-round. "**So the large model is not an invention of miracles, but a well-designed engineering creation." **

And looking back at the domestic AI industry, it will also face interference from many other factors. At that time, Bodhidharma Academy had two main teams working on large-scale models, one was the machine intelligence team led by Jin Rong, and Si Luo was in charge of AliciMind; the other was the natural language laboratory led by Zhou Jingren, of which Yang Hongxia was in charge of the large-scale model M6.

In the evaluation at the end of 2022, the results of the M6 large model have a slight advantage, and the two are finally integrated into the current Tongyi large model. "Actually, there are only 20 or 30 people in the big model team of Dharma Institute, and his pre-training is mainly placed on Alibaba Cloud." Zhang Cong told AI Whale Selection Agency, but now Tongyi is an important project of the group. There are more than 600 people, and many resources are now devoted to large models. The CEO of the group asks about technical progress every 2 weeks.

For Baidu, this wave of AGI boom, but the AI era that he has predicted since 2016, will naturally not miss it.

It was officially approved internally on February 7 this year and officially released on March 16. During this period, it was directly promoted to the highest priority project of Baidu Group. Li Yanhong personally supervised the battle, and CTO Dr. Wang Haifeng directly took charge. At that time, Baidu Yangquan Supercomputing Center was dedicated to large-scale model training.

Baidu algorithm engineer Zhao Hui told AI Whale that the Baidu Natural Language Processing Department has been researching NLP and other technologies, and the chief scientist Wu Hua has also been the leader. There are hundreds of people in this department. Baidu's ERNIE2.0 has been transformed into a large model of Wenxin, "I used to make Baidu brains, but now it is said to be a large model of Wenxin."

There are similarities in what they do, but of course there are differences. Zhao Hui mentioned that in the past, Baidu would do a lot of vertical search Rank, just to reorder the search results based on human clicks. After the emergence of the large model, these capabilities will be deposited in the algorithm of the large model, which will also help to give more accurate answers.

For Baidu, the large model promotes qualitative changes in the next generation of search, which has been written into Robin Li's OKR. However, in terms of ecology, Baidu’s Wenxin model is based on the Bert model. “GLM, including Zhiyuan’s, is an independent technical route, which is different from the international GPT.” A Baidu cloud employee told AI Jingxuan In fact, there is no need to worry about this. Wenxin Qianfan has all types of models, and GPT2, 3, and 4 are also very different.

As for Yang Hongxia, who resigned from Ali, she was also hired by ByteDance to be the head of research and development of North American large models after she went overseas. Zhang Yiming has been researching whether the large model will be open source or closed source, so he didn't ask to concentrate on the work. "There will be a real breakthrough before the end of the year." Yang Hongxia told AI Whale Selection Agency.

On the whole, ByteDance should be a company that better matches the big model in business after Baidu. A headhunter told the AI Whale Selection Agency that although the big model is not in a hurry, it is still quite radical in the field of AIGC. For example, Tiktok is doing advertising creative business AIGC, the director position gives a budget of 100-150W, and the requirement is to lead the team after 1988.

So far, except for Pinduoduo, all major Internet companies have entered the big model. The enthusiasm of big manufacturers to enter the game is even more than that of O2O and live broadcasting back then.

**3.The night when the watershed suddenly appeared

In June, in Sohu Building in Beijing, this large-scale model company with the most financing is in full swing.

The Oneflow deep learning framework of the original first-class technology is still thinking about continuing to do it, but many people have been transferred by the large model business. But on June 23, someone suddenly revealed on social media that Wang Huiwen was ill. At that time, someone from the company went to verify it, but they got no such news. However, on the evening of the 25th, Meituan suddenly announced that the co-founder Wang Huiwen was hospitalized due to depression and resigned from the company's director.

For a while, the news that Wang Huiwen ran away early became the guesswork of some people. The news Jingxuan got from the investor circle is that Wang Huiwen's condition is indeed very serious. In the end, Wang Huiwen's brother who slept on the upper bunk, Wang Xing, the founder of Meituan, helped to take over the business light years away.

Is the big model really dead? Everyone has this question. During that period, Zhu Xiaohu, a well-known investor, and Fu Sheng, the founder of Cheetah, were also arguing in the circle of friends whether there was a bubble in the large-scale model industry. Zhu Xiaohu is extremely pessimistic about the current situation of the market flocking to make general-purpose large-scale models, and believes that most of them will die by the end of the year.

Does the active change that is light years away also confirm Zhu Xiaohu's remarks?

According to the information obtained from the AI Whale Selection Agency, the acquisition of Meituan, which is light years away, has not stopped the pace of large models. Not only has it exclusively invested several hundred million yuan in Zhipu AI, but it is also currently recruiting project directors for large-scale models, with an annual salary of up to 3 million yuan, and even established a technology research institute in the United States. Meituan, which earns hard-earned money, does not want to fall behind in this wave of technology, especially after Ele.me clearly wants to connect to the Tongyi large model, and Ctrip, which has business competition, has also launched a large model.

But for the domestic market, there are indeed too many general-purpose large models. According to incomplete statistics, in less than 8 months, more than 85 large models have been released, many of which have become cash-out concepts of listed companies.

Wind data shows that in 2023, 24 "AIGC concept stocks" have undergone a total of 67 shareholding reductions, and the divorce wave of major shareholders is also amazing. Since the beginning of 2023, the families of major shareholders of nearly ten AI sector companies have been exposed to divorce. What has attracted much attention is that Kunlun Wanwei, an A-share AI company, recently reported that Ms. Li Qiong (the ex-wife of the founder Zhou Yahui), who accounts for 11% of the shares, plans to reduce her holdings by 3% of the shares (about 1.3 billion yuan), and then pay interest. lent to the company. According to the insider’s introduction to Jingxuan, Kunlun Wanwei, who has felt the benefits of AGI, not only made large-scale models, but also intensively formed a team recently, and went all out to make Copilot, which is a benchmark against Microsoft.

Listed companies use AGI to seize the concept of speculation and cash out. Big-model startups introvert to death.

Zhang Yang, an investor who recently established the AIGC Fund, told AI Whale that with the launch of the open source, free and powerful Llama 2, many large-scale enterprises will inevitably face financing difficulties in the second half of the year.

Now everything is already on the horizon. On July 11, Baichuan Intelligent launched the Baichuan-13B, a large model with tens of billions of parameters. It not only announced that it was open source, but also free for commercial use. Although the parameter scale of Baichuan-13B is not large, based on accurate Chinese corpus training, Baichuan often ranks first among large models with tens of billions of parameters.

The free strategy of Baichuan-13B has greatly impacted the paid market for large-scale models in China. At present, Zhiyuan AI announced on the 14th that the enterprise registration has been authorized to allow free commercial use of ChatGLM-6B and ChatGLM2-6B.

After more and more large models are open source and free, the death knockout competition for large models officially begins. A CTO of a start-up company based on large-scale models told AI Whale Selection Agency that Zhiyuan’s large-scale models cost 20 million yuan from the initial private domain deployment, and the calling price at the beginning of the year was 1.8 million to 300,000 packages. The industry is changing very fast. **Fu Sheng believes that this is the market's transition from a competition of large model parameters to a competition of ecological scale. **

Big Internet companies are not worried about ecological construction. Since there are many internal models, there are also free and paid models. The most important large models are still closed source and paid. It is more difficult for start-up companies to establish an ecology. Many start-ups have exhausted their efforts to build a large-scale model, and it is inevitable that they will not be able to do the ecology. It is understood that MiniMax is currently one of the few start-up companies that adheres to the public cloud and is a large-scale enterprise in the MaaS model.

Fan Kai, CTO of Clove Garden, described this wave of open source and free, like bringing water plants (big models) to users’ homes for free, so that everyone has a faucet. For those closed source water plants, it’s best that your water is invincible and delicious. Everyone is willing to pay to go to you.

**4.AGI development enters a fork in the road

After the emergence of the watershed, the former chairman of the technical committee of Jingdong Group and the founder of Lianyuan Technology told AI Whale that the current entrepreneurial competition has already developed three factions.

One group insists on fully self-developed large models, and this group is all powerful players. This faction is mainly composed of major Internet companies such as Baidu, Ali, and Byte, as well as start-ups such as Zhipu, MiniMAX, and Lianyuan. But these powerful players are also divided into two types of enterprises.

The first category is to insist on doing self-developed general-purpose large models, benchmarking against ChatGPT, and constantly catching up with the iteration speed of ChatGPT.

In the view of Chen Yu, managing partner of Yunqi Capital, general-purpose large-scale models are the only way to go, and the development of vertical large-scale models is limited. "Because for the general large model, the vertical field does not need to be retrained. The general large model can be deepened through the vector database, but the vertical large model is difficult to emerge intelligently."

Judging from the current situation, those who have a dream must make a general-purpose large-scale model. After all, it can become the next big Internet company. ChatGPT has demonstrated this disruptive potential in the fields of collaborative office, e-commerce, code generation, and auxiliary design.

The second category is to recognize the reality, focus on landing, and insist on making vertical large-scale models. This group includes Zhu Xiaohu and Fu Sheng who finally reached a consensus. Both of them believe that vertical large-scale models will have more industrial applicability.

General-purpose large models generally have more than 100 billion parameters, while vertical large models have a scale of around 10 billion or 7 billion. Similar to the large-scale model product ProductGPT of Lianyuan Technology and the large-scale model product Cao Zhi of Daguan Data, the parameters are in the tens of billions.

Lianyuan Technology is not a parametric school of large models. "We have the basic capabilities of general large models, but we believe that technical algorithms, model iterations, and scene closed loops are all more critical."

Compared with Lianyuan, which pays more attention to the scene, Daguan pays more attention to the data.

According to Chen Yunwen, CEO of Daguan Data, told AI Jingxuan that the "Cao Zhi" large model adopts a mixed training data scheme, which is 50% general mixed corpus + 50% vertical professional corpus. "We have been doing text intelligence in the fields of finance and government affairs for many years. A lot of data is unique, and customers also ask us to do privatization training." Chen Yunwen told AI Whale Selection Agency, "In the past, four or five people used to do one job a week. Report, now the AI is done in half a day."

Only by giving up the dream of becoming the next ChatGPT can it land as soon as possible in the commercial scene, which is the sober cognition of many vertical models.

** From the perspective of the industry, the second route to the AGI temple is based on other people's models (such as GPT), and then combined with their own industry Know-how for training. ** "I think it will take time to verify whether the second category can be successful. It is not yet clear. The reason is that everyone still does not know how to integrate the industry Know-how with the big model. At the same time, there is a sustainable business model, which is still unknown.”

"In particular, many large models are suspected of being shelled," Zhang Yang, an investor, told AI Whale Selection Agency. When combining industries for commercialization, they will face many problems. It is reported that the large models of two well-known entrepreneurs with the same surname are based on Facebook's open source Llama; a game and protection company uses the basic framework of the Zhiyuan large model.

A stronger open source model is coming. Meta recently released the free commercial version Llama 2. Fu Sheng said in the circle of friends: "I don't know how many companies wake up laughing in the middle of the night, how many companies cry in the toilet... "Zhu Xiaohu also commented on this: Many people are about to wake up, everyone can take a free ride."

Fu Sheng's "wake up with a smile" is slightly different from Zhu Xiaohu's. Fu Shengxiaoxing refers to companies that use open source large models to develop AIGC applications, and they can also produce better products. Zhu Xiaohu Xiaoxing refers to those large-scale model companies that claim to be self-developed but actually cover their shells. Recently, they will announce an upgrade . What everyone refers to is the same. They are all large-scale model companies that claim to be self-developed. Llama 2, one of the most powerful foreign large-scale models, announced that it is open source and free. Everyone is based on the same open source large model. How to make unique capabilities in the industry? ?

**The third route of AGI is pure application, which is to use the model directly, which will have lower barriers. **Zhu Xiaohu is also not optimistic about this type of model, thinking that if 90% of the capabilities are provided by ChatGPT, then AIGC applications have no investment value.

In the OpenAI ecosystem, Sam Altman promised to avoid doing the application layer as much as possible and compete with ecosystem developers, so he imitated Google and made the ChatGPT Plugin. From the current point of view, no one in China has made such a commitment.

There have been hundreds of function updates for Wenxinyiyan and Tongyiqianwen, and these functions have also covered the work of some developers. Although Wenxin Yiyan also has plug-ins, there are currently only two, one is Baidu Search, and the other is ChatFile (analysis and processing capabilities for long documents). How to coordinate the ecological development of Baidu Qianfan and Alimodao is still a challenging proposition.

Zhu Xiaohu, who is firmly optimistic about AGI, believes that the entrepreneurship and investment window for general-purpose large-scale models has passed, and AIGC, which does not absolutely rely on the capabilities of a certain large-scale model, is the low-hanging fruit of the AGI era. For example, the "Miaoya Camera", which has suddenly become popular in the circle of friends recently, is the first AIGC product to become popular in China, and a similar product "lensa" has also become popular abroad, with a monthly income of US$8 million.

When the wave of the times hit, investor Wu Shichun once jokingly asked Zhu Xiaohu: "Has the money earned from investing in AI made up for the money lost from investing in SaaS?" Zhu Xiaohu replied, not yet, but AI has unlimited prospects.

At present, the venture capital circle also recognizes the truth expressed by Zhu Xiaohu, but many people do not expect Zhu Xiaohu to tell the truth, such as "ChatGPT is very unfriendly to start-up companies, please give up financing illusions in the next two to three years" These chilling words.

"The big model is the real estate of the Internet, even if there is a bubble, it is a beautiful bubble," said an entrepreneur who just finished AI vocational education and training. Lei Jun once shouted when the Internet dividend disappeared in 2013, we must believe The power of the Internet, today we also have to believe in AGI, no matter whether it has a bubble or not.

Note: Li Ming, Zhang Cong, Zhao Hui, etc. are pseudonyms in this article.