It’s too curly! 36 Notes and Truth About Big Models and AIGC

**Source:**i dark horse

On the afternoon of September 12, the Beijing headquarters of Startup Dark Horse ushered in a wave of “new AI forces”.

Huawei Cloud, APUS, Tors, SenseTime, Kuaishou, 360 Group, Qingbo Intelligence, Dark Horse Tianqi, MiniMax, Sinovation Ventures, Qiji Chuangtan, China Academy of Information and Communications Technology... There are listed companies, leading companies, and unicorns animal companies, as well as leading investment institutions and scientific research institutes in the field of AI.

The guests attending the meeting went straight to the topic——

"The current industry status of large models? How can companies better commercialize? What are the new trends and opportunities worth paying attention to?"

I am honored to participate in this closed-door AI meeting. I will share 38 notes and truths with you.

How many volumes does the industry have?

According to the latest statistics, more than 130 large models have been launched in China, and more than 70 algorithm models have been registered with the Cyberspace Administration of China. Internet giants such as BAT have all released large AI models. In 2023 alone, more than 60 startups have received financing, and the products are full of basic layers, model layers, and application layers. The new generation of generative AI may need to look back at the pitfalls of the previous generation of AI, and avoid the industry’s complacency to avoid the reincarnation of the previous winter. Practitioners in this field must clearly see the involution of the industry and the pain points of customers, and don't be fooled by the big guys' chicken soup.

1. Now a customer comes to us and says that there are 20 scenes, and each scene costs X million yuan? Then I went door-to-door to ask for prices. Would Huawei do it? Will Alibaba do it? Will Baidu do it? If you continue like this, you won't make any money in the end.

2. 80%-90% of the money raised by everyone is used for computing power. This is the current situation. You know, it's just training now. The cost of training is controllable, but the cost of inference is uncontrollable.

3. We have 30,000 GPUs and a computing power of 6,000p. We will strive to have a computing power of over 12,000p by the end of the year. In terms of data, 2 trillion tokens are cleaned and annotated every month, and there will be 10 trillion tokens by the end of the year.

4. At present, the industry still lacks some disruptive killer applications, making it difficult to realize commercialization.

5. How to find a balance between cost and effect? This is a difficult point. They all use large models and the cost is too high.

6. After a while, what everyone will compete for is the ability to optimize infrastructure. For example, in terms of network optimization capabilities, if you continue when others fail, you will be able to train more times than others.

7. In the case of tight computing power, we are making some forward-looking technical attempts. You may not have thought that in our large model company, there are more students doing infrastructure than modeling. Their prices are generally quite expensive and difficult to recruit.

8. Large models are now in an embarrassing situation. They cannot sell themselves at a high price. In the end, only those who sell clouds, cards and computing power make money.

9. At first I thought the model was quite valuable, but now it has fallen into involution again. I met a customer some time ago, and BAT and others quoted him. The initial quote was quite expensive, more than 10 million. Does anyone know what the final unit price was? Too curly.

10. Large models with tens of billions of parameters are considered free by some special customers.

Voices from the front lines

Companies in the field of artificial intelligence should always remember what Chairman Mao said, "come from practice to practice." Only by taking off your robe and mandarin jacket and walking to the fields can you get the most authentic feedback when you are next to your customers. There are too many pseudo-experts in the AI field now, so it’s better to listen to more voices from the front lines.

11. We talked to about 150 customers. The requirements for the large model itself are mainly divided into two categories. One is the textual requirements of the large model itself. Customers' requirements for the large language model are 100% accurate. The other is the AI agent, which includes function calls, code displays, and calls to third-party tools.

12. During our cooperation, a conflict arose. Customers will feel that the data cannot be given to you before they decide to deploy it internally. But without this data, how can we train a model that suits customer needs?

13. When we were working on the project, we found that users are not willing to pay for large models, but they still pay for your application. Some customers will directly ask, with large models, are some of the previous intelligent middle platforms and knowledge graphs no longer needed? In the end, I found that the scene is the core.

14. We need to find some sexy scenes. There are several standards. First, the small incision. Second, match the advantages of large models. Third, let users who make the decision to pay have a strong perception. For example, in the past, obtaining some data, conclusions or services required different processes. Now, through large models, decision makers can quickly obtain and complete them on mobile devices.

15. To succeed in a large model, three things are needed: 1) Whether you can get enough money to buy computing power. 2) Can we get enough data? 3) Is the talent density high enough? It’s not about the quantity, but whether there are enough high-quality scientists.

16. Now we encounter three types of customers. One type is anxious customers, such as financial customers who want to do it because their competitors are doing it, but they don’t know what their needs are. One type is customers who want to reduce costs. The customer's management believes that this is mainly a matter of cost reduction and expenditure reduction. However, it is difficult to judge the specific value of how much savings the model has achieved. The last category is customers who hope to make money by deploying open source models. They rely on this model to generate income. It is most convenient for these customers to pay.

ToC or ToB?

It is understood that the cost of training GPT-4 once is approximately US$63 million and requires a huge amount of 1.8 trillion parameters. For some large models currently released in China, the ToC direction is still the free model of the Internet. But practitioners all know that the development and operation of large models require a lot of costs, making it easier for ToB's business model to operate smoothly. In terms of commercialization exploration, ToB or ToC has always been a matter of concern to the industry. During the closed-door meeting, we heard two key words repeatedly: 1) genes and 2) transgression. "You cannot be what you are not."

17. We believe that opportunities still lie in the tob vertical model, and the core point is the data and scenario itself, which is the core.

18. We ourselves are also thinking about how artificial intelligence can be combined with the Internet, digitization, etc., and how to better make the original scenes more intelligent.

19. ToC or ToB, frankly speaking, this is related to everyone’s genes. Just because we can't do it doesn't mean others can't do it. For example, some ToC applications are played by children born in 2000 or 10, which are beyond our age understanding.

20. Toc and Tob are still very different. ToC has a relatively high fault tolerance rate. TOB is the opposite. Take, for example, intelligent question and answer. How to ensure accuracy? Like the government, there are red lines. How to avoid the illusion of the model is currently being explored accordingly.

21. On the contrary, we think it is easier to make money in ToC. There is a problem with ToB. The process of a project is relatively long. The money cycle is very long from the client’s budget approval to project establishment and implementation.

22. I think there are too many opportunities now. Don’t allocate resources to places you are not qualified for. It is very important to make strategic choices.

ToB and ToG are also very difficult

23. The biggest bug on the B side is that it ended up being advanced human outsourcing.

24. All projects have cycles, and all payments are based on the nodes of the cycle. It is impossible for me to help you train and optimize a model without restrictions.

25. AIGC creations are more tolerant and may have some mistakes. But when it comes to production and manufacturing by some companies, the requirements for accuracy are very high. It is often easier for us to pick one or two better cases from the model, but it is still quite difficult to maintain it at a high level without bad cases.

26. When we were working on a smart digital government project in an economically developed province, we promoted more than 5 scenarios, and the customer finally approved 3 scenarios. Then, we’ll get to the bottom of things about security, data, and the bottom layer. Then I will ask, what are the differences and advantages between you and other large models? Finally, all parties need to sit down and come up with an evaluation system. After passing the evaluation system, we still need to evaluate the performance.

Use projects to incubate products and solutions

27. Use projects to incubate products. After completing several projects, extract corresponding technical solutions. This set of solutions is most likely not a model, but a large model + small models, and finally a comprehensive solution formed by multiple models.

28. In the past one or two years, it may be the process of innovation and product production, and the recovery cycle of cash will be relatively long.

Agent

Imagine AI mimicking everyday human tasks to handle large numbers of humans’ complex social behaviors. A paper from Stanford University titled "Generative Agents: Interactive Simulacra of Human Behavior, titled" takes an in-depth look at AI Agents that remember, react, and plan. AI Agent is considered to be the next direction of OpenAI's efforts. The co-founder of OpenAI also said at a recent event: "Compared with model training methods, OpenAI is currently paying more attention to changes in the Agent field. Whenever a new AI Agents paper comes out, we will be very excited and discuss it internally seriously." .

29. We always imagine that large models are omnipotent and can solve various problems? Is this the case? Big models are just big models.

30. We manage AI internally, which is called invisible AI. In front of users, we will not emphasize what model it is or how many parameters it has. Our definition of AI is human assistance.

31. Bypassing the model and computing power, the next opportunity may be Agent.

32. The biggest problem currently affecting customer use: input-output ratio. Once you reach the end of the conversation with the client and talk about the project budget, if it is purely text-related and the investment is a few million or a few million, the client will not be very satisfied. In addition, if large models are embedded into actual production environments using AI agents to solve actual problems, customers will be very willing to pay.

33. Based on the large model, AI Agent has enhanced capabilities such as memory, planning and execution. We have invested in more than 60 start-up projects this time, more than 20 of which are Agents.

34. ToC products, payment forms and product forms are very different between China and foreign countries. Recently, we have invested in some agent companies.

35. However, at this stage, AI Agent is only in a new experimental stage, and there is still a certain gap between it and general intelligence. In the future, in addition to the comprehensive capabilities of a single AI Agent, breakthroughs in collaboration and emotion capabilities between multiple AI Agents will also need to be solved.

36. Big model players must ensure that they stay at the poker table so that they can have a chance to see new things come out in the second half.

(over)

​