Punching Apple, kicking Intel, Qualcomm chips are the future of AI

Original source: Geek Park

Image source: Generated by Unbounded AI

"Wait a minute, wait a minute, you want to shoot the next PPT more!"

In the early morning of October 25, Beijing time, in the opening speech of the 2023 Qualcomm Snapdragon Summit, Cristiano Amon, president and CEO of Qualcomm, grandly introduced the Snapdragon X Elite Oryon CPU. In the PPT presentation, the new CPU not only overshadowed Apple's M2 Max and Intel i9-13980HX, the two most powerful processors in the world, but also had lower peak energy consumption than the previous two.

At the same time, in addition to making a name for X Elite on the PC side, Qualcomm also exposed the Snapdragon 8 Gen 3 processor on the same day. As the latest mobile chip, Snapdragon 8 Gen 3 not only improves performance, but also optimizes generative AI processing, and can even directly run models with tens of billions of parameters**.

Inside and outside the market, CEOs and presidents of consumer electronics giants such as Microsoft, Meta, Google, HP, Lenovo, and Xiaomi have stood for Qualcomm. Among them, Xiaomi Group President Lu Weibing not only exposed the upcoming Xiaomi 14 flagship at the scene, but also officially previewed that Xiaomi cars will be launched in 2024.

The new PC series chip products, the Snapdragon 8 Gen 3 chip, make the "terminal AI" emphasized by Qualcomm a reality within reach.

01 Snapdragon X Elite, straight to Apple Intel

The focus of the whole audience that day was none other than the Snapdragon X Elite.

To support future high-load intelligent tasks, Qualcomm introduced the company's most powerful computing processor to date for PCs: the Snapdragon X Elite, with Oryon CPUs directly inflicted on Intel and Apple.

Built for AI, Snapdragon X Elite** integrates a new customized Qualcomm Oryon CPU, supports generative AI models with more than 13 billion parameters running on the terminal side**, processing 4.5 times faster than competing products, and can generate 30 tokens per second for 7 billion parameter large models. According to Qualcomm, this mobile computing CPU performs twice as well as the competition, consuming only one-third of the power consumption at the same peak performance.

Snapdragon X Elite can generate 30 tokens per second for 7 billion parameter large models

"Hurry up and take pictures." Qualcomm CEO Amon said at the summit that the single-threaded performance of Oryon CPUs exceeds Apple's M2 Max and Intel's i9-13980HX, and the peak performance power consumption is 30% less than the M2 Max and 70% less than the i9-13980HX.

The single-threaded performance of Oryon CPUs surpasses Apple's M2 Max | Qualcomm

The single-threaded performance of the Oryon CPU surpasses Intel's i9-13980HX |

In terms of multi-threaded performance, Qualcomm also compared Intel's i7-1355U and i7-1360P, saying that the performance of the Snapdragon X Elite can be twice as high, and the peak performance power consumption can be 68% less; There is also Intel's i7-13800H, which claims to exceed it by 60% in performance and 65% less in peak performance power consumption. In addition, Qualcomm also focused on comparing Apple's M2, emphasizing that the performance of the Snapdragon X Elite exceeds it by 50%.

The performance of the Snapdragon X Elite exceeds that of the Apple M2 by 50% | Qualcomm

Kedar Kondap, Senior Vice President and General Manager of Compute & Gaming at Qualcomm Technologies, said: "The Snapdragon X Elite marks a quantum leap forward in computing technology innovation, with the new custom Qualcomm Oryon CPU delivering incredible energy efficiency to consumers and taking creativity and productivity to a whole new level. Powerful end-side AI will enable seamless multitasking and a new intuitive user experience, empowering consumers and businesses to create and grow."

Snapdragon X Elite Performance |

It is reported that OEM manufacturers are expected to launch PCs equipped with Snapdragon X Elite in mid-2024.

02 3rd Generation Snapdragon 8,1 seconds to generate Stable Diffusion image

At the summit, Qualcomm also released the first mobile platform carefully built for generative AI: Snapdragon 8 Gen 3.

Compared with the previous generation, the third-generation Snapdragon 8 has 30% higher CPU performance, 25% higher GPU performance, and 25% higher NPU performance, supporting running generative AI models with 10 billion parameters. Supposedly running Stable Diffusion on smartphone devices, it takes less than 1 second to generate images with text.

3rd Generation Snapdragon 8 Performance | Qualcomm

Chris Patrick, senior vice president and general manager of mobile phones at Qualcomm Technologies, said, "The platform will usher in a new era of generative AI—empowering users to create unique content, help boost productivity, and enable other breakthrough use cases."

Based on the live demo, Snapdragon 8 Gen 3 will support some handy and creative AI camera tools, including features such as removing objects from video, expanding the photo area, and generating virtual backgrounds.

As a leader in Android flagship smartphone SoCs, it is reported that the third-generation Snapdragon 8 will be widely adopted on the terminals of global OEMs and smartphone brands, including Asus, Honor, iQOO, Meizu, NIO, Nubia, OnePlus, OPPO, Realme, Redmi, Red Devil, Sony, vivo, Xiaomi and ZTE.

Xiaomi Group President Lu Weibing also appeared at the Hawaii Summit to demonstrate the upcoming release of the Mi 14 mobile phone equipped with the third-generation Snapdragon 8, as well as Xiaomi's "long-term AI strategy growth". According to it, the Xiaomi 14 series will make the world premiere of the third-generation Snapdragon 8 mobile platform.

Xiaomi President Lu Weibing at the Hawaii Snapdragon Summit | Qualcomm

In addition to Xiaomi, according to executives of various manufacturers, NIO and Qualcomm based on the Snapdragon platform in the car and mobile phone in-depth cooperation, e-sports mobile phone Red Devil will be equipped with the third generation of Snapdragon 8, the upcoming realme GT5 Pro will also be equipped with this flagship chip, vivo and iQOO will release flagship products equipped with the third generation of Snapdragon 8, ZTE based on the third generation of Snapdragon 8 nubia flagship mobile phone will soon be available.

3rd Generation Snapdragon 8 Performance | Qualcomm

According to Qualcomm, Android flagship terminals equipped with the third-generation Snapdragon 8 are expected to be available in the coming weeks.

03 The era of "terminal AI"

Promoting generative AI in various categories of terminals, including a wide range of consumer electronics, has become Qualcomm's biggest goal this year.

"We are entering the era of AI, and end-side generative AI is critical to creating powerful, fast, personalized, efficient, secure, and highly optimized experiences. **," said Qualcomm CEO Amon Anmeng.

The Snapdragon X Elite, the third-generation Snapdragon 8, announced at the Snapdragon Summit, is said to be able to handle generative AI tasks at extreme speeds, for example, the terminal-side chat assistant for Windows 11 PCs can process 30 tokens per second, and it takes less than a second to generate images on smartphones using Stable Diffusion.

From text to image, image to video, and text to 3D, the advent of multimodal AI is changing the way humans interact with devices, according to Ziad Asghar, senior vice president of product management at Qualcomm, "Fully operational on end devices, which means you can fly on an airplane and still use generative AI." And, "this is definitely just the beginning of this revolution."

According to Asghar, "Generative AI in the cloud faces many challenges that can be solved by processing on the device." For example, some companies are integrating proprietary code into GPT tools to detect code errors. In the future, PCs may be able to use information known by the device, such as user status, preferences, calendar, and age, combined with generative AI queries for better generative AI experiences that are "far superior to the cloud."

Moreover, compared with the interactive gameplay where people hand over personal information to various chatbots, the biggest selling point of AI running on devices is: privacy, and there is no need to pass private context information to the cloud for processing. In addition, there is the benefit of immediacy, which is that you don't have to wait for information to return from the cloud.

But there is another key problem with running AI in the terminal, what if model inference requires a lot of computation?

According to Qualcomm executives, 16-bit or 32-bit floating-point numbers are typically used when training large language models in the cloud, resulting in computation-intensive computation for each model inference. Instead, they've been focusing on compressing models, quantizing and pruning, such as using only 4 bits, saving power, and being able to perform more concurrent AI processing on devices, saying that "the growth of large language models over the past few months has proven strategies to make models smaller by quantizing them."

In addition, over time, people continued to improve the algorithm and the model began to get smaller. Another approach that people will take is "domain-specific models," where models are trained for specific applications, "models can become very small and have very high accuracy."

"Based on Qualcomm's years of AI research and development, including the combination of CPU, NPU and GPU with excellent performance in the terminal, and our support for local operation of many leading models, we are able to bring the benefits of generative AI to users around the world and to different end categories," Amon said.

At the same time, in order to allow Snapdragon terminals using Android, Windows and other operating systems to discover each other and achieve seamless cooperation between multiple terminals, Qualcomm launched the cross-platform technology Snapdragon Seamless at the Snapdragon Summit.

Cross-platform technology Snapdragon Seamless |

According to the Deloitte Connected Consumer Survey, each U.S. household currently has an average of 21 digital terminals, but the transfer of information between different terminals, especially between terminals of different manufacturers, is often not smooth. This will limit consumer choice and lead to a lock-in effect, where consumers have to buy products only from the same manufacturer in order to work together between their terminals.

With Snapdragon Seamless, multiple terminals can operate as if they were a "single unit", for example: mouse and keyboard can be used seamlessly on PCs, mobile phones and tablets; Files and windows can be dragged and dropped between different types of terminals; The earbuds can be intelligently switched according to the priority of the sound source; XR offers extended functionality for smartphones.

It is reported that Snapdragon Seamless will expand to XR, automotive and IoT platforms in the future. Companies including Microsoft, Android, Xiaomi, Asus, Honor, Lenovo and OPPO are working with Qualcomm to adopt Snapdragon Seamless, a technology that will land on terminal platforms released worldwide as early as this year.

From smartphones, augmented reality devices, cars to personal computers, the penetration of generative AI into a wide range of end devices through chips has begun. Qualcomm is the most important first and most important step in promoting "terminal AI" to all consumers.