Cloud Hosting

Artificial Intelligence (AI) is the tech term of the moment, and everyone in the PC space wants a piece of it. Every major computer company, from the ones that make the best ultrabooks and laptops to those that put together the best gaming PCs and desktops, as well as most component manufacturers, want to say they're offering an AI PC.

But what is an AI PC, exactly? What does it do differently from the computer you already have? Do you need one at all?

Here's how to cut through the noise to learn what an AI PC actually is:

The many definitions of an AI PC

Since every PC company and component manufacturer wants to say it is offering an AI PC, there are a ton of different definitions out there. Much of the press and the industry has seemingly coalesced around Microsoft's definition, which Intel shared at an AI PC developer program showing off its Core Ultra "Meteor Lake" processors. That definition had three requirements, suggesting that an AI PC:

This definition did rule out some existing PCs that had AMD and Intel's NPU and Copilot, but hadn't included the Copilot key. Most major laptop releases since then have included that key. If you take the Copilot key away, previous Windows on Arm devices, like those running Qualcomm 8cx would also qualify, as those have NPUs and can run Copilot, too. The Copilot key is little more than a branding exercise as it simply launches Copliot by simulating Shift + WIndows + F23, and one could just hit Windows + C to get the same efffect.

Since then, Microsoft has introduced its Copilot+ PCs, with laptops from Asus, Dell, Acer, Samsung, HP, Lenovo, Samsung, and Microsoft's Surface brand. Those all use Qualcomm's Snapdragon X Elite and Plus processors at the moment, though Microsoft has said upcoming AMD Strix Point (aka Ryzen AI 300) and Intel Lunar Lake chips may also fit the bill. To be considered a Copilot+ PC, laptops need to have at least 16GB RAM, 256GB storage, and an on-board NPU that's capable of 40 TOPS (trillions of operations per second). The Qualcomm Snapdragon X Elite chips support 45 TOPS on the NPU.

Copilot+ PCs, the first of which will launch on June 18, 2024, will have a series of four unique Windows AI features that other PCs cannot access. These include Cocreator (image generation in Paint), Windows Studio Effects (webcam blurring and special effects), real-time translation and captions for audio and Recall. Recall, the controversial feature which keeps a record of almost everything you do on your PC so you can remember it, was just pulled from the Copilot+ launch date build of Windows.

So, anyone who doesnt buy a Snapdragon X-powered laptop will have to wait on those special Windows features, likely for many months. AMD recently confirmed that its Ryzen AI 300 PCs wont be getting the Copilot+ features when they launch later this year, but will eventually. Desktop users are left out of the cold until at least the launch of Intel Arrowlake in Q4. And anyone with a current-gen laptop or desktop is probably left out permanently.

Given the weak set of Copilot+ features, anyone who is actually paying attention probably isnt that sad about having to miss them or wait for them. There are many other ways to get an offline (or online) AI image generator, to do real-time translation and to blur your webcam background. Recall is somewhat unique, but many people wont want it, because of the privacy risks associated with taking constant screenshots of your work.

Intel, on its website, has taken a more general approach: "An AI PC has a CPU, a GPU and an NPU, each with specific AI acceleration capabilities."

AMD, via a staff post on its forums, has a similar definition: "An AI PC is a PC designed to optimally execute local AI workloads across a range of hardware, including the CPU (central processing unit), GPU (graphics processing unit), and NPU (neural processing unit)."

Who is making AI PCs?

If you're following the definition that most of the industry is using: CPU, GPU, and NPU, then the answer is most laptop manufacturers are making them. They include Dell, HP, Lenovo, Asus, Acer, Samsung, Microsoft, among others.

These are laptops with Intel (Core Ultra "Meteor Lake"), AMD (Ryzen 7040 or 8040), or Qualcomm (Snapdragon X Elite or Plus) processors.

What is an NPU, exactly?

A neural processing unit, or NPU, are processors that specialize in parallel computing specifically for the purpose of AI workloads (GPUs often also use parallel processing in order to render advanced graphics). Intel, AMD, Qualcomm, and Apple have these attached onto the processor with the CPU and integrated GPU.

They're also highly efficient, allowing for longer battery life than running these processes on a CPU or a GPU (even if, in some cases, those might be the more performant options). NPUs run matrix math, allowing them to do things such as video decoding, upscaling and background removal at a fraction of the power.

An NPU's performance is measured in tera operations per second, or TOPS. Intels Meteor Lake processors and their NPUs can only do about 10 TOPS while Qualcomms Snapdragon X processors, AMDs Ryzen AI 300 processors and Intels Lunar Lake chips will all deliver 45 or more TOPS from their NPUs.

What about desktop PCs? What about GPUs?

At the moment, every PC with an NPU is a laptop (or, in some cases, tablets or gaming handhelds). But neither AMD nor Intel has released a desktop chip with an NPU. A big reason to include the NPU on mobile devices is that they are extremely power efficient, which helps with battery life. Desktop systems don't have batteries, so processor manufacturers can still pump their processors using more wattage.

Additionally, desktops are more likely to have discrete GPUs, which have also proven to be extremely adept at certain AI tasks (though this is mostly high-end parts, like the Nvidia GeForce RTX 4090, which has 24GB of RAM to work with).

In fact, in a recent blog post, Nvidia's vice president of consumer AI, Jason Paul, suggested that the company started the AI PC boom all the way back in 2018, when it launched its first GPUs with RTX tensor cores and DLSS with the RTX 20-series and Turing architecture. That's yet another different definition from the ones laptop and CPU companies are making.

While this doesn't fit the conventional, NPU-focused definition that many are working with, there are a number of companies putting the AI label on desktop PCs, too.

Newegg, for example, is selling its ABS desktops (which start with consumer-grade parts like the Nvidia GeForce RTX 4070 Super and an Intel Core i5-14400F for $1,800 and goes up from there), on an AI PC page alongside laptops using Intel NPUs, as well as desktop parts ("AI CPU," "API GPU"). MSI lists a number of its desktops using 14th Gen Intel processors as "AI Gaming Desktops" because of software that the company includes.

The desktops that make more sense to call AI PCs are workstations that have the power to train some models. Maingear, for instance, sells its Pro AI systems with Intel Xeon chips and Nvidia's RTX 5000 and RTX 6000 Ada GPUs. These range from $28,000 to $60,000 and are definitely not for people looking just to generate images or photos.

Are Macs AI PCs?

Apple is introducing its take on the AI to Mac laptops and desktops this fall. The company is launching a beta version of Apple Intelligence with generative writing, images, custom emoji, and a more capable version of its Siri assistant in macOS Sequoia this fall (as well as iPadOS 18 and iOS 18).

Apple will support Intelligence on Macs and iPads using its M1, M2, M3, and M4 families of processors. On the iPhone, it will support the A17 Pro on the iPhone 15 Pro and Pro Max (and presumably whatever is in the next iPhone this fall).

Because Apple uses its silicon across all of its Macs, its desktops, like the iMac, Mac Studio, and Mac Pro also have NPU (or as Apple refers to them, Neural Engines). That means macOS will have AI-specific features on desktop running on NPUs before Windows.

Do I need an AI PC?

Right now? "Need" is a strong word.

AI features are still in their infancy. In many cases they're still in beta. Many popular chatbots, like OpenAI's ChatGPT and Google Gemini, are totally cloud-based, as is most of what Microsoft Copilot does.

That's not to say there's no features you'll miss. On Copilot+ PCs, image generation built into Windows 11 and Restyle photo editing are exclusive to the new Snapdragon systems. Some other software, like the Adobe Photoshop, Lightroom, and Express as well as Da Vinci Resolve will use NPUs for some AI features and the cloud for others. Those NPU features may be slower or non-existent on older computers.

NPUs are also being used to power background blur and other camera effects on some PCs, though you don't inherently need an NPU for that kind of work (though it does free up the CPU and GPU).

But across the board, these features are still just rolling out, and it's unclear which will be the most useful to you. Local AI is more secure, as you don't have to send your information to another company's servers, but if you're using it in limited scenarios, the cloud functionality the most popular apps currently offer should more than suffice.

There's a lot of hype around AI. And while it has some legitimately cool uses, there are still plenty of places where it's unclear how much people will want to use it. If your current PC is still doing what you need it to do and is getting security updates, it may be worth waiting as more powerful tech comes out to support, presumably, more local AI tools, and see what you actually need.

It's already clear that companies will lock certain features to newer PCs. While early adopters may jump at the chance to try them, there's also no harm in letting those people be the beta testers (and many of these AI features are being labeled as beta by the companies that make them) and grab something when you know what you want.

Are AI PCs more secure?

One of the biggest pushes for AI on a laptop, rather than in the cloud, is security. After all, running an AI workload on an NPU in your computer means you don't necessarily need to send your information to the cloud.

That being said, that also means AI features need to be built securely, too. After security researchers discovered how easy it was to steal data from, Recall, Microsofts new AI feature that takes screenshots of all your activity, for later reference was pulled from the initial Copilot+ set of features as Microsoft promised more security and testing with Windows Insider members.

If you are running a business and having an LLM manage top-secret corporate data, having it processed locally would be more meaningful. But most of the AI features currently being marketed are not mission-critical business tools. Perhaps if Microsoft 365 Copilot ran completely locally, that would be a plus for some companies.

No matter what type of PC you're using, you should still adhere to good security practices. Other humans are still outsmarting us there.

Right now, the term AI PC is still somewhat vague. CPU vendors and Microsoft are using the term to sell you new computers (currently, new laptops only) that have powerful NPUs built into their processors. Most of the generative AI features people actually use today (chatbots, image generators) are free to use in the cloud, making them less than must-haves in their local form.

But NPUs do promise to save battery life by performing some common tasks such as video playback at much lower power. Some web browsers, such as Edge, can use the GPU today to do AI upscaling of videos, but soon that will be offloaded to the NPU. For creative professionals who are doing audio, video or photo editing while unplugged, the NPU will tackle tasks such as background noise removal at much lower power than your CPU or graphics card; of course, the software has to be optimized to do so. And, in the near future, well see more tasks transferred to NPUs, which in turn, will increase system efficiency even more.

So, in the end, the killer feature of AI PCs (at least if they are laptops) could really be longer endurance. If your laptop goes from lasting 12 hours on a charge to lasting 20 hours on a charge because of NPU usage, that could be a huge deal. But the generative features are still in their infancy, so if you're not ready to upgrade, there's still time to wait and see what evolves.

Continue reading here:

What is an AI PC and do you actually need one? - Tom's Hardware