– Debate Between Intel and Microsoft: Running Copilot on PCs vs. Cloud

We are aware that AI PCs will come with several hardware components that are not currently typical in Windows systems, such as a built-in neural processing unit (NPU) and Microsoft’s new Copilot key for keyboards. This aligns with Microsoft’s statement from January. However, details on the actual functionality of these AI PCs for users have been scarce.

In an Intel announcement concerning a new AI Computer programmer program aimed at encouraging software developers to integrate AI features into their applications using local hardware, Microsoft and Intel have initiated discussions.

According to Tom’s Hardware, AI PCs will have the capability to handle “more aspects of Copilot,” Microsoft’s AI assistant, locally on the user’s device. Currently, Copilot relies on server-side processing even for minor tasks, leading to noticeable delays, especially for simple tasks like adjusting settings or obtaining basic information. By running generative AI models locally, user privacy can be enhanced, enabling AI-powered software usage without the need to transmit data to external servers for additional model training.

Presently, Windows does not extensively utilize local NPUs due to their absence in most current PCs. NPUs can enhance the Surface Studio camera features for power-efficient video effects and background alterations, but their utilization is limited. In contrast, Apple and Google operating systems leverage NPUs for a broader spectrum of image and audio processing functions, including facial recognition, image recognition, OCR, live transcription, translation, and more.

Intel has indicated that Microsoft will mandate NPUs capable of achieving speeds of 40 trillion operations per second (TOPS) in “next-generation AI Computers” to meet its requirements. TOPS serves as a significant performance metric for NPUs, with Intel’s Meteor Lake laptop chips delivering 10 TOPS, while AMD’s Ryzen 7040 and 8040 laptop chips offer 10 TOPS and 16 TOPS, respectively.

Qualcomm is poised to introduce an NPU suitable for local Copilot usage ahead of Intel. The upcoming Snapdragon X processors, often viewed as Windows’ response to Apple’s M-series Mac chips, promise up to 45 TOPS. Despite Microsoft’s historical reliance on both Intel and Qualcomm options, the Surface tablet is now exclusively available for business purchases. Rumors suggest that Microsoft may transition the consumer version to Qualcomm chips.

TOPS, while a simplified performance metric, may not fully capture the nuances in how each NPU handles various tasks, similar to the limitations of using FLOPS for comparing graphics performance. Additionally, Windows’ Arm version continues to face compatibility challenges with hardware and software, potentially hindering its advancement.

As part of its developer program, Intel is introducing an “AI PC development kit” featuring an Asus NUC Pro 14, a mini PC based on Intel’s Meteor Lake silicon. Last year, Intel ceased production of its NUC mini PCs, transferring the brand and designs to Asus. Asus offers support for older Intel NUCs, including warranty services and software. The ROG NUC mini gaming PC and the NUC Pro 14 are the first new NUC models announced since the transition.