“AI-capable computers” took center stage at CES 2024 recently, with key industry players like Intel, AMD, NVIDIA, and Qualcomm showcasing the prowess of their hardware in managing AI-related tasks. Microsoft has heralded this as the dawn of the “year of AI-powered Windows PCs.”
While these “AI-capable computers” are now available for purchase, it is crucial to look beyond the marketing hype and focus on the actual value they provide. Despite ambitious claims, these PCs may not immediately deliver the transformative experience promised at the start of 2024. Although they hold the potential for exciting features down the line, that vision has yet to materialize.
Let’s examine the current landscape and explore why, even for genAI enthusiasts, investing in this current wave of AI-accelerating chips may not be the most rewarding choice. Redirecting attention to alternative hardware solutions, some of which users may already possess, could prove more beneficial.
Current State of AI Processing
Before diving into specifics, it’s essential to understand the prevailing dynamics of most existing genAI tools. When using tools like Copilot on Windows 11, accessing ChatGPT, leveraging Adobe Firefly for image generation, or interacting with similar popular genAI applications, it’s vital to note that the computational heavy lifting doesn’t happen on your local machine.
Instead, the processing occurs in remote data centers equipped with robust computing resources to execute AI algorithms and generate outputs like images or responses. While some tasks can be performed locally using open-source tools like the Stable Diffusion text-to-image model or the LLAMA large language model, achieving optimal results typically requires a PC with significant processing capabilities, especially a high-performance GPU.
The aim is to have mainstream PCs capable of running these genAI models locally without depending on external data centers. However, the shift to a fully localized processing environment is gradual, with complex tasks likely to continue being offloaded to data centers while simpler functions are managed locally, such as contextual suggestions and email assistance.
The Significance of Neural Processing Units (NPUs)
The recent excitement surrounding AI PCs stems from hardware advancements, particularly the integration of “neural processing units” (NPUs) in Intel’s latest Meteor Lake chips and recent AMD processors. NPUs are specialized, energy-efficient components designed to run genAI models locally and accelerate AI operations. The vision is for every PC to incorporate an NPU, functioning seamlessly even on battery power due to their low energy consumption.
Apart from PC manufacturers, Google’s Tensor G3 chip in devices like the Google Pixel 8 series follows a similar path, enabling on-device genAI functionalities such as audio summarization and smart replies. While these NPUs enhance AI capabilities on PCs, they currently do not support running advanced models like Copilot or ChatGPT locally.
Although “AI-capable computers” with NPUs are now on the market, their transformative potential is yet to be fully realized.
Windows’ Readiness for ‘AI PCs’
Despite Microsoft’s focus on the era of Windows-powered AI PCs leveraging NPUs, practical applications are currently limited. The introduction of a Copilot key as part of Microsoft’s AI PC initiative exemplifies this scenario. The Copilot function remains consistent across Windows PCs, with no additional AI-specific features utilizing advanced hardware. Windows 11’s use of NPUs is currently confined to features like “Windows Studio Effects” for video and voice meetings, offering enhancements like background blurring and eye contact simulation.
While NPUs may enhance system performance and battery efficiency, the current software implementations, such as Windows Studio Effects, do not present a significant advantage over existing third-party solutions for webcam enhancements.
The much-anticipated “AI release” of Windows, rumored to debut later this year with NPU-driven features, holds promise for improving user interactions with PCs. However, the full integration of NPUs into Windows functionalities remains a future prospect, with tangible outcomes still on the horizon.
Harnessing NPUs and AI Workloads
Given the limitations of Windows’ NPU utilization, exploring third-party software becomes crucial to maximize the potential of NPUs. Intel’s demonstrations of NPU-accelerated genAI workloads showcase plugins for popular applications like GIMP and Audacity, enabling users to independently configure these plugins to leverage NPU capabilities.
However, the current landscape indicates that NPU-driven AI applications are not yet compelling, with manufacturers primarily highlighting minor enhancements like improved noise reduction and intelligent fan control. While these incremental advancements are noteworthy, they do not represent a significant departure from existing functionalities.
GPUs vs. NPUs in AI Performance
When assessing the effectiveness of NPUs for AI tasks, benchmarks indicate that traditional GPUs outperform NPUs in efficiently handling current genAI workloads. While NPUs excel in energy efficiency and potential future software optimizations, GPUs remain the preferred choice for running demanding AI applications like Stable Diffusion image generation and LLMs directly on PCs.
The focus on NPUs revolves around enabling future AI capabilities rather than immediate performance gains in current AI workloads.
Recommendations for ‘AI PC’ Purchases in 2024
Given the evolving landscape of AI-enabled hardware, making informed decisions about PC purchases is crucial:
-
Anticipating Future AI Features: For users keen on upcoming AI enhancements integrated into Windows, exercising patience until late 2024 may provide access to newer AI-centric laptops or potential discounts on existing models.
-
Optimizing AI Workloads: To effectively address current AI demands, prioritizing computers equipped with powerful GPUs from leading manufacturers like NVIDIA or AMD is advisable. While AMD hardware offers competitive options, NVIDIA GPUs are renowned for optimal performance in AI tasks.
-
Navigating the NPU Hype: When considering the purchase of an “AI laptop” featuring NPUs, it’s essential to acknowledge that future Windows functionalities may leverage NPUs or high-end GPUs. Investing in contemporary hardware with modern CPUs, such as Intel’s Meteor Lake series or equivalent AMD chips, can align with future AI requirements.
Ultimately, the decision to invest in AI-enabled hardware should be guided by present capabilities rather than speculative future advancements. As the industry progresses towards integrating NPUs and enhancing AI functionalities, users can anticipate a more enriched computing experience in the near future.
