A new vulnerability known as ‘LeftoverLocals’ has been identified, affecting graphics processing units (GPUs) manufactured by AMD, Apple, Qualcomm, and Imagination Technologies. This vulnerability allows unauthorized access to data stored in the local memory space of these GPUs.
Referred to as CVE-2023-4969, this security flaw poses a risk of data extraction from susceptible GPUs, particularly in the context of large language models (LLMs) and machine learning (ML) processes.
The discovery of LeftoverLocals was credited to Trail of Bits researchers Tyler Sorensen and Heidy Khlaaf. They responsibly disclosed the issue to the respective vendors before releasing a detailed technical analysis.
Overview of LeftoverLocals
The vulnerability originates from inadequate memory isolation within certain GPU frameworks, enabling one kernel to read data from the local memory that was written by another kernel on the same machine.
According to the researchers, an attacker can exploit this flaw by executing a GPU compute application, such as OpenCL, Vulkan, or Metal, to access data remnants left in the GPU’s local memory.
By deploying a ‘listener’ GPU kernel, an attacker can extract uninitialized local memory data and store it in a persistent location, like the global memory.
If the local memory is not properly cleared, the attacker can utilize the listener to retrieve information left by the ‘writer’ program, which stores values in the local memory.
The vulnerability allows attackers to intercept and extract sensitive information, such as model inputs, outputs, weights, and intermediate computations, from the victim’s computations.
In scenarios where multiple users share a GPU to run LLMs, LeftoverLocals can facilitate eavesdropping on other users’ activities and extracting data from the local memory of the victim’s “writer” process.
A proof of concept (PoC) developed by the researchers demonstrates that an attacker can recover up to 5.5MB of data per GPU invocation, depending on the GPU framework being targeted.
For instance, on an AMD Radeon RX 7900 XT running the LLM llama.cpp, an attacker could potentially retrieve as much as 181MB per query, allowing for accurate reconstruction of the LLM’s responses.
Impact and Mitigation
The CVE-2023-4969 vulnerability was identified in September 2023, prompting Trail of Bits researchers to notify CERT/CC for coordinated disclosure and remediation efforts.
Some vendors have already addressed the issue with patches, while others are still in the process of developing mitigation strategies.
Apple has confirmed that the latest iPhone 15 is not affected, and fixes have been released for A17 and M3 processors. However, devices powered by M2 processors are still vulnerable.
AMD is actively investigating mitigation options for the affected GPU models, while Qualcomm has issued a patch through firmware v2.0.7 for some chips but remains vulnerable on others.
Imagination addressed the vulnerability in DDK v23.3 in December 2023, although Google has cautioned that certain GPUs from this vendor are still at risk as of January 2024.
Intel, NVIDIA, and ARM GPUs have been reported to be unaffected by the data leak issue.
To mitigate the vulnerability, Trail of Bits recommends that GPU vendors implement an automatic local memory clearing mechanism between kernel calls to ensure the isolation of sensitive data. Despite potential performance implications, this measure is deemed necessary given the severity of the security risks posed.
Additional mitigation strategies include avoiding multi-tenant GPU environments in security-sensitive scenarios and implementing user-level protections.
