Despite the sophisticated capabilities of generative AI models, instances of these models behaving unpredictably, hallucinating, or containing exploitable vulnerabilities have been observed. In an effort to address this issue, Microsoft has introduced a new tool designed to identify risks within generative AI systems.
Recently, Microsoft unveiled the Python Risk Identification Toolkit for generative AI (PyRIT), a tool that has been utilized by Microsoft’s AI Red Team to assess risks within their generative AI systems, including Copilot.
Over the past year, Microsoft has conducted red team assessments on over 60 high-value generative AI systems. Through these assessments, it became evident that the evaluation process for these systems significantly differs from that of classical AI or traditional software, as highlighted in a blog post.
The evaluation process is distinct due to the need to address not only standard security risks but also responsible AI risks. These responsible AI risks encompass preventing the intentional generation of harmful content and ensuring that the models do not propagate misinformation.
Furthermore, generative AI models exhibit a wide range of architectural variances, leading to diverse outcomes from identical inputs. This variability makes it challenging to establish a uniform evaluation process applicable to all models.
Consequently, manually scrutinizing these diverse risks proves to be laborious, monotonous, and time-consuming. Microsoft advocates for the utilization of automation to assist red teams in pinpointing areas of concern that warrant further scrutiny and automating repetitive tasks, a role fulfilled by PyRIT.
The toolkit, validated through extensive use by the Microsoft AI team, operates by sending malevolent prompts to the generative AI system. Subsequently, the scoring agent evaluates the system’s response, assigning a score that informs the generation of subsequent prompts based on prior feedback.
Microsoft underscores that PyRIT’s primary advantage lies in its ability to enhance the efficiency of red team operations significantly, drastically reducing the time required to complete tasks.
For example, during a red team exercise involving a Copilot system, Microsoft was able to select a harm category, generate thousands of malicious prompts, and leverage PyRIT’s scoring engine to assess the Copilot system’s output within hours, a process that previously would have taken weeks.
The PyRIT toolkit is now accessible and includes a range of demonstrations to facilitate user familiarity. Microsoft is also conducting a webinar on PyRIT to showcase its application in red team assessments of generative AI systems, with registration available on Microsoft’s website.
