By leveraging artificial intelligence (AI) technology in guiding small drones during active combat scenarios, the pursuit of autonomous weapons systems for military utilization has reached a new phase. The IRIS robot, also recognized as the “throwbot,” has been deployed by the Israel Defense Forces in the recent Gaza conflicts commencing on October 7. This advanced system is proficient in dismantling Hamas’ intricate network of underground tunnels by utilizing sensors to detect objects and individuals, transmitting captured images to its operator, and potentially neutralizing booby-traps.
Israel’s arsenal also includes Nova 2, a diminutive AI-equipped aircraft developed by U.S.-based Shield AI. Operating without GPS, communication links, or human pilots, Nova 2 possesses the capability to autonomously navigate and map complex subterranean or multi-level structures. Additionally, Israel has introduced Fleet-X, an innovation by Elbit Systems, designed to oversee multiple drones capable of carrying grenade-sized explosive charges, transforming them into loitering munitions. As Israeli forces engage in clandestine warfare against Hamas in the imminent days, the efficacy of these aerial assets will be rigorously tested.
The “Political Declaration on the Concerned Defense Use of Artificial Intelligence and Autonomy,” unveiled by U.S. Vice President Harris on November 1, 2023, has garnered support from 46 states. This Declaration, disclosed in February, aims to promote optimal practices for the responsible integration of AI in security applications without altering existing legal obligations or introducing new ones under international law. Upholding current legal frameworks such as the law of armed conflict (LOAC), which mandates the scrutiny of novel weaponry, is deemed imperative for ensuring responsible AI utilization, as emphasized by Shawn Steene and Chris Jenks in their analysis of the Declaration. Given the unique self-learning attributes of AI-enabled military systems, meticulous testing and fail-safe mechanisms are imperative. To establish requisite safeguards, pertinent legal and ethical considerations should be integrated throughout the entire lifecycle of these capabilities, spanning from conceptualization and design to operational deployment.
While the Declaration is predominantly non-binding, a notable exception highlighted in Point J underscores the imperative need for:
“In order to mitigate the risk of malfunctions in military AI systems, measures should be implemented to prevent unintended consequences and respond promptly, including deactivation or disengagement of deployed systems.”
This article deliberates on the extent to which these protective measures must be enforced in alignment with contemporary international law and the legal implications of surpassing the statutory requirements outlined. It is imperative to veer away from the notion of human supremacy as the linchpin of governmental deliberations concerning autonomous weaponry, as elucidated herein. To ensure the lawful operation of AI-driven military functionalities, the focus should shift towards “networks of control,” a perspective already embraced by numerous stakeholders in the domain.
Enhancing Safety Protocols
In specific domains of arms regulations, safety protocols are pivotal. For instance, anti-personnel mines may be equipped with self-destruct mechanisms or mechanisms for self-neutralization after a specified duration to prevent inadvertent harm. Similarly, underwater mines must be designed to render them harmless if control is lost. This concept aligns with the requirement for termination and neutralization as stipulated in Article 57(2)(b) of Additional Protocol I, as previously highlighted. Such measures could potentially garner broader support from states.
Determining the activation criteria for safety mechanisms to prevent AI-enabled functionalities from operating poses a regulatory challenge. Imposing time-based restrictions may not always trigger safeguards when necessary, potentially compromising operational effectiveness. For instance, triggering a safety mechanism upon loss of communication could impair the system’s autonomy, a core feature of AI-enabled capabilities. Evaluating a safeguard mechanism activated by “unintended behavior,” as proposed in the Declaration, is intricate due to various factors contributing to functional degradation.
From a legal standpoint, AI-enabled weapons systems like Legion-X drones carrying explosives may lose their ability to differentiate between intended targets and civilians or non-combatants. Given the criticality of distinction in LOAC, states could argue that AI functionalities must consistently exhibit high reliability, with even minor performance degradation warranting reversible measures when the outcomes are potentially devastating. Conversely, a decline in non-essential features may be tolerable if it does not compromise core operational aspects.
Steene and Jenks underscore the importance of assessing the potential repercussions of losses to determine suitable safeguards. Establishing discernible standards is fundamental, necessitating developers and programmers to articulate and encode these requirements in quantifiable terms. Defining protective metrics to classify failure events and requisite protective actions is paramount for ensuring operational integrity.
Anticipated Implications
The Declaration’s endorsement by an increasing number of states signifies a burgeoning consensus on the regulatory framework governing military AI capabilities. Although more work is needed to bridge the gap between high-level policy directives and technical requisites, the inclusion of safeguard requirements may herald a new era in international law. Field experiences with novel AI-enabled assets like Nova 2 and Legion-X will shed light on the integration of legal and ethical considerations into control systems from the inception stages of development and design.
In conclusion, the evolution of safeguard systems within the realm of AI-driven military capabilities, whether mandated as a novel obligation or adopted as a best practice, mitigates the potential risks associated with unintended consequences. This trajectory mirrors past agreements in arms control regulations and signifies a pivotal shift in governance discussions surrounding AI, transcending the simplistic notion of human control. The transition from the “systems of control” paradigm to “essential people control” underscores the imperative for a sophisticated approach to regulating AI functionalities in military contexts, necessitating a comprehensive framework encompassing diverse control measures tailored to specific operational requirements. As the international community navigates the complexities of AI integration in defense mechanisms, the development and implementation of safeguard mechanisms will play a pivotal role in shaping the future landscape of military AI applications.
