The U.S. Department of Defense has chosen Scale AI to assist in the testing and evaluation of generative artificial intelligence for military applications. Scale AI, a company based in California, disclosed this collaboration with the Chief Digital and AI Office on February 20, coinciding with the commencement of the CDAO conference in Washington, D.C., which featured discussions on this subject.
Generative AI has the capability to produce text, images, or other forms of data outputs based on algorithmic models in response to user prompts. Large language models, a category of generative AI, utilize statistical relationships within text documents or other inputs to generate essays, computer code, human-like conversations, and more, either independently or through supervised training. Scale AI will establish benchmarks for such systems as part of the contract.
While the Defense Department has shown a growing interest in generative AI, its applications are still a topic of debate. While tools like smart assistants or chatbots can efficiently locate files, answer common queries, or retrieve contact details, they also have the potential to be used in disinformation campaigns, spoofing attacks, and cyber threats. In August, the CDAO initiated Task Force Lima to explore and provide guidance on the use of generative AI for national security purposes.
Alexandr Wang, the founder and CEO of Scale, emphasized in a statement that the company’s efforts will facilitate the implementation of verifiable and cyber-secure models. Scale’s testing and evaluation processes are designed to detect misinformation, bias, data breaches, and inaccurate advice, including medical, legal, or financial information.
“Testing and evaluating generative AI will help the DOD understand the strengths and limitations of the technology, so it can be deployed responsibly,” Wang stated. “Scale is honored to partner with the DOD on this framework.”
The announcement made on Tuesday did not specify the financial value of the collaboration.
In a July testimony to Congress, Wang highlighted how outdated data retention and management policies were impeding the Defense Department. He stressed the importance of enhancing rapid procurement methods and fostering innovation to ensure the effectiveness of AI systems, noting that the quality of these systems is directly linked to the data they are trained on.
Scale secured a contract worth nearly $250 million in 2022 to provide federal agencies with access to its technologies. This blanket purchasing agreement was issued by the Joint AI Center, which now operates under the CDAO.
Colin Demarest, a reporter at C4ISRNET, specializes in covering military networks, cyber issues, and IT. Previously, he reported on the Department of Energy and the National Nuclear Security Administration, focusing on Cold War cleanup and nuclear weapons development for a newspaper in South Carolina. Colin is also recognized for his award-winning photography.
