Cloud Hosting

The Pentagon is shopping around for ideas from industry regarding how it might better test and evaluate future artificial intelligence products to ensure they are safe and effective.

In a request for information this week, the Pentagons Joint Artificial Intelligence Center, or JAIC, seeks input on cutting-edge testing and evaluation capabilities to support the full spectrum of the Defense Departments emerging AI technologies, including machine learning, deep learning and neural networks.

According to the solicitation, the Pentagon wants to augment the JAICs Test and Evaluation office, which develops standards and conducts algorithm testing, system testing and operational testing on the militarys many AI initiatives.

The Pentagon stood up the JAIC in 2018 to centralize coordination and accelerate the adoption of AI and has been building out its ranks in recent months, hiring an official to implement its new AI ethical principles for warfare.

The JAIC is requesting testing tools and expertise in planning, data management, and analysis of inputs and outputs associated with those tools. The introduction of AI-enabled systems is bringing changes to the process, metrics, data, and skills necessary to produce the level of testing the military needs, and that is why the JAIC is requesting information, Dr. Jane Pinelis, Chief, Test, Evaluation and Assessment at the JAIC, said in a statement. Testing and Evaluation provides knowledge of system capabilities and limitations to the acquisition community and to the warfighter. The JAIC's T&E team will make rigorous and objective assessments of systems under operational conditions and against realistic threats, so that our warfighters ultimately trust the systems they are operating and that the risks associated with operating these systems are well-known to military acquisition decision-makers."

The solicitation indicates it plans to use feedback from the solicitation to guide how it further builds out its capabilities. Specifically, the Pentagon is interested in tech testing tools that focus on:

In addition, the Pentagon wants feedback regarding evaluation services in five mission areas: dataset curation, test harness development, model output analysis, test reporting and testing services. Lastly, it seeks other technologies it may not be aware of that may be beneficial to testing and evaluation efforts.

Responses to the RFI are due May 10.

Original post:
Pentagon Needs Tools to Test the Limits of Its Artificial Intelligence Projects - Nextgov