Automatic Risk Discovery for Agents: Testing Artificial Intelligence Systems with Black Box Method

Artificial intelligence systems start to perform more complex tasks automatically. Agents that work especially in the production environment work unsecured inputs, using tools with real permissions and make independent decisions. This situation opens a range of new security risks beyond chat-based models.

The existing artificial intelligence assessment methods remain insufficient to this complexity:

-Single round tests can not capture errors that the agents can do in multiple steps following each other

• Standard security controls do not simulate multi-step attack scenarios made by autonomous systems in the real world
• Not taking into account the scenarios available risk assessments, agents access to vehicles and external systems
• Tests made with closed box methods can only be based on basic system definitions

New frame developed by researchers, designed to close this deficit. The system aims to automatically explore the risks that agents can occur during very step operations, starting with basic information. The frame is structured based on a plugsonomy consisting of seven basic components and can quickly find new security open in practice.

This research is seen as an important step to ensure their safe operation, when agents pass to practical applications quickly. Automatic risk discovery methods will allow developer and security teams to test complex AI systems more efficiently.