User Attacks for Artificial Intelligence Systems

Artificial intelligence security research often concentrates on the damages caused by models, an important issue overlooks: behaviors that users can show against artificial intelligence systems. A new study published in arXiv systematically examines how and when such negative interactions occur.

Basic findings of research:

• Two independent sensors were used by examining 777 thousand English chat recording
• An eight-categorized word audit was created to categorize hostility, printing and counter prints for models
• Negative behavior directed by the user can complex the correct interpretation of model behavior
• The alignment deviation of such interactions and the actual world distribution risks
• The study reveals a ignored size on the user side while evaluating the security of the chat systems

This research shows that security assessments should focus not only on threats created by models, but on negative behaviors directed by users. While interacting with chat-based AI systems is becoming increasingly common, the understanding of this dynamics has become critical for pre-distribution tests and security protocols.

As a result, to see the full picture of how AI systems are used in the real world environment, it is necessary to evaluate the interaction between model behavior and behavior on the user side together.