CUSP: New Approach to Measure Deficiency of Artificial Intelligence Models

Take advantage of the power of multiple artificial intelligence models is becoming increasingly common in machine learning. But when combining these models, it creates a big challenge to measure the uncertainties that arise. Developed by researchers CUSP (Collective Uncertainty through Semantic Opinion Pooling) offers a solution that does not require training to this problem.

• What’s new:*

• It is possible to compare multiple visual-language models (VLM) responses to a common symmagnetic space
• Evaluates the real reliability of the system level, rather than the trust of individual model or the trust of batch response
• As a non-trained frame, the calculation cost increases accuracy while minimizing
• Combines different outputs of models with semantic view pooling (opinion pooling) method

Today, the combination of multiple models is mainly used to design complex artificial intelligence systems. CUSP can produce more reliable results by sitting on a standard meaning foundation from models in this heterogeneous structure. The frame calculates how consistent and reliable the answer that each model arises, rather than independently measure the reliability of each model.

This development opens the use of multi-model artificial intelligence systems in industrial applications.