How to Change Language Model Learning Format with Tropical Reinforcement Learning

Consolidated learning methods used in education of large language models are based on the principle of maximizing the expected return for a long time. However, a new research released in arXiv reveals the important limitations of this classic approach and offers a more effective alternative.

• The available method shows how often the model is successful by collecting the possibilities of all successful ways, but which solution can not determine what really works
• Due to the fact that potentials are equal to one, applying to a solution can never forget another solution that does not prove to be wrong
• Tropical learning uses a new formulation on mathematical basis to solve this hierarchical problem
• The new approach helps the model make more reliable decisions by making more clear distinctions among successful examples
• Research opened the way to improve model performance in complex tasks Tags

The researchers highlight that the expected value formulation only reports the statistical success rate, is insufficient for model training. Especially in tasks that offer multiple solutions, the model can not determine which one should prefer, which leads to cognitive conflicts.

Tropical learning approaches this problem in terms of a new math and allows models to make more informed preferences. This development can significantly improve the ability of language models to solve complex problems and contribute to becoming more reliable than next generation artificial intelligence systems.