The Calculation Power Measured in Token Production of Language Models
Large language models (LLM) separate an equal amount of calculation resources to each while generating tokens. Yet some coins can be estimated much harder than others. The new research released in ArXiv aims to measure and optimize this calculation wasting.
The main findings and approach of research:
-
Mixture-of-Agents (MoA) Method: The actual calculation requirement of each token is measured using 15 different language models in increased capacity.
-
Spekulative Solution and Model Routing: The existing optimization techniques assume that many tokens are taken unnecessary processing, but this claim is verified by systematic data for the first time.
-
Accounting Expense Issue: The separation of equal resources to all tokens, the significant resources of tokens made simple predictions areted.
-
** Optimization Opportunity with the Method of Charge**: The determination of the actual calculation cost of tokens opens doors to dynamic resource allocation and more efficient methods of interest.
This study reveals a new size of language model yield. If there is a variable calculation cost according to the difficulty level of tokens, this information can be used to develop smart** resource allocation algorithms. As a result, both the speed of the interest can be increased and energy consumption can be reduced. Research is an important step for LLM applications, especially on mobile devices and low source.