Routing Network

A routing network, also called a router or gating network, is the decision-making component in a Mixture of Experts architecture. Instead of sending every input token through every expert sub-network, the router scores the available experts and selects the top-k experts most likely to handle that token. The selected experts process the token, their outputs are combined, and the model continues with a sparse fraction of its total parameters active for that step.

How routing works

The router usually applies a learned projection to the token representation, producing one score per expert. A softmax or similar normalization turns those scores into selection weights. Top-1 routing sends a token to one expert; top-2 or top-k routing sends it to several and combines their outputs. The router is trained with the rest of the model, so it gradually learns patterns such as which experts are useful for code, different languages, mathematics, factual recall, or particular styles of reasoning.

The router does not usually understand a token in human terms or carry a hand-written label such as “send this to the coding expert.” It learns routing patterns from the training objective. In practice, one expert can specialize, but specialization is an emergent behavior rather than a fixed job description.

Why routing networks matter

Routing is what makes a very large sparse model practical. An MoE model can store far more total parameters than it activates for each token, increasing its representational capacity without multiplying inference compute by the full parameter count. This is why a model can be described as having a large total parameter count while behaving, in compute terms, more like a much smaller dense model on each step.

The trade-off is systems complexity. Tokens must be dispatched to experts, sometimes across different GPUs or machines, then gathered back together. Communication, memory placement, batch size, and the router's decisions all affect real-world latency. A theoretically efficient router can still produce a slow model if its dispatch pattern creates network bottlenecks or leaves hardware underused.

Training and failure modes

The router is commonly trained with auxiliary load-balancing losses that encourage tokens to spread across experts. Capacity factors limit how many tokens an expert can accept in a batch; overflow tokens may be dropped, sent to another expert, or handled through a fallback path depending on the architecture. Noisy top-k gating can add exploration during training so the router does not commit too early to a small group of experts.

Poor routing creates several recognizable problems:

• Expert collapse: most tokens go to a few experts while others receive little training or inference work.
• Overloaded experts: too many tokens are assigned to one expert, causing overflow, padding, or extra communication.
• Misrouting: a token is sent to experts that have not learned useful behavior for that input, reducing quality even when the model has enough total capacity.
• Unstable specialization: experts appear to specialize during one phase of training but lose that separation after later updates.

Router quality therefore affects both capability and cost. A model can have excellent experts on paper and still underperform if the router cannot consistently send the right tokens to them.

Applications

Routing networks are used most visibly in large language models such as sparse transformer MoE systems, where they select experts for each token during text generation. The same idea applies to multimodal models, where different experts may become useful for text, images, audio, or cross-modal reasoning, and to domain-adapted systems that need capacity across programming, science, translation, or business tasks.

They are also relevant to conditional-compute systems beyond classic MoE models. A router can choose among task-specific adapters, retrieval tools, vision encoders, or specialized inference paths. In all of these cases, the goal is similar: spend computation where the input needs it instead of running every available component for every example.

Bottom line

A routing network is the traffic controller that turns a collection of experts into one sparse model. It determines which capacity is used, how balanced the hardware remains, and whether specialization improves the result or becomes wasted infrastructure. When reading an MoE model description, total parameters tell you how much capacity exists; active parameters and routing behavior tell you how much of that capacity each token actually uses.