EAGLE-3

EAGLE-3 is a speculative decoding method for making a large language model generate text faster without changing the distribution of its output. Its name expands to Extrapolation Algorithm for Greater Language-model Efficiency. EAGLE-3 was introduced by researchers from Peking University, Microsoft Research, the University of Waterloo, and the Vector Institute, and was accepted at NeurIPS 2025.

EAGLE-3 is not a chatbot or a standalone foundation model. It is a small, separately trained draft model attached to a larger target model such as Llama 3.1 8B or Llama 3.3 70B. The draft model proposes several likely next tokens cheaply. The target model checks those proposals together, accepts the valid prefix, and corrects the first rejected token. When enough proposals are accepted, the target model produces several tokens per expensive verification pass instead of one.

The plain-language idea

Normal autoregressive decoding writes one token, checks the entire model, writes the next token, and repeats. EAGLE-3 is closer to a junior writer preparing several possible continuations while a senior editor reviews a whole batch at once. A good draft saves the editor from writing every word. A bad draft is rejected, so the final text still follows the senior model's decisions.

The term lossless is important here. It does not mean compression without larger files or that every implementation is bug-free. It means the speculative sampling acceptance rule preserves the same token distribution as ordinary decoding when implemented correctly. EAGLE-3 accelerates the target model; it does not improve that model's knowledge, reasoning, or factual accuracy.

How EAGLE-3 works

The original EAGLE predicted the target model's next hidden feature and then passed that prediction through the target model's language-model head to obtain draft tokens. EAGLE-3 removes the feature-regression constraint and trains directly for token prediction. Its draft model receives a fusion of low-, middle-, and high-level hidden features recorded during the target model's previous pass. Those layers carry different kinds of information, giving the drafter a richer signal than the top layer alone.

EAGLE-3's core drafter is a lightweight, single-layer Transformer decoder. During what the paper calls training-time test, the draft model generates intermediate predictions and feeds its own predicted states back into later simulated draft steps. This exposes it during training to the same imperfect inputs it will encounter during inference. Without that simulation, a drafter trained only on correct target-model states can deteriorate rapidly after its first self-generated token.

At runtime, EAGLE-3 uses the context-aware dynamic draft tree introduced by EAGLE-2. Instead of betting on only one continuation, it proposes a tree of likely tokens and allocates more draft capacity to branches with higher estimated acceptance. The target model verifies that tree using tree attention.

EAGLE, EAGLE-2, and EAGLE-3

Version	Main idea	Reported Vicuna 13B speedup
EAGLE	Predict the target model's next hidden feature.	About 3.05x mean
EAGLE-2	Build and prune a dynamic draft tree using confidence.	About 4.22x mean
EAGLE-3	Predict tokens directly, fuse multiple target layers, and simulate drafting during training.	About 5.51x mean

These figures come from the EAGLE-3 paper at temperature 0 across MT-bench, HumanEval, GSM8K, Alpaca, and CNN/DailyMail. They are controlled research results, not a promise that every application will become 5.5 times faster.

Paper benchmark results

The following table reports mean wall-clock speedup over ordinary autoregressive decoding across the paper's five evaluation tasks.

Target model	EAGLE-2, temperature 0	EAGLE-3, temperature 0	EAGLE-3, temperature 1
Vicuna 13B	4.22x	5.51x	4.65x
Llama 3.1 8B Instruct	3.23x	4.44x	3.45x
Llama 3.3 70B Instruct	2.85x	4.12x	3.95x
DeepSeek-R1-Distill-Llama 8B	3.26x	4.16x	3.52x

The largest individual result was 6.47x on HumanEval with Vicuna 13B at temperature 0. Code contains repeated syntax and templates that can be easier for a draft model to predict. Across all tasks and target models in the paper, EAGLE-3 reported roughly 3.0x to 6.5x acceleration and a 20% to 40% improvement over EAGLE-2.

The paper also tested throughput against vLLM without speculative decoding. EAGLE-3 reported 1.75x throughput at batch size 2, 1.49x at batch size 16, 1.21x at batch size 48, and 1.01x at batch size 56. That shrinking gain shows the central limitation: speculative decoding is most useful when generation is memory-bound and spare compute is available. Large batches already use the GPU's arithmetic capacity more fully, leaving less free computation for drafting and verification.

Output quality and acceptance length

EAGLE-3 does not need a separate quality score because accepted output is verified by the unchanged target model. The paper instead measures speedup ratio and average acceptance length, the average number of tokens completed per draft-and-verify cycle. On Llama 3.1 8B at temperature 0, EAGLE-3 increased mean acceptance length from 4.11 tokens with EAGLE-2 to 6.23 while raising mean speedup from 3.23x to 4.44x.

Higher acceptance is not automatically higher end-to-end speed. The draft model, tree construction, target verification, memory movement, and serving framework all add overhead. The useful production metric is measured latency or throughput on the exact model, hardware, batch size, prompt distribution, and decoding settings.

Models, training, and deployment

EAGLE-3 requires a compatible draft checkpoint trained for the specific target model. A checkpoint for Llama 3.1 8B cannot simply be attached to an unrelated model with a different vocabulary, hidden size, or internal layers. Creating a new drafter requires access to the target model's intermediate features and a training pipeline; it does not change the target model's weights.

The official Apache 2.0 implementation provides checkpoints for several Llama, Vicuna, DeepSeek-distilled, and other models. EAGLE support is also available across serving and optimization projects including SGLang, vLLM-related speculators, TensorRT-LLM, NVIDIA Model Optimizer, NeMo, AMD ROCm, MLC-LLM, and PaddleNLP, though supported versions and checkpoint formats differ. The project recommends comparing official checkpoints and official draft-tree configurations because unofficial drafters can produce different acceptance and speed results.

When to use EAGLE-3

EAGLE-3 is most attractive for latency-sensitive, batch-one or low-batch generation; long responses where decode time dominates prompt processing; repeated production use that justifies training or obtaining a matched draft checkpoint; and self-hosted deployments where both target-model features and inference configuration are controllable.

When it may not help

It may provide little benefit when prompts are long but outputs are short, because speculative decoding accelerates the decode phase rather than the initial prompt prefill. Gains also shrink at high batch sizes, with poor draft-target alignment, on compute-bound hardware, or when framework overhead dominates. Hosted API users generally cannot add EAGLE-3 unless the provider exposes it. Memory use can also rise because the system must load the drafter and keep extra states and draft-tree data.

EAGLE-3 vs. DFlash

Both methods use cheap proposals plus target-model verification, but their drafters are different. EAGLE-3 autoregressively builds a dynamic token tree from fused internal features of the target model. DFlash uses a small block-diffusion drafter to propose a group of tokens in parallel and iteratively refine it. Both need a drafter matched to the target model, and neither makes unverified draft tokens part of the final answer.

Bottom line

EAGLE-3 is one of the strongest specialized methods for lossless, low-batch LLM decoding acceleration. Its headline speedups are credible within the paper's controlled setup, and its support in major serving stacks makes it more than a research diagram. The practical question is not whether EAGLE-3 can reach 6.5x on one benchmark. It is whether a matched checkpoint, low enough batch size, and memory-bound workload let its accepted drafts save more time than the drafting machinery costs.