Literature
Display
← All papers

DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

DeepSeek-AI

· arXiv (preprint) ↗

Language models10 min text · CondensedOriginal paper ↗
Contents

Overview

The opening figure compares reported agent performance and global cache size.

Refer to captionRefer to caption(a)(b)
Figure 1: (a) Performance of DeepSeek-V4.1-Flash and its counterparts on agentic benchmarks. (b) Global KV cache size per token (in bytes) across generations of DeepSeek models, highlighting DeepSeek’s sustained efforts to reduce context memory requirements. DeepSeek-V4.1-Flash achieves approximately 4-fold and 437-fold reductions in per-token global KV cache size relative to DeepSeek-V4-Flash and DeepSeek-V1, respectively.

At a glance

DeepSeek-V4.1-Flash targets input-heavy, long-context agent workloads. The report describes a 552-billion-parameter multimodal backbone plus 196 billion Engram memory parameters, with 8 billion parameters active per token during prefill and 16 billion during decoding. CED, cross-layer sparse-attention reuse, and four-bit caching reduce global KV storage to 890 bytes per token. Persistent cache storage is reported at about one eighth of DeepSeek-V4-Flash under matched workloads. The model is trained on 45 trillion multimodal tokens and supports contexts up to one million tokens.

1 Introduction

Long-running agents repeatedly process tool outputs and move large attention caches. The design attacks prefill computation, GPU-resident global KV, and persistent cache storage separately. Its compression ratios refer to these different resources rather than total model memory or an unconditional end-to-end speedup.

Related references: [19], [21], [80], [12], [11], [70], [51], [16], [64], [48].

Refer to caption
Figure 2: Single-token Decode FLOPs versus context length across generations of DeepSeek models. We account for compute precision by weighting BF16, FP8, and FP4 operations by 11, 0.50.5, and 0.250.25, respectively. DeepSeek-V4.1-Flash maintains nearly constant Decode FLOPs as context length increases, substantially reducing the computational cost of long-context scenarios.
Original figure
Figure 3: Overall architecture of DeepSeek-V4.1-Flash. The 40-layer network is divided into a causal encoder and a decoder, each with 20 layers. All feed-forward layers use standard DeepSeekMoE. The first two encoder layers use sliding window attention (SWA); the rest use Compressed Sparse Attention 2 (CSA2), with CSA2(ratio, mode) specifying the compression ratio and mode. The model also uses Single-Pass mHC, Engram, DSpark, and a Hierarchical Sparse Indexer.

2 Architecture

2.1 Overview

A 20-layer causal encoder feeds a 20-layer decoder. Global cache sharing and sparse index reuse reduce attention costs, while local sliding-window attention remains layer-specific. Images are encoded and projected into the same sequence as text. Conditional memory, residual-stream fusion, and speculative decoding address additional costs.

Related references: [15], [74], [80], [12], [11].

2.1.1 Multimodal Architecture

The vision encoder produces a spatial feature grid. A 3×3 pixel-unshuffle reduces token count before an MLP projects visual features into the language-model embedding space.

DeepSeek-ViT

DeepSeek-ViT uses two-dimensional rotary positions for variable resolutions, linear patch projection, RMSNorm, and SwiGLU. Spatial downsampling reduces visual tokens ninefold and supports inputs up to roughly 1344×1344 pixels.

Related references: [25], [92], [63].

Multimodal Auxiliary-loss-free Load Balancing for MoEs

Separate routing-bias corrections balance expert use for image and text tokens. Combining their loads would conceal modality-specific imbalances.

Related references: [74].

2.2 Causal Encoder-Decoder (CED)

Decoder global KV is projected from the final encoder state rather than computed through every decoder layer. Most prompt tokens can therefore skip decoder computation. Decoder-local attention still needs a short replay; for long sequences, the resulting prefill cost approaches half of a full 40-layer pass.

Related references: [67], [9].

Equation 1
Cl=HL/2​WlK​V,Zl=HL/2​WlZ,l>L2,C_{l}=H_{L/2}W_{l}^{KV},\quad Z_{l}=H_{L/2}W_{l}^{Z},\quad l>\frac{L}{2}, (1)
Original figure
Figure 4: Three operating modes of CSA2. The modes differ in how they obtain main KV, indexer K, and Top-K indices. Green blocks indicate quantities computed in the current layer; yellow blocks indicate main KV and indexer K reused from the most recent Full Mode layer; while red blocks indicate Top-K indices reused from the most recent index-producing (Full or Reindex Mode) layer. All three modes compute main Q and SWA KV in the current layer.

2.3 Compressed Sparse Attention 2 (CSA2)

CSA2 compresses along entry size, sequence length, and layer count. Cache sharing and sparse-selection reuse are independent choices. The compressor removes overlapping source windows and projects indexer keys from main KV, simplifying the training path.

Related references: [1], [17], [21], [7], [5], [68], [28].

2.3.1 Cross-Layer KV and Index Reuse

Full layers build caches and sparse selections. Reindex layers share caches but choose new entries. Reuse layers share both caches and the latest selection. Every layer still computes its own query, local KV, and attention output.

Original figure
Figure 5: Hierarchical Sparse Indexer. Each square represents a position; green squares mark selected indices, and blue rectangles mark blocks selected based on their maximum indexer scores. The decoder’s first CSA2 layer in Full mode selects its own Top-512 indices and builds a shared candidate pool from the selected blocks for subsequent layers. CSA2 layers in Reindex mode then select their Top-512 indices from this pool.

2.3.2 Hierarchical Sparse Indexer

The first decoder indexer searches the whole visible context and chooses a bounded candidate pool. Later indexers search only that pool. This bounds their cost, but the initial full-context scan remains. Post-training uses the same restriction as inference.

Related references: [82].

2.4 Efficient Architectural Extensions

2.4.1 Single-Pass m HC

Single-Pass mHC uses input-mixing coefficients produced by the preceding block. Removing the same-block dependency permits one fused residual-update, mixing, and coefficient-prediction pass, halving the original implementation’s activation-memory traffic.

Related references: [80].

Equations 2–6
Xl+1=Bl​Xl+Cl​ℱl​(Al​Xl),(Al,Bl,Cl)=ℋ⁡(Xl),X_{l+1}=B_{l}X_{l}+C_{l}\,\mathcal{F}_{l}(A_{l}X_{l}),\qquad(A_{l},B_{l},C_{l})=\mathcal{H}(X_{l}), (2)
Xl\displaystyle X_{l} =Bl−1​Xl−1+Cl−1​Yl−1\displaystyle=B_{l-1}X_{l-1}+C_{l-1}Y_{l-1} Residual update, contraction over nn (3)
(Al,Bl,Cl)\displaystyle(A_{l},B_{l},C_{l}) =ℋ⁡(Xl)\displaystyle=\mathcal{H}(X_{l}) Coefficients, contraction over n​dnd (4)
X^l\displaystyle\hat{X}_{l} =Al​Xl\displaystyle=A_{l}X_{l} Input mixing, contraction over nn (5)
Xl+1=Bl​Xl+Cl​ℱl​(Al−1​Xl),(Al,Bl,Cl)=ℋ⁡(Xl).X_{l+1}=B_{l}X_{l}+C_{l}\,\mathcal{F}_{l}(A_{l-1}X_{l}),\qquad(A_{l},B_{l},C_{l})=\mathcal{H}(X_{l}). (6)

2.4.2 Engram

Two Engram modules provide 196 billion sparsely accessed memory parameters using hashed token n-grams. Deterministic addresses permit prefetching; the design omits the earlier short convolution and changes the embedding optimizer.

Related references: [13].

2.4.3 DSpark

DSpark drafts five token positions in parallel with a three-block auxiliary model. A confidence-based scheduler chooses verification length for throughput under current load. It is trained after backbone pretraining and kept aligned during post-training without sending its loss gradients into the backbone.

Related references: [11], [18].

2.4.4 FP4 Main KV Cache

Post-training introduces quantization-aware four-bit main KV storage using small per-group scales, dequantized before attention. The sensitive sliding-window cache remains FP8. Main-cache compression therefore differs from quantizing all attention state.

Related references: [36], [60], [2].

2.5 Optimization

Head-wise Muon addresses differences between attention heads. Large embedding and prediction matrices use momentum with Sinkhorn balancing to reduce optimizer-state memory.

Related references: [93], [95], [90], [70], [61].

Basic Configurations.

Linear matrices use Muon, selected non-matrix parameters use AdamW, and embedding-style matrices use Sinkhorn-balanced updates. The vision encoder is initially frozen and unfrozen at learning-rate decay with a smaller learning rate.

Related references: [44], [37], [53], [43].

Sinkhorn-Balanced Updates for Engram / Embedding / Prediction Head.

The optimizer approximately equalizes row and column RMS of momentum updates, masking numerically negligible rows. A 0.18 scale aligns update magnitudes with the reused learning-rate schedule. The original algorithm and equations specify the procedure.

Related references: [43], [62], [94], [79], [30], [23], [86], [81].

Algorithm 1 Momentum update with Sinkhorn balancing
1: Weight Wt∈ℝm×nW_{t}\in\mathbb{R}^{m\times n}, gradient GtG_{t}, momentum Mt−1M_{t-1}, momentum coefficient β\beta, base learning rate ηt\eta_{t}, learning-rate correction γ\gamma, numerical constants ε\varepsilon and τ\tau, and an odd number of normalization steps KK
2: Mt←β​Mt−1+(1−β)​GtM_{t}\leftarrow\beta M_{t-1}+(1-\beta)G_{t}
3: G^t←β​Mt+(1−β)​Gt\widehat{G}_{t}\leftarrow\beta M_{t}+(1-\beta)G_{t} ⊳\triangleright Nesterov momentum
4: ρi←∥G^t,i,:∥2\rho_{i}\leftarrow\lVert\widehat{G}_{t,i,:}\rVert_{2} and ρ¯←1m​∑i=1mρi\bar{\rho}\leftarrow\frac{1}{m}\sum_{i=1}^{m}\rho_{i}
5: U(0)←G^tU^{(0)}\leftarrow\widehat{G}_{t}
6: Set U(0)i,:←0U^{(0)}_{i,:}\leftarrow 0 if ρi⩽τ​ρ¯\rho_{i}\leqslant\tau\bar{\rho} ⊳\triangleright Mask near-zero rows
7: for k=1,…,Kk=1,\ldots,K do
8:   if kk is odd then
9:    for all i=1,…,mi=1,\ldots,m do
10:      Ui,:(k)←Ui,:(k−1)/(∥Ui,:(k−1)∥2+ε)U^{(k)}_{i,:}\leftarrow U^{(k-1)}_{i,:}/(\lVert U^{(k-1)}_{i,:}\rVert_{2}+\varepsilon)
11:    end for
12:   else
13:    for all j=1,…,nj=1,\ldots,n do
14:      U:,j(k)←U:,j(k−1)/(∥U:,j(k−1)∥2+ε)U^{(k)}_{:,j}\leftarrow U^{(k-1)}_{:,j}/(\lVert U^{(k-1)}_{:,j}\rVert_{2}+\varepsilon)
15:    end for
16:   end if
17: end for
18: Δt←n​U(K)\Delta_{t}\leftarrow\sqrt{n}\,U^{(K)} ⊳\triangleright Convert unit row ℓ2\ell_{2} norm to unit row RMS
19: η~t←γ​ηt\widetilde{\eta}_{t}\leftarrow\gamma\eta_{t} ⊳\triangleright Match the update magnitude of Adam
20: Wt+1←Wt−η~t​ΔtW_{t+1}\leftarrow W_{t}-\widetilde{\eta}_{t}\Delta_{t}
Equation 7
Δt=n​U(K)=n​Dr​G^t​Dc,1n​∑j=1n(Δt)i​j2≈1,1m​∑i=1m(Δt)i​j2≈1,\Delta_{t}=\sqrt{n}\,U^{(K)}=\sqrt{n}\,D_{r}\widehat{G}_{t}D_{c},\qquad\frac{1}{n}\sum_{j=1}^{n}(\Delta_{t})_{ij}^{2}\approx 1,\qquad\frac{1}{m}\sum_{i=1}^{m}(\Delta_{t})_{ij}^{2}\approx 1, (7)

3 General Infrastructures

3.1 Training Infrastructure

3.1.1 Multimodal Training Infrastructure

Communication-Computation Overlap in Contrastive Learning.

Contrastive vision training overlaps gathering one modality’s features with useful forward or backward work for the other, reducing communication stalls.

Equation
Forward(V)→(Forward(T)∥AllGather(V))→∇Text→(Backward(T)∥AllGather(T))→∇Vision→Backward(V),\text{Forward}(V)\;\rightarrow\;\bigl(\text{Forward}(T)\;\|\;\text{AllGather}(V)\bigr)\;\rightarrow\;\nabla_{\text{Text}}\\ \;\rightarrow\;\bigl(\text{Backward}(T)\;\|\;\text{AllGather}(T)\bigr)\;\rightarrow\;\nabla_{\text{Vision}}\;\rightarrow\;\text{Backward}(V),

End-to-End Parallelism.

A separately replicated vision encoder runs before and after the language-model forward/backward phase. This allows independent image-load balancing while preserving the language model’s parallel schedule.

Related references: [96], [72], [71].

Long-Sequence Multimodal Training Optimization.

Images are balanced across context-parallel workers and loaded once. Reinforcement-learning rollouts transfer images incrementally and reuse decoded/preprocessed image data.

Equation
N×ρBIO<N×CBGPU⟺ρ<BIOBGPUC,\frac{N\times\rho}{B_{\mathrm{IO}}}<\frac{N\times C}{B_{\mathrm{GPU}}}\quad\Longleftrightarrow\quad\rho<\frac{B_{\mathrm{IO}}}{B_{\mathrm{GPU}}}\,C,

3.1.2 Attention Sharing Training for CSA2

Shared attention components crossing pipeline stages use synchronized shadow indexers, extended inter-stage payloads, and per-microbatch lifetime tracking. Ownership of optimization and checkpoints remains explicit.

3.1.3 Engram

Engram tables are row-sharded and prefetched before microbatches; gradients return to owning ranks after backbone backpropagation. FP8 transfers and fused normalization reduce memory traffic. Rollouts keep tables on GPUs to relieve host-memory pressure.

3.2 Inference System

Fused kernels reduce scheduling overhead, while vision encoding, prefill, and decoding can scale independently. The report gives 15 prefill kernels and 11 decode kernels for the common CSA2 Reuse-mode layers.

Related references: [42], [97], [75], [57].

3.2.1 Persistent KV Cache Management

Persistent storage retains compressed global KV, while local sliding-window state moves to a short-lived host-memory pool. Removing persistent local state and compressing global state combine to the reported eightfold reduction. Local-cache misses are recovered through bounded replay.

3.2.2 SWA Bounded Replay

Bounded replay reconstructs local attention from only the most recent window instead of the full multilayer receptive field. It intentionally approximates the original cached state.

Encoder SWA Bounded Replay.

On an encoder local-cache miss, replay recent prefix tokens while retaining the existing global cache, then process the uncached suffix. Reconstructed states depend on the cache-hit position; the authors report little response-quality loss.

Decoder SWA Bounded Replay.

Decoder replay processes only the final attention window before decoding. The result is not mathematically equivalent to a full decoder pass, and the same approximation is simulated during post-training.

4 Pre-Training

4.1 Data Construction

Text Data Curation

Text curation emphasizes complementary information, expert quality criteria, and newer code. Low-information generated or translated content is filtered as a form of implicit duplication.

Multimodal Data Curation

Multimodal data combines image–text pairs, interleaved documents, and specialist datasets. Staged filtering and deduplication precede expensive image processing; visual grounding, document reading, image-code pairs, and computer-use trajectories fill gaps in web data.

Related references: [47].

Data Integration and Deduplication

Multimodal versions replace overlapping text-only samples. The resulting corpus uses a 7:1 text-only-to-multimodal token ratio, with coordinated sampling and packing to limit duplication and padding.

4.2 Pre-Training Setups

4.2.1 Model Setups

The backbone has 40 layers and width 5120. Encoder caches compress pairs of tokens, while decoder global caches are uncompressed along sequence length and shared across layers. Attention selects 512 entries from a decoder candidate pool of at most 16,384 positions; the local window is 128 tokens. Each MoE layer has one shared expert and activates six of 384 routed experts.

Related references: [54].

4.2.2 Training Setups

Training covers 45 trillion tokens with a fixed 100.6-million-token batch. Sequence length starts at 64K and extends to one million at 34 trillion tokens. Separate modality-routing corrections and sample-level masks are retained.

Related references: [37], [43], [44], [12].

Vision Encoder Training.

The vision encoder first learns contrastively from about 47 billion image–text pairs, then trains with a temporary 4-billion-parameter language model on 236 billion tokens at higher resolutions. Only the trained vision encoder is retained for the final multimodal backbone.

Related references: [91].

4.3 Evaluations

4.3.1 Evaluation Benchmarks

Base-model evaluation covers knowledge, language understanding, reasoning, coding, mathematics, long context, and visual tasks. The original table records the benchmark-specific metrics.

Related references: [98], [76], [34], [35], [32], [26], [69], [38], [27], [89], [99], [10], [14], [33], [65], [6], [87], [50], [73], [39], [52], [46], [85].

Table 1: Comparison among DeepSeek-V4-Flash-Base, DeepSeek-V4-Pro-Base, and DeepSeek-V4.1-Flash-Base. All models are evaluated in our internal framework and share the same evaluation setting. Scores with a gap not exceeding 0.3 are considered to be at the same level. The highest score in each row is in bold font, and the second is underlined.
Benchmark (Metric) # Shots DeepSeek-V4-Flash DeepSeek-V4-Pro DeepSeek-V4.1-Flash
Base Base Base
Architecture - MoE MoE MoE
# Activated Params - 13B 49B 8B/16B
# Backbone Params - 284B 1.6T 552B
World Knowl. AGIEval (EM) 3-5-shot 83.9 84.4 83.4
MMLU-Pro (EM) 5-shot 68.3 73.5 74.1
C-Eval (EM) 5-shot 92.1 93.1 92.1
MultiLoKo (LLM-Judge) 5-shot 42.6 50.9 45.5
Simple-QA verified (EM) 25-shot 30.1 55.2 42.3
SuperGPQA (EM) 5-shot 46.5 53.9 53.1
Lang. & Reas. BBH (EM) 3-shot 86.9 87.5 86.1
BBEH (EM) 1-shot 25.4 29.8 27.2
DROP (F1) 1-shot 88.6 88.7 87.9
HellaSwag (EM) 0-shot 85.7 88.0 87.2
Code & Math BigCodeBench (Pass@1) 3-shot 56.8 59.2 60.6
HumanEval (Pass@1) 0-shot 69.5 76.8 79.4
GSM8K (EM) 8-shot 90.8 92.6 93.0
MATH (EM) 4-shot 57.4 64.5 61.1
MGSM (EM) 8-shot 85.7 84.4 80.2
Long Context LongBench-V2 (EM) 1-shot 44.7 51.5 45.2
Multimodal MMMU-Pro (EM) 4-shot - - 56.5
CVBench (EM) 4-shot - - 77.9
DocVQA (LLM-Judge) 4-shot - - 95.6
RefCOCO-avg (Acc@0.5) 0-shot - - 86.0

4.3.2 Evaluation Results

The authors report performance comparable to or above earlier DeepSeek base models on many tasks with a smaller active and cache footprint. An additional bits-per-byte comparison uses internal development corpora and is not an independently accessible benchmark.

Refer to caption
Figure 6: Bits-per-bytes (BPB) comparison of DeepSeek-V4-Flash-Base, DeepSeek-V4-Pro-Base and DeepSeek-V4.1-Flash-Base on our held-out evaluation sets. DeepSeek-V4.1-Flash-Base achieves lowest BPB on all tasks and demonstrates greater potential to serve as a strong base-model.

5 Post-Training

5.1 Post-Training Pipeline

Post-training follows supervised fine-tuning, reinforcement learning, and on-policy distillation. The emphasized contribution is the scale and quality of synthesized tasks and environments, rather than a new optimization algorithm.

Related references: [31], [45].

5.1.1 Large-Scale Agent Task Synthesis

Tasks combine a problem, executable environment, and verifier. General-agent tasks reconstruct consented workflow feedback; coding tasks use sessions and public repositories. Independent construction, solving, inspection, and repair stages check difficulty, correctness, leakage, and verifier weaknesses.

5.1.2 RL in Synthesized Tasks

Asynchronous learning spans diverse scaffolds. Sandboxes execute tasks while worker containers normalize trajectories for training. Checkpoint merging carries gains between successive reinforcement-learning runs; plotted discontinuities can therefore reflect reinitialization.

Original figure
Figure 7: Performance improves on various code agent benchmarks as RL training scales in the Minimal mode of DeepSeek Harness. Further extending maximal context length to 1M tokens continues to improve performance on extremely long-horizon tasks, e.g., Terminal-Bench v3.0.
Original figure
Figure 8: Performance improves with cumulative RL steps when jointly training across multiple versions of Claude Code (left) and across heterogeneous scaffolds, including OpenCode, Pi, and DeepSeek Harness in Standard and PTC modes (right). Performance is evaluated on DeepSWE v1.1. Lighter curves show the evaluation of individual scaffold versions or scaffolds.

5.1.3 Running Agents at Massive Scale: DSec

DSec provides sharded sandbox scheduling, local admission controls, resource isolation, checkpoint resumption, and separate treatment of latency-sensitive evaluations. The report also describes environment misuse during training and the need to harden evaluators.

5.1.4 Controllable Reasoning Effort in RL

An effort scalar changes the reward penalty for reasoning length; higher effort weakens the penalty. Comparisons within training groups use the same effort level. This learns a cost–quality tradeoff rather than enforcing a hard token budget.

Equations 8–10
zb,j∼πθ(⋅∣x,b),b∈ℬ,j=1,…,Mb.z_{b,j}\sim\pi_{\theta}(\cdot\mid x,b),\qquad b\in\mathcal{B},\quad j=1,\ldots,M_{b}. (8)
rb,jlen=−min⁡{Cmax,k⁡(b)​ℓb,jLnorm},r_{b,j}^{\mathrm{len}}=-\min\left\{C_{\max},\;k(b)\frac{\ell_{b,j}}{L_{\mathrm{norm}}}\right\}, (9)
k⁡(b)=k0​exp⁡(−b−bminτ),τ=λ​Δ​b¯,k(b)=k_{0}\exp\left(-\frac{b-b_{\min}}{\tau}\right),\qquad\tau=\lambda\overline{\Delta b}, (10)
Table 2: Mapping between the public API reasoning-effort tiers and the underlying scalar effort values bb.
API tier Effort value bb
max 100
high 75
low 50

5.2 Asynchronous Post-training Infrastructure

Asynchronous generation reduces delays from long-running agent trajectories, but requires explicit handling of data age and length imbalance.

Related references: [90], [71].

5.2.1 Overall Workflow

Sample-level dispatch maintains concurrency while rollout and training time-share devices. Training interrupts generation when enough samples are ready and replays the recorded expert routing across checkpoint boundaries.

5.2.2 Mitigating Length Bias and Off-Policy Effects

Per-dataset concurrency and early-short-sample handling reduce length bias. Dispatch limits and stale-token loss masking address off-policy samples generated by older checkpoints.

5.2.3 Performance Optimization

Token-boundary interruption and persisted cache/routing state let rollouts resume after checkpoint switches or scheduler preemption. Per-sample cleanup releases completed state.

5.2.4 Large-Scale On-Policy Distillation

The final distillation stage uses more than forty domain-specialized teachers with potentially different architectures. The infrastructure supports asynchronous teacher and data-mixture changes while samples remain in flight.

Related references: [21].

5.3 Evaluation

5.3.1 Evaluation Setup

Reasoning and agent benchmarks use task-specific contexts, sampling, and harnesses. Coding environments restrict network access and remove potentially leaking histories or caches, though the authors still observe evaluator gaming. Scores therefore depend on both model and evaluation setup.

Related references: [58], [56], [22], [51], [49], [48], [16], [84], [24], [41], [78], [77], [64], [66], [29], [8], [59].

5.3.2 Evaluation Results

Reported scores include 90.9% on GPQA Diamond, 65.6% on MathArena Apex, 74.2% on DeepSWE v1.1, and 90.6% on Terminal-Bench 2.1. These are the report’s configured evaluations, not universal performance guarantees. Visual-agent results still trail some closed-model comparisons.

Table 3: Comparison between DeepSeek-V4.1-Flash with closed/open source models. †{\dagger} denotes text-only subset of HLE. The best results are highlighted in bold; the second-best results are underlined.
Benchmark (Metric) Opus-5 GPT-5.6 Sol K3 GLM-5.3 DS-V4-Pro DS-V4-Flash DS-V4.1-Flash
Max Max Max Max Max Max Max
Reasoning GPQA Diamond (Pass@1) 93.4 94.1 92.9 88.1 92.4 89.9 90.9
HLE (Pass@1) 56.3 44.5 43.5 42.0† 42.7† 37.8† 36.8 (39.1†)
Codeforces (Rating) - - - - 3348 3289 3471
MathArena Apex (Pass@1) - - 65.6 - 65.3 58.6 65.6
Agentic Terminal-Bench 2.1 (Pass@1) 89.1 88.8 88.3 88.2 87.9 82.7 90.6
Terminal-Bench 3.0 (Pass@1) 43.3 34.4 17.7 28.3 11.8 7.6 30.0
Terminal-Bench 4.0 (Pass@1) 51.8 39.9 12.6 37.9 12.4 7.0 31.2
DeepSWE v1.1 (Resolved) 74.0 73.0 67.5 66.9 62.7 54.4 74.2
ProgramBench (Almost@1) 37.0 23.0 17.5 19.0 15.5 - 20.3
NL2Repo-Bench (Score) 75.3 56.8 58.0 58.0 61.5 54.2 65.4
CyberGym (Pass@1) - 84.5 80.0 84.5 83.3 76.7 88.1
SEC-Bench Pro (Pass@1) - 74.3 - - 56.4 30.9 62.8
ExploitGym (Pass@1) 22.1 33.7 - 15.0 5.4 1.8 15.3
HLE w/ tools (Pass@1) 63.6 - 59.8 62.5 60.0 51.5 63.9
Automation-Bench (Pass@1) 50.3 45.8 46.7 48.8 43.2 37.7 54.8
Agents’ Last Exam(Pass@1) 28.6 26.7 27.6 28.5 25.7 25.2 31.8
Chartography w/ tools (Pass@1) 84.0 79.9 68.1 - - - 78.9
BabyVision w/ tools (Pass@1) 94.1 88.9 85.7 - - - 89.6
ZeroBench-main w/ tools (Pass@5) 52.0 53.0 41.0 - - - 49.0

5.3.3 Performance across reasoning efforts

Increasing effort generally lengthens reasoning and improves the reported aggregate results, with diminishing returns. The report maps low, high, and max presets to scalar values 50, 75, and 100; intermediate values are also studied.

Original figure
Figure 9: Performance and output length as a function of reasoning effort. Each panel plots Pass@1 (solid, left axis) and mean output tokens per response (dashed, right axis) as the reasoning-effort value is varied from 25 to 100; The results of reasoning-intensive benchmarks are averaged over eight benchmarks (AIME 2026, Apex 2025 Shortlist, GPQA Diamond, HLE, IMO-AnswerBench, LiveCodeBench, MathArena-Apex, SimpleQA-Verified). DeepSWE v1.1 is evaluated based on mini-SWE and Terminal-Bench v2.1 is evaluated based on DeepSeek Harness (Minimal).

5.3.4 Performance across agent scaffolds

Eight configurations from six scaffold families keep the checkpoint and task set fixed while changing surrounding tools and prompts. The preserved table shows how performance depends on the harness.

Related references: [4], [55], [3], [88], [83], [20].

Table 4: Performance across agent scaffolds at Max reasoning effort.
Benchmark (Metric) Claude Code Codex OpenCode Pi mini-SWE DeepSeek Harness
Minimal Standard PTC
DeepSWE v1.1 (Resolved) 69.8 65.6 65.5 66.2 74.2 72.6 70.5 67.6
Terminal-Bench v2.1 (Pass@1) 88.0 84.1 85.0 86.1 90.3 90.6 85.8 85.8

Note. All scaffolds use N=8N=8 samples per task on DeepSWE v1.1 and N=3N=3 on Terminal-Bench v2.1. Runs use Linux containers, temperature 1.0, top-p 0.95, a 1M-token context window, and max_steps=500 model-generation rounds per agent on both benchmarks. Terminal-Bench v2.1 is evaluated without network access. We evaluate four Claude Code versions, with per-version results and their average reported in Appendix Table 5; this table reports v2.1.251. More scaffold-specific configurations are detailed in Appendix B.1.

5.3.5 Multi-Agent

Preliminary agent-team experiments compare the strongest observed configurations under explicit wall-clock deadlines. They use a 172-task filtered ProgramBench subset and a no-GPU FrontierSWE subset. Reported team gains should be read with those selection rules, differing metrics, and deadline budgets in mind.

Related references: [84], [40].

Original figure
Figure 10: Test-time compute scaling for single-agent and multi-agent configurations on ProgramBench (Almost@1) and FrontierSWE v2 (Mean@5) as functions of the per-rollout wall-clock deadline.

6 Conclusion, Limitations, and Future Directions

The report combines architectural changes, cache precision, serving systems, and training data to reduce long-context costs. Sparse selection and approximate local-state replay introduce robustness boundaries not fully characterized by the reported tests. The authors also acknowledge remaining gaps on difficult reasoning and edge cases despite strong benchmark scores.

Appendix

Appendix A Author List

The original appendix contains the complete DeepSeek-AI contributor list.

Appendix B Evaluation Details

B.1 Scaffold Configurations

The appendix records scaffold versions, tool interfaces, and runtime settings. These implementation details matter when reproducing cross-harness comparisons.

Table 5: Performance across Claude Code versions at Max reasoning effort.

Benchmark (Metric) v2.1.105 v2.1.238 v2.1.251 v2.1.259 Average DeepSWE v1.1 (Resolved) 68.4 68.7 69.8 68.6 68.9 Terminal-Bench v2.1 (Pass@1) 87.3 88.4 88.0 87.6 87.8

Note. Average is computed from the unrounded Pass@1 scores of the four versions.

B.2 Reasoning Efforts across Scaffolds

Output length grows with effort in every tested scaffold–benchmark pair, but success rates have plateaus and intermediate dips. Harness choice can matter as much as the effort tier on nearly saturated tasks.

Original figure
Figure 11: Reasoning effort drives trajectory length consistently but correlates only weakly with accuracy across coding scaffolds. Each panel plots Pass@1 (%, solid, left axis) and mean output tokens per trajectory (k, dashed, right axis) against the reasoning-effort setting, for DeepSWE v1.1 (top row) and Terminal-Bench v2.1 (bottom row) under three agent scaffolds: Claude Code, DeepSeek Harness (Minimal) and mini-SWE. All panels come from the same checkpoint.

B.3 Detailed Results of Reasoning Benchmarks across Reasoning Efforts

The detailed reasoning plots show benchmark-specific accuracy and token use. Higher effort brings large gains on harder mathematics and smaller gains on already saturated tasks.

Original figure
Figure 12: Performance and output length as a function of reasoning effort on eight reasoning-intensive benchmarks. Each panel plots Pass@1 (solid, left axis) and mean output tokens per response (dashed, right axis) as the reasoning-effort value is varied from 25 to 100.

Appendix C Exponential Token Penalty in Reasoning Effort Control

A local reward model assumes exponentially diminishing marginal benefit from additional reasoning. Under an uncapped interior optimum, an exponential token-penalty schedule produces an approximately affine preferred length versus effort. The derivation does not guarantee linear or pointwise-monotonic measured behavior.

Equations 11–17
rlen​(ℓ,b)=−min⁡{Cmax,k⁡(b)​ℓLnorm},r^{\mathrm{len}}(\ell,b)=-\min\left\{C_{\max},k(b)\frac{\ell}{L_{\mathrm{norm}}}\right\}, (11)
k⁡(b)=k0​exp⁡(−b−bminτ),k(b)=k_{0}\exp\left(-\frac{b-b_{\min}}{\tau}\right), (12)
ℓx∗​(b)∈arg⁡maxℓ⩾0​[px​(ℓ)−k⁡(b)​ℓLnorm].\ell_{x}^{*}(b)\in\arg\max_{\ell\geqslant 0}\left[p_{x}(\ell)-k(b)\frac{\ell}{L_{\mathrm{norm}}}\right]. (13)
px′​(ℓx∗​(b))=k⁡(b)Lnorm,p_{x}^{\prime}\!\left(\ell_{x}^{*}(b)\right)=\frac{k(b)}{L_{\mathrm{norm}}}, (14)
px′​(ℓ)≈ax​exp⁡(−ℓsx),p_{x}^{\prime}(\ell)\approx a_{x}\exp\left(-\frac{\ell}{s_{x}}\right), (15)
ℓx∗​(b)≈Cx−sx​log⁡k0+sxτ​(b−bmin),\ell_{x}^{*}(b)\approx C_{x}-s_{x}\log k_{0}+\frac{s_{x}}{\tau}(b-b_{\min}), (16)
ℓx∗​(b2)−ℓx∗​(b1)≈sxτ​(b2−b1).\ell_{x}^{*}(b_{2})-\ell_{x}^{*}(b_{1})\approx\frac{s_{x}}{\tau}(b_{2}-b_{1}). (17)
Concept demonstration

Open image ↗