Overview
At a glance
Jeva treats fast language decisions as an evolving request state reading an unordered memory of candidate options. A shared encoder processes each text independently; a compact attention block makes the final judgment depend on the full candidate set while leaving option representations reusable.
A separate coverage gate estimates whether any option applies. Shared scoring supports changing candidate counts, and symmetric aggregation makes probabilities permutation-equivariant. Jeva-4B performs strongly across typed decisions and null detection, scores up to 2,048 candidates, and reduces repeated-request latency through caching.
The guarantees concern fixed model settings and exact arithmetic; empirical speedups depend on batching and reuse. Null is answer absence, not merely uncertainty among acceptable options.
1 Introduction
Choosing tools, retrieving documents, checking conditions, and assigning scores often require a compact output rather than generated reasoning [18] [19] [20]. Jointly encoding the request and candidate list couples reusable content to each request and exposes choices to candidate order.
Jeva separates semantic encoding, set-dependent judgment, and answer availability. It aims to provide four structural properties together: explicit null outcomes, permutation symmetry, variable candidate counts, and reusable option computation. Its set-conditioned state can reverse a pairwise preference when surrounding alternatives change, unlike independent scoring.
2 Related Work
Typed models read answer-token logits, decision markers, or dedicated prediction heads [15] [16] [22] [24]. Independent encoders enable reuse in retrieval and contrastive decision models [5] [6] [17]. Jeva adds collective judgment over the option set.
Its shared transformations and symmetric aggregation build on set models and latent attention [2] [3] [4]. Coverage estimates answerability; it differs from selective abstention to control error and from post-training probability calibration [8] [9] [10].
3 Jeva: A Native Decision Architecture
The backbone supplies reusable text representations; a small decision module updates the request against fixed candidate memory.
3.1 Independent Encoding, Collective Decisions
The request template includes decision type, question, and state. Each option template includes only decision type and its own text, excluding position, current request, and other candidates. Last-nonpadding-token pooling and learned projections form the request state and normalized option vectors.
Independent softmax scores have pairwise log odds unaffected by adding other options. Jeva instead refines the scoring state using the candidate set, allowing context-dependent preference changes without recomputing the options.
Equations 1–2
| (1) | ||||
| (2) |
3.2 Candidate-Conditioned State Refinement
Each multi-head block forms normalized queries and keys, then aggregates option values with temperature-controlled attention. Residual mixer and SwiGLU updates alter only the request state. A learned coupling bounds the mixer coefficient divided by temperature by 1/√L.
Proposition 1: a single block can uniquely choose A from one set and B from another containing the same A and B. The constructive example changes only a third candidate. Independent scoring with fixed tie-breaking cannot reproduce both choices. This is an expressivity result, not a claim that adding an option should always change the answer.
| Typed | BANKING77 | MASSIVE | Mind2Web | |||||
| Model | Test | Task | Website | Domain | ||||
| Jev-1.13.0 | 74.10 | 86.14 | 82.50 | 84.63 | 81.61 | 66.26 | 59.97 | 62.47 |
| JevK5 | 62.30 | 78.18 | 69.03 | 80.16 | 69.70 | 51.47 | 43.30 | 45.60 |
| InternDecision-4B | 78.70 | 80.78 | 75.03 | 81.20 | 75.79 | 48.02 | 41.17 | 43.61 |
| InternDecision-2B | 75.85 | 48.31 | 35.10 | 45.33 | 41.69 | 29.43 | 27.47 | 26.12 |
| InternDecision-0.8B | 69.35 | 29.74 | 22.63 | 28.58 | 24.61 | 25.33 | 22.53 | 21.48 |
| Re·Flex-4B | 58.85 | 78.77 | 74.38 | 76.63 | 71.39 | 52.08 | 45.05 | 47.64 |
| SemiF-4B | 62.75 | 69.97 | 67.18 | 69.70 | 67.38 | 48.60† | 41.16† | 43.67† |
| KEV-4B (BF16) | 66.00 | 83.47 | 82.47 | 76.60 | 73.54 | 40.32 | 34.17 | 35.62 |
| KEV-4B (FP32) | 66.05 | 83.44 | 82.34 | 76.53 | 73.64 | 40.53 | 34.25 | 35.81 |
| CLM-8B | 43.05 | 26.95 | 18.15 | 33.36 | 23.23 | 21.73 | 21.16 | 19.79 |
| Valen-Preview-2B | 37.75 | 19.61 | 19.09 | 19.60 | 21.72 | 13.27 | 14.16 | 14.38 |
| Laya-English | 34.50 | 58.05 | 50.49 | 44.69 | 37.63 | 15.96 | 11.34 | 15.26 |
| Qwen3.5-4B label LP | 50.85 | 49.84 | 43.12 | 47.31 | 41.63 | 18.14 | 15.30 | 18.34 |
| Qwen3.5-2B label LP | 41.70 | 48.25 | 35.65 | 30.93 | 25.25 | 12.46 | 10.73 | 11.47 |
| Jeva-4B | 78.30 | 92.05 | 91.62 | 91.09 | 90.22 | 79.99 | 62.25 | 66.23 |
Equations 3–5
| (3) | ||||
| (4) | ||||
| (5) |
3.3 Competition, Coverage, and Typed Outputs
Final state–option dot products yield a conditional distribution q. A sigmoid of the maximum score relative to a learned threshold gives coverage ρ. Full probabilities are ρq for candidates and 1−ρ for null.
Proposition 2 applies at fixed final scores: adding scores below the existing maximum leaves coverage unchanged, unlike a flat softmax whose null probability shrinks with candidate count. Actual set refinement can change scores, and the joint argmax can change even with fixed coverage.
Choice selects an option; Noul returns the affirmative candidate's conditional probability; Score returns a numerical expectation under q. Separate temperatures control attention, competition, and coverage.
| Noul | Score | Choice permutations | |||||
| Model | Brier | MAE | MAE | RMSE | Flip (%) | All-order (%) | Mean TV |
| Jev-1.13.0 | 0.0342 | 0.1456 | 0.3744 | 0.4795 | 17.00 | 66.67 | 0.0858 |
| JevK5 | 0.0987 | 0.2448 | 0.4370 | 0.5935 | 26.17 | 47.83 | 0.116 |
| InternDecision-4B | 0.0340 | 0.1431 | 0.2488 | 0.3208 | 22.67 | 64.50 | 0.106 |
| InternDecision-2B | 0.0440 | 0.1649 | 0.2641 | 0.3433 | 26.83 | 60.33 | 0.124 |
| InternDecision-0.8B | 0.0503 | 0.1835 | 0.3145 | 0.4013 | 45.00 | 45.33 | 0.135 |
| Re·Flex-4B | 0.1768 | 0.3464 | 0.4795 | 0.6490 | 26.50 | 54.67 | 0.116 |
| SemiF-4B | 0.0913 | 0.2439 | 0.4722 | 0.6134 | 32.33 | 50.33 | 0.165 |
| KEV-4B (BF16) | 0.0888 | 0.2282 | 0.3987 | 0.5669 | 17.50 | 55.50 | 0.0658 |
| KEV-4B (FP32) | 0.0887 | 0.2283 | 0.3989 | 0.5675 | 17.17 | 55.83 | 0.0657 |
| Valen-Preview-2B | 0.1509 | 0.3348 | 0.7786 | 0.9274 | 31.17 | 24.17 | 0.139 |
| Laya-English | 0.1514 | 0.3206 | 0.7083 | 0.8383 | 66.83 | 11.83 | 0.139 |
| Qwen3.5-4B label LP | 0.1623 | 0.3213 | 0.8095 | 1.0235 | 84.50 | 14.17 | 0.414 |
| Qwen3.5-2B label LP | 0.1411 | 0.3224 | 0.6221 | 0.7730 | 76.83 | 18.17 | 0.199 |
| Jeva-4B | 0.0184 | 0.0939 | 0.2315 | 0.3222 | 0.00 | 76.67 | 3.17e-08 |
Equations 6–7
| (6) |
| (7) |
3.4 Symmetry, Reuse, and Scaling
Atomic memory stores each option vector and its layer-specific key/value projections. Exact-set caching reuses an ordered candidate tuple; hybrid caching also assembles new tuples from previously seen options. Every request still undergoes encoding, refinement, and readout.
Proposition 3: with fixed settings, deterministic exact arithmetic, and identical cached entries, permutation only reorders candidate probabilities, and reuse preserves outputs. Unique winners retain identity; tied winners require identity-based tie-breaking. Prepared-memory decision work is linear in candidate count at fixed width/depth. Each text still has an individual length limit.
Equation 8
| (8) |
3.5 Direct Supervised Learning
Open-set training minimizes joint cross-entropy: a coverage term plus conditional candidate ranking. All-null examples supply no ranking target under this primary loss. Closed-set examples use candidate cross-entropy; type-specific auxiliaries are detailed below.
Jeva-4B uses Qwen3.5-4B, a 768-wide decision state, one block, eight heads, and a 3,072-wide FFN. Rank-64 LoRA trains alongside the projections and 13.37M-parameter decision machinery [7]. A staged curriculum covers typed judgments, verification, reasoning, and large-set retrieval. Serving merges LoRA and prepares memories for fixed weights.
Equation 9
| (9) |
4 Experiments
Evaluation combines 2,000 Typed Decisions examples, BANKING77 and MASSIVE intent classification at 8/16 candidates, and three Mind2Web slices at 16 candidates [11] [12] [26] [27]. Baselines keep native interfaces and runtimes. Open-set adapters use either an explicit None candidate or a separate availability query.
Accuracy includes null and uses first-argmax on fully supported slices; unsupported requests are reported separately. Latency uses synchronized H200 measurements. Within-model cache/batching comparisons hold the checkpoint and request stream fixed.
4.1 General and typed decision quality
Jeva leads seven of eight general-evaluation slices, exceeding 90% on both intent datasets at both candidate counts. Its Typed Decisions accuracy is 78.30%, slightly below InternDecision-4B's 78.70%.
Jeva achieves Noul Brier 0.0184 and Score MAE 0.2315; Score RMSE is 0.3222 versus InternDecision-4B's 0.3208. Across 600 Choice examples and up to five orders, it records zero flips, 76.67% all-order accuracy, and mean TV 3.17×10⁻⁸. This tests specific permutations, while the structural proof supplies the exact-arithmetic symmetry claim.
4.2 Null detection
On 3,080 matched BANKING77 requests with 16 candidates, including 588 answer-absent cases, Jeva reaches 98.66% null-detection AUROC. It detects 93.88% of absent answers at 5.02% FPR. Operating thresholds are empirical 95th percentiles of each model's answer-present scores; these measurements do not establish calibration on new distributions.
4.3 Candidate scaling and batching
ToolRet uses 500 requests in six distinct-query cohorts spanning 32–2,048 candidates; 389 requests have multiple positives, and added negatives can be weak or unjudged. Jeva covers all requests with 78.40% aggregate accuracy versus recursive JevK5's 64.00%. Unsupported slices are excluded from accuracy comparisons.
Length-aware batching cuts Jeva's median latency at 2,048 options from 94.459 to 12.040 seconds without changing cohort accuracy. At that size Jeva scores 69.88%, versus JevK5's 67.47% at 28 seconds.
A separate test expands nested sets for the same 83 queries. Jeva stays correct at every size on 58 queries (69.88%), compared with 48 for JevK5 and seven for CLM.
4.4 Reuse across states and candidate sets
In the main-text MLDR experiment, all novel candidate tuples miss the exact-set cache. Hybrid reuse cuts median latency from 2,102.0 to 75.1 ms at full reuse, a 27.98× ratio of medians; median paired speedup is 27.86×. Partial reuse yields 1.95×. Shared-bank Typed Decisions latency roughly halves.
Only option computation is reused. MLDR priming is excluded from request latency, while Typed Decisions timing includes cold starts. No decisions change across cache modes, although maximum absolute probability difference reaches 0.00736. Main text/Figure 9 report 50 novel-set queries per reuse level; Appendix C.5 instead reports 46.
5 Conclusion
Jeva combines independent option encoding with collective decision-making, explicit answer availability, and reusable memory. Its results support these architectural choices within the tested tasks and hardware. Larger candidate counts, alternative distributions, near ties, and end-to-end cache-priming costs still require separate evaluation.
Appendix A Derivations and Structural Properties
A.1 Entropy-Regularized Candidate Reading
The attention weights uniquely maximize compatibility plus temperature-weighted entropy over the probability simplex. Learned value and output maps convert this evidence allocation into a request-state update.
Equation 10
| (10) | ||||
Why this aggregation rule?
Subtracting KL divergence from the log-partition expression proves the softmax optimum. These attention weights gather evidence; they are not the final decision probabilities.
Equation 11
| (11) |
Block parameterization.
Projection matrices start at identity and the SwiGLU output starts at zero. Positive temperatures use a softplus parameterization; coupling the mixer scale to temperature and depth bounds their ratio.
Separate temperatures for separate questions.
For fixed compatibility logits, attention entropy increases with temperature. Keeping attention, competition, and coverage temperatures separate permits independent tuning; their numerical values are not directly comparable across different score scales.
A.2 Geometry and Single-Block Expressivity
Geometry of the selection correction.
The score-space expansion shows how attention to candidate j can affect candidate i through learned geometry. The two-dimensional counterexample uses identity projections and zero FFN output to reverse A/B preference by swapping a third candidate. Copying its coordinates across heads preserves the construction.
The implied pairwise interaction matrix is an algebraic description; the implementation computes the aggregated message without materializing a quadratic matrix.
Equations 12–15
| (12) |
| (13) |
| (14) |
| (15) |
A.3 Coverage and Candidate Count
The max-based coverage gate removes the compulsory log K effect of a flat null-inclusive softmax at fixed scores. Candidate competition still redistributes probability, and state refinement can change the scores themselves. Coverage thresholding and joint maximum-probability selection therefore answer different questions.
Equation 16
| (16) | ||||
A.4 Detailed Reuse and Symmetry Proofs
Exact reuse.
Cached option entries depend on neither the request nor other candidates. Substitution therefore preserves every subsequent operation when entries match exactly. Preparing projections costs O(LKd²); prepared-memory decision work costs O(L(d²+Kd+df)), plus request encoding. Linear output cost is unavoidable for a dense K-candidate distribution.
Candidate symmetry.
Reordering complete candidates and their metadata only reindexes attention sums. Induction preserves the request state and coverage while permuting candidate probabilities; Noul and Score remain unchanged. Multiplicity and changes in set composition are meaningful inputs and fall outside this permutation statement.
Equation 17
| (17) |
A.5 Joint Cross-Entropy Identity
Expanding cross-entropy under p=(1−ρ,ρq) yields the coverage-plus-ranking loss. All-null targets train availability without ranking candidates; answer-present targets train both components.
Equation 18
| (18) | ||||
Appendix B Method Proofs and Implementation Details
B.1 The Single-Block Construction in Higher Dimensions
Replicating the two-dimensional construction across heads with 1/√H scaling preserves dot products, normalized attention, and final scores, including at the deployed 768 dimensions/eight heads. A shared coverage factor cannot fix the independent scorer's pairwise-order limitation.
Equation
B.2 Temperature and the Sensitivity of Candidate Aggregation
Entropy at fixed compatibility.
Differentiation gives attention-entropy derivative Var(c)/τ³≥0 for fixed logits. This is a statement about changing temperature while holding compatibilities fixed.
Equations 19
| (19) |
Why the matrix model involves cross-covariance.
The value-mean derivative is a key/value cross-covariance divided by temperature. With unit-bounded keys and value norm at most R, its operator norm is bounded by R/τ. Different keys and values mean the matrix need not be symmetric or positive semidefinite; a full block derivative also includes projections, normalization, and residual branches.
Equation 20
| (20) | ||||
B.3 A Quantitative Bound for Added Candidates
For a fixed incoming query, added candidates separated by a compatibility gap receive bounded total attention mass. The resulting value-mean change is at most twice that mass times the value-norm bound. Output projection and local FFN sensitivity extend this to a pairwise-score bound.
The single-block comparison has the same incoming query automatically. Multiple blocks additionally need bounds on changes to preceding states.
Equations 21–22
| (21) |
| (22) |
B.4 Joint Log Loss and the Meaning of Null
Excess population log loss splits into coverage KL plus conditional competition KL weighted by true answer availability. If no candidate can be valid, the competition term vanishes.
Null wins the joint argmax when the coverage logit is below −log(qmax). Under uniform competition this threshold is log K: joint selection still depends on candidate count even when coverage itself is fixed.
Equations 23–24
| (23) | ||||
| (24) |
B.5 Symmetry, Cache Identity, and Numerical Evaluation
Permutation preserves the set of tied semantic winners, but first-index tie-breaking can select a different member. Floating-point reductions and batching can perturb nearly tied scores.
Reuse requires unchanged weights, tokenizer, templates, length policy, and numeric configuration. Repeated options preserve multiplicity. Training updates invalidate cached activations; inference disables dropout.
B.6 Detailed Computational Cost
Prepared memories eliminate the 2Kd² option-projection term per layer. Attention, aggregation, readout, and gathering remain linear in candidate count; no quadratic interaction tensor is formed. Each atomic entry stores d+2Ld scalars, with keys in FP32; assembled-set caches add storage.
Equation
B.7 Encoding and Numerical Configuration
Inputs use no chat template, right padding, and last-nonpadding pooling. The collator can truncate, so full-text evaluation must check lengths explicitly. Initial/final states are not normalized; the FFN uses RMSNorm with ε=10⁻⁶.
The backbone runs in BF16 while normalization, compatibility, softmax, and coverage use FP32. LoRA rank/scaling are 64/128 with 0.05 training dropout. Initial set temperature is 0.15, competition/coverage temperatures 0.10, and mixer-to-temperature ratio 0.60/√L.
B.8 Type-Specific Losses and Optimization
Score auxiliaries penalize numerical expectation error and cumulative-distribution discrepancy; Noul uses squared probability error and a margin term for hard labels. Unlike the primary joint loss, enabled Score auxiliaries are still evaluated for all-null targets in the current implementation.
Training clips global gradient norm at 1.0 and rejects nonfinite loss/gradient values. Gradient checkpointing reduces LoRA activation memory; a frozen backbone runs without gradients while projections and the decision core train.
Equation
Appendix C Detailed Experimental Protocols
C.1 Training curriculum
The curriculum progresses through basic typed/intent decisions, preferences and reasoning, counterfactual verification, evidence-grounded open sets, reading/logical reasoning, difficult examples with replay, targeted refinement, long-text retrieval, capability retention, and numerical scoring.
Reported per-stage sizes include replay. Teacher probabilities are accepted in one stage only when the teacher agrees with the original top target under two shuffled orders. Supervision combines hard labels, soft distributions, null targets, and numerical rubrics.
C.2 General evaluation and typed targets
Typed Decisions contains 600 Choice, 600 Noul, and 800 Score examples. Intent evaluation includes answer-present/withheld requests; Mind2Web selects from supplied pools. General accuracy uses first-argmax including null. Noul Brier measures probability error; Score MAE/RMSE use original rubric units.
C.3 Null operating points and candidate permutations
Null evaluation uses 2,492 answer-present and 588 absent BANKING77 cases. The 600-example permutation test uses original, reversed, and up to three seeded random orders, aligning outputs by candidate identity. A flip means any changed decision; all-order accuracy requires every tested order to be correct.
C.4 Large-set support and bounded option encoding
ToolRet assigns 83–84 distinct queries to each of six candidate-count cohorts. Jeva completes all 500; both InternDecision variants cover only 65/84 requests at 32 candidates, with larger slices outside their interfaces. Accuracy accepts a top-scoring target and is plotted only for full-support slices. H200 request timing excludes loading and uses each model's own runtime.
C.5 Cache conditions, timing, and numerical agreement
The appendix describes 171 cache episodes and 2,188 measured request identities with 32-row/16,384-token batching, cyclic mode ordering, and resets. Novel-set priming uses different questions and sets, guaranteeing no exact-set hit, and is excluded from request latency.
This section states 46 novel-set requests per reuse level, whereas the main text and Figure 9 state 50. The edition preserves the inconsistency rather than treating the counts as interchangeable.
C.6 Input Templates and Typed Examples
Encoder inputs.
The following exact templates separately encode the request and each candidate. Candidate inputs exclude the current request, other candidates, and position identifiers.
Decision type: {decision_type}
Question: {question}
State:
{state}
Option for {decision_type}: {option}
Choice.
The duplicate-charge example selects among Billing, Technical support, and Delivery. Closed-set selection uses q; allowing null uses the coverage-weighted distribution without encoding a new None candidate.
Decision type: choice Question: Which team should handle this? State: I was charged twice.
Option for choice: Billing Option for choice: Technical support Option for choice: Delivery
Noul.
The default Boolean candidates are false/true. Zero-based positive_index=1 selects the affirmative candidate's probability, not answer-absence probability. Their representations can be reused across Noul questions.
Option for noul: false Option for noul: true
Score.
With Low/Medium/High rubric values 0/1/2, Score returns q₂+2q₃. Numeric values, positive_index, and allow_null control output interpretation rather than adding encoder fields.
Option for score: Low Option for score: Medium Option for score: High
C.7 Evaluation Adapters and Cache Examples
Auxiliary coverage queries.
Several baselines estimate availability with the exact auxiliary instruction below, preserving their native interfaces. SemiF instead answers it as a separate two-option categorical request. The affirmative probability scales the baseline's conditional candidate distribution; remaining mass becomes null.
{question}
Determine whether at least one of the following candidates correctly applies to the state:
- {option_1}
- {option_2}
…
- {option_K}true: At least one supplied candidate applies. false: None of the supplied candidates applies.
Equation
Explicit null candidates.
JevK5 appends the literal candidate None; CLM appends the exact text below. Evaluation uses first-argmax over null and original candidates. This rule differs from the threshold sweep used for null-detection AUROC.
None of the above. None of the listed options applies to this request.
Reuse across new states and sets.
For sets (A,B,C) then (C,A,D), hybrid caching reuses C and A, computes D, and gathers entries in the new order. Exact-set caching misses the second tuple. Here A/B/C/D denote Billing, Technical support, Delivery, and Account security; letters are not inserted into encoder inputs.
New request encoding and decision computation always run. The same pattern applies to long documents, persistent tool registries, and repeated scoring rubrics.
Equation
Cache identity.
Individual-option cache keys use formatted token sequences; set keys use their ordered tuples. Decision type therefore enters cache identity. Weight or encoder-setting changes require invalidation; exact reuse assumes cached entries equal recomputed entries.