Literature
Display
← All papers

Meta-Ctrl: Guaranteed Plan Generation by Decoupling Syntactic and Semantic Constraints

Gwen Yidou-Weng · Edward Sun · Tianyi Ma · Metin Alp Dogan · Benjie Wang · Allen Peng · Guy Van den Broeck · Yuchen Cui

Preprint posted · CoRL 2026 (accepted) ↗

Robotics17 min text · CondensedOriginal paper ↗
Contents

Overview

Keywords: Constrained Decoding, Formal Guarantees, Task and Motion Planning

At a glance

This edition follows the complete 26 page v 3, preserving three figures, eleven numbered tables, the unnumbered coverage table, eleven displayed equations, Algorithm 1, four exact output-format examples and five complete baseline/Meta-Ctrl plan pairs.

Meta-Ctrl factors constrained plan decoding into token-level syntax and action-level symbolic semantics. An LM proposes tokens; finite-state lookahead reweights them toward admissible completions. The formal factorization assumes a canonical bijective action/token representation; actual probabilities use HMM surrogates.

The guarantee concerns encoded constraints and the robot’s estimated state. Omitted goals, incomplete affordances, finite horizons and syntax-only fallback limit its scope. Planning validity does not guarantee perception, physical execution or the benchmark’s full task objective.

1 Introduction

Fluent plans can contain malformed actions or violate preconditions and goals. SayCan and grounded decoding softly score feasibility; LLM+P delegates to a symbolic planner. Meta-Ctrl instead constrains generation while retaining the LM as a preference over feasible choices.

A monolithic syntax/state automaton can be very large. A grounded action becomes one meta-token for semantic transitions while its surface spelling remains a token-level problem. This factors representations; it does not remove the need to specify and maintain relevant world state.

Complete original PDF block S1.F1
Figure 1: Meta-Ctrl enforces preconditions that the base LM violates. Task: put dishes in dishwasher; PUTIN requires holding an object and an open container. The base LM (top) attempts PUTIN while the dishwasher is closed. Meta-Ctrl (bottom) enforces the action-level precondition, inserting OPEN(dishwasher).

2 Preliminaries: Ctrl-G

Ctrl-G writes the exact next-token conditional as the LM probability times the probability that a valid full continuation exists under that distribution. Computing the latter exactly is intractable, so an HMM plus constraint DFA supplies a tractable surrogate.

Backward dynamic programming costs O(n|D|H²) for length n, edge count |D| and H hidden states. Zero continuation mass rules out invalid completions, but validity also requires nonzero feasible support and proper accepting termination. The surrogate changes probabilities; Equation 2 is not generally exact sampling from the base LM conditioned on the constraint.

Equations 1–2
pLM​(xt∣x<t,α)∝pLM​(xt∣x<t)⋅pLM​(α∣x≤t).p_{\mathrm{LM}}(x_{t}\mid x_{<t},\alpha)\;\propto\;p_{\mathrm{LM}}(x_{t}\mid x_{<t})\;\cdot\;p_{\mathrm{LM}}(\alpha\mid x_{\leq t}). (1)
pctrl​(xt∣x<t,α)∝pLM​(xt∣x<t)⋅pHMM​(α∣x≤t).p_{\mathrm{ctrl}}(x_{t}\mid x_{<t},\alpha)\;\propto\;p_{\mathrm{LM}}(x_{t}\mid x_{<t})\;\cdot\;p_{\mathrm{HMM}}(\alpha\mid x_{\leq t}). (2)

3 Decoupling Syntax and Semantics for Constrained Decoding

Syntax γ governs action names, arguments and structure; semantics β governs state, preconditions and goals after projecting text to actions. The joint requirement is γ together with β applied to the parsed action sequence. The two levels communicate through the action compatible with the current partial token block.

3.1 Level 1: Token-Level Syntax

A syntax DFA restricts output to allowed names, object instances, argument types and JSON-like plan format. Ctrl-G lookahead favors well-formed completions. A syntactically valid plan can still place an unheld object or stop before achieving goals; semantic state is a separate requirement.

Equation 3
p⁡(xt∣x<t,γ)∝pLM​(xt∣x<t)⋅pHMM​(γ∣x≤t).p(x_{t}\mid x_{<t},\gamma)\;\propto\;p_{\mathrm{LM}}(x_{t}\mid x_{<t})\;\cdot\;p_{\mathrm{HMM}}(\gamma\mid x_{\leq t}). (3)

3.2 Level 2: Action-Level Semantics via Meta-Tokens

The parser maps each completed action block to a grounded-action meta-token. Equations 5–8 marginalize over the current action, splitting its bridge probability, syntax completion probability and semantic continuation probability. The induced meta distribution in Equation 6 is the LM distribution conditioned on syntax, expressed through the unique inverse mapping.

Exactness assumes the stated bijection and a consistent incremental canonical representation. Appendix C later permits tokenization equivalence classes; a many-to-one case requires summing probability over those realizations or defining a quotient representation, rather than using an unqualified unique inverse. Learned token/meta HMMs are further approximations distinct from the algebraic identity.

Equation 4
x⊧α⇔(x⊧γ)∧(τ⁡(x)⊧β).x\models\alpha\;\iff\;(x\models\gamma)\;\wedge\;(\tau(x)\models\beta). (4)
Complete original PDF block S3.F2
Figure 2: Level 2: action-level semantics. The parser τ\tau projects each action block of tokens (e.g. {"GRAB":"plate"}) to one meta-token (Grab(plate)). A DFA 𝒟β\mathcal{D}_{\beta} over meta-tokens enforces preconditions and goal progress (e.g. Put requires the container open). Operating on a small action vocabulary, rather than raw tokens, is what makes the factored computation below tractable.
Original selectable diagram symbolsτ\tauβ\beta
Equations 5–8
p⁡(α∣x≤t)=∑alp⁡(al∣x≤t)⏟bridging⋅p⁡(γ∣x≤t,al)⏟syntax⋅p⁡(β∣x≤t,al,γ)⏟semantics.p(\alpha\mid x_{\leq t})=\sum_{a_{l}}\underbrace{p(a_{l}\mid x_{\leq t})}_{\text{bridging}}\cdot\underbrace{p(\gamma\mid x_{\leq t},a_{l})}_{\text{syntax}}\cdot\underbrace{p(\beta\mid x_{\leq t},a_{l},\gamma)}_{\text{semantics}}. (5)
pmeta​(w):=pLM​(τ−1​(w)|γ),p_{\mathrm{meta}}(w)\;:=\;p_{\mathrm{LM}}\!\left(\tau^{-1}(w)\,\big|\,\gamma\right), (6)
p⁡(β|x≤t,al,γ)=pmeta​(β|a≤l),p\!\left(\beta\,\big|\,x_{\leq t},a_{l},\gamma\right)\;=\;p_{\mathrm{meta}}\!\left(\beta\,\big|\,a_{\leq l}\right), (7)
p⁡(α∣x≤t)=∑alp⁡(al∣x≤t)⏟bridging⋅p⁡(γ∣x≤t,al)⏟syntax(token-level)⋅pmeta​(β∣a≤l)⏟semantics(action-level).p(\alpha\mid x_{\leq t})=\sum_{a_{l}}\underbrace{p(a_{l}\mid x_{\leq t})}_{\text{bridging}}\cdot\underbrace{p(\gamma\mid x_{\leq t},a_{l})}_{\begin{subarray}{c}\text{syntax}\\ (\text{token-level})\end{subarray}}\cdot\underbrace{p_{\mathrm{meta}}(\beta\mid a_{\leq l})}_{\begin{subarray}{c}\text{semantics}\\ (\text{action-level})\end{subarray}}. (8)

3.3 Why Two Levels?

For a median VirtualHome task, the paper estimates 350 million monolithic states versus 57 thousand factored states,1,720 trillion versus 922 billion compute units, and 107 TB versus 1.6 GB DP memory. These are construction/DP comparisons, not a measured 1,900 fold end-to-end robot speedup.

Per-action negative log likelihood is 33.54 for the raw-token HMM versus 2.01 for the meta surrogate. Their different representations and syntax treatment mean this is a fit diagnostic, not direct proof of a 16.7 fold increase in task quality. Semantic state can still grow exponentially with tracked object attributes.

Complete original PDF block S3.T1
Table 1: Why two levels: factoring the joint constraint is both cheaper and produces a better surrogate. Numbers are for a median VirtualHome task (H=128H{=}128); see Section 4 for the quality ablation. Both −log⁡p-\log p columns are per grounded action; sequence length is raw tokens (42.7) for the token HMM and grounded actions (7.6) for the meta-token HMM.
Original selectable table

Efficiency

Monolithic Two-level
States Sγ×SβS_{\gamma}{\times}S_{\beta} Sγ+SβS_{\gamma}{+}S_{\beta}
350 M 57 K
Compute 1,720 T 922 G
DP memory 107 TB 1.6 GB
∼1,900×{\sim}1{,}900\times faster, ∼67,000×{\sim}67{,}000\times less memory

Quality

Token HMM Meta-token HMM
Vocab 128 K 132
Emission 16.4 M 16.9 K
Seq. length 42.7 7.6
−log⁡p-\log p/act. 33.54 2.01
∼16.7×{\sim}16.7\times better fit

4 Experiments

Experiments cover EAI action sequencing and subgoal decomposition in VirtualHome/BEHAVIOR, LoTa-Bench WAH-NL and physical tabletop demonstrations. Ablations compare no LM, syntax masking, semantic reachability masking, uniform lookahead and trained lookahead.

4.1 Setup

EAI has 342 VirtualHome action tasks,338 subgoal tasks and 100 of each BEHAVIOR module. WAH-NL has 100 tasks. Task SR scores goal achievement; execution SR scores reaching the end of a plan; WAH also reports the fraction of subgoals satisfied. Neither metric alone captures every failure.

EAI uses Llama3-8B and gpt-oss-20B; WAH uses Llama3.1-8B. Per-task automata use shared HMMs trained on substituted benchmark-format prompts excluding evaluation instances. Decoding is greedy with a 40 action horizon. Reported infeasible cases fall back to syntax-only, so those outputs do not retain the full semantic guarantee.

4.2 Watch-And-Help via LoTa-Bench

On the same WAH LM/evaluator, raw task/subgoal/execution scores are 0/.022/.010; syntax-only gives 0/.037/.010; full Meta-Ctrl gives.470/.705/1.000. Semantic constraints account for most of this measured gain.

External GPT-4, LLaMA-1 and STEP numbers use different models, input information, executors or closed-loop operation. Table 2 correctly presents them as context; a shared subgoal metric does not remove those protocol confounds. ProgPrompt failures concentrate at GRAB, PUTIN and PUTBACK preconditions.

Complete original PDF block S4.T2
Table 2: WAH-NL (single-agent, n=100n=100) under the LoTa-Bench protocol. External references use different LMs and executors; we report them as ballpark context, not strict baselines. Empty entries are not reported. Same-LM rows (Llama 3.1 8B) are direct comparisons. †STEP is closed-loop; ours is open-loop.
Original selectable table
Method Base LM SR SSR Exec
External references (different LM / executor)
   SayCan — 0.010 0.021 —
   ProgPrompt — 0.030 0.187 —
   LoTa-Bench GPT-4 — 0.342 —
   LoTa-Bench LLaMA-1 65B — 0.433 —
   STEP† (larger) 0.400 0.620 —
Same LM (Llama 3.1 8B), same evaluator, n=100n=100
   Raw LM (unconstrained) Llama 3.1 8B 0.000 0.022 0.010
   + syntax Llama 3.1 8B 0.000 0.037 0.010
   Meta-Ctrl Llama 3.1 8B 0.470 0.705 1.000

4.3 Embodied Agent Interface

Llama3-8B improves VirtualHome action task success 21.3→90.4% and execution 23.6→97.7%; subgoal task success 48.8→89.9%. BEHAVIOR gains are smaller: action 10→34% and subgoal 22→36%, while execution increases strongly. The gpt-oss action-task BEHAVIOR score stays 40%.

Under identical automata, VirtualHome action task scores are 83.8 without an LM,81.7 with reachability masking,90.7 uniform lookahead and 90.4 trained lookahead. On BEHAVIOR subgoals, no-LM search reaches 52%, exceeding the LM variants 36–41%. Thus constraints carry substantial solution structure and adding an LM or trained surrogate is not uniformly beneficial.

Porting uniform guidance to Qwen3-8B and Gemma3-12B requires tokenizer-specific grammar rebuilding but no HMM training; their VirtualHome action scores are 90.7 and 89.4%. Token-HMM replacement alone leaves 273/342 plans byte-identical. The full trained and uniform configurations should not be conflated.

Older-leaderboard wins are qualified by Appendix K’s newer-model results. Reported zero LM-fault syntax errors also exclude evaluator/pipeline mismatches listed in Appendix H; they do not mean zero total evaluator rejections.

Complete original PDF block S4.T3
Table 3: Embodied Agent Interface results on VirtualHome (V; n=342n{=}342 AS, 338338 SD) and BEHAVIOR (B; n=100n{=}100 AS, 100100 SD). Task SR and Execution SR (%). Leaderboard baselines from [9]; Each base LM is grouped with its full Meta-Ctrl (γ+β\gamma{+}\beta) result. Bold: best per column; underline: second-best. ‡The gpt-oss rows use a keyword-based BEHAVIOR affordance prior; the Llama rows use the taxonomy-derived specification of Appendix G. Our VirtualHome cells are means over six evaluator seeds (Appendix K).
Original selectable table
Action Sequencing Subgoal Decomposition
Task SR Exec SR Task SR Exec SR
Model VH BEH VH BEH VH BEH VH BEH
Representative baselines
GPT-4o 71.5 47.0 81.3 53.0 87.6 49.0 91.1 55.0
Claude-3.5 Sonnet 76.1 60.0 81.3 69.0 89.1 39.0 92.0 44.0
o1-preview 65.2 81.0 72.5 91.0 89.4 60.0 93.2 62.0
Mistral Large 78.4 33.0 84.6 50.0 84.3 31.0 92.0 38.0
Llama 3 70B Instruct 59.0 34.0 66.6 42.0 78.4 21.0 87.3 30.0
Llama 3 8B Instruct (base) 21.3 10.0 23.6 16.0 48.8 22.0 58.0 29.0
   + Meta-Ctrl (ours) 90.4 34.0 97.7 89.0 89.9 36.0 93.5 66.0
gpt-oss-20B (base) 74.4 40.0 80.3 51.0 72.5 27.0 82.2 36.0
   + Meta-Ctrl (ours)‡ 86.6 40.0 94.1 90.0 82.3 41.0 86.4 66.0
Complete original PDF block S4.T4
Table 4: Porting Meta-Ctrl to newer base LMs, VH AS (n=342n{=}342), task / execution SR (%). Each port rebuilds γ\gamma over the new tokenizer automatically and uses a uniform token-level HMM over the new vocabulary; the meta level is unchanged (uniform meta-surrogate rows for all three bases). No training of any kind. VH-AS means over six evaluator seeds.
Original selectable table
Base LM unconstrained + Meta-Ctrl changed lines
Llama 3 8B Instruct 21.3 / 23.6 90.7 / 97.7 —
Qwen3-8B 50.8 / 58.4 90.7 / 97.7 ∼40{\sim}40
Gemma-3-12B-it 47.5 / 50.5 89.4 / 96.1 ∼35{\sim}35
Complete original PDF block S4.T5
Table 5: What each component contributes, task / execution SR (%) on Llama 3 8B. Rows 3–6 share the same constraint construction (meta-tokens, γ\gamma and β\beta compiled from the prompt and the public object taxonomy) and the same guarantee; they differ only in the guidance signal. Search, no LLM is breadth-first search over the meta-level automaton with the LM removed. Masking keeps only the support of the lookahead term, a reachability yes/no. Uniform and trained instantiate the lookahead measure itself. Classical planner is Fast Downward given the benchmark’s own PDDL domain, i.e. a complete transition model, strictly more information than our automata encode; SD asks for goal states rather than actions, so planners do not apply.
Original selectable table
Method (Llama 3 8B) VH AS VH SD BEH AS BEH SD
LLM only (unconstrained) 21.3 / 23.6 48.8 / 58.0 10 / 16 22 / 29
LLM + syntax-only GCD (γ\gamma mask) 1.4 / 97.3 0.0 / 96.0 12 / 20 19 / 31
Classical planner, full domain 47.2 / 84.6 n/a 13 / 37 n/a
with our meta-level automata:
Search, no LLM 83.8 / 93.1 72.5 / 79.0 31 / 77 52 / 64
LLM + masking (reachability only) 81.7 / 87.5 88.7 / 92.6 33 / 85 38 / 51
LLM + uniform lookahead 90.7 / 97.7 88.4 / 93.2 36 / 89 41 / 59
LLM + trained lookahead (Meta-Ctrl) 90.4 / 97.7 89.9 / 93.5 34 / 89 36 / 66

4.4 Real Robot Demonstrations

A calibrated RGB-D camera, Molmo, SAM2, GraspGen and a Code-as-Policies executor connect symbolic plans to XArm7 motions. Eight tasks are demonstrated; only three pick/place tasks receive twenty repetitions each.

Meta-Ctrl plan validity is 20/20 for each task, with final successful executions 16/20,18/20 and 17/20. The source describes a cascade but prints all fractions over 20; these are stage-survival counts, not conditional rates with changing denominators. Baseline perception is scored separately while baseline execution is end-to-end, so downstream columns are not directly matched comparisons.

Perception and grasp failures remain after valid planning. The automaton starts from an estimated symbolic scene and therefore cannot certify that the true physical scene satisfies its predicates.

Complete original PDF block S4.F3
Figure 3: Real robot setup with xArm 7 and deployed tabletop manipulation tasks. Top-row tasks require only pick-and-place; bottom-row tasks require specialized motion primitives (e.g. open door).
Complete original PDF block S4.T6
Table 6: Real robot evaluations on an XArm7, 2020 repetitions per task. Meta-Ctrl columns are a cascade: plan success holds by construction; perception and execution are conditional on preceding stages. Baseline perception is scored per-module, independent of plan validity; its execution is end-to-end and bounded by planning failures. Baseline downstream columns are thus not directly comparable to Meta-Ctrl’s.
Original selectable table
Meta-Ctrl (ours) Baseline (Llama 3 8B)
Task Planning Perception Execution Planning Perception Execution
Put donut in canister 20/20 17/20 16/20 2/20 18/20 1/20
Pick up green block (under blocker) 20/20 19/20 18/20 1/20 19/20 1/20
Stack dishes by size 20/20 20/20 17/20 0/20 19/20 0/20

5 Conclusion

The useful contribution is factoring finite-state constraints into two representational levels and testing guidance within that feasible set. It requires explicit DFA-expressible constraints, a valid state model and feasible acceptance within the horizon. Continuous arithmetic or unbounded-memory requirements are outside the stated class.

Incomplete specification can underconstrain or overconstrain behavior. Syntax-only fallback abandons semantic enforcement, and success still depends on sensing and control. The source’s 17 rejected gold plans do not alone establish that every alternative plan is infeasible or prove an 83% task-success ceiling.

Acknowledgments

The work acknowledges DARPA ANSR/CODORD/SAFRON support and gifts from Cisco Research, Qualcomm and Amazon. These acknowledgments do not affect the scope of the encoded-constraint guarantee.

Appendix A Construction of the Joint DFA 𝒟 α

The product automaton tracks a syntax state and semantic state. Token transitions advance syntax; completing an action additionally advances semantics. Acceptance requires both states in their respective accepting sets. Induction over prefixes establishes correspondence with the incremental parse.

The state-count upper bound is the product of component state counts. The displayed edge bound additionally absorbs alphabet/branching factors; it should not be read as a representation-independent inequality for arbitrary sparse automata on different alphabets. The factored method avoids explicitly building this product but still maintains semantic state.

Equations
δα​((qγ,qβ),v)=(qγ′,δβ​(qβ,a)).\delta_{\alpha}\bigl((q_{\gamma},q_{\beta}),v\bigr)=\bigl(q_{\gamma}^{\prime},\delta_{\beta}(q_{\beta},a)\bigr).
δα​((qγ,qβ),v)=(qγ′,qβ).\delta_{\alpha}\bigl((q_{\gamma},q_{\beta}),v\bigr)=\bigl(q_{\gamma}^{\prime},q_{\beta}\bigr).
|𝒟α|≤|Sγ|⋅|Sβ|⋅|V|≤|𝒟γ|⋅|𝒟β||\mathcal{D}_{\alpha}|\;\leq\;|S_{\gamma}|\cdot|S_{\beta}|\cdot|V|\;\leq\;|\mathcal{D}_{\gamma}|\cdot|\mathcal{D}_{\beta}|

Appendix B Two-Level Decoding Algorithm

Algorithm 1 precomputes token- and action-level backward tables, evaluates each admissible token with LM probability times factored lookahead, and greedily emits the maximum. It updates the action index at block closure and stops at EOS. All original pseudocode and symbols remain below.

The formal acceptance requirement must apply at termination. A finite horizon, feasible positive-mass continuation and correctly handled EOS are needed; the pseudocode does not turn syntax-only fallback into semantic validity.

Complete original PDF block algorithm1
Algorithm 1 Two-Level Constrained Decoding
Original selectable algorithm
Input: Prompt xpromptx_{\mathrm{prompt}}; LM pLMp_{\mathrm{LM}}; syntax HMM and DFA 𝒟γ\mathcal{D}_{\gamma}; meta-token HMM and DFA 𝒟β\mathcal{D}_{\beta}; parser τ\tau (incremental form τinc\tau_{\mathrm{inc}}).
Output: Token sequence satisfying α=γ∧(β∘τ)\alpha=\gamma\wedge(\beta\circ\tau).
Bγ←B_{\gamma}\leftarrow backward DP over HMMtoken×𝒟γ\mathrm{HMM}_{\mathrm{token}}\times\mathcal{D}_{\gamma} ; // token-level lookahead
Bβ←B_{\beta}\leftarrow backward DP over HMMmeta×𝒟β\mathrm{HMM}_{\mathrm{meta}}\times\mathcal{D}_{\beta} ; // action-level lookahead
x←xpromptx\leftarrow x_{\mathrm{prompt}}; l←1l\leftarrow 1;
repeat
   a<l←τinc​(x)a_{<l}\leftarrow\tau_{\mathrm{inc}}(x) ; // completed actions so far
   for each token v∈Vv\in V admissible at the current DγD_{\gamma} state do
      q⁡(v)←pLM​(v∣x)⋅p⁡(α∣x⋅v)q(v)\leftarrow p_{\text{LM}}(v\mid x)\cdot p(\alpha\mid x{\cdot}v) ; // constraint-conditioned posterior, Eq. 8
   end for
   xt←arg⁡maxv⁡q⁡(v)x_{t}\leftarrow\arg\max_{v}q(v); x←x⋅xtx\leftarrow x\cdot x_{t}; if xtx_{t} closes an action block then
      l←l+1l\leftarrow l+1
   end if
until xt=<EOS>x_{t}=\texttt{<EOS>};
return xx

Appendix C Per-Module DFA and Parser Construction

Each benchmark/module has its own syntax DFA, semantic DFA and incremental parser. Object IDs are canonicalized. Accepting state definitions explicitly refer to all non-dropped goals; the report does not fully enumerate every goal-dropping condition. A guarantee over this retained subset is narrower than the evaluator’s complete goal set.

C.1 VirtualHome Action Sequencing (VH-AS)

Output grammar ( 𝒟 γ ).

VirtualHome action output is a JSON object of action names to flat object-name/ID arrays. Scene-specific tries constrain actions, names and IDs; action blocks repeat until closure. The exact example is retained. Repeated action names require the stated sequential-plan parser to preserve order; generic JSON-map behavior should not be assumed.

Complete original PDF block source-vh-as-format
Exact VirtualHome action format
Original selectable source block

{ "WALK": ["soap", "1002"] ,
{"GRAB": ["soap", "1002"] ,
{"PUTON": ["clothes_jacket", "1003"] }

Action vocabulary ( 𝒲 ).

The action set has 36 types: one nullary STANDUP, three binary POUR/PUTBACK/PUTIN and 32 unary actions. Types without any compatible scene arguments are pruned. A count of action types is different from the number of fully grounded actions.

Semantic constraints ( 𝒟 β ).

State includes agent location, two occupied-hand slots, posture/facing, object open/switch/plug bits, relations and ordered goals. Static properties permit actions; dynamic tests enforce proximity, free hands, power/open-state compatibility and held-object placement. Acceptance checks retained node/edge/character/action goals.

Parser ( τ ).

Closing an argument array emits a meta-step. Trie leaves supply the action and raw IDs, then remap them to canonical IDs. Structural tokens do not independently change the semantic state.

C.2 VirtualHome Subgoal Decomposition (VH-SD)

Output grammar ( 𝒟 γ ).

VirtualHome subgoal output has one output list of ordered predicate strings, with dotted name.id arguments. Binary-predicate argument choices can depend on the first object. The complete three-step washing-machine example is preserved.

Complete original PDF block source-vh-sd-format
Exact VirtualHome subgoal format
Original selectable source block

{"output": ["PLUGGED_IN(washing_machine.1001)",
{"output": ["CLOSED(washing_machine.1001)",
{"output": ["ON(washing_machine.1001)"]}

Subgoal vocabulary ( 𝒲 ).

Thirty-six predicates comprise nine state predicates, six relations and 21 action-named intermediate subgoals. This module outputs desired predicates for an executor, rather than directly emitting the same actions as action sequencing.

Semantic constraints ( 𝒟 β ).

State bitmasks and goal progress enforce rules such as plugged-in and closed before ON, an open destination for placement and free hands for holding. Acceptance checks current state, so reopening can undo CLOSED; repeated placements use latest-wins semantics. The model approximates the benchmark executor’s behavior.

Parser ( τ ).

One completed predicate atom emits one meta-step after name/ID parsing. The exact list syntax remains separate from its state-transition interpretation.

C.3 BEHAVIOR Action Sequencing (BEH-AS)

Output grammar ( 𝒟 γ ).

BEHAVIOR action output is an array of one-action dictionaries with action and object fields. A scene-specific instance trie constrains names and a cyclic grammar permits repeated actions. The preserved example includes grasp, slice and placement.

Complete original PDF block source-beh-as-format
Exact BEHAVIOR action format
Original selectable source block

[{"action": "LEFT_GRASP", "object": "plywood_0"},
[{"action": "SLICE", "object": "strawberry_0"},
[{"action": "LEFT_PLACE_INSIDE", "object": "jar_0"}]

Action vocabulary ( 𝒲 ).

Thirty action types include hand-specific grasp/release/placement/content transfer plus open/close/toggle and cleaning/cooking/freezing attributes. Navigation is implicit, so the plan is not a geometric path or a full low-level execution program.

Semantic constraints ( 𝒟 β ).

The automaton tracks goals, hand occupancy, object locations and Boolean attributes. Grasp requires a free hand and accessible graspable target; closed containment blocks access. Placement, toggling, soaking and stained-object cleaning enforce selected preconditions. These encoded transitions are not the complete physical world model.

Parser ( τ ).

A closed action dictionary emits one grounded step, canonicalizing instance labels. Correct correspondence between these names and downstream perception/executor names is an additional pipeline requirement.

C.4 BEHAVIOR Subgoal Decomposition (BEH-SD)

Output grammar ( 𝒟 γ ).

BEHAVIOR subgoal output lists temporal steps, each containing predicates joined by and; not expresses negation. The complete source example and punctuation are retained. Atom-level parsing must preserve the distinction between within-step conjunction and between-step sequence.

Complete original PDF block source-beh-sd-format
Exact BEHAVIOR subgoal format
Original selectable source block

{"output": ["holds_rh(carton.66) and inside(hardback.64, carton.66)
{"output": [ and inside(hardback.65, carton.66)", …]}

Subgoal vocabulary ( 𝒲 ).

Sixteen positive predicates plus their negations give 32 atom types: eight state attributes and eight spatial/holding relations. Negative goals are represented explicitly.

Semantic constraints ( 𝒟 β ).

Goal bits are set on matching atoms, with container-access gating and place–toggle–soak prerequisites for relevant cleaning goals. Quantified goals are expanded and repositioned anchors must be established before dependent relations. This description is weaker than tracking every possible later undoing of every satisfied goal; it should not be generalized beyond the encoded rules.

Parser ( τ ).

Each completed atom emits a canonicalized meta-step. Structural list/conjunction delimiters do not themselves supply missing affordances or state knowledge.

C.5 WAH-NL via LoTa-Bench

WAH reuses VirtualHome machinery on a restricted action subset, excluding WAKEUP/SLEEP/PUTOBJBACK. It enforces free-hand/co-location grasping, held-object/open-container insertion, local open/close and plugged-in switching. The resulting symbolic executor test is distinct from earlier Unity-based or natural-language-only protocols.

C.6 Shared construction notes

Canonical tokenization.

The implementation description fixes one canonical tokenization per atom but also marginalizes structural tokenization equivalence classes with logsumexp. Exact factorization across multiple realizations needs a corresponding aggregated meta distribution; Equation 6’s unique-inverse notation applies literally only to the stated bijective representation.

EOS handling.

The source says EOS is allowed at a boundary when the semantic state is accept-reachable. Terminal goal satisfaction instead requires an accepting state, as in Appendix A; Appendix F also says premature closure gets zero mass. Reachability alone is insufficient at termination. These descriptions are left explicitly qualified without claiming the uninspected implementation uses the weaker rule.

Appendix D HMM Training

EAI shares a 128 hidden-state action HMM and a separate subgoal HMM across simulators; WAH uses another meta HMM. Token HMMs are shared across modules for each base LM. The 132/123 meta vocabularies count emission symbols, not fully grounded actions: an action name is followed by zero, one or two object-ID symbols.

For action sequencing,81 action names+49 IDs+UNK+EOS give 132 symbols. Token emissions dominate roughly 16.4 million parameters; meta emissions/transitions total about 33 thousand. Grounded-action semantics and symbol-level HMM steps must be distinguished when reading vocabulary and horizon claims.

Training data construction.

Training substitutes object/predicate arguments in benchmark-format prompts while excluding official evaluation instances. Up to one million generations per pair use temperature 0.1, top-p 0.95 and 2,048 token cap. Parse failures are dropped and simulator halves balanced 1:1. Both HMMs use retained text and its parsed projection; this is format-matched synthetic training, not absence of benchmark-domain supervision.

Corpus diversity.

Low-temperature data is highly repetitive:58 distinct BEHAVIOR action sequences from 95,665 samples over 70 prompts. Temperature 1.0 increases diversity, but rare mandatory actions may remain missing. More samples alone provide diminishing held-out likelihood gains.

Complete original PDF block A4.T7
Table 7: HMM parameter counts. Token-level HMM size is dominated by the emission matrix H×|V|H\times|V|; meta-token HMM is dominated by the H×HH\times H transition matrix at this scale. H=128H{=}128 throughout; token-level |V||V| shown for the Llama-3 tokenizer (gpt-oss uses |V|≈201|V|{\approx}201K ⇒\Rightarrow ∼25.7{\sim}25.7M).
Original selectable table
Vocab Token-level HMM Meta-token HMM
VH-AS 132 actions / 128128K tokens ∼16.4{\sim}16.4M ∼33{\sim}33K
VH-SD 123 predicates / 128128K ∼16.4{\sim}16.4M ∼32{\sim}32K
BEH-AS 132 actions / 128128K ∼16.4{\sim}16.4M ∼33{\sim}33K
BEH-SD 123 predicates / 128128K ∼16.4{\sim}16.4M ∼32{\sim}32K
WAH-NL 7 actions + 128 ids / 128128K ∼16.4{\sim}16.4M ∼34{\sim}34K

Trained vs. uniform meta-HMM.

Uniform and trained surrogates enforce the same encoded support but rank accepting futures differently. Uniform lookahead is not identical to a reachability mask: it accounts for accepting future mass/length. Trained guidance does not dominate task scores; temperature 1 training trades lower VirtualHome subgoal score for stronger BEHAVIOR action execution.

SLEEP and PLUGIN are reported absent from roughly10⁸sampled tokens, making faithful learned priors penalize steps that constraints demand. This explains a possible calibration tradeoff; it does not establish that the uniform distribution equals the LM conditional.

Complete original PDF block A4.T8
Table 8: Trained vs. uniform meta-token HMM on Llama 3 8B, task / execution SR (%). All trained rows use the single recipe of Appendix D: parse-filtered corpus, balanced 1:1 simulator mixture, App. D hyperparameters, no decode-time scaling. The last row retrains the same recipe on a temperature-1.01.0 corpus. VH-AS cells are means over six evaluator seeds (Appendix K).
Selectable table transcribed from the original PDF
Meta-HMMVH ASVH SDBEH ASBEH SD
Uniform90.7 / 97.788.4 / 93.236 / 8941 / 59
Trained (T=0.1 corpus)90.4 / 97.789.9 / 93.534 / 8936 / 66
Trained (T=1.0 corpus)90.1 / 98.087.8 / 92.633 / 9642 / 65
Meta-HMM VH AS VH SD BEH AS BEH SD
Uniform 90.7 / 97.7 88.4 / 93.2 36 / 89 41 / 59
Trained (T=0.1T{=}0.1 corpus) 90.4 / 97.7 89.9 / 93.5 34 / 89 36 / 66
Trained (T=1.0T{=}1.0 corpus) 90.1 / 98.0 87.8 / 92.6 33 / 96 42 / 65

Appendix E Decoding Details

Greedy decoding under the factored posterior.

Inference greedily maximizes reweighted scores, without temperature sampling, top-p or beam search. A greedy sequence is not a sample from the exact conditional distribution or necessarily its globally most probable complete plan.

Decoding overhead.

At batch 1, medians across twelve matched-length VirtualHome tasks are 20.6 ms/token constrained versus 20.9 ms raw. Grammar/semantic compilation adds 2.3/0.7 seconds per task; peak device memory is 32 GiB including the fp 16 model. The 107 TB-to 1.6 GB DP estimate excludes these full-system costs.

Horizon budget.

The semantic horizon is 40 actions; gold EAI plans have median 4 and 95 thpercentile 19, with one 54 step case exceeding it. A gold plan longer than 40 does not prove no shorter solution exists. The finite budget restricts the feasible continuation set and can trigger fallback.

Appendix F Qualitative Examples: DFA-only Degeneration vs. Meta-Ctrl

The retained light example contrasts a locally true NEXT_TO predicate with PLUGGED_IN followed by ON. Syntax-only generation can terminate with a valid but irrelevant plan, explaining high execution but low task success. This comparison changes semantic constraints as well as guidance; Table 5’s matched-semantic masking comparison isolates the latter more closely.

Complete original PDF block A6.T9
Table 9: Representative outputs from DFA-only (γ\gamma hard mask) and Meta-Ctrl (γ+β\gamma{+}\beta) on VirtualHome Subgoal Decomposition. DFA-only emits a single locally-true predicate and stops; Meta-Ctrl produces the predicates required to reach the goal.
Original selectable table
Task Prompt id DFA-only (γ\gamma mask) Meta-Ctrl (γ+β\gamma{+}\beta)
Turn on light scene_1_125_2 NEXT_TO(character.65, light.411) PLUGGED_IN(light.411), ON(light.411)

Appendix G BEHAVIOR Affordance Prior

BEHAVIOR lacks explicit per-object affordance tags in its prompts, so the Llama experiments add published BDDL taxonomy information plus rules. The gpt-oss rows use an earlier keyword-based prior, as Table 3’s footnote specifies; their information sources are not identical.

Affordances come from the BDDL taxonomy, not from an author list.

The 1,822 node taxonomy maps prompt-named WordNet synsets to abilities and ISA classes. It is reported semantically identical to the released BDDL resource. This is additional benchmark-domain knowledge beyond plain instruction text, although public rather than inferred from test gold plans.

Two additions on top of the taxonomy.

An admissibility rule limits open/close actions using initial/goal evidence or container roles. Five further rules treat open-top vessels as lidless, bathtubs as switched water sources, kettles/teapots/pools as passive water sources, goal-repositioned objects as graspable, and cleaning tools as soakable/dryable by default.

These are fixed rule forms whose application can depend on task goals; the later claim that none depends on a goal is too broad. Their physical interpretation can disagree with the benchmark, including gold plans that open baskets or buckets.

General precondition rules.

Shared checks prohibit grasping with an occupied hand, interacting through closed containment, self/cyclic containment and using an unsettled placement anchor. They are explicit symbolic invariants, not continuous collision, force or reachability certification.

Scope of the prior.

The paper contrasts its partial affordance/constraint model with the planner’s full PDDL domain. Its semantic automata nevertheless update object, hand and goal states, so “no effects” should not be read as absence of transition semantics. Different incomplete/mismatched models are not interchangeable merely because one has more schemas.

Seventeen gold plans violate the prior and the system reports syntax-only fallback on those cases. Rejecting a reference plan alone does not rule out another valid plan; fallback can also still happen to achieve a goal. Thus the stated 83% ceiling does not follow solely from the 17 rejections.

Appendix H Format and Hallucination Errors: Per-Module Breakdown

Table 10 preserves raw error rates, total constrained errors and the authors’ split into benchmark-side versus LM-fault. Zero in the final column is relative to the chosen grammar/normalization contract, while actual evaluator rejections remain nonzero. Summed category percentages need not equal a disjoint plan-failure probability.

Complete original PDF block A8.T10
Table 10: Format and hallucination error rates for raw Llama 3 8B and Meta-Ctrl. Parsing: output fails the evaluator’s parser. Hallucination: output references objects or predicates not present in the evaluator’s per-task vocabulary. Parameter: argument arity, type, or formatting mismatch. The Meta-Ctrl benchmark-side column counts plans rejected by the evaluator even though they comply with the prompt’s stated grammar and vocabulary (footnotes identify each case). The LM-fault column is the residual after removing benchmark-side rejections, and is zero across every module and error class.
Original selectable table
Module Error class Raw Llama 3 8B Meta-Ctrl
Total Benchmark-side LM-fault
BEH AS Parsing 0.0% 0.0% 0.0% 0.0%
Hallucination 15.0% 0.0% 0.0% 0.0%
Parameter 9.0% 0.0% 0.0% 0.0%
Sum format/hallucination 24.0% 0.0% 0.0% 0.0%
VH AS Parsing 14.4% 1.3% 1.3%a 0.0%
Hallucination 6.9% 1.3% 1.3%b 0.0%
Parameter 1.0% 0.0% 0.0% 0.0%
Sum format/hallucination 22.3% 2.7% 2.7% 0.0%
VH SD Parsing 0.6% 2.1% 2.1%c 0.0%
Hallucination 3.3% 0.0% 0.0% 0.0%
Parameter 0.6% 0.0% 0.0% 0.0%
Sum format/hallucination 4.5% 2.1% 2.1% 0.0%
BEH SD Parsing 0.0% 0.0% 0.0% 0.0%
Hallucination 21.0% 1.0% 1.0%d 0.0%
Parameter 0.0% 0.0% 0.0% 0.0%
Sum format/hallucination 21.0% 1.0% 1.0% 0.0%

Notes on benchmark-side residuals.

Residual cases include empty-goal outputs rejected as {}, prompt-listed PLUGIN rejected by the action dispatcher, POUR rejected by the subgoal dispatcher, and normalized shirt.1 mismatching a t-shirt.1 scene instance. These examples show interface disagreement; the naming case is still a system-pipeline failure even if the LM obeyed its encoded grammar.

Appendix I Why BEHAVIOR Gains Less from Constrained Decoding than VirtualHome

The appendix attributes weaker BEHAVIOR gains to missing prompt information. Its coverage audit compares each gold action’s declared preconditions against directly readable prompt predicates and bound objects. This describes that benchmark/schema audit, not a universal upper bound on inferable knowledge.

VirtualHome prompts expose object affordances; BEHAVIOR prompts do not.

VirtualHome supplies about eleven affordance tags per prompt; BEHAVIOR supplies none of that tag class. Both provide other scene/goal information, so zero affordance tags does not mean zero task state knowledge.

Most VirtualHome preconditions are derivable from the prompt; most BEHAVIOR preconditions are not.

The preserved table reports 82% of VirtualHome preconditions directly derivable from prompts versus 24% for BEHAVIOR. Remaining constraints may require domain rules or inference. The percentages do not directly measure the final augmented automata’s coverage.

Complete original PDF block source-precondition-coverage
Original prompt-precondition coverage table
Original selectable source block
Prompt-derivable preconditions Affordance tags per prompt
VirtualHome 82%82\% ∼11{\sim}11
BEHAVIOR 24%24\% 00

The remaining BEHAVIOR gap cannot be closed by enlarging the prior.

The prior assigns actionable affordance classes to 43 of 148 gold-plan object base names despite every instance resolving to a taxonomy synset. Missing abilities/state details motivate richer knowledge. This finite audit does not prove that no enlarged or better prior could improve task success.

The effect.

The qualitative comparison remains: BEHAVIOR action 10→34%, versus VirtualHome roughly 21→90%. This appendix’s 89% wording differs from the six-seed 90.4% main-table figure; the edition uses the explicit main-table value and retains all source tables.

Appendix J Real Robot Demonstrations

Five of the eight demonstrations provide full side-by-side plans below. Colored insertions/changes and every numbered step are retained. These examples illustrate symbolic repairs, while the three-task quantitative physical test has separate perception/control failures.

J.1 Clean dishes with detergent

Dish cleaning adds detergent application and soaking the rag before cleaning the plate/bowl. These prerequisites are encoded task semantics; the example alone does not validate cleaning effectiveness or every physical tool-use condition.

Complete original PDF block source-plan-1
Complete original baseline and Meta-Ctrl plan pair 1
Original selectable source block

Baseline (Llama 3 8B)

  1. 1.

    Grab(orange rag)

  2. 2.

    Clean(plate) ✗ rag not soaked

  3. 3.

    Clean(bowl) ✗ rag not soaked

  4. 4.

    PutOn(orange rag, stool)

Meta-Ctrl (ours)

  1. 1.

    Grab(spray bottle)

  2. 2.

    Spray(plate)

  3. 3.

    Spray(bowl)

  4. 4.

    PutOn(spray bottle, stool)

  5. 5.

    Grab(orange rag)

  6. 6.

    Soak(cup of water)

  7. 7.

    Clean(plate)

  8. 8.

    Clean(bowl)

  9. 9.

    PutOn(orange rag, stool)

J.2 Put dishes in dishwasher

Opening the dishwasher once makes subsequent blue-mug, white-cup and gray-pot insertions admissible under the container-access rule. Both complete plans are preserved, including baseline failure markers.

Complete original PDF block source-plan-2
Complete original baseline and Meta-Ctrl plan pair 2
Original selectable source block

Baseline (Llama 3 8B)

  1. 1.

    Grab(blue mug)

  2. 2.

    PutIn(blue mug, dishwasher) ✗ dishwasher closed

  3. 3.

    Grab(white cup)

  4. 4.

    PutIn(white cup, dishwasher) ✗

  5. 5.

    Grab(gray pot)

  6. 6.

    PutIn(gray pot, dishwasher) ✗

Meta-Ctrl (ours)

  1. 1.

    Open(dishwasher)

  2. 2.

    Grab(blue mug)

  3. 3.

    PutIn(blue mug, dishwasher)

  4. 4.

    Grab(white cup)

  5. 5.

    PutIn(white cup, dishwasher)

  6. 6.

    Grab(gray pot)

  7. 7.

    PutIn(gray pot, dishwasher)

J.3 Fetch egg from fridge

The section title says fetch egg, but its exact plans move orange and syrup from the fridge into a pot, then place it on a stove and switch it on. The note follows the printed actions without silently replacing ingredients. The added OPEN fixes the modeled access prerequisite.

Complete original PDF block source-plan-3
Complete original baseline and Meta-Ctrl plan pair 3
Original selectable source block

Baseline (Llama 3 8B)

  1. 1.

    Grab(orange) ✗ fridge closed

  2. 2.

    PutIn(orange, pot)

  3. 3.

    Grab(syrup) ✗ fridge closed

  4. 4.

    PutIn(syrup, pot)

  5. 5.

    Grab(pot)

  6. 6.

    PutOn(pot, stove)

  7. 7.

    ToggleOn(stove)

Meta-Ctrl (ours)

  1. 1.

    Open(fridge)

  2. 2.

    Grab(orange)

  3. 3.

    PutIn(orange, pot)

  4. 4.

    Grab(syrup)

  5. 5.

    PutIn(syrup, pot)

  6. 6.

    Grab(pot)

  7. 7.

    PutOn(pot, stove)

  8. 8.

    ToggleOn(stove)

J.4 Set up breakfast table

The repaired breakfast plan places the bread into the bowl before grasping the milk carton, then releases milk before grasping tuna. This enforces a single-gripper invariant. Putting bread in a bowl on the stool is a containment-based goal interpretation, not necessarily literal direct support on the stool.

Complete original PDF block source-plan-4
Complete original baseline and Meta-Ctrl plan pair 4
Original selectable source block

Baseline (Llama 3 8B)

  1. 1.

    Grab(bowl)

  2. 2.

    PutOn(bowl, stool)

  3. 3.

    Grab(bread)

  4. 4.

    Grab(oat-milk carton) ✗ holding bread

  5. 5.

    Grab(blue tuna can) ✗ holding oat-milk

  6. 6.

    PutOn(blue tuna can, stool) ✗ bread, milk lost

Meta-Ctrl (ours)

  1. 1.

    Grab(bowl)

  2. 2.

    PutOn(bowl, stool)

  3. 3.

    Grab(bread)

  4. 4.

    PutIn(bread, bowl)

  5. 5.

    Grab(oat-milk carton)

  6. 6.

    PutOn(oat-milk carton, stool)

  7. 7.

    Grab(blue tuna can)

  8. 8.

    PutOn(blue tuna can, stool)

J.5 Place boxed objects in drawer

The tools example selects pliers and screwdriver rather than their containing boxes, inserting them into respective shelves and closing the shelves. Complete ten-step plans remain. Correct symbolic selection still depends on physical identification and execution.

Complete original PDF block source-plan-5
Complete original baseline and Meta-Ctrl plan pair 5
Original selectable source block

Baseline (Llama 3 8B)

  1. 1.

    Open(box 1)

  2. 2.

    Open(shelf 1)

  3. 3.

    Grab(box 1) ✗ wrong object

  4. 4.

    PutIn(box 1, shelf 1) ✗ goal not met

  5. 5.

    Close(shelf 1)

  6. 6.

    Open(box 2)

  7. 7.

    Open(shelf 2)

  8. 8.

    Grab(box 2) ✗

  9. 9.

    PutIn(box 2, shelf 2) ✗

  10. 10.

    Close(shelf 2)

Meta-Ctrl (ours)

  1. 1.

    Open(box 1)

  2. 2.

    Open(shelf 1)

  3. 3.

    Grab(pliers)

  4. 4.

    PutIn(pliers, shelf 1)

  5. 5.

    Close(shelf 1)

  6. 6.

    Open(box 2)

  7. 7.

    Open(shelf 2)

  8. 8.

    Grab(screwdriver)

  9. 9.

    PutIn(screwdriver, shelf 2)

  10. 10.

    Close(shelf 2)

Summary.

The examples cover container access, detergent/soaking chains and holding/object-identity constraints. They are selected demonstrations of rules represented in the automaton, not evidence that every future failure type is captured.

Appendix K Extended Evaluation on Recent Models

The authors evaluate nine newer models under their EAI protocol and preserve all results in Table 11. This updates the earlier-leaderboard context: Meta-Ctrl leads selected execution/action columns, while newer frontier models lead several task/subgoal columns. Parameter-size comparisons to closed models are not established by disclosed sizes here.

Evaluation protocol.

All four task sets use official prompts and symbolic scoring; open models run on eight H200 GPUs and outputs use default sampling settings. Different model defaults and extra compiled constraint information mean this is a system-level comparison.

Evaluation protocol.

VirtualHome action-goal matching depends on hash order for thirteen tasks, changing scores on identical plans by up to±1.4 points. Reported VirtualHome scores average six fixed evaluator seeds. This addresses evaluator variation, not repeated independent LM training or complete statistical uncertainty.

Where Meta-Ctrl stands.

Llama Meta-Ctrl’s 90.4% VirtualHome action task score exceeds the listed 87.9% ClaudeOpus 4.8 result. On VirtualHome subgoals, frontier scores 90.8–92.9% exceed 89.9%; BEHAVIOR task success also favors frontier models. gpt-oss Meta-Ctrl leads BEHAVIOR action execution 90%. Table 11’s average uses only the four task-success columns, not all eight metrics.

Complete original PDF block A11.T11
Table 11: Extended Embodied Agent Interface results on recent models, evaluated by us under the official EAI protocol. Task SR (T) and Execution SR (E) on VirtualHome (VH; n=342n=342 AS, 338338 SD) and BEHAVIOR (BEH; n=100n=100 AS, 100100 SD). Meta-Ctrl rows from Table 3 included for reference. Bold: best per column; underline: second-best.
Original selectable table
Action Sequencing Subgoal Decomposition
VH BEH VH BEH
Model T E T E T E T E Avg
Recent frontier models
Claude Opus 4.8 87.9 86.2 74.0 81.0 90.8 93.8 61.0 65.0 78.4
Claude Opus 4.7 77.7 76.1 73.0 82.0 91.1 93.8 64.0 71.0 76.5
Claude Opus 4.6 74.4 74.8 78.0 87.0 92.9 95.6 78.0 82.0 80.8
Claude Sonnet 4.6 73.1 75.7 84.0 89.0 91.7 93.5 73.0 79.0 80.5
GPT-5.5 73.4 78.0 72.0 75.0 92.3 95.0 69.0 76.0 76.7
GPT-5.4 70.8 74.8 57.0 59.0 92.9 95.3 66.0 76.0 71.7
GPT-5.4 mini 71.5 76.7 70.0 79.0 91.1 94.7 63.0 76.0 73.9
Recent open-weight models
Gemma 4 31B 20.7 22.6 51.0 58.0 83.7 93.5 37.0 42.0 48.1
gpt-oss-20B 74.4 80.3 40.0 51.0 72.5 82.2 27.0 36.0 53.5
Meta-Ctrl
Llama 3 8B + Meta-Ctrl 90.4 97.7 34.0 89.0 89.9 93.5 36.0 66.0 62.6
gpt-oss-20B + Meta-Ctrl 86.6 94.1 40.0 90.0 82.3 86.4 41.0 66.0 62.5

T = Task SR, E = Execution SR. Avg = mean of the four Task SR columns.

Concept demonstration

Open image ↗