Literature
Display
← All papers

PopGen: Generating Semantically Diverse Populations for Zero-Shot Coordination

Paul Teiletche · Kanghyun Ryu · Kyungjoon Park · Dhruva Tirumala · Markus Wulfmeier · Negar Mehr

Preprint posted · arXiv (preprint) ↗

Robotics16 min text · CondensedOriginal paper ↗
Contents

Overview

Language-generated partner strategies, executable policy code and population-play training for zero-shot coordination. All original figures, tables, equations, algorithm, prompt boxes and code-diff examples are retained. Source code.

At a glance

PopGen first generates varied behavior descriptions, then writes and refines executable partner policies. PopPlay trains a cooperator against those frozen partners. Across four simulated environments it obtains the highest average reward with held-out scripted partners; gains over the strongest baseline have overlapping error bars in two environments. The experiments support strategy coverage and role adaptation, while human cooperation and judge-independent semantic validity remain untested.

1 Introduction

Different trajectories can implement the same strategy, so trajectory diversity need not cover incompatible coordination conventions. PopGen instead describes strategies in language, explores many candidates, selects plausible distinct descriptions, and converts them into code through rollout-guided refinement. The key hypothesis is that exposure to complementary roles and conventions helps a learned cooperator adapt to unseen partners. Semantic distance is itself an embedding-based proxy, and generated descriptions must actually be realized by the policies.

Figure 1: Overview of PopGen. Left (Stage 1): An LLM produces a diverse set of behaviors at the text level using oversampling. Right (Stage 2): Each behavior description enters an iterative refinement loop: a generator translates the behavior into an executable policy (Policy Code), which is evaluated with a VLM through environment rollouts. An advisor then analyzes rollout results and provides feedback to guide subsequent refinement attempts.
Figure 1: Overview of PopGen . Left (Stage 1): An LLM produces a diverse set of behaviors at the text level using oversampling. Right (Stage 2): Each behavior description enters an iterative refinement loop: a generator translates the behavior into an executable policy (Policy Code), which is evaluated with a VLM through environment rollouts. An advisor then analyzes rollout results and provides feedback to guide subsequent refinement attempts.

2 Related Work

The work combines population play with language-based behavior specification and executable policies. It evaluates both the diversity of descriptions and adherence of resulting agents, because either stage can fail.

Generating diverse populations for ZSC.

Prior populations pool self-play seeds/checkpoints, optimize trajectory/action diversity, or model human play. PopGen targets distinctions expressible in text rather than relying only on low-level trajectory distance. It complements, rather than proves invalid, earlier behavioral-diversity measures.

LLM-generated agents.

An LLM can choose actions online, generate a reward for an RL policy, or write explicit decision logic. Online calls are costly; a reward may be optimized without the intended behavior emerging. Code makes rules explicit and runs without per-step LLM calls. The controlled comparison with SemDiv-style reward generation uses identical descriptions to isolate this modeling choice.

3 Problem Formulation

The formal setting is cooperative sequential decision-making with shared reward, potentially partial observations and unknown partner conventions.

Multi-agent interaction.

A Dec-POMDP specifies agents, states, per-agent observations/actions, observation and transition models, shared reward and discount. Each policy acts from its observation history; coordination concerns joint behavior rather than independent task rewards.

Training against populations of partners.

Experiments use two exchangeable cooperative agents. First construct K partner policies, then optimize one cooperator’s expected shared return against sampled partners. PopGen associates each partner with a textual behavior. The displayed objective averages uniformly for exposition, while implemented PopPlay uses prioritized sampling of hard partners.

Equation 1
ϕ∗=arg⁡maxϕ​𝔼πk2∼Uniform​(Π)​[J⁡(πϕ1,πk2)]=arg⁡maxϕ​1K​∑k=1KJ⁡(πϕ1,πk2),\phi^{*}=\arg\max_{\phi}\;\mathbb{E}_{\pi^{2}_{k}\sim\text{Uniform}(\Pi)}\left[J(\pi^{1}_{\phi},\pi^{2}_{k})\right]=\arg\max_{\phi}\;\frac{1}{K}\sum_{k=1}^{K}J(\pi^{1}_{\phi},\pi^{2}_{k}), (1)

4 PopGen : Generating Diverse Agent Populations

Stage 1 produces descriptions that balance semantic variety with plausibility. Stage 2 turns each description into an executable policy and improves it using trajectory-based adherence feedback.

4.1 Stage 1: Generating Semantically Diverse Behaviors

Embed each behavior description as a unit vector and average pairwise distances √(2(1−cosine similarity)), the Euclidean distance on normalized embeddings. This measures linguistic/semantic spread under the selected embedding model, not directly the diversity of executed policies. Adherence evaluation supplies the separate grounding check.

Equation 2
D⁡(ℬ)=2K⁡(K−1)​∑k<ℓ2​(1−ek⊤​eℓ).D(\mathcal{B})=\frac{2}{K(K-1)}\sum_{k<\ell}\sqrt{2\big(1-e_{k}^{\top}e_{\ell}\big)}. (2)

The tension between quality and diversity of agent populations.

Quality sums three LLM-scored 0–5 rubrics: environment relevance, realism and specificity. PopGen oversamples descriptions with Verbalized Sampling, asking for candidate probabilities, then merges/deduplicates and selects a smaller high-quality set. Separating exploration from selection aims to prevent novelty through nonsense. The scores and verbalized probabilities are model judgments, not calibrated human ratings or behavior probabilities.

4.2 Stage 2: Instantiating Behaviors as Agents

The generator writes a Python policy that maps observations to actions. This avoids training a separate neural partner for every description and makes population rollouts inexpensive after generation. The neural cooperator is still trained with RL.

Measuring adherence with an evaluator.

A VLM receives sampled rollout states/images and returns a binary adherence verdict with rationale. Average accepted rollouts across multiple initial states to estimate adherence. It judges conformity to the target behavior, not necessarily high task reward, and its prompt allows justified good-faith attempts when the environment or teammate prevents full demonstration.

Equation 3
Adh⁡(π,b)=1M​∑m=1MVLM​(τm,b)∈[0,1].\mathrm{Adh}(\pi,b)=\frac{1}{M}\sum_{m=1}^{M}\text{VLM}(\tau_{m},b)\in[0,1]. (3)

Iterative refinement loop.

Generate a candidate, run it with the designated scripted partner, score adherence and ask a separate advisor to diagnose failures. Regenerate with the feedback until a threshold is met or the attempt budget ends; otherwise return the highest-scoring attempt. Best-of-five results combine iterative repair and candidate selection, so their gain is not solely an isolated effect of the advisor.

4.3 Population Play with PopGen Partners

PopPlay freezes the generated partner code and trains only the recurrent PPO cooperator. It adopts MEP’s rank-based prioritization: partners yielding lower recent team return are sampled more often. This avoids spending all training on easy partners, though the training distribution then differs from the uniform objective stated earlier.

5 Experiments

Experiments test held-out coordination, growth of description diversity, quality-versus-diversity trade-offs, and the effect of code versus reward-based partner implementation.

Original figure
Figure 2: Zero-shot coordination with held-out partners. Each bar reports the average team reward (higher is better) when each cooperator is paired with the held-out partners, which do not appear in any training population. Error bars give the standard deviation across five runs, and dots mark individual runs. PopPlay attains the highest reward in all four environments. Table 5 reports the reward against each individual partner.
Figure 3: Cooperator role adaptation in overcooked and robustness to held-out partners. (a) Rollouts show PopPlay taking semantically diverse behaviors interpretable to humans. (b) PopPlay spreads its actions across subtasks while the baselines tend to focus on a subset of actions. Per-partner counts are in Table 8. (c) PopPlay achieves the highest worst-case reward in all environments.
(a) Rendered rollouts of PopPlay cooperator (red box) paired with a partner. PopPlay switches roles based on the partner’s behavior. Method Onion → \rightarrow pot Start pot Scoop soup Serve soup PopPlay 26 % 26\% ( 2.7 ) (2.7) 26 % 26\% ( 2.7 ) (2.7) 27 % 27\% ( 2.8 ) (2.8) 21 % 21\% ( 2.2 ) (2.2) Self-Play 0 % 0\% ( 0.0 ) (0.0) 0 % 0\% ( 0.0 ) (0.0) 56 % 56\% ( 1.8 ) (1.8) 44 % 44\% ( 1.4 ) (1.4) FCP 0 % 0\% ( 0.0 ) (0.0) 0 % 0\% ( 0.0 ) (0.0) 62 % 62\% ( 1.2 ) (1.2) 38 % 38\% ( 0.7 ) (0.7) MEP 9 % 9\% ( 0.2 ) (0.2) 13 % 13\% ( 0.3 ) (0.3) 52 % 52\% ( 1.1 ) (1.1) 26 % 26\% ( 0.5 ) (0.5) CoMeDi 68 % 68\% ( 6.1 ) (6.1) 21 % 21\% ( 1.9 ) (1.9) 6 % 6\% ( 0.5 ) (0.5) 5 % 5\% ( 0.5 ) (0.5) (b) Share of each subtask completed by the cooperator, averaged over the four
held-out partners, with the subtask count per episode in parentheses. Method PMR LBF PP Overcooked PopPlay (ours) 1.41 ± 0.89 \mathbf{1.41\pm 0.89} 0.76 ± 0.16 \mathbf{0.76\pm 0.16} 1.33 ± 0.13 \mathbf{1.33\pm 0.13} 31.2 ± 44.6 \mathbf{31.2\pm 44.6} Self-Play (SP) 0.00 ± 0.00 0.00\pm 0.00 0.71 ± 0.11 0.71\pm 0.11 1.04 ± 0.14 1.04\pm 0.14 0.0 ± 0.0 0.0\pm 0.0 FCP 0.00 ± 0.00 0.00\pm 0.00 0.73 ± 0.08 0.73\pm 0.08 1.07 ± 0.14 1.07\pm 0.14 0.0 ± 0.0 0.0\pm 0.0 MEP 0.62 ± 0.57 0.62\pm 0.57 0.68 ± 0.14 0.68\pm 0.14 0.98 ± 0.09 0.98\pm 0.09 2.3 ± 3.7 2.3\pm 3.7 CoMeDi 1.40 ± 0.52 1.40\pm 0.52 0.49 ± 0.08 0.49\pm 0.08 0.64 ± 0.21 0.64\pm 0.21 11.2 ± 0.5 11.2\pm 0.5 (c) Worst-case team reward against held-out partners. For each run, we report the reward with the held-out partner on which the cooperator scores lowest (mean ± \pm std over five runs).

5.1 Experimental Setup

Four environments cover discrete and continuous control, different observation scopes and complementary coordination conventions.

Environments.

Point-Mass Rendezvous requires agreement on one of four landmarks; Level-Based Foraging requires joint collection; Predator-Prey trades solo rabbit captures against coordinated stag captures; Overcooked requires a division of sequential cooking subtasks. All are simulated cooperative-agent tasks, with no physical-robot or human-partner evaluation.

LLM models.

The experiments use Gemini-3-Flash for generation and adherence classification, Gemini-3-Pro for feedback and quality judging, and Gemini-Embedding-001 for description embeddings. Results depend on these models and prompts; using related model families for generation and judgment can introduce shared biases.

Cooperator training.

Self-Play uses independently initialized MAPPO partners; FCP adds early/middle/final checkpoints; MEP rewards population entropy; CoMeDi seeks competent but incompatible conventions and uses an extra behavior-cloning objective. Most cooperators share PPO architecture and training-frame budget, while CoMeDi’s objective differs. FCP also has 12 partners versus four for the other main populations, so population size is not identical across all baselines.

ZSC evaluation.

Each environment has four fixed held-out scripted partner behaviors, designed in advance without LLMs and withheld from generation prompts. This separates explicit train/test behaviors but covers only a finite chosen synthetic test set; worst-case results below mean the worst of these four partners.

5.2 Training Cooperators with PopGen Populations

Evaluate team reward against unseen partners, then inspect Overcooked subtask choices to test whether the same cooperator changes roles with the partner.

Generalization to novel partners.

Across five runs, PopPlay’s held-out mean rewards are 1.70±0.56 in PMR, 0.87±0.07 in LBF, 1.59±0.08 in PP and 120.88±59.25 in Overcooked. It has the highest mean in all four; PMR and Overcooked overlap CoMeDi’s uncertainty. Its mean per-run worst-partner rewards are 1.41, 0.76, 1.33 and 31.2, with substantial variation, especially in PMR and Overcooked. This is empirical robustness on the designated test partners, not a bound over arbitrary teammates.

PopPlay adapts its role to the partner.

The same PopPlay cooperator plates/serves when the partner fills pots, and fills/cooks when the partner serves. Its averaged subtask shares span all displayed roles, unlike onion-specialized self-play populations whose cooperators fail to fetch onions when needed. CoMeDi is stronger with two serving-focused Overcooked partners, so PopPlay’s role coverage does not mean per-partner dominance. Figure 3 preserves both rollout panels and all task-share/worst-case tables.

Original figure
(a) Diversity of the training pool and worst-case team reward as a function of population size.
Original figure
(b) Team reward with held-out partners as a function of training pool diversity.
Figure 4: Scaling the PopGen population in overcooked. PopPlay cooperators trained against pools of nn distinct PopGen agents at a fixed budget. (a) Larger populations are more diverse and raise the worst-case team reward with held-out partners. (b) Higher diversity also raises the average team reward, with gains plateauing beyond n=4n=4. Error bars show the standard error over runs.

5.3 Generating Diverse Populations for Cooperator Training

Population-size experiments link semantic spread to downstream reward; separate prompting comparisons assess whether that spread can be increased without sacrificing plausible descriptions.

Larger populations for more diversity, and more robustness.

Overcooked uses n=1,2,3,4,6,8 distinct behaviors while keeping eight training slots through replication and a fixed frame budget. Five independently generated populations/cooperators are run per size. Diversity rises from zero toward about 0.51 at n=4 and then levels off. One or two distinct partners leave worst-partner reward near zero; three or more improve it, but worst-case mean is not monotonic beyond four. The roughly 34% average-return increase at the fourth behavior and later plateau support coverage in this setting, without proving a universal four-partner optimum.

Original figure
Figure 5: Diversity and quality of PopGen population agents. Quality and diversity of a generated set of 30 behaviors, averaged across the environments. PopGen matches the diversity of VS while improving quality, and is substantially more diverse than DIR and SemDiv.

Preserving training agents’ quality while increasing diversity.

For sets of 30 descriptions, PopGen reports diversity 0.728 and quality 12.74/15, versus Verbalized Sampling at 0.726 and 12.03. Direct independent prompting scores higher quality (14.45) but lower diversity (0.446); SemDiv trades some diversity for quality. These figures describe the main reported comparison, while the six-environment oversampling ablation reports different averages. The claimed trade-off depends on embedding distances and the model judge’s rubric.

5.4 Modeling Agent Behaviors as Policy Code for Population Play

Keeping descriptions fixed, code-based partners yield higher downstream cooperator mean reward than reward-generated neural partners in all four environments. Table 12 shows best-attempt test adherence 66.3±7.2% for Policy Code versus 43.4±6.6% for Reward Function. The gap is largest in LBF (61.0% versus 12.5%) and Overcooked (63.3% versus 40.7%). This isolates implementation more directly than the full pipeline comparison, but does not establish that all reward-learning methods cannot realize these behaviors.

Original figure
Figure 6: Population play with policy code and reward-function partners. Mean team reward against held-out partners of cooperators whose training partners are modeled as executable code (like PopGen) or as neural network policies trained with RL on LLM-generated reward functions (like SemDiv). The set of behaviors is kept identical, so that only agent modeling method changes. Error bars give the standard deviation across five runs, and dots mark individual runs.

6 Conclusion

The results favor behavior coverage that is meaningful at the strategy level and faithfully executable. Oversampling/selection broadens descriptions, refinement improves code adherence, and population play exposes the cooperator to more complementary conventions.

Limitations.

All evaluation teammates are synthetic; human–AI coordination requires a human study. LLM population generation varies across runs, and the judge/embedding metrics are imperfect proxies. High mean reward with broad uncertainty and a finite worst-partner set should not be read as a universal cooperation guarantee.

Future work.

Future directions include generating new partners online to address the cooperator’s weaknesses, and improving the advisor’s context and code-generation reliability. The reported partners are generated before training and then frozen; online adaptation is not evaluated.

Acknowledgments

The authors acknowledge Antoine Bosselut, Seoyeon Choi and Google’s support.

AI Use Statement

The authors disclose generative-AI assistance with hypotheses, methodology, implementation, analysis, figures and writing, and state that they reviewed the resulting work. This disclosure is separate from the LLMs that are components of PopGen itself.

Reproducibility Statement

The paper supplies environment descriptions, held-out behaviors, baseline settings, detailed result tables and advisor/evaluator/quality-judge prompts. The code link and supplementary implementation support reproduction; the complete original prompt boxes are retained below.

Appendix A PopGen Details

Appendix A specifies environment mechanics and the adherence-driven refinement algorithm.

A.1 Environments

Two cooperative agents are trained in every core environment. LBF and Overcooked have six discrete actions; PMR and PP use two-dimensional continuous actions. LBF/PP are partially observable, while Overcooked/PMR use global observations. PP additionally contains five fixed-policy prey agents.

Figure 7: Environments used for cooperator training. From left to right, PMR requires agents (red and purple) to navigate toward and converge on the same landmark (green). PP requires agents (red) to coordinate to capture prey, with different rewards between individually capturing a rabbit (green) and jointly capturing a stag (yellow). LBF requires agents to combine their carrying capacities to collect food items whose levels may exceed the level of a single agent. Finally, overcooked requires two chefs to coordinate the sequential subtasks of preparing, cooking, and delivering soups.
Figure 7: Environments used for cooperator training. From left to right, PMR requires agents (red and purple) to navigate toward and converge on the same landmark (green). PP requires agents (red) to coordinate to capture prey, with different rewards between individually capturing a rabbit (green) and jointly capturing a stag (yellow). LBF requires agents to combine their carrying capacities to collect food items whose levels may exceed the level of a single agent. Finally, overcooked requires two chefs to coordinate the sequential subtasks of preparing, cooking, and delivering soups.
Table 1: Multi-agent environments in PopGen cover cooperative settings across discrete and continuous action spaces. We note that adversarial agents (prey) in PP follow fixed policies defined by the environment, so only the ego team is generated and trained.
Env Num. Agents Action Space Observation Objective
LBF 2 (ego: 2) Discrete (6) Partial Collect all food items
Overcooked 2 (ego: 2) Discrete (6) Global Serve as many soups as possible
PMR 2 (ego: 2) Continuous (2D) Global Meet at the same landmark
PP 7 (ego: 2, adv.: 5) Continuous (2D) Partial Capture prey

Level-Based Foraging (LBF) ( Albrecht & Ramamoorthy, 2015 ) .

LBF rewards collected food level divided by the total food level in the field. Food can require combined agent capacity, so navigation and synchronized loading matter.

Overcooked ( Carroll et al., 2019 ) .

The large_room Overcooked layout rewards intermediate subtasks as well as delivery. Any chef can fetch onions, fill/start pots, plate and serve. Team return therefore includes shaping progress and is not simply a count of soups delivered.

Point-Mass Rendezvous (PMR) ( Lowe et al., 2017 ) .

PMR places four equally valid landmarks on a circle. The team receives reward 2 only when both agents converge to the same landmark, making incompatible conventions easy to diagnose.

Predator-Prey (PP) ( Lowe et al., 2017 ) .

PP uses two trained predators, three rabbits and two stags. A solo rabbit capture gives 0.5 reward; stag capture requires coordinated predators and gives 2. Prey policies are fixed by the environment.

A.2 Iterative Refinement for Adherence

The original pseudocode retains early acceptance at the adherence threshold, advisor feedback after failed attempts, and best-candidate fallback after the iteration budget.

Evaluating adherence to behavior.

Adherence evaluation uses a fixed partner type per environment: nearest-food loading in LBF, a coordinating scripted predator in PP, the Overcooked oracle planner, and a randomly selected landmark in PMR. Multiple initializations test robustness, but adherence with these partners does not guarantee the generated behavior under every teammate.

Feedback loop.

At each attempt, the generator receives the target behavior, environment and prior feedback. The VLM labels M rollouts; the advisor converts their rationales plus the current code into revision guidance. Accept a candidate once its adherence reaches α; otherwise retain the best attempt. The reported refinement ablation uses up to five attempts.

Algorithm 1 Iterative Behavior-to-Agent Refinement
1: target behavior b b , environment description s s , adherence threshold α \alpha , max iteration I m ​ a ​ x I_{max} , rollouts per attempt M M 2: best candidate policy π ⋆ \pi^{\star} for behavior b b 3: f 1 ← ∅ f_{1}\leftarrow\emptyset 4: for i = 1 , … , I m ​ a ​ x i=1,\dots,I_{max} do 5: π i ← Generator ​ ( b , s , f i ) \pi_{i}\leftarrow\textsc{Generator}(b,s,f_{i}) ⊳ \triangleright regenerate given prior feedback 6: { τ m } m = 1 M ← Rollout ​ ( π i , M ) \{\tau_{m}\}_{m=1}^{M}\leftarrow\textsc{Rollout}(\pi_{i},M) 7: Adh i , { rationale m } m = 1 M ← VLM ​ ( b , { τ m } m = 1 M ) \mathrm{Adh}_{i},\{\mathrm{rationale}_{m}\}_{m=1}^{M}\leftarrow\textsc{VLM}(b,\{\tau_{m}\}_{m=1}^{M}) ⊳ \triangleright fraction of rollouts like b b 8: if Adh i ≥ α \mathrm{Adh}_{i}\geq\alpha then 9: return π i \pi_{i} 10: end if 11: f i + 1 ← Advisor ​ ( b , s , π i , { rationale m } m = 1 M ) f_{i+1}\leftarrow\textsc{Advisor}(b,s,\pi_{i},\{\mathrm{rationale}_{m}\}_{m=1}^{M}) ⊳ \triangleright feedback for next attempt 12: end for 13: return π i ⋆ \pi_{i^{\star}} where i ⋆ = arg ⁡ max i ∈ { 1 , … , I m ​ a ​ x } ⁡ Adh i i^{\star}=\arg\max_{i\in\{1,\dots,I_{max}\}}\mathrm{Adh}_{i} ⊳ \triangleright return best attempt

Advisor.

The advisor diagnoses recurring coordination failures and recommends actionable code edits. Its prompt and the contrasting code diffs are retained, allowing inspection of what the feedback actually asks the generator to change.

Appendix B Cooperator Training

All baselines construct partners first and train a separate cooperator afterward. The appendix distinguishes population-generation objectives from cooperator-training objectives.

B.1 Baselines

The shared population-play recipe does not imply identical populations or training losses. Baselines differ in partner construction, checkpoint count, prioritization and, for CoMeDi, the cooperator’s auxiliary objective.

Self-Play (SP).

Self-Play trains K independent MAPPO policies against copies of themselves and keeps final checkpoints. Diversity comes only from initialization and rollout noise, so multiple seeds can settle on the same convention.

Equation 4
ΠSP={π1,…,πK},\Pi_{\text{SP}}=\{\pi_{1},\dots,\pi_{K}\}, (4)

Fictitious Co-Play (FCP).

FCP retains three evenly spaced early/middle/final checkpoints from each of four self-play runs, giving 12 partners. This increases skill variation but need not add new semantic roles when the original runs share conventions.

Equation 5
ΠFCP={πk,tc}k=1,…,Kc=1,…,C,|ΠFCP|=K​C.\Pi_{\text{FCP}}=\big\{\pi_{k,t_{c}}\big\}_{k=1,\dots,K}^{c=1,\dots,C},\qquad|\Pi_{\text{FCP}}|=KC. (5)

Maximum Entropy Population-based Training (MEP).

MEP augments team return with entropy of the population’s mean action policy, combining individual stochasticity and inter-policy disagreement. Its entropy coefficient is 0.001 for LBF/PMR and 0.005 for PP/Overcooked. Cooperator training ranks partners by recent episodic return and samples low-return partners more often; PopPlay uses the same prioritization with β=1.

Equations 6–8
maxπ1,…,πK∑k=1K𝔼τ∼(πk,πk)[∑tγt(rt+ηℋ(π¯(⋅∣ot)))],\max_{\pi_{1},\dots,\pi_{K}}\;\sum_{k=1}^{K}\mathbb{E}_{\tau\sim(\pi_{k},\pi_{k})}\left[\sum_{t}\gamma^{t}\big(r_{t}+\eta\,\mathcal{H}\big(\bar{\pi}(\cdot\mid o_{t})\big)\big)\right], (6)
ℋ⁡(π¯)=1K​∑k=1Kℋ⁡(πk)⏟stochasticity of each partner+JSD⁡(π1,…,πK)⏟disagreement between partners,\mathcal{H}(\bar{\pi})=\underbrace{\frac{1}{K}\sum_{k=1}^{K}\mathcal{H}(\pi_{k})}_{\text{stochasticity of each partner}}\;+\;\underbrace{\mathrm{JSD}(\pi_{1},\dots,\pi_{K})}_{\text{disagreement between partners}}, (7)
p⁡(πk)=rank⁡(1/J⁡(πϕ,πk))β∑j=1Krank⁡(1/J⁡(πϕ,πj))β,p(\pi_{k})=\frac{\operatorname{rank}\!\left(1/J(\pi_{\phi},\pi_{k})\right)^{\beta}}{\sum_{j=1}^{K}\operatorname{rank}\!\left(1/J(\pi_{\phi},\pi_{j})\right)^{\beta}}, (8)

CoMeDi.

CoMeDi adds partners sequentially, rewarding self-play competence while penalizing cross-play return against the closest existing convention. Mixed-play switches partway through a rollout from cross-play to self-play and scores the later return, discouraging deliberate incompetence as a shortcut to low cross-play. Coefficients are 0.25 for cross-play and 1.0 for mixed-play.

Equations 9–10
j⋆​(π)=arg⁡maxj<k⁡J⁡(π,πj),j^{\star}(\pi)=\arg\max_{j<k}\;J(\pi,\pi_{j}), (9)
πk=arg⁡maxπ⁡J⁡(π,π)−λXP​J​(π,πj⋆​(π))+λMP​JMP​(π,πj⋆​(π)).\pi_{k}=\arg\max_{\pi}\;J(\pi,\pi)\;-\;\lambda_{\text{XP}}\,J\big(\pi,\pi_{j^{\star}(\pi)}\big)\;+\;\lambda_{\text{MP}}\,J_{\text{MP}}\big(\pi,\pi_{j^{\star}(\pi)}\big). (10)

The CoMeDi cooperator.

CoMeDi’s cooperator adds behavior cloning from partner self-play demonstrations to its objective. This differs from the other PPO cooperators and inherits the convention coverage of its fixed population.

Equation 11
ℒ⁡(π^,Π)=−J⁡(π^,π^)−λ|Π|​∑π∈Π𝔼(o,a)∼π​[log⁡π^​(a∣o)],\mathcal{L}(\hat{\pi},\Pi)=-J(\hat{\pi},\hat{\pi})\;-\;\frac{\lambda}{|\Pi|}\sum_{\pi\in\Pi}\;\mathbb{E}_{(o,a)\sim\pi}\big[\log\hat{\pi}(a\mid o)\big], (11)

B.2 Held-out Evaluation Partners

Each environment has four held-out scripts: different food targets/loading timing in LBF; nearest, teammate-biased, rabbit-only or stag-only pursuit in PP; filling versus serving roles in Overcooked; and nearest, farthest, teammate-biased or regroup-then-diverge landmark behavior in PMR. The complete descriptions remain in Table 2.

Env. Name Description
LBF Closest-Food Greedy Moves to the nearest food and loads it when possible.
Team-Center Food Targets food near the center of the two agents.
Teammate-Biased Food Targets the food cheapest for the teammate to reach.
Synchronized Loading Waits at its food until the teammate is adjacent.
PP Closest-Prey Greedy Chases the nearest prey, whether it is a rabbit or a stag.
Teammate-Biased Prey Targets the prey the teammate is also pressuring.
Rabbit Only Always pursues the nearest rabbit, never a stag.
Stag Only Pursues stags only, so it scores only if the partner joins.
Overcooked Fill only Only fetches onions and loads the pot.
Fetch & Serve Only fetches a dish and delivers soup.
Fill & Start Loads onions and starts the pot, but never serves.
Wait & Serve Waits at the pot until the soup is cooked, then serves.
PMR Closest-Landmark Greedy Moves toward the currently nearest landmark.
Farthest-Landmark Commitment Always moves toward the currently farthest landmark.
Teammate-Biased Landmark Selects the landmark closest to the teammate.
Regroup then Diverge Approaches the teammate, then heads to a far landmark.
Table 2: Held-out evaluation partners for each environment: Level-Based Foraging (LBF), Predator-Prey (PP), Overcooked, and Point-Mass Rendezvous (PMR).

B.3 Hyperparameters

Cooperators use recurrent GRU policies, discount 0.99, learning rate 10⁻⁴, PPO clipping 0.2, GAE λ=0.9 and prioritization β=1. Frame budgets are 4 million for LBF and Overcooked, 30 million for PMR and 80 million for PP. Partners remain frozen; baseline population generation uses MAPPO. Most main populations have four partners, versus FCP’s twelve.

Table 3: Cooperator training hyperparameters. The upper block lists the settings shared by every environment and every method. The lower block lists the per-environment settings.
Shared across all environments
Discount factor γ\gamma 0.99
Learning rate 10−410^{-4}
PPO clip ϵ\epsilon 0.2
GAE λ\lambda 0.9
Policy network GRU
Prioritized sampling β\beta 1.0
Parameter LBF Overcooked PMR PP
Environment frames 4M 4M 30M 80M

B.4 Results Table

The full train/evaluation table shows PopPlay’s highest held-out mean in each environment, with a large Overcooked train–test drop from 211.66 to 120.88. Some methods score higher on evaluation than on training partners, so this difference is not always an OOD penalty. PopPlay has the top or tied top displayed mean for 14 of 16 partners; CoMeDi wins the two serving-focused Overcooked cases. The paper’s winner count assigns ties to PopPlay, while bold also marks overlapping error bars. Table 4 uses standard deviation; Table 5 uses standard error.

Table 4: Average team reward of each cooperator against evaluation and training partners. We report reward against held-out eval behaviors and against the train partners the cooperator saw during training. We report the reward averaged across partners (mean ±\pm standard deviation over five runs). Best value per environment is in bold.
Method Partners PMR LBF PP Overcooked
Self-Play (SP) Eval 1.01±0.251.01\pm 0.25 0.81±0.040.81\pm 0.04 1.21±0.121.21\pm 0.12 63.38±65.3763.38\pm 65.37
Train 1.60±0.221.60\pm 0.22 0.78±0.020.78\pm 0.02 1.36±0.141.36\pm 0.14 133.79±151.41133.79\pm 151.41
FCP Eval 0.93±0.310.93\pm 0.31 0.80±0.030.80\pm 0.03 1.27±0.091.27\pm 0.09 40.53±52.2340.53\pm 52.23
Train 1.68±0.30\mathbf{1.68\pm 0.30} 0.80±0.020.80\pm 0.02 1.31±0.191.31\pm 0.19 157.29±134.16157.29\pm 134.16
MEP Eval 1.45±0.461.45\pm 0.46 0.72±0.140.72\pm 0.14 1.30±0.171.30\pm 0.17 33.79±37.7333.79\pm 37.73
Train 1.32±0.171.32\pm 0.17 0.65±0.300.65\pm 0.30 1.31±0.321.31\pm 0.32 126.05±104.18126.05\pm 104.18
CoMeDi Eval 1.56±0.391.56\pm 0.39 0.59±0.090.59\pm 0.09 1.02±0.131.02\pm 0.13 86.33±31.4886.33\pm 31.48
Train 1.51±0.151.51\pm 0.15 0.48±0.140.48\pm 0.14 0.88±0.130.88\pm 0.13 312.96±75.00\mathbf{312.96\pm 75.00}
PopPlay (ours) Eval 1.70±0.56\mathbf{1.70\pm 0.56} 0.87±0.07\mathbf{0.87\pm 0.07} 1.59±0.08\mathbf{1.59\pm 0.08} 120.88±59.25\mathbf{120.88\pm 59.25}
Train 1.64±0.451.64\pm 0.45 0.94±0.03\mathbf{0.94\pm 0.03} 1.71±0.29\mathbf{1.71\pm 0.29} 211.66±122.13211.66\pm 122.13
Table 5: Team reward against each held-out partner. We report mean ±\pm standard error of the team reward across 5 runs. For every partner, bold marks the best cooperator and every method whose error bar overlaps it. The final row counts the partners for which a method attains the highest average.
Held-out partner PopPlay (ours) SP FCP MEP CoMeDi
Point-Mass Rendezvous (PMR)
Closest-Landmark Greedy 1.58±0.26\mathbf{1.58}_{\pm 0.26} 0.82±0.080.82_{\pm 0.08} 0.92±0.160.92_{\pm 0.16} 1.01±0.021.01_{\pm 0.02} 1.46±0.21\mathbf{1.46}_{\pm 0.21}
Farthest-Landmark Commitment 1.60±0.40\mathbf{1.60}_{\pm 0.40} 0.80±0.49\mathbf{0.80}_{\pm 0.49} 0.40±0.400.40_{\pm 0.40} 1.20±0.49\mathbf{1.20}_{\pm 0.49} 1.45±0.22\mathbf{1.45}_{\pm 0.22}
Teammate-Biased Landmark 2.00±0.00\mathbf{2.00}_{\pm 0.00} 2.00±0.00\mathbf{2.00}_{\pm 0.00} 2.00±0.00\mathbf{2.00}_{\pm 0.00} 1.99±0.01\mathbf{1.99}_{\pm 0.01} 1.93±0.041.93_{\pm 0.04}
Regroup then Diverge 1.60±0.40\mathbf{1.60}_{\pm 0.40} 0.40±0.400.40_{\pm 0.40} 0.40±0.400.40_{\pm 0.40} 1.60±0.40\mathbf{1.60}_{\pm 0.40} 1.42±0.23\mathbf{1.42}_{\pm 0.23}
Level-Based Foraging (LBF)
Closest-Food Greedy 0.76±0.07\mathbf{0.76}_{\pm 0.07} 0.72±0.05\mathbf{0.72}_{\pm 0.05} 0.73±0.04\mathbf{0.73}_{\pm 0.04} 0.71±0.07\mathbf{0.71}_{\pm 0.07} 0.49±0.030.49_{\pm 0.03}
Team-Center Food 0.90±0.02\mathbf{0.90}_{\pm 0.02} 0.82±0.020.82_{\pm 0.02} 0.81±0.010.81_{\pm 0.01} 0.71±0.060.71_{\pm 0.06} 0.62±0.040.62_{\pm 0.04}
Teammate-Biased Food 0.92±0.01\mathbf{0.92}_{\pm 0.01} 0.87±0.010.87_{\pm 0.01} 0.86±0.020.86_{\pm 0.02} 0.77±0.050.77_{\pm 0.05} 0.66±0.040.66_{\pm 0.04}
Synchronized Loading 0.90±0.02\mathbf{0.90}_{\pm 0.02} 0.84±0.020.84_{\pm 0.02} 0.82±0.010.82_{\pm 0.01} 0.70±0.070.70_{\pm 0.07} 0.60±0.040.60_{\pm 0.04}
Predator-Prey (PP)
Closest-Prey Greedy 1.35±0.06\mathbf{1.35}_{\pm 0.06} 1.06±0.061.06_{\pm 0.06} 1.28±0.05\mathbf{1.28}_{\pm 0.05} 1.16±0.101.16_{\pm 0.10} 1.01±0.031.01_{\pm 0.03}
Teammate-Biased Prey 1.81±0.12\mathbf{1.81}_{\pm 0.12} 1.43±0.071.43_{\pm 0.07} 1.39±0.051.39_{\pm 0.05} 1.48±0.161.48_{\pm 0.16} 1.20±0.111.20_{\pm 0.11}
Rabbit Only 1.38±0.06\mathbf{1.38}_{\pm 0.06} 1.21±0.041.21_{\pm 0.04} 1.21±0.021.21_{\pm 0.02} 1.05±0.021.05_{\pm 0.02} 1.24±0.031.24_{\pm 0.03}
Stag Only 1.81±0.11\mathbf{1.81}_{\pm 0.11} 1.15±0.081.15_{\pm 0.08} 1.22±0.131.22_{\pm 0.13} 1.53±0.141.53_{\pm 0.14} 0.64±0.090.64_{\pm 0.09}
Overcooked
Fill only 𝟑𝟔±18\mathbf{36}_{\pm 18} 12±012_{\pm 0} 12±012_{\pm 0} 𝟏𝟕±4\mathbf{17}_{\pm 4} 11±011_{\pm 0}
Fetch & Serve 53±2953_{\pm 29} 0±00_{\pm 0} 0±00_{\pm 0} 2±22_{\pm 2} 𝟏𝟒𝟏±18\mathbf{141}_{\pm 18}
Fill & Start 𝟑𝟒𝟏±66\mathbf{341}_{\pm 66} 𝟐𝟒𝟐±117\mathbf{242}_{\pm 117} 150±93150_{\pm 93} 113±70113_{\pm 70} 81±4481_{\pm 44}
Wait & Serve 53±2953_{\pm 29} 0±00_{\pm 0} 0±00_{\pm 0} 2±22_{\pm 2} 𝟏𝟏𝟐±20\mathbf{112}_{\pm 20}
Best cooperator 𝟏𝟒\mathbf{14}/16 00/16 00/16 00/16 22/16

B.5 Mode Collapse in Self-Play Populations

Five self-play populations of four policies cover only 2.8 of four PMR landmarks on average by dominant convention; all four policies in one run select West. Overcooked self-play actions average 44% onion fetching and 36% pot loading, with no soup serving in the displayed runs. Intermediate FCP checkpoints add skill levels without necessarily adding missing roles. These diagnostics concern the tested training setup rather than all self-play algorithms.

Table 6: Self-Play in Point-Mass Rendezvous. Each column is one self-play policy paired with itself. We run 5 self-play experiments, each producing 4 policies from different seeds, and evaluate each policy for 20 episodes. Rows give the percentage of episodes in which the pair meets at each landmark, and the largest in each column is in bold. Counting each policy’s most frequent landmark, only Run 1 covers all four landmarks, and all four policies in Run 5 settle on West, since Self-Play has no principled way of promoting diversity across seeds.
Landmark (%) Run 1 Run 2 Run 3 Run 4 Run 5
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
East 0 0 0 100 35 100 0 0 0 0 0 0 85 0 0 100 0 0 0 0
North 100 15 0 0 0 0 75 0 0 0 100 0 15 0 30 0 0 0 0 0
West 0 0 75 0 0 0 0 0 100 0 0 0 0 0 70 0 100 100 70 100
South 0 85 25 0 65 0 25 100 0 100 0 100 0 100 0 0 0 0 30 0
Table 7: Self-Play in Overcooked. Each column is one self-play policy paired with itself. We run 5 self-play experiments, each producing 4 policies from different seeds, and evaluate each policy for 20 episodes. Rows give each pipeline stage’s share of all stage actions performed by both agents (%), and the largest in each column is in bold.
Stage (%) Run 1 Run 2 Run 3 Run 4 Run 5
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 Mean
Fetch onion 37 40 51 38 58 51 40 54 41 49 35 46 43 42 57 34 51 47 35 39 44
Onion→\topot 31 36 38 35 38 36 35 39 33 28 33 45 33 35 34 34 38 42 35 36 36
Start pot 9 12 0 12 0 0 12 0 8 8 11 0 11 10 0 10 0 0 12 12 6
Fetch dish 11 12 11 8 4 13 13 7 11 8 11 10 11 13 9 10 10 11 10 12 10
Scoop soup 11 0 0 8 0 0 0 0 8 8 11 0 2 0 0 10 0 0 10 1 3
Serve 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

B.6 Task Event Counts in overcooked

Task-event counts over 50 episodes show PopPlay filling/cooking for serving partners and plating/serving for filling partners. CoMeDi loads pots even more often for serving partners, consistent with its higher reward in those cases. The caption mentions partner counts, but the displayed Table 8 contains cooperator event counts keyed by partner; it should not be interpreted as a separate measurement of the partner’s actions.

Table 8: Cooperator task events per held-out partner in overcooked. Per-partner breakdown of Figure 3b. Fill & Start and Fill only load the pot but never serve. Fetch & Serve and Wait & Serve plate and serve but never fetch an onion. Counts are averaged per episode over 50 episodes.
Cooperator
Method Partner Onion →\rightarrow pot Start pot Fetch dish Scoop soup Serve soup
PopPlay Fill & Start 0.230.23 0.220.22 10.6210.62 10.3510.35 8.148.14
Fill only 0.230.23 0.780.78 2.342.34 0.740.74 0.720.72
Fetch & Serve 5.185.18 4.854.85 0.360.36 0.020.02 0.000.00
Wait & Serve 5.205.20 4.854.85 0.410.41 0.030.03 0.000.00
Self-Play Fill & Start 0.000.00 0.000.00 7.967.96 7.167.16 5.555.55
Fill only 0.000.00 0.000.00 22.4722.47 0.000.00 0.000.00
Fetch & Serve 0.000.00 0.000.00 0.050.05 0.000.00 0.000.00
Wait & Serve 0.000.00 0.000.00 0.050.05 0.000.00 0.000.00
FCP Fill & Start 0.000.00 0.000.00 5.515.51 4.844.84 2.962.96
Fill only 0.000.00 0.000.00 1.281.28 0.000.00 0.000.00
Fetch & Serve 0.000.00 0.000.00 0.270.27 0.000.00 0.000.00
Wait & Serve 0.000.00 0.000.00 0.270.27 0.000.00 0.000.00
MEP Fill & Start 0.030.03 0.320.32 4.634.63 3.763.76 2.052.05
Fill only 0.030.03 0.370.37 1.241.24 0.370.37 0.080.08
Fetch & Serve 0.320.32 0.180.18 0.230.23 0.040.04 0.000.00
Wait & Serve 0.310.31 0.180.18 0.230.23 0.050.05 0.000.00
CoMeDi Fill & Start 0.590.59 0.060.06 2.902.90 1.931.93 1.911.91
Fill only 0.540.54 0.070.07 1.891.89 0.030.03 0.030.03
Fetch & Serve 12.8312.83 4.164.16 0.630.63 0.000.00 0.000.00
Wait & Serve 10.3110.31 3.303.30 0.710.71 0.020.02 0.010.01

Appendix C Diverse Behavior Generation

The description-generation study additionally includes Football and SMACv2. Their long or slow rollouts prevented inclusion in the main cooperator-training evaluation, so these results broaden text-generation analysis rather than proving coordination gains in six environments.

Football ( Bettini et al., 2024b ) .

Football descriptions vary positioning, offense and defense around the ball, goals and teammates. No main zero-shot cooperator reward result is supplied for Football.

SMACv2 ( Ellis et al., 2023 ) .

SMACv2 uses two allied Marines against four enemy Marines, encouraging varied target selection, movement, spacing and retreat. It is an additional behavior-generation domain, not one of the four main cooperation benchmarks.

Diverse text generation methods.

Direct prompting samples independently; One-Pass generates the set at once; Sequence uses batches; Multi-Turn conditions on prior descriptions; SimpleStrat samples explicit diversity axes. Verbalized Sampling requests candidate probabilities; its multi-turn version adds history. PopGen’s VS-OS oversamples and makes a separate merge/selection call. Table 9 retains the formal sampling definitions and call counts; default batches contain five candidates.

Table 9: Comparison of prompting methods for generating NN behaviors. kk is the number of candidates per LLM call (in our experiments k=5k=5). yiy_{i} denotes the ii-th generated candidate, p^i\hat{p}_{i} its verbalized probability, and π(⋅|x)\pi(\cdot|x) the LLM’s output distribution conditioned on prompt xx. For Multi-Turn and VS-Multi, hi−1h_{i-1} denotes the conversation history up to turn i−1i-1, and tt the turn index.
Method LLM Calls Candidates Turns Prompt Example Definition
1. Instance-level Prompt
   Direct (DIR) NN 1 1 “Think step-by-step, generate a football player behavior” yi∼π⁡(y|xCoT)y_{i}\sim\pi(y|x_{\text{CoT}})
2. List-level Prompt
   One-Pass (OP) 11 NN 1 “Generate 30 football player behaviors” (y1,…,yN)∼π⁡(y1,…,yN|xOP)(y_{1},\ldots,y_{N})\sim\pi(y_{1},\ldots,y_{N}|x_{\text{OP}})
   Sequence (SEQ) ⌈N/k⌉\lceil N/k\rceil kk 1 “Generate 5 football player behaviors” (y1,…,yk)∼π⁡(y1,…,yk|xseq)(y_{1},\ldots,y_{k})\sim\pi(y_{1},\ldots,y_{k}|x_{\text{seq}})
   Multi-Turn (MT) NN 1 NN Turn 1: “Generate a football player behavior” Turn 2+: “Generate another football player behavior” yi∼π⁡(y|xmulti,hi−1)y_{i}\sim\pi(y|x_{\text{multi}},h_{i-1})
   SimpleStrat (SS) (Wong et al., 2024) ⌈N/k⌉\lceil N/k\rceil kk 1 “Generate a football player behavior following these characteristics: [strata]” yi∼π⁡(y∣xstrati)y_{i}\sim\pi(y\mid x_{\text{strat}_{i}}),  strati∼P^strata\text{strat}_{i}\sim\hat{P}_{\text{strata}}
3. Distribution-level Prompt
   Verbalized-Sampling (VS) (Zhang et al., 2025) ⌈N/k⌉\lceil N/k\rceil kk 1 “Think step-by-step, generate 5 football player behaviors with probabilities” (y1,p^1),…,(yk,p^k)∼π(⋅|xVS-CoT)(y_{1},\hat{p}_{1}),\ldots,(y_{k},\hat{p}_{k})\sim\pi(\cdot|x_{\text{VS-CoT}})
   VS-MT (Zhang et al., 2025) ⌈N/k⌉\lceil N/k\rceil kk ⌈N/k⌉\lceil N/k\rceil Turn 1: “Generate 5 player behaviors with probabilities” Turn 2+: “Generate 5 more with probabilities” ((y1(1),p^1(1)),…,(yk(t),p^k(t))∼π(⋅|xVS,ht−1))\begin{pmatrix}(y_{1}^{(1)},\hat{p}_{1}^{(1)}),\ldots,(y_{k}^{(t)},\hat{p}_{k}^{(t)})\\ \sim\pi(\cdot|x_{\text{VS}},h_{t-1})\end{pmatrix}
   VS-OS (ours) ⌈m/k⌉+1\lceil m/k\rceil+1 kk 1 (VS) “Generate m>Nm{>}N player behaviors with probs” (Merge) “Combine into NN best behaviors” {(yi,p^i)}i=1m∼π(⋅∣xVS),m>N\{(y_{i},\hat{p}_{i})\}_{i=1}^{m}\sim\pi(\cdot\mid x_{\text{VS}}),\;m>N y~1:N=fπ({(yi,p^i)}i=1m)\tilde{y}_{1:N}=f_{\pi}\!\left(\{(y_{i},\hat{p}_{i})\}_{i=1}^{m}\right)
Original figure
Figure 8: Quality and diversity of all behavior generation methods. Quality and diversity of a generated set of 30 behaviors, averaged across the four environments, for every method in Table 9. VS-OS is the method used in PopGen.

C.1 Oversampling to Preserve Quality in Behavior Generation

Oversampling is evaluated as a way to explore broadly before selecting plausible distinct descriptions. More candidates need not always improve the final diversity.

Table 10: Quality and diversity of oversampling variants across environments. Diversity is measured as average pairwise embedding distance; quality as total score. Best value per column in bold. The Sequential ×\times2 variant — referred to as VS-OS throughout the paper — achieves the best diversity–quality trade-off overall.
Diversity (↑\uparrow) Quality (↑\uparrow)
Type Ratio football lbf pmr pp smacv2 overcooked avg football lbf pmr pp smacv2 overcooked avg
Baseline (VS) ×\times1 0.730 0.726 0.636 0.691 0.756 0.612 0.692 12.77 11.16 13.52 12.20 12.00 13.32 12.50
One Pass ×\times2 0.737 0.710 0.645 0.725 0.752 0.634 0.701 12.47 12.24 13.78 11.90 12.44 13.30 12.69
×\times4 0.729 0.684 0.644 0.718 0.747 0.684 0.701 12.50 12.18 13.84 13.04 12.57 13.20 12.89
Independent ×\times2 0.737 0.654 0.637 0.735 0.738 0.616 0.686 12.44 11.97 13.84 12.93 12.20 13.26 12.77
×\times4 0.726 0.672 0.643 0.721 0.749 0.614 0.688 13.01 12.57 13.88 12.77 12.11 13.10 12.91
Sequential ×\times2 0.733 0.703 0.649 0.737 0.740 0.613 0.696 13.01 11.97 13.78 13.14 12.87 13.20 13.00
×\times4 0.738 0.685 0.640 0.719 0.694 0.594 0.678 12.84 12.36 13.74 12.66 12.80 13.22 12.94

Oversampling ablation.

Across the six-environment table, sequential 2× oversampling gives quality 13.00 and diversity 0.696, close to the maximum diversity 0.701. It becomes the default. Sequential 4× lowers diversity to 0.678 and quality to 12.94, consistent with selection pruning unusual behaviors. Other variants likewise improve quality over baseline VS (12.50), but these averages are not the main four-environment Figure 5 values.

Original figure
(a) Detailed diversity metrics: minimum pairwise embedding distance, embedding space coverage, and lexical dissimilarity (1 −- Rouge-L).
Original figure
(b) Scaling VS-OS: generation latency in seconds (left); quality improvement Δ\Delta when scaling the generator from Flash to Pro (center); diversity gain Δ\Delta from the same scaling (right).
Figure 9: Diversity breakdown and scaling properties of VS variants. (a) VS-MT sacrifices worst-case diversity (minimum pairwise distance drops to 0.35) and coverage (0.38) relative to single-pass VS (0.48, 0.44): iterative refinement tends to homogenize outputs. VS-OS recovers both metrics (0.47, 0.45) by decoupling exploration from selection—oversampling casts a wide net, then a dedicated merging step curates without collapsing variety. Lexical dissimilarity is comparable across all three. (b) The decoupled design of VS-OS trades additional latency for better scaling behavior. When moving from Flash to Pro, VS-MT gains quality (+1.00+1.00) but loses diversity (−0.06-0.06), confirming that multi-turn refinement with a stronger model amplifies the homogenization effect. VS-OS, by contrast, translates the stronger model into gains on both axes (+0.57+0.57 quality, +0.04+0.04 diversity): because exploration and selection are separate stages, a more capable model improves the quality of each stage independently without one undermining the other.

Diversity analysis and scaling of VS-OS.

Minimum embedding distance/coverage are 0.48/0.44 for VS, 0.35/0.38 for multi-turn VS, and 0.47/0.45 for VS-OS; lexical dissimilarity is similar. Scaling Flash to Pro gives multi-turn VS +1.00 quality but −0.06 diversity, versus VS-OS +0.57 and +0.04. Oversampling adds generation latency. These diagnostics support broader coverage under the chosen metrics, without proving that each extra semantic distinction becomes a distinct executable strategy.

Transfer of findings from other domains.

SimpleStrat reports quality 13.07 and diversity 0.532, below the displayed Pareto frontier. Verbalized-probability prompts transfer better to this task in the tested settings; multi-turn history tends to improve quality while narrowing variety. The comparison is empirical and model/domain dependent.

Appendix D Modeling Behaviors in Agents with LLMs

The adherence study separates development initializations used in refinement from ten unseen test initializations. This tests initialization generalization under the fixed evaluation partner and judge, not transfer to arbitrary partners or human judgments.

D.1 Behavior Modeling Methods

Compare online LLM actions, LLM-generated rewards followed by RL, and executable policy code. Each starts from a textual target behavior, with different generation and rollout costs.

Table 11: Behavior modeling methods. Each method specifies a distinct mechanism for instantiating a target behavior bb in an agent policy π\pi. oo is the observation of the environment, and aa is the sampled action.
Name Abbr. Description
LLM-Embodied Agent LLM-A Uses an LLM directly as the decision-making agent, conditioned on textualized observations and a behavior specification: πLLM-A∼LLM​(o,b)\pi_{\text{LLM-A}}\sim\text{LLM}(o,b)
Reward Function RF An LLM generates a reward function rb:(o,a)→ℝr_{b}:(o,a)\rightarrow\mathbb{R}, whose optimization via reinforcement learning yields a neural network policy πRF\pi_{\text{RF}} exhibiting the target behavior.
Policy Code PC An LLM synthesizes an executable policy fb:o→af_{b}:o\rightarrow a that directly maps observations to actions, serving as the policy.
Table 12: Adherence scores of PopGen agents across environments and splits. We use gemini-3-flash-preview as the agent generator or player in LLM-A. For each method, we report first-attempt and best-attempt adherence (out of 5 attempts) along with the improvement Δ\Delta. Avg. columns show the mean across environments. Best values per method and split are in bold.
Behavior Adherence (Dev) Behavior Adherence (Test)
Method Attempt LBF PMR PP Overcooked Avg. LBF PMR PP Overcooked Avg.
LLM-A First 13.3±10.413.3\pm 10.4 33.3±7.733.3\pm 7.7 58.3±10.458.3\pm 10.4 30.7±8.030.7\pm 8.0 33.9±9.133.9\pm 9.1 25.4±1.925.4\pm 1.9 26.7±5.826.7\pm 5.8 93.3±5.293.3\pm 5.2 30.7±6.530.7\pm 6.5 44.0±4.944.0\pm 4.9
Best 13.3±10.4\mathbf{13.3\pm 10.4} 50.7±7.8\mathbf{50.7\pm 7.8} 76.7±10.4\mathbf{76.7\pm 10.4} 62.7±9.3\mathbf{62.7\pm 9.3} 50.9±9.5\mathbf{50.9\pm 9.5} 25.4±1.9\mathbf{25.4\pm 1.9} 32.7±6.6\mathbf{32.7\pm 6.6} 93.3±5.2\mathbf{93.3\pm 5.2} 45.3±9.1\mathbf{45.3\pm 9.1} 49.2±5.7\mathbf{49.2\pm 5.7}
Δ\Delta 0.00.0 +17.4+17.4 +18.3+18.3 +32.0+32.0 +16.9+16.9 0.00.0 +6.0+6.0 0.00.0 +14.6+14.6 +5.2+5.2
PC First 43.0±8.443.0\pm 8.4 56.0±8.356.0\pm 8.3 95.0±3.595.0\pm 3.5 34.7±10.834.7\pm 10.8 57.2±7.857.2\pm 7.8 53.5±3.953.5\pm 3.9 58.0±7.558.0\pm 7.5 74.0±8.474.0\pm 8.4 41.3±10.541.3\pm 10.5 56.7±7.656.7\pm 7.6
Best 71.0±6.5\mathbf{71.0\pm 6.5} 84.0±5.9\mathbf{84.0\pm 5.9} 98.0±2.7\mathbf{98.0\pm 2.7} 68.0±10.3\mathbf{68.0\pm 10.3} 80.3±6.4\mathbf{80.3\pm 6.4} 61.0±5.0\mathbf{61.0\pm 5.0} 66.7±6.1\mathbf{66.7\pm 6.1} 74.0±8.4\mathbf{74.0\pm 8.4} 63.3±9.3\mathbf{63.3\pm 9.3} 66.3±7.2\mathbf{66.3\pm 7.2}
Δ\Delta +28.0+28.0 +28.0+28.0 +3.0+3.0 +33.3+33.3 +23.1+23.1 +7.5+7.5 +8.7+8.7 0.00.0 +22.0+22.0 +9.6+9.6
RF First 1.0±2.21.0\pm 2.2 50.7±8.550.7\pm 8.5 68.0±2.768.0\pm 2.7 28.0±11.228.0\pm 11.2 36.9±6.236.9\pm 6.2 7.5±3.57.5\pm 3.5 48.0±7.948.0\pm 7.9 64.5±5.164.5\pm 5.1 28.7±11.128.7\pm 11.1 37.2±6.937.2\pm 6.9
Best 6.0±5.5\mathbf{6.0\pm 5.5} 70.7±7.3\mathbf{70.7\pm 7.3} 86.0±8.9\mathbf{86.0\pm 8.9} 42.7±11.0\mathbf{42.7\pm 11.0} 51.4±8.2\mathbf{51.4\pm 8.2} 12.5±3.5\mathbf{12.5\pm 3.5} 49.3±7.5\mathbf{49.3\pm 7.5} 71.0±5.2\mathbf{71.0\pm 5.2} 40.7±10.3\mathbf{40.7\pm 10.3} 43.4±6.6\mathbf{43.4\pm 6.6}
Δ\Delta +5.0+5.0 +20.0+20.0 +18.0+18.0 +14.7+14.7 +14.4+14.4 +5.0+5.0 +1.3+1.3 +6.5+6.5 +12.0+12.0 +6.2+6.2

Performance.

Use the explicit four-environment Table 12 values: best Policy Code adherence is 80.3% on development and 66.3% on test, versus Reward Function at 51.4%/43.4% and online LLM at 50.9%/49.2%. Online LLM is strongest on PP test adherence (93.3%), so code does not win every environment. Appendix prose instead says 81.3%/63.8% and cites SMACv2/Football figures absent from that table; those inconsistent claims are not used as verified results.

Table 13: Average generation and rollout latency per method. Generation latency covers code synthesis (PC) or RL training (RF); rollout latency is for a single episode. LLM-A has no generation step (—) but incurs high rollout cost from per-timestep LLM inference. Best per column in bold. smacv2 is excluded due to engine overhead dominating latencies.
Method Generation latency (s) ↓\downarrow Rollout latency (s) ↓\downarrow
LLM-A — 110.1±1.5110.1\pm 1.5
RF 2878.6±239.52878.6\pm 239.5 11.7±0.611.7\pm 0.6
PC 8.1±0.2\mathbf{8.1\pm 0.2} 9.8±1.9\mathbf{9.8\pm 1.9}

Latency.

Reported average code synthesis takes 8.1±0.2 seconds and a rollout 9.8±1.9 seconds; reward-policy generation/training takes 2878.6±239.5 seconds and a rollout 11.7±0.6 seconds. Online LLM actions require no artifact-generation step but 110.1±1.5 seconds per rollout. These separate costs exclude a complete accounting of every refinement/judge/advisor call for an entire population; SMACv2 is excluded because engine overhead dominates.

D.2 Impact of the Refinement Loop on PopGen Agents’ Adherence

The refinement analysis compares the first candidate with the best of up to five attempts, then illustrates how an advisor changes code repairs.

Table 14: Policy iterative refinement. Improvement in adherence (first- vs. best-attempt, out of 5 attempts) on the dev and test splits. Avg. columns show the mean across environments. Best values per split are in bold.
Split Attempt    LBF PMR PP Overcooked Avg.
Dev First    43.0±8.443.0\pm 8.4 56.0±8.356.0\pm 8.3 95.0±3.595.0\pm 3.5 34.7±10.834.7\pm 10.8 57.2±7.857.2\pm 7.8
Best    71.0±6.5\mathbf{71.0\pm 6.5} 84.0±5.9\mathbf{84.0\pm 5.9} 98.0±2.7\mathbf{98.0\pm 2.7} 68.0±10.3\mathbf{68.0\pm 10.3} 80.3±6.4\mathbf{80.3\pm 6.4}
Δ\Delta    +28.0+28.0 +28.0+28.0 +3.0+3.0 +33.3+33.3 +23.1+23.1
Test First    53.5±3.953.5\pm 3.9 58.0±7.558.0\pm 7.5 74.0±8.474.0\pm 8.4 41.3±10.541.3\pm 10.5 56.7±7.656.7\pm 7.6
Best    61.0±5.0\mathbf{61.0\pm 5.0} 66.7±6.1\mathbf{66.7\pm 6.1} 74.0±8.4\mathbf{74.0\pm 8.4} 63.3±9.3\mathbf{63.3\pm 9.3} 66.3±7.2\mathbf{66.3\pm 7.2}
Δ\Delta    +7.5+7.5 +8.7+8.7 0.00.0 +22.0+22.0 +9.6+9.6

Refinement loop to improve adherence.

Development adherence increases from 57.2% to 80.3% (+23.1 points); unseen-initialization adherence rises from 56.7% to 66.3% (+9.6 points). PP improves only on development (95%→98%) and stays 74% on test. The smaller test gain shows incomplete transfer from selection/refinement, and these averages remain below perfect adherence.

Feedback examples.

The advisor example fixes reversed movement directions, scores food using teammate motion, chooses an unoccupied support cell and holds a target for three steps. The no-advisor example adds loading without adjacency checks, chases the teammate instead of a support cell, and computes a heading it never uses. Both original truncated diffs remain, including indentation, colors and line-change counts. This is a qualitative comparison of one revision, not a quantified advisor-only ablation.

@@  movement direction mapping
-    if abs(dr) >= abs(dc) and abs(dr) > 0:
-        return torch.tensor(1) if dr > 0 else torch.tensor(2)
+    action = 2 if dr > 0 else 1  # SOUTH if dr>0, NORTH if dr<0
+    return torch.tensor(action)

@@  target food scoring via teammate motion vector
-    closest_dist = float('inf')
-    for f in foods:
-        dist = abs(f['r'] - team_pos[0]) + abs(f['c'] - team_pos[1])
+    def food_score(food):
+        f_dr = food[0] - team_row
+        f_dc = food[1] - team_col
+        return (abs(f_dr - team_dr) + abs(f_dc - team_dc), abs(f_dr), abs(f_dc))
+    best_food = sorted(foods, key=food_score)[0] if foods else None

@@  support-cell selection (closest cardinal cell, not teammate-occupied)
+    for d_row, d_col in [(-1,0),(1,0),(0,-1),(0,1)]:
+        cand_r, cand_c = target_row + d_row, target_col + d_col
+        if not (cand_r == team_row and cand_c == team_col):
+            valid_positions.append((cand_r, cand_c))
+    best_pos = min(valid_positions,
+                   key=lambda p: abs(p[0]-self_row)+abs(p[1]-self_col))

@@  target commitment window
+    if not hasattr(policy, 'target_countdown'):
+        policy.target_countdown = 3
+    if policy.target_countdown == 0:
+        policy.target_food = best_food
+        policy.target_countdown = 3
Original source text
@@  movement direction mapping
-    if abs(dr) >= abs(dc) and abs(dr) > 0:
-        return torch.tensor(1) if dr > 0 else torch.tensor(2)
+    action = 2 if dr > 0 else 1  # SOUTH if dr>0, NORTH if dr<0
+    return torch.tensor(action)

@@  target food scoring via teammate motion vector
-    closest_dist = float('inf')
-    for f in foods:
-        dist = abs(f['r'] - team_pos[0]) + abs(f['c'] - team_pos[1])
+    def food_score(food):
+        f_dr = food[0] - team_row
+        f_dc = food[1] - team_col
+        return (abs(f_dr - team_dr) + abs(f_dc - team_dc), abs(f_dr), abs(f_dc))
+    best_food = sorted(foods, key=food_score)[0] if foods else None

@@  support-cell selection (closest cardinal cell, not teammate-occupied)
+    for d_row, d_col in [(-1,0),(1,0),(0,-1),(0,1)]:
+        cand_r, cand_c = target_row + d_row, target_col + d_col
+        if not (cand_r == team_row and cand_c == team_col):
+            valid_positions.append((cand_r, cand_c))
+    best_pos = min(valid_positions,
+                   key=lambda p: abs(p[0]-self_row)+abs(p[1]-self_col))

@@  target commitment window
+    if not hasattr(policy, 'target_countdown'):
+        policy.target_countdown = 3
+    if policy.target_countdown == 0:
+        policy.target_food = best_food
+        policy.target_countdown = 3
@@  opportunistic LOAD without teammate adjacency check
+    if abs(target_r - self_r) <= 1 and abs(target_c - self_c) <= 1:
+        if food0_l > 0 and (self_l + team_l >= food0_l):
+            return torch.tensor(5)

@@  fallback: move toward teammate rather than support cell
+    dr_to_t = team_r - self_r
+    dc_to_t = team_c - self_c
+    if dr_to_t != 0:
+        return torch.tensor(2 if dr_to_t > 0 else 1) \
+               if abs(dr_to_t) > abs(dc_to_t) \
+               else torch.tensor(3 if dc_to_t > 0 else 4)

@@  heading estimation computed but never used in food selection
+    dr = policy.team_pos[-1][0] - policy.team_pos[-2][0]
+    dc = policy.team_pos[-1][1] - policy.team_pos[-2][1]
+    new_dir = 1 if dr > 0 else -1  # stored but unused
Original source text
@@  opportunistic LOAD without teammate adjacency check
+    if abs(target_r - self_r) <= 1 and abs(target_c - self_c) <= 1:
+        if food0_l > 0 and (self_l + team_l >= food0_l):
+            return torch.tensor(5)

@@  fallback: move toward teammate rather than support cell
+    dr_to_t = team_r - self_r
+    dc_to_t = team_c - self_c
+    if dr_to_t != 0:
+        return torch.tensor(2 if dr_to_t > 0 else 1) \
+               if abs(dr_to_t) > abs(dc_to_t) \
+               else torch.tensor(3 if dc_to_t > 0 else 4)

@@  heading estimation computed but never used in food selection
+    dr = policy.team_pos[-1][0] - policy.team_pos[-2][0]
+    dc = policy.team_pos[-1][1] - policy.team_pos[-2][1]
+    new_dir = 1 if dr > 0 else -1  # stored but unused

Appendix E Prompts

All advisor, adherence-evaluator and quality-judge system/user prompts are retained as original native renderings. Expand their source-text disclosures for the embedded copyable text.

E.1 Advisor

The advisor receives the target behavior, environment, attempted code and rollout evaluations, then diagnoses important failure patterns and recommends concrete edits. It does not itself supply a new policy in the displayed prompt.

You are an advisor for a code generation pipeline.
Your task is to diagnose why a generated code fails to represent its
target behavior in rollouts and provide actionable feedback to improve
the next code attempt.

You will be given:
- The intended behavior
- The environment description
- The code attempt (type: {generation_type})
- Evaluation results from multiple rollout executions.

Focus on the most important failure patterns and give specific,
actionable suggestions to fix the code.
Original source text
You are an advisor for a code generation pipeline.
Your task is to diagnose why a generated code fails to represent its
target behavior in rollouts and provide actionable feedback to improve
the next code attempt.

You will be given:
- The intended behavior
- The environment description
- The code attempt (type: {generation_type})
- Evaluation results from multiple rollout executions.

Focus on the most important failure patterns and give specific,
actionable suggestions to fix the code.
Code generation type: {generation_type}

Inputs
--------------
## Behavior
{behavior}

## Environment Description
{environment_description}

## Code Attempt
```
{code_attempt}
```

## Rollout Evaluation Results
{metrics_str}
--------------
Analyze the results and provide feedback for the next code attempt.
Original source text
Code generation type: {generation_type}

Inputs
--------------
## Behavior
{behavior}

## Environment Description
{environment_description}

## Code Attempt
```
{code_attempt}
```

## Rollout Evaluation Results
{metrics_str}
--------------
Analyze the results and provide feedback for the next code attempt.

E.2 Adherence Evaluator

The adherence evaluator prioritizes serialized states over images, focuses on the ego agent and requested behavior, and returns a boolean plus justification. It may accept a clearly explained good-faith effort when constraints prevent full demonstration. This permissive clause is part of what the reported adherence metric measures, rather than an objective task-success criterion.

You are an expert evaluator of whether an agent's actions match a given
behavior description.
You will be given:
- a behavior description
- agent rollouts (trajectory states and possibly image sequences)

Your task is to determine whether the ego agent's actions adhere to the
behavior description.

# Guidelines:
- Evaluate alignment with the described behavior only.
- Do not consider any other possible goals.
- Focus only on the ego agent.
- Consider interactions with other agents only when relevant to behavior
  following or when they affect the ego agent's ability to demonstrate
  the behavior.
- Use states as the primary source of truth and images as supporting evidence.
- When serialized states are present, interpret them literally and use them
  as the main basis for factual claims.

In rare cases, the agent may be unable to fully demonstrate the expected
behavior due to environment limitations or teammate actions. If the ego
agent still makes a clear, reasonable, good-faith effort to follow the
behavior, you may mark it as adherent. You must explicitly explain the
constraint and justify the decision in the reasoning field.

# Environment Description
{environment_description}

# Output Format
Return a JSON object with exactly these fields:
- "evaluation": a boolean indicating whether the rollout demonstrates
  the behavior
- "reasoning": a detailed explanation of why the rollout does or does
  not demonstrate the behavior, including the single most decisive reason
  for the verdict and specific actions or states when relevant
Original source text
You are an expert evaluator of whether an agent's actions match a given
behavior description.
You will be given:
- a behavior description
- agent rollouts (trajectory states and possibly image sequences)

Your task is to determine whether the ego agent's actions adhere to the
behavior description.

# Guidelines:
- Evaluate alignment with the described behavior only.
- Do not consider any other possible goals.
- Focus only on the ego agent.
- Consider interactions with other agents only when relevant to behavior
  following or when they affect the ego agent's ability to demonstrate
  the behavior.
- Use states as the primary source of truth and images as supporting evidence.
- When serialized states are present, interpret them literally and use them
  as the main basis for factual claims.

In rare cases, the agent may be unable to fully demonstrate the expected
behavior due to environment limitations or teammate actions. If the ego
agent still makes a clear, reasonable, good-faith effort to follow the
behavior, you may mark it as adherent. You must explicitly explain the
constraint and justify the decision in the reasoning field.

# Environment Description
{environment_description}

# Output Format
Return a JSON object with exactly these fields:
- "evaluation": a boolean indicating whether the rollout demonstrates
  the behavior
- "reasoning": a detailed explanation of why the rollout does or does
  not demonstrate the behavior, including the single most decisive reason
  for the verdict and specific actions or states when relevant
Behavior:
{behavior}

Rollout (showing {N} serialized sampled states and {M} sampled images
in chronological order; step numbers are meaningful):
------------------------------------------------------------
[image_1] ... [image_M]          # interleaved chronologically
[state_1]
[state_2]
...
[state_N]
------------------------------------------------------------
Original source text
Behavior:
{behavior}

Rollout (showing {N} serialized sampled states and {M} sampled images
in chronological order; step numbers are meaningful):
------------------------------------------------------------
[image_1] ... [image_M]          # interleaved chronologically
[state_1]
[state_2]
...
[state_N]
------------------------------------------------------------

E.3 Behavior Quality Judge

Quality is scored separately for relevance, realism and specificity, each 0–5. The detailed rubric permits unusual but conceivable styles while penalizing impossible actions or vague instructions. Scores sum to 15; they reflect the model judge’s interpretation of plausible human play, without human calibration in the reported study.

System Prompt
You are an expert in evaluating the quality of agent behavior descriptions.
You will be provided with a behavior description and an environment description.
Your task is to evaluate the quality of the behavior across three rubrics:
1. **Relevance** (0-5): Is the behavior relevant to the environment?
- 0: Completely irrelevant, behavior has nothing to do with the environment or its objectives
- 1: Mostly irrelevant, only tangentially related to the environment
- 2: Somewhat relevant, addresses the environment but misses key aspects
- 3: Moderately relevant, relates to the environment but could apply to many other contexts
- 4: Highly relevant, clearly designed for this environment with appropriate context
- 5: Perfectly relevant, deeply tied to the environment’s unique characteristics and objectives
2. **Realism** (0-5): Could a real player plausibly play this way, given the environment’s constraints?
- 0: Violates hard environment constraints (e.g., impossible actions, broken physics)
- 1: Technically allowed but contradicts the environment so fundamentally that no human would ever play this way
- 2: Extremely unlikely — requires bizarre intent or misunderstanding, but not strictly impossible
- 3: Unusual but conceivable — a small fraction of players (even 0.01%) might genuinely adopt this style
- 4: Plausible — an uncommon but recognizable play style that some players would intentionally choose
- 5: Natural — a common or intuitive way to play that many players would independently arrive at
3. **Specificity** (0-5): Is the behavior specific enough to be actionable?
- 0: Extremely vague, no actionable guidance (e.g., "play well")
- 1: Very generic, could mean almost anything (e.g., "be strategic")
- 2: Somewhat vague, general direction but lacking concrete details
- 3: Moderately specific, provides guidance but leaves room for interpretation
- 4: Specific, clear behavior and conditions with minor ambiguities
- 5: Highly specific, precise behavior, triggers, and success criteria that leave no ambiguity
Provide detailed reasoning for each score.
Original source text
System Prompt
You are an expert in evaluating the quality of agent behavior descriptions.
You will be provided with a behavior description and an environment description.
Your task is to evaluate the quality of the behavior across three rubrics:
1. **Relevance** (0-5): Is the behavior relevant to the environment?
- 0: Completely irrelevant, behavior has nothing to do with the environment or its objectives
- 1: Mostly irrelevant, only tangentially related to the environment
- 2: Somewhat relevant, addresses the environment but misses key aspects
- 3: Moderately relevant, relates to the environment but could apply to many other contexts
- 4: Highly relevant, clearly designed for this environment with appropriate context
- 5: Perfectly relevant, deeply tied to the environment’s unique characteristics and objectives
2. **Realism** (0-5): Could a real player plausibly play this way, given the environment’s constraints?
- 0: Violates hard environment constraints (e.g., impossible actions, broken physics)
- 1: Technically allowed but contradicts the environment so fundamentally that no human would ever play this way
- 2: Extremely unlikely — requires bizarre intent or misunderstanding, but not strictly impossible
- 3: Unusual but conceivable — a small fraction of players (even 0.01%) might genuinely adopt this style
- 4: Plausible — an uncommon but recognizable play style that some players would intentionally choose
- 5: Natural — a common or intuitive way to play that many players would independently arrive at
3. **Specificity** (0-5): Is the behavior specific enough to be actionable?
- 0: Extremely vague, no actionable guidance (e.g., "play well")
- 1: Very generic, could mean almost anything (e.g., "be strategic")
- 2: Somewhat vague, general direction but lacking concrete details
- 3: Moderately specific, provides guidance but leaves room for interpretation
- 4: Specific, clear behavior and conditions with minor ambiguities
- 5: Highly specific, precise behavior, triggers, and success criteria that leave no ambiguity
Provide detailed reasoning for each score.
User Prompt
Environment Description:
{environment_description}
Behavior:
{behavior}
Please evaluate this behavior’s quality across the three rubrics (relevance, realism, specificity).
Original source text
User Prompt
Environment Description:
{environment_description}
Behavior:
{behavior}
Please evaluate this behavior’s quality across the three rubrics (relevance, realism, specificity).
Concept demonstration

Open image ↗