Overview
At a glance
RULER trains a language model to produce SVG code using a rubric tailored to each prompt. A frozen visual judge scores six criteria spanning meaning, visual quality, and rendering style; their weighted average rewards reinforcement learning. On 300 illustration and icon prompts, the trained Qwen3-8B improves the separate evaluation judge’s scores from 0.432/0.395 to 0.693/0.683. Human comparisons favor it over several baselines, although its 53.3% win rate against VectorFusion is a narrow observed edge. Tests stay within one benchmark family, and rubric design can still induce reward hacking.
1 Introduction
Many different vector drawings can satisfy the same instruction, so matching a single reference image is an unsuitable training target. Generic image metrics can also reward attractive but irrelevant graphics. RULER turns a prompt into explicit visual criteria, scores rendered candidates against them, and uses those scores to improve the generator. Its central contribution is the reward specification; generation remains ordinary autoregressive code prediction.
2 Related Work
| Reward Paradigm | Representative Metric | Visual | Semantic | GT-Free | Multi-Axis | Instance- |
|---|---|---|---|---|---|---|
| Granularity | Granularity | Aware | ||||
| Pixel-based | SSIM / PSNR Wang et al. (2004) | Fine-grained | N/A | ✗ | ✗ | ✗ |
| Rule-based | Code Length Rodriguez et al. (2025b) | N/A | N/A | ✓ | ✗ | ✗ |
| Embedding-based | CLIPScore Hessel et al. (2021) | Coarse-grained | Coarse-grained | ✓ | ✗ | ✗ |
| Rubric-based | Universal Rubric He et al. (2026) | Fine-grained | Fine-grained | ✗ | ✓ | ✗ |
| Rubric-based | Instance-aware Rubric (RULER) | Fine-grained | Fine-grained | ✓ | ✓ | ✓ |
2.1 SVG Code Generation
Earlier methods learn vector representations, optimize paths through a diffusion model, or generate SVG programs directly. VectorFusion and SVGDreamer use iterative optimization; IconShop and newer language models generate token sequences. These approaches establish generation capability, while RULER addresses the feedback used to train it [4], [15], [37], [34].
2.2 Rubric-Based Evaluation and Rewards
Rubrics expose separate quality requirements instead of hiding them inside one learned score. Previous work applies this idea to language evaluation and reinforcement learning. RULER adapts the criteria to each visual prompt, where valid solutions vary in shape, composition, and style. Explicit criteria still require testing: optimizing an imperfect reward can exploit its weaknesses [12], [10], [3], [8].
3 RULER
The pipeline generates and caches a rubric for each training prompt, samples several SVG programs, renders them, and scores each rendering. Group-relative reinforcement learning then increases the probability of better candidates. The rubric generator, training judge, and final evaluation judge have distinct roles.
3.1 Task Definition
Generation is modeled as an MDP: the state is the instruction plus the current code prefix, each action appends a token, and the completed program is rendered for a terminal reward. No externally supplied target image is required.
3.2 Human Assessment of Existing Metrics
The authors collect 900 images from three generators on 300 prompts and compare automated metrics with validated human ratings. Rubric scores achieve Spearman correlation 0.7929, versus 0.6051 for aesthetic scores and 0.5518 for CLIP. Pairwise ranking agreement is also higher. This supports the rubric on this sample; it does not establish universal agreement with human taste.
3.3 Instance-Aware Rubric Generation
Claude Opus 4.6 first creates an ideal SVG, then six prompt-specific criteria. The criteria cover concept recognition, essential components, silhouette, composition, execution quality, and stylistic coherence, with weights 5, 5, 5, 4, 5, and 5. They describe design intentions rather than prescribe exact pixels or paths. The internally generated example guides construction and filtering; the training judge does not receive it as a reference image [1].
Equation 1
| (1) |
3.4 Policy Optimization
Reward Calculation.
A frozen Qwen3-VL-8B receives the prompt, its rubric, and the rendered candidate. It assigns each item a satisfaction score between zero and one. The normalized weighted sum becomes the candidate’s reward; the default reward adds no separate CLIP, aesthetic, or text-penalty term [2].
Equation 2
| (2) |
Group Relative Advantage.
GRPO samples eight candidates per instruction and normalizes their rewards within the group. A clipped policy objective favors candidates with positive relative advantage. The displayed formulation uses a sequence-level importance ratio; the experimental configuration omits KL regularization [27], [11], [26].
Equations 3–4
| (3) |
| (4) | |||
4 Experiments
Experiments test overall generation quality, reward alternatives, rubric components, sensitivity to model choices, and visual failure modes. The main results use one inference run; separate appendix experiments repeat inference with five random seeds.
| Method | MMSVG-Illustration | MMSVG-Icon | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| CLIP | Aesth. | HPS | Rubric | Tokens | CLIP | Aesth. | HPS | Rubric | Tokens | |
| Diffusion-Optimized Methods | ||||||||||
| VectorFusion | 0.313 | 4.930 | 0.265 | 0.611 | 31.4k | 0.309 | 4.624 | 0.254 | 0.510 | 31.4k |
| SVGDreamer | 0.293 | 5.119 | 0.266 | 0.547 | 124.3k | 0.276 | 4.850 | 0.253 | 0.413 | 124.3k |
| Foundation LLMs | ||||||||||
| Qwen3-8B | 0.244 | 4.309 | 0.233 | 0.432 | 0.4k | 0.266 | 4.424 | 0.234 | 0.395 | 0.3k |
| Qwen3-32B | 0.224 | 4.392 | 0.215 | 0.651 | 0.5k | 0.228 | 4.514 | 0.218 | 0.681 | 0.3k |
| DeepSeek-V3 | 0.285 | 4.653 | 0.251 | 0.686 | 0.5k | 0.290 | 4.610 | 0.248 | 0.673 | 0.2k |
| Specialist Models | ||||||||||
| IconShop | 0.236 | 4.480 | 0.219 | 0.262 | 2.6k | 0.285 | 4.541 | 0.243 | 0.586 | 1.3k |
| OmniSVG-8B | 0.215 | 4.505 | 0.221 | 0.288 | 6.9k | 0.272 | 4.629 | 0.244 | 0.565 | 5.7k |
| JanusCoder-8B | 0.228 | 4.323 | 0.227 | 0.390 | 1.0k | 0.256 | 4.432 | 0.234 | 0.384 | 1.0k |
| RULER | 0.249 | 4.859 | 0.246 | 0.693 | 2.0k | 0.269 | 4.873 | 0.247 | 0.683 | 0.7k |
4.1 Experimental Setup
Baselines.
Comparisons include iterative VectorFusion and SVGDreamer, general language models, and specialized IconShop, OmniSVG-8B, and JanusCoder-8B. RULER starts from Qwen3-8B; the appendix also trains a 4-billion-parameter version [15], [37], [34], [39], [29], [38].
Benchmarks.
Tests use 150 MMSVG-Illustration prompts and 150 MMSVG-Icon prompts. Training and test prompts are separately generated within the same benchmark distribution, so these results measure generalization to held-out prompts rather than a new domain [39].
Metrics.
The primary score comes from GPT-5-mini applying a shared, universal rubric, separate from the Qwen3-VL-8B training judge. CLIP, aesthetic scores, and HPS provide additional views. Automated scores include only successfully rendered outputs; render success is reported separately. Token counts measure generated SVG code length, not total training or inference computation [14], [35], [20].
4.2 Main Results (RQ1)
Consistent improvements across baselines.
RULER raises the primary rubric score from 0.432 to 0.693 on Illustration and from 0.395 to 0.683 on Icon. The comparison with identical-base-model reward alternatives supports the contribution of the training reward. Improvements are not uniform across every auxiliary image metric.
Competitive performance on auxiliary metrics, top-tier on the rubric score.
RULER’s single-run rubric scores slightly exceed DeepSeek-V3’s 0.686/0.673 and Qwen3-32B’s 0.651/0.681. These numerical differences should not be read as established statistical superiority. Its outputs average 2.0K/0.7K tokens: far shorter than iterative vectorization outputs, but longer than the untrained Qwen3-8B’s 0.4K/0.3K.
Human preference confirms the performance gains.
Across 150 prompts per comparison, human win rates excluding ties are 89.7% against Qwen3-8B, 66.1% against Qwen3-32B, 53.3% against VectorFusion, 66.9% against OmniSVG-8B, and 96.5% against JanusCoder-8B. The VectorFusion comparison is close: 73 wins, 64 losses, and 13 ties.
| Method | MMSVG-Illustration | MMSVG-Icon | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| CLIP | Aesth. | HPS | Rubric | Tokens | CLIP | Aesth. | HPS | Rubric | Tokens | |
| Zero-Shot | 0.244 | 4.309 | 0.233 | 0.432 | 0.4k | 0.266 | 4.424 | 0.234 | 0.395 | 0.3k |
| C+A+H RL | 0.196 | 6.697 | 0.235 | 0.464 | 0.7k | 0.232 | 6.210 | 0.239 | 0.262 | 6.3k |
| Universal Rubric RL | 0.244 | 4.813 | 0.241 | 0.660 | 0.7k | 0.266 | 4.793 | 0.244 | 0.591 | 0.3k |
| RULER | 0.249 | 4.859 | 0.246 | 0.693 | 2.0k | 0.269 | 4.873 | 0.247 | 0.683 | 0.7k |
4.3 Reward Design Analysis (RQ2)
Reward comparisons hold the Qwen3-8B starting policy and optimization settings fixed. They contrast a mixture of conventional metrics, a universal rubric, and the instance-aware rubric.
RULER delivers the strongest balanced gains.
RULER reaches 0.693/0.683, compared with 0.660/0.591 for reinforcement learning with a universal rubric. The gains are 0.033 on Illustration and 0.092 on Icon, supporting prompt-specific criteria in this setting.
Universal rubrics improve steadily but lack instance-level granularity.
A universal rubric already improves substantially over the untuned model, indicating that structured visual feedback helps even without instance-specific descriptions.
Conventional metrics collapse into reward hacking.
Optimizing the CLIP, aesthetic, and HPS mixture raises its aesthetic scores but reduces Icon’s universal rubric score to 0.262 from 0.395. Outputs can accumulate overlapping paths and lose their intended meaning. Higher proxy rewards therefore need not indicate better instruction following.
4.4 Ablation Study (RQ3)
Effectiveness of the three rubric axes.
Removing any rubric axis hurts Illustration performance: the primary score falls from 0.693 to 0.580 without visual quality, 0.599 without rendering style, and 0.623 without semantic fidelity. All three axes contribute under this ablation.
Sensitivity to rubric design.
The stricter, structure-focused Rubric-S scores only 0.536 on Illustration and encourages prompt-related text inside drawings despite an explicit text penalty. Rubric wording and calibration can redirect optimization toward shortcuts; adding criteria does not guarantee a better reward.
4.5 Robustness Analysis (RQ4)
Robustness across base models.
Across five inference seeds, the 4-billion-parameter model improves from 0.372 ± 0.017 to 0.561 ± 0.008 on Illustration and from 0.338 ± 0.017 to 0.559 ± 0.006 on Icon. RULER-8B reaches 0.692 ± 0.007 and 0.662 ± 0.016. These are repeated inference runs, not independent training replications.
Robustness across rubric generators.
Replacing Claude Opus 4.6 with GPT-5.5 for rubric generation and retraining Qwen3-8B preserves gains on Icon: 0.646 ± 0.007 versus 0.662 ± 0.016 with Claude-generated rubrics. The same frozen training judge is retained [1], [21].
4.6 Qualitative Analysis (RQ5)
The illustrated examples show clearer subjects, more coherent compositions, and cleaner vector construction after training. They help explain score changes but remain selected qualitative examples, not a measure of failure frequency.
5 Conclusion
Instance-aware visual rubrics provide useful reinforcement-learning rewards for text-to-SVG generation without externally paired target images. The gains depend on how criteria are written and judged; the unsuccessful Rubric-S variant shows that reward hacking remains possible.
Limitations
Rubric generation inherits the frontier model’s preferences, and the frozen visual judge can introduce further biases. Rendering and judging every rollout add training cost. Fixed quality axes may undervalue legitimate artistic choices. Evaluation covers two tasks in one benchmark family; performance across other visual domains and independently repeated training runs remains unestablished.
Acknowledgments
The authors acknowledge support from the Ant Group internship program and feedback from colleagues.
Appendix A Training Details
Training uses VERL and vLLM with thinking disabled, eight rollouts per prompt, sampling temperature 1.0, and maximum prompt/response lengths of 512/4,096 tokens. The batch size is 128, learning rate 10⁻⁶, and entropy coefficient 0.001, with no KL term. Experiments use BF16, gradient checkpointing, and tensor parallelism across eight NVIDIA H800 GPUs with 80 GB each.
Appendix B Dataset Construction
From 32,000 sampled training prompts, rubric construction removes malformed responses and cases where the generated ideal SVG scores below 0.9 under its own rubric. This retains 19,531 Icon and 11,206 Illustration examples, or 30,737 total. The ideal images are generated for this process, not supplied as human ground truth. Filtering couples the dataset to the generator’s and judge’s preferences.
Appendix C Reward Computation and SVG Rendering
CairoSVG 2.9.0 renders candidates at 512 × 512 pixels on a white background. Render failures and effectively blank images receive zero reward. Successful images are judged item by item and aggregated with the displayed weighted mean. Rubric-S’s negative text penalty enters only the numerator; its denominator uses the five positive weights.
Equation
Appendix D Human Evaluation
Two human studies answer different questions: whether metrics track holistic image ratings, and whether people prefer RULER’s final outputs to competing generators.
D.1 Human Alignment Study
For metric alignment, 10 annotators rate 900 images from Claude Opus 4.6, Qwen3-32B, and Qwen3-8B on 300 prompts. Three validators review unreasonable scores. Each image receives one validated holistic rating from 0 to 100. Correlation and pairwise ranking agreement are computed against those ratings.
D.2 Final-Output Human Preference Study
The preference study compares RULER with five baselines on 150 prompts, producing 750 blinded, randomly ordered pairs distributed among three annotators; each pair is judged once. Judges consider meaning, visual quality, and rendering coherence. Ties are excluded from win rates. Unlike automated quality scores, these comparisons penalize a render failure when the opposing output succeeds.
Equation
| Comparison | Win | Tie | Loss | Win Rate |
|---|---|---|---|---|
| RULER vs. Qwen3-8B | 122 | 14 | 14 | 89.7% |
| RULER vs. Qwen3-32B | 84 | 23 | 43 | 66.1% |
| RULER vs. VectorFusion | 73 | 13 | 64 | 53.3% |
| RULER vs. OmniSVG | 93 | 11 | 46 | 66.9% |
| RULER vs. JanusCoder | 139 | 6 | 5 | 96.5% |
Appendix E Additional Evaluation and Robustness Analyses
E.1 Stability Across Inference Runs
Five independently seeded inference runs report means and standard deviations. They assess sampling variability for trained policies, not variation from retraining the entire method. These repeated-run averages differ from the single-run main table.
E.2 Render Success Rates
RULER renders successfully on 99.3% of Icon prompts and 100% of Illustration prompts, versus 94.7% and 89.3% for Qwen3-8B. This separate accounting matters because failed renders are excluded from automated quality averages.
| Method | MMSVG-Icon | MMSVG-Illustration |
|---|---|---|
| RULER | 99.3% | 100.0% |
| OmniSVG | 99.3% | 99.3% |
| DeepSeek-V3 | 100.0% | 99.3% |
| JanusCoder | 68.0% | 68.7% |
| Qwen3-8B | 94.7% | 89.3% |
| Qwen3-32B | 99.3% | 100.0% |
| IconShop | 100.0% | 100.0% |
| VectorFusion | 100.0% | 100.0% |
| SVGDreamer | 100.0% | 100.0% |
E.3 Policy-Scale Analysis
Training helps both the 4-billion- and 8-billion-parameter base policies under repeated inference. This supports the method at the two tested scales, without establishing scaling behavior beyond them.
E.4 Rubric Generator Analysis
The generator comparison changes which frontier model writes the training rubrics while retaining the Qwen3-8B policy and Qwen3-VL-8B judge. Both rubric sources improve Icon results, although their averages differ.
E.5 Comparison with Iterative Diffusion Methods
Iterative diffusion methods generate much longer path programs. The authors report VectorFusion averaging 72.1 minutes per sample in their implementation, while RULER uses one autoregressive generation pass. No corresponding numerical RULER latency is supplied, so the paper does not establish an end-to-end speedup factor.
Appendix F Rubric Design Details
F.1 Ground-Truth-Free Rubric Construction
Rubric construction takes only the instruction as external input but internally synthesizes an ideal SVG before writing criteria. Reward computation later evaluates a candidate against the rubric without comparing it with that ideal. Criteria target recognizable design intentions, leaving room for different valid drawings.
F.2 Rubric-S
Rubric-S emphasizes readability, components, shape complexity, internal details, and layering. A sixth item penalizes text hints with weight −2; a higher score on that item means stronger evidence of a shortcut. Its lower quality score and increased textual shortcuts demonstrate sensitivity to the reward specification.
Appendix G Rubric Generation Cost
The cached training rubrics cost $2,013.12 to generate, averaging $0.0655 per retained prompt. The reported 60.18 million input and 68.49 million output tokens include ideal SVG generation as well as rubric text. This is preprocessing cost, not total training cost: the frontier generator is absent during optimization, but rendering and visual judging continue for each rollout.
| Statistic | MMSVG-Icon | MMSVG-Illustration | Overall |
|---|---|---|---|
| Prompts | 19,531 | 11,206 | 30,737 |
| Avg. Input Tokens | 1,959 | 1,957 | 1,958 |
| Avg. Rubric Tokens | 1,010 | 1,057 | 1,027 |
| Avg. Output Tokens | 1,934 | 2,740 | 2,228 |
| Total Cost | $1,135.77 | $877.35 | $2,013.12 |
| Cost / Prompt | $0.0582 | $0.0783 | $0.0655 |
Appendix H Prompt Templates
H.1 Rubric Generation Prompt
The preserved template requests an ideal SVG followed by six weighted, independently scorable criteria. It emphasizes observable design intentions and calibrated scoring instead of exact reconstruction.



H.2 Rubric-based Judge Prompt
The item-level judge receives the instruction, candidate rendering, and instance-specific rubric. It returns reasoning and a zero-to-one satisfaction score for each criterion, using only the stated requirements.

H.3 Universal Rubric Judge Prompt
The universal evaluation prompt applies shared semantic, visual, and rendering criteria to all candidates and returns an overall score. It also discourages using written labels as substitutes for drawing the requested subject.



H.4 Rubric-S Generation Prompt
The alternative generation prompt specifies five structural criteria with stricter scoring guides. The separate preserved panel supplies the fixed negative-weight text-hint penalty used during training.


