Overview
This edition reviews the complete fourteen-page arXiv v1 dated 17 December 2025, including the supplementary material. The title and all eight authors match; the existing CVPR 2026 publication link is retained.
At a glance
VLIC trains a diffusion image compressor using pairwise reconstruction preferences from a vision-language model. A discrete encoder and stochastic decoder first learn reconstruction; Diffusion DPO then favors samples judged closer to the original. The main reward combines Gemini 2.5-Flash judgments with LPIPS agreement.
Human studies and image metrics show gains over selected baselines, especially on MS-COCO, while PO-ELIC remains stronger on CLIC 2022. Preference agreement, perceptual quality, pixel accuracy and compressed size are separate measurements; no universal improvement across all of them is established.
1 Introduction
Compression trades transmitted bits against reconstruction quality. Pixel errors need not reflect the importance of readable text, recognizable faces or other semantic details. Learned perceptual losses help, but optimizing their scores can exploit weaknesses that do not improve what people see.
The paper asks whether a pretrained VLM can judge two reconstructions directly, avoiding a separately distilled differentiable reward network. It extends FlowMo with scalar quantization, entropy coding and Diffusion DPO. Its main system still uses LPIPS during pretraining and preference filtering, so it does not eliminate learned perceptual metrics entirely.
The contribution is a codec/post-training design plus evidence on two preference benchmarks, three compression datasets and reward ablations. Figure 2 preserves qualitative comparisons at their displayed, sometimes different, bitrates; these images are examples rather than a matched-rate statistical test.
2 Related Work
The relevant alternatives differ in how they decode images, approximate human visual similarity and optimize a preference signal.
Perceptually-oriented Image Compression.
GAN codecs such as HiFiC and PO-ELIC often decode quickly. Diffusion methods either reconstruct from a compressed bottleneck or use the diffusion reverse process itself as an entropy code. VLIC follows the bottleneck-autoencoder family, using FlowMo’s architecture with a separate autoregressive entropy model.
Approximations to Human Perception of Visual Similarity.
LPIPS, E-LPIPS, DreamSim and DISTS approximate aspects of human image similarity. Text-guided codecs and pretrained visual models also target semantic information. VLIC instead obtains direct judgments from a VLM; its final training pipeline retains LPIPS as a complementary constraint.
Aligning Diffusion Models to Preferences.
Differentiable reward methods backpropagate a score, whereas preference optimization can use discrete rankings. VLIC uses the diffusion adaptation of DPO, inspired by earlier VLM rewards for image generation. The paper’s DDPO shorthand refers here to Diffusion Direct Preference Optimization, which must be distinguished from the separately cited Denoising Diffusion Policy Optimization method.
3 Method
An encoder deterministically maps an image to a discrete latent sequence. An autoregressive entropy model compresses that sequence; a diffusion decoder samples a reconstruction. During post-training, two samples from the same code become a preference pair.
Architecture and training.
The transformer codec replaces FlowMo’s lookup-free quantizer with finite scalar quantization, removing its commitment and entropy losses. Pretraining combines rectified-flow reconstruction with LPIPS on the one-step denoised prediction.
Diffusion DPO compares the current decoder’s denoising error against a fixed reference for winning and losing samples. Its logistic objective favors the winner relative to the loser; β controls departure from the reference and the timestep weight depends on signal-to-noise ratio. Both samples share an original image and latent conditioning, reducing variation from differing inputs.
The original flow loss is retained with weight λFlow to stabilize longer post-training. The encoder remains trainable so its representation can adapt to the new reward. Afterwards, the encoder and decoder are frozen while a separate autoregressive transformer learns token probabilities for arithmetic coding.
All five numbered objectives remain below. The source defines velocity as v=ε+x, but does not specify the interpolation convention needed to interpret that sign; this original expression is retained. Numerical β and λFlow settings, decoder step count and guidance scale are not supplied in the paper.
Equations 1–5
| (1) |
| (2) |
| (3) |
| (4) |
| (5) |
The VLIC reward function.
Gemini 2.5-Flash receives the original and two reconstructions, explains visible differences and rates relative similarity from −5 to +5. Negative favors A; positive favors B. The full prompt prioritizes human-relevant semantic changes over texture changes.
Each seed evaluates both reconstruction orders, reverses the corresponding signs and takes the sign of the summed ratings. Votes are then summed over three seeds. The final pair is used only when the VLM ranking agrees with LPIPS; disagreement is discarded. Thus ordinary accepted-pair scoring uses six VLM calls before filtering. Exact handling of tied votes or equal LPIPS scores is not specified.
Order reversal reduces positional bias, repeated seeds reduce noise, and LPIPS agreement constrains the reward. These measures improve selected outcomes but do not guarantee a correct judgment; the filtering also changes which training pairs survive.
Equations
4 Experiments
The experiments evaluate image quality and human preferences, then separate the effects of preference post-training, the VLM, repeated judgments and LPIPS filtering.
Our setup.
Two nominal pre-entropy rates, 0.07 and 0.21 bits per pixel, are trained on 256×256 ImageNet images. Pretraining uses one million Adam steps at 10⁻⁴; DPO uses 8,000 steps at 5×10⁻⁷. Both use batch 256, bfloat16 JAX and 256 TPUv4 accelerators.
Online preference buffers contain about 2,560 candidates and refresh every 250 steps; LPIPS/VLM disagreement can reduce their usable size. VLM queries run asynchronously while optimization uses a slightly older buffer. The authors report online buffers work better than an offline set, without a numerical offline comparison here.
At inference, overlapping tiles extend the 256-pixel model to larger images. A shifted diffusion schedule and classifier-free guidance are used; discrete conditioning is dropped in 10% of training cases. Nominal rates before entropy coding should not be substituted for the plotted realized bitrates.
4.1 Main results.
The main comparison uses dataset-specific baseline availability and evaluates both perceptual fidelity and conventional pixel distortion.
Baselines.
HiFiC is run across all datasets. PerCo uses an open-source replication, and is evaluated only on low-resolution MS-COCO. HFD’s CLIC 2020 scores come from its paper after reproducing its evaluation pipeline. PO-ELIC uses released CLIC 2022 reconstructions. Missing entries reflect unavailable implementations or unsupported resolution; this is not one fully rerun common baseline suite.
Datasets.
MS-COCO uses 30,000 validation images cropped to 256×256 with the DALL-E protocol, compared with HiFiC and PerCo. CLIC 2020 uses 428 high-resolution training-set images, up to four megapixels, compared with HiFiC and HFD. CLIC 2022 uses thirty high-resolution test images, compared with HiFiC and PO-ELIC.
| MS-COCO | Human Elo | FD-DINO | FID | LPIPS | PSNR |
|---|---|---|---|---|---|
| Ours (0.07bpp) | 858 | 62.25 | 2.20 | 0.274 | 21.78 |
| LPIPS post-training only | 838 | 63.63 | 2.33 | 0.274 | 21.77 |
| Ours (0.21bpp) | 1112 | 16.83 | 1.35 | 0.168 | 26.50 |
| LPIPS post-training only | 1103 | 16.96 | 1.30 | 0.169 | 26.54 |
Metrics.
LPIPS and PSNR measure paired-image discrepancy; FID and FD-DINO measure distributional differences. The authors give greatest weight to human Elo and then FD-DINO, while acknowledging that perceptual quality and pixel accuracy can conflict at a fixed rate.
Human studies collect 1,812 CLIC 2020, 2,523 CLIC 2022 and 15,705 MS-COCO pairwise ratings. Elo is averaged over 10,000 random orderings to reduce update-order dependence. Elo values are relative to the compared pool and study design, not percentages or directly comparable absolute quality across datasets.
High-resolution distributional evaluation uses random 256×256 crops. The prose specifies one hundred crops per CLIC 2022 image and ten per MS-COCO image, despite discussing CLIC 2020 in that passage; a CLIC 2020 crop count is not explicitly given. Human raters also see crops for the high-resolution datasets. The paper does not report rater counts/demographics, confidence intervals or repeated-training-seed uncertainty.
Results.
Figure 4 shows particularly strong human/perceptual performance on MS-COCO and generally favorable comparisons with HiFiC, PerCo and HFD. PO-ELIC outperforms VLIC on CLIC 2022, whose test set contains thirty images; absent PO-ELIC results elsewhere do not establish VLIC’s superiority there.
Improved perceptual scores can accompany worse PSNR, notably against HFD and HiFiC. This supports a different rate/perception tradeoff rather than lossless recovery or a claim that all reconstruction details are more accurate.
4.2 Analysis
The analysis tests whether VLM judgments agree with people, whether they add value beyond LPIPS, and how noise affects the post-training signal.
Replicating human judgments.
The new compressed-image preference set has 5,401 tuples, each rated at least three times. The human baseline holds out one rater and compares that answer with the remaining raters’ average; it is not a human correctness ceiling.
On BAPPS, VLM agreement is 69.44%, versus LPIPS 69.56% and held-out humans 73.99%. On compressed images, VLM reaches 83.80%, below LPIPS 92.32% but above the single-human agreement of 72.15%. These results support useful zero-shot correlation, not superior judgment to LPIPS or humans in every sense. The authors attribute noisy human agreement to highly similar reconstructions.
Importance of the VLM.
Adding VLM agreement to LPIPS-only preference training raises human Elo from 838 to 858 at nominal 0.07 bpp and from 1103 to 1112 at 0.21 bpp. FD-DINO also improves. At the higher rate, however, FID worsens from 1.30 to 1.35 and PSNR falls from 26.54 to 26.50. At the lower rate LPIPS ties at 0.274. The advantage is metric-dependent, and the table provides no uncertainty on those differences.
Mitigating noise.
Figure 6 shows the same two reconstructions ranked inconsistently after swapping their order. Repeated seeds reduce this failure mode. Figure 5 rises from roughly 67% agreement with one seed toward 71% with seven or eight, then falls slightly; more calls are not strictly monotonically better in the plotted run. Main training uses three seeds as a compute/quality compromise.
Ablation study.
Removing DPO worsens all four Table 3 metrics. Removing LPIPS filtering improves distributional FD-DINO/FID (67.83/2.31 to 67.68/2.10), but worsens LPIPS/PSNR (0.278/21.68 to 0.280/21.29). Removing self-ensembling similarly improves FID to 2.15 while degrading the other three metrics.
Consequently the full reward is not best on every measure, and this ablation does not include human Elo. Its full-model numbers differ from Table 1’s nominal 0.07-bpp configuration, so the two tables should not be treated as identical runs.
Limitations.
Diffusion decoding is slower than typical GAN decoding, and multi-call VLM rewards cost more than a small perceptual network. The paper does not quantify decoding latency, VLM query expense or preference-pair acceptance rate. A stochastic perceptual reconstruction can preserve plausible appearance while changing details; the method is lossy.
5 Conclusion
Pairwise VLM judgments can help train a perceptual compressor when combined with diffusion preference optimization and noise mitigation. The evidence is strongest for the tested datasets, model and reward design; broader improvements from future VLMs remain a hypothesis.
Acknowledgments.
The authors thank Ben Poole, David Minnen and Dina Bashkirova for discussions.
Appendix A Models and training
All three components are transformers. Low/high-rate models each total 1.01B parameters. The encoder uses width 768 and eight layers; decoder width 1152 and sixteen layers; entropy coder width 768 and sixteen layers. Both codec transformers use patch size four, sequence length 4352 and eight scalar-quantizer levels. Token sizes are six versus eighteen; entropy sequences have length 384 versus 1152. Tables 4–5 retain the exact configuration.
Training uses ImageNet at 256-pixel resolution, random horizontal flips and center crops. Pretraining has no dropout or weight decay. DPO refreshes its 2,560-sample buffer every 250 steps. Finally, the NanoGPT-based entropy model trains for 200k AdamW steps at 10⁻⁴ with dropout 0.1 and decay 0.025, while codec weights stay frozen.
Appendix B VLM prompt
The exact Gemini 2.5-Flash prompt is retained below. It establishes the A/B score sign, favors semantic fidelity, discourages extreme scores without clear evidence and requires a final RATING field. Post-training uses the parsed numerical rating rather than directly optimizing the explanation text.
“In this task, you’ll be asked to compare an original image with two AI reconstructions of that image, Reconstruction A followed by Reconstruction B. You’ll see triplets of images like (original, A, B). You’ll give a relative rating of the two images. This is between -5 and 5, inclusive. Higher scores mean that Reconstruction B is relatively better. So if you give a score of -2, you think A is kind of better, and if you give a score of +5, you think B is obviously significantly better. Be sparing with the higher magnitude scores - you should expect that most triplets you see won’t have an obviously better reconstruction. In your ratings, make sure to prioritize differences that are semantically important to a human observer. If a distortion changes the meaning of an image to a human observer, then it’s more significant than if a distortion changes the texture of an image. Your response should conclude with “RATING: X”, where X is your rating, i.e. -2 or 3. Now here is the image triplet that you need to rate. Make sure to provide a lot more reasoning and make sure to carefully look at each of the three images and provide meticulous visual justifications based on evidence from each image! Remember, negative scores mean that A is better, and positive scores mean that B is better.”
Appendix C Inference
Choose the smallest nonnegative tile counts whose coverage reaches the image height and width, using native resolution r=256 and margin p=8. Resize to those coverage dimensions, encode each overlapping tile independently, then jointly diffuse the tiles using MultiDiffusion. Overlap communicates context across borders but costs extra coded area. This is a resolution-extension procedure, not preservation of the exact original pixel grid.
Equation
Appendix D User studies
Raters see A, the original and B in that order and select one of five responses: A much/slightly closer, about the same, or B slightly/much closer. Images are displayed at width 480 pixels; CLIC 2020/2022 use random crops. The exact interface is not disclosed for the authors’ stated policy reasons. The mapping from these five responses to Elo updates, including ties, is not fully specified.
Appendix E Additional capabilities
An additional experiment asks the VLM to transcribe text and rewards small edit distance from the original transcription. The codec learns to censor readable text, bounding the empty-output penalty by the original text length instead of preserving characters. Figure 8 documents reward exploitation; alternative edit-distance formulations give only small, generally imperceptible improvements.
Appendix F Disclaimer
Some benchmark preparation flips images horizontally. For visual presentation the authors may flip both VLIC and baseline reconstructions back, especially when text makes orientation noticeable. This display adjustment is disclosed and should not be mistaken for a new reconstruction result.
Appendix G Image attributions
The full original image-source, creator, identifier and license attribution list is preserved below, including credits for the external supplementary gallery.
The authors’ interactive gallery provides additional scene and method comparisons beyond the manuscript figures.
Figure 1, Truck
- Sourced From: ImageNet
- Image ID: n03594945_15055
- License: ImageNet Agreement
Figure 1, Street Sign
Figure 1, Bar
- Sourced From: COCO
- Image ID: 000000000723
- Original Creator: Henry Zbyszynski
- License: CC BY 2.0
- URL: https://www.flickr.com/photos/hankzby/7385695522/
Figure 2, Chicago Skyline
Figure 2, Street Sign
- Sourced From: COCO
- Image ID: 000000000250
- Original Creator: Unknown
- License: CC BY-NC-SA 2.0
- URL: http://images.cocodataset.org/train2014/COCO_train2014_000000000250.jpg
Figure 2, Women on Phone
- Sourced From: COCO
- Image ID: 000000000536
- Original Creator: Unknown
- License: CC BY-NC-SA 2.0
- URL: http://images.cocodataset.org/val2014/COCO_val2014_000000000536.jpg
Figure 2, Soldiers
- Sourced From: COCO
- Image ID: 000000001149
- Original Creator: Picatinny Arsenal
- License: CC BY-NC 2.0
- URL: https://www.flickr.com/photos/picatinnyarsenal/5202323780/
Figure 3, Gazelles
- Sourced From: ImageNet
- Image ID: n02423022_31692
- License: ImageNet Agreement
Figure 6, Mouse
- Sourced From: ImageNet
- License: ImageNet Agreement
Gallery, Bridge
Gallery, Tennis player
Gallery, Woman
- Sourced From: COCO
- Image ID: 000000001569
- Original Creator: Tom Conger
- License: CC BY-NC 2.0
- https://www.flickr.com/photos/tomconger/3578082722/
Gallery, Hunter