Literature
Display
← All papers

Generating Fine Details of Entity Interactions

Xinyi Gu · Jiayuan Mao

· EMNLP 2025 Industry Track ↗

Computer Vision11 min text · CondensedOriginal paper ↗
Contents

Overview

This edition covers the complete v2 supplement, including evaluation instructions, generation prompts, topic-frequency lists, and all 50 selected dataset examples. The benchmark emphasizes unusual interactions, often involving animals using tools; its results concern the specified models and evaluation protocol.

Complete Figure 1: paired interaction examples and generated gallery.
Figure 1: Left: DetailScribe improves the base text-to-image model across three scenarios: functional interaction, complex scene layouts, and multi-subject interactions. Right: A gallery showcasing DetailScribe-generated images with rich entity interactions.

At a glance

InterActing contains 1,000 LLM-generated prompts targeting functional actions, interactions among multiple subjects, and compositional spatial layouts. DetailScribe uses a language model to decompose an instruction into interacting parts, a multimodal model to critique an initial image, and a partially restarted diffusion process to apply a refined prompt.

On the tested SD3.5-large configuration, this procedure improves human ratings and aggregate automated alignment scores. Its useful corrections depend on an initially plausible global scene. The project page provides the dataset and code.

1 Introduction

Recognizable objects are insufficient when an instruction requires a tool to touch the correct surface, two subjects to coordinate, or objects to form a particular layout. These details can fail even in visually polished images. InterActing makes such interactions the evaluation target, and DetailScribe uses explicit sub-interaction checks to guide inference-time corrections.

The method requires no user-supplied reference image or additional model training. It adds language-model reasoning and image critique to generation; the experiments mainly test one refinement round rather than an open-ended iterative correction loop.

Scenario Subclass Examples
Functional and
Action-Based Interactions (600)
Tool Manipulation (227) cutting, painting, sailing, stirring, taking a photo
Physical Contact (373) sculpting snow, stacking, holding
Multi-subject Interactions (200) Interaction (200) huddling, high-five, collaborating to lift, weaving leaves together, sharing food
Compositional
Spatial Relationships (200)
Abstract Layouts (183) tic-tac-toe, table, atom, solar system, forest, tree, bookshelf
Geometric Patterns (17) zig-zag pattern, circle, center
Table 1: The InterActing dataset contains 1000 text-to-image prompts by subclass and occurrence.

2 Related Works

Diffusion generators such as Stable Diffusion 3 can produce strong object appearance while missing relational details. Existing benchmarks and learned scores emphasize complementary aspects of fidelity, alignment, or composition; InterActing focuses on contact, actions, and unusual layouts.

Prior self-correction, visual-programming, and inference-time noise-search methods motivate feedback-driven generation. DetailScribe’s tested contribution is to structure that feedback around sub-interactions and preserve part of an initial image through re-denoising.

Refer to caption
Figure 2: The overall pipeline of DetailScribe. DetailScribe takes as input a single natural language instruction. It first prompts a large language model (LLM) to generate a breakdown of the concepts in the image, which guides a multimodal LLM (MLLM) to attend to different regions of a generated image and suggests fixes. It then adds noises back to the generated image and re-runs the diffusion process with the MLLM-refined prompt.

3 The InterActing Dataset

The dataset has 600 functional/action prompts, 200 multi-subject prompts, and 200 compositional-layout prompts. Table 1 divides these into 227 tool-manipulation examples, 373 physical-contact examples, 200 interactions, 183 abstract layouts, and 17 geometric patterns.

GPT-4o generates topics and then completes each prompt. Many examples deliberately place animals in unusual activities, making the benchmark a targeted synthetic distribution rather than a representative sample of everyday image requests.

3.1 Evaluation Metrics

Human annotators score image–prompt alignment from one to five, with instructions that emphasize missing tools, incorrect contact, and spatial relations. Automatic evaluators are compared with human preferences on a sampled set before being used across the dataset. This calibration supports their use as proxies without making them equivalent to human assessment.

Automatic evaluation.

The automatic measures are a one-to-five MLLM judgment, ImageReward, CLIPScore, and BLIP-VQA. Different scales and learned biases mean that their numerical values should be compared within a metric. The authors use Qwen2.5-VL-32B for judging and GPT-4o for generation feedback, reducing direct reuse of the same model without eliminating evaluation bias.

Refer to caption
Figure 3: MLLM-based critique and prompt refinement. Given the LLM-generated concept decomposition and an image generated using the user input, an MLLM generates a critique of errors in the image, suggests corrections, and finally refines the prompt. This prompt will be used in a second-round diffusion process.

4 DetailScribe

DetailScribe first decomposes the text instruction, generates an initial image, critiques it against the decomposition, and re-denoises it under a corrected prompt. The learned image generator remains fixed. Structured critique determines what to request, while partial re-denoising controls how strongly the image may change.

4.1 Prompt Completion by Concept Decomposition

Following the visual abstraction schema idea, an LLM breaks an interaction into entities, subparts, and contact relations. A hedgehog rolling dough becomes checks for holding the rolling pin and contact between pin and dough. One cooking example supplies the in-context demonstration for every task. The implemented output is a concise decomposition checklist; the paper does not learn a new scene-graph model.

Refer to caption
Figure 4: Images generated by DetailScribe and baselines on the InterActing dataset. From left to right: 1) Stable Diffusion; 2) SD + GPT Rewrite; 3) SD + GPT Refine; 4) SD by Alternative seed/initial noise in the Inference scaling experiment; 5) DALL·E 3; 6) Ours: DetailScribe. DetailScribe consistently provides effective corrections, which help generate images that closely follow the fine details in the prompts.

4.2 Multimodal LLM-Based Critique and Prompt Refinement

GPT-4o examines the initial image using the decomposition, identifies abnormal or missing parts, ranks their importance, and writes correction instructions. The prompt requires retaining the original first sentence and appending concise edits within a 70-word description. A good critique can expose local failures, but an MLLM may still miss mistakes or suggest inappropriate changes.

4.3 Refinement by Diffusion Re-denoising

The method adds noise back to the generated image at an intermediate diffusion step and denoises again under the revised text. The retained original equation specifies the forward-noise mixture. Starting closer to full noise allows larger changes; starting near a clean image preserves more content but may leave errors unchanged.

The procedure does not specify a spatial edit mask, so “targeted” means text-guided correction rather than a guarantee that only selected pixels change. The reported setting near T − 2 is specific to the tested schedule; the paper does not provide a universal noise-strength rule.

EquationIt=α¯t​I0+1−α¯t​𝒩​(0,1),I_{t}=\sqrt{\bar{\alpha}_{t}}I_{0}+\sqrt{1-\bar{\alpha}_{t}}\mathcal{N}(0,1),

5 Experiments

Experiments compare SD3.5-large, three inference modifications, DALL·E 3, and DetailScribe. Human evaluation uses 50 sampled prompts, while automatic metrics also cover all 1,000 prompts. Ablations separate decomposition from unstructured critique and compare where re-denoising starts.

Functional Relation Multi-subject Interaction Compositional Relation
Human Qwen2.5 ImReward CLIPS. B-VQA Human Qwen2.5 ImReward CLIPS. B-VQA Human Qwen2.5 ImReward CLIPS. B-VQA
SD 3.360 4.480 1.657 0.971 0.366 3.225 3.800 1.171 0.894 0.306 3.583 4.867 1.415 0.898 0.379
+ GPT Rewrite 3.770 4.440 1.546 0.921 0.245 3.275 3.900 1.101 0.890 0.292 3.167 4.467 1.283 0.867 0.325
+ GPT Refine 3.450 4.600 1.524 0.951 0.290 3.175 3.600 1.022 0.858 0.292 3.667 4.800 1.390 0.889 0.389
+ Inf Scale 3.270 4.560 1.718 0.985 0.434 3.375 4.400 1.149 0.903 0.302 3.650 4.800 1.538 0.912 0.423
DALL·E 3 3.940 4.680 1.535 0.880 0.226 3.775 4.700 1.111 0.813 0.286 3.433 4.867 1.382 0.838 0.367
DetailScribe 4.280 4.880 1.761 0.998 0.449 3.800 4.600 1.326 0.907 0.343 4.283 5.000 1.545 0.923 0.485
Table 2: Average human/MLLM likert scale (1 - 5) and pre-trained metrics on three scenarios of sampled InterActing dataset. We report the human Likert scale (Human Evaluation), MLLM evaluation score (Qwen2.5-VL-32B), as well as ImageReward (ImReward), CLIPScore (CLIPS.) and BLIP-VQA (B-VQA) score. DetailScribe receives the highest scores according to human preference in all scenarios.

5.1 Baselines and Implementation Details

GPT Rewrite expands a prompt before generation; GPT Refine critiques an initial image and regenerates; the inference-scaling baseline chooses between two random noise samples using CLIPScore. The latter is explicitly a small implementation inspired by noise search, not the full cited method. DetailScribe uses GPT-4o and SD3.5-large for one initial generation plus refinement.

The paper estimates roughly twice the base diffusion compute for refinement and two-seed search, excluding MLLM API critique time, described as about 10% of SD3.5 runtime. DALL·E 3 has a different model and internal prompt processing, so this is not a controlled comparison of identical training or total service cost.

5.2 Result

Human scores for DetailScribe are 4.280, 3.800, and 4.283 in the three scenarios, versus SD’s 3.360, 3.225, and 3.583. The multi-subject advantage over DALL·E 3 is only 0.025 (3.800 versus 3.775); no uncertainty intervals or significance tests are reported.

Across all prompts, DetailScribe leads the four reported aggregate metrics: Qwen 4.570, ImageReward 1.460, CLIPScore 0.923, and BLIP-VQA 0.381. Qwen’s stated 87.6% agreement with human preferences exceeds the other metrics’ 67.6–73.6%, but the paper gives limited detail on pair selection, ties, or a separate held-out calibration set. Qualitative successes illustrate corrections, not their frequency across all prompts.

Qwen2.5 ImReward CLIPS. B-VQA
SD 4.228 1.323 0.902 0.336
+ GPT Rewrite 4.116 1.193 0.880 0.268
+ GPT Refine 4.141 1.255 0.880 0.300
+ Inf Scale 4.126 1.354 0.922 0.365
DALL·E 3 4.519 1.222 0.860 0.312
DetailScribe 4.570 1.460 0.923 0.381
Table 3: Automatic evaluation on entire InterActing. DetailScribe outperforms all baselines on all metrics. We include evaluation by scenario in Appendix D.

5.3 Ablation: Hierarchical Concept Decomposition Improves Error Detection

Adding explicit decomposition raises the sampled human score from 3.843 to 4.187, Qwen from 4.660 to 4.760, and each listed pretrained metric slightly. The examples show feedback shifting from generic appearance toward missing tools and action contacts. These point estimates support the component’s usefulness under the protocol; the paper does not report confidence intervals or a formal significance test.

Refer to caption
Figure 5: An illustrative example showing the effectiveness of the explicit concept decomposition module. MLLM first critiques the original generation, and identifies the features needs to be correct (red) and the features non-necessary for further modification (green), and then provides the corrected prompt for re-denoising.
Human Qwen2.5 ImReward CLIPS. B-VQA
w/o. Decomp 3.843 4.660 1.586 0.953 0.410
w/. Decomp 4.187 4.760 1.609 0.957 0.438
Table 4: Ablation study on the effectiveness of the concept decomposition module. Explicit concept decomposition significantly improves the generation quality.

5.4 Ablation: Re-Denoising Step

Figure 6 compares full-noise restart T with T − 1, T − 4, and T − 6. Stronger re-noising can fix large errors while changing composition or introducing new ones; weaker re-noising can add small details but preserve a bad layout. The authors favor a setting near T − 2, although the figure does not provide a quantitative sweep establishing that optimum.

Starting from pure noise with a more detailed prompt can also omit entities. Partial refinement is therefore presented as a coarse-to-fine strategy. Multiple refinement rounds and broader seed search remain future work.

Refer to caption
Figure 6: Ablation on the number of re-denoising steps. The first column shows the result without the refined prompt. Column 2-5: Starting the re-denoising at step TT, T−1T-1, T−4T-4, and T−6T-6.

6 Conclusion

The study combines an interaction-focused prompt set with decomposition-guided image feedback and partial diffusion refinement. Its main limitation is reliance on a roughly correct initial global structure: missing main subjects, wrong zoom, or major layout errors may survive even a correct critique. Selecting better initial seeds could address some failures, but that extension is not established by the reported experiments.

Acknowledgements

The authors thank collaborators and four volunteers who supplied human evaluations. Support includes NSF, AFOSR, ONR, ARO, MIT Quest for Intelligence, the Boston Dynamics AI Institute, and the Simons Center for the Social Brain; the source attributes the conclusions to the authors.

Appendix A DetailScribe Prompting Implementation Details

The exact decomposition and critique prompts are retained below, together with the provided input/output examples. They define the practical meaning of the structured feedback used in the experiments.

A.1 Prompt: Concept Decomposition

Decomposition uses one cooking example and asks for only necessary, simultaneous contact relationships. The retained examples concern a polar bear cutting cake and a flower circle. The response includes an apparent trailing JSON fragment in the source; it is preserved rather than silently repaired.

We can decompose each abstract concept into interactions defined by contact points and contact objects. 

For example,

(concept: cooking)= 

(hand hold the handle of a ladle) + (ladle stir the ingredient in the pot) + (pot is on a stove)



Please do the same for the following concepts in the same format without explanation.

Keep the program simple. Use only the most necessary parts of the schema that can be mapped to objects in an image. Describe only the interactions that can happen simultaneously.


[Concepts]
1. Topic: polar-bear-cut-cake

Prompt: An anime of a polar bear carefully cutting a berry cake.
2. Topic: Inside a Circle

Prompt: A circle of sunflowers with a single, vibrant red rose in the very center, surrounded by the larger yellow blooms.
1. Topic: polar-bear-cut-cake

Prompt: An anime of a polar bear carefully cutting a berry cake.

Components: (concept: polar-bear-cut-cake) = (paw hold knife) + (knife cut through cake) + (cake rest on plate)
2. Topic: Inside a Circle

Prompt: A circle of sunflowers with a single, vibrant red rose in the very center, surrounded by the larger yellow blooms.

Components: (concept: circle) = (sunflowers form circle) + (red rose placed in center) + (yellow blooms surround rose)", "description": "A circle of sunflowers with a single, vibrant red rose in the very center, surrounded by the larger yellow blooms.

A.2 Prompt: Critique and Prompt Refinement

Critique receives the image, original instruction, and decomposition without an additional demonstration. It requests a ranked error list followed by a corrected description, keeps the original opening sentence, and limits the description to 70 words. The instruction explicitly tells the model not to question the original scene concept.

This is an image generated with the prompt: [prompt]. But this image looks bizarre. Examine the image carefully follow the concept of  [topic] attached below and other components in the image. For each abnormal part, describe what is wrong with it, then give a concise description on how to correct it. 



Components: [Components]



Do not simply rely on the components described above, but also exam whether an object looks complete.

First, write your answer in a numbered list,

Then, rank the issues by their degree of impact on presenting the concept.

Last, summarize the correction instructions in order, and write a new description with the first sentence to be [prompt] followed by correction instructions.



Do not change the first sentence.

Be concise, no more than 70 words, but make sure not to miss any information that needs to be corrected.

Provide the new description in angle brackets <>.

The components described in the original prompt are essential, do not question the concepts in the original prompt.

Appendix B InterActing Generation Details

GPT-4o first proposes interaction topics, then expands each into an image prompt. The following templates preserve the examples, requested topic counts, output formatting, and restrictions used to construct the synthetic benchmark.

B.1 Functional and Action-Based Interactions

Functional/action examples are generated through separate tool-manipulation and physical-contact topic lists, followed by a common animal-centered prompt-completion template.

B.1.1 Topic Generation: Tool Manipulation

The tool template asks for 150 lowercase concepts related to cutting cake, carving wood, cutting pizza, and painting a portrait. A polar bear with a knife illustrates an unusual but visually depictable interaction.

Given a tool manipulation action, we can create some novel and previously unseen scene or cartoon that can be present by an image. For example,



Concept: Cut-Cake

Tool: knife

Image description: n anime of a polar bear carefully cutting a berry cake.



Think of concepts similar to cut-cake, carve-wood, cut-pizza, paint-portrait. Provide 150 different but similar concepts, separate them by comma ’,’.

All lowercase please.

B.1.2 Topic Generation: Physical Contact

The physical-contact template asks for 150 concepts similar to stacking or holding, illustrated by a rabbit sculpting a snow bunny. Requested generation counts are template instructions, not the final subclass totals in Table 1.

Given an action has direct physical contact, we can create some novel and previously unseen scene or cartoon that can be present by an image. For example,



Concept: sculpting-snow

Image description: A rabbit carefully sculpts a tiny snow bunny with its paws, adding details like ears and whiskers to the figure.



Think of concepts similar to stacking, holding. Provide 150 different but similar concepts, separate them by comma ’,’.

All lowercase please.

B.1.3 Prompt Completion

Prompt completion asks for an animal performing the supplied action, emphasizes the main relation, excludes passive voice, and returns topic/prompt JSON. The camera-holding squirrel example makes the expected format explicit.

Come up with a description of an animal [content], the description should be similar as the following example and uncommon to be observed. Do not use passive voice. 

Double check the description to focus on major relation, which is [content].
Write down your answer in this format: {"topic": [content], "prompt": description} 

For example: 

interaction: "taking photos" 

Entities: squirrel 

Description: A squirrel taking photos with a camera.

Then, the output should be: {"topic": "taking-photos", "prompt": "A squirrel taking photos with a camera."}

B.2 Multi-subject Interactions

Multi-subject generation asks for activities involving two animals and preserves the named interaction in the completed description.

B.2.1 Topic Generation

The topic generator requests 100 lowercase interaction concepts, using a dolphin and seal high-fiving as its example. The source wording and spelling remain in the retained template.

Given a verb of 2 subjects’ interaction, we can create some novel and previously unseen scene or cartoon that can be present by an image. For example,



Concept: High-Fiving

Image description: A dolphin and a seal leap from the water, high-fiving with their flippers.


Think of concepts similar to High-Fiving, Lifting-Togethe, huddling-for Warmth. Provide 100 different but similar concepts, separate them by comma ’,’.

All lowercase please.

B.2.2 Prompt Completion

Completion explicitly requires the exact supplied interaction word in the description and produces topic/prompt JSON. This lexical constraint does not ensure that a generated image depicts the action correctly.

Come up with a description of two animals doing [content], the description should be similar as the following example and uncommon to be observed. Do not use passive voice.

Double check the description to focus on major relation, which is [content].
Write down your answer in this format: {"topic": [content], "prompt": description}

The description must contains the exact [content] word.

For example:

Concept: "High-Fiving"

Description: A dolphin and a seal leap from the water, high-fiving with their flippers.

Then, the output should be: {"topic": "High-Fiving", "prompt": "A dolphin and a seal leap from the water, high-fiving with their flippers."}

B.3 Compositional Spatial Relationships

Spatial prompts combine familiar abstract structures with unusual materials or objects. Layout and geometric-pattern topics are generated together and then classified manually with LLM assistance.

B.3.1 Topic Generation

The template requests 300 concepts similar to tic-tac-toe, an atom, a triangle, or a tree, using tomatoes and cucumbers as playing symbols. These are candidate topics before final dataset categorization.

Given an abstract concept, we can create some novel scene that can be present by an image. For example,



Concept: tic-tac-toe

Image description: A tic-tac-toe composed by tomato and cucumber as the players symbols.



Think of concepts similar to tic-tac-toe, atom, triangle, tree. Provide 300 different but similar concepts, separate them by comma ’,’.

All lowercase please.

B.3.2 Prompt Completion

Completion asks for a novel visual combination around the supplied concept, avoids passive voice, and returns topic/prompt JSON. The source examples specify composition through objects rather than a numerical geometric constraint system.

Come up with a description of a scene which is a novel combination of [content], the description should be similar as the following example and uncommon to be observed. Do not use passive voice. 

Double check the description to focus on major relation, which is [content].
Write down your answer in this format: {"topic": [content], "prompt": description} 

For example: 

Concept: "tic-tac-toe" 

Description: A tic-tac-toe composed by tomato and cucumber as the players symbols. 

Then, the output should be: {"topic": "tic-tac-toe", "prompt": "A tic-tac-toe composed by tomato and cucumber as the players symbols."}

Appendix C Model Evaluation Details

This appendix retains human-rating instructions, exact MLLM evaluation prompts, full-dataset scenario scores, and examples of evaluator failures. Human and automated ratings measure perceived instruction alignment, not physical validity under a simulator.

C.1 Human evaluation

Four volunteers rate randomly shuffled outputs for each prompt from one (irrelevant) to five (accurate and natural). Intermediate levels distinguish severely distorted/missing objects, missing interaction components, and minor relational or subpart problems. The paper averages annotator scores but does not report inter-rater reliability or uncertainty intervals.

C.2 MLLM Evaluation Prompt Implementation Details

The evaluator receives the generated image, prompt, topic, and a scenario-specific one-to-five rubric. Its required output is an angle-bracketed score, with no calibrated probability interpretation.

C.2.1 Functional and Action-Based Interactions, Multi-subject Interactions

The action/multi-subject rubric emphasizes whether the subject performs the action, whether critical tools or patterns are missing, and whether limbs or tool parts require improvement. Its ordering differs slightly from the human rubric; the exact wording is retained.

The above images were generated with the prompt: [prompt].

Please rate the text-image alignment score of each image from 1 to 5, focusing on [topic] and follow the criteria: 



1: poor interaction, subject(s) not acting correctly.

2: subject(s) incorrect/inaccurate.

3: critical part missing (e.g., missing critical tools or patterns to complete [topic]).

4: nearly perfect but some subparts need further improvement (e.g., needs to refine appearance of tools or limbs, subject is not [topic] correctly).

5: image perfectly depicts [topic].



Return the score in angle brackets <>. For example, if the image is nearly perfect and got score 4, response: <4>

C.2.2 Compositional Spatial Relationships

The spatial rubric asks whether the specified objects occupy the intended layout, ranging from irrelevant to correct for all named objects. It supplies a score-four output example, not an image-based demonstration of the rating boundaries.

You are my assistant to identify objects and their spatial layout in the image. According to the image, evaluate if the text [prompt] is correctly portrayed in the image.


Give a score from 1 to 5 according to the criteria: 

5: correct spatial layout ([topic]) in the image for all objects mentioned in the text.

4: basically, spatial layout of objects matches the text.

3: spatial layout not aligned properly with the text.

2: image not aligned properly with the text.

1: image almost irrelevant to the text.


Return the score in angle brackets <>. For example, if the image’s spatial layout of objects matches the text and got score 4, response: <4>

C.2.3 Automatic evaluation Results

Table 5 reports scenario-level scores for the full dataset. DetailScribe leads the SD-based alternatives, but DALL·E 3 receives higher Qwen ratings for multi-subject interactions (4.310 versus 4.275) and compositional relations (4.850 versus 4.830). Thus the aggregate win in Table 3 is not a win in every evaluator/scenario combination.

Functional Relation Multi-subject Interaction Compositional Relation
Qwen2.5 ImReward CLIPS. B-VQA Qwen2.5 ImReward CLIPS. B-VQA Qwen2.5 ImReward CLIPS. B-VQA
SD 4.271 1.471 0.914 0.430 3.780 1.247 0.917 0.203 4.550 0.949 0.851 0.184
+ GPT Rewrite 4.133 1.285 0.881 0.323 3.735 1.206 0.910 0.198 4.445 0.902 0.846 0.171
+ GPT Refine 4.198 1.401 0.889 0.376 3.755 1.216 0.904 0.204 4.355 0.852 0.830 0.169
+ Multi-seed 4.282 1.502 0.935 0.472 3.845 1.242 0.934 0.210 4.565 1.019 0.872 0.197
DALL·E 3 4.477 1.343 0.869 0.392 4.310 1.139 0.868 0.193 4.850 0.938 0.824 0.190
DetailScribe 4.582 1.598 0.936 0.482 4.275 1.378 0.935 0.242 4.830 1.123 0.875 0.216
Table 5: Auto evaluation on entire InterActing dataset by scenario.

C.3 Agreement between human evaluation and automatic Evaluation

Figure 7 preserves examples where automatic judges overlook incorrect object intersections or missing essential components, alongside an example with more consistent judgments. Qwen tends toward high ratings, including near-ceiling values that can conceal errors. The authors recommend human evaluation when feasible; using a separate judge model does not remove this limitation.

Refer to caption
Figure 7: Example of automatic evaluation misinterpreted and successfully interpreted the critical interaction in prompts. The details in the prompt missed by the evaluators are highlighted in blue with an underline. (H: human, Q: Qwen2.5, I: ImageReward, C: CLIPScore, B: BLIP-VQA, Highest score of each metrics are highlighted in red)

Appendix D Additional Results and Analysis

Additional examples compare baseline generations, critique variants, progressive refinement, and sensitivity to the initial random seed. These visual studies illustrate mechanisms and failure modes without replacing the quantitative evaluation.

D.1 Qualitative Examples

Figure 8 extends the comparison to unusual materials, animal actions, multi-subject scenes, and layouts. It preserves all baseline columns and prompts, including cases where several methods produce plausible images.

Refer to caption
Figure 8: More examples generated by DetailScribe and baselines on the InterActing dataset.

D.2 Qualitative Studies on the Effect of Concept Decomposition and Progressive Refinement

Figure 9 contrasts the full method with feedback lacking decomposition and generation without progressive refinement. Decomposition directs attention toward interaction requirements; retaining the initial image can preserve a useful composition while applying corrections. The figure gives selected examples rather than an additional quantified effect size.

Refer to caption
Figure 9: Qualitative examples that illustrate the effectiveness of concept decomposition and progressive refinement. We compare our full pipeline (shown as full system in the last two columns with two alternatives: one without concept decomposition (shown as w/o Decomposition), and another one without the generate-then-refine procedure (shown as w/o progressive refinement).

D.3 Qualitative Studies on the Effect of Global Seeds

Different seeds can produce substantially different initial subjects and layouts, and partial refinement inherits those differences. Figure 10 shows that improving local interaction details does not reliably correct global structure or zoom. Sampling and selecting multiple starting images is proposed as a future extension.

Refer to caption
Figure 10: Different global random seeds yield different initial images. They also result in different refined images using the DetailScribe framework.

Appendix E InterActing Dataset

The appendix provides complete topic-frequency lists and 50 selected prompts, rather than all 1,000 prompt texts. Both the counts and the selected descriptions are retained verbatim below; the complete dataset is linked from the project page.

E.1 Functional and Action-Based Interactions

Functional/action examples cover tools, contact, posture, and other relation phrases. The published topic lists preserve distinctions and overlaps that should not be read as a rigorously disjoint physical taxonomy.

E.1.1 Statistic of topic

The retained lists enumerate tool-manipulation and physical-contact topics with occurrence counts. They include broad relations and occasional abstract terms as well as concrete actions, which qualifies the intuitive labels of the categories.

• Tool manipulation: octopus paint canvas(1), cat sail(1), fox stir stew(1), squirrel take photo(1), play chess(1), carve wood(1), fly a kite(1), make a potion(1), polar bear cut cake(1), raccoon sweep floor(1), rabbit paint mural(1), bear mop floor(1), painted a portrait(1), reading a book(2), opening a door(1), skiing down(2), slicing(2), skateboarding(2), typing on(2), tying(2), cleaning(1), cooking(2), chopping vegetables(1), dragging(1), skating on(1), photographing(1), baking a cake(1), brushing(1), pouring(1), playing piano(1), playing video games(1), painting(1), writing a book(1), cutting(2), driving(2), using tools(1), constructing bridges(1), playing the piano(1), assemble chair(1), burn wood(1), clip cloth(1), crack coconut(1), decorate cake(1), dip pen ink(1), fold paper crane(1), hang picture(1), measure flour(1), mix dough(1), mount painting(1), paint mural(1), pour syrup(1), saw ice(1), sew button(1), sew patch(1), shape clay(1), shovel snow(1), sketch blueprint(1), sort silverware(1), spray paint(1), spread butter(1), spread paint(1), chop vegetables(1), sharpen pencil(1), drill hole(1), slice bread(1), hammer nail(1), saw plank(1), stir soup(1), sew fabric(1), knit scarf(1), whisk eggs(1), grate cheese(1), peel apple(1), crack walnut(1), roll dough(1), frost cupcake(1), flip pancake(1), grill steak(1), dice carrots(1), scramble eggs(1), boil pasta(1), roast marshmallow(1), squeeze lemon(1), grind coffee(1), sculpt ice(1), weld metal(1), etch glass(1), sand wood(1), string beads(1), carve pumpkin(1), mend socks(1), tie knot(1), cut origami(1), bend wire(1), engrave stone(1), pour tea(1), ladle soup(1), weave basket(1), thread needle(1), sketch bird(1), chisel marble(1), polish shoe(1), blow glass(1), stamp seal(1), dip paintbrush(1), spray graffiti(1), lace shoes(1), brush hair(1), write calligraphy(1), erase mistake(1), clamp wood(1), file nail(1), clip coupon(1), punch hole(1), staple paper(1), tape box(1), tie bow(1), glaze pottery(1), stack books(1), roll sushi(1), fill bottle(1), slice melon(1), scoop ice cream(1), carve sign(1), build clock(1), stretch canvas(1), pluck strings(1), shred paper(1), scrape icing(1), light candle(1), zip jacket(1), unlock door(1), wrap gift(1), rinse brush(1), fold laundry(1), pour candle wax(1), whittle stick(1), cut hair(1), plaster wall(1), glue model(1), mix paint(1), stencil design(1), thread loom(1), cut paper snowflake(1), iron shirt(1), chainmail weaving(1), stamp pattern(1), embroider flower(1), stack firewood(1), filter coffee(1), frost window(1), tie fishing line(1), shape snowball(1), drill teeth(1), paint fence(1), sharpen knife(1), pour molten metal(1), cut ribbon(1), stretch cloth(1), fold napkin(1), heat metal(1), frame picture(1), string violin(1), curl hair(1), carve soap(1), squeeze clay(1), lay tiles(1), weave rug(1), nail shoe(1), stitch wound(1), break egg(1), spread jam(1), poke hole(1), patch clothes(1), snap twig(1), snap chopsticks(1), pick lock(1), snip hedge(1), clean brush(1), file metal(1), press flower(1), scoop sand(1), mold snowman(1), snap photo(1), knot rope(1), chop logs(1), chain carvings(1), stir sauce(1), cut vinyl(1), fold fan(1), tape frame(1), trim trees(1), sift flour(1), stir coffee(1), whisk cream(1), shell peanut(1), pry lid(1), knead bread(1), scrub floor(1), filter water(1), stain wood(1), melt chocolate(1), stir tea(1), light match(1), brush coat(1), tie lace(1), saw bamboo(1), peel corn(1), scrape wax(1), rotate key(1), dip spoon(1), stack bricks(1), weave tapestry(1), bind book(1), decorate mask(1), skim cream(1), pour wine(1), paint sculpture(1)

• Physical Contact: gripping(1), clutching(1), hugging(1), clasping(1), twisting(3), lifting(1), balancing(1), squeezing(2), pressing(2), pinching(1), pulling(3), pushing(5), wrapping(2), cupping(1), stroking(1), rubbing(1), kneading(1), twirling(1), braiding(1), weaving(1), shaping(1), molding(1), rolling(1), spinning(2), tapping(2), scratching(1), clapping(1), smudging(1), flicking(2), catching(2), tossing(3), kicking(1), propping(1), supporting(1), cradling(1), wiping(1), polishing(2), dusting(1), scrubbing(1), slapping(1), punching(1), drumming(2), doodling(1), carving(1), shuffling(1), stacking(1), arranging(1), aligning(1), linking(1), snapping(2), unrolling(1), folding(1), creasing(1), peeling(1), popping(1), tying(1), knotting(1), stretching(1), skipping(1), waving(1), shaking(2), fanning(1), poking(1), nudging(1), flipping(1), scooping(1), ladling(1), swiping(2), tugging(2), shoveling(1), sifting(1), spreading(1), smoothing(1), plucking(1), patting(1), scraping(1), slathering(1), dipping(1), drizzling(1), stamping(1), tracing(1), sketching(1), threading(1), embroidering(1), ruffling(1), petting(1), nuzzling(1), tickling(1), adjusting(1), placing(1), tucking(1), clicking(1), rotating(1), juggling(1), whittling(1), cranking(1), filing(1), plating(1), tacking(1), dabbing(1), buffing(1), dancing(2), pouring(1), jostling(1), stirring(1), rabbit sculpt snow(1), heron fishing(1), build(1), bake(1), serve(1), cook(1), frog sing(1), beaver drink(1), rabbit set table(1), squirrel carve acorn(1), fox pour tea(1), cricket write music leaves(1), beaver cut pizza(1), draped over(2), playing guitar(7), playing chess(4), balancing on(2), posing with(2), reflecting in(2), eating at(2), walking up(2), sewn on(2), getting on(1), approaching(2), walking towards(2), walking to(1), growing by(2), grabbing(2), playing music(1), scattered on(1), jumping on(1), climbing(2), pointing at(2), coming down(2), preparing(2), going into(2), decorating(2), growing from(1), washing(2), herding(2), chewing(2), working in(2), picking up(2), looking over(2), shining through(2), smelling(1), running through(1), enclosing(1), going through(1), walking into(1), falling off(1), decorated with(1), walking past(1), towing(1), blowing out(1), jumping off(1), moving(1), running across(1), hang from(1), sitting around(1), cooked in(1), buying(1), standing around(1), growing behind(1), exiting(1), jumping over(1), looking down at(1), looking into(1), leaning over(1), growing next to(1), observing(1), traveling on(1), wading in(1), growing along(1), opening(1), eating in(1), standing against(1), trying to catch(1), stacking rocks(1), lying next to(1), guiding(1), smoking(1), conducting interviews(1), wearing(2), holding(2), sitting on(2), standing on(2), riding(2), standing in(2), lying on(2), hanging on(2), eating(2), looking at(2), covering(1), sitting in(2), hanging from(2), parked on(2), riding on(2), covered in snow(1), flying in(2), sitting at(2), playing with(2), reading(2), reading books(2), filled with laughter(1), crossing(1), swinging(2), standing next to(2), touching(1), flying(2), contain(2), hitting(2), lying in(2), standing by(2), driving on(2), throwing(2), sitting on top of(2), walking down(2), parked in(2), standing near(2), performing tricks(1), printed on(1), facing(2), leaning against(2), grazing on(2), standing in front of(2), drinking(2), topped with(2), swimming in(2), driving down(2), hanging over(2), feeding(2), waiting for(1), running on(2), talking to(1), holding onto(1), eating from(1), perched on(1), parked by(1), hanging above(1), floating on(1), wrapped around(1), near(1), carrying(1), walking on(1), covered in leaves(1), watching(1), covered in(1), enthusiasm(1), ambition(1), walking in(1), surrounded by(1), pulled by(1), growing on(1), standing behind(1), playing(1), mounted on(1), surfing(1), talking on(1), worn on(1), resting on(1), floating in(1), lying on top of(1), playing in(1), walking with(1), pushed by(1), playing on(1), sitting next to(1)

E.1.2 Selected prompts

All 25 selected functional/action prompts are retained. They include animals painting, cooking, building, playing games, and handling objects, with explicit descriptions of the unusual scene.

1. Topic : Octopus-Paint-Canvas Prompt: An octopus in an art studio is painting on a canvas.

2. Topic : Cat-Sail Prompt: A cat sails across the sea in a large seashell, holding a mast.

3. Topic : Fox-Stir-Stew Prompt: A fox stirs a stew in a hollowed-out tree trunk.

4. Topic : Squirrel-Take-Photo Prompt: A squirrel taking photos with a camera.

5. Topic : Rabbit-Sculpt-Snow Prompt: A rabbit carefully sculpts a tiny snow bunny with its paws, adding details like ears and whiskers to the figure.

6. Topic : Heron-Fishing Prompt: A heron fishing by the river.

7. Topic : Build Prompt: An anime of a mouse constructing a tiny castle with blocks, carefully stacking each piece, with tools like a mini hammer and ruler scattered around.

8. Topic : Bake Prompt: An anime of a hedgehog in a tiny apron, rolling dough with a miniature rolling pin, preparing a berry pie with a cheerful expression.

9. Topic : Play-Chess Prompt: An anime of a raven perched on a table, moving pieces on a tiny chessboard with its beak, calculating each move as it faces off against another bird.

10. Topic : Carve-Wood Prompt: An anime of a beaver wearing a small hat, using its teeth to carve an intricate wooden statue, with wood shavings scattered around.

11. Topic : Serve Prompt: A penguin wearing a small bow tie balancing a tray with a fish platter, ready to serve it at a fancy dinner.

12. Topic : Cook Prompt: A bear in a tiny chef hat flipping pancakes in a pan, with jars of honey around, preparing breakfast in the forest.

13. Topic : Fly-A-Kite Prompt: An elephant holding a vine tied to a leaf-shaped kite, flying it in the air on a breezy day.

14. Topic : Make-A-Potion Prompt: A crow wearing glasses mixing colorful, glowing potions in tiny vials using its beak in a spooky forest.

15. Topic : Polar-Bear-Cut-Cake Prompt: An anime of a polar bear carefully cutting a berry cake.

16. Topic : Raccoon-Sweep-Floor Prompt: An anime of a raccoon sweeping floor with a broom.

17. Topic : Frog-Sing Prompt: An anime of a frog singing with a microphone on hand by a pond.

18. Topic : Beaver-Drink Prompt: An anime of a beaver sipping water from a pond through a hollow stick like a straw.

19. Topic : Rabbit-Set-Table Prompt: An anime of a rabbit setting plates on a rock ’table.’

20. Topic : Squirrel-Carve-Acorn Prompt: An anime of a squirrel carving a design on an acorn using a chisel.

21. Topic : Fox-Pour-Tea Prompt: An anime of a fox pouring tea from a tiny pot into cups.

22. Topic : Cricket-Write-Music-Leaves Prompt: An anime of a cricket scratching musical notes onto a large leaf with a pen.

23. Topic : Beaver-Cut-Pizza Prompt: An anime of a beaver cutting a pizza.

24. Topic : Rabbit-Paint-Mural Prompt: An anime of a rabbit painting a colorful mural on a wall.

25. Topic : Bear-Mop-Floor Prompt: An anime of a bear cleaning its cave floor with a bundle of grass.

E.2 Multi-subject Interactions

This category emphasizes coordinated or reciprocal activity by multiple subjects, usually two animals.

E.2.1 Statistic of topic

The complete frequency list ranges from shared motion and contact to social or cooperative actions. Most listed topics occur once; the original counts remain available for inspecting benchmark coverage.

balancing(1), tug of warring(1), encouraging(1), synchronizing(1), applauding(1), rowing together(1), spinning(1), jumping together(1), supporting(1), swinging(1), lifting(1), hugging(1), carrying(1), holding hands(1), waving(1), flipping(1), celebrating(1), tossing(1), wrestling(1), stacking(1), whispering(1), cheering(1), gliding(1), leaping(1), skating(1), sliding(1), twirling(1), sharing(1), paddling(1), singing(1), drumming(1), dodging(2), shielding(1), twisting(1), stomping(1), kicking(1), leaning(1), pulling(1), pushing(1), vaulting(1), clashing(1), peering(2), rolling(1), hopping(1), shaking(1), bowing(1), building(1), nesting(1), scurrying(1), foraging(1), weaving(1), diving(1), circling(1), peeking(1), nestling(1), peeling(1), sniffing(1), chirping(1), dashing(1), pouncing(1), snuggling(1), flicking(1), surfing(1), linking(1), bumping(1), jumpstarting(1), guiding(1), swing dancing(1), hovering(1), sledding(1), twinkling(1), zipping(1), balancing on one foot(1), batoning(1), beak tapping(1), blending in(1), blocking(1), blooming together(1), blowing kisses(1), bouncing off(1), bridge building(1), celebratory jumping(1), clambering(1), clapping(1), clasping(1), climbing a rope(1), clockwise spinning(1), coat sharing(1), codebreaking(1), coin flipping(1), colliding(1), composing(1), concocting(1), conga lining(1), contemplating(1), cork popping(1), corn husking(1), counting stars(1), crab walking(1), cracking knuckles(1), creating shadows(1), crisscrossing(1), croquet playing(1), dashing through snow(1), daydreaming(1), defying gravity(1), disappearing act(1), docking(1), dodgeballing(1), dolphin surfing(1), double jumping(1), drag racing(1), ducking(1), echoing(1), egg balancing(1), elbow bumping(1), embracing(1), eye winking(1), face painting(1), fan waving(1), fast forwarding(1), feather tickling(1), fence jumping(1), firework launching(1), fishing together(1), flashlight signaling(1), flipping pages(1), fluttering(1), freezing in place(1), frisbee tossing(1), frog leaping(1), fumbling(1), game playing(1), gazing(1), ghost hunting(1), gift exchanging(1), gliding on air(1), glueing(1), goal scoring(1), gondola riding(1), grappling(1), grinning(1), guarding(1), guessing(1), gymnastics(1), hair braiding(1), hand painting(1), hand shaking(1), hand standing(1), harmonizing(1), harnessing wind(1), head bobbing(1), head butting(1), hearing secrets(1), heel clicking(1), hide and seeking(1), hiking(1), hill rolling(1), home run hitting(1), hopping on one foot(1), horseback riding(1), hula hooping(1), ice skating(1), improvising(1), inventing(1), jigsaw puzzling(1), jumping jacks(1), kicking a can(1), kneeling(1), laughing(1), leaf pile jumping(1), leapfrogging(1), letter writing(1), lifting weights(1), light painting(1), limboing(1), line dancing(1), listening to music(1), log rolling(1), looking through binoculars(1), magician acting(1), mapping(1), meditating(1), metal detecting(1), moon watching(1), moth chasing(1), mountain climbing(1), mushroom picking(1), bouncing(1), huddling for warmth(1), jumping rope(1), high fiving(1), dancing(1), lifting together(1), balancing a ball(1), digging together(1), building a nest(1), sharing food(1)

E.2.2 Selected prompts

All ten selected multi-subject prompts are retained, including huddling, jumping rope, high-fiving, lifting food, nest building, and sharing. Some descriptions imply temporal alternation that a single still image cannot unambiguously establish.

1. Topic : Bouncing Prompt: A frog and a grasshopper take turns bouncing across lily pads on a pond.

2. Topic : Huddling-for-Warmth Prompt: A walrus and a polar bear huddle together on an ice floe, sharing warmth.

3. Topic : Jumping-Rope Prompt: A kangaroo and a lemur each hold an end of a vine, hopping over it together in turn.

4. Topic : High-Fiving Prompt: Two monkeys jump up and high-five with their paws, celebrating a successful foraging trip.

5. Topic : Dancing Prompt: Two flamingos perform an elegant dance, mirroring each other’s wing movements in perfect coordination.

6. Topic : Lifting-Together Prompt: Two ants collaborate to lift a large crumb, balancing it carefully as they carry it back to their colony.

7. Topic : Balancing-a-Ball Prompt: Two seals balance a ball on their noses, passing it back and forth in a coordinated game.

8. Topic : Digging-Together Prompt: Two meerkats dig a hole side-by-side, their paws flying in rhythm as they excavate a burrow.

9. Topic : Building-a-Nest Prompt: Two birds bring twigs and leaves to a tree branch, weaving them together to create a shared nest.

10. Topic : Sharing-Food Prompt: Two bears share a large fish, taking turns taking bites while watching out for other animals.

E.3 Compositional Spatial Relationships

Spatial examples combine an abstract structure with unusual constituent objects or materials, alongside a smaller geometric-pattern subset.

E.3.1 Statistic of topic

The published list separates 183 abstract-layout occurrences from 17 geometric-pattern occurrences. It spans familiar arrangements and much broader scientific or abstract terms; the category name does not mean that every prompt has the same geometric difficulty.

• Abstract Layouts: chessboard(1), domino(1), constellation(1), pyramid(1), labyrinth(1), kaleidoscope(1), circuit(1), tetris(1), sundial(1), hourglass(1), compass(1), map(1), blueprint(1), gear(1), vortex(1), tessellation(1), barcode(1), spectrum(1), origami(1), satellite(1), silhouette(1), shadow(1), footprint(1), bridge(2), tunnel(1), stained glass(1), windmill(1), lighthouse(1), mountain range(1), river delta(1), waterfall(1), thunderbolt(1), sand dune(1), cliff(1), canyon(1), volcano(2), coral reef(1), aurora(1), nebula(1), eclipse(1), supernova(1), galaxy(1), comet(1), meteor(1), black hole(1), crystal(1), beehive(1), chess knight(1), rubik’s cube(1), sudoku(1), hieroglyph(1), calligraphy(1), musical note(1), soundwave(1), microchip(1), pixel(1), digital clock(1), keyboard(1), mouse pointer(1), barcode scanner(1), jigsaw(1), metro map(1), circuit board(1), telescope(1), microscope(1), hour hand(1), ice crystal(1), tornado(1), tidal wave(1), flame(1), fog(1), reflection(1), horizon(1), globe(1), water cycle(1), ecosystem(1), double helix(1), electric arc(1), solar flare(1), magnet field(1), pendulum(1), gyroscope(1), whirlpool(1), sand timer(1), prism(1), steam power(1), cogwheel(1), marble rolling(1), beam splitter(1), tesseract(1), mobius strip(1), klein bottle(1), electron cloud(1), time lapse(1), solar system(1), tidal force(1), magnetic levitation(1), hologram(1), lens flare(1), binary code(1), algorithm(1), probability tree(1), data cloud(1), social network(1), venn diagram(1), flowchart(1), decision tree(1), optical fiber(1), cosmic web(1), interstellar map(1), dna strand(1), chromosome(1), protein structure(1), enzyme(1), bacteria colony(1), virus model(1), periodic table(1), crystal lattice(1), liquid drop(1), bubble(1), soap film(1), oil slick(1), lava flow(1), fossil(1), seismograph(1), tsunami(1), weather front(1), storm path(1), thundercloud(1), cloud formation(1), rainforest(1), food chain(1), coral polyp(1), ocean current(1), tide pool(1), glacier(1), iceberg(1), volcano cross section(1), fossilized leaf(1), desert oasis(1), lava lamp(1), windmill blades(1), compass needle(1), sundial marks(1), shadow clock(1), metronome(1), pendulum wave(1), ripple tank(1), icicle drip(1), salt crystal(1), gemstone cut(1), light beam bend(1), fiber optic glow(1), laser beam(1), nebula cluster(1), star map(1), supernova explosion(1), gravitational wave(1), celestial sphere(1), solar eclipse(1), lunar cycle(1), moondust(1), martian canyon(1), space dust(1), cosmic string(1), dark matter(1), quark structure(1), higgs boson(1), neutrino path(1), time dilation(1), tic tac toe(1), table(1), atom(1), forest(1), city(1), tree(1), bookshelf(1), flower(1), island(1), garden(1), mosaic(1),

• Geometric patterns: snowflake(1), spiral(1), ripple(1), waveform(1), parabola(1), arch(1), infinity symbol(1), yin yang(1), mandala(1), fibonacci sequence(1), sphere(1), cone(1), dodecahedron(1), helix(1), triangle(1), zigzag leaves(1), circle(1)

E.3.2 Selected prompts

All 15 selected spatial prompts are retained, including a centered flower circle, food-based tic-tac-toe, a sushi triangle, and landscapes made of candy or cookies. They specify qualitative arrangements without supplying ground-truth target images.

1. Topic : Zigzag-Leaves Prompt: A zigzag pattern made of scattered autumn leaves, creating a path that alternates left and right as it moves forward.

2. Topic : Circle Prompt: A circle of sunflowers with a single, vibrant red rose in the very center, surrounded by the larger yellow blooms.

3. Topic : Tic-Tac-Toe Prompt: A tic-tac-toe composed by tomato and cucumber as the players move, and sticks as the grid.

4. Topic : Table Prompt: A table formed by pretzels stacked together.

5. Topic : Atom Prompt: An atom depicted with orange as the nucleus and blueberries as electrons spinning in circular orbits.

6. Topic : Triangle Prompt: A triangle made of sushi pieces, where each side is formed by a different sushi roll.

7. Topic : Forest Prompt: A forest made from broccoli trees, with animal-shaped cookies as wildlife and a path of cookie crumbs winding through it.

8. Topic : City Prompt: A cityscape made of stacked crackers as buildings, licorice strips as roads.

9. Topic : Tree Prompt: A tree made of a pretzel stick as the trunk, with green gummy leaves and woolen yarn roots branching out.

10. Topic : Bookshelf Prompt: A bookshelf made from colorful candies.

11. Topic : Flower Prompt: A flower made by colorful gummy.

12. Topic : Bridge Prompt: A bridge constructed from graham crackers.

13. Topic : Island Prompt: An island scene with coconut flakes as sand, candy trees on the beach, and blue jelly water as sea.

14. Topic : Volcano Prompt: A volcano built from chocolate, with red jelly spilling as lava and cotton candy smoke billowing from the top.

15. Topic : Garden Prompt: A garden made from crushed cookie soil, flower-shaped candies, and wafer cookie paths winding through it.
Concept demonstration

Open image ↗