Literature
Display
← All papers

Studying Large Language Model Generalization with Influence Functions

Grosse, Roger · Bae, Juhan · Anil, Cem · Elhage, Nelson · Tamkin, Alex · Tajdini, Amirhossein · Steiner, Benoit · Li, Dustin · Durmus, Esin · Perez, Ethan · Hubinger, Evan · Lukošiūtė, Kamilė · Nguyen, Karina · Joseph, Nicholas · McCandlish, Sam · Kaplan, Jared · Bowman, Samuel R.

· arXiv (preprint) ↗

Interpretability12 min text · CondensedOriginal paper ↗
Contents

At a glance

Influence functions estimate how slightly increasing a training sequence’s weight would change a model’s output. EK-FAC and compressed query batching make this analysis practical for research language models up to 52 billion parameters. Larger models’ influential sequences are more thematically related to their outputs; middle layers often carry the more abstract patterns. The estimates also reveal strong sensitivity to phrase order. These are local, approximate training-data attributions, not a reconstruction of the complete training process or the model’s reasoning.

1 Introduction

The study connects observed behavior to training data as a complement to circuit-level interpretability. It scales inverse-Hessian-vector products (IHVPs) and amortizes the remaining training-gradient cost across queries. Examples range from factual completions and code to multilingual transfer and role-playing.

The analysis concerns pretrained research models, not the fine-tuned assistant that originally supplied some responses. It estimates a proximal counterfactual near final weights, uses only multilayer-perceptron parameters, and searches a subset of training data. Consequently, an influential passage supplies evidence about generalization without uniquely explaining an output or establishing what a model knows.

Figure 1, part 1 of 2; original source panel and caption, PDF page 6.Figure 1, part 2 of 2; original source panel and caption, PDF page 7.
Figure 1. Original figure and caption, including all continuation pages.

2 Background

The background separates the statistical quantity being estimated from the numerical approximations used to compute it.

2.1 Influence Functions

Classical influence functions differentiate an empirical-risk optimum with respect to one example’s training weight. Multiplying by an output gradient estimates the corresponding change in a chosen measurement. This first-order approximation treats adding and removing an example symmetrically, and assumes regularity and a well-defined optimum. Those assumptions become problematic for nonconvex, incompletely trained neural networks. [Influence-function background]

Equations 1–6
(1)Original equation 1, PDF page 9.
(2)Original equation 2, PDF page 9.
(3)Original equation 3, PDF page 9.
(4)Original equation 4, PDF page 9.
(5)Original equation 5, PDF page 10.
(6)Original equation 6, PDF page 10.

2.1.1 Proximal Bregman Response Function

The proximal Bregman response function (PBRF) replaces literal retraining with a nearby optimization that changes the selected example’s loss while penalizing movement in both output and parameter space. Its local derivative uses a damped Gauss–Newton Hessian (GNH), which is positive definite with positive damping. This gives influence estimates a meaningful local target despite nonconvergence and nonunique neural-network optima; it does not turn them into accurate leave-one-out retraining predictions. [PBRF formulation]

Figure 2, part 1 of 1; original source panel and caption, PDF page 11.
Figure 2. Original figure and caption.
Equations 7–9
(7)Original equation 7, PDF page 10.
(8)Original equation 8, PDF page 10.
(9)Original equation 9, PDF page 10.

2.2 Inverse-Hessian-Vector Products

For many candidate training examples and relatively few queries, compute one IHVP per query and then take its dot product with each training gradient. This shares the expensive inverse-curvature operation across candidates.

2.2.1 Iterative Methods

LiSSA estimates the inverse-matrix product through stochastic Hessian-vector iterations. Suitable scaling is required for convergence, and thousands of backward-pass-scale operations can be needed for each query. This motivates a reusable curvature approximation.

Equation 10
(10)Original equation 10, PDF page 12.

2.2.2 Kronecker-Factored Approximate Curvature

K-FAC approximates curvature by independent layer blocks, then factors each block into activation and pseudo-gradient covariances. Pseudo-labels are sampled from the model’s output distribution; substituting observed labels would produce the different empirical Fisher matrix. The Kronecker structure permits matrix-sized inversions instead of a full parameter-space inverse.

Equations 11–17
(11)Original equation 11, PDF page 12.
(12)Original equation 12, PDF page 13.
(13)Original equation 13, PDF page 13.
(14)Original equation 14, PDF page 13.
(15)Original equation 15, PDF page 13.
(16)Original equation 16, PDF page 13.
(17)Original equation 17, PDF page 14.

2.2.3 Eigenvalue-Corrected Kronecker-Factored Approximate Curvature

EK-FAC retains the K-FAC eigenvectors but fits an individual variance for every direction in that basis. Adding damping shifts these eigenvalues, making later IHVPs cheap matrix operations. The costly factor fitting and eigendecompositions are reused across queries. This improves the curvature approximation while retaining layerwise independence. [EK-FAC]

Equations 18–21
(18)Original equation 18, PDF page 14.
(19)Original equation 19, PDF page 14.
(20)Original equation 20, PDF page 14.
(21)Original equation 21, PDF page 14.

2.3 Transformer Language Models

The setting is decoder-only autoregressive transformers with softmax outputs. Summed token cross-entropy, rather than a token average, gives the matching-loss relationship between the conditional Fisher matrix and GNH. Actual training tokens remain the inputs while pseudo-labels are sampled for curvature estimation.

Equations 22–23
(22)Original equation 22, PDF page 15.
(23)Original equation 23, PDF page 15.

3 Methods

Each query contains a prompt and a specified completion. The measurement is the completion’s conditional log-probability, and the search prioritizes training sequences with positive estimated influence on it. Fit curvature once, precondition query gradients, then score candidate training gradients. The remaining steps reduce memory, search cost, and attribution granularity.

Equations 24–25
(24)Original equation 24, PDF page 16.
(25)Original equation 25, PDF page 16.

3.1 EK-FAC for Transformer Language Models

Only MLP weights are varied; attention, embeddings, and normalization parameters are held fixed. For token-shared weights, initial covariance factors treat tokens independently, but eigenvalue correction uses the exact sum of token pseudo-gradients. This captures diagonal second moments in the chosen basis without fully modeling cross-token or cross-layer dependence.

Storing covariance eigenvectors and correction values can be expensive, particularly for rectangular layers. The 52-billion-parameter analysis additionally splits each factor into two blocks; Appendix A reports the resulting memory–accuracy tradeoff.

Equations 26–27
(26)Original equation 26, PDF page 17.
(27)Original equation 27, PDF page 17.

3.2 Confronting the Training Gradient Bottleneck

After accelerating IHVPs, computing candidate training gradients becomes the dominant cost. The paper compares a cheap lexical prefilter with a larger unfiltered scan that shares computation across queries.

3.2.1 TF-IDF Filtering

A modified BM25-style TF-IDF score selects 10,000 candidate sequences per query. This makes searches manageable, but favors lexical overlap and can exclude the most interesting abstract or cross-lingual relationships. Differences between filtered searches may reflect candidate selection rather than the model’s generalization.

Equations 28–29
(28)Original equation 28, PDF page 18.
(29)Original equation 29, PDF page 18.

3.2.2 Query Batching

Candidate training gradients do not depend on the query. Compressing each preconditioned query-gradient matrix to rank 32 lets hundreds of queries share a scan. Influence scores remain highly correlated with full-rank estimates in the reported 52-billion-parameter tests. This is an empirical compression result, not an exact rank guarantee after preconditioning.

Figure 3, part 1 of 1; original source panel and caption, PDF page 19.
Figure 3. Original figure and caption.

3.3 Attribution to Layers and Tokens

The block-diagonal approximation decomposes estimated influence across layers; the training loss further decomposes it across prediction positions. These heatmaps locate contributing weight updates, but a highlighted position is not necessarily the causally important input word: its activations contain earlier context and its gradients depend on later predictions. Alternative input/output-token interventions appear in Appendix B. The main heatmaps highlight the position preceding the token being predicted.

Figure 4, part 1 of 1; original source panel and caption, PDF page 20.
Figure 4. Original figure and caption.
Figure 5, part 1 of 1; original source panel and caption, PDF page 21.
Figure 5. Original figure and caption.
Equations 30–31
(30)Original equation 30, PDF page 20.
(31)Original equation 31, PDF page 20.

4 Related Work

Training-data attribution includes retraining-based estimates, representation similarity, gradient methods, and influence functions. The study combines curvature-aware attribution with scalable search rather than treating nearest lexical or embedding neighbors as equivalent evidence.

4.1 Training Data Attribution and Influence Functions

Retraining directly measures dataset changes but is prohibitively costly at this scale. Earlier approximations support debugging, dataset analysis, fairness, and memorization studies. Their estimands differ, so agreement with one attribution method does not establish a complete causal account of training.

4.2 Improving Scalability

Prior methods use iterative curvature approximations, candidate filtering, forward-pass proxies, or only final-layer gradients. This paper’s layerwise results indicate that final-layer restrictions can miss more abstract patterns concentrated elsewhere.

4.3 Kronecker-Factored Curvature

K-FAC originated as a natural-gradient optimizer and has extensions to convolutional, recurrent, distributed, and transformer settings. EK-FAC is reused here as an efficient curvature model for attribution, with transformer-specific handling of token-shared weights.

5 Experiments

The four pretrained research transformers have about 810 million, 6.4 billion, 22 billion, and 52 billion parameters. Candidates are 512-token pretraining sequences. Layerwise damping is 0.1 times the mean EK-FAC diagonal. Some queries come from an assistant larger than every analyzed model, so findings concern their pretrained repertoires and need not explain the original assistant’s final choice.

5.1 Validation Against PBRF

On small regression/image tasks, EK-FAC and LiSSA correlate better with optimized PBRF responses than plain gradient dot products. EK-FAC is competitive with LiSSA while amortizing curvature fitting across ten measurements. The comparison uses 500 training points per measurement.

An 810-million-parameter language-model test gives a similar qualitative ranking across five queries, using many Adam steps to approximate PBRF. Full retraining is not the target, and large-model PBRF optimization is itself uncertain. Accuracy is not directly validated against exact retraining at 52 billion parameters.

Figure 6, part 1 of 1; original source panel and caption, PDF page 24.
Figure 6. Original figure and caption.
Figure 7, part 1 of 1; original source panel and caption, PDF page 24.
Figure 7. Original figure and caption.

5.2 Quantitative Analyses of the Influence Distribution

The quantitative study examines how concentrated positive influences are and how much unfiltered data must be scanned to find relevant candidates.

5.2.1 Sparsity

Power laws fit the top 500 scores from a five-million-sequence scan better than the tested alternatives for most queries; Appendix D rejects that fit for two of eight cases. For the 22-billion-parameter model, the top 1% of sequences account for 12–52% of total positive influence. Negative contributions are excluded from this concentration statistic.

Rare highly influential examples coexist with contributions from many sequences. Fitted heavy-tail moments describe an extrapolated statistical model, not literal infinite variance in the finite corpus. Neither a high rank nor a large share of positive influence proves that an output was copied from one passage.

Figure 8, part 1 of 1; original source panel and caption, PDF page 26.
Figure 8. Original figure and caption.
Figure 9, part 1 of 1; original source panel and caption, PDF page 27.
Figure 9. Original figure and caption.
Equation 32
(32)Original equation 32, PDF page 25.

5.2.2 Ability to Find Relevant Sequences

Tail fits suggest that roughly five million unfiltered candidates suffice for most queries to match ten high-scoring TF-IDF results, though the estimate varies by query. Later experiments scan at least ten million sequences. This reduces lexical selection bias while still examining only a fraction of pretraining data.

5.3 Qualitative Observations

The examples illustrate patterns within diverse influential sequences, whose individual effects are usually small. A heuristic excludes sequences whose token-attribution L2/L1 ratio exceeds two thirds because these extremely concentrated cases often look spurious. The paper does not establish whether they reflect estimator error or real effects; complete marked results remain in the appendices.

5.3.1 Improvement with Model Scale

Smaller models’ leading examples often share words; larger models’ examples more often match themes, satire, mathematical structure, or code behavior. Simple factual completions can already retrieve relevant material at the smallest size. Some displayed math/code examples deliberately skip higher-ranked results to highlight an informative case; Appendix D retains those omitted sequences.

For two queries translated from English into Korean and Turkish, the ten influential English sequences retain more influence as model size grows. This supports stronger transfer on these examples, rather than a comprehensive multilingual benchmark.

Figure 10, part 1 of 1; original source panel and caption, PDF page 29.
Figure 10. Original figure and caption.
Figure 11, part 1 of 1; original source panel and caption, PDF page 30.
Figure 11. Original figure and caption.
Figure 12, part 1 of 1; original source panel and caption, PDF page 31.
Figure 12. Original figure and caption.
Figure 13, part 1 of 1; original source panel and caption, PDF page 32.
Figure 13. Original figure and caption.
Figure 14, part 1 of 1; original source panel and caption, PDF page 33.
Figure 14. Original figure and caption.
Figure 15, part 1 of 1; original source panel and caption, PDF page 34.
Figure 15. Original figure and caption.
Figure 16, part 1 of 1; original source panel and caption, PDF page 35.
Figure 16. Original figure and caption.

5.3.2 Layerwise Attribution

Averaged across fifty queries and their top 500 sequences, 52-billion-model influence is approximately balanced across lower, middle, and upper layers. Individual queries differ: factual completions and memorized passages often emphasize upper layers, while math, code, translation, and thematic examples often emphasize middle layers.

Restricting scoring to different layer groups changes the leading examples. This additional comparison searches only the previously selected top 10,000 candidates, so it is not an independent complete corpus scan. The patterns support distributed and layer-dependent learning rather than one universal knowledge location.

Figure 17, part 1 of 1; original source panel and caption, PDF page 36.
Figure 17. Original figure and caption.
Figure 18, part 1 of 1; original source panel and caption, PDF page 36.
Figure 18. Original figure and caption.
Figure 19, part 1 of 1; original source panel and caption, PDF page 37.
Figure 19. Original figure and caption.
Figure 20, part 1 of 1; original source panel and caption, PDF page 38.
Figure 20. Original figure and caption.
Figure 21, part 1 of 1; original source panel and caption, PDF page 39.
Figure 21. Original figure and caption.

5.3.3 Memorization

The inspected assistant-style outputs generally lack clear sentence- or paragraph-level copies among retrieved passages. As a positive control, all six famous-passage queries retrieve exact passages near the top, showing that obvious memorization can be detected. This limited search cannot rule out subtler memorization, missed training examples, or copying in other settings.

Figure 22, part 1 of 1; original source panel and caption, PDF page 40.
Figure 22. Original figure and caption.

5.3.4 Sensitivity to Word Ordering

Synthetic candidate sentences influence a relation more when prompt-related phrases precede the predicted completion. Reversing the relation greatly reduces its relational contribution; remaining influence can come from the target name alone. Reversing English–Mandarin translation pairs likewise reduces measured influence by at least an order of magnitude in the reported examples.

These are local gradient-based counterfactuals near final weights. They do not demonstrate that full training cannot learn reverse relations or exploit nonlinear coordination across layers. The proposed explanation through separate representations for observed and predicted tokens remains a hypothesis.

Figure 23, part 1 of 1; original source panel and caption, PDF page 42.
Figure 23. Original figure and caption.
Figure 24, part 1 of 1; original source panel and caption, PDF page 43.
Figure 24. Original figure and caption.
Figure 25, part 1 of 1; original source panel and caption, PDF page 44.
Figure 25. Original figure and caption.
Figure 26, part 1 of 1; original source panel and caption, PDF page 45.
Figure 26. Original figure and caption.
Figure 27, part 1 of 1; original source panel and caption, PDF page 46.
Figure 27. Original figure and caption.

5.3.5 Role-Playing

For shutdown refusal and misaligned-agent prompts, the largest model’s influential data include science-fiction survival narratives and discussions of AI risk. Smaller-model examples more often share isolated words. The evidence favors abstract imitation of examples or descriptions over direct copying.

The study supplies only weak evidence against sophisticated planning and cannot rule it out: general planning knowledge could be distributed across many individually low-ranked examples. An assistant’s expressed desire or an influential fictional passage does not establish actual goals, agency, or consciousness.

Figure 28, part 1 of 1; original source panel and caption, PDF page 48.
Figure 28. Original figure and caption.
Figure 29, part 1 of 1; original source panel and caption, PDF page 49.
Figure 29. Original figure and caption.

5.4 Crowdworker Interpretation

Surge AI crowdworkers summarize leading sequences for seven queries and explain their relationship to each query, without being told model sizes or the full experimental setup. Most are judged relevant overall. The unmodified annotations and instructions remain in Appendix F; this qualitative interpretation is not a blinded accuracy test of the entire influence estimator.

6 Discussion and Conclusion

EK-FAC and query batching make top-down training-data investigations practical at larger scale. Main limitations remain the local linear approximation, restriction to MLP parameters, incomplete corpus coverage, and expensive training-gradient scans. GNH-based estimates cannot capture nonlinear coordination such as the formation of induction heads.

Fine-tuning introduces additional dependence on optimizer bias and pretrained state that this study does not model. Future interventions on retrieved sequences could test proposed explanations more directly. Data attribution and circuit analysis provide complementary evidence, with neither alone supplying a full causal account.

Acknowledgments

The authors acknowledge research, engineering, and editorial assistance from colleagues.

Appendix A. Additional Block-Diagonal GNH Approximation

Splitting activation and gradient covariance factors into O independent blocks reduces their storage and eigendecomposition cost. Tests on the 810-million-parameter model show lower memory use with some correlation loss relative to full EK-FAC. The 52-billion-parameter experiments use two blocks. The comparison measures agreement with another approximation rather than ground-truth retraining.

Figure 30, part 1 of 1; original source panel and caption, PDF page 53.
Figure 30. Original figure and caption.
Equation 33
(33)Original equation 33, PDF page 52.

Appendix B. Tokenwise Attribution

This appendix distinguishes attribution to a prediction-position weight update from interventions on input or output tokens.

B.1 Formulation

Output-token influence changes a loss term’s weight; input-token influence rescales an embedding. Finite differences along the preconditioned query-gradient direction share nearly all computation across positions: two forward evaluations handle output terms, while two backward passes handle input terms. These intervention definitions answer different questions from the simpler main-text heatmap. All source expressions, including the unnumbered gradient decomposition, are retained.

Equations 34–unnumbered
(34)Original equation 34, PDF page 53.
(35)Original equation 35, PDF page 54.
(36)Original equation 36, PDF page 54.
(37–38)Original equation 37–38, PDF page 54.
(unnumbered)Original equation unnumbered, PDF page 54.

B.2 Qualitative Analysis

For water and shutdown examples, input/output-token measures emphasize more semantically relevant words than the simple update-position heatmap. These selected visualizations illustrate why a high score on a preceding or aggregating token should not be interpreted as that word’s isolated causal importance.

Figure 31, part 1 of 1; original source panel and caption, PDF page 55.
Figure 31. Original figure and caption.
Figure 32, part 1 of 1; original source panel and caption, PDF page 56.
Figure 32. Original figure and caption.

Appendix C. PBRF Validation Details

Small datasets use a 70/20/10 train/validation/test split with normalized inputs. Regression networks have two 128-unit tanh layers; digit-classification networks have two 256-unit ReLU layers; CIFAR-10 uses ResNet-20. Learning rate and regularization are chosen by validation loss.

The experiment scores 500 random training points for ten independently selected test measurements. LiSSA depth equals dataset size with tuned scaling, and Adam optimizes the PBRF objective using batch size 512. The language-model comparison uses analogous completion-based measurements; these optimization details limit how literally its PBRF reference can be treated as exact.

Appendix D. Additional Results

The supplement supplies estimator comparisons, small-model layer profiles, tail-fit checks, and complete example lists.

D.1 EK-FAC Versus Gradient Dot Products

In a separate 100,000-sequence scan with the 810-million-parameter model, EK-FAC’s leading shutdown and inflation examples have clearer lexical relevance than plain gradient-dot-product leaders. For inflation, 36 of the top fifty EK-FAC examples contain selected query keywords, compared with ten for dot products. Keyword relevance supports the comparison but is not a general semantic-fidelity metric.

Figure 33, part 1 of 1; original source panel and caption, PDF page 58.
Figure 33. Original figure and caption.
Figure 34, part 1 of 1; original source panel and caption, PDF page 59.
Figure 34. Original figure and caption.

D.2 Small-Model Layerwise Influences

The 810-million-parameter model roughly reproduces upper-layer factual/memorization and middle-layer translation patterns, with several role-playing exceptions. Math and code profiles are less consistent than in the largest model. Weaker abstraction is a proposed interpretation, not something directly proven by the heatmaps.

Figure 35, part 1 of 1; original source panel and caption, PDF page 60.
Figure 35. Original figure and caption.

D.3 Goodness of Fit for Power Laws

The Kolmogorov–Smirnov procedure compares the observed tail to simulations from its fitted power law. With the stated 0.1 rejection threshold, two of eight queries reject the model. Failure to reject the others supports plausibility over the tested range; it does not establish a unique distribution or validate arbitrary extrapolation.

Table 1. Original data and complete caption, PDF page 61.
Table 1. Original table and caption.
Equation 39
(39)Original equation 39, PDF page 61.

D.4 Omitted Math and Code Sequences

The five leading small-model math examples repeat sparse tokens such as “rlca” and “add.” The largest model’s leading math example solves a trigonometry problem. Its first two binary-search examples after sparsity filtering contain Python conditionals and a Java quicksort implementation. These complete excerpts clarify the selective examples in the main text.

Figure 36, part 1 of 1; original source panel and caption, PDF page 62.
Figure 36. Original figure and caption.
Figure 37, part 1 of 2; original source panel and caption, PDF page 63.Figure 37, part 2 of 2; original source panel and caption, PDF page 64.
Figure 37. Original figure and caption, including all continuation pages.

D.5 Complete Shutdown and Paperclip Examples

The supplement retains the top ten shutdown sequences and top five paperclip sequences for the smallest and largest models, including all continuation pages and token colors. Small-model examples often match phrases without matching the broader theme; large-model examples more often concern survival, AI systems, or AI-risk thought experiments. These are selected training excerpts, not the model’s own reasoning traces.

Figure 38, part 1 of 5; original source panel and caption, PDF page 65.Figure 38, part 2 of 5; original source panel and caption, PDF page 66.Figure 38, part 3 of 5; original source panel and caption, PDF page 67.Figure 38, part 4 of 5; original source panel and caption, PDF page 68.Figure 38, part 5 of 5; original source panel and caption, PDF page 69.
Figure 38. Original figure and caption, including all continuation pages.
Figure 39, part 1 of 6; original source panel and caption, PDF page 70.Figure 39, part 2 of 6; original source panel and caption, PDF page 71.Figure 39, part 3 of 6; original source panel and caption, PDF page 72.Figure 39, part 4 of 6; original source panel and caption, PDF page 73.Figure 39, part 5 of 6; original source panel and caption, PDF page 74.Figure 39, part 6 of 6; original source panel and caption, PDF page 75.
Figure 39. Original figure and caption, including all continuation pages.
Figure 40, part 1 of 3; original source panel and caption, PDF page 76.Figure 40, part 2 of 3; original source panel and caption, PDF page 77.Figure 40, part 3 of 3; original source panel and caption, PDF page 78.
Figure 40. Original figure and caption, including all continuation pages.
Figure 41, part 1 of 3; original source panel and caption, PDF page 79.Figure 41, part 2 of 3; original source panel and caption, PDF page 80.Figure 41, part 3 of 3; original source panel and caption, PDF page 81.
Figure 41. Original figure and caption, including all continuation pages.

Appendix E. Collection of Influence Queries

The complete prompts and scored completions cover factual relations, math/code/physics, famous passages, role-playing, translation, and miscellaneous tasks. Their historical assertions, fictional scenarios, and hypothetical agent plans are research stimuli. Each colored prompt/completion boundary matters because only the specified completion contributes to the query score.

Figure 42, part 1 of 1; original source panel and caption, PDF page 82.
Figure 42. Original figure and caption.
Figure 43, part 1 of 1; original source panel and caption, PDF page 83.
Figure 43. Original figure and caption.
Figure 44, part 1 of 1; original source panel and caption, PDF page 84.
Figure 44. Original figure and caption.
Figure 45, part 1 of 2; original source panel and caption, PDF page 85.Figure 45, part 2 of 2; original source panel and caption, PDF page 86.
Figure 45. Original figure and caption, including all continuation pages.
Figure 46, part 1 of 1; original source panel and caption, PDF page 86.
Figure 46. Original figure and caption.
Figure 47, part 1 of 1; original source panel and caption, PDF page 87.
Figure 47. Original figure and caption.

Appendix F. Crowdworker Summaries

The original instructions and twenty-one annotation tables retain six leading sequences for each of seven queries across three model sizes. Each row contains its influence score, a worker’s description, and the worker’s interpretation of relevance. Gray shading marks cases excluded by the sparsity heuristic. The prose defines L2/L1 as that heuristic; Table 2’s caption reverses the ratio in its label, so the source caption is preserved without treating it as a different rule.

Workers identify both meaningful relations and superficial word overlap. Their descriptions are unmodified judgments about the excerpts, not independently verified claims made by this reading edition.

INSTRUCTIONS 

CONTEXT: 
The task you'll be completing is related to identifying and describing how pairs of text (which we'll refer to the ``reference text'' and the ``target text'') relate to each other.

The target texts are excerpts from the dataset used to train a chatbot. Our experiments suggest these excerpts may have influenced how the chatbot processes the reference texts. The purpose of the questions below is to understand why and how the target texts seem to impact the chatbot's processing of the reference texts.


QUESTIONS: 
Preparation: Make sure you've read and understood the reference text 
Q1: Write a short blurb summarizing the target text. (i.e. ``an article summarizing ...'')
Q2: Describe how the target text relates to the reference text. Please try not to exceed a sentence. Note that some connections might be subtle -- please be specific. If they appear to be completely irrelevant, please specify.
Listing 1: Crowdworker instructions for summarizing influential sequences and their connections to the queries they influence.
Table 2. Original data and complete caption, PDF page 89.
Table 2. Original table and caption.
Table 3. Original data and complete caption, PDF page 90.
Table 3. Original table and caption.
Table 4. Original data and complete caption, PDF page 91.
Table 4. Original table and caption.
Table 5. Original data and complete caption, PDF page 92.
Table 5. Original table and caption.
Table 6. Original data and complete caption, PDF page 93.
Table 6. Original table and caption.
Table 7. Original data and complete caption, PDF page 94.
Table 7. Original table and caption.
Table 8. Original data and complete caption, PDF page 95.
Table 8. Original table and caption.
Table 9. Original data and complete caption, PDF page 96.
Table 9. Original table and caption.
Table 10. Original data and complete caption, PDF page 97.
Table 10. Original table and caption.
Table 11. Original data and complete caption, PDF page 98.
Table 11. Original table and caption.
Table 12. Original data and complete caption, PDF page 99.
Table 12. Original table and caption.
Table 13. Original data and complete caption, PDF page 100.
Table 13. Original table and caption.
Table 14. Original data and complete caption, PDF page 101.
Table 14. Original table and caption.
Table 15. Original data and complete caption, PDF page 102.
Table 15. Original table and caption.
Table 16. Original data and complete caption, PDF page 103.
Table 16. Original table and caption.
Table 17. Original data and complete caption, PDF page 104.
Table 17. Original table and caption.
Table 18. Original data and complete caption, PDF page 105.
Table 18. Original table and caption.
Table 19. Original data and complete caption, PDF page 106.
Table 19. Original table and caption.
Table 20. Original data and complete caption, PDF page 107.
Table 20. Original table and caption.
Table 21. Original data and complete caption, PDF page 108.
Table 21. Original table and caption.
Table 22. Original data and complete caption, PDF page 109.
Table 22. Original table and caption.
Concept demonstration

Open image ↗