At a glance
FALCON learns a new visual concept from one or a few image examples, a sentence identifying the relevant object, and optional sentences relating the concept to known concepts. A pretrained language interface resolves the reference; a meta-learned module then predicts an axis-aligned box in a shared embedding space. The new concept can be reused in downstream visual questions.
The strongest results are on birds and synthetic scenes, where supplied conceptual relations reduce ambiguity. Improvements on natural-image GQA are modest. The system assumes pretrained perception and semantic parsing, generated language templates, and structured concept relations; it does not learn these components or discover the ontology from a single example.
1 Introduction
A single purple cylinder cannot reveal whether a new word names its color or shape. A description can identify the object, while a supplemental relation such as “is a color” constrains the intended property. Figure 1 illustrates this combination across CLEVR, birds, and natural images.
FALCON stores explicit concept embeddings and learns how to infer a new embedding from examples and related concepts. The main contribution is a sample-update-select inference procedure trained over many base-concept tasks. Its learned concepts support subsequent questions and can supply context for learning later concepts.
2 Related Works
The approach extends neuro-symbolic concept learning and concept-relation learning to a meta-learning setting. It also draws on few-shot classifier generation and geometric embeddings. Boxes represent category extent and containment, while object features occupy very small regions in the same space. Figure 2 gives the complete model pipeline.
3 FALCON
Each task supplies a novel concept, paired example images and descriptions, optional supplemental relations, and downstream image-question queries. Training concepts, validation concepts, and test concepts are separated. The system learns an inference procedure on base concepts, then applies it to previously unseen concept identities.
3.1 Visual Representations and Embedding Spaces
Mask R-CNN detects objects, and ResNet-34 extracts object features from the image and masks. Object-pair features support relational concepts in a separate embedding space. GQA uses the dataset’s supplied pretrained detector features.
A concept box is specified by its center and offsets; an object is represented by a tiny box. Box volumes and intersection volumes define conditional membership scores. Containment naturally expresses a narrow category belonging to a broader one. The original definitions below preserve the precise notation, and softplus smoothing makes the geometric scoring easier to optimize.
These geometric relations are an inductive bias. Predicted boxes are not guaranteed to satisfy every supplied relation, because the system learns to infer embeddings rather than enforcing all constraints exactly.
Equations
3.2 Neuro-Symbolic Reasoning
Pretrained semantic parsers turn descriptions and questions into typed programs and supplemental sentences into concept-relation tuples. A fixed differentiable executor uses known concepts to locate the example object, then uses the inferred new concept to answer later questions. Intermediate object sets are soft membership vectors.
This modular interface exposes grounding errors, but an incorrect reference can contaminate the new concept. Parser supervision and the restricted operation vocabulary are part of the setup.
3.3 Learning to Learn Concepts
FALCON samples candidate concept embeddings from a learned prior, updates them using related concepts and visual examples, then selects the candidate assigning the highest likelihood to the observed example objects. The retained equations give the updates and the training objective.
FALCON-G applies two graph networks in sequence: one over concept relations, one over example objects. FALCON-R processes relation/example edges recurrently in randomized order; Appendix A describes its unified implementation. Both receive downstream question-answer loss plus a prior-density term. Their learned updates replace a hand-written geometric constraint solver.
Equations
3.4 Training and Testing Paradigm
Pretraining learns visual representations and base-concept embeddings. Meta-training fixes base embeddings and learns the new-concept inference modules from tasks built around base concepts, selecting models on validation concepts. Meta-testing supplies examples and relations for held-out concepts and measures downstream answers. Appendix B retains the training schedule and the adaptations needed for comparison models.
4 Experiments
Experiments use procedurally generated descriptions and questions to control which concepts and relations are available.
4.1 Dataset
CUB’s 200 bird species are split into 100 base, fifty validation, and fifty test species. Adding higher taxonomic categories yields 211, 83, and 72 concepts. Relations include hypernyms and concepts sharing a hypernym.
CLEVR uses fifteen attribute concepts split into ten base, two validation, and three test concepts, with results averaged over four splits. GQA uses 105 frequent concepts split 89/6/10. Images may be natural, but the descriptions, supplemental sentences, and evaluation questions are generated from annotations and templates rather than unrestricted user language.
4.2 Baseline
CNN+LSTM and MAC process examples, descriptions, and test queries together. Concept-centric comparisons instead infer a reusable concept embedding and share the same parser/executor. NSCL+LSTM combines recurrent encodings; NSCL+GNN initializes from averaged examples and propagates through the relation graph. FALCON adds sampled initialization, learned example updates, and candidate selection. Figure 3 contrasts the designs.
The study compares box, hyperplane, and hypercone representations. Their original scoring formulas are retained. Appendix A clarifies that the one-example box baseline uses an average training-concept box size, despite a main-text sentence suggesting that this baseline cannot operate in that setting.
Equations
4.3 Results on CUB
CUB queries ask whether the bird belongs to the learned category, emphasizing classification rather than complex reasoning. With one example plus relations, Table 1 reports 81.33% for FALCON-G and 79.83 for FALCON-R, compared with 79.53 for NSCL+LSTM and 73.88 for MAC. The main text’s statement that FALCON-R is highest conflicts with this table; this edition follows the reported values.
Without supplemental relations, Table 2 gives 76.37 for FALCON-G and 75.84 for FALCON-R. Figure 4 preserves successes and mistakes. Later ablations examine continuing to learn through previously acquired concepts, reducing the base vocabulary, and withholding related-concept information.
| Model(Type) | QA Accuracy | ||
| CNN+LSTM (E2E) | 51.37 | ||
| MAC (E2E) | 73.88 | ||
| Box | Hyperplane | Hypercone | |
| NSCL+LSTM (C.C.) | 79.53 | 71.71 | 72.48 |
| NSCL+GNN (C.C.) | 78.50 | 67.60 | 72.82 |
| FALCON-G (C.C.) | 81.33 | 74.11 | 68.44 |
| FALCON-R (C.C.) | 79.83 | 72.82 | 69.74 |
| Model(Type) | QA Accuracy | ||
| CNN+LSTM (E2E) | 51.50 | ||
| MAC (E2E) | 73.55 | ||
| Box | Hyperplane | Hypercone | |
| NSCL+LSTM (C.C.) | 72.23 | 63.18 | 68.48 |
| NSCL+GNN (C.C.) | 73.38 | 62.00 | 72.91 |
| FALCON-G (C.C.) | 76.37 | 64.60 | 70.42 |
| FALCON-R (C.C.) | 75.84 | 63.32 | 69.69 |
4.4 Results on CLEVR
On CLEVR, one object often leaves a new attribute ambiguous. Table 3 shows FALCON-G improving from 68.47% with examples alone to 88.40 with supplemental sentences; NSCL+GNN rises from 65.01 to 84.11. The result supports using supplied concept relations to resolve ambiguity.
A separate biased-data test forces the novel concept to co-occur with another novel concept. Appendix Table 9 reports 87.29% for FALCON-G with supplemental sentences, whereas the main narrative states 88.29. The appendix also gives different baseline values; the condensed discussion uses its complete table.
| Model | Example Only | Example+Supp. | |
| CNN+LSTM | 41.92 | 42.69 | 0.77 |
| MAC | 61.81 | 62.05 | 0.24 |
| NSCL+LSTM | 56.82 | 61.35 | 4.53 |
| NSCL+GNN | 65.01 | 84.11 | 19.11 |
| FALCON-G | 68.47 | 88.40 | 19.93 |
| FALCON-R | 68.57 | 87.36 | 18.78 |
4.5 Results on GQA
GQA asks whether a referred object has the new property, using a restricted frequent-concept vocabulary. FALCON-G with boxes scores 55.96%, compared with 55.41 for NSCL+LSTM and 54.99 for MAC. These are small gains, and without supplemental sentences FALCON-R’s 54.60 is close to MAC’s 54.45. Natural-scene complexity remains a substantial limitation.
5 Conclusion
Explicit reusable concepts and learned inference from visual examples plus relational guidance can improve data efficiency in controlled tasks. Their usefulness depends on reliable grounding, supplied language structure, the base vocabulary, and the geometry of the new category. The paper demonstrates downstream question answering, with broader task transfer remaining to be established. The authors acknowledge ONR, CBMM, MIT Quest for Intelligence, and the MIT–IBM AI Lab.
Appendix A Model Details
Appendix A retains the detailed representations, update equations, code, parser operations, and comparison-model implementations.
A.1 Remarks on the overall model structure
The concept-centric design separates learning a reusable object classifier from answering queries. Images and descriptions are compressed into a concept embedding, which avoids reprocessing every learning example for each future question.
A.2 Concept Embedding
Box embeddings use 100 dimensions for centers and another 100 for offsets; hyperplane and hypercone vectors use 512 dimensions. Box smoothing uses temperature 0.2. Thus the representation comparisons also differ in dimensionality and parameterization.
A.3 Prior Distribution
A shared Dirichlet prior samples the position and extent of each box dimension, while vector alternatives use isotropic Gaussian priors. The original sampling formulas are preserved below. The prior parameters are learned through the meta-learning objective, with the prior-density loss coefficient set to one.
Equations
A.4 FALCON-G
Each graph network has two message-passing layers. Relation-specific networks produce messages, elementwise maximum aggregates neighbors, and gated residual updates modify the target embedding. Box updates share small networks across dimensions; vector embeddings use larger fully connected networks. The equations retain the exact center/offset and vector updates.
Equations
A.5 FALCON-R
The appendix combines concept and example edges into one graph, randomly orders the relational tuples, and processes each through a recurrent update implemented as message passing on a two-node graph. Snippet 1 retains the complete published PyTorch-style propagation code. It is source pseudocode, not a newly validated executable implementation.
for j, i, k in G.edges:
split = torch.stack([
*h[j].chunk(2, -1),
*h[i].chunk(2, -1)
], dim=-1)
m[(j, i)] = mlp_1[k](split)
for i in G.nodes:
h_N[i] = torch.stack([
m[(jj, ii)] for (jj, ii) in m if i == ii
]).max(dim=0).values
for i in G.nodes:
o, delta, o_prime, delta_prime = mlp_2(torch.stack([
*h[i].chunk(2, dim=-1),
*h_N[i].unbind(dim=-1)
])).unbind(dim=-1)
h[i] = torch.cat([
center + torch.sigmoid(o) * delta,
offset + torch.sigmoid(o_prime) * delta_prime
], dim=-1)
A.6 Semantic parser
The sequence-to-sequence parser predicts flattened operation/concept pairs, then reconstructs hierarchical programs. It is trained on 10% of the generated data. Table 4 preserves all CLEVR operations; CUB uses only Scene, Filter, and Exist because each image has one bird. Table 5 retains description and query examples across all datasets.
| Operation | Semantics | ||
| Scene | Return all objects in the scene. | ||
| Filter |
| ||
| Relate |
| ||
| AERelate | Filter out the set of objects that have the same attribute value | ||
| as the input object. | |||
| Intersection | Return the intersection of two object sets. | ||
| Union | Return the union of two object sets. | ||
| Query | Query the attribute of the input object. | ||
| AEQuery | Query if two objects have the same attribute value (e.g., color). | ||
| Exist | Query if an object set is empty. | ||
| Count | Return the size of an object set. | ||
| CountLessThan |
| ||
| CountGreaterThan |
| ||
| CountEqual |
|
| Dataset | Input | Output | |||||||||||
|
|
| |||||||||||
|
|
| |||||||||||
|
There is a frigatebird. | Scene() | |||||||||||
|
Is there a frigatebird? |
| |||||||||||
|
The clouds are a white object. | Filter(Scene(),clouds) | |||||||||||
|
|
|
A.7 Neuro-Symbolic Progrm Execution
Execution follows NS-CL with two changes: concept membership uses the selected embedding geometry, and same-attribute tests are represented as learned relational concepts such as has_same_color. Soft object memberships allow downstream answer loss to train the underlying representations. Table 4’s original Exist wording says “empty,” whereas the intended operation tests existence; the table is preserved as published.
A.8 Baseline Implementation
The following implementations distinguish reusable-concept models from comparisons that process the full task context for each query.
CNN+LSTM
CNN+LSTM extracts global ResNet-34 features for the example and query images, encodes concatenated descriptions and questions with a bidirectional LSTM, and predicts the answer through an MLP. It does not explicitly construct a reusable novel-concept classifier.
MAC
MAC concatenates example/query image feature maps and description/query text, then uses its attention-based reasoning network. The image backbone is ResNet-101, and other settings follow the original architecture.
NSCL+LSTM
NSCL+LSTM shares the parser and executor, then separately encodes grounded visual examples and related-concept information with bidirectional LSTMs. Their final states predict the new concept embedding through a linear layer. This contrasts with FALCON’s sequential or graph-based updates to sampled candidates.
NSCL+GNN
NSCL+GNN initializes a concept from normalized mean vectors or the smallest box enclosing its example embeddings, then applies a relation graph network. Without supplemental relations, the single-example box version initializes size from the average training-concept box. This appendix detail explains the numerical box baseline in Table 2.
Appendix B Training Details
Figure 5 preserves the three-stage learning protocol. CLEVR pretraining uses a curriculum over scene size and question complexity. Pretraining runs 50,000 iterations with batch size ten and Adam learning rate 0.0001; meta-training runs 10,000 task iterations with batch size one and learning rate 0.001. Validation occurs every 1,000 iterations, with a tenfold learning-rate reduction after five evaluations without improvement.
For CNN+LSTM and MAC, the novel concept name is replaced by an unknown token during meta-training and meta-testing because those models assume a fixed vocabulary. This is an implementation choice affecting the comparison, not a universal limit of all end-to-end architectures.
Appendix C Experiment Details
Appendix C preserves the data templates, additional result tables, biased-concept examples, graph alternatives, and entailment evaluation.
C.1 The CUB Dataset
Table 6 gives the bird description, taxonomy, and query templates. The generated data include roughly 1,800 descriptions, 54,000 downstream questions, and 11,800 pretraining image-question pairs. Training supplemental sentences randomly omit a related concept with probability 0.2. Standard image resizing, cropping, and horizontal flips augment the training data.
| (a) The descriptive sentence template in CUB. | |
| Template | There is a/an <CONCEPT>. |
| Example | There is a white-eyed vireo. |
| Description | <CONCEPT> is the word representing concept . |
| (b) The supplemental sentence template in CUB. | |
| Template | <CONCEPT-1>, <CONCEPT-2> <CONCEPT-N> are a kind of |
| <HYPERNYM>. | |
| Example | Black-capped vireo, philadelphia vireo, warbling vireo, white-eyed vireo are |
| a kind of vireos. | |
| Description | <CONCEPT-i> are words representing and that share a cohypernym relation |
| with . <HYPERNYM> denotes ’s hypernym. | |
| (c) The test question template in CUB. | |
| Template | Is there a/an <CONCEPT>. |
| Example | Is there a white-eyed vireo. |
| Description | <CONCEPT> is the word representing concept . |
Extension: continual learning.
Continual learning introduces test concepts in increasing taxonomy distance from known concepts, freezing newly learned concepts before using them to support later ones. Table 7 reports 79.67% for FALCON-G with relations versus 75.21 for NSCL+LSTM. This tests ordered expansion of the concept repository; it is not a general benchmark of forgetting under repeated updates to all previous classifiers.
| Model | Performance | ||
| Box | Hyperplane | Hypercone | |
| NSCL+LSTM | 71.31 | 62.45 | 67.50 |
| NSCL+GNN | 72.53 | 61.56 | 71.25 |
| FALCON-G | 75.28 | 64.16 | 68.73 |
| FALCON-R | 75.01 | 62.78 | 67.96 |
C.2 The CLEVR Dataset.
Each CLEVR split holds out two colors for validation and two further colors plus one shape for testing. Descriptions identify an object using only base concepts; test questions contain the novel concept once. Table 8 retains the templates. The protocol generates about 7,500 descriptions, 225,000 downstream image-question pairs, and 402,000 pretraining questions.
| (a) The descriptive sentence template in CLEVR. | |
| Template | There is a <REFEXP>; it is <CONCEPT>. |
| Example | There is a small object to the right of the yellow object; it is a cyan object. |
| Description | <REFEXP> is a referential expression generated by CLEVR-Ref+ |
| Liu et al. (2019). <CONCEPT> is the word for concept . | |
| (b) The supplemental sentence template in CLEVR. | |
| Template | <CONCEPT-1>,<CONCEPT-2><CONCEPT-N> are <ATTRIBUTE>. |
| Example | Cube, sphere and cylinder are shapes. |
| Description | <CONCEPT-i> are words representing and its related concepts . |
| <ATTRIBUTE> is the word for an attribute of a object, one of | |
| shape, color, material and size. | |
| (c) The test question generation template in CLEVR. | |
| Template | <QUEST> |
| Example | Does the small blue thing have the same material as the gray object? |
| Description | <QUEST> are question generated from the original CLEVR paper |
| Johnson et al. (2017a). | |
Learning from biased data.
Figure 6 uses the same purple-cylinder appearance for two different novel concepts. Supplemental sentences distinguish the shape interpretation from the color interpretation. Table 9 reports FALCON-G improving from 69.71 to 87.29%; NSCL+GNN improves from 64.75 to 84.83. The table’s FALCON improvement is smaller than NSCL+GNN’s absolute gain, despite the higher final score.
| Model | Example Only | Example+Supp. | |
| CNN+LSTM | 41.61 | 42.38 | 0.77 |
| MAC | 62.38 | 62.40 | 0.02 |
| NSCL+LSTM | 57.76 | 61.79 | 4.03 |
| NSCL+GNN | 64.75 | 84.83 | 20.08 |
| FALCON-G | 69.71 | 87.29 | 17.57 |
| FALCON-R | 69.70 | 86.21 | 16.50 |
Handling non-prototypical concepts.
Attribute categories such as “color” do not have a single visual prototype. Figure 7 compares connecting a new concept to sibling concepts against introducing the abstract attribute itself as a graph node. The former reaches 88.40%, the latter 71.34; the experiments therefore use sibling links. Table 10 also preserves the natural-image language templates.
| (a) The descriptive sentence template in GQA. | |
| Template | The <CONCEPT-1> (<CONCEPT-2>) (object) is a/an <CONCEPT> (object). |
| Example | The small man is a sitting object. |
| Description | <CONCEPT> is the word for concept . <CONCEPT-i> are other concepts. |
| (b) The supplemental sentence template in GQA. | |
| Template | <CONCEPT-1>,<CONCEPT-2><CONCEPT-N> describes the same |
| property of an object. | |
| Example | Sitting, standing describes the same property of an object. |
| Description | <CONCEPT-i> are words representing and its related concepts . |
| (c) The test question generation template in GQA. | |
| Template | Is the <CONCEPT-1> (<CONCEPT-1>) (object) a/an <CONCEPT> (object)? |
| Example | Is the white woman a sitting object? |
| Description | <CONCEPT> is the word for concept . <CONCEPT-i> are other concepts. |
C.3 The GQA dataset
GQA uses supplied Faster R-CNN object features and generated descriptions and questions. Tables 11(a) and 11(b) retain results with and without relations across all embedding geometries. The source’s stated total and split counts for generated questions do not sum consistently, so they should not be treated as a verified dataset-size audit. The reported accuracies show only modest advantages over the strongest comparisons.
| Model | QA Accuracy | ||
| CNN+LSTM | 48.73 | ||
| MAC | 54.45 | ||
| Box | Hyperplane | Hypercone | |
| NSCL+LSTM | 53.54 | 53.16 | 52.83 |
| NSCL+GNN | 50.15 | 50.74 | 52.03 |
| FALCON-G | 53.89 | 53.24 | 52.22 |
| FALCON-R | 54.60 | 52.86 | 52.37 |
C.4 The connections between the box embedding and our formulation
The geometric motivation and the empirical entailment test are distinct: containment can express category inclusion, but learned updates need not satisfy every relation exactly.
Modeling concept entailments (intuition).
A narrower concept’s box should lie inside its broader hypernym’s box. Same-kind relations, such as two colors, do not have the same simple containment interpretation. The source appendix reverses the containment direction in one explanatory sentence; the interpretation here follows the main text and its conditional-probability definition.
Comparison with other box embedding spaces (empirical support).
The authors score every concept pair for entailment and vary a threshold to compute ROC AUC. The original classifier and scoring formulas are retained. Table 12 reports 0.855 for boxes, 0.638 for hyperplanes, and 0.585 for hypercones. The alternatives use symmetric similarity measures that are poorly suited to directional entailment, so the comparison favors the box representation’s intended geometry.
Equations
Appendix D Ablation Details
Two ablations vary the base-concept vocabulary and the amount of relational context.
D.1 The Effect of the Number of Base Concepts
Reducing the training species from 100 to fifty changes the base vocabulary from 211 to 130 concepts and also changes validation/test composition. FALCON-G accuracy falls from 81.33 to 76.32%; Table 13 retains the comparison models. This supports the value of prior concepts, while the changed evaluation split limits an interpretation as a pure vocabulary-size intervention.
D.2 The Effect of The Number of Related Concepts in Supplemental Sentences
Table 14 gives FALCON-G accuracies of 76.37, 80.20, 80.78, 81.20, and 81.33% with 0, 25, 50, 75, and 100% of related-concept information. Most of the gain comes from the first quarter of relations.
Figure 8 is retained unchanged, but its MAC and FALCON-G legend labels conflict with the corresponding series in Table 14: the highest curve matches FALCON-G’s tabulated values. The numerical discussion follows the table.
| % of all related concepts | FALCON-G | NSCL+GNN | MAC |
| 0 | 76.37 | 73.38 | 73.55 |
| 25 | 80.20 | 77.16 | 73.94 |
| 50 | 80.78 | 76.93 | 74.13 |
| 75 | 81.20 | 77.77 | 74.23 |
| 100 | 81.33 | 78.50 | 73.88 |
Appendix E Failure Mode Details
The failure analysis separates object detection, language parsing, and concept-grounding errors. A correct final answer can coexist with an incorrect intermediate representation.
E.1 Failure mode in detection module
Table 15 preserves missed-object and false-positive examples. Missing a small or occluded object can redirect the description to the wrong instance; a spurious mask may have no effect if execution ignores it. Errors in the chosen training example can alter the learned concept and affect later questions.
E.2 Failure mode in semantic parser
Table 16 retains syntactically invalid programs repaired by heuristics and syntactically valid but semantically wrong programs. A wrong program can still produce the expected answer when scene properties are redundant. The implementation has no general semantic-error recovery; visual-feedback parser refinement is proposed as future work.
E.3 Failure mode in concept learning module
Figure 9 preserves two concept-grounding failures: an overly broad spatial relation chooses the wrong example object, while an overly narrow newly learned shape excludes the correct test object. Such errors can arise in base, relational, or newly learned concepts and propagate through otherwise valid programs.
|
|
Descriptive sentence:The blue matte thing to the right of the big cyan object is a cube. Expected example object: The blue cube around the upper-right corner of the cyan cube. Predicted example object: The purple metal cylinder. ✗ Detection error: The blue cube is not detected by the detection module. |
|
|
Descriptive sentence: The white small object is a plate. Expected example object: The white plate. Predicted example object: The lower side of the plate. ✗ Detection error: The entirety of the white plate is not detected as one object |
|
|
Descriptive sentence:The metallic thing that is right of the tiny sphere is a cube. Expected example object: The green cube. Predicted example object: The green cube. ✓ Detection error: The union of the blue cube and the gray cube is detected as one object by the detection module. |
|
|
Descriptive sentence:The large long object is a silver object. Expected example object: The long knife. Predicted example object: The long knife. ✓ Detection error: Shadows of the knife and the apple are detected as objects by the detection module. |
![[Uncaptioned image]](https://storage.literature.yihao.one/media/sha256/54/54848f34726562c38d8b9cb7fdf7c373959cc863053d0714e24743e1ee6d7a32.webp)
| Text |
| ||
| Expected Program | CountEqual(Filter(Filter(Relate( | ||
| Filter(Filter(Scene(), small), ball), | |||
| front), small), ball), | |||
| Filter(Filter(Scene(), big), cyan)) | |||
| Predicted Program | CountEqual(Filter(Filter(Relate( | ||
| Filter(Filter(Scene(), small), ball), | |||
| front), small), Filter), | |||
| Filter(Filter(Scene(), big), cyan)) | |||
| Fixed Program | CountEqual(Filter(Relate( | ||
| Filter(Filter(Scene(), small), ball), | |||
| front), small), | |||
| Filter(Filter(Scene(), big), cyan)) |
| Text |
| ||||
| Expected Program | Exists(Filter(Filter(Relate( | ||||
| Filter(Filter(Scene(), big), brown), | |||||
| left), metal), cyan)) | |||||
| Predicted Program | Exists(Filter(Filter(Relate( | ||||
| Filter(Filter(Scene(), big), purple), | |||||
| left), metal), cyan)) | |||||
| |||||
| Text |
| ||||
| Expected Program | Exists(Filter(Filter(Relate( | ||||
| Filter(Filter(Scene(), big), brown), | |||||
| right), big), cyan)) | |||||
| Predicted Program | Exists(Filter(Filter(Relate( | ||||
| Filter(Filter(Scene(), big), purple), | |||||
| right), big), cyan)) | |||||
| |||||

Descriptive sentence: The shiny block that are behind small rubber thing is a red object.
Expected example object: The metallic red cube.
Predicted example object: The metallic grey cube.
Explanation: The relational concept behind is not accurate enough; it is so extensive that even the grey cube is behind the yellow cube.

Question: How many red cylinders are to the right of the blue sphere?
Expected answer: 1. (The red cylinder)
Predicted answer: 0.
Explanation: The meta-learned concept cylinder is not accurate enough; it is too exclusive that even the red cylinder is not considered a cylinder.