Literature
Display
← All papers

Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering

Zikui Cai · Kaushal Janga · Tan Dat Dao · Seungjae Lee · Shivin Dass · Mingyo Seo · Kaiyu Yue · Mintong Kang · Nandhu Pillai · Monte Hoover · Aadi Palnitkar · Ruchit Rawal · Ruijie Zheng · Bo Li · Yuke Zhu · Roberto Martín-Martín · Tom Goldstein · Furong Huang

Preprint posted · arXiv (preprint) ↗

Robotics8 min text · CondensedOriginal paper ↗
Contents

Reading guide

This edition covers all eight pages, six complete figures, both exact tables, nine equations and twenty-seven references. The paper asks whether an existing EQA agent can reuse evidence from earlier questions in the same scene without retraining. Its strongest result is a large within-agent gain for 3D-Mem. Comparisons across four complete systems support a useful design hypothesis; they do not prove that 3 D geometry is the only architecture capable of sequential memory.

At a glance

Sequential-EQA preserves scene memory between questions instead of resetting after each independent episode. Occupancy-only maps may retain explored space without visual attributes; short-horizon implicit context can become mismatched to later questions. Among four tested systems, 3D-Mem combines a substantial accuracy gain with shorter paths. The evidence concerns these implementations, scenes and adaptation rules, rather than a general necessity theorem or a days-long deployment demonstration.

I Introduction

Embodied question answering requires navigation, evidence collection and deciding when an answer is justified. Repeating exploration for each related query wastes information, but leaving memory on need not make it useful. The diagnostic changes persistence while holding each agent’s weights and question sequence fixed. Architectural failure explanations are plausible interpretations of system-level results; persistent versus reset runs do not separately ablate mapping, retrieval, perception and training history.

Complete original Fig. 1 : Comparison between Episodic and Sequential Evaluation paradigms. (a) In episodic evaluation , the agent’s memory is cleared after every task; for each new query (e.g., Q2), the agent must repeat the full exploration process as if the environment were novel.
(b) Conversely, sequential evaluation allows the agent to maintain a persistent memory across successive tasks. While the initial task (Q1) requires full exploration, the agent leverages its prior knowledge in subsequent tasks (Q2) to navigate directly to targets, eliminating redundant re-exploration and significantly improving efficiency.
Fig. 1: Comparison between Episodic and Sequential Evaluation paradigms. (a) In episodic evaluation, the agent’s memory is cleared after every task; for each new query (e.g., Q2), the agent must repeat the full exploration process as if the environment were novel. (b) Conversely, sequential evaluation allows the agent to maintain a persistent memory across successive tasks. While the initial task (Q1) requires full exploration, the agent leverages its prior knowledge in subsequent tasks (Q2) to navigate directly to targets, eliminating redundant re-exploration and significantly improving efficiency.
Original protocol definitions and mathematical context

We show that spatially grounded 3D memory is necessary to break the accuracy–efficiency tradeoff, simultaneously achieving +33.3%+33.3\% accuracy gain and +53.3%+53.3\% navigation cost reduction in sequential settings.

II Related Work

Earlier EQA, lifelong navigation, structured mapping and long-horizon active QA already address parts of this problem. The claimed contribution is a controlled persistence protocol applied to existing episodic agents, rather than the first long-horizon EQA task or a newly trained memory algorithm. The related work distinguishes spatial fields, scene graphs and visual libraries that differ in how they associate observations across time.

III Sequential Evaluation Protocol

Questions are grouped by environment and answered in a fixed scene-level order. Episodic evaluation starts each question with empty memory; sequential evaluation inherits the previous terminal memory. Both use the same questions and frozen agent. The first question has no prior-question evidence, and later positions can benefit from incidental coverage acquired while answering earlier queries.

III-A Problem Formulation: Episodic vs. Sequential

Equations 1–2 formalize the difference as the memory argument passed to the policy. They abstract away the detailed map update, retrieval and stopping rules. Here continuous evaluation means evidence persists across queries; it does not necessarily mean the robot begins the next question at its previous physical endpoint. Real trials explicitly reset position, and the later released adapter contract also resets simulator pose per question.

Original protocol definitions and mathematical context

We define the EQA task within an environment ℰ\mathcal{E} containing an ordered set of NN questions 𝒬={q1,q2,…,qN}\mathcal{Q}=\{q_{1},q_{2},\dots,q_{N}\}. The agent is characterized by a policy π\pi and an internal state (memory) m∈ℳm\in\mathcal{M}, which may include spatial maps, latent embeddings, or navigation history.

Original protocol definitions and mathematical context

Episodic Evaluation: In the standard episodic protocol, each question qi∈𝒬q_{i}\in\mathcal{Q} is treated as a disjoint Markov decision process (MDP). The agent’s memory is re-initialized to a null state m0=∅m_{0}=\emptyset at the start of every question. The trajectory τi\tau_{i} for question qiq_{i} is generated as:

Complete original Equation 1
Original selectable equation
Equation 1
τi∼π⁡(qi,m0).\tau_{i}\sim\pi(q_{i},m_{0}). (1)
Original protocol definitions and mathematical context

Sequential Evaluation: In the sequential protocol, the memory is accumulated over the set of questions 𝒬\mathcal{Q}. If mim_{i} is the memory at the terminal state for question qiq_{i}, then the trajectory for question qi+1q_{i+1} is generated as

Complete original Equation 2
Original selectable equation
Equation 2
τi+1∼π⁡(qi+1,mi).\tau_{i+1}\sim\pi(q_{i+1},m_{i}). (2)

III-B Sequentializing an Existing Benchmark

Equation 3 groups all selected questions from each scene into a variable-length sequence. The manuscript says questions are shuffled with a fixed seed and the same order is used for every comparison. The later official benchmark release provides fifty HM3D scenes and 498 questions, but its documentation states that the original source revision and generation seed were not preserved. Canonical released order is available; the missing provenance fields should not be guessed, and robustness across alternate orders is not established.

Original protocol definitions and mathematical context

To isolate memory reuse, we group questions from the same environment ℰ\mathcal{E} into a sequence 𝒮\mathcal{S}. Given the original dataset 𝒟episodic={(qj,ℰj)}j=1M\mathcal{D}_{\text{episodic}}=\{(q_{j},\mathcal{E}_{j})\}_{j=1}^{M}, the sequential version is:

Complete original Equation 3
Original selectable equation
Equation 3
𝒟seq={(𝒮k,ℰk)}k=1K.\mathcal{D}_{\text{seq}}=\{(\mathcal{S}_{k},\mathcal{E}_{k})\}_{k=1}^{K}. (3)
Original protocol definitions and mathematical context

Each 𝒮k=(qk,1,…,qk,Nk)\mathcal{S}_{k}=(q_{k,1},\dots,q_{k,N_{k}}) contains all selected questions from scene ℰk\mathcal{E}_{k}; thus NkN_{k} varies by scene. Sequence order is randomly shuffled with a fixed seed and held constant for all agents and episodic/sequential comparisons. We will release the generated sequences, split code, and evaluation configuration for reproducibility.

III-C Evaluation Metrics

Equations 4–9 pool question scores, define memory advantage as sequential minus episodic accuracy, and define step advantage from the fractional reduction in path length. Thus 25.5→58.8 is +33.3 percentage points, about 130.6% relative, while the reported 53.3% is a relative cost reduction. The manuscript permits binary or real-valued success scores without fully specifying their conversion. The later evaluation code explicitly separates normalized OpenEQA score, mean((score−1)/4)×100, from binary success at score≥4. That release clarification does not independently reconstruct the manuscript table. Equations define total PL, while Figure 3 labels mean distance; identical question sets make total and mean ratios agree. Printed rounded 5.6→2.6 gives 53.6%, so the reported 53.3% should be retained rather than silently recalculated.

Original protocol definitions and mathematical context

We quantify the efficacy of memory retention through both task success and navigational efficiency. Let yk,iy_{k,i} be the success indicator (binary or real-valued score) for the ii-th question in sequence 𝒮k\mathcal{S}_{k}, and Lk,iL_{k,i} be the geodesic path length (in meters) traversed during that question.

Original protocol definitions and mathematical context

Accuracy metrics. Success Rate (S​RSR) is the mean accuracy under episodic resets:

Complete original Equation 4
Original selectable equation
Equation 4
S​R=1∑k=1KNk​∑k=1K∑i=1Nkyk,is.t.m0,k,i=∅.SR=\frac{1}{\sum_{k=1}^{K}N_{k}}\sum_{k=1}^{K}\sum_{i=1}^{N_{k}}y_{k,i}\quad\text{s.t.}\quad m_{0,k,i}=\emptyset. (4)
Original protocol definitions and mathematical context

Success Rate with Memory (S​RmemSR_{\text{mem}}) is the mean accuracy under state persistence:

Complete original Equation 5
Original selectable equation
Equation 5
S​Rmem=1∑k=1KNk​∑k=1K∑i=1Nkyk,is.t.m0,k,i=mTk,i−1,k,i−1.SR_{\text{mem}}=\frac{1}{\sum_{k=1}^{K}N_{k}}\sum_{k=1}^{K}\sum_{i=1}^{N_{k}}y_{k,i}\quad\text{s.t.}\quad m_{0,k,i}=m_{T_{k,i-1},k,i-1}. (5)
Original protocol definitions and mathematical context

Memory Advantage (M​AMA) is the absolute performance gain of using sequential memory:

Complete original Equation 6
Original selectable equation
Equation 6
M​A=S​Rmem−S​R.MA=SR_{\text{mem}}-SR. (6)
Original protocol definitions and mathematical context

Efficiency metrics. Path Length (P​LPL) is the total navigation cost required without information reuse:

Complete original Equation 7
Original selectable equation
Equation 7
P​L=∑k=1K∑i=1NkLk,is.t.m0,k,i=∅.PL=\sum_{k=1}^{K}\sum_{i=1}^{N_{k}}L_{k,i}\quad\text{s.t.}\quad m_{0,k,i}=\emptyset. (7)
Original protocol definitions and mathematical context

Path Length with Memory (P​LmemPL_{\text{mem}}) is the total cost when leveraging prior exploration:

Complete original Equation 8
Original selectable equation
Equation 8
P​Lmem=∑k=1K∑i=1NkLk,is.t.m0,k,i=mTk,i−1,k,i−1.PL_{\text{mem}}=\sum_{k=1}^{K}\sum_{i=1}^{N_{k}}L_{k,i}\quad\text{s.t.}\quad m_{0,k,i}=m_{T_{k,i-1},k,i-1}. (8)
Original protocol definitions and mathematical context

Step Advantage (S​ASA) is the normalized efficiency gain, expressed as a percentage:

Complete original Equation 9
Original selectable equation
Equation 9
S​A=(P​L−P​LmemP​L)×100%.SA=\left(\frac{PL-PL_{\text{mem}}}{PL}\right)\times 100\%. (9)

IV Memory Representation Paradigms in EQA

Table I compares what each agent stores and how it retrieves evidence. Frozen VLM agents expose explicit map or memory structures, while the VLA’s action predictions depend on learned visual context. The comparison changes whole systems and their original training, rather than swapping only one memory component into an otherwise identical agent. It diagnoses the tested agents under this protocol and motivates more controlled memory ablations.

Complete original TABLE I : Comparison of memory paradigms in embodied EQA. We categorize each method by its architectural paradigm, the specific content stored, and the primary retrieval mechanism employed.
TABLE I: Comparison of memory paradigms in embodied EQA. We categorize each method by its architectural paradigm, the specific content stored, and the primary retrieval mechanism employed.
Accessible original table, all values and grouped headers
Method Paradigm What is Stored Retrieval Mechanism
Explore-EQA VLM-Agent Occupancy + frontier scores Frontier ranking by VLM
MemoryEQA VLM-Agent RGB + pose + language description + embedding Entropy-based adaptive retrieval
3D-Mem VLM-Agent 3D point cloud + visual embeddings Spatially indexed 3D lookup
UniNavid VLA Transformer hidden state Implicit attention

IV-A Evaluated Agents

ExploreEQA keeps occupancy and frontier information, useful for exploration but limited for later appearance questions. MemoryEQA stores RGB, pose, language and embeddings in separate entries; 3D-Mem associates visual features with a metric reconstruction; UniNavid relies on short visual histories and implicit context. All four original rows and retrieval mechanisms remain below. Richer memory alone does not guarantee successful query selection, and episodic training history differs across agents.

IV-B Memory Adaptation Protocol

The manuscript carries terminal memory forward without truncation or summarization, replaces the question, resets goal-directed planners and freezes model weights. This tests off-the-shelf persistence rather than each architecture’s best possible sequential finetuning. The later adapter documentation makes the operational scope more precise: only designated scene memory survives; pose, prompts, active targets, stopping state and pending actions reset. These release details are versioned evidence, not proof that every manuscript run used precisely the same implementation.

Original protocol definitions and mathematical context

To isolate the contribution of memory architecture from the confounding effects of sequential training, we adopt a minimal persistence protocol that transitions each agent from episodic to sequential evaluation without any modification to model weights or training procedures. This design choice is deliberate: our goal is not to find the best-performing sequential agent after task-specific optimization, but rather to diagnose which architectural properties intrinsically support knowledge accumulation across queries. The protocol applies uniformly to all agents. At the boundary between consecutive questions qiq_{i} and qi+1q_{i+1} within a scene, we perform three operations: (1) state inheritance: The agent’s terminal memory state, mTi,im_{T_{i},i}, is carried forward as the initial state for the next query, m0,i+1m_{0,i+1}, with no truncation or summarization. (2) query update: The task-specific input is replaced with qi+1q_{i+1}, and any goal-directed planners are re-initialized accordingly. (3) frozen weights: All network parameters θ\theta remain strictly unchanged throughout the sequence, and no uncertainty recalibration or auxiliary adaptation is applied. This minimal intervention ensures that any performance difference between episodic and sequential conditions is attributable solely to the structure of the inherited representation, not to any learned sequential behavior. Agents that benefit from this protocol do so because their memory architecture is compositional by design; agents that fail do so because their representations were never structured to support cross-query reuse.

V Experimental Results and Analysis

The evaluation considers pooled accuracy and cost, variation by query position, selected trajectories and a small physical experiment. Separating accuracy from cost is valuable: short paths alone cannot tell whether the agent retrieved sufficient evidence, stopped prematurely or simply gave up. The table nevertheless shows almost no overall path savings for three of the four systems, so broad claims that all persistent memories substantially reduce navigation should be read narrowly.

V-A Experimental Setup

Questions come from OpenEQA and are grouped by scene without forcing a fixed horizon. Figure 2 spans six to fourteen questions, with most scenes around ten to thirteen; later positions therefore average fewer, different scenes. The manuscript reports Qwen3-VL 8B-Instruct, “FB8” quantization and A5000 GPUs; the precision label is preserved as written. Exact shuffled files and hyperparameters were promised. The later public release supplies an identifiable sequence artifact, but the paper does not report alternate-order variance, a memory-budget scaling experiment or a days/weeks retention test.

Complete original Fig. 2 : Distribution of questions per scene in the sequentialized OpenEQA benchmark. Bars show the number of scenes containing exactly N k N_{k} questions; the overlaid line shows the cumulative percentage of scenes with at least N k N_{k} questions. We evaluate all generated scene-level sequences; aggregate metrics pool all questions, while query-index analyses average over scenes with N k ≥ i N_{k}\geq i .
Fig. 2: Distribution of questions per scene in the sequentialized OpenEQA benchmark. Bars show the number of scenes containing exactly NkN_{k} questions; the overlaid line shows the cumulative percentage of scenes with at least NkN_{k} questions. We evaluate all generated scene-level sequences; aggregate metrics pool all questions, while query-index analyses average over scenes with Nk≥iN_{k}\geq i.
Original protocol definitions and mathematical context

Dataset. We construct our sequential benchmark by re-grouping questions from the OpenEQA [14] dataset by scene and evaluate the aforementioned 4 different types of representative agents. OpenEQA provides real indoor environments paired with free-form questions that require spatial and semantic reasoning. Under our sequential protocol, all selected questions belonging to the same scene are concatenated into one ordered sequence, and the agent’s internal state is preserved across them without any resets. We evaluate all resulting scene-level sequences rather than truncating every scene to a fixed horizon: aggregate results pool all questions from all sequences, while per-query-index plots include, at index ii, only scenes whose sequence length satisfies Nk≥iN_{k}\geq i. Figure 2 shows the distribution of sequence lengths across scenes in the resulting benchmark. The majority of scenes contain between 10 and 13 questions, with over 60% of scenes providing sequences of at least 10 queries. The exact shuffled sequence files, split-generation code, and evaluation hyperparameters will be released with the benchmark to make the random ordering reproducible and to support future controlled comparisons.

V-B Sequential Memory Does Not Uniformly Improve Performance

Table II gives positive accuracy changes for every agent: ExploreEQA+2.7, MemoryEQA+1.4, UniNavid+0.9 and 3D-Mem+33.3 points. Thus 3D-Mem is the only substantial joint accuracy/efficiency improvement here, not the only positive accuracy change. Its 58.8 sequential score remains below MemoryEQA’s 62.4. Path lengths are 84.3→84.3,43.6→43.9,5.6→2.6 and 12.2→12.4 respectively: only 3D-Mem has a large pooled reduction. Mechanisms such as semantic interference, distribution shift and premature stopping are interpretations, not directly isolated causal measurements. Figures 3–4 retain their standard errors, log scales and all query positions. In Figure 4, green means positive sequential-minus-episodic change, which is favorable for score but unfavorable for time; negative/red time changes save time. Changing scene composition at later positions prevents treating the curves as a fixed cohort’s causal learning trajectory. Neither exponential memory utility nor universal necessity of spatial fusion is quantified.

Complete original TABLE II : Performance comparison on Sequential-EQA benchmark. Results show success rates under episodic (SR %) and sequential (SR mem {}_{\text{mem}} %) conditions, memory advantage (MA %), path lengths (PL), and step advantage (SA %). ↑ \uparrow indicates higher is better, ↓ \downarrow indicates lower is better. Standard errors shown in Fig. 3 .
TABLE II: Performance comparison on Sequential-EQA benchmark. Results show success rates under episodic (SR %) and sequential (SRmem{}_{\text{mem}} %) conditions, memory advantage (MA %), path lengths (PL), and step advantage (SA %). ↑\uparrow indicates higher is better, ↓\downarrow indicates lower is better. Standard errors shown in Fig. 3.
Accessible original table, all values and grouped headers
Method SR ↑\uparrow SRmem{}_{\text{mem}} ↑\uparrow MA ↑\uparrow PL ↓\downarrow PLmem{}_{\text{mem}} ↓\downarrow SA ↑\uparrow
ExploreEQA 43.8 46.5 2.7 84.3 84.3 0.0
MemoryEQA 61.0 62.4 1.4 43.6 43.9 -0.5
3D-Mem 25.5 58.8 33.3 5.6 2.6 53.3
UniNavid 36.4 37.3 0.9 12.2 12.4 -1.5
Original protocol definitions and mathematical context

Accuracy and efficiency. Table II shows that memory persistence alone does not reliably improve answer quality. ExploreEQA, UniNavid, and MemoryEQA all obtain near-zero memory advantage (MA <3%<3\%), but for different architectural reasons. ExploreEQA retains traversed geometry but not object-level semantic evidence, so it remembers where the agent has been rather than what was observed. UniNavid inherits recurrent context across query boundaries, although it was trained on short episodic horizons; this makes sequential context an out-of-distribution input rather than a reusable scene model. MemoryEQA stores richer RGB observations, poses, descriptions, and embeddings, but these entries accumulate as independent events rather than a globally aligned scene representation, increasing retrieval noise as sequences grow.

Original protocol definitions and mathematical context

In contrast, 3D-Mem improves both answer accuracy and navigation efficiency, with +33.3%+33.3\% MA and +53.3%+53.3\% SA. This improvement is not merely trajectory compression: the agent answers more questions correctly while navigating less. By anchoring observations to metric 3D geometry, each query enriches a coherent scene representation rather than adding unstructured history.

Complete original Fig. 3 : Accuracy–efficiency tradeoff under sequential memory reuse. From left to right: mean answer score (1–5; ↑ \uparrow ), mean navigation time (seconds; ↓ \downarrow ), mean navigation distance (meters; ↓ \downarrow ), and a joint efficiency-vs-accuracy scatter plot. Light bars denote episodic (no memory) evaluation; dark bars denote sequential (with memory) evaluation; error bars indicate standard error.
Fig. 3: Accuracy–efficiency tradeoff under sequential memory reuse. From left to right: mean answer score (1–5; ↑\uparrow), mean navigation time (seconds; ↓\downarrow), mean navigation distance (meters; ↓\downarrow), and a joint efficiency-vs-accuracy scatter plot. Light bars denote episodic (no memory) evaluation; dark bars denote sequential (with memory) evaluation; error bars indicate standard error.
Key Takeaways from Sequential Evaluation Persistence ≠\neq Accumulation. Merely retaining state across sequential queries does not guarantee successful environmental knowledge accumulation. Spatial Grounding is Critical. Only spatially grounded memory consistently translates accumulated observations into simultaneous gains in answer accuracy and navigation efficiency. Decoupling of Efficiency and Success via Premature Termination. Shorter navigation paths do not imply better reasoning; unstructured memory can bias agents to stop early in familiar regions without finding correct evidence. Structured Memory Supports Cross-query Accumulation. Structuring observations geometrically allows the agent’s representation to become exponentially more useful over later queries.
Original protocol definitions and mathematical context

Dynamics across query position. Aggregate metrics show average gains, but they do not reveal whether memory becomes more useful as a sequence progresses. We therefore analyze accuracy and navigation time by query position ii.

Complete original Fig. 4 : Per-query-index accuracy and navigation time under sequential memory reuse. Panel A shows mean answer score (1–5; ↑ \uparrow , log scale); Panel B shows mean navigation time (seconds; ↓ \downarrow , log scale). The x-axis is query position i i within a scene sequence; each position averages over all sequences with N k ≥ i N_{k}\geq i , not only sequences of length 13. Δ \Delta denotes sequential minus episodic performance and is color-coded by sign: green for Δ > 0 \Delta>0 and red for Δ < 0 \Delta<0 . Because metric direction differs across panels, green indicates improvement only for Panel A, whereas red/negative Δ \Delta in Panel B indicates an improvement because lower time is better.
Fig. 4: Per-query-index accuracy and navigation time under sequential memory reuse. Panel A shows mean answer score (1–5; ↑\uparrow, log scale); Panel B shows mean navigation time (seconds; ↓\downarrow, log scale). The x-axis is query position ii within a scene sequence; each position averages over all sequences with Nk≥iN_{k}\geq i, not only sequences of length 13. Δ\Delta denotes sequential minus episodic performance and is color-coded by sign: green for Δ>0\Delta>0 and red for Δ<0\Delta<0. Because metric direction differs across panels, green indicates improvement only for Panel A, whereas red/negative Δ\Delta in Panel B indicates an improvement because lower time is better.
Original protocol definitions and mathematical context

Figure 4 averages each position over scenes with Nk≥iN_{k}\geq i, so later points use the subset of longer sequences. Navigation time generally decreases under memory reuse, since even weak memory can reduce repeated exploration or bias the agent toward familiar regions. However, lower navigation cost is not sufficient evidence of knowledge accumulation: it may also reflect premature stopping or over-reliance on incomplete history.

Original protocol definitions and mathematical context

Accuracy reveals this distinction. ExploreEQA, UniNavid, and MemoryEQA show no stable improvement across query positions; gains and losses alternate as history accumulates. This suggests that their retained state does not monotonically improve scene understanding. Only 3D-Mem shows sustained positive accuracy Δ\Delta across query positions, consistent with a representation that becomes more complete over time. Taken together, the aggregate and per-query analyses show that useful sequential memory must be both persistent and spatially compositional.

V-C Qualitative Analysis

Figure 5 contains two different scenes and both methods’ examples. 3D-Mem improves from one of three to three of three answers with less repeated exploration; UniNavid improves from two of three to three of three while its paths change little. These selected three-query illustrations explain possible reuse behavior and keep all maps, paths, question colors and correctness markers. They do not estimate how often the mechanism occurs, and the UniNavid example should not be omitted simply because the aggregate result favors 3D-Mem.

Complete original Fig. 5 : Qualitative comparison of agent trajectories under episodic (left) and sequential (right) evaluation for three consecutive questions (Q1: blue, Q2: orange, Q3: purple).
Fig. 5: Qualitative comparison of agent trajectories under episodic (left) and sequential (right) evaluation for three consecutive questions (Q1: blue, Q2: orange, Q3: purple).
Original protocol definitions and mathematical context

The right panel shows the same three queries under sequential evaluation. With a persistent 3D map accumulated from prior queries, the agent navigates directly to previously observed regions rather than re-exploring from scratch. Path length drops sharply across Q2 and Q3, consistent with the aggregate SA of 53.3%53.3\% reported in Table II. Critically, accuracy improves alongside efficiency: all three questions are answered correctly, including the smoke detector query, which the agent now answers correctly by drawing on observations already fused into its 3D representation from the first query’s exploration.

VI Real-Robot Validation

Physical tests ask whether the simulated persistence behavior also appears with imperfect sensing and actuation. They support feasibility and expose failures on the tested robot. Small question counts, different environment coverage across agents and absence of a reported confidence analysis limit any claim that the simulation mechanism is uniquely established in deployment.

VI-A Experimental Setup

A Unitree Go2 carries a RealSense D435i and L2 LiDAR. Each method gets an episodic and a sequential trial per environment, with five questions and the robot returned to the same start before every query. Memory alone persists in sequential trials. Three shared environments are a lab, lobby and hallway; UniNavid additionally runs on a rooftop terrace. This is physical navigation with retained evidence, rather than uninterrupted pose-continuous execution or controlled equal outdoor coverage for every method.

VI-B Results and Analysis

Reported accuracy is 15%→15% for UniNavid,20%→40% for 3D-Mem,33%→26% for ExploreEQA and 40%→47% for MemoryEQA. Five questions in three shared environments suggest only fifteen question outcomes per setting; UniNavid’s additional environment gives a different scope. The rounded percentages and small sample should remain source results rather than an invented precise significance test. MemoryEQA’s sequential 47% is higher than 3D-Mem’s 40%, although 3D-Mem has the larger change. Figure 6 and both original project movies show indoor/outdoor UniNavid behavior, not comparative video evidence for all four agents.

Complete original Fig. 6 : Real-world deployment of the Unitree Go2. In both environments, UniNavid fails to leverage previous knowledge. In the indoor setting (left), the agent re-navigates to the chair for the second question as if the first encounter never occurred. Similarly, in the outdoor setting (right), the agent repeats its navigation pattern for the second query. In both cases, the agent exhibits erratic rotation and fails to answer correctly, confirming that the sequential memory reuse bottleneck generalizes to physical deployment.
Fig. 6: Real-world deployment of the Unitree Go2. In both environments, UniNavid fails to leverage previous knowledge. In the indoor setting (left), the agent re-navigates to the chair for the second question as if the first encounter never occurred. Similarly, in the outdoor setting (right), the agent repeats its navigation pattern for the second query. In both cases, the agent exhibits erratic rotation and fails to answer correctly, confirming that the sequential memory reuse bottleneck generalizes to physical deployment.
Original protocol definitions and mathematical context

Figure 6 shows the UniNavid in both indoor and outdoor environments. UniNavid achieved 15%15\% accuracy in both episodic and sequential settings across all scenes. Failure severity is similar in both indoor and outdoor settings. 3D-Mem stayed consistent with its performance in simulation, improving its success rate from 20%20\% to 40%40\% when switching from episodic to sequential. ExploreEQA actually had a decrease in accuracy from 33%33\% to 26%26\%; however, that could be due to the small test size. Finally, MemoryEQA improved from 40%40\% to 47%47\% accuracy when maintaining memory.

VII Conclusion

The defensible conclusion is that persistent memory must retain retrievable task-relevant evidence, and geometric visual fusion worked especially well for the tested 3D-Mem pipeline. Keeping state on does not ensure useful accumulation. Broader claims about preventing forgetting, exponential usefulness or the only deployable memory structure go beyond these comparisons. Next evidence would include controlled component swaps, matched budgets, multiple question orders, scene-aware uncertainty and changing environments over longer deployments.

Original project figures and robot demonstrations

The official site’s original diagrams, complete plot images and trajectory panels are preserved alongside the paper’s native figures. Its two original MOV clips show the Unitree’s indoor and rooftop trials; source posters remain attached. H.264 copies provide browser playback while the unchanged HEVC originals remain linked. These are qualitative demonstrations, not additional independent trial counts or evidence for methods absent from the clips. The project’s schematic images illustrate the protocol and are labeled separately from measured result plots.

Episodic Q1 — full exploration trajectory
Episodic Q1 — full exploration trajectory
Episodic Q2 — full exploration trajectory
Episodic Q2 — full exploration trajectory
Episodic Q3 — full exploration trajectory
Episodic Q3 — full exploration trajectory
Sequential Q1 — full exploration trajectory
Sequential Q1 — full exploration trajectory
Sequential Q2 — direct navigation reusing memory
Sequential Q2 — direct navigation reusing memory
Sequential Q3 — direct navigation reusing memory
Sequential Q3 — direct navigation reusing memory
Bar charts comparing mean answer score, navigation time and navigation distance between episodic and sequential evaluation across four methods, plus a joint efficiency-vs-accuracy scatter plot.
Bar charts comparing mean answer score, navigation time and navigation distance between episodic and sequential evaluation across four methods, plus a joint efficiency-vs-accuracy scatter plot.
Two-panel line chart of mean answer score and mean navigation time as a function of query position within a scene sequence, comparing episodic and sequential evaluation.
Two-panel line chart of mean answer score and mean navigation time as a function of query position within a scene sequence, comparing episodic and sequential evaluation.
3D-Mem trajectories for three questions in the same apartment under episodic and sequential evaluation, showing reduced path length and correct answers under sequential evaluation.
3D-Mem trajectories for three questions in the same apartment under episodic and sequential evaluation, showing reduced path length and correct answers under sequential evaluation.
UniNavid trajectories for three questions on an MP3D scene under episodic and sequential evaluation, showing a correction on Q2 with minimal change in navigation efficiency.
UniNavid trajectories for three questions on an MP3D scene under episodic and sequential evaluation, showing a correction on Q2 with minimal change in navigation efficiency.
Indoor Sequential-EQA robot trial — H.264 playback copy; unchanged original HEVC movie
Outdoor Sequential-EQA robot trial — H.264 playback copy; unchanged original HEVC movie
Concept demonstration

Open image ↗