At a glance
NOIR-EEG combines noninvasive EEG decoding, parameterized robot skills, and learned intention suggestions. Users select an object, a skill, and its parameters; the robot executes the resulting low-level movement. Predictions from a previous demonstration can bypass much of the slow decoding process, while users retain confirmation and correction.
The preprint reports 16 participants and 15 household tasks, with performance plots evaluated over 30 trials per condition. Mean task time fell from 17.54 to 12.60 minutes with intention prediction. The evidence concerns a controlled tabletop study; it does not establish independent home use, clinical effectiveness, or universal intention decoding.
Introduction
The design avoids asking noisy, low-resolution EEG to specify every motor command. Instead, neural signals communicate a sequence of high-level, parameterized skills, and robot control supplies the detailed motion. A staged decoder combines visually evoked signals with motor imagery, while foundation models learn reusable suggestions from earlier interaction.

Performing complex tasks
An object detector identifies candidates in the camera image [1]. Objects receive flicker masks at 6, 7.5, 8.57, or 10 Hz; canonical correlation with reference signals identifies which object the user attends to [5]. Four-class motor imagery then chooses a skill and moves a cursor in the image plane; two-class imagery selects height.
The system uses a 128-channel saline EEG net. Motor-imagery signals are filtered over 8–30 Hz and classified with filter-bank common spatial patterns plus a support vector machine [7]. Jaw-clench EMG confirms or rejects choices. Thus the complete interface includes a voluntary muscle signal as well as brain-signal decoding.
Task completion and decoding accuracy
Sixteen participants—11 male and five female—performed tasks spanning seven meal-preparation, four cleaning, two personal-care, and two entertainment activities. They sat separately from the Franka Panda robot. The paper reports that all task trials completed on the first attempt; this is task-level completion with confirmation and correction, not error-free decoding.
Mean completion time was 17.54 minutes. CookPasta averaged about 34 minutes and ten executed skills. Although the task description says 3–9 steps, execution can require additional skills. Decision-making and decoding occupied 63% of total time: object, skill, and parameter selection accounted for 5%, 13%, and 45%. Cursor-based parameter selection was the main bottleneck.
Mean SSVEP accuracy was 90.00% during calibration and 86.20% in testing; four-class motor-imagery accuracy was 50.10% and 49.60%. Paired participant-level tests found no significant calibration/test difference. This does not mean the approximately 50% motor-imagery decoder became highly accurate. Figure 2 retains the distributions, standard deviations, and cursor-path efficiency.
Participants sometimes adapted primitive skills creatively: pushing a wide book to the edge before side-grasping it, or repeatedly moving a knife with Push to slice a banana. These examples illustrate flexible skill composition rather than new learned low-level manipulation skills.

Intention prediction
After one successful demonstration, the system stores images and chosen objects, skills, and parameters. GPT-4o compares a new image with that recorded sequence and suggests the next object–skill pair. After confirmation, DINOv2 semantic correspondence proposes the parameter location [9]. Suggestions can accommodate changed object placement and appearance within the demonstrated task.

Time and effort with prediction
Mean trial time decreased from 17.54 ± 7.07 to 12.60 ± 5.20 minutes, about a 28% reduction; the paired trial-level test reports t(29)=5.59, p<.001. Mean per-step selection times fell from 0.56 to 0.15 minutes for object–skill choice, 0.92 to 0.19 for image-plane coordinates, and 0.46 to 0.19 for height.
Median cursor travel fell from 3,382.50 to zero pixels for image-plane selection and from 847.55 to zero for height. Reported suggestion accuracy exceeded 94%, allowing many suggestions to be accepted directly. Zero median adjustment does not imply every suggested parameter was correct.
Survey medians improved from four to five on five-point scales for experience, perceived robot intelligence, and ease of providing signals. The survey labels the first trial as without prediction and the second as with prediction. The manuscript does not report counterbalancing; practice and condition order therefore remain potential contributors. Thirty trial observations should not be read as thirty independent participants.

Discussion
The central contribution is an integrated interaction system: staged neural decoding makes skill selection possible, while robot intelligence reduces how often that decoder must be used. Earlier EEG robot studies focused on narrower navigation or reaching tasks [8] [10] [13]. The paper’s broader task set is not a controlled head-to-head comparison with every prior system.
The results depend on calibrated EEG hardware, a prepared laboratory, a fixed skill library, visual attention to flicker, and jaw-clench feedback. The manuscript gives no long-term home evaluation or patient-specific efficacy study. It also does not establish generalization to arbitrary unseen tasks or guarantee that confirmation catches every unsafe action.
Author contributions
Ruohan Zhang, Tasha Kim, and Yingke Wang co-led the investigation. Team members developed the decoding pipeline, robot skills, intention learning, and experimental procedures; Kim, Cho, Hodges, and Wang conducted the human experiments. Anthony Norcia, Li Fei-Fei, and Jiajun Wu jointly supervised. The preprint declares no competing interest.
Appendix A: Staged decoding
Appendix A specifies the decoders. SSVEP uses selected visual-cortex channels, motor imagery uses selected motor/visual and peripheral channels, and clench detection uses all channels. Figures 5 and 6 preserve the complete control flow and electrode maps.

SSVEP object selection
For an EEG matrix with C channels and T time samples, the decoder compares the signal against a set of candidate flicker frequencies. The attended frequency is chosen using canonical correlation with sinusoidal harmonics [4] [5].

Reference signals
Each candidate frequency receives sine and cosine components through H harmonics. Samples occur at times 0, 1/f_s, …, (T−1)/f_s; the reference matrix has 2H rows and T columns. Equation 1 is retained exactly.
Equation
Canonical correlation analysis
CCA finds projections of the EEG matrix and reference signals that maximize their correlation. The source’s normalized covariance expression is preserved in Equation 2.
Equation
Frequency detection
The method computes one canonical correlation for each candidate frequency and selects the maximum. Equations 3–4 preserve both the frequency-specific objective and the final decision rule.
Equations
Motor-imagery skill selection
Filter-bank common spatial patterns separate motor-imagery classes using frequency-dependent spatial variance [7]. The appendix presents the pairwise covariance projection used within its multiclass decoding pipeline.
Band-pass filtering
The EEG signal is passed through B band-pass filters, yielding one channel-by-time matrix per frequency band.
Equation
Covariance matrices
For each band, the spatial covariance is normalized by its trace so that overall signal magnitude does not directly determine the covariance scale.
Equation
Spatial projection
A generalized eigenvalue problem between class covariance matrices produces spatial filters. Applying their transpose to the band-filtered EEG yields the projected signal.
Equations
Feature extraction
The source computes each component’s mean squared amplitude and logs its share of the total component variance. Equations 9–10 retain the exact formulas; the displayed “variance” formula does not include an explicit mean-subtraction term.
Equations
Feature concatenation and classification
Features from all selected bands and spatial components are concatenated, then classified by a support vector machine to predict the imagined action.
Equation
Cursor parameter selection
Cursor selection maintains a penalty for each of four image-plane directions. Penalties discourage previously rejected directions, and a temporary boost favors a selected direction. Equation 12 gives the initial penalty state.
Equation
Direction penalties
At each step, the penalty is reduced by a constant and clipped at zero. The prose calls this exponential decay, but Equation 13 specifies subtractive decay; the equation is preserved unchanged. Direction probabilities are divided by one plus the penalty and normalized.
Equations
Direction boost
The boost also decays subtractively to zero. A favored direction’s probability is multiplied by one plus the boost, followed by normalization. The paper switches between penalized and unpenalized probability symbols in these steps; the equations and algorithm retain that notation rather than silently resolving the implementation ambiguity.
Equations
Jaw-clench correction and algorithm
A detected jaw clench increases the penalty on the current direction. Algorithm 1 preserves the full selection loop, including collection, weighting, movement, decay, rejection, and final cursor return.
Equation
Clench decoder
The clench decoder distinguishes jaw-muscle tension from background activity and blink artifacts through signal variance, frequency content, spatial distribution, and duration. These are engineered EMG features measured by the EEG equipment.
Signal variance
Median channel variance supplies a global muscle-activity statistic.
Equation
Blink artifacts
Median variance over frontal channels supplies a blink-related statistic.
Equation
Frequency features
Low-frequency power covers 1–4 Hz, while high-frequency power covers 20 Hz and above. Their ratio includes a small denominator stabilizer.
Equations
Spatial features
Median temporal-channel variance supplies spatial evidence for clenching.
Equation
Duration
The event duration is the number of samples divided by sampling frequency.
Equation
Final classification
The binary decision requires all conditions to hold. The appendix does not give the numeric feature thresholds inside this final rule, so the equations alone do not fully specify a reproducible detector.
Equation
Appendix B: Object–skill retrieval
Appendix B adapts retrieval-based imitation learning [19] [20] [21]. A pretrained vision-language model aligns the current scene with a single recorded task demonstration, allowing the system to propose the next demonstrated action rather than requiring a new EEG choice at every stage.
State understanding and mapping
GPT-4o first interprets gripper and top views, aligns objects with recorded object labels and scene states with skill labels, and understands the order of the demonstration. The original sample prompt is preserved, including its ellipses; the paper does not provide a fully expanded production prompt.
The gripper view image captures what is held in the robot’s gripper during the current state. ... The top-view image captures the spatial relationships and status of objects on the table during the current state. ... I will give you a sequence of all the states in the current task, arranged in order. Please do the following: 1. Fully understand the object types in the gripper view images and pair them with the ‘obj’ text annotations. ... 2. Fully understand the objects in the top-view images, the spatial relationships between these objects, and the current status of each object, pairing them with the ‘skill’ text annotations. ... After that, I will send you two query images. ...
State inference and retrieval
For a new image pair, the model identifies the closest recorded state and returns the object and skill at the following state, ranked by likelihood. The exact sample query prompt follows.
Identify the state index from the sequence that best matches this new query pair. Then, return the ‘obj’ and ‘skill’ information of the next state (i.e., index + 1). ... Rank your answer by the order of possibility. ...
One-shot parameter learning
DINOv2 matches semantic features between a 360×240 demonstration image and the test image. A 3×3 neighborhood around the demonstrated point supplies the reference feature; cosine similarity finds a corresponding location.
The top view determines image-plane coordinates. A side-view match determines height by projecting onto the vertical axis associated with the predicted image-plane location, then converting pixels into robot coordinates. The paper describes a 75×100 feature map and “768 patch tokens”; it does not clarify that terminology further.
Appendix C: Experimental procedure
The Stanford institutional review board approved the study under IRB-67685. The manuscript describes hardware setup, participant instructions, calibration, and survey questions, but does not fully tabulate how the 30 trial observations are distributed across all 16 participants.
EEG acquisition and preparation
A 128-channel saline HydroCel Geodesic SensorNet streams EEG through Net Station 5.4 in an electrically and acoustically isolated lab. Preparation includes ten minutes of soaking in conductive solution, fitting the net, reducing each electrode’s impedance to at most 50 kΩ, and covering the net to slow drying. This preparation is additional to the task-completion times reported above.
Participant instructions
Participants attend to the desired flickering object. For motor imagery, they consistently imagine wrist movement or squeezing a ball for left/right classes, depressing a pedal with both feet for the leg class, and fixation for rest. Jaw tension communicates the clench signal.
Interface and calibration
Motor-imagery calibration uses a one-second warning, a 5.5-second imagery cue with the final five seconds recorded, and randomized 0.5–2-second rest. Four blocks yield twenty trials per action. Task-time skill decoding follows a two-second delay after the class-to-skill mapping appears.
Clench calibration collects three 500-millisecond trials for each of rest and clench. At decision time, written instructions and a countdown precede a 500-millisecond opportunity to respond.
Post-experiment survey
The preserved questionnaire separately rates the first and second trials for overall experience, robot intelligence, and difficulty providing signals, then invites comments. These subjective ratings complement measured time and cursor travel; they are not a direct physiological measure of cognitive workload.
Thank you for your participation! 1. Please rate your overall user experience for the two trials • Trial 1: (1-extremely unsatisfied, 5-extremely satisfied) • Trial 2: (1-extremely unsatisfied, 5-extremely satisfied) 2. How intelligent was the robot’s decision during the experiment? • Trial 1: (1-extremely unintelligent, 5-extremely intelligent) • Trial 2: (1-extremely unintelligent, 5-extremely intelligent) 3. How difficult was it for you to provide signals? • Trial 1: (1-extremely difficult, 5-extremely easy) • Trial 2: (1-extremely difficult, 5-extremely easy) 4. Please leave any comments you have here.
Appendix D: Task definitions
Appendix D defines success through initial and goal predicates in the BEHAVIOR Domain Definition Language [24] [25]. All fifteen original task illustrations below retain their scene examples, initial conditions, and complete goal conditions.















Robot platform and skills
The system uses a Franka arm with three RealSense cameras. Table 1 preserves nine skill families and all parameter choices. An operational-space pose controller executes waypoint trajectories through the Deoxys interface [26] [27]; numerical interpolation supplies reaching trajectories.
Supplementary videos
The original supplement provides two PourTea demonstrations: Video 1, without intention prediction, and Video 2, with intention prediction. These links retain access to the publisher-hosted videos.