At a glance
DAVATaR learns manipulation policies for a robot whose end-effector repeatedly crosses the air–water interface. It combines a fluidically sensorized gripper, tactile pressure and immersion-depth inputs, and visual training augmentations that approximate glare and apparent scale changes.
Across four evaluated tasks, mean success is 71%, compared with 41% for the augmented vision-only variant and 55% for tactile policies without the submerged-domain augmentations. These are averages over normal and adversarial settings. The evidence supports complementary benefits from sensing and augmentation in this laboratory setup.
1. Introduction
A camera above water can see objects at misleading positions or scales, and foam or glare can hide the surface and contact region. Tasks such as sampling at a valid depth, grasping a spoon, or wiping a plate therefore require physical information that vision alone may not reliably supply.
The system treats changing hydrostatic pressure as useful depth information while placing sensing electronics above the wet region. Its scope is submerged end-effector interaction from a dry robot, rather than a fully underwater robot.
3. System design
Three components work together: long waterproof sensing fingers, training-time visual perturbations, and a Transformer behavior-cloning policy that fuses recent images with tactile and depth channels. Only the end-effector enters the water.
3.1. Submerged gripper
Each finger combines a 305 mm aluminum beam with a 90 mm compliant tip. Five pairs of L-shaped air channels per fingertip change pressure under deformation. Differential transducers near the finger roots reduce common-mode disturbances while keeping readout electronics away from the submerged region.
An open/sealed tube pair supplies hydrostatic depth information. The reported policy sensing vector has eleven channels: ten contact-sensitive pressure channels plus one immersion-depth channel, sampled at 20 Hz. Figure 2 preserves the full construction and dimensions.
3.2. Visual augmentation
Standard translation and color jitter are followed by two submerged-domain perturbations. Reflection-like augmentation adds bright translucent patches whose brightness and spatial scattering vary. Refraction-like augmentation resizes the image and crops or pads it back to its original size.
These perturbations mimic occlusion and apparent scale changes. They do not reproduce complete refractive geometry, dynamic water motion, or optical defocus. Images alone are augmented; tactile signals, depth readings, and demonstrated actions remain paired with their original values.
3.3. Multimodal policy
The policy receives five synchronized observations. Each includes a 256 × 256 wrist image and, for tactile variants, all eleven local sensing channels. ResNet-18 with spatial softmax encodes images; a multilayer perceptron encodes local sensing. A Transformer fuses both modalities with learned readout tokens.
The readout parameterizes a mixture-density action distribution for operational-space motion and gripper commands. Training minimizes negative log likelihood of demonstrated actions. Augmentation is disabled during validation and robot execution. Figure 4 retains the complete training and architecture diagram.
4. Evaluation
The evaluation first characterizes contact-force and depth sensing, then compares task performance with and without tactile inputs and visual augmentation. The two ablations remove either all local tactile/depth sensing or the submerged-domain visual augmentations.
Force and immersion-depth characterization
Force calibration uses repeated one-minute pressing trials at dry, 5 cm, 10 cm, and 20 cm immersion. A wrist force-torque sensor supplies reference measurements. Support vector regression with a radial basis function kernel maps tactile features to force under five-fold cross-validation.
Figure 5 retains every calibration row. Grip-force root mean squared error (RMSE) rises from 0.11 N dry to 0.85 N at 20 cm; push-force RMSE is 0.46 N dry and 0.80 N at 20 cm. Overall reported values are 0.37 N for grip and 0.66 N for push. These characterize the tested 0–20 N range; performance varies with depth and force direction.
Depth characterization uses five two-minute trials with randomized motion at 1 cm/s in a 25 cm tank. Robot kinematics relative to the water surface provide reference depth. Linear calibration gives R² = 0.9996 and 1.6 mm RMSE, with repeatability of 0.30% of full-scale range. This is calibration evidence in the tested setup, not a guarantee under arbitrary temperature or water conditions.
Task setup
A Franka Emika Panda performs water sampling, spoon pickup, sink-plug removal, and plate wiping. Each task has forty demonstrations collected under normal conditions. All policy variants use the same behavior-cloning loss.
Each evaluation condition uses three trained seeds and six rollouts per seed, for eighteen rollouts. Adversarial conditions add thicker foam, bubbles, reflection, or deeper water. Robot evaluation runs at 5 Hz with a 300-step cap, distinct from the gripper’s 20 Hz sensing rate.
Task-level performance
Table 1 retains all task means. DAVATaR scores 0.67 for water sampling, 0.75 for spoon pickup, 0.61 for sink-plug removal, and 0.81 for plate wiping. The overall mean is 0.71 versus 0.28 for unaugmented vision-only, 0.41 for augmented vision-only, and 0.55 for tactile without submerged-domain augmentation.
The 71% versus 41% difference is thirty percentage points. Each task mean gives equal weight to normal success and its averaged adversarial success; it is not a single pooled success fraction over all rollouts.
Contribution of tactile sensing
Adding contact and immersion-depth channels helps infer hidden physical state, such as insertion depth beneath foam or whether fingers have reached the tank bottom. The task comparison supports a useful contribution from the combined tactile/depth stream.
Figure 7 shows attention changes during representative approach, contact, and completion phases. Its curves are normalized separately within each modality and cannot be interpreted as percentages of total modality importance. They also do not separate contact-force information from depth information or establish that attention caused a particular action.
Contribution of visual augmentation
Augmentation raises overall success from 28% to 41% without tactile sensing and from 55% to 71% with it. Under adversarial settings, the corresponding changes are 9% to 25% and 37% to 61%. For adversarial sink-plug removal, success rises from 0% to 25% without tactile and from 11% to 50% with tactile.
The single-augmentation spoon tests distinguish the two effects. In reflective conditions, reflection-only augmentation raises vision-only success from 0% to 54%, and tactile success from 46% to 71%. In deep water, refraction-only augmentation changes vision-only success from 38% to 39%, but tactile success from 37% to 67%.
The suffix names indicate what is removed: “-RefrA” retains reflection-like augmentation, while “-ReflA” retains refraction-like augmentation. Figure 8 preserves every panel and the original explanatory caption.
5. Conclusion and limitations
The combination of local physical sensing and visually perturbed imitation training performs best in the reported submerged tasks. The sensor contributes information hidden by water, while augmentation improves tolerance to corrupted images.
Repeated large forces can deform the compliant tips and peel channel sealant. Long air pathways are susceptible to temperature-induced pressure drift. The patch and zoom perturbations omit dynamic lensing, changing water motion, and refractive defocus. The experiments do not establish long-term hardware durability or generalization to every submerged environment.
A.1. Normal and adversarial conditions
Demonstrations use normal conditions; stronger visual disturbances are reserved for evaluation. Figure 9 and Table 2 retain all settings, including normal depth ranges, pose and plate-angle variation, 3–15 cm thick foam, glare, bubbles, and the 17–20 cm deep-water spoon condition.
Water sampling
A syringe must enter through surface foam, fill within the valid depth interval, and return without touching bottom sand. No water collection, sand contact, or timeout counts as failure. Foam hides the surface while sand limits how deep insertion may go.
Spoon pickup
The policy must grasp a spoon from the tank bottom and lift it out. Early closure, missing or dropping the spoon, and timeout are failures. Changing depth and refraction make the apparent gripper-to-spoon geometry unreliable.
Sink plug removal
A submerged plug must be aligned, rotated into its unlocked configuration, then removed. Premature lifting, over-rotation or misalignment, lost contact, and timeout are failures. Visual appearance alone does not reliably reveal the fixture’s mechanical state.
Plate wiping
A sponge must maintain contact with a tilted submerged plate and remove the marked residue. Missing the plate, losing contact, incomplete residue removal, and timeout are failures. The task tests sustained contact and surface alignment.
A.2. Complete result breakdown
Table 3 preserves every normal, adversarial, and averaged result. The full model is strongest on the reported task means, but does not strictly beat every ablation in every condition: normal spoon pickup ties the tactile/no-augmentation model at 0.81, and adversarial water sampling ties it at 0.56.
Where a task has multiple adversarial conditions, their results are averaged before normal/adversarial averaging. The table reports aggregate rates rather than per-seed variability or confidence intervals.
A.3. Attention visualization
Attention rollout averages heads, adds the residual identity, row-normalizes, and multiplies layer matrices. The current-output readout token supplies scores for visual and tactile token groups; all three original equation displays are retained.
The tactile token contains both contact and immersion-depth channels. Each modality’s trajectory is independently min–max normalized and smoothed over five frames. These plots are qualitative attribution proxies, as the appendix explicitly states, and provide neither causal evidence nor a calibrated comparison of absolute reliance between modalities.
Equations
B.1. Sensor construction
Tips use EPU40 elastomer, approximately Shore A 72, printed on a Carbon M2 and sealed with Sil-Poxy. Air channels are 1 mm in diameter and lattice structures 2 mm. DLHR-L05D-E1BD-C-NAV8 differential transducers sit near the finger roots.
Tactile tubing has 1.59 mm inner and 3.18 mm outer diameter. Depth tubing uses larger 3.18 mm inner and 6.35 mm outer diameters to reduce flow resistance along the sensing pathway.
B.2. Augmentation implementation
The retained equation gives the order: translation, color jitter, reflection-like patches, then refraction-like zoom. Table 4 retains all parameter ranges. Translation is up to fifteen pixels with a common crop across temporal history; color jitter probability is 0.9.
Each submerged-domain augmentation has probability 0.3. Patches target about 6% of image area, with brightness sampled from 0.3–0.8 and scattering from 0.2–0.8. Zoom is sampled from 0.95–1.05. One stochastic augmented sequence is generated per forward pass; the multiple variants in Figure 4 represent possible outcomes rather than a fixed expanded dataset.
Equation
B.3. Network architecture
The ImageNet-pretrained ResNet-18 uses spatial softmax with 32 keypoints. Inputs and actions are normalized from dataset statistics. Table 5 retains the architecture as reported: five-step history, 64-dimensional tokens, four Transformer layers, six attention heads, and five mixture-density components.
Transformer dropout is 0.1. Tactile policies use modality dropout of 0.2, while vision-only variants use zero modality dropout. The reported token and head dimensions are preserved without inferring an unreported attention projection implementation.
Training details
Forty demonstrations per task are collected with SpaceMouse teleoperation and split 90%/10% at sequence level. AdamW uses learning rate 0.0001, weight decay 0.01, batch size 32, and 101 epochs with 300 gradient steps each. Cosine annealing reaches 0.00001; validation occurs every five epochs and checkpoints every twenty.
The public project hosts an anonymous submission-format PDF; author names and the University of Texas at Austin affiliation are verified from the project page. The edition preserves that PDF’s complete figures, tables, equations, and appendix.