Overview
Project page. This edition follows the complete sixteen-page, 10 October 2026 v2, including nine figures, six numbered tables, both numbered equations and all 54 references.
At a glance
KPGrasp generates dexterous grasps as Euclidean hand keypoints, conditioned on object geometry. A Transformer flow model learns from large analytically synthesized datasets with standard flow-matching supervision. Likelihood ranks candidates; rigid registration and inverse kinematics recover executable hand configurations. Strong simulation results and a retrained hardware deployment support this approach, with data quality, partial visibility and kinematic/trajectory failures still limiting performance.
1 Introduction
Large analytic grasp datasets offer a way to amortize expensive physics-based synthesis. KPGrasp asks whether the learned distribution can provide sufficient contact priors without contact-specific training losses or contact-based refinement. Its output places hand keypoints in the same Cartesian frame as the object, making spatial relationships available directly to attention.
The method still relies on analytic data generation, kinematic projection and candidate selection. It removes contact optimization from the generator, not all geometric computation from deployment. The headline Dexonomy gain is 47.4 percentage points, from 28.9% to 76.3%, rather than a 47.4% relative increase.
2 Related Work
2.1 Data-Driven Dexterous Grasp Generation
Generative approaches capture multiple valid grasps better than deterministic regression. DexGrasp Anything, GraspTTA and DexDiffuser use physics/contact guidance, adaptation or auxiliary evaluation. D(R,O) Grasp explicitly models hand-object relationships at quadratic pairwise cost. KPGrasp instead learns a keypoint distribution from high-quality grasps, retaining model-internal likelihood ranking.
2.2 Hand Parameterization
Conventional outputs combine wrist pose and joint angles, requiring treatment of rotations, mixed coordinates and kinematic error propagation. Geometric flow models and equivariant networks address these issues differently. Keypoint manipulation policies and hand-tracking work motivate Cartesian outputs; here they become the distribution modeled by a grasp generator.
3 Preliminary
Flow matching regresses the velocity of a linear path from Gaussian noise to data. Equation 1 retains the squared-error objective; integrating the learned field produces samples. Equation 2 gives the continuous change-of-variables density relation.
Following continuous normalizing flows, a Hutchinson probe estimates divergence using a vector-Jacobian computation. The trace estimator is unbiased at a fixed state, but the deployed five-step integration gives an approximate model score. Learned density is not physical stability probability, and need not equal the true data density written in the ideal formula.
Equations 1–2
| (1) |
| (2) |
4 Method
4.1 Keypoint Grasp Parameterization
Twenty-one Shadow Hand keypoints describe joints, fingertips and rigid-palm anchors in the object point-cloud frame. Forward kinematics converts training grasps into these coordinates. Five palm points support SVD rigid registration; Mink inverse kinematics then recovers joint angles from the remaining points.
This projection enforces hand kinematic compatibility without consulting the object for contact, penetration or force closure. Palm registration error is below 0.4 mm for half the grasps and 0.99 mm for 99%; mean IK reconstruction error is about 0.7 mm. Rare invalid configurations remain.
Equation
4.2 Conditional Keypoint Flow Model
A Minkowski sparse-convolution encoder processes 1,024 object points; farthest-point sampling yields 128 geometry tokens of width 512. Each noisy keypoint becomes a token. Eight DiT blocks alternate hand self-attention and object cross-attention, using eight heads, FFN width 2,048 and time-conditioned adaptive normalization.
The 67.19-million-parameter model trains for 200,000 steps with batch 4,096, Muon at learning rate 2×10⁻⁴, and eight RTX 4090 GPUs, taking about two days. Inference generates one hundred candidates with five Euler steps, estimates scores using one Rademacher probe per trajectory and retains the top ten before rigid registration/IK. The score ranks learned plausibility without an external discriminator.
5 Experiment
Simulation uses complete object point clouds to isolate grasp synthesis. The real system is retrained on simulated single-view observations for a Sharpa hand. Baselines retain their native selection/refinement pipelines; those without rankers are not given an added ranking module. These comparisons assess complete reported methods, not uniformly identical inference budgets.
5.1 Benchmark on Dexonomy
Dexonomy contains 9.5 million grasps over 10.7 thousand objects and 31 grasp types. Evaluation follows its object-level split and six-direction external-force test in MuJoCo. GSR scores retained grasp proposals; OSR requires at least one successful proposal for an object. Contact consistency, penetration and diversity follow the official definitions.
KPGrasp reaches 76.3% GSR, 97.3% OSR, 5.6 mm contact-distance consistency and 2.4 mm penetration. The directly comparable Dexonomy baseline reaches 28.9% GSR; its extra-classifier variant reaches 63.9%, giving a 12.4-point rather than relative-percent gap. KPGrasp ties DexGrasp Anything’s diversity value 23.3 instead of strictly winning that metric.
The qualitative montage illustrates cleaner generated contacts, but selected images do not independently establish every grasp’s stability. Training-data synthesis and simulator force tests carry the physical prior.

Original selectable table
| Method | Penetration Depth (mm) | Success Rate (%) | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GRAB | RealD | MD | UDG | DGN | Avg. | GRAB | RealD | MD | UDG | DGN | Avg. | |
| UniDexGrasp [47] | 37.4 | 39.0 | 13.5 | 24.5 | 31.9 | 29.3 | 20.8 | 27.1 | 21.6 | 23.7 | 33.9 | 25.4 |
| GraspTTA [23] | 51.4 | 40.1 | 19.0 | 21.2 | 24.5 | 31.2 | 14.4 | 13.3 | 30.3 | 21.0 | 18.6 | 19.5 |
| SceneDiffuser [22] | 41.1 | 42.0 | 14.6 | 25.1 | 31.0 | 30.8 | 39.1 | 21.7 | 69.8 | 28.3 | 26.6 | 37.1 |
| UGG [33] | 33.2 | 34.4 | 10.3 | 24.5 | 25.2 | 25.5 | 42.7 | 32.7 | 55.3 | 46.0 | 46.9 | 44.7 |
| DexGraspAnything [51] | 30.4 | 27.7 | 11.4 | 18.8 | 17.8 | 21.2 | 57.9 | 44.8 | 79.1 | 53.1 | 57.5 | 58.5 |
| KPGrasp (Ours) | 20.8 | 9.6 | 6.6 | 13.0 | 14.3 | 12.9 | 59.8 | 67.1 | 62.2 | 62.3 | 67.5 | 63.8 |
5.2 Benchmark on DexGrasp Anything
The Dexonomy-trained model transfers to five DexGrasp Anything object sets without fine-tuning, with training/evaluation object overlap excluded. This protocol uses IsaacGym and point-cloud-based penetration, so its numbers are not directly interchangeable with Dexonomy metrics.
Mean success is 63.8% versus DGA 58.5%, and mean penetration is 12.9 versus 21.2 mm. KPGrasp wins four of five success columns; on MultiDex it trails DGA 62.2% versus 79.1%, and SceneDiffuser also scores higher at 69.8%.
Retraining DGA on Dexonomy changes its mean only to 59.1%; removing its contact refinement gives 46.6% on original training data and 46.8% on Dexonomy. Data source alone therefore does not explain the comparison, while Appendix E shows that data quality remains essential for KPGrasp itself.
Original selectable table
| Method | GSR() | OSR() | CDC() | PD() | Diversity |
| Pose-Joint (w/ Euclidean Flow) | 62.5 | 92.0 | 10.3 | 3.7 | 25.7 |
| Pose-Joint (w/ Flow) | 55.0 | 93.9 | 11.8 | 5.7 | 23.8 |
| Pose-Joint (w/ 24 tokens) | 63.5 | 94.6 | 9.7 | 3.3 | 25.8 |
| Pose-Joint (w/ 34 tokens) | 66.1 | 95.1 | 9.5 | 3.2 | 25.7 |
| Keypoint (w/ Single Token) | 55.4 | 93.2 | 10.1 | 8.6 | 23.7 |
| Keypoint (w/o likelihood ranking) | 67.0 | 96.4 | 7.4 | 4.4 | 23.2 |
| Keypoint (Ours) | 76.3 | 97.3 | 5.6 | 2.4 | 23.3 |

5.3 Ablation Study
Pose-joint outputs with Euclidean rotation projection obtain 62.5% GSR, versus 55.0% with an SE(3) flow. Giving pose-joint models 24/34 tokens yields 63.5%/66.1%; collapsing keypoints to one token gives 55.4%. Representation and spatial tokenization both matter; token count alone does not explain 76.3%.
Without likelihood ranking, success falls to 67.0%, a 9.3-point decrease, while the diversity metric slightly improves from 23.3 to 23.2. Ranking improves selected quality rather than every metric.
Figure 5 varies data, block count and batch size. Larger batches at fixed steps also see more examples. A sample-count-matched batch-512 run takes 1.6 million steps/about thirteen days and reaches 73.5%, versus 76.3% for batch 4,096. Sixteen blocks reach 77.6%; the eight-block default balances capacity and cost. These sweeps do not establish a universal scaling law.
5.4 Real World Experiments
The Galbot platform uses a Sharpa right hand and left-arm RealSense D405 camera. Retargeted analytic synthesis produces 3.3 million tabletop grasps over five thousand objects with simulated partial clouds; KPGrasp is retrained for this setup.
GroundedSAM segments the object, then likelihood-ranked candidates undergo collision filtering and trajectory interpolation. Five trials on each of twenty objects yield 83/100 successful lifts. This is deployability evidence for a retrained, filtered pipeline, not zero-shot transfer of the Shadow Hand model or a broad hardware benchmark. Small pose/calibration errors can invalidate sensitive grasps.
5.5 Inference Time Analysis
Reported inference includes network sampling, likelihood ranking and IK: four objects per batch, one hundred candidates per object and ten retained per object give 0.032 seconds per returned grasp. This is amortized throughput, not single-grasp request latency or the full perception-to-executed-lift cycle. IK additionally uses 64 CPU workers.
The paper compares roughly 2.8 seconds per DGA grasp with contact refinement and estimates about 24 successful grasps per second by multiplying throughput by 76.3% simulated success. Its comparison with analytic dataset generation uses separately reported hardware/workload statistics; it is not a matched end-to-end experiment.
6 Limitation
Quality depends on the coverage and correctness of analytic training data. Occlusion and sensing errors degrade grasps; likelihood is not a stability certificate, IK can fail, and execution needs collision handling. Real tests cover only twenty objects and one retrained hand/platform. A unified model across different hands remains open.
7 Conclusion
Cartesian keypoint flows plus large grasp datasets provide an effective learned proposal distribution. The strongest evidence is high selected-grasp quality in simulation and successful retrained hardware deployment, with explicit dependence on dataset quality, kinematic recovery and execution filtering. The full gallery below retains all 48 examples.
Appendix A Inverse-Kinematics Module Analysis
The appendix measures how closely projected robot configurations reproduce generated keypoints and reports the small but nonzero failure tail.
Implementation Details
SVD aligns canonical and predicted palm anchors to recover wrist pose. Position-only keypoint tasks in Mink with the quadprog backend solve the remaining configuration, integrating IK velocity for at most five iterations. This is kinematic optimization rather than object-contact refinement.
Evaluation Settings
The test covers 128,340 attempts: 2,139 objects × six scales × ten proposals. Forward kinematics reconstructs solved keypoints, and Euclidean discrepancies measure agreement with the generated points. These reconstruction errors are distinct from grasp success and hand-object penetration.
Results
Average mean/max errors are 0.669/1.861 mm; 97.37% have maximum error at most 5 mm. Half have mean error below 0.5 mm, and 99% below 2.5 mm. About 0.2% exhibit clear failures such as backward-folded fingers, illustrated in Figure 8.
Adding feasibility constraints to intermediate terminal estimates reduces that failure rate to 0.08% but raises GSR by only 0.4 percentage points. Amortized IK runtime is 0.002 seconds with 64 workers on a 32-core AMD EPYC CPU, not serial per-request latency.
Original selectable table
| Metric | Value |
|---|---|
| IK max keypoint error 5 mm | 97.37% |
| Avg. mean keypoint error | 0.669 mm |
| Avg. max keypoint error | 1.861 mm |
| 64-worker amortized runtime | 0.002 s |
Appendix B Differentiable FK loss for Pose-Joint variant
A pose-joint model trained only through differentiable forward-kinematics error produces grasp-like configurations but roughly 1% GSR. Adding that loss to standard flow matching gives 66.0%, essentially the 66.1% pose-joint baseline. Cartesian supervision alone does not reproduce the benefit of modeling the flow directly over spatial keypoint tokens.
Appendix C Partial Observation Results
Retraining on single-view point clouds gives 67.4% simulated GSR, 8.9 percentage points below the complete-geometry result 76.3%. Occluded geometry is the authors’ explanation; this is a separate simulated partial-observation comparison, not the 83% physical lift metric.
Appendix D Comparison of Different Sampling Steps
Naive one-step flow sampling nearly fails. A separately distilled FlowMap using Lagrangian Self-Distillation reaches 52.7% at 0.010 seconds per grasp. Three/five/seven Euler steps give 66.5%/76.3%/76.0% at 0.024/0.032/0.040 seconds. Five is the selected quality-throughput trade-off.
Scores use discretized divergence integration; the one-step variant approximates the integral with average velocity. Its distillation and score approximation differ from merely reducing the original model’s solver steps, and finite-step scores are not exact likelihoods of an ideal continuous flow.
Original selectable table
| Method | Success Rate (%) | Inference Time (s) |
|---|---|---|
| Ours (1-step FlowMap) | 52.7 | 0.01 |
| Ours (3-step flow matching) | 66.5 | 0.024 |
| Ours (5-step flow matching) | 76.3 | 0.032 |
| Ours (7-step flow matching) | 76.0 | 0.04 |
Appendix E Additional Results of DexGrasp Anything Benchmark
Training KPGrasp separately on the original five benchmark datasets yields mean success 47.8%, below DGA’s 58.5%; Dexonomy-trained KPGrasp reaches 63.8%. The original training grasps themselves score only 42.8% under the DGA evaluation, suggesting filtering/evaluation mismatch and uneven data quality.
This qualifies the headline: the simple generator benefits substantially from high-quality large-scale data. The comparison does not isolate dataset size, quality and protocol mismatch as independent causes, and DGA’s explicit priors can be more helpful under weaker training data.
Appendix F Additional Details of Real World Experiment
The hardware pipeline changes the hand, retrains on tabletop partial observations and adds collision checks around grasp execution.
F.1 Sharpa Hand Keypoint Annotation
Sharpa receives its own twenty-one-keypoint annotation at mechanical joints, fingertips and rigid-palm anchors. Reusing the representation design does not imply the same trained network transfers unchanged between hand embodiments.

F.2 Implementation Details
Shadow Hand templates are retargeted to Sharpa before analytic regeneration. Simulated cameras vary height 30–40 cm and horizontal distance 30–50 cm. At deployment, object segmentation and an estimated table plane align observations with this training distribution.
Candidate interpolation is filtered for collisions. Documented hazards include near-table hand poses, the unused right-wrist camera hitting the table after arm IK, and paths crossing the object when start/target grasps lie on opposite sides. Collision-aware planning and larger synthesis margins are proposed future improvements; grasp quality alone does not ensure a safe approach trajectory.