One email each morning — yesterday's AI, sortedGet it in your inbox

METAL LAB

CoToGrasp: Contact-Topology-Conditioned Dexterous Grasp Synthesis via Canonical Workspace Learning

arXiv:2608.197762026-08-19

Teaching robot hands to grasp objects the right way, not just any way that works

Most dexterous robot grasp planners only care about whether a grasp is stable, so they overwhelmingly default to full-hand enveloping grips even when a task needs a precise pinch or a tool-specific hold. CoToGrasp instead trains a model to reproduce 21 human-inspired grasp types (precision, power, and object-specific) by learning only the robot gripper's own contact geometry, with no object-specific data at all, so it generalizes to unseen objects immediately. Tested on the large DexGraspNet dataset and on a real robot arm, it matched requested grasp types more faithfully than prior taxonomy-based methods while staying physically stable.

What they did

  1. Existing grasp planners mainly check whether an object can be held stably, which causes them to collapse toward generic enveloping power grasps and fail to produce the specific precision or tool-use grips that real tasks need.
  2. CoToGrasp builds a shared 'canonical workspace' anchored to the gripper itself rather than the object, letting it learn which parts of the hand can contact which kinds of surfaces independent of what object is in front of it.
  3. Training uses only the gripper's own surface geometry paired with 21 human-derived grasp taxonomy templates, generating 210,000 training samples purely geometrically, without any object meshes or physics simulation.
  4. At inference, the target object is transformed into this gripper-centric space, and a network combining DGCNN, a Transformer encoder, and a CVAE predicts which hand zones should touch the object, followed by a validation step and energy-based optimization to compute final joint angles.
  5. On the DexGraspNet benchmark, CoToGrasp beat the taxonomy-aware baseline Dexonomy in both stability and how accurately it matched the requested grasp type, and real-world tests on an Allegro Hand with YCB objects confirmed the grasps are physically executable.
Figure 1: Contact-Topology-Conditioned Grasp Synthesis. Given a desired semantic contact-topology condition (top left) – categorized into Precision, Object-Specific (highly constrained topologies tailored for specific tool use) or Power functional groups – and a novel, unseen object (bottom left), our framework synthesizes functionally diverse and physically stable grasps (right). Rather than learning contact topologies directly on the object geometry, we project local object features into a feature-based canonical workspace. This unified spatial representation effectively decouples the functional intent from the specific object identity. Within this workspace, we learn a latent manifold (center) that models the intrinsic contact capabilities of the gripper, enabling zero-shot generalization to diverse target geometries.
Figure 1: Contact-Topology-Conditioned Grasp Synthesis. Given a desired semantic contact-topology condition (top left) – categorized into Precision, Object-Specific (highly constrained topologies tailored for specific tool use) or Power functional groups – and a novel, unseen object (bottom left), our framework synthesizes functionally diverse and physically stable grasps (right). Rather than learning contact topologies directly on the object geometry, we project local object features into a feature-based canonical workspace. This unified spatial representation effectively decouples the functional intent from the specific object identity. Within this workspace, we learn a latent manifold (center) that models the intrinsic contact capabilities of the gripper, enabling zero-shot generalization to diverse target geometries.
Figure 2: CoToGrasp Method Overview. The proposed framework operates in two distinct phases. Top (Object-Agnostic Training): The model learns an intrinsic, gripper-centric contact manifold within a canonical feature-based workspace, independent of object geometry. Bottom (Grasp Synthesis): At inference, a target object is transformed into the canonical frame. The network’s contact-topology-conditioned prediction is strictly filtered through a validation pipeline before energy-based optimization aligns the gripper to yield the final stable grasp (Q∗).
Figure 2: CoToGrasp Method Overview. The proposed framework operates in two distinct phases. Top (Object-Agnostic Training): The model learns an intrinsic, gripper-centric contact manifold within a canonical feature-based workspace, independent of object geometry. Bottom (Grasp Synthesis): At inference, a target object is transformed into the canonical frame. The network’s contact-topology-conditioned prediction is strictly filtered through a validation pipeline before energy-based optimization aligns the gripper to yield the final stable grasp (Q∗).
Table 1: Comparison with taxonomy-unaware baselines. CoToGrasp achieves the highest semantic entropy (HT​C) and generation speed, overcoming the functional mode collapse typical of unconditioned planners.
MethodObject-Agnostic TrainingSR ↑HT​C ↑Speed (sec. / grasps)Diversity (avg.) ↑
𝐭 (m)R (rad)Q (rad)
DFC [26]✓72.150.7389>18000.06071.4240.3579
GenDexGrasp [24]✗71.150.595614.650.05191.4160.2567
DRO-Grasp [43]✗63.300.65041.720.05461.5150.2892
GOAG [32]✓77.900.65270.200.04791.4010.3170
CoToGrasp✓36.940.830.110.06741.49270.3458
Figure 3: Semantic Grasp Taxonomy and Contact Mapping. (Left) Correspondences between the classical Feix [11] (F) taxonomy (top), the haptic Gonzalez [12] (M) taxonomy (middle row) and our derived point cloud contact templates 𝒜m (bottom row). We categorized the 21 templates into three distinct functional groups: Precision, Power and Object-Specific (highly constrained topologies tailored for specific tool use). (Right) The 22 anatomical contact zones defined by Gonzalez (top) and the direct surjective mapping (ζ) onto our discrete gripper handprint ℋ (bottom).
Figure 3: Semantic Grasp Taxonomy and Contact Mapping. (Left) Correspondences between the classical Feix [11] (F) taxonomy (top), the haptic Gonzalez [12] (M) taxonomy (middle row) and our derived point cloud contact templates 𝒜m (bottom row). We categorized the 21 templates into three distinct functional groups: Precision, Power and Object-Specific (highly constrained topologies tailored for specific tool use). (Right) The 22 anatomical contact zones defined by Gonzalez (top) and the direct surjective mapping (ζ) onto our discrete gripper handprint ℋ (bottom).
Figure 4: Functional contact topology distribution across taxonomy-unaware planners. The histogram illustrates the distribution of grasps generated by unconditioned baselines compared to CoToGrasp on the Multidex objects set. The unknown category represents physically stable grasps with unnatural contact patterns that fail to match any contact topology. Notably, unconditioned baselines exhibit a severe generative bias (mode collapse) toward enveloping power grasps (red box).
Figure 4: Functional contact topology distribution across taxonomy-unaware planners. The histogram illustrates the distribution of grasps generated by unconditioned baselines compared to CoToGrasp on the Multidex objects set. The unknown category represents physically stable grasps with unnatural contact patterns that fail to match any contact topology. Notably, unconditioned baselines exhibit a severe generative bias (mode collapse) toward enveloping power grasps (red box).
Table 2: Taxonomy-aware grasp synthesis. CoToGrasp outperforms the baseline in both physical stability (SR) and semantic accuracy (TC), particularly on highly constrained precision grasps.
MethodSR (%)HS​RTC (%)HT​C
PowerPrecisionObj. Spe.Avg. Topo.Avg. Obj.
Dexonomy [5]27.1612.3619.6221.1323.800.9114.280.77
CoToGrasp29.7522.7125.5026.7227.560.9617.180.84
CoToGrasp (w/o Label-Consistency)25.1114.7720.8521.1422.970.9414.450.81
CoToGrasp (w/o Force-Closure)26.9014.8721.0822.0823.650.9516.260.81
CoToGrasp (No Check)25.0914.7320.5821.0623.000.9414.720.81
Figure 5: Real-World kinematic validation. CoToGrasp synthesizes diverse, topology-compliant grasps that are physically executable on a physical Allegro Hand using YCB [4] objects. The target contact topologies (indicated above each frame) demonstrate the physical viability of the generated grasps across both precision and power categories.
Figure 5: Real-World kinematic validation. CoToGrasp synthesizes diverse, topology-compliant grasps that are physically executable on a physical Allegro Hand using YCB [4] objects. The target contact topologies (indicated above each frame) demonstrate the physical viability of the generated grasps across both precision and power categories.
Figure 6: Handprint areas and workspace constitution. Left: Discretized handprints of the Shadow and Allegro hands, illustrating the manually defined anatomical zone divisions. Right: A three-quarter view of the Shadow Hand’s workspace.
Figure 6: Handprint areas and workspace constitution. Left: Discretized handprints of the Shadow and Allegro hands, illustrating the manually defined anatomical zone divisions. Right: A three-quarter view of the Shadow Hand’s workspace.
Table 3: Per-Topology results: 3 Power grasps (M13, M18, M21), 2 Precision (M2, M6) and 1 Object-Specific (M11). †Indicates artificially inflated scores due to mode collapse, where Dexonomy defaults to unverified enveloping grasps.
MethodSR (%)
M2M6M11M13M18M21
Dexonomy [5]10.515.260.5†20.329.637.2†
CoToGrasp30.321.729.829.631.333.5
Figure 7: Taxonomy transfer mapping for anthropomorphic grippers. The handprints of the Shadow Hand (left) and the Allegro Hand (right) are segmented into manually defined, corresponding anatomical zones (𝒜1–𝒜21).
Figure 7: Taxonomy transfer mapping for anthropomorphic grippers. The handprints of the Shadow Hand (left) and the Allegro Hand (right) are segmented into manually defined, corresponding anatomical zones (𝒜1–𝒜21).
Figure 8: t-SNE visualization of features after canonical workspace projection.
Figure 8: t-SNE visualization of features after canonical workspace projection.
Table 4: Implementation Details and Network Hyperparameters for the CoToGrasp framework.
Module / ParameterNotationValue / Size
Training & Optimization
Batch Size–32
Learning Rate–10−5
Training Epochs–50
KLD Regularization Weightβ0.1
Attention Factorα3.0
Workspace & Geometric Projection
Workspace ResolutionNW8,192
Projection kNNk5
Aligned Distance Scalingγ2.0
Pointwise Feature Extraction
Local Graph kNNK16
Hidden Layer Sizes–[12,64,64,128,256,512]
Output Feature Dimensionℱ𝒫,ℱ𝒲1,024
Conditioning Embeddings
Topology Embedding Dim.ℱ𝒯64
Label Embedding (MLP)ℱΛ[128,64]
Self-Attention Modules
Transformer Encoder Feature Dim.Φ1088
Transformer Encoder Blocks–4
Transformer Encoder Heads–8
Set Transformer (Pooling) Heads–8
Global Latent Descriptor Dim.𝒵1,024
CVAE & Latent Space
Latent Encoder Hidden Dim.–512
Latent Variable Dimensionψ64
AdaLN Decoder Output ClassesN+123
Figure 9: Topology-Conditioned Pose Sampling. For specific grasp topologies (such as M4 and M12), the initial object pose is sampled within a restricted kinematic region. Middle: The template’s full active sub-workspace is shown in light blue, while the truncated sub-workspace – filtered for reachability and palm clearance – is highlighted in red. Right: Examples of the initialized object point cloud 𝒪~ (green) successfully placed within this feasible region after the sampled spatial transformation.
Figure 9: Topology-Conditioned Pose Sampling. For specific grasp topologies (such as M4 and M12), the initial object pose is sampled within a restricted kinematic region. Middle: The template’s full active sub-workspace is shown in light blue, while the truncated sub-workspace – filtered for reachability and palm clearance – is highlighted in red. Right: Examples of the initialized object point cloud 𝒪~ (green) successfully placed within this feasible region after the sampled spatial transformation.
Figure 10: Illustration of metric-induced misclassification. While CoToGrasp strictly respects the target contact topology, the kinematic optimization may result in incidental contacts where an adjacent phalanx rests against the object surface. Despite the repulsive term in our optimization energy function, these phalanges often cannot be pushed away due to inherent kinematic constraints. For instance, the left grasp illustrates an intended M3 pinch reclassified as M12, while the right shows an M4 grasp reclassified as M15. Although the intended contacts are successfully achieved and the grasps remain physically stable, these incidental contacts trigger a strict reclassification by our automated pipeline.
Figure 10: Illustration of metric-induced misclassification. While CoToGrasp strictly respects the target contact topology, the kinematic optimization may result in incidental contacts where an adjacent phalanx rests against the object surface. Despite the repulsive term in our optimization energy function, these phalanges often cannot be pushed away due to inherent kinematic constraints. For instance, the left grasp illustrates an intended M3 pinch reclassified as M12, while the right shows an M4 grasp reclassified as M15. Although the intended contacts are successfully achieved and the grasps remain physically stable, these incidental contacts trigger a strict reclassification by our automated pipeline.
Table 5: Feature Alignment (Cosine Similarity).
Raw DGCNNWorkspace+ Attn.
Matched Pairs0.350.610.66
Random Pairs0.230.330.27
Figure 11: Histogram comparing the frequencies of effective topologies and attempted topologies (as defined Sec. 4.2) among all stable grasps generated by CoToGrasp (top) and Dexonomy [5] (bottom).
Figure 11: Histogram comparing the frequencies of effective topologies and attempted topologies (as defined Sec. 4.2) among all stable grasps generated by CoToGrasp (top) and Dexonomy [5] (bottom).
CoToGrasp: Contact-Topology-Conditioned Dexterous Grasp Synthesis via Canonical Workspace Learning figure 11
Table 6: Topology compliance ablation.
LabelEval.HS​RTC (%)HT​C
ConsistencyIsaac
✓✓0.9617.180.84
✓✗21.920.53
✗✓0.9414.450.81
✗✗19.340.50
Table 7: Object-Level Analysis. Evaluating performance across geometric complexity (c=Vo​b​j/Vh​u​l​l). CoToGrasp shows superior retention of both physical stability (SR) and semantic compliance (TC) on challenging non-convex objects.
Object ComplexityMetricCoToGraspDexonomy [5]
Convex → Non-ConvexSR Retained58.72%37.42%
Severe Concavities (c<0.4, ∼4% data)SR18.30%12.05%
TC19.17%8.94%

Why it matters

As robots move toward high-degree-of-freedom hands for tasks like tool use or handle grasping, simply avoiding drops isn't enough; the hand must grasp in a way that supports the downstream task. This approach removes the need for expensive object-specific grasp datasets, making functional, task-aware grasping practical for new objects without retraining.

Terms in this paper

  • Gripper · the robotic hand or claw that physically holds an object
  • Contact Topology · a pattern defining which parts of the hand touch which parts of the object, borrowed from human grasp classification systems
  • Zero-shot generalization · the ability to work correctly on new, previously unseen objects without additional training
  • CVAE (Conditional Variational Auto-Encoder) · a neural network that generates varied outputs conditioned on given input information, in a probabilistic way
  • Mode collapse · when a generative model produces only a few repetitive patterns instead of diverse outputs

Original abstract (English)

Current dexterous grasp planners primarily optimize for physical stability, focusing on whether an object can be grasped rather than how it should be grasped to support downstream functional tasks. However, conditioning grasp synthesis on specific human grasp taxonomies typically requires prohibitively expensive, object-annotated datasets. To address these limitations, we propose CoToGrasp, a novel generative framework that synthesizes diverse, stable grasps strictly conditioned on specific contact topologies. To bypass the data collection bottleneck, CoToGrasp is trained entirely in an object-agnostic manner. We introduce a feature-based canonical workspace that projects local object features into a unified gripper-centric domain, effectively decoupling the semantic functional intent from the arbitrary object geometry. By learning the intrinsic contact manifold of the gripper within this workspace, our model achieves zero-shot generalization to unseen objects at inference. Extensive evaluations on the large-scale DexGraspNet dataset demonstrate that CoToGrasp achieves state-of-the-art performance, outperforming existing taxonomy-guided planners. Finally, we demonstrate the physical viability and kinematic feasibility of our synthesized contact topologies on a physical robot platform. Code is available on our project website https://cea-list.github.io/cotograspweb/ .

Authors · Julien Merand

Read on arXiv

Latest papers

All papers →

Latest from METAL LAB

Figures: Julien Merand et al., arXiv:2608.19776, CC BY 4.0