每天早上一封邮件，把昨天的 AI 梳理好订阅邮件

METAL LAB

CoToGrasp: Contact-Topology-Conditioned Dexterous Grasp Synthesis via Canonical Workspace Learning

arXiv:2608.197762026-08-19

让机器人的手不再只是随便抓住东西,而是按需要的方式去抓

目前大多数灵巧机械手的抓取规划方法只关心能不能稳稳抓住物体,结果绝大多数都变成用整只手包裹式抓取,难以完成需要精确捏取或特定工具握法的任务。CoToGrasp通过只学习机械手自身的接触几何、完全不依赖物体标注数据,让模型能够复现21种源自人类抓握方式的接触模式(精确抓、力量抓、专用工具抓),从而对没见过的新物体也能立刻适用。在大规模DexGraspNet数据集和真实机械臂实验中,该方法比现有基于分类体系的方法更准确地复现了指定的抓取方式,同时保持了较高的稳定性。

他们做了什么

  1. 现有抓取规划方法主要只判断抓取是否稳定,导致模型普遍退化为包裹式力量抓取,无法产生任务所需的精确捏取或特定工具握法(即所谓的模式坍塌)。
  2. CoToGrasp构建了一个以机械手自身为中心、而非以物体为中心的统一工作空间,使模型可以在不依赖具体物体形状的情况下学习手的哪些部位能接触哪些类型的表面。
  3. 训练阶段只使用机械手自身的表面几何数据,搭配21种源自人类的抓握分类模板,纯几何方式生成21万条训练样本,完全不需要物体网格或物理仿真。
  4. 推理阶段将目标物体变换到这个以手为中心的坐标系中,再由结合DGCNN、Transformer编码器和CVAE的网络预测应接触的手部区域,随后经过有效性校验和基于能量的优化计算出最终关节角度。
  5. 在DexGraspNet基准测试中,CoToGrasp在抓取稳定性和抓取类型匹配准确度上均优于分类体系基线方法Dexonomy,并在Allegro机械手抓取YCB物体的真实实验中验证了其物理可执行性。
Figure 1: Contact-Topology-Conditioned Grasp Synthesis. Given a desired semantic contact-topology condition (top left) – categorized into Precision, Object-Specific (highly constrained topologies tailored for specific tool use) or Power functional groups – and a novel, unseen object (bottom left), our framework synthesizes functionally diverse and physically stable grasps (right). Rather than learning contact topologies directly on the object geometry, we project local object features into a feature-based canonical workspace. This unified spatial representation effectively decouples the functional intent from the specific object identity. Within this workspace, we learn a latent manifold (center) that models the intrinsic contact capabilities of the gripper, enabling zero-shot generalization to diverse target geometries.
Figure 1: Contact-Topology-Conditioned Grasp Synthesis. Given a desired semantic contact-topology condition (top left) – categorized into Precision, Object-Specific (highly constrained topologies tailored for specific tool use) or Power functional groups – and a novel, unseen object (bottom left), our framework synthesizes functionally diverse and physically stable grasps (right). Rather than learning contact topologies directly on the object geometry, we project local object features into a feature-based canonical workspace. This unified spatial representation effectively decouples the functional intent from the specific object identity. Within this workspace, we learn a latent manifold (center) that models the intrinsic contact capabilities of the gripper, enabling zero-shot generalization to diverse target geometries.
Figure 2: CoToGrasp Method Overview. The proposed framework operates in two distinct phases. Top (Object-Agnostic Training): The model learns an intrinsic, gripper-centric contact manifold within a canonical feature-based workspace, independent of object geometry. Bottom (Grasp Synthesis): At inference, a target object is transformed into the canonical frame. The network’s contact-topology-conditioned prediction is strictly filtered through a validation pipeline before energy-based optimization aligns the gripper to yield the final stable grasp (Q∗).
Figure 2: CoToGrasp Method Overview. The proposed framework operates in two distinct phases. Top (Object-Agnostic Training): The model learns an intrinsic, gripper-centric contact manifold within a canonical feature-based workspace, independent of object geometry. Bottom (Grasp Synthesis): At inference, a target object is transformed into the canonical frame. The network’s contact-topology-conditioned prediction is strictly filtered through a validation pipeline before energy-based optimization aligns the gripper to yield the final stable grasp (Q∗).
Table 1: Comparison with taxonomy-unaware baselines. CoToGrasp achieves the highest semantic entropy (HT​C) and generation speed, overcoming the functional mode collapse typical of unconditioned planners.
MethodObject-Agnostic TrainingSR ↑HT​C ↑Speed (sec. / grasps)Diversity (avg.) ↑
𝐭 (m)R (rad)Q (rad)
DFC [26]✓72.150.7389>18000.06071.4240.3579
GenDexGrasp [24]✗71.150.595614.650.05191.4160.2567
DRO-Grasp [43]✗63.300.65041.720.05461.5150.2892
GOAG [32]✓77.900.65270.200.04791.4010.3170
CoToGrasp✓36.940.830.110.06741.49270.3458
Figure 3: Semantic Grasp Taxonomy and Contact Mapping. (Left) Correspondences between the classical Feix [11] (F) taxonomy (top), the haptic Gonzalez [12] (M) taxonomy (middle row) and our derived point cloud contact templates 𝒜m (bottom row). We categorized the 21 templates into three distinct functional groups: Precision, Power and Object-Specific (highly constrained topologies tailored for specific tool use). (Right) The 22 anatomical contact zones defined by Gonzalez (top) and the direct surjective mapping (ζ) onto our discrete gripper handprint ℋ (bottom).
Figure 3: Semantic Grasp Taxonomy and Contact Mapping. (Left) Correspondences between the classical Feix [11] (F) taxonomy (top), the haptic Gonzalez [12] (M) taxonomy (middle row) and our derived point cloud contact templates 𝒜m (bottom row). We categorized the 21 templates into three distinct functional groups: Precision, Power and Object-Specific (highly constrained topologies tailored for specific tool use). (Right) The 22 anatomical contact zones defined by Gonzalez (top) and the direct surjective mapping (ζ) onto our discrete gripper handprint ℋ (bottom).
Figure 4: Functional contact topology distribution across taxonomy-unaware planners. The histogram illustrates the distribution of grasps generated by unconditioned baselines compared to CoToGrasp on the Multidex objects set. The unknown category represents physically stable grasps with unnatural contact patterns that fail to match any contact topology. Notably, unconditioned baselines exhibit a severe generative bias (mode collapse) toward enveloping power grasps (red box).
Figure 4: Functional contact topology distribution across taxonomy-unaware planners. The histogram illustrates the distribution of grasps generated by unconditioned baselines compared to CoToGrasp on the Multidex objects set. The unknown category represents physically stable grasps with unnatural contact patterns that fail to match any contact topology. Notably, unconditioned baselines exhibit a severe generative bias (mode collapse) toward enveloping power grasps (red box).
Table 2: Taxonomy-aware grasp synthesis. CoToGrasp outperforms the baseline in both physical stability (SR) and semantic accuracy (TC), particularly on highly constrained precision grasps.
MethodSR (%)HS​RTC (%)HT​C
PowerPrecisionObj. Spe.Avg. Topo.Avg. Obj.
Dexonomy [5]27.1612.3619.6221.1323.800.9114.280.77
CoToGrasp29.7522.7125.5026.7227.560.9617.180.84
CoToGrasp (w/o Label-Consistency)25.1114.7720.8521.1422.970.9414.450.81
CoToGrasp (w/o Force-Closure)26.9014.8721.0822.0823.650.9516.260.81
CoToGrasp (No Check)25.0914.7320.5821.0623.000.9414.720.81
Figure 5: Real-World kinematic validation. CoToGrasp synthesizes diverse, topology-compliant grasps that are physically executable on a physical Allegro Hand using YCB [4] objects. The target contact topologies (indicated above each frame) demonstrate the physical viability of the generated grasps across both precision and power categories.
Figure 5: Real-World kinematic validation. CoToGrasp synthesizes diverse, topology-compliant grasps that are physically executable on a physical Allegro Hand using YCB [4] objects. The target contact topologies (indicated above each frame) demonstrate the physical viability of the generated grasps across both precision and power categories.
Figure 6: Handprint areas and workspace constitution. Left: Discretized handprints of the Shadow and Allegro hands, illustrating the manually defined anatomical zone divisions. Right: A three-quarter view of the Shadow Hand’s workspace.
Figure 6: Handprint areas and workspace constitution. Left: Discretized handprints of the Shadow and Allegro hands, illustrating the manually defined anatomical zone divisions. Right: A three-quarter view of the Shadow Hand’s workspace.
Table 3: Per-Topology results: 3 Power grasps (M13, M18, M21), 2 Precision (M2, M6) and 1 Object-Specific (M11). †Indicates artificially inflated scores due to mode collapse, where Dexonomy defaults to unverified enveloping grasps.
MethodSR (%)
M2M6M11M13M18M21
Dexonomy [5]10.515.260.5†20.329.637.2†
CoToGrasp30.321.729.829.631.333.5
Figure 7: Taxonomy transfer mapping for anthropomorphic grippers. The handprints of the Shadow Hand (left) and the Allegro Hand (right) are segmented into manually defined, corresponding anatomical zones (𝒜1–𝒜21).
Figure 7: Taxonomy transfer mapping for anthropomorphic grippers. The handprints of the Shadow Hand (left) and the Allegro Hand (right) are segmented into manually defined, corresponding anatomical zones (𝒜1–𝒜21).
Figure 8: t-SNE visualization of features after canonical workspace projection.
Figure 8: t-SNE visualization of features after canonical workspace projection.
Table 4: Implementation Details and Network Hyperparameters for the CoToGrasp framework.
Module / ParameterNotationValue / Size
Training & Optimization
Batch Size–32
Learning Rate–10−5
Training Epochs–50
KLD Regularization Weightβ0.1
Attention Factorα3.0
Workspace & Geometric Projection
Workspace ResolutionNW8,192
Projection kNNk5
Aligned Distance Scalingγ2.0
Pointwise Feature Extraction
Local Graph kNNK16
Hidden Layer Sizes–[12,64,64,128,256,512]
Output Feature Dimensionℱ𝒫,ℱ𝒲1,024
Conditioning Embeddings
Topology Embedding Dim.ℱ𝒯64
Label Embedding (MLP)ℱΛ[128,64]
Self-Attention Modules
Transformer Encoder Feature Dim.Φ1088
Transformer Encoder Blocks–4
Transformer Encoder Heads–8
Set Transformer (Pooling) Heads–8
Global Latent Descriptor Dim.𝒵1,024
CVAE & Latent Space
Latent Encoder Hidden Dim.–512
Latent Variable Dimensionψ64
AdaLN Decoder Output ClassesN+123
Figure 9: Topology-Conditioned Pose Sampling. For specific grasp topologies (such as M4 and M12), the initial object pose is sampled within a restricted kinematic region. Middle: The template’s full active sub-workspace is shown in light blue, while the truncated sub-workspace – filtered for reachability and palm clearance – is highlighted in red. Right: Examples of the initialized object point cloud 𝒪~ (green) successfully placed within this feasible region after the sampled spatial transformation.
Figure 9: Topology-Conditioned Pose Sampling. For specific grasp topologies (such as M4 and M12), the initial object pose is sampled within a restricted kinematic region. Middle: The template’s full active sub-workspace is shown in light blue, while the truncated sub-workspace – filtered for reachability and palm clearance – is highlighted in red. Right: Examples of the initialized object point cloud 𝒪~ (green) successfully placed within this feasible region after the sampled spatial transformation.
Figure 10: Illustration of metric-induced misclassification. While CoToGrasp strictly respects the target contact topology, the kinematic optimization may result in incidental contacts where an adjacent phalanx rests against the object surface. Despite the repulsive term in our optimization energy function, these phalanges often cannot be pushed away due to inherent kinematic constraints. For instance, the left grasp illustrates an intended M3 pinch reclassified as M12, while the right shows an M4 grasp reclassified as M15. Although the intended contacts are successfully achieved and the grasps remain physically stable, these incidental contacts trigger a strict reclassification by our automated pipeline.
Figure 10: Illustration of metric-induced misclassification. While CoToGrasp strictly respects the target contact topology, the kinematic optimization may result in incidental contacts where an adjacent phalanx rests against the object surface. Despite the repulsive term in our optimization energy function, these phalanges often cannot be pushed away due to inherent kinematic constraints. For instance, the left grasp illustrates an intended M3 pinch reclassified as M12, while the right shows an M4 grasp reclassified as M15. Although the intended contacts are successfully achieved and the grasps remain physically stable, these incidental contacts trigger a strict reclassification by our automated pipeline.
Table 5: Feature Alignment (Cosine Similarity).
Raw DGCNNWorkspace+ Attn.
Matched Pairs0.350.610.66
Random Pairs0.230.330.27
Figure 11: Histogram comparing the frequencies of effective topologies and attempted topologies (as defined Sec. 4.2) among all stable grasps generated by CoToGrasp (top) and Dexonomy [5] (bottom).
Figure 11: Histogram comparing the frequencies of effective topologies and attempted topologies (as defined Sec. 4.2) among all stable grasps generated by CoToGrasp (top) and Dexonomy [5] (bottom).
CoToGrasp: Contact-Topology-Conditioned Dexterous Grasp Synthesis via Canonical Workspace Learning figure 11
Table 6: Topology compliance ablation.
LabelEval.HS​RTC (%)HT​C
ConsistencyIsaac
✓✓0.9617.180.84
✓✗21.920.53
✗✓0.9414.450.81
✗✗19.340.50
Table 7: Object-Level Analysis. Evaluating performance across geometric complexity (c=Vo​b​j/Vh​u​l​l). CoToGrasp shows superior retention of both physical stability (SR) and semantic compliance (TC) on challenging non-convex objects.
Object ComplexityMetricCoToGraspDexonomy [5]
Convex → Non-ConvexSR Retained58.72%37.42%
Severe Concavities (c<0.4, ∼4% data)SR18.30%12.05%
TC19.17%8.94%

为什么重要

随着机器人转向高自由度灵巧手以完成工具使用、把手抓取等任务,仅仅避免物体掉落已经不够,抓取方式本身必须服务于具体任务需求。该方法无需为每个物体单独采集抓取数据,使功能导向、任务感知的抓取规划能够直接应用于新物体,大幅降低了数据成本。

本文术语

  • 机械手(Gripper) · 机器人用来抓握物体的手或夹爪部分
  • 接触拓扑(Contact Topology) · 定义手的哪些部位接触物体哪些部位的模式,源自人类抓握分类体系
  • 零样本泛化(Zero-shot generalization) · 无需额外训练即可正确处理训练中未见过的新物体的能力
  • 条件变分自编码器(CVAE) · 一种根据给定条件以概率方式生成多样化结果的神经网络结构
  • 模式坍塌(Mode collapse) · 生成模型只产生少数几种重复模式、缺乏多样性的现象

论文原文摘要（英文）

Current dexterous grasp planners primarily optimize for physical stability, focusing on whether an object can be grasped rather than how it should be grasped to support downstream functional tasks. However, conditioning grasp synthesis on specific human grasp taxonomies typically requires prohibitively expensive, object-annotated datasets. To address these limitations, we propose CoToGrasp, a novel generative framework that synthesizes diverse, stable grasps strictly conditioned on specific contact topologies. To bypass the data collection bottleneck, CoToGrasp is trained entirely in an object-agnostic manner. We introduce a feature-based canonical workspace that projects local object features into a unified gripper-centric domain, effectively decoupling the semantic functional intent from the arbitrary object geometry. By learning the intrinsic contact manifold of the gripper within this workspace, our model achieves zero-shot generalization to unseen objects at inference. Extensive evaluations on the large-scale DexGraspNet dataset demonstrate that CoToGrasp achieves state-of-the-art performance, outperforming existing taxonomy-guided planners. Finally, we demonstrate the physical viability and kinematic feasibility of our synthesized contact topologies on a physical robot platform. Code is available on our project website https://cea-list.github.io/cotograspweb/ .

作者 · Julien Merand

在 arXiv 阅读

最新论文

全部论文 →

METAL LAB 最新报道

图片来源： Julien Merand et al., arXiv:2608.19776, CC BY 4.0