매일 아침, 어제의 AI를 한 통으로 정리해 보내드립니다메일로 받아보기

METAL LAB

GOAG: Generative and Object-Agnostic Grasp Planner for Dexterous Robotic Manipulation

arXiv:2608.197592026-08-19

물건을 하나도 안 배워도, 손가락 로봇이 물건 잡는 법을 알아낸다

로봇 손이 물건을 능숙하게 쥐게 하려면 보통 물건-그리퍼(로봇 손) 조합 데이터를 잔뜩 학습시켜야 하는데, 새로운 물건에는 잘 안 통한다는 문제가 있었다. GOAG는 물건 데이터 없이 그리퍼 자체의 관절 구조와 표면 모양만 학습해서, 처음 보는 물건도 손이 스스로 접촉 지점을 찾아내게 만든다. MultiDex라는 기존 데이터셋 기준으로 평균 86.93%의 성공률을 냈고, 기존 방법과 성능은 비슷하면서도 다수의 파지를 생성할 때 훨씬 빠르다.

무엇을 했나

  1. 기존 방식은 특정 물건-그리퍼 조합으로 만든 데이터셋에 의존해 새 물건에 일반화가 안 되는 문제가 있었다
  2. GOAG는 물건 대신 그리퍼(로봇 손) 자체의 표면과 관절 움직임만으로 학습 데이터를 만들어 학습 단계에서는 물건을 전혀 보지 않는다
  3. 손 모양의 잡는 방식(그랩 유형) 6가지를 기준으로 손 표면에서 접촉 가능한 영역을 무작위로 샘플링해 300만 개의 학습 데이터를 생성했으며, 데이터 생성에 GPU 1시간이 걸려 기존 방법의 1400시간보다 훨씬 빠르다
  4. 추론 시에만 새로운 물건의 3D 형태를 입력받아, CVAE(조건부 변분 오토인코더)라는 생성 모델로 물건 표면에서 손이 닿아야 할 접촉점을 찾아내고, 힘 폐쇄 검사와 최적화를 거쳐 최종 손 자세를 완성한다
  5. MultiDex 데이터셋에서 평균 86.93% 성공률로 해당 데이터셋에 맞춰 학습된 기존 최고 방법들과 비슷한 성능을 냈고, 5개의 서로 다른 데이터셋에서도 재학습 없이 Shadow Hand 하나로 두 번째로 높은 성공률을 기록했으며 실제 로봇 팔로도 YCB 물건 11개를 성공적으로 집었다
Fig. 1: GOAG Paradigm. A successful grasp on an object induces dual contact zones on both object and gripper, at the intersection of the two geometries. Our method is built on this key observation: these contact zones (𝒞(.)) are closely the same from either perspective. GOAG capitalizes on this by training exclusively on gripper geometry, allowing it to learn a robust and generalizable grasping strategy without ever being exposed to a grasp database with specific objects geometries.
Fig. 1: GOAG Paradigm. A successful grasp on an object induces dual contact zones on both object and gripper, at the intersection of the two geometries. Our method is built on this key observation: these contact zones (𝒞(.)) are closely the same from either perspective. GOAG capitalizes on this by training exclusively on gripper geometry, allowing it to learn a robust and generalizable grasping strategy without ever being exposed to a grasp database with specific objects geometries.
TABLE I: Comparison of Dexterous Grasp Planning Methods.
Grasp RepresentationGraspRepresentationGripper PoseGripperPoseGripper Joint ValuesGripperJoint ValuesForce ClosureForceClosureNon- PenetrationNon-PenetrationTraining SetTrainingSetWorking Reference Frame
Grasp
Representation
Gripper
Pose
Gripper
Joint Values
Force
Closure
Non-
Penetration
Training
Set
Working
Reference Frame
Optional Grasp
Preference Interface
DFC [14]DirectOptimizedOptimized✓✓NoObject✗
UniGrasp [24]IntermediateIK SolvedIK Solved✓✗Objects + GrippersObject✗
GeoMatch [2]IntermediateIK SolvedIK Solved✓✗Objects + GrippersObject✗
GenDexGrasp [12]IntermediateOptimizedOptimized✓✓Objects + GrippersObject✗
ManiFM [37]IntermediateOptimizedOptimized✓✓Objects + GrippersObjectContact Region
DRO-Grasp [32]IntermediateOptimizedOptimized✓✓Objects + GrippersObjectPalm Orientation
DexDiffuser [33]DirectLearnedLearnedA posterioriA posterioriObjects + GrippersObject✗
DexGrasp Anything [39]DirectLearnedLearned✓✓Objects + GrippersObject✗
GOAG (Ours)IntermediateSampledOptimized✓✓Gripper OnlyGripperPalm Full Pose
Fig. 2: Grasp Taxonomy Adaptation and Contact Sampling. (Top) We adapt the human grasp taxonomy from [7] to the Allegro Hand geometry. For each grasp type (e.g., C6, F27), we define a corresponding admissible contact region (black points), distinguishing it from the non-contact surface (blue points). (Bottom) Data generation mechanism: We randomly sample specific contact points (red) strictly within the admissible black regions. This allows the model to learn structured, feasible contact distributions based solely on gripper kinematics, independent of any object.
Fig. 2: Grasp Taxonomy Adaptation and Contact Sampling. (Top) We adapt the human grasp taxonomy from [7] to the Allegro Hand geometry. For each grasp type (e.g., C6, F27), we define a corresponding admissible contact region (black points), distinguishing it from the non-contact surface (blue points). (Bottom) Data generation mechanism: We randomly sample specific contact points (red) strictly within the admissible black regions. This allows the model to learn structured, feasible contact distributions based solely on gripper kinematics, independent of any object.
TABLE II: In-depth grasp performance analysis on the Multidex [12] test set. We report our grasp results, on three classic dexterous grippers in terms of success rate, efficiency and diversity.
MethodData DrivenObject-Agnostic TrainingSuccess Rate (%) ↑Efficiency (sec. / grasps) ↓Diversity (avg.) ↑
BarrettAllegroShadowHandAvg.BarrettAllegroShadowHandT (m)R (rad)Q (rad)
DFC [14]✗✓83.1082.7172.1579.32>1800>1800>18000.06071.4240.3579
GenDexGrasp [12] (full)✓✗70.2671.4871.1570.969.7816.4514.650.05191.4160.2567
DRO-Grasp [32] (pretrain, w/o controller)✓✗78.3075.8063.3072.470.880.421.720.05461.5150.2892
GOAG (w/o FC)✓✓86.3091.2074.7084.070.090.130.150.04801.3960.3162
GOAG✓✓87.4093.2077.9086.930.180.190.200.04791.4010.3170
Fig. 3: Overview of GOAG. Geometrical graspability is learned in an object-agnostic manner by focusing on the gripper’s capabilities. Training: We sample gripper configurations Q to generate ℋ⁡(Q) and corresponding contact points (𝒞⁡(ℋ⁡(Q))). To ensure transferability, we use a Basis Point Set (BPS) encoding tied to the gripper’s workspace. A Conditional Variational Autoencoder (CVAE) is trained to reconstruct these contact distributions, while a Links Mapper (PointNet++) learns to associate contact points with specific gripper links. Inference: A novel object 𝒪, positioned at the inverse gripper pose [R,T]−1, is BPS-encoded. By sampling a latent variable z∈ℝψ, the CVAE Decoder generatively predicts diverse, plausible contact points 𝒞^​(𝒪). The Links Mapper then labels which gripper link should reach each point. Generation: Finally, a Force Closure check ensures the predicted contacts yield a stable grasp, and a Grasp Optimization step outputs the final, refined gripper configuration Q∗.
Fig. 3: Overview of GOAG. Geometrical graspability is learned in an object-agnostic manner by focusing on the gripper’s capabilities. Training: We sample gripper configurations Q to generate ℋ⁡(Q) and corresponding contact points (𝒞⁡(ℋ⁡(Q))). To ensure transferability, we use a Basis Point Set (BPS) encoding tied to the gripper’s workspace. A Conditional Variational Autoencoder (CVAE) is trained to reconstruct these contact distributions, while a Links Mapper (PointNet++) learns to associate contact points with specific gripper links. Inference: A novel object 𝒪, positioned at the inverse gripper pose [R,T]−1, is BPS-encoded. By sampling a latent variable z∈ℝψ, the CVAE Decoder generatively predicts diverse, plausible contact points 𝒞^​(𝒪). The Links Mapper then labels which gripper link should reach each point. Generation: Finally, a Force Closure check ensures the predicted contacts yield a stable grasp, and a Grasp Optimization step outputs the final, refined gripper configuration Q∗.
TABLE III: Grasp generalization assessment across multiple grasp datasets. Following [39] we evaluate our method performances with the Shadow hand on multiple grasp test sets. We report the success rates as defined in IV-C. It is worth noting that state-of-the-art methods are retrained for each dataset. GOAG has only been trained once on Shadow hand kinematics.
MethodPer-Dataset TrainingDexGraspNet ↑UniDexGrasp ↑MultiDex ↑RealDex ↑DexGRAB ↑Avg. (%)
UniDexGrasp [35]✓33.923.721.627.120.825.42
GraspTTA [10]✓18.621.030.313.314.419.52
SceneDiffuser [9]✓26.628.369.821.739.137.10
UGG [16]✓46.946.055.332.742.744.72
DGA [39]✓57.553.179.144.857.958.48
GOAG✗43.0749.5177.9037.3762.1353.97
Fig. 4: GOAG grasp results on Multidex [12] objects. Grasps are shown for the Barrett (green), Allegro (pink), and Shadow Hand (purple) grippers.
Fig. 4: GOAG grasp results on Multidex [12] objects. Grasps are shown for the Barrett (green), Allegro (pink), and Shadow Hand (purple) grippers.

왜 중요한가

로봇이 창고나 가정에서 마주치는 물건은 셀 수 없이 다양한데, 매번 새 물건마다 데이터를 모아 재학습하는 방식은 현실적으로 비용이 크다. GOAG는 손 자체의 구조만 한 번 학습해두면 되므로, 학습 비용과 시간을 크게 줄이면서도 새로운 물건에 바로 대응할 수 있는 실용적인 로봇 손 제어 방법을 제시한다.

Fig. 5: Real-world setup and results with Allegro hand on YCB [5] objects. First row presents the real robot grasps. Second row presents corresponding virtual grasps. Objects have been rotated around the z-axis for a better understanding of the grasp poses.
Fig. 5: Real-world setup and results with Allegro hand on YCB [5] objects. First row presents the real robot grasps. Second row presents corresponding virtual grasps. Objects have been rotated around the z-axis for a better understanding of the grasp poses.

이 논문의 용어

  • 그리퍼(Gripper) · 물건을 집는 로봇 손 부분
  • CVAE(조건부 변분 오토인코더) · 조건(입력 정보)에 맞춰 다양한 결과를 생성해내는 딥러닝 생성 모델
  • BPS(Basis Point Set) · 3D 점들의 모양을 고정된 크기의 숫자 표현으로 바꾸는 인코딩 방식
  • 힘 폐쇄(Force Closure) · 손가락들이 만든 접촉점만으로 물건을 흔들림 없이 붙잡을 수 있는지 확인하는 물리적 조건
  • PointNet++ · 3D 점 구름 데이터를 처리하는 딥러닝 네트워크 구조

논문 원문 초록 (영문)

Multifingered grasping is a crucial robotic skill, but current deep-learning grasp planners often struggle to generalize to new objects because they are trained on limited, object-specific datasets. We introduce a fundamentally different approach, grounded in the observation that the gripper and the object share identical surface geometry at their mutual contact points. We propose GOAG: Generative and Object-Agnostic Grasp Planner for Dexterous Robotic Manipulation, a novel deep generative model that learns a compact latent representation of a specific gripper's contact surface distribution, enabling the efficient sampling of valid grasp configurations without relying on object-specific training data. We show that by introducing object features only at inference time, our model can effectively retrieve admissible contact areas that are compatible with the gripper's capabilities. We validate our approach through extensive experiments on established grasp protocols in both simulated and real-world scenarios, demonstrating its effectiveness with different grippers from the literature. Our method delivers state-of-the-art results on the objects from the MultiDex dataset, achieving an average success rate of 86.93%. It offers significantly faster processing when generating numerous grasps, while matching the performance of leading approaches specifically trained on this dataset. Unlike these methods, our approach does not rely on object-specific training data, highlighting the advantages of object-agnostic learning. It effectively addresses the generalization challenges faced by traditional data-driven grasp planners. Code and videos are available on our project website https://cea-list.github.io/goagweb/ .

저자 · Julien Merand

arXiv에서 원문 보기

최신 논문

논문 전체 보기 →

METAL LAB 최신 기사

그림 출처: Julien Merand et al., arXiv:2608.19759, CC BY 4.0