ArtManip: Category-Level Articulated In-Hand Manipulation
Learning dexterous policies of in-hand articulated objects that generalize across intra-class instances and initial grasps, with zero-shot sim2real capability.
Abstract
Category-level in-hand manipulation of articulated objects is a formidable yet underexplored challenge for dexterous robotic hands. This difficulty stems from two core bottlenecks: first, controlling an object's internal degrees of freedom is tightly coupled with maintaining grasp stability on a free-floating base; second, acquiring diverse object models and functional grasps at scale is highly labor-intensive, yet vital for generalization given the system's sensitivity to initial configurations. In this work, we present ArtManip, the first category-level articulated in-hand manipulation method that generalizes across object instances and diverse initial grasps. For initial configuration construction, we develop an automated pipeline that procedurally generates diverse articulated objects and synthesizes task-oriented functional grasps. For policy learning, we propose a robust two-stage training strategy that incorporates articulation physics randomization, reward curriculum, and latent representation distillation to handle complex contact and joint dynamics during deployment. Extensive experiments across four object categories demonstrate that our policy generalizes to unseen instances and varied configurations in simulation, and achieves zero-shot transfer to 12 real-world objects featuring diverse shapes and joint mechanics.
Primitive assets → functional grasps → robust policy
The pipeline builds a scalable training distribution, then learns a deployable student policy under partial observations.
Primitive articulated assets
Generate two-link objects with one prismatic or revolute joint using simple box primitives.
Functional grasp synthesis
Use category-level contact-region templates to synthesize diverse task-oriented initial grasps.
Teacher policy learning
Train with privileged state, reward curriculum, and articulation physics randomization.
Student deployment
Distill a latent representation from observable history for zero-shot real-world execution.
Initial Configuration Construction
Teacher-Student Training