ShanghaiTech University Knowledge Management System
| AffordDP: Generalizable Diffusion Policy with Transferable Affordance | |
| 2024-12-04 | |
| 状态 | 已发表 |
| 摘要 | Diffusion-based policies have shown impressive performance in robotic manipulation tasks while struggling with out-of-domain distributions. Recent efforts attempted to enhance generalization by improving the visual feature encoding for diffusion policy. However, their generalization is typically limited to the same category with similar appearances. Our key insight is that leveraging affordances--manipulation priors that define "where" and "how" an agent interacts with an object--can substantially enhance generalization to entirely unseen object instances and categories. We introduce the Diffusion Policy with transferable Affordance (AffordDP), designed for generalizable manipulation across novel categories. AffordDP models affordances through 3D contact points and post-contact trajectories, capturing the essential static and dynamic information for complex tasks. The transferable affordance from in-domain data to unseen objects is achieved by estimating a 6D transformation matrix using foundational vision models and point cloud registration techniques. More importantly, we incorporate affordance guidance during diffusion sampling that can refine action sequence generation. This guidance directs the generated action to gradually move towards the desired manipulation for unseen objects while keeping the generated action within the manifold of action space. Experimental results from both simulated and real-world environments demonstrate that AffordDP consistently outperforms previous diffusion-based methods, successfully generalizing to unseen instances and categories where others fail. |
| 语种 | 英语 |
| DOI | arXiv:2412.03142 |
| 相关网址 | 查看原文 |
| 出处 | Arxiv |
| 收录类别 | PPRN.PPRN |
| WOS记录号 | PPRN:119697735 |
| WOS类目 | Computer Science, Artificial Intelligence |
| 文献类型 | 预印本 |
| 条目标识符 | https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/471023 |
| 专题 | 信息科学与技术学院_硕士生 信息科学与技术学院_PI研究组_虞晶怡组 信息科学与技术学院_PI研究组_汪婧雅组 信息科学与技术学院_PI研究组_石野组 信息科学与技术学院_PI研究组_顾家远组 |
| 通讯作者 | Wang, Jingya |
| 作者单位 | ShanghaiTech Univ, Shanghai, Peoples R China |
| 推荐引用方式 GB/T 7714 | Wu, Shijie,Zhu, Yihang,Huang, Yunao,et al. AffordDP: Generalizable Diffusion Policy with Transferable Affordance. 2024. |
| 条目包含的文件 | ||||||
| 条目无相关文件。 | ||||||
修改评论
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。