Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition

MVIG-RHOS, SJTU

demodemodemo
Employing a teleoperation system for gathering demonstrations offers the potential for more efficient learning of robot manipulation. However, teleoperating a robot arm equipped with a dexterous hand or gripper, via a teleoperation system poses significant challenges due to its high dimensionality, complex motions, and differences in physiological structure. In this study, we introduce a novel system for joint learning between human operators and robots, that enables human operators to share control of a robot end-effector with a learned assistive agent, facilitating simultaneous human demonstration collection and robot manipulation teaching. In this setup, as data accumulates, the assistive agent gradually learns. Consequently, less human effort and attention are required, enhancing the efficiency of the data collection process. It also allows the human operator to adjust the control ratio to achieve a trade-off between manual and automated control. We conducted experiments in both simulated environments and physical real-world settings. Through user studies and quantitative evaluations, it is evident that the proposed system could enhance data collection efficiency and reduce the need for human adaptation while ensuring the collected data is of sufficient quality for downstream tasks.

Demo

News and Olds

[2023.4] Our paper is available on arXiv.
[2023.3] Trail run

Results

pentagon

Download

Our data and code will come very soon!

Publications

Before using our data and code in your project, please cite:
@misc{luo2024humanagentjointlearningefficient,
      title={Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition}, 
      author={Shengcheng Luo and Quanquan Peng and Jun Lv and Kaiwen Hong and Katherine Rose Driggs-Campbell and Cewu Lu and Yong-Lu Li},
      year={2024},
      eprint={2407.00299},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
}
© Copyright 2022 MVIG-RHOS • Based on tbakerx