Precise Multi-Modal In-Hand Pose Estimation using Low-Precision Sensors for Robotic Assembly (ICRA ‘21)

Felix von Drigalski, Kennosuke Hayashi, Yifei Huang, Ryo Yonetani, Masashi Hamaya, Kazutoshi Tanaka, Yoshihisa Ijiri, "Precise Multi-Modal In-Hand Pose Estimation using Low-Precision Sensors for Robotic Assembly", ICRA 2021

Transcript

1. © 2021 OMRON SINIC X Corporation. All Rights Reserved. Precise Multi-Modal In-Hand Pose Estimation using Low-Precision Sensors for Robotic Assembly (ICRA ‘21) Felix von Drigalski, Kennosuke Hayashi, Yifei Huang*, Ryo Yonetani, Masashi Hamaya, Kazutoshi Tanaka, Yoshihisa Ijiri (OMRON SINIC X) *(University of Tokyo) International Conference on Robotics and Automation (ICRA) 2021 Jun. 1, 2021
2. OMRON SINIC X Precise Multi-Modal In-Hand Pose Estimation using Low-Precision Sensors for Robotic Assembly ICRA 2021 Felix von Drigalski, Kennosuke Hayashi, Yifei Huang, Ryo Yonetani, Masashi Hamaya, Kazutoshi Tanaka, Yoshihisa Ijiri
3. 2 © OMRON SINIC X Corporation All Rights Reserved - A framework to obtain high-precision poses of grasped objects - Using very simple calibration methods - With commonplace, off-the-shelf sensors (camera, force sensor) In-Hand Pose Estimation Summary / Objective
4. 3 © OMRON SINIC X Corporation All Rights Reserved - Small position errors cause large problems - Insertion failures, stuck parts, drops… Automatic assembly Background Pose estimate is imperfect due to noise Insertion fails
5. 4 © OMRON SINIC X Corporation All Rights Reserved - Task separated into simple problems with known state - Inflexible, high engineering cost, long changeover times - Customized jigs used to fix part position + orientation Classic robot workcells Current generation workcells iREX 2017
6. 5 © OMRON SINIC X Corporation All Rights Reserved WRS2018 Assembly Challenge Human demonstration Objective: Jigless assembly Video: Osaka University
7. 6 © OMRON SINIC X Corporation All Rights Reserved Problem Uncertain pose after grasping High confidence Task execution ? How to obtain in-hand pose (post-grasp) with high precision?
8. 7 © OMRON SINIC X Corporation All Rights Reserved Proposal: In-Hand Pose Estimation Uncertain pose after grasping High confidence Touch tip etc. Task execution Touch side Discrete actions to reduce uncertainty Interact with environment to gain information Look
9. 8 © OMRON SINIC X Corporation All Rights Reserved In-Hand Pose Estimation Uncertain pose High confidence Touch tip Touch side Look Compute likelihood Select action Sequential improvement until desired precision reached
10. 9 © OMRON SINIC X Corporation All Rights Reserved • Belief about the object pose is a probability distribution • Representation: Object pose (xyz, quaternion = 7 DOF) + Covariance matrix defined in gripper frame • Lower covariance = lower uncertainty Pose Belief p = [xyz, xyzw] Gripper frame Mesh frame (object frame) Probability distribution
11. 10 © OMRON SINIC X Corporation All Rights Reserved • Observations 𝑦𝒕 , 𝒄𝒕 , obtained from actions improve quality of belief • We sample particles from the distribution and calculate their likelihood • Then, we use particle likelihood to approximate the posterior probability: Actions + Belief Belief distribution New belief distribution Action/Observation
12. 11 © OMRON SINIC X Corporation All Rights Reserved Touch action Belief distribution Sampled Particles Belief Collision evaluation New belief distribution Weighted Particles Collision body Sensor detects contact (in simulated environment) No contact Too deep Contact ✔
13. 12 © OMRON SINIC X Corporation All Rights Reserved Look action (idealized) Binary (real) RGB (real) Belief distribution Sampling Binary (sim) Particles Project (one particle) Belief Observation Evaluation Calculate similarity Filter via similarity (all particles) New belief distribution Object contour
14. 13 © OMRON SINIC X Corporation All Rights Reserved Look action (bad calibration) Binary (real) RGB (real) Belief distribution Sampling Binary (sim) Particles Project (one particle) Belief Observation Evaluation Calculate similarity Filter via similarity (all particles) ? New belief distribution
15. 14 © OMRON SINIC X Corporation All Rights Reserved Look action (with reference object) Binary (real) RGB (real) Belief distribution Sampling Binary (sim) Particles Project (one particle) Belief Observation Evaluation Calculate similarity Filter via similarity (all particles) Align via reference New belief distribution
16. 15 © OMRON SINIC X Corporation All Rights Reserved • We use a reference object in the scene and P3P calibration to confirm the camera pose • Easy calibration: Move robot gripper to cone tip • Independent of initial camera calibration Online calibration
17. 16 © OMRON SINIC X Corporation All Rights Reserved - We grasped three objects in 4 different orientations and positions, and used Look and Touch actions to determine their in-hand pose - Compared: Precision for different poses, objects, number of particles order of actions Experiments Grasp poses and depths
18. 17 © OMRON SINIC X Corporation All Rights Reserved - Look alone does not achieve high precision (low-cost camera) - Order matters: Look first, then Touch delivers best results - Costliest operation: mesh projection into image Results
19. 18 © OMRON SINIC X Corporation All Rights Reserved 1. Current belief distribution is 6D pose of mesh origin, but rotation around mesh origin is asymmetric à Include offset or different centers of rotation in belief representation 2. Binary outcomes cannot be represented by normal distribution (e. g. symmetries, flipped orientation, shaft with a hole on one end) à Non-continuous distributions 3. High calculation cost when sampling many individual particles à Apply observations to distribution directly to increase efficiency Considerations Mesh offset
20. 19 © OMRON SINIC X Corporation All Rights Reserved - A framework to determine post-grasp object pose with high precision, easy calibration, commonplace sensors - Allows both touch & look actions - Touching environment takes time but can offer higher precision - Future work: - Use gravity + environment - Determine optimal action - Release code package Conclusion