Save 37% off PRO during our Black Friday Sale! »

Learning Robotic Contact Juggling (IROS'21)

753eb5a167cf43a033413f08e63f3632?s=47 OMRON SINIC X
October 21, 2021

Learning Robotic Contact Juggling (IROS'21)

Learning Robotic Contact Juggling (IROS'21)

Kazutoshi Tanaka, Masashi Hamaya, Devwrat Joshi, Felix von Drigalski, Ryo Yonetani, Takamitsu Matsubara, and Yoshihisa Ijiri

Presented at International Conference on Intelligent Robots and Systems (IROS 2021)
September 27, 2021

753eb5a167cf43a033413f08e63f3632?s=128

OMRON SINIC X

October 21, 2021
Tweet

Transcript

  1. © 2021 OMRON SINIC X Corporation. All Rights Reserved. Learning

    Robotic Contact Juggling (IROS’21) Kazutoshi Tanaka1, Masashi Hamaya1, Devwrat Joshi1, Felix von Drigalski1, Ryo Yonetani1, Takamitsu Matsubara2, and Yoshihisa Ijiri1 (1OMRON SINIC X Corporation, 2Nara Institute of Science and Technology) International Conference on Intelligent Robots and Systems (IROS 2021) September 27, 2021
  2. Learning Robotic Contact Juggling Kazutoshi Tanaka1, Masashi Hamaya1, Devwrat Joshi1,

    Felix von Drigalski1, Ryo Yonetani1, Takamitsu Matsubara2, and Yoshihisa Ijiri1 1OMRON SINIC X Corporation, 2Nara Institute of Science and Technology (c) 2021 OMRON SINIC X 2
  3. Outline • Motivation • Method • Experiment (c) 2021 OMRON

    SINIC X 3
  4. Outline • Motivation • Method • Experiment (c) 2021 OMRON

    SINIC X 4
  5. Robotic juggling (c) 2021 OMRON SINIC X 5 Devil sticking

    Diabolo Paddle juggling [Schaal & Atkeson 1994] Toss juggling [Ploeger+ 2020] [Kober+ 2010] [Drigalski+ 2021] Agile manipulation benchmark
  6. Contact juggling • Move an object with contact • Nonprehensile

    manipulation (c) 2021 OMRON SINIC X 6
  7. Robotic contact juggling (c) 2021 OMRON SINIC X 7 1

    joint [Lynch+ 1998] Modeling 3 joints [Woodruff & Lynch 2021] Butterfly Ball Hand Manual modeling and building a controller
  8. Model-based reinforcement learning • Sample-efficient • Complex dynamics • Behavior

    primitives (ours) Learning robotic contact juggling Contact juggling • Many interactions • High acceleration • High risk of breakdown (c) 2021 OMRON SINIC X 8
  9. Concept: behavior primitives 1. Simple behavior: e.g., free flying 2.

    Simple model: fast learning (c) 2021 OMRON SINIC X 9 Primitive A: contact Primitive B: no contact ... Contact juggling
  10. Outline • Motivation • Method • Experiment (c) 2021 OMRON

    SINIC X 10
  11. Model based RL (MBRL) • 𝑓 ~ 𝑇, state-transition dynamics

    𝑠𝑡+1 = 𝑇(𝑠𝑡 , 𝑎𝑡 ) • A model 𝑓(𝑠, 𝑎; 𝜃) (θ: parameter) • Learned using collected samples • Model predictive control 11 (C) 2021 OMRON SINIC X Corporation
  12. Concept: behavior primitives 1. Simple behavior: e.g., free flying 2.

    Simple model: fast learning (c) 2021 OMRON SINIC X 12 Primitive A: contact Primitive B: no contact ... Contact juggling
  13. Switched multiple model-based reinforcement learning (c) 2021 OMRON SINIC X

    13 Primitive A: contact Primitive B: no contact ...
  14. One goal of the model construction (c) 2021 OMRON SINIC

    X 14
  15. One goal of our method (c) 2021 OMRON SINIC X

    15 Primitive A: contact Primitive B: no contact
  16. One goal of our method (c) 2021 OMRON SINIC X

    16
  17. Dividing database to sub-databases (c) 2021 OMRON SINIC X 17

    Based on features of samples
  18. Switching model using sub-databases (c) 2021 OMRON SINIC X 18

    Based on features of samples
  19. (1) Extract all samples (c) 2021 OMRON SINIC X 21

  20. (2) Fit (c) 2021 OMRON SINIC X 22

  21. (3) Add a new model (c) 2021 OMRON SINIC X

    23 Maximum error
  22. (4) Remove maximum error sample (c) 2021 OMRON SINIC X

    24
  23. (5) Random sample (c) 2021 OMRON SINIC X 25

  24. End: Random sample (c) 2021 OMRON SINIC X 26 Fit

    sample number
  25. (6) Learn a switching model (c) 2021 OMRON SINIC X

    27
  26. Outline • Motivation • Method • Experiment (c) 2021 OMRON

    SINIC X 30
  27. Environment of learning robotic contact juggling (c) 2021 OMRON SINIC

    X 31 Ball: radius 0.05 m, weight 0.45 kg Robot: UR5e + flat hand Control: 25 Hz Gazebo: timestep 0.001 s
  28. Definition of learning robotic contact juggling (c) 2021 OMRON SINIC

    X 32 State: Reward: reference trajectory Action: joint target velocity Hand pose Ball position
  29. Curriculum learning (c) 2021 OMRON SINIC X 33 • Episode

    11-20 • 1.0 s • Throw & Catch • Episode 1-10 • 0.4 s • Throw • Trains models continuously • Same reward functions
  30. Baseline method (c) 2021 OMRON SINIC X 34 Baseline1 Ours

    One complex dynamics models Multiple simple models Baseline2 Ours Linear combination Selecting a model Baseline3 Ours No random extraction With random extraction *Simple/complex: low/high representation performance of models [Chua+ 2018] [Doya+ 2002]
  31. (5) Random sample (c) 2021 OMRON SINIC X 35

  32. Baseline method (c) 2021 OMRON SINIC X 36 Baseline1 Ours

    One complex dynamics models Multiple simple models Baseline2 Ours Linear combination Selecting a model Baseline3 Ours No random extraction With random extraction *Simple/complex: low/high representation performance of models [Chua+ 2018] [Doya+ 2002]
  33. Learned robotic contact juggling (c) 2021 OMRON SINIC X 37

  34. Learned robotic contact juggling (Realtime) (c) 2021 OMRON SINIC X

    38
  35. Returns: Our method learns contact juggling faster (c) 2021 OMRON

    SINIC X 40 Baseline1 Ours One complex dynamics models Multiple simple models
  36. Class: Behavior primitives were divided automatically (c) 2021 OMRON SINIC

    X 41 Free flying Contact
  37. Baseline 2 and baseline3 do not learn the juggling (c)

    2021 OMRON SINIC X 42 Baseline2 Ours Linear combination Selecting a model Baseline3 Ours No random extraction With random extraction Fundamental • Selection • Random sample extraction
  38. Conclusion • S-MMRL: a novel MBRL method to learn robotic

    contact juggling. • S-MMRL learns the juggling faster than baselines in simulations. • Future work: apply S-MMRL to • Real robots • Other robotic contact juggling • Practical tasks e.g., peg-in-hole (c) 2021 OMRON SINIC X 43