Robotic Simulation in Real World

Introduction to simularion in robotics, presenting not only MuJoCo, a general purpose physics engine, aiming to facilitate research and development in robotics, biomechanics, graphics and animation, machine learning, but also its applications.

In particular, the presentation introduces:
- OpenAI Gym, a standard API for reinforcement learning, and a diverse collection of reference environments
- DeepMind dm_control, the software stack for physics-based simulation and Reinforcement Learning environments, using MuJoCo physics.
- MoCapAct, which uses motion capture (MoCap) clips to learn low-level motor skills for the "CMU Humanoid" from the dm_control package

Transcript

1. Robotic Simulation in Real World @davilagrau
2. About me… Andrés-Leonardo Martínez-Ortiz a.k.a almo, holds a PhD on Software, Systems and Computing and a Master on Computer Science. Based on Zurich, almo is a member of the Google Machine Learning Site Reliability Engineering team, leading several programs aiming for reliability, efficiency & convergence. He is also a member of IEEE, ACM, Linux Foundation and Computer Society. @davilagrau almo.dev almo
3. Catharsis Computing Club bit.ly/3T6CqAB
4. About this talk… Feynman Effect • What is Simulation? • Why do robotics researchers do simulation? • How do robotics researchers do simulations?
5. A simulation is the imitation of the operation of a real-world process or system over time. Simulations require the use of models; the model represents the key characteristics or behaviors of the selected system or process, whereas the simulation represents the evolution of the model over time. Wikipedia, accessed Oct 2022.
6. Simulated Worlds
7. Photo by Ant Rozetsky on Unsplash Revolution of the Industrial Production Automatization
8. Photo by fabio on Unsplash Challenge of the data acquisition
9. Photo by Amir-abbas Abdolali on Unsplash Twins are trendy! The digital ones!
10. Why robotics researchers do simulations?
11. (I) Robots…
12. Experimentation might be hard…
13. … harder
14. … or even impossible
15. Physical experimentation is limited in scale
16. Project Skyway: UK set to have world's biggest automated drone superhighway
17. How do (some) robotics researchers do simulations?
18. MuJoCo • MuJoCo stands for Multi-Joint dynamics with Contact. • It is a general purpose physics engine (Robotics, biomechanics, graphics and animation, machine learning) • It is a C/C++ library with a C API, including an interactive visualization with a native GUI, rendered in OpenGL. It also offer a wrapper in Python. • Scene descriptions (models) are describe in the native MJCF language (an XML file format). • 3D rendering helps the user gain insight into the physics simulation • Integrated/Supported by OpenAI Gym, PyBullet and others
19. [Off Topic] Engineers & Design Photo by Hal Gatewood on Unsplash
20. Modelling Elements • Inspired by the idea of Cascading Style Sheets (CSS) inlined in HTML. • Native 3D visualizer provides rendering, textures, etc… • Modelling ropes, cloth, and deformable 3D objects. • The positions, orientations and inertial properties are inferred. • Modelling elements: ◦ Options ◦ Assets ◦ Kinematic Tree ◦ Stand-alone elements.
21. Modelling Elements: Options Established by default, but customizable using the model file. Can be updated dynamically. • mjOption: physics simulation • mjVisual: visualization options (non OpenGL options) • mjStatistic: average body mass, spatial extent of the model etc. Photo by Victoriano Izquierdo on Unsplash
22. Modelling Elements: Assets Asset changes the properties of t model elements. • Mesh: OBJ files and binary STL • Skin: meshes whose shape can deform at runtime. • Height fields: model terrain maps. • Textures: only be referenced from another asset (namely material) • Material: control the appearance of geoms, sites and tendons. Photo by Nick Nice on Unsplash
23. Modelling Elements: Kinematic Tree System state represented in joint coordinates and the bodies are explicitly organized into kinematic trees • Body: Bodies have mass and inertial properties but do not have any geometric properties. • Joints: are defined within bodies, creating motion degrees of freedom (DOFs) between the body and its parent Photo by Nick Nice on Unsplash
24. Modelling Elements: Kinematic Tree System state represented in joint coordinates and the bodies are explicitly organized into kinematic trees • Geom: 3D shapes attached to the bodies. • Camera: There is always a default camera, but multiple cameras can be defined in a model. • Lights: full lighting model in OpenGL, including Photo by Nick Nice on Unsplash
25. Let’s look some examples
26. Example (C++) Loading file (MJCF or URDF) mjModel contains the model description (constant) Getting preallocated and internally managed stack, mjData contains all dynamic variables and intermediate results. Simulation: input(m,d) - output(d)
27. mjModel MuJoCo Simulation Workflow XML File User defined Mujoco App (C++) Mujoco Library OpenGL mjData Compilation time Execution time Function 1 Function 2 Function 2 Function N
28. Generalized Coordinates • Suitable for rigid body in 3D space • Joins add degrees of freedom • Forward Kinematic Images from Wikipedia
29. Photo by Claudio Schwarz on Unsplash Applications
30. Application I Digital Gym for non existence Robots bit.ly/3EprnOQ
31. OpenAI Gym • Python environment for developing and testing learning agenda • Suitable for Reinforcement Learning (RL) but… • It can also be used for training Deep Learning algorithms • Implementation of a simple RL loop
32. Environment for Atari, MuJoco, Toy Text and others
33. Example Let’s Play
34. Application II Software and Tasks for Continuous Control (dm_control) bit.ly/3fPkVGD
35. DeepMind dm_control • Collection of Python libraries and task suites for reinforcement learning agents in an articulated-body simulation. • A MuJoCo wrapper provides convenient bindings to functions and data structures. • PyMJCF and Composer libraries enable procedural model manipulation. • Control Suite offers a set of tasks with standardised structure, intended to serve as performance benchmarks. • Locomotion framework provides high-level abstractions and examples of locomotion tasks.
36. Let’s play bit.ly/3Vd2lbS
37. Application III What would it take to get humanoid, bipedal robots to dance like Mick Jagger? bit.ly/3MgkG3u
38. Photo by Ben Sweet on Unsplash Human vs Computer Simulations
39. MoCapAct - Motion Capture with Actions • MoCapAct is a large library of pre-trained humanoid control models along with enriched data for training new ones. • Based on using motion capture (MoCap) data. • Aiming to train AI agents with humanoid morphology to match human performance across the entire diversity of human motion. • Repositories such as CMU Motion Capture Dataset contain hours of clips for just about any common motion of a human body. • Based on dm_control* (MuJoCo) *DeepMind's software stack for physics-based simulation and Reinforcement Learning environments, using MuJoCo physics.
40. Re-using low-level component of the mutli-clip policy to aid in learning new humanoid tasks.
41. Use case I
42. Use case II Inspired by sentence completion from NLP, we train a GPT policy to perform motion completion
43. Final considerations
44. Next steps: Robot Benchmark • Program simulated robots online. • Open source • From middle school to PhD • Python and other languages coming soon! https://robotbenchmark.net
45. Others Simulation Environment Simulator Pros Gazebo Opensource, ROS, sponsored by the Open Source Robotic Foundations, Integrated w/ 4 physics engines PyBullet Opensource, Python, Blender. MuJoCo Models accepted Webots Opensouce, C++, Python, ROS, Matlab, OpenStreetMap
46. Robotic Simulation in Real World @davilagrau Thank you! @davilagrau almo.dev almo