Slide 1

Slide 1 text

Robotic Simulation in Real World @davilagrau

Slide 2

Slide 2 text

About me… Andrés-Leonardo Martínez-Ortiz a.k.a almo, holds a PhD on Software, Systems and Computing and a Master on Computer Science. Based on Zurich, almo is a member of the Google Machine Learning Site Reliability Engineering team, leading several programs aiming for reliability, efficiency & convergence. He is also a member of IEEE, ACM, Linux Foundation and Computer Society. @davilagrau almo.dev almo

Slide 3

Slide 3 text

Catharsis Computing Club bit.ly/3T6CqAB

Slide 4

Slide 4 text

About this talk… Feynman Effect ● What is Simulation? ● Why do robotics researchers do simulation? ● How do robotics researchers do simulations?

Slide 5

Slide 5 text

A simulation is the imitation of the operation of a real-world process or system over time. Simulations require the use of models; the model represents the key characteristics or behaviors of the selected system or process, whereas the simulation represents the evolution of the model over time. Wikipedia, accessed Oct 2022.

Slide 6

Slide 6 text

Simulated Worlds

Slide 7

Slide 7 text

Photo by Ant Rozetsky on Unsplash Revolution of the Industrial Production Automatization

Slide 8

Slide 8 text

Photo by fabio on Unsplash Challenge of the data acquisition

Slide 9

Slide 9 text

Photo by Amir-abbas Abdolali on Unsplash Twins are trendy! The digital ones!

Slide 10

Slide 10 text

Why robotics researchers do simulations?

Slide 11

Slide 11 text

(I) Robots…

Slide 12

Slide 12 text

Experimentation might be hard…

Slide 13

Slide 13 text

… harder

Slide 14

Slide 14 text

… or even impossible

Slide 15

Slide 15 text

Physical experimentation is limited in scale

Slide 16

Slide 16 text

Project Skyway: UK set to have world's biggest automated drone superhighway

Slide 17

Slide 17 text

How do (some) robotics researchers do simulations?

Slide 18

Slide 18 text

MuJoCo ● MuJoCo stands for Multi-Joint dynamics with Contact. ● It is a general purpose physics engine (Robotics, biomechanics, graphics and animation, machine learning) ● It is a C/C++ library with a C API, including an interactive visualization with a native GUI, rendered in OpenGL. It also offer a wrapper in Python. ● Scene descriptions (models) are describe in the native MJCF language (an XML file format). ● 3D rendering helps the user gain insight into the physics simulation ● Integrated/Supported by OpenAI Gym, PyBullet and others

Slide 19

Slide 19 text

[Off Topic] Engineers & Design Photo by Hal Gatewood on Unsplash

Slide 20

Slide 20 text

Modelling Elements ● Inspired by the idea of Cascading Style Sheets (CSS) inlined in HTML. ● Native 3D visualizer provides rendering, textures, etc… ● Modelling ropes, cloth, and deformable 3D objects. ● The positions, orientations and inertial properties are inferred. ● Modelling elements: ○ Options ○ Assets ○ Kinematic Tree ○ Stand-alone elements.

Slide 21

Slide 21 text

Modelling Elements: Options Established by default, but customizable using the model file. Can be updated dynamically. ● mjOption: physics simulation ● mjVisual: visualization options (non OpenGL options) ● mjStatistic: average body mass, spatial extent of the model etc. Photo by Victoriano Izquierdo on Unsplash

Slide 22

Slide 22 text

Modelling Elements: Assets Asset changes the properties of t model elements. ● Mesh: OBJ files and binary STL ● Skin: meshes whose shape can deform at runtime. ● Height fields: model terrain maps. ● Textures: only be referenced from another asset (namely material) ● Material: control the appearance of geoms, sites and tendons. Photo by Nick Nice on Unsplash

Slide 23

Slide 23 text

Modelling Elements: Kinematic Tree System state represented in joint coordinates and the bodies are explicitly organized into kinematic trees ● Body: Bodies have mass and inertial properties but do not have any geometric properties. ● Joints: are defined within bodies, creating motion degrees of freedom (DOFs) between the body and its parent Photo by Nick Nice on Unsplash

Slide 24

Slide 24 text

Modelling Elements: Kinematic Tree System state represented in joint coordinates and the bodies are explicitly organized into kinematic trees ● Geom: 3D shapes attached to the bodies. ● Camera: There is always a default camera, but multiple cameras can be defined in a model. ● Lights: full lighting model in OpenGL, including Photo by Nick Nice on Unsplash

Slide 25

Slide 25 text

Let’s look some examples

Slide 26

Slide 26 text

Example (C++) Loading file (MJCF or URDF) mjModel contains the model description (constant) Getting preallocated and internally managed stack, mjData contains all dynamic variables and intermediate results. Simulation: input(m,d) - output(d)

Slide 27

Slide 27 text

mjModel MuJoCo Simulation Workflow XML File User defined Mujoco App (C++) Mujoco Library OpenGL mjData Compilation time Execution time Function 1 Function 2 Function 2 Function N

Slide 28

Slide 28 text

Generalized Coordinates ● Suitable for rigid body in 3D space ● Joins add degrees of freedom ● Forward Kinematic Images from Wikipedia

Slide 29

Slide 29 text

Photo by Claudio Schwarz on Unsplash Applications

Slide 30

Slide 30 text

Application I Digital Gym for non existence Robots bit.ly/3EprnOQ

Slide 31

Slide 31 text

OpenAI Gym ● Python environment for developing and testing learning agenda ● Suitable for Reinforcement Learning (RL) but… ● It can also be used for training Deep Learning algorithms ● Implementation of a simple RL loop

Slide 32

Slide 32 text

Environment for Atari, MuJoco, Toy Text and others

Slide 33

Slide 33 text

Example Let’s Play

Slide 34

Slide 34 text

Application II Software and Tasks for Continuous Control (dm_control) bit.ly/3fPkVGD

Slide 35

Slide 35 text

DeepMind dm_control ● Collection of Python libraries and task suites for reinforcement learning agents in an articulated-body simulation. ● A MuJoCo wrapper provides convenient bindings to functions and data structures. ● PyMJCF and Composer libraries enable procedural model manipulation. ● Control Suite offers a set of tasks with standardised structure, intended to serve as performance benchmarks. ● Locomotion framework provides high-level abstractions and examples of locomotion tasks.

Slide 36

Slide 36 text

Let’s play bit.ly/3Vd2lbS

Slide 37

Slide 37 text

Application III What would it take to get humanoid, bipedal robots to dance like Mick Jagger? bit.ly/3MgkG3u

Slide 38

Slide 38 text

Photo by Ben Sweet on Unsplash Human vs Computer Simulations

Slide 39

Slide 39 text

MoCapAct - Motion Capture with Actions ● MoCapAct is a large library of pre-trained humanoid control models along with enriched data for training new ones. ● Based on using motion capture (MoCap) data. ● Aiming to train AI agents with humanoid morphology to match human performance across the entire diversity of human motion. ● Repositories such as CMU Motion Capture Dataset contain hours of clips for just about any common motion of a human body. ● Based on dm_control* (MuJoCo) *DeepMind's software stack for physics-based simulation and Reinforcement Learning environments, using MuJoCo physics.

Slide 40

Slide 40 text

Re-using low-level component of the mutli-clip policy to aid in learning new humanoid tasks.

Slide 41

Slide 41 text

Use case I

Slide 42

Slide 42 text

Use case II Inspired by sentence completion from NLP, we train a GPT policy to perform motion completion

Slide 43

Slide 43 text

Final considerations

Slide 44

Slide 44 text

Next steps: Robot Benchmark ● Program simulated robots online. ● Open source ● From middle school to PhD ● Python and other languages coming soon! https://robotbenchmark.net

Slide 45

Slide 45 text

Others Simulation Environment Simulator Pros Gazebo Opensource, ROS, sponsored by the Open Source Robotic Foundations, Integrated w/ 4 physics engines PyBullet Opensource, Python, Blender. MuJoCo Models accepted Webots Opensouce, C++, Python, ROS, Matlab, OpenStreetMap

Slide 46

Slide 46 text

Robotic Simulation in Real World @davilagrau Thank you! @davilagrau almo.dev almo