Robotic Simulation in Real World @davilagrau

About me… Andrés-Leonardo Martínez-Ortiz a.k.a almo, holds a PhD on Software, Systems and Computing and a Master on Computer Science. Based on Zurich, almo is a member of the Google Machine Learning Site Reliability Engineering team, leading several programs aiming for reliability, efficiency & convergence. He is also a member of IEEE, ACM, Linux Foundation and Computer Society. @davilagrau almo.dev almo

Catharsis Computing Club bit.ly/3T6CqAB

About this talk… Feynman Effect ● What is Simulation? ● Why do robotics researchers do simulation? ● How do robotics researchers do simulations?

A simulation is the imitation of the operation of a real-world process or system over time. Simulations require the use of models; the model represents the key characteristics or behaviors of the selected system or process, whereas the simulation represents the evolution of the model over time. Wikipedia, accessed Oct 2022.

Simulated Worlds

Photo by Ant Rozetsky on Unsplash Revolution of the Industrial Production Automatization

Photo by fabio on Unsplash Challenge of the data acquisition

Photo by Amir-abbas Abdolali on Unsplash Twins are trendy! The digital ones!

Why robotics researchers do simulations?

(I) Robots…

Experimentation might be hard…

… harder

… or even impossible

Physical experimentation is limited in scale

Project Skyway: UK set to have world's biggest automated drone superhighway

How do (some) robotics researchers do simulations?

MuJoCo ● MuJoCo stands for Multi-Joint dynamics with Contact. ● It is a general purpose physics engine (Robotics, biomechanics, graphics and animation, machine learning) ● It is a C/C++ library with a C API, including an interactive visualization with a native GUI, rendered in OpenGL. It also offer a wrapper in Python. ● Scene descriptions (models) are describe in the native MJCF language (an XML file format). ● 3D rendering helps the user gain insight into the physics simulation ● Integrated/Supported by OpenAI Gym, PyBullet and others

[Off Topic] Engineers & Design Photo by Hal Gatewood on Unsplash

Modelling Elements ● Inspired by the idea of Cascading Style Sheets (CSS) inlined in HTML. ● Native 3D visualizer provides rendering, textures, etc… ● Modelling ropes, cloth, and deformable 3D objects. ● The positions, orientations and inertial properties are inferred. ● Modelling elements: ○ Options ○ Assets ○ Kinematic Tree ○ Stand-alone elements.

Modelling Elements: Options Established by default, but customizable using the model file. Can be updated dynamically. ● mjOption: physics simulation ● mjVisual: visualization options (non OpenGL options) ● mjStatistic: average body mass, spatial extent of the model etc. Photo by Victoriano Izquierdo on Unsplash

Modelling Elements: Assets Asset changes the properties of t model elements. ● Mesh: OBJ files and binary STL ● Skin: meshes whose shape can deform at runtime. ● Height fields: model terrain maps. ● Textures: only be referenced from another asset (namely material) ● Material: control the appearance of geoms, sites and tendons. Photo by Nick Nice on Unsplash

Modelling Elements: Kinematic Tree System state represented in joint coordinates and the bodies are explicitly organized into kinematic trees ● Body: Bodies have mass and inertial properties but do not have any geometric properties. ● Joints: are defined within bodies, creating motion degrees of freedom (DOFs) between the body and its parent Photo by Nick Nice on Unsplash

Modelling Elements: Kinematic Tree System state represented in joint coordinates and the bodies are explicitly organized into kinematic trees ● Geom: 3D shapes attached to the bodies. ● Camera: There is always a default camera, but multiple cameras can be defined in a model. ● Lights: full lighting model in OpenGL, including Photo by Nick Nice on Unsplash

Let’s look some examples

Example (C++) Loading file (MJCF or URDF) mjModel contains the model description (constant) Getting preallocated and internally managed stack, mjData contains all dynamic variables and intermediate results. Simulation: input(m,d) - output(d)

mjModel MuJoCo Simulation Workflow XML File User defined Mujoco App (C++) Mujoco Library OpenGL mjData Compilation time Execution time Function 1 Function 2 Function 2 Function N

Generalized Coordinates ● Suitable for rigid body in 3D space ● Joins add degrees of freedom ● Forward Kinematic Images from Wikipedia

Photo by Claudio Schwarz on Unsplash Applications

Application I Digital Gym for non existence Robots bit.ly/3EprnOQ

OpenAI Gym ● Python environment for developing and testing learning agenda ● Suitable for Reinforcement Learning (RL) but… ● It can also be used for training Deep Learning algorithms ● Implementation of a simple RL loop

Environment for Atari, MuJoco, Toy Text and others

Example Let’s Play

Application II Software and Tasks for Continuous Control (dm_control) bit.ly/3fPkVGD

DeepMind dm_control ● Collection of Python libraries and task suites for reinforcement learning agents in an articulated-body simulation. ● A MuJoCo wrapper provides convenient bindings to functions and data structures. ● PyMJCF and Composer libraries enable procedural model manipulation. ● Control Suite offers a set of tasks with standardised structure, intended to serve as performance benchmarks. ● Locomotion framework provides high-level abstractions and examples of locomotion tasks.

Let’s play bit.ly/3Vd2lbS

Application III What would it take to get humanoid, bipedal robots to dance like Mick Jagger? bit.ly/3MgkG3u

Photo by Ben Sweet on Unsplash Human vs Computer Simulations

MoCapAct - Motion Capture with Actions ● MoCapAct is a large library of pre-trained humanoid control models along with enriched data for training new ones. ● Based on using motion capture (MoCap) data. ● Aiming to train AI agents with humanoid morphology to match human performance across the entire diversity of human motion. ● Repositories such as CMU Motion Capture Dataset contain hours of clips for just about any common motion of a human body. ● Based on dm_control* (MuJoCo) *DeepMind's software stack for physics-based simulation and Reinforcement Learning environments, using MuJoCo physics.

Re-using low-level component of the mutli-clip policy to aid in learning new humanoid tasks.

Use case I

Use case II Inspired by sentence completion from NLP, we train a GPT policy to perform motion completion

Final considerations

Next steps: Robot Benchmark ● Program simulated robots online. ● Open source ● From middle school to PhD ● Python and other languages coming soon! https://robotbenchmark.net

Others Simulation Environment Simulator Pros Gazebo Opensource, ROS, sponsored by the Open Source Robotic Foundations, Integrated w/ 4 physics engines PyBullet Opensource, Python, Blender. MuJoCo Models accepted Webots Opensouce, C++, Python, ROS, Matlab, OpenStreetMap

Robotic Simulation in Real World @davilagrau Thank you! @davilagrau almo.dev almo