Using RLLib in an Enterprise Scale Reinforcement Learning Solution (Jeroen Bédorf & Ishaan Sood, minds.ai)

Slide 1

Slide 1 text

Using RLlib in an enterprise scale reinforcement learning solution Ray Summit 2021 Jeroen Bédorf, jeroen@minds.ai Ishaan Sood, ishaan@minds.ai

Slide 2

Slide 2 text

©minds.ai Problem Statements Integration and usage of RLlib and Tune DeepSim Platform Adaptive Cruise Control Demo Hybrid Electric Vehicle Demo Outline

Slide 3

Slide 3 text

©minds.ai Trend: Exploding complexity and proliferation of smart systems DeepSim: Bring RL to Subject Matter Experts Electrification Autonomy Automation Renewables

Slide 4

Slide 4 text

©minds.ai DeepSim: Bring RL to Subject Matter Experts Controllers: Brains behind complex systems Reinforcement Learning Controllers: Trained for operating complex systems PID Controller Process Feedback Input Output RL Agent (neural network) Environment Input Output

Slide 5

Slide 5 text

©minds.ai DeepSim: Platform Overview Environment Integration & Scenario support Training libraries Data Analysis & Visualization Toolkit HPO & NAS Neural Network Models & definition method Front end TFAgents RLlib Ray Internal MPI Horovod Tune Internal Public Cloud Backend Algorithms Distribution method

Slide 6

Slide 6 text

©minds.ai DeepSim: Usage of Ray, RLlib and Tune Custom Action Distributions Easy Model Definition Method Custom Logging Custom Models Export Methods Analysis Tools RLlib Tune Ray Inference Methods

Slide 7

Slide 7 text

©minds.ai Typical end-user workflow Configure simulation, reward, etc. Status & Progress information Export trained Agent 1 2 3 Set up training runs (HPO & NAS) Tune Train

Slide 8

Slide 8 text

©minds.ai Optimizer (ONNX, TensorRT, etc.) Trained Agent Ray Serve Embedded Laptop/Workstation Inference System / Controller RLlib Checkpoint Inference Library Deployment