Controlling Robots like NASA

In this demo, Adrian Todorov and I explain the do's and dont's when building a remote-controlled robot.

This version of the talk was given at the Southern California Linux Expo (22nd Edition) in March 2025.

Transcript

1. Controlling Robots (like NASA, but on a budget)
2. Senior Developer Advocate at HashiCorp and former IIoT Engineer @ksatirli Kerim Satirli
3. Staff Solutions Architect at HashiCorp and former Site Reliability Engineer @sofixa Adrian Todorov
4. Remotely control a device to move in any logical and supported direction. Goal (aka: lofty aspiration)
5. Bad Attitude apnews.com/article/88ccdbc183bfd90b7f40ad266d56ad02
6. Bad Attitude apnews.com/article/88ccdbc183bfd90b7f40ad266d56ad02
7. Goal (aka: lofty aspiration) Build, accessorize, and remotely drive a robot, but make it extra fun.
8. Hardware Overview
9. Hardware Overview ▪ ESP32-based control board ▪ GPIO interface for host computer ▪ support for LiDAR, cameras, arms ▪ open source(ish) demo available ▪ 4WD, with 2 extra support wheels ▪ powered by commonly used batteries ▪ integrated inertial measurement unit ▪ commercially available, at low cost
10. ▪ Arduino-based C++ "demo" code ▪ Arduino libraries as force multiplier ▪ refactored to modularize components ▪ Terraform Provider Software Overview api_controller.h String jsonAPIHandler(String stringInput) { StaticJsonDocument<512> jsonInput; deserializeJson(jsonInput, stringInput); int cmdType = jsonInput["T"].as<int>(); switch (cmdType) { case CMD_EMERGENCY_STOP: setGoalSpeed(0, 0); emergencyStopProcessing(); break; case CMD_GIMBAL_CTRL_MOVE: gimbalMove( jsonInput["X"], jsonInput["Y"], jsonInput["SX"], jsonInput["SY"] ); break; !!" in progress: ▪ object detection with YOLO ▪ edge orchestration with Nomad
11. Going from A to B
12. • pulses of light or sound • accuracy depends on environmental conditions • not a great option when hunted by submarines Sonar / LiDAR • need pre-positioning • accuracy depends on quality of deployment • not a great option when ad-hoc movement is key Beacon / Satnav • easy to calculate • accuracy depends on lots of dynamic variables • not a great option for long movement plans Dead Reckoning Going from A to B
13. Movement Plan c419.pathfinder.svcs.dev/map
14. Movement Plan c419.pathfinder.svcs.dev/map
15. Movement Plan c419.pathfinder.svcs.dev/map
16. Movement Plan with Angle c419.pathfinder.svcs.dev/map 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 0 { "name": "Movement Plan #1", "steps": [ ] } plan-1.json { "angle": 0, "direction": "forward", "distance": 60 }
17. Movement Plan with Angle c419.pathfinder.svcs.dev/map 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 0 plan-1.json { "name": "Movement Plan #1", "steps": [ { "angle": 0, "direction": "forward", "distance": 60 } ] } { , { "angle": -50, "direction": "forward", "distance": 10 }
18. Movement Plan with Angle github.com/workloads/pathfinder-openapi-specs/blob/main/movement-plan/README.md 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 0 plan-1.json { "name": "Movement Plan #1", "steps": [ { "angle": 0, "direction": "forward", "distance": 60 } ] } movement-plan.md Movement Plan for Pathfinder > This document describes v1.1 of the "movement plan" schema for Pathfinder. 
 Table of Contents Definition Movement Plan Movement Step Angle Direction Distance Track Map Map Legend Example Movement Plan Definition A movement plan represents the sequence of steps that Pathfinder will follow to navigate a track.
19. • imperial, metric, pulses • accuracy depends on environmental conditions • attitude of device affects precision of movement Distance • battery quality and actual charge levels • state and quality of wheel fairing and cables • interface between software and hardware Maintenance • traction and friction of wheels on surface • "invisible" to sensors and partial obstruction • RF interference and distance to control unit Environment Failure Factors
20. Future Improvements • 360 degree vision via a multi-camera array • improved LiDAR • extra RF capabilities • off-load to control unit Hardware • CV for object detection, and AI for decision making • better obstacle avoidance • backhaul switching • swarm logic Software
22. This talk wasn't about how to control a robot.
23. This talk was about navigating the unknown.
24. Demo pathfinder-0bc4.svcs.dev
25. Demo Code github.com/workloads/pathfinder
26. Thank you speakerdeck.com/ksatirli