RASPBERRY SI David Garlan CMU Co-I Bradley Schmerl CMU Co-I Pooyan Jamshidi USC PI Javier Camara York Collaborator Ellen Czaplinski Arkansas Consultant Katherine Dzurilla Arkansas Consultant Jianhai Su USC Graduate Student Matt DeMinico NASA Co-I AISR: Autonomous Robotics Research for Ocean Worlds (ARROW) Resource Adaptive Software Purpose-Built for Extraordinary Robotic Research Yields - Science Instruments

Current Practice of Science Discovery in Remote Planets Problem: Slow science discovery due to lack of full autonomy 2 (v) Only deal with: 
 Known Knowns ~2.5 hours ~2.5 hours (iv) Does not scale (i) Delay in science discovery (iii) High risks Planners Spacecraft Engineers Command Sequences [uplink] Telemetry Image [downlink] State Science Activities Scientific Data [images, measurements] Postmortem Analysis (ii) High mission costs

Autonomy Planning Perception Command Sequences [onboard] Telemetry Image [onboard] Perceived State Science Mission Mission Planning x f 12 / 38 Engineers Telemetry Image [downlink] Actual State Planners correction [uplink] Spacecraft Slow Fast Scientific Data [images, measurements] Postmortem Analysis Ideal Vision of Science Discovery in Remote Planets Solution: Fast science discovery with AI-based full autonomy 3 (v) Can deal with:
 Unknown Unknowns High Frequency Low Frequency (i) Fast science discovery (ii) Low mission costs (iv) Does scale (iii) Low risks

Challenges and Opportunities • Large data to train an accurate and reliable model • Data collection on other planets is slow. • Data from previous explorations with similar physics and characteristics • Physics-based simulation data 4

Transfer Learning from Simulation to Ocean Worlds Sim2Real Transfer 5 Deployed Environment Autonomous 
 System Simulation Environment VxSIM – Virtual Exercise Framework Sensor Models Camera LIDAR GPS/IMU RADAR HD Virtual Environment Simulation Network Vehicle and Articulated Model Dynamics Autonomous, AI System Scenario Builder Exercise Management Process and Parameter Interface Status Commands Sensor Data

Transfer Learning from the Earth to Ocean Worlds 6 Transfer Learning Well-known Physics Big Data Limited known Physics Small Data Earth2Europa Causal Invariances Causal AI Earth Ocean Worlds 
 (Europa, Enceladus, and Titan)

Simulation using OceanWATERS 7 Component Test Integration Test Model Learning Transfer Learning Model Compression Online Learning A B C D Quantitative Planning E Learning A E Case (Baseline) A E B A E B C A E B C D Case 2 (Transfer) Case 3 (Compress) Case 4 (Online) Test 1 Expected Performance Case 1 < Case 2 < Case 3 < Case 4 OWLAT Code: https://github.com/nasa/ow_simulator Physical Autonomy Testbed: https://www1.grc.nasa.gov/wp-content/uploads/2020_ASCE_OWLAT_20191028.pdf

Real-World Experiments using OWLAT • Models learned from simulation • Adaptive System (Learning + Planning) • Sets of tests Adaptive System Machine learning Models Mission Environment Continual Learning: refining models Log Mission Reports Local Machine Cloud Storage

Test Coverage • Mission Types: landing and scientific explorations -> sampling • Mission Difficulty: • Rough regions for landing • Number of locations where a sample needs to be fetched • Unexpected events: • Changes in the environments: e.g., uneven terrain and weather • Changes to the lander capabilities: e.g., deploy new sensors • Faults (power, instruments, etc)

Success Criteria of Evaluation • Correctness/Safety: all operating parameters of the spacecraft are within tolerances of their expected values. • Accuracy: the state estimates generated by the onboard learning algorithms to ground truth values. • Efficiency: computation times, energy consumption, and data bandwidth consumed. • Quality of Mission Completion • Major metric: mission goal (land successfully or collect expected amount of materials) • Minor metric: efficiency of the mission • If the mission violates correctness requirements, then set it to failure.

Evaluation Infrastructure Test Generator Autonomy Module Test 1 Test Harness Mission Configuration Testbed Monitoring & Logging Communication Logging Logs Log Analysis Evaluation Report Environment & Lander Simulation Adapter Interface Learning & Planning Plan Executive

Discussions (Virtual Testbed) • Virtual testbed capabilities: Battery charge; Position, pose, time, orientation? • Is the power model parametric (discharge rate)? • Telecommunication in virtual testbed – bandwidth consumption? • New features in virtual testbed?

Discussions (Physical Testbed) • Will the physical testbed have the lander move around? • Concurrent actions are common during mission? Language support both sequential and concurrent actions? • Time constraints for reconfigurations: how strict are the constraints? Impact on planning? • The gap between physical lander testbed and virtual testbed • Physical testbed provides a physical area for the lander to move around? How will be landing simulated in physical testbed? • Any way to implement a saw blade into testbed (like in Europa Lander mission)? • For bulk excavation to depth (10 cm or greater) • Largely agnostic to local surface topography

Discussions (Test Case Design) • Challenge problems for the lander to guide research and evaluation. • Test case design in agile way from Day-1! • Mission Types: landing and science instrument • Test Scenarios: • Are there any guidelines for creating test cases? • How would you engage in the design of test scenarios? • We would like an agile approach from Day-1 to design realistic test cases. • Transfer learning scenarios?

Test Cases: Surface scenarios/events • Would radiation or light from a distant supernova affect surface operations? • Thermal/power conservation during eclipses, which occur frequently on Europa • Virtual testbed darkens scene evenly for eclipses, but doesn’t simulate subtle gradation of a planet’s penumbra • Big unknowns in near-field features • Good estimates from Death Valley and Atacama desert research • But how to account for unknown surface features • “Europa-quakes” • Europa’s plate tectonics and icy shell provides opportunities to study quakes from the surface - also need to account for this type of event and how lander would respond if vibrations caused a key instrument to fail • Nearby Europa plumes • Exciting event for life detection and sampling material from subsurface ocean • How would lander respond?

Test Cases: Surface scenarios/events • From Europa lander report • “Tal” used to represent orbital period of the carrier/relayer spacecraft (24 Earth hours) • 5 sampling tals planned over a 20 day mission • Sample acquisition ~5 hours • Sample cycle is expected to be fully autonomous sequence - how might this be autonomously adjusted if the lander has to account for an “unknown event” (e.g., intense quakes or plume) • Will testbeds simulate deorbit, descent, and landing (DDL)? • If so, it’s possible that the hydrazine exhaust could deposit material on the surface near the landing site • How to implement this into testbed, if applicable? • How would instruments differentiate between Europa-native species or hydrazine-native species of nitrogen, ammonia, hydrogen, water, carbon dioxide, chloride, for example? • Europa lander mission has plans to reduce amount of exhaust contamination on surface

Test Cases: Surface scenarios/events • Radiation • Instruments protected via radiation shields/radiation vault • Could radiation affect comms relay between lander-orbiter or orbiter-Earth? • Will we have access to the carrier/relayer spacecraft in either testbed?

Discussions (Collaboration Infrastructure) • System requirements for using the two testbeds • ROS (Melodic), Gazebo 9.13+, Ubuntu 18.04 • Plan execution: YAML file + PLEXIL language vs Instruction Graph • To facilitate third-party evaluation, Dockerize Test Harness, Testbed and Adaptive Lander System. • GitFlow to facilitate collaboration during the project. PLEXIL: Plan Execution Interchange Language

Physical Space Lander Testbed at JPL Physical Autonomy Testbed: https://www1.grc.nasa.gov/wp-content/uploads/2020_ASCE_OWLAT_20191028.pdf E2M Technologies six DOF Stewart Platform representing spacecraft lander Barrett WAM seven DOF manipulator arm mounted to lander with wrist FTS and tool changer Modular instruments to be mounted on robot arm Testbed setup and major components HITL simulator of lander and manipulator

Computing and Software Architecture of the Physical Lander Testbed Physical Autonomy Testbed: https://www1.grc.nasa.gov/wp-content/uploads/2020_ASCE_OWLAT_20191028.pdf Emulation of Ocean World body dynamics within testbed Operator Interface used as a stand-in for the autonomy software Computing and Software Architecture

Important Test Cases: OWLAT Pressure- sinkage Test and Scooping Operation Physical Autonomy Testbed: https://www1.grc.nasa.gov/wp-content/uploads/2020_ASCE_OWLAT_20191028.pdf

Program Information • Program Manager: Carolyn Mercer @ NASA • Physical testbed contact: Hari Nayar @ NASA JPL • Virtual testbed contact: Mike Dalal @ NASA Ames • Selected Projects (out of 17 submissions): • Project 1: RASPBERRY SI: Resource Adaptive Software Purpose-Built for Extraordinary Robotic Research Yields - Science Instruments (USC) • PI: Pooyan Jamshidi (University of South Carolina) • Project 2: Robust Autonomy for Planetary Sampling • PI: Jonathan Bohren (Honeybee Robotics, Ltd) • Further Info: https://nspires.nasaprs.com/external/viewrepositorydocument/ cmdocumentid=773394/ solicitationId=%7B6FD283AF-7FD6-7A9F-1546-0FBFD722B6C2%7D/ viewSolicitationDocument=1/AISR19%20Abstracts.pdf

Discussions August 26th

Notes • The AI and Autonomy Technologies of this project will be potentially used in Ocean Worlds missions • Develop something cool, demonstrate feasibility, evaluate and demonstrate, NASA would love to take • Infusion of technology • We can tap into technologies outside of NASA • Community of practice • Formed collaboration them, each other hardware, program • Synergies, worthwhile to propose solicitation

Notes • Remote testing and evaluation for physical testbed • Encourage physical presence and stay at JPL to work closely with the JPL team • Resident at JPL • Co-developing be able to refine improve the functionality • Schedule work • Start with the virtual testbed • Physical after initial approval

Notes • Virtual testbed still under development • Changes to virtual testbed • Maintain some level of compatibility Command on virtual side and be able to test to physical • Stay with some standards

Notes • Simple operation first and make it more complicated • Adequate to get things done • Fault injection of you know • Bounds on what the system can do and what you may not expect • Unknown Unknowns are important • Characterize the nominal, software, autonomy software, unexpected things that can happen

Notes • Extending PLEXIL itself • Orienting logistical contractual fault injection • Planned feature • Basic fault injection model • Prioritize feature • Basic model: inject faults as ROS parameters • Fault spaces • Open source collaboration • Draft open source contribution

Notes • Physical will be ready end of November • Priori to that time Interface details • Capabilities in the system • Doing our development the simulator • Low-level autonomy command when we have the whole infrastructure, command actuators and sensors • Setup simulator low level capabilities with the software simulators • Virtual motors and tested out Completely platforms on a computer without physical • Full capability without driving the system • Different virtual testbed, all the capabilities of the physical testbed in virtual — hardware in the loop • Acceptance test, pass through the test in simulated version • Driving the physical Remote login to computer, remotely send command • Safety someone in the lab Model

Action items 1. Sending technical draft to testbed contacts. 2. Setting up regular meetings with testbed contacts.

Goal: Our Innovations in AI and Autonomy to be used in the Europa and other Ocean Worlds missions. This is a once in a lifetime chance to make a difference. Thanks, NASA, for giving us this opportunity! We are hiring under-represented groups in STEM: Female, Blacks, Hispanics, Native Americans. Contact: Pooyan Jamshidi (University of South Carolina) https://pooyanjamshidi.github.io/