Flying Penguins: Embedded Linux Applications for Autonomous UAVs

Presented at the 2015 Embedded Linux Conference.

Accompanying videos: https://vimeo.com/claymcclure

Transcript

1. Embedded  Linux  applications   for  autonomous  UAVs Flying Penguins
2. Clay McClure github.com/claymation twitter.com/claymcclure
9. autopilots
10. RC input motor mixing stabilization telemetry missions failsafes
11. autopilots
13. ≠ AUTO PILOT AUTO NOMOUS
14. where to go how to get there what to do next “finds its own goal positions”
15. SO MANY ALGORITHMS, SO LITTLE COMPUTER http://ra3ndy.deviantart.com/art/Sad-Panda-69204875
16. +
17. Autopilot runs on Linux
18. Autopilot talks to Linux
19. ODROID-XU3 Lite • Samsung Exynos5422 octa core • 4x Cortex™-A15 2.0GHz • 4x Cortex™-A7 1.4GHz • 2 GB RAM • 32+ GB flash • 4x USB 2.0 + 1x USB 3.0
20. UAV Your App Linux ? Auto Pilot Architecture
21. UAV MAV Link Your App Linux Auto Pilot Architecture
22. MAVLink is the HTTP of drones
23. (it’s also the libcurl)
24. MAVLink
25. status configuration position / attitude setpoints missions
26. UAV MAV Link Your App Linux ? Auto Pilot Architecture
27. UAV MAV Link Your App Linux Middle ware Auto Pilot Architecture
28. Middleware DroneAPI • Python • Go to Kevin Hester’s talk tomorrow mavros • Python, C++, Lisp (really) • Access to a wealth of robotics research and tools
29. UAV MAV Link mavros Your App ROS Linux Auto Pilot Architecture
30. ROS Crash Course
31. Robot Operating System “ROS is an open-source, meta-operating system for your robot.”
32. “ROS is an open-source, meta-operating system for your robot.” Robot Operating System
34. Nodes Node Node Node Node Node Node Node Node Node
35. Topics Node Node Node Node Topic
36. Services Node Node
37. but wait, that’s not all… parameters dynamic reconfig coordinate frames transformations record/playback visualization logging
38. UAV MAV Link mavros Your App ROS Linux Auto Pilot Architecture
39. mavros is the Babel fish of drones
40. Mavros Node Topic Topic Topic Auto Pilot
41. Topics /mavros/state /mavros/imu/data /mavros/global_position/global /mavros/local_position/local /mavros/setpoint_position/local_position /mavros/setpoint_velocity/cmd_vel
42. Services /mavros/cmd/arming /mavros/cmd/land /mavros/cmd/takeoff /mavros/set_mode /mavros/set_stream_rate
43. • FCU • MAVLink • mavros • ROS • Application nodes PX4 + ROS Credit: Kabir Mohammed MAVROS
44. UAV MAV Link mavros Your App ROS Linux Auto Pilot Architecture
45. Event-driven programming • “Don’t call me, I’ll call you” • Your application code responds to events • Message arrival • “vehicle position is (x, y, z)” • Timer expiry • “it’s time to run the control loop”
46. YAPL Yet Another Precision Lander
47. Nodes • Tracker • Processes video stream, looks for landing pad • Publishes target position/velocity messages • Commander • Subscribes to vehicle state and position messages • Subscribes to target tracker messages • Controls vehicle velocity
48. Tracker
51. class  TrackerNode(object):   !        def  __init__(self):                  rospy.init_node("tracker")   !                camera_matrix  =  rospy.get_param("~camera_matrix")                  use_sim  =  rospy.get_param("~use_sim",  False)   !              self.image_publisher  =  \                          rospy.Publisher("tracker/image",                                  sensor_msgs.msg.Image,  queue_size=1)   !                self.track_publisher  =  \                          rospy.Publisher("tracker/track",                                  Track,  queue_size=1)
52. class  TrackerNode(object):   !        def  run(self):                  """                  Capture  frames  from  the  camera  and  publish  them.                  """                  self.wait_for_position()   !                while  not  rospy.is_shutdown():                          frame  =  self.camera.get_frame()                          if  frame  is  not  None:                                  position,  velocity  =  \                                          self.process_frame(frame)                                  self.publish_track(position,  velocity)
53.        def  publish_track(self,  position,  velocity):                  """                  Publish  Track  messages  for  the  commander  node.                  """                  msg  =  Track()                  msg.track.is_tracking.data  =  self.is_tracking   !                if  self.is_tracking:                          msg.track.position.x  =  position[0]                          msg.track.position.y  =  position[1]                          msg.track.position.z  =  position[2]                          msg.track.velocity.x  =  velocity[0]                          msg.track.velocity.y  =  velocity[1]                          msg.track.velocity.z  =  velocity[2]   !                self.track_publisher.publish(msg)  
54.        def  publish_track(self,  position,  velocity):                  """                  Publish  Track  messages  for  the  commander  node.                  """                  msg  =  Track()                  msg.track.is_tracking.data  =  self.is_tracking   !                if  self.is_tracking:                          msg.track.position.x  =  position[0]                          msg.track.position.y  =  position[1]                          msg.track.position.z  =  position[2]                          msg.track.velocity.x  =  velocity[0]                          msg.track.velocity.y  =  velocity[1]                          msg.track.velocity.z  =  velocity[2]   !                self.track_publisher.publish(msg)  
55. $  cat  msg/Track.msg     #  Whether  we're  tracking  an  object   std_msgs/Bool  is_tracking   ! #  Relative  position  and  velocity  of  the  tracked  object   geometry_msgs/Vector3  position   geometry_msgs/Vector3  velocity  
56.        def  publish_image(self,  image):                  """                  Publish  processed  tracker  images  for  telemetry.                  """                  msg  =  self.image_bridge.cv2_to_imgmsg(image,  "bgr8")                  self.image_publisher.publish(msg)  
57. Commander
58. PENDING SEEK APPROACH DESCEND LAND LANDED
59. class  CommanderNode(object):        def  __init__(self,  vehicle):                rospy.init_node("commander")   !                #  Initialize  flight  controllers                  self.controllers  =  {                          FlightState.PENDING    :  PendingController(),                          FlightState.SEEK          :  SeekController(),                          FlightState.APPROACH  :  ApproachController(),                          FlightState.DESCEND    :  DescendController(),                          FlightState.LAND          :  LandController(),                  }   !                #  ...
60.                #  Initialize  the  control  loop                  control_loop_rate  =  \                                  rospy.get_param("~control_loop_rate",                                  DEFAULT_CONTROL_LOOP_RATE)                  self.control_loop_rate  =  \                                  rospy.Rate(control_loop_rate)   !                rospy.Subscriber("/mavros/state",                                  mavros.msg.State,                                  self.handle_state_message)                  rospy.Subscriber("/tracker/track",                                  Track,  self.handle_track_message)   !                #  Initialize  state  machine                  self.controller  =  None                  self.state  =  FlightState.INIT                  self.transition_to_state(FlightState.PENDING)
61.        def  run(self):                  """                  Spin  the  ROS  event  loop,  running  the  active                  flight  controller  on  each  iteration.                  """                  while  not  rospy.is_shutdown():                          self.controller.run()                          self.control_loop_rate.sleep()
62.        def  handle_state_message(self,  msg):                  """                  Handle  mavros/State  messages,  published  by  the                  FCU  at  ~1  Hz.                  """                  mode  =  msg.mode   !                if  (mode  in  GUIDED_MODES  and                          self.state  ==  FlightState.PENDING):                                  self.transition_to_state(FlightState.SEEK)   !                elif  (mode  not  in  GUIDED_MODES  and                          self.state  !=  FlightState.PENDING):                                  self.transition_to_state(FlightState.PENDING)
63. #  Guided  modes  differ  between  ArduCopter  and  PX4  native   GUIDED_MODES  =  ("GUIDED",  "OFFBOARD")
64. Vehicle
65. class  Vehicle(object,  PositionMixin):          """          Simple  model  of  an  aerial  vehicle.          """          def  __init__(self):                self.location_setpoint_publisher  =  \                          rospy.Publisher(                                  "/mavros/setpoint_position/local_position",                                  geometry_msgs.msg.PoseStamped,                                  queue_size=1)   !                self.velocity_setpoint_publisher  =  \                          rospy.Publisher(                                  "/mavros/setpoint_velocity/cmd_vel",                                  geometry_msgs.msg.TwistStamped,                                  queue_size=1)
66. class  Vehicle(object,  PositionMixin):   !        #  ...                  def  set_location_setpoint(self,  setpoint):                  """                  Publish  a  SET_POSITION_TARGET_LOCAL_NED  message                  with  position  x,  y,  z  and  yaw.                  """                  x,  y,  z,  yaw  =  setpoint   !                msg  =  geometry_msgs.msg.PoseStamped()                  msg.pose.position.x  =  x                  msg.pose.position.y  =  y                  msg.pose.position.z  =  z                  msg.pose.orientation.z  =  yaw   !                self.location_setpoint_publisher.publish(msg)
67. class  Vehicle(object,  PositionMixin):   !        #  ...                  def  set_armed_state(self,  state):                  set_armed_state  =  rospy.ServiceProxy(                                  "/mavros/cmd/arming",                                  CommandBool)                  set_armed_state(value=state)   !        def  arm(self):                  self.set_armed_state(True)   !        def  disarm(self):                  self.set_armed_state(False)
68. Simulation
69. HITL • Hardware in the loop • Flight software runs on flight hardware • Simulated sensor and control inputs
70. SITL • Software in the loop • Flight software runs on (Linux) desktop • Simulated sensor and control inputs and HAL
74. –Johnny Appleseed “Type a quote here.” “In theory there is no difference between theory and practice. 
 In practice there is.” ! ~ Yogi Berra
75. Practical Considerations
79. Connections • UART recommended • Requires 6-pin DF-13, possibly a level shifter • USB works for me • Use hot glue gun • sudo  apt-­‐get  remove  modemmanager
80. Power 5V 5A UBEC ! ODROID + USB camera + WiFi = 5 hours
81. Launch files • ROS feature that makes it easy to start and manage multiple nodes and their parameters • roslaunch  lander  lander.launch
82. Startup • Use ubuntu’s upstart to launch ROS + mavros + application nodes • rosrun  robot_upstart  install  \
        lander/launch/lander.launch
83. Telemetry • WiFi • Ad-Hoc mode (man  wireless) • Need high-gain antenna and an antenna tracker
 (a.k.a. unsuspecting friend or bystander) • sudo  apt-­‐get  remove  wpasupplicant   • GSM? • Use image_transport to transmit compressed images
84. Coordinate Frames • Global / Local • NED (UAV) • ENU (ROS) • Body-fixed • tf library
85. In closing…
86. What will you make?
87. Thanks
88. For more information… ros.org ardupilot.com pixhawk.org/start pixhawk.ethz.ch/mavlink github.com/mavlink/mavros github.com/claymation/lander