Tweet
Tweet

Slide 1

Slide 1 text

felixge NodeCopter Flying robots over a 10.000 mile distance with JavaScript Felix Geisendörfer

Slide 2

Slide 2 text

@felixge felixge Background 2005 - 2008 2008 - now 2009 - now Felix Geisendörfer

Slide 3

Slide 3 text

@felixge felixge npm modules Felix Geisendörfer

Slide 4

Slide 4 text

@felixge felixge NodeCopter

Slide 5

Slide 5 text

@felixge felixge What is NodeCopter? • Started as a one-time event in Berlin on Oct 5th, 2012 • 60 programmers to program 25 AR Drones for a full day • Amazing venue, old swimming hall with 13m ceilings • Non-profit / cheap tickets (25 EUR)

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

@felixge felixge 1 day

Slide 13

Slide 13 text

@felixge felixge 1 day • Computer vision

Slide 14

Slide 14 text

@felixge felixge 1 day • Computer vision • Voice recognition

Slide 15

Slide 15 text

@felixge felixge 1 day • Computer vision • Voice recognition • Gesture recognition

Slide 16

Slide 16 text

@felixge felixge 1 day • Computer vision • Voice recognition • Gesture recognition • Hardware controls

Slide 17

Slide 17 text

@felixge felixge 1 day • Computer vision • Voice recognition • Gesture recognition • Hardware controls • and more

Slide 18

Slide 18 text

@felixge felixge Previous Events • Berlin, Oct 5th, 2012 • Dublin, Oct 20th, 2012 • San Francisco, Oct 31st, 2012 • Seattle, Nov 8th - 9th, 2012 • Brighton, Nov 10th, 2012

Slide 19

Slide 19 text

@felixge felixge Upcoming Events • San Francisco, Dec 1st, 2012 • Florida, May 30, 2013 • + Summer of Drones

Slide 20

Slide 20 text

felixge Summer of Drones June 2013

Slide 21

Slide 21 text

@felixge felixge Summer of Drones

Slide 22

Slide 22 text

@felixge felixge Summer of Drones • 4 weeks in June 2013

Slide 23

Slide 23 text

@felixge felixge Summer of Drones • 4 weeks in June 2013

Slide 24

Slide 24 text

@felixge felixge Summer of Drones • 4 weeks in June 2013 • 10+ NodeCopter events, and 20+ MiniNodeCopter events

Slide 25

Slide 25 text

@felixge felixge Summer of Drones • 4 weeks in June 2013 • 10+ NodeCopter events, and 20+ MiniNodeCopter events

Slide 26

Slide 26 text

@felixge felixge Summer of Drones • 4 weeks in June 2013 • 10+ NodeCopter events, and 20+ MiniNodeCopter events • North America, Europe, South America and Asia

Slide 27

Slide 27 text

@felixge felixge Summer of Drones

Slide 28

Slide 28 text

@felixge felixge Summer of Drones • Drone games / competitions

Slide 29

Slide 29 text

@felixge felixge Summer of Drones • Drone games / competitions

Slide 30

Slide 30 text

@felixge felixge Summer of Drones • Drone games / competitions • Half autonomous / half human controlled

Slide 31

Slide 31 text

@felixge felixge Summer of Drones • Drone games / competitions • Half autonomous / half human controlled

Slide 32

Slide 32 text

@felixge felixge Summer of Drones • Drone games / competitions • Half autonomous / half human controlled • Spectators for the tournaments

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

felixge http://nodecopter.com/

Slide 35

Slide 35 text

felixge This Talk

Slide 36

Slide 36 text

felixge Flying robots over a 10.000 mile distance with JavaScript

Slide 37

Slide 37 text

felixge

Slide 38

Slide 38 text

felixge Somebody in Berlin presses a button. Nothing happens.

Slide 39

Slide 39 text

felixge

Slide 40

Slide 40 text

felixge “NodeCopter” terror cell under investigation. Drone strike causes several injuries at Sidney JavaScript meetup.

Slide 41

Slide 41 text

felixge Drone Self-Defense

Slide 42

Slide 42 text

felixge Tip 1: Flip drone up-side down

Slide 43

Slide 43 text

felixge Tip 2: Nerf Gun

Slide 44

Slide 44 text

felixge just kidding

Slide 45

Slide 45 text

@felixge felixge Parrot AR Drone 2.0

Slide 46

Slide 46 text

@felixge felixge The Specs

Slide 47

Slide 47 text

@felixge felixge The Specs • WiFi (b/g/n) Router

Slide 48

Slide 48 text

@felixge felixge The Specs • WiFi (b/g/n) Router • 1 GHz 32Bit ARM Cortex A8 processor

Slide 49

Slide 49 text

@felixge felixge The Specs • WiFi (b/g/n) Router • 1 GHz 32Bit ARM Cortex A8 processor • 125 MB DDR2 Memory

Slide 50

Slide 50 text

@felixge felixge The Specs • WiFi (b/g/n) Router • 1 GHz 32Bit ARM Cortex A8 processor • 125 MB DDR2 Memory • Linux 2.6.32 you can telnet into it (Busybox)

Slide 51

Slide 51 text

@felixge felixge The Specs • WiFi (b/g/n) Router • 1 GHz 32Bit ARM Cortex A8 processor • 125 MB DDR2 Memory • Linux 2.6.32 you can telnet into it (Busybox) • 720p front facing camera (30 FPS)

Slide 52

Slide 52 text

@felixge felixge The Specs • WiFi (b/g/n) Router • 1 GHz 32Bit ARM Cortex A8 processor • 125 MB DDR2 Memory • Linux 2.6.32 you can telnet into it (Busybox) • 720p front facing camera (30 FPS) • 240p vertical camera (60 FPS)

Slide 53

Slide 53 text

@felixge felixge The Specs • WiFi (b/g/n) Router • 1 GHz 32Bit ARM Cortex A8 processor • 125 MB DDR2 Memory • Linux 2.6.32 you can telnet into it (Busybox) • 720p front facing camera (30 FPS) • 240p vertical camera (60 FPS) • USB 2.0 connector

Slide 54

Slide 54 text

@felixge felixge The Specs

Slide 55

Slide 55 text

@felixge felixge The Specs • 3 axis gyroscope (2000°/second precision)

Slide 56

Slide 56 text

@felixge felixge The Specs • 3 axis gyroscope (2000°/second precision) • 3 axis accelerometer (+-50mg precision)

Slide 57

Slide 57 text

@felixge felixge The Specs • 3 axis gyroscope (2000°/second precision) • 3 axis accelerometer (+-50mg precision) • 3 axis magnetometer (6° precision)

Slide 58

Slide 58 text

@felixge felixge The Specs • 3 axis gyroscope (2000°/second precision) • 3 axis accelerometer (+-50mg precision) • 3 axis magnetometer (6° precision) • Pressure sensor (+-10 Pa precision)

Slide 59

Slide 59 text

@felixge felixge The Specs • 3 axis gyroscope (2000°/second precision) • 3 axis accelerometer (+-50mg precision) • 3 axis magnetometer (6° precision) • Pressure sensor (+-10 Pa precision) • Ultrasound sensors

Slide 60

Slide 60 text

@felixge felixge The Specs • 3 axis gyroscope (2000°/second precision) • 3 axis accelerometer (+-50mg precision) • 3 axis magnetometer (6° precision) • Pressure sensor (+-10 Pa precision) • Ultrasound sensors • Fully reparable - all components can be ordered

Slide 61

Slide 61 text

@felixge felixge Normal Setup Parrot AR Drone 2.0 Smartphone or Laptop WiFi

Slide 62

Slide 62 text

felixge WiFi Range is ~50m

Slide 63

Slide 63 text

felixge Berlin -> Australia

Slide 64

Slide 64 text

@felixge felixge

Slide 65

Slide 65 text

felixge ~10.000 miles (~16.000 km)

Slide 66

Slide 66 text

felixge We need to extend range by 320.000x

Slide 67

Slide 67 text

felixge “No problem, we’ll just set up an ssh tunnel, should take ~30 minutes” --me

Slide 68

Slide 68 text

felixge 30 minutes - yeah right

Slide 69

Slide 69 text

felixge ~8 hours

Slide 70

Slide 70 text

@felixge felixge The Idea Parrot AR Drone 2.0 @SydJS Laptop @SydJS WiFi @SydJS Router Ethernet Router @Berlin Laptop @Berlin THE INTERNET WiFi

Slide 71

Slide 71 text

@felixge felixge Reverse SSH Tunnel ssh -NT \ -R 2023:192.168.1.1:23 \ sydjs@91.64.83.193 • Connect to my laptop in Berlin (sydjs@91.64.83.193) • Forward port 2023 on my laptop to port 23 at 192.168.1.1 (the AR Drone) • -N (do not execute a cmd) -T (disable pseudo-tty)

Slide 72

Slide 72 text

felixge Works great for TCP

Slide 73

Slide 73 text

felixge Does not work with UDP

Slide 74

Slide 74 text

felixge netcat to the rescue!

Slide 75

Slide 75 text

@felixge felixge netcat udp forwarding # @sydjs ssh -NT \ -R 6556:localhost:6556 \ sydjs@91.64.83.193 & mkfifo navdata.fifo nc -l localhost 6556 < navdata.fifo \ | nc -u 192.168.1.1 5556 > navdata.fifo # @berlin mkfifo navdata.fifo nc -l -u localhost 5556 < navdata.fifo \ | nc localhost 65556 > navdata.fifo

Slide 76

Slide 76 text

felixge No luck : (

Slide 77

Slide 77 text

felixge Node.js to the rescue

Slide 78

Slide 78 text

@felixge felixge In Sidney • SSH Reverse tunnel for telnet and video (ports 23 and 5555) • SSH Reverse tunnel for udp (port 2024) • node.js TCP server on port 2024 listening for incoming connection from Berlin • Incoming tcp data = udp messages • Forwards messages received on tcp server via udp to drone • Also forwards udp messages from drone back to berlin via tcp

Slide 79

Slide 79 text

@felixge felixge In Berlin • node.js UDP server listening on port 5554 (navdata) and 5556 (control) • node.js TCP connection to port 2024 on localhost which is hooked up to port 2024 in sidney • Incoming UDP messages are send to Sidney via TCP • Data received via TCP from Sidney is resend as UDP locally

Slide 80

Slide 80 text

@felixge felixge Problem • UDP guarantees atomic messages • TCP does not (one message may be split into multiple parts, or merged with others)

Slide 81

Slide 81 text

@felixge felixge Simple Protocol • 4 Bytes: Message Length • 2 Bytes: Source UDP Port • 2 Bytes: Destination UDP Port • : Message

Slide 82

Slide 82 text

felixge what could go wrong?

Slide 83

Slide 83 text

felixge

Slide 84

Slide 84 text

felixge https://github.com/felixge/nodecopter-ssh-tunnel https://github.com/felixge/ar-drone https://github.com/bkw/node-dronestream

Slide 85

Slide 85 text

felixge So little JS!

Slide 86

Slide 86 text

felixge Thanks!

Slide 87

Slide 87 text

felixge Questions? Felix Geisendörfer