Koti Home Automation Robot

Slides for the talk I gave at Queens.js in New York on January 6, 2016. Video: https://youtu.be/Z-NpPj6aKJE

Links from the slides (or just download the original PDF https://goo.gl/HWxPSL, links work there):

---

I travel & write stories at http://arturpaikin.com

My small pop-up coffee shop Tipi Coffee https://vimeo.com/118402160

---

Comic strip by [KC Green](http://gunshowcomic.com/513)

---

# Beagle Bone

[a lot of fiddling](http://www.bonebrews.com/temperature-monitoring-with-the-ds18b20-on-a-beaglebone-black/)
[confusing pinout](http://www.element14.com/community/servlet/JiveServlet/showImage/38-17874-209857/bbb-pinout.jpg)

---

# Interesting controllers / chips

[LightBlue Bean](https://punchthrough.com/bean)

ESP8266 ([NodeMcu Lua Whatever]
(http://www.aliexpress.com/item/2015-New-product-Wireless-module-NodeMcu-Lua-Wifi-Nodemcu-WIFI-Network-Development-Board-Based-ESP8266-High/32341789414.html))
More about it: http://makezine.com/2015/04/01/esp8266-5-microcontroller-wi-fi-now-arduino-compatible/

[C.H.I.P.](https://www.kickstarter.com/projects/1598272670/chip-the-worlds-first-9-computer/description)

[Tessel 2](https://tessel.io)

[Arduino YUN](https://www.arduino.cc/en/Main/ArduinoBoardYun) (also OpenWRT)

[PINE](https://www.kickstarter.com/projects/pine64/pine-a64-first-15-64-bit-single-board-super-comput?ref=thanks_tweet)
[UDOO NEO](http://www.udoo.org/udoo-neo/)

---
# Awesome Johnny Five

http://johnny-five.io

And some issues I mentioned:
https://github.com/rwaldron/johnny-five/issues/112
https://github.com/rwaldron/johnny-five/issues/534

---

My home automation control panel in action: https://youtu.be/wXo2cdHL4Ys

---

What are Telegram Bots: https://core.telegram.org/bots

---

# Inspiration, cool projects

* Leonid Kaganov’s [home server](https://translate.google.com/translate?hl=en&sl=ru&tl=en&u=https%3A%2F%2Flleo.me%2Fdnevnik%2F2010%2F04%2F27.html)
* Rachel’s [cat feeder](https://github.com/rachelnicole/robokitty)
* DIY [Raspberry Pi Mirror](http://blog.dylanjpierce.com/raspberrypi/magicmirror/tutorial/2015/12/27/build-a-magic-mirror.html)
* Amazing [SidLee Dashboard](http://dashboard.sidlee.com)
* [Netflix socks](https://makeit.netflix.com)
* [PubNub](https://www.pubnub.com/blog)

---

Koti Home is open source: http://github.com/arturi/kotihome

---

# Follow me

* [arturpaikin.com](http://arturpaikin.com)
* [twitter.com/arturi](http://twitter.com/arturi)
* [instagram.com/arturi](http://instagram.com/arturi)
* [github.com/arturi](http://github.com/arturi)

Transcript

1. DIVIDE AND CONQUER YOUR HOUSE BUILDING A HOME AUTOMATION ROBOT
2. Hello, my name is ARTUR PAIKIN
3. ▸ Travel & write stories about it at arturpaikin.com ▸ Web developer, also teach web development for beginners ▸ Run a small pop-up coffee shop Tipi Coffee
4. And I’M INTO HOME AUTOMATION
5. IT STARTED AS A BIRTHDAY PRESENT !
6. BRIEF HISTORY OF MY ADDICTION 1. Built a blinking LED & read thermistor temperature data 2. Made it available on the Internet with an Ethernet shield and Dynamic DNS 3. Told everything about in (too much) detail to my girlfriend 4. Was hooked
7. WHY DO WE REALLY NEED A REASON? COMIC BY KC GREEN
8. ARDUINO PEACE OF ! ▸ Tons of great tutorials & cool community ▸ Robust libraries for sensors, protocols and whatever ▸ Digital and Analog I/O
9. void setup() { pinMode(13, OUTPUT); } void loop() { digitalWrite(13, HIGH); delay(1000); digitalWrite(13, LOW); delay(1000); }
10. ! WOW, I CAN CHECK THE TEMPERATURE & TURN THE LIGHT ON AND OFF THROUGH THE INTERNET "
12. IT’S NOT SECURE NO SSL You wouldn’t want anybody checking your home temperature, would you?
13. Actually, on some forum I found a message that was like: “I thought the whole point of this home automation stuff is that anybody can turn the light on at your house” !
14. YOU WANT CAMERAS & SOUND AND OTHER USB BELLS & WHISTLES
15. ARDUINO IS BEST AT HANDLING DIGITAL & ANALOG I/O, SO LET’S LEAVE IT AT THAT Arduino-like low-level I/O + smart Linux controller = ❤
16. ENTER BEAGLEBONE var b = require('bonescript'); b.digitalWrite('P8_13', 1);
17. ▸ Powerful CPU, built in 4GB flash memory, low-power usage ▸ Relatively small community, lack of libraries (One Wire) ▸ Connecting some sensors requires a lot of fiddling (like building a “device tree” !) ▸ Pre-installed distro with a lot of cruft I did’t need and had to delete (not easy) ▸ Sometimes confusing pinout ▸ No audio, micro-HDMI, one USB port (MacBook-style) ▸ “Just use JavaScript” = Bonescript lib. Not well maintained, not following error first node callback guidelines, so promisify fails Not saying it’s bad and most of it applies to other boards too. Just wasn’t for me.
18. ANYWAY. USED IT FOR SOME TIME, THEN SOLD IT
20. ENTER RASPBERRY PI 2 ▸ 4 USB ports, audio & standard HDMI ▸ Huge community, less bugs, great tutorials & cooler website
21. Technically you can use just the Pi for home automation, and some people do, it’s got 40 GPIO pins. But I wanted analog sensors and all the libs Arduino community provides. Might reconsider.
22. MULTIPLE “DUMB” CLIENTS WITH SENSORS + “SMART” HUB & HOST APP TO RULE THEM ALL
23. THESE ARE TWO OR MORE SEPARATE DEVICES
24. ARDUINO + RASPBERRY PI LET’S CONNECT THEM
26. NOT VERY PRETTY, BUT HEY, IT WORKS!
27. WHAT ELSE IS OUT THERE ▸ ESP8266 (NodeMcu Lua Whatever) basically an Arduino Nano, but with Wi-Fi and for $5 (official Wi-Fi module for Arduino is $49.90) ! ▸ LightBlue Bean bluetooth, fun, $30
28. ▸ C.H.I.P. promises Wi-Fi & Bluetooth, battery power circuit, not sure about analog I/O and ease of use (see BeagleBone). Impressive, $9 ▸ Tessel 2 promises awesomeness (might be a bit slow for some projects, OpenWRT might be limited for some cases) ▸ Arduino YUN (also OpenWRT) ▸ PINE is on the way for $15
29. UDOO NEO promises best of all worlds: Linux & Android, Arduino-compatible digital & analog pins, Wi-Fi + Bluetooth and even some sensors. $60 ! I’ll have to check it out.
30. IT’S A GOOD TIME WE LIVE IN BUT ENOUGH HARDWARE ✋
31. LET’S MAKE THEM TALK TO EACH OTHER With NODE SERIALPORT ! OR WEB SOCKETS, OR REST
32. JOHNNY FIVE IS GREAT but Doesn’t support some sensors and chips yet, like DHT 11 / 22 or radio 433 Mhz RF Not that big a deal, but I didn’t want to be tied to Firmata / Johnny Five + wanted to see what it’s like out there ! Plus, with a lot of controllers requires tethering
33. ROCKING JSON WITH ARDUINO void loop() { Serial.println(""); Serial.print("{"); Serial.print("\"temp\": "); Serial.print(getTemp()); Serial.print(","); Serial.print("\"lux\": "); Serial.print(getLux()); Serial.print("}"); delay(1000); }
35. Inspired by
36. SEE IT IN ACTION YOUTU.BE/WXO2CDHL4YS
37. DEMO
38. THREE WAYS TO CONTROL THE ROBOT 1. Look & click on the dashboard 2. Talk & listen to the dashboard 3. Send & receive messages through Telegram app3 3 Like Whatsapp, but multi-platform and with bots
39. LOOK, REACT! EACH BUTTON IS A COMPONENT
40. Dashboard.js import React from 'react'; import { LightBtn } from './LightBtn'; import { TempMeter } from './TempMeter'; import { Microphone } from './Microphone'; export class Dashboard extends React.Component { constructor(props) { super(props); this.state = { temp: 0, lightIsOn: 0, bgColor: Colors.coral }; this.socket = io(); this.socket.on('status', data => { this.setState( {temp: data.temp || 0, lightIsOn: data.lightIsOn || 0} ); }); } render() { return ( <div className="dashboard" style={{backgroundColor: this.state.bgColor}}> <LightBtn lightIsOn={this.state.lightIsOn} /> <TempMeter temp={this.state.temp} /> <Microphone /> </div> ); } }
41. LightBtn.js import React from 'react'; export class LightBtn extends React.Component { onClick() { fetch('/command/lightSwitch', {credentials: 'include'}) .then(function(response) { return response.text(); }); } render() { let classString = 'lampIcon' + (this.props.lightIsOn ? ' isOn' : ''); return ( <button className="dashboardItemBlock lampBlock" title="turn light on / off"> <svg className={classString} onClick={this.onClick}> <path d="M2.51791892,8.1966 C2.54591892,8.15385 ..."></path> <path d="M7.54164865,9.45136 C7.25275676,9.31133 ..."></path> </svg> </button> ); } };
42. VOICE RECOGNITION & SYNTHESIS API RIGHT IN YOU BROWSER1, ACCESSIBLE VIA JAVASCRIPT ! const recognition = new webkitSpeechRecognition(); const utterance = new SpeechSynthesisUtterance(msg); const, such ES2015 1 That currently means Chrome
43. TELEGRAM BOT Easy and cool. Custom commands & keyboards.
45. ▸ Problem: Dynamic IPs & Firewalls ▸ Solution: hub establishes a connection to VPS server, not the other way around ▸ HTTP is too bloated, so send messages through TCP. 5 days all good, then connection hang (that’s how I learned what keep alive is for) ▸ Trying to reinvent the wheel (sometimes a good thing)
46. ! MQTT & WEB SOCKETS ! ▸ Publish-Subscribe ▸ Quality of Service (must be good) ▸ Minimal overhead The hub connects to a VPS which is running MQTT via Web Sockets, through NGINX, which is using SSL cert2 2 This is done so that I can use the same cert I have for my domain, where the control panel lives
47. var mqttClient = mqtt.connect(broker, { username: user, password: pass }); // Get data from Arduino’s sensors, check if it’s new // and if so, send to the server var oldData = controller.getData(); function publishNewData() { var newData = controller.getData(); if (oldData !== newData) { oldData = newData; mqttClient.publish('/koti/data', newData); } } mqttClient.on('connect', function() { mqttClient.subscribe('/koti/commands'); publishNewDataInterval = setInterval(publishNewData, 1000); });
48. ROBOT ▸ Robot receives a command through POST-request, thinks about, maybe takes some action, then replies back ▸ Parses sentences, splits them into words ▸ if containsAny(words, dictionary.greetings), reply with hello ▸ Random answers from specific dictionaries (greetings, confirmations, sex)
49. function containsAny(haystack, needles) { if ( _.intersection(haystack, needles).length > 0 ) { return true; } return false; } if ( containsAny(words, this.dictionary.greetings) ) { matchFound = true; return this.react('hello', this.lastUpdate); }
50. case 'light': function lightSwitch() { self.controller.sendCommand('lightSwitch'); return self.react('confirmation', lastUpdate); } if (this.lastUpdate.method === 'telegram') { // checks if the msg is coming from an actual house owner if ( telegram.tremblingCreatureOrHaveITheRight(lastUpdate.from.username) ) { return lightSwitch(); } else { return this.react('howDareYou', lastUpdate); } } else { return lightSwitch(); } break;
51. WHAT I LEARNED ▸ Arduino, low-level hardware stuff & what is possible ▸ Raspberry Pi & Unix stuff ▸ Continuously running apps (I used PM2) ▸ React ▸ If your project features cats, it will get more stars on GitHub
52. SHIT HAPPENED ▸ Exceeded IFTTT’s monthly message cap with movement alert ▸ Node App was leaking memory. Turns out: setInterval on every client connect, had to use clearInterval on disconnect (thanks to my friend, real Java developer) ▸ Saw that there was some movement at home when no one was supposed to be there. Ran home. Turns out: supplied less power than necessary to Arduino and it freaked out. ▸ Raspberry Pi was hanging dead each time I turned the light on. Turns out: power issues again — the USB hub was either bad or needed more power.
53. INSPIRATION, COOL PROJECTS ▸ Leonid Kaganov’s home server ▸ Rachel’s cat feeder ▸ DIY Raspberry Pi Mirror ▸ Amazing SidLee Dashboard ▸ Netflix socks ▸ PubNub
54. THE END ▸ Make something fun, weird and non-profitable ▸ Celebrate your little achievements and be proud of them
55. OPEN SOURCE at github.com/arturi/kotihome pull requests and suggestions are very welcome
56. FOLLOW ME ▸ arturpaikin.com ▸ twitter.com/arturi ▸ instagram.com/arturi ▸ github.com/arturi