Koti Home Automation Robot

Koti Home Automation Robot

Slides for the talk I gave at Queens.js in New York on January 6, 2016. Video: https://youtu.be/Z-NpPj6aKJE

Links from the slides (or just download the original PDF https://goo.gl/HWxPSL, links work there):

---

I travel & write stories at http://arturpaikin.com

My small pop-up coffee shop Tipi Coffee https://vimeo.com/118402160

---

Comic strip by [KC Green](http://gunshowcomic.com/513)

---

# Beagle Bone

[a lot of fiddling](http://www.bonebrews.com/temperature-monitoring-with-the-ds18b20-on-a-beaglebone-black/)
[confusing pinout](http://www.element14.com/community/servlet/JiveServlet/showImage/38-17874-209857/bbb-pinout.jpg)

---

# Interesting controllers / chips

[LightBlue Bean](https://punchthrough.com/bean)

ESP8266 ([NodeMcu Lua Whatever]
(http://www.aliexpress.com/item/2015-New-product-Wireless-module-NodeMcu-Lua-Wifi-Nodemcu-WIFI-Network-Development-Board-Based-ESP8266-High/32341789414.html))
More about it: http://makezine.com/2015/04/01/esp8266-5-microcontroller-wi-fi-now-arduino-compatible/

[C.H.I.P.](https://www.kickstarter.com/projects/1598272670/chip-the-worlds-first-9-computer/description)

[Tessel 2](https://tessel.io)

[Arduino YUN](https://www.arduino.cc/en/Main/ArduinoBoardYun) (also OpenWRT)

[PINE](https://www.kickstarter.com/projects/pine64/pine-a64-first-15-64-bit-single-board-super-comput?ref=thanks_tweet)
[UDOO NEO](http://www.udoo.org/udoo-neo/)

---
# Awesome Johnny Five

http://johnny-five.io

And some issues I mentioned:
https://github.com/rwaldron/johnny-five/issues/112
https://github.com/rwaldron/johnny-five/issues/534

---

My home automation control panel in action: https://youtu.be/wXo2cdHL4Ys

---

What are Telegram Bots: https://core.telegram.org/bots

---

# Inspiration, cool projects

* Leonid Kaganov’s [home server](https://translate.google.com/translate?hl=en&sl=ru&tl=en&u=https%3A%2F%2Flleo.me%2Fdnevnik%2F2010%2F04%2F27.html)
* Rachel’s [cat feeder](https://github.com/rachelnicole/robokitty)
* DIY [Raspberry Pi Mirror](http://blog.dylanjpierce.com/raspberrypi/magicmirror/tutorial/2015/12/27/build-a-magic-mirror.html)
* Amazing [SidLee Dashboard](http://dashboard.sidlee.com)
* [Netflix socks](https://makeit.netflix.com)
* [PubNub](https://www.pubnub.com/blog)

---

Koti Home is open source: http://github.com/arturi/kotihome

---

# Follow me

* [arturpaikin.com](http://arturpaikin.com)
* [twitter.com/arturi](http://twitter.com/arturi)
* [instagram.com/arturi](http://instagram.com/arturi)
* [github.com/arturi](http://github.com/arturi)

C32ae87863e9a773986f65f5d369e551?s=128

Artur Paikin

January 06, 2016
Tweet

Transcript

  1. 3.

    ▸ Travel & write stories about it at arturpaikin.com ▸

    Web developer, also teach web development for beginners ▸ Run a small pop-up coffee shop Tipi Coffee
  2. 6.

    BRIEF HISTORY OF MY ADDICTION 1. Built a blinking LED

    & read thermistor temperature data 2. Made it available on the Internet with an Ethernet shield and Dynamic DNS 3. Told everything about in (too much) detail to my girlfriend 4. Was hooked
  3. 8.

    ARDUINO PEACE OF ! ▸ Tons of great tutorials &

    cool community ▸ Robust libraries for sensors, protocols and whatever ▸ Digital and Analog I/O
  4. 9.

    void setup() { pinMode(13, OUTPUT); } void loop() { digitalWrite(13,

    HIGH); delay(1000); digitalWrite(13, LOW); delay(1000); }
  5. 10.

    ! WOW, I CAN CHECK THE TEMPERATURE & TURN THE

    LIGHT ON AND OFF THROUGH THE INTERNET "
  6. 11.
  7. 13.

    Actually, on some forum I found a message that was

    like: “I thought the whole point of this home automation stuff is that anybody can turn the light on at your house” !
  8. 15.

    ARDUINO IS BEST AT HANDLING DIGITAL & ANALOG I/O, SO

    LET’S LEAVE IT AT THAT Arduino-like low-level I/O + smart Linux controller = ❤
  9. 17.

    ▸ Powerful CPU, built in 4GB flash memory, low-power usage

    ▸ Relatively small community, lack of libraries (One Wire) ▸ Connecting some sensors requires a lot of fiddling (like building a “device tree” !) ▸ Pre-installed distro with a lot of cruft I did’t need and had to delete (not easy) ▸ Sometimes confusing pinout ▸ No audio, micro-HDMI, one USB port (MacBook-style) ▸ “Just use JavaScript” = Bonescript lib. Not well maintained, not following error first node callback guidelines, so promisify fails Not saying it’s bad and most of it applies to other boards too. Just wasn’t for me.
  10. 19.
  11. 20.

    ENTER RASPBERRY PI 2 ▸ 4 USB ports, audio &

    standard HDMI ▸ Huge community, less bugs, great tutorials & cooler website
  12. 21.

    Technically you can use just the Pi for home automation,

    and some people do, it’s got 40 GPIO pins. But I wanted analog sensors and all the libs Arduino community provides. Might reconsider.
  13. 25.
  14. 27.

    WHAT ELSE IS OUT THERE ▸ ESP8266 (NodeMcu Lua Whatever)

    basically an Arduino Nano, but with Wi-Fi and for $5 (official Wi-Fi module for Arduino is $49.90) ! ▸ LightBlue Bean bluetooth, fun, $30
  15. 28.

    ▸ C.H.I.P. promises Wi-Fi & Bluetooth, battery power circuit, not

    sure about analog I/O and ease of use (see BeagleBone). Impressive, $9 ▸ Tessel 2 promises awesomeness (might be a bit slow for some projects, OpenWRT might be limited for some cases) ▸ Arduino YUN (also OpenWRT) ▸ PINE is on the way for $15
  16. 29.

    UDOO NEO promises best of all worlds: Linux & Android,

    Arduino-compatible digital & analog pins, Wi-Fi + Bluetooth and even some sensors. $60 ! I’ll have to check it out.
  17. 32.

    JOHNNY FIVE IS GREAT but Doesn’t support some sensors and

    chips yet, like DHT 11 / 22 or radio 433 Mhz RF Not that big a deal, but I didn’t want to be tied to Firmata / Johnny Five + wanted to see what it’s like out there ! Plus, with a lot of controllers requires tethering
  18. 33.

    ROCKING JSON WITH ARDUINO void loop() { Serial.println(""); Serial.print("{"); Serial.print("\"temp\":

    "); Serial.print(getTemp()); Serial.print(","); Serial.print("\"lux\": "); Serial.print(getLux()); Serial.print("}"); delay(1000); }
  19. 34.
  20. 37.
  21. 38.

    THREE WAYS TO CONTROL THE ROBOT 1. Look & click

    on the dashboard 2. Talk & listen to the dashboard 3. Send & receive messages through Telegram app3 3 Like Whatsapp, but multi-platform and with bots
  22. 40.

    Dashboard.js import React from 'react'; import { LightBtn } from

    './LightBtn'; import { TempMeter } from './TempMeter'; import { Microphone } from './Microphone'; export class Dashboard extends React.Component { constructor(props) { super(props); this.state = { temp: 0, lightIsOn: 0, bgColor: Colors.coral }; this.socket = io(); this.socket.on('status', data => { this.setState( {temp: data.temp || 0, lightIsOn: data.lightIsOn || 0} ); }); } render() { return ( <div className="dashboard" style={{backgroundColor: this.state.bgColor}}> <LightBtn lightIsOn={this.state.lightIsOn} /> <TempMeter temp={this.state.temp} /> <Microphone /> </div> ); } }
  23. 41.

    LightBtn.js import React from 'react'; export class LightBtn extends React.Component

    { onClick() { fetch('/command/lightSwitch', {credentials: 'include'}) .then(function(response) { return response.text(); }); } render() { let classString = 'lampIcon' + (this.props.lightIsOn ? ' isOn' : ''); return ( <button className="dashboardItemBlock lampBlock" title="turn light on / off"> <svg className={classString} onClick={this.onClick}> <path d="M2.51791892,8.1966 C2.54591892,8.15385 ..."></path> <path d="M7.54164865,9.45136 C7.25275676,9.31133 ..."></path> </svg> </button> ); } };
  24. 42.

    VOICE RECOGNITION & SYNTHESIS API RIGHT IN YOU BROWSER1, ACCESSIBLE

    VIA JAVASCRIPT ! const recognition = new webkitSpeechRecognition(); const utterance = new SpeechSynthesisUtterance(msg); const, such ES2015 1 That currently means Chrome
  25. 44.
  26. 45.

    ▸ Problem: Dynamic IPs & Firewalls ▸ Solution: hub establishes

    a connection to VPS server, not the other way around ▸ HTTP is too bloated, so send messages through TCP. 5 days all good, then connection hang (that’s how I learned what keep alive is for) ▸ Trying to reinvent the wheel (sometimes a good thing)
  27. 46.

    ! MQTT & WEB SOCKETS ! ▸ Publish-Subscribe ▸ Quality

    of Service (must be good) ▸ Minimal overhead The hub connects to a VPS which is running MQTT via Web Sockets, through NGINX, which is using SSL cert2 2 This is done so that I can use the same cert I have for my domain, where the control panel lives
  28. 47.

    var mqttClient = mqtt.connect(broker, { username: user, password: pass });

    // Get data from Arduino’s sensors, check if it’s new // and if so, send to the server var oldData = controller.getData(); function publishNewData() { var newData = controller.getData(); if (oldData !== newData) { oldData = newData; mqttClient.publish('/koti/data', newData); } } mqttClient.on('connect', function() { mqttClient.subscribe('/koti/commands'); publishNewDataInterval = setInterval(publishNewData, 1000); });
  29. 48.

    ROBOT ▸ Robot receives a command through POST-request, thinks about,

    maybe takes some action, then replies back ▸ Parses sentences, splits them into words ▸ if containsAny(words, dictionary.greetings), reply with hello ▸ Random answers from specific dictionaries (greetings, confirmations, sex)
  30. 49.

    function containsAny(haystack, needles) { if ( _.intersection(haystack, needles).length > 0

    ) { return true; } return false; } if ( containsAny(words, this.dictionary.greetings) ) { matchFound = true; return this.react('hello', this.lastUpdate); }
  31. 50.

    case 'light': function lightSwitch() { self.controller.sendCommand('lightSwitch'); return self.react('confirmation', lastUpdate); }

    if (this.lastUpdate.method === 'telegram') { // checks if the msg is coming from an actual house owner if ( telegram.tremblingCreatureOrHaveITheRight(lastUpdate.from.username) ) { return lightSwitch(); } else { return this.react('howDareYou', lastUpdate); } } else { return lightSwitch(); } break;
  32. 51.

    WHAT I LEARNED ▸ Arduino, low-level hardware stuff & what

    is possible ▸ Raspberry Pi & Unix stuff ▸ Continuously running apps (I used PM2) ▸ React ▸ If your project features cats, it will get more stars on GitHub
  33. 52.

    SHIT HAPPENED ▸ Exceeded IFTTT’s monthly message cap with movement

    alert ▸ Node App was leaking memory. Turns out: setInterval on every client connect, had to use clearInterval on disconnect (thanks to my friend, real Java developer) ▸ Saw that there was some movement at home when no one was supposed to be there. Ran home. Turns out: supplied less power than necessary to Arduino and it freaked out. ▸ Raspberry Pi was hanging dead each time I turned the light on. Turns out: power issues again — the USB hub was either bad or needed more power.
  34. 53.

    INSPIRATION, COOL PROJECTS ▸ Leonid Kaganov’s home server ▸ Rachel’s

    cat feeder ▸ DIY Raspberry Pi Mirror ▸ Amazing SidLee Dashboard ▸ Netflix socks ▸ PubNub
  35. 54.

    THE END ▸ Make something fun, weird and non-profitable ▸

    Celebrate your little achievements and be proud of them