Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Sentinel - The First Home Security Robot Powered by Android Things (DroidCon NYC 2017)

Sentinel - The First Home Security Robot Powered by Android Things (DroidCon NYC 2017)


Oscar Salguero

September 26, 2017


  1. Sentinel The First Home Security Robot Powered by Android Things™

    https://www.hackster.io/oscarsalguero/sentinel-8234b3 Oscar Salguero @OscarSalguero
  2. Agenda • Demo • Motivation • Hardware • Software •

    Roadmap • Tips
  3. Demo

  4. Video

  5. Video

  6. Motivation

  7. • Home security cameras are expensive • Most surveillance services

    have crazy high monthly fees • Gotta have a camera on each room • IP cameras are hackable • Setup is complicated • Can’t see anything from a WAN without DDNS • OEM and 3rd party remote viewing apps are unreliable Motivation
  8. • Don’t talk to other smart home/IoT hardware • Still

    use email to send you alerts • Alert! A leaf moved! (notifications are irrelevant) • No face detection nor facial recognition • No image classification • Can’t find your pet if it walked out of frame Motivation
  9. • Pets move from room to room or go outside

    • Entertainment…? Throw a tennis ball to Fido from anywhere! • Check if your pets have food or water • Keep an eye on the kids • See if a delivery arrived Motivation
  10. Hardware

  11. Rover 5 Platform https://www.pololu.com/product/1535

  12. Raspberry Pi 3, 32GB SD Card with Android Things ™

  13. Adafruit DC and Stepper Motor HAT for RPi https://www.adafruit.com/product/2348

  14. Adafruit 16-Channel PWM Servo HAT for RPi https://www.adafruit.com/product/2327

  15. Raspberry Pi Camera Module v2 https://www.adafruit.com/product/3099

  16. https://www.sparkfun.com/products/ 10335 Pan/Tilt Bracket

  17. 3D Printed Camera Mount https://goo.gl/hMTwSb

  18. 8Bitdo NES30 Pro Bluetooth Controller http://www.8bitdo.com/n30pro-f30pro/

  19. Android Phone https://store.google.com/category/phones

  20. • Stereo speakers • USB Mic • 30,000 mah Power

    bank • Couple of plastic expansion plates • USB and barrel jack cables, wires • Standoffs, nuts, screws, etc. Other Stuff...
  21. Software

  22. Android... Android Everywhere! • Android Studio 3.0 Beta • Android

    Things ™ Developer Preview • Antonio Zugaldia’s Android Robocar Project (Adafruit PWM and Motor Hat Java Library) http://androidrobocar.com
  23. Firebase & Google Cloud

  24. DC and Stepper Motor Hat Driver // I2C Bus Name

    public static final String I2C_DEVICE_NAME = "I2C1"; // Motor Hat Address private static final int MOTOR_HAT_I2C_ADDRESS = 0x60; // Motor Hat private AdafruitMotorHat mMotorHat; // Motors private AdafruitDcMotor motorFrontLeft, motorFrontRight, motorBackLeft, motorBackRight; https://github.com/zugaldia/android-robocar
  25. DC and Stepper Motor Hat Driver mMotorHat = new AdafruitMotorHat(I2C_DEVICE_NAME,

    MOTOR_HAT_I2C_ADDRESS); motorFrontLeft = mMotorHat.getMotor(1); motorBackLeft = mMotorHat.getMotor(2); motorFrontRight = mMotorHat.getMotor(3); motorBackRight = mMotorHat.getMotor(4);
  26. private void moveForward() { Log.d(LOG_TAG, "Moving forward."); motorFrontLeft.run(AdafruitMotorHat.FORWARD); motorFrontRight.run(AdafruitMotorHat.BACKWARD); motorBackLeft.run(AdafruitMotorHat.BACKWARD);

    motorBackRight.run(AdafruitMotorHat.FORWARD); } private void turnRight() { Log.d(LOG_TAG, "Turning right."); motorFrontLeft.run(AdafruitMotorHat.FORWARD); motorFrontRight.run(AdafruitMotorHat.FORWARD); motorBackLeft.run(AdafruitMotorHat.BACKWARD); motorBackRight.run(AdafruitMotorHat.BACKWARD); } DC and Stepper Motor Hat Driver
  27. 16 Channel Servo HAT Driver // Servo Hat Address private

    static final int SERVO_HAT_I2C_ADDRESS = 0x41; // Servo Hat private AdafruitServoHat mServoHat; // Servos private AdafruitServo mTiltServo; private AdafruitServo mPanServo; https://github.com/RacZo/adafruit-16-channel-servo-hat-driver
  28. 16 Channel Servo HAT Driver // Servo Hat initialization mServoHat

    = new AdafruitServoHat(I2C_DEVICE_NAME, SERVO_HAT_I2C_ADDRESS); // Variables to configure the servos according to their manufacturer specs double minAngle = 20.0; double maxAngle = 150.0; double minPulseDurationRange = 0.7; // ms double maxPulseDurationRange = 1.50; // ms int frequency = 40;
  29. // Add the servos to the hat in the order

    they are connected from index 0 to 15 mPanServo = new AdafruitServo(0, frequency); mPanServo.setPulseDurationRange(minPulseDurationRange, maxPulseDurationRange); mPanServo.setAngleRange(minAngle, maxAngle); mPanServo.setAngle(SERVO_INITIAL_PAN_ANGLE); mServoHat.addServo(mPanServo); mTiltServo = new AdafruitServo(1, frequency); ... mServoHat.addServo(mTiltServo); … // Add the servos to the hat in the order they are connected from index 0 to 15 mServoHat.rotateToAngle(mPanServo, 90); mServoHat.rotateToAngle(mTiltServo, 90); 16 Channel Servo HAT Driver
  30. Companion Android App • See your pet from anywhere! •

    Remote Control • Live feed (kind off) • Motion and loud sound alerts • Image history & management
  31. Tips

  32. Tips • Draw, sketch or diagram your idea first •

    Write down a list of materials and a list of desirable features • Research about the technologies such as APIs, protocols, etc. you’ll need to achieve the desired features, research about the parts and materials before purchasing them; pros/cons matter! • Use Etcher to push Android Things image to an SD Card • Learn to love adb and make a command cheat sheet
  33. Tips • Add speech-to-text help you debug stuff • Run

    and analyze sample code (GitHub), ask questions (forums, Stack Overflow), watch how it is done (YouTube) • Redundancy (if you can, buy more than 1 board) • Learn about circuits, voltages and currents beforehand and be mindful about power requirements • Use rechargeable batteries • Always have a wired USB keyboard, mouse and an HDMI cable at hand • Try not to burn your fingers when soldering headers (It hurts!)
  34. The Road Ahead Software Improvements • Real time video &

    2-way audio (Web RTC) • Face detection and feature tracking (OpenCV) • Facial recognition (TF) • Real time image classification (TF) • Voice control (Google Voice Actions) tinyLiDAR (HW + SW) • Indoor mapping and navigation using • Follow moving targets • Autonomous patrolling
  35. tinyLIDAR

  36. tinyLIDAR

  37. Questions? This presentation is available at: https://goo.gl/EgvWFd

  38. Thank You! Oscar Salguero @OscarSalguero