Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Android Things Workshop - Building a Motion Sensing Camera

Android Things Workshop - Building a Motion Sensing Camera

In this workshop, attendees built an Android Things motion sensing camera using the Google Android Things libraries. The slides acted as a guide throughout the workshop.
The full hackster project can be found here - https://www.hackster.io/riggaroo/smart-motion-sensing-camera-with-intruder-notifications-b6c613
With the Github Repository for the workshop with the different steps: https://github.com/riggaroo/android-things-workshop-camera

Rebecca Franks

November 11, 2017
Tweet

More Decks by Rebecca Franks

Other Decks in Programming

Transcript

  1. View Slide

  2. View Slide

  3. View Slide

  4. View Slide

  5. View Slide

  6. extension
    IoT embedded

    View Slide

  7. powerful intelligent
    secure

    View Slide

  8. Android SDK Play Services Firebase
    Android Studio Cloud Platform

    View Slide

  9. No Play Store Deploy OTAs
    Subset of
    APIs
    Custom
    Hardware
    Single Purpose
    Device

    View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. SoM
    Architecture
    Google
    Managed BSP

    View Slide

  14. Android Framework
    Hardware Libraries
    Linux Kernel
    Apps
    User Drivers

    View Slide





  15. View Slide

  16. View Slide

  17. View Slide

  18. View Slide

  19. View Slide

  20. View Slide

  21. Button GPS
    PWM Servo
    RGB LED Strip
    Temperature
    Sensor
    Capacitive
    Touch Buttons
    GPIO PWM
    I2C SPI
    UART I2S

    View Slide

  22. View Slide

  23. View Slide

  24. View Slide

  25. View Slide

  26. View Slide

  27. View Slide

  28. Electronics Basics

    View Slide

  29. Components
    Breadboard - Construction base for
    prototyping electronics
    Jumper Wire - Used to interconnect the
    components of a breadboard

    View Slide

  30. LEDs - Emits visible light when an
    electric current passes through it
    Push Switches - Allows electricity
    to flow between its two contacts
    Components

    View Slide

  31. Resistor -Used to reduce current flow
    Source: http://www.electronicshub.org/resistor-color-code/
    Components
    and many more...

    View Slide

  32. Ohm’s Law
    The current through a conductor
    between two points is directly
    proportional to the voltage across the
    two points.
    V = I * R
    V is Voltage (volts)
    I is Current (amps)
    R is Resistance (ohms)
    I = 0.023A
    V = 5V
    R = 5V / 0.023A
    R = ± 217 ohms

    View Slide

  33. View Slide

  34. View Slide

  35. Red, Red, Brown = 220 ohm
    Brown, Black, Orange = 1k ohm

    View Slide

  36. View Slide

  37. View Slide

  38. View Slide

  39. View Slide

  40. View Slide

  41. Pico Setup: bit.ly/pico-pro-setup

    View Slide

  42. View Slide

  43. //Boot into fastboot mode
    $ adb reboot bootloader
    // Get list of fastboot devices
    $ fastboot devices
    1b2f21d4e1fe0129 fastboot
    //Locate directory with factory image
    $ ./flash_all.sh //windows ./flash_all.bat
    // Get list of devices once flashed
    $ adb devices
    List of devices attached
    1b2f21d4e1fe0129 device

    View Slide

  44. //Start internet
    $ adb shell am startservice -n com.google.wifisetup/.WifiSetupService -a
    WifiSetupService.Connect -e ssid "Jozihub Events" -e passphrase Jozihub#

    View Slide

  45. View Slide

  46. View Slide

  47. View Slide

  48. View Slide

  49. View Slide

  50. //Automatically added in app level build.gradle
    compileOnly 'com.google.android.things:androidthings:...'

    View Slide

  51. //Automatically added in AndroidManifest.xml






    android:name="android.intent.category.IOT_LAUNCHER"/>




    View Slide

  52. //Automatically added in AndroidManifest.xml






    android:name="android.intent.category.IOT_LAUNCHER"/>




    View Slide

  53. View Slide

  54. //Accessing Sensor Data
    private val buttonGpio: Gpio = PeripheralManagerService().openGpio(“GPIO_175”)
    fun start() {
    buttonGpio.setDirection(Gpio.DIRECTION_IN)
    buttonGpio.setActiveType(Gpio.ACTIVE_HIGH)
    buttonGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    buttonGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    ledGpio.value = true
    } else {
    //do something else
    }
    return true
    }
    })
    }

    View Slide

  55. //Accessing Sensor Data
    private val buttonGpio: Gpio = PeripheralManagerService().openGpio(“GPIO_175”)
    fun start() {
    buttonGpio.setDirection(Gpio.DIRECTION_IN)
    buttonGpio.setActiveType(Gpio.ACTIVE_HIGH)
    buttonGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    buttonGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    ledGpio.value = true
    } else {
    //do something else
    }
    return true
    }
    })
    }

    View Slide

  56. //Accessing Sensor Data
    private val buttonGpio: Gpio = PeripheralManagerService().openGpio(“GPIO_175”)
    fun start() {
    buttonGpio.setDirection(Gpio.DIRECTION_IN)
    buttonGpio.setActiveType(Gpio.ACTIVE_HIGH)
    buttonGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    buttonGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    ledGpio.value = true
    } else {
    //do something else
    }
    return true
    }
    })
    }

    View Slide

  57. View Slide

  58. //Accessing Sensor Data
    private val motionSensorGpio: Gpio = PeripheralManagerService().openGpio(“GPIO_35”)
    fun start() {
    motionSensorGpio.setDirection(Gpio.DIRECTION_IN)
    motionSensorGpio.setActiveType(Gpio.ACTIVE_HIGH)
    motionSensorGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    motionSensorGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    motionListener.onMotionDetected()
    } else {
    motionListener.onMotionStopped()
    }
    return true
    }
    })
    }

    View Slide

  59. //Accessing Sensor Data
    private val motionSensorGpio: Gpio = PeripheralManagerService().openGpio(“GPIO_35”)
    fun start() {
    motionSensorGpio.setDirection(Gpio.DIRECTION_IN)
    motionSensorGpio.setActiveType(Gpio.ACTIVE_HIGH)
    motionSensorGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    motionSensorGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    motionListener.onMotionDetected()
    } else {
    motionListener.onMotionStopped()
    }
    return true
    }
    })
    }

    View Slide

  60. //Accessing Sensor Data
    private val motionSensorGpio: Gpio = PeripheralManagerService().openGpio(“GPIO_35”)
    fun start() {
    motionSensorGpio.setDirection(Gpio.DIRECTION_IN)
    motionSensorGpio.setActiveType(Gpio.ACTIVE_HIGH)
    motionSensorGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    motionSensorGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    camera.takePhoto()
    } else {
    //do something else
    }
    return true
    }
    })
    }

    View Slide

  61. View Slide

  62. val peripheralManagerService = PeripheralManagerService()
    ledGpio = peripheralManagerService.openGpio(“GPIO_174”)
    ledGpio.setDirection(Gpio.DIRECTION_OUT_INITIALLY_LOW)
    ledGpio.value = true
    ledGpio.close()

    View Slide

  63. val peripheralManagerService = PeripheralManagerService()
    ledGpio = peripheralManagerService.openGpio(“GPIO_174”)
    ledGpio.setDirection(Gpio.DIRECTION_OUT_INITIALLY_LOW)
    ledGpio.value = true
    ledGpio.close()

    View Slide

  64. val peripheralManagerService = PeripheralManagerService()
    ledGpio = peripheralManagerService.openGpio(“GPIO_174”)
    ledGpio.setDirection(Gpio.DIRECTION_OUT_INITIALLY_LOW)
    ledGpio.value = true
    ledGpio.close()

    View Slide

  65. View Slide

  66. View Slide

  67. private fun setupCamera() {
    camera = CustomCamera.getInstance()
    camera.initializeCamera(this, Handler(), imageAvailableListener)
    }
    private val imageAvailableListener = object : CustomCamera.ImageCapturedListener {
    override fun onImageCaptured(bitmap: Bitmap) {
    motionImageView.setImageBitmap(bitmap)
    motionViewModel.uploadMotionImage(bitmap)
    }
    }
    camera.takePicture()

    View Slide

  68. private fun setupCamera() {
    camera = CustomCamera.getInstance()
    camera.initializeCamera(this, Handler(), imageAvailableListener)
    }
    private val imageAvailableListener = object : CustomCamera.ImageCapturedListener {
    override fun onImageCaptured(bitmap: Bitmap) {
    motionImageView.setImageBitmap(bitmap)
    motionViewModel.uploadMotionImage(bitmap)
    }
    }
    camera.takePicture()

    View Slide

  69. private fun setupCamera() {
    camera = CustomCamera.getInstance()
    camera.initializeCamera(this, Handler(), imageAvailableListener)
    }
    private val imageAvailableListener = object : CustomCamera.ImageCapturedListener {
    override fun onImageCaptured(bitmap: Bitmap) {
    motionImageView.setImageBitmap(bitmap)
    motionViewModel.uploadMotionImage(bitmap)
    }
    }
    camera.takePicture()

    View Slide

  70. View Slide

  71. View Slide

  72. View Slide

  73. //data class for saving
    data class FirebaseImageLog(val timestamp: Long? = null, val imageRef: String? = null)
    //upload image
    val storageRef = FirebaseStorage.getInstance().getReference(FIREBASE_MOTION_REF)
    val imageStorageRef = storageRef.child(FIREBASE_IMAGE_PREFIX + System.currentTimeMillis() + ".jpg")
    val uploadTask = imageStorageRef.putBytes(imageStream.toByteArray())
    uploadTask.addOnFailureListener {
    Log.d(TAG, "onFailure uploadMotionImage")
    }.addOnSuccessListener {
    // save image log to realtime db
    val downloadUrl = imageStorageRef.path
    val ref = FirebaseDatabase.getInstance().getReference(FIREBASE_MOTION_LOGS).push()
    ref.setValue(FirebaseImageLog(System.currentTimeMillis(), downloadUrl))
    }

    View Slide

  74. //data class for saving
    data class FirebaseImageLog(val timestamp: Long? = null, val imageRef: String? = null)
    //upload image
    val storageRef = FirebaseStorage.getInstance().getReference(FIREBASE_MOTION_REF)
    val imageStorageRef = storageRef.child(FIREBASE_IMAGE_PREFIX + System.currentTimeMillis() + ".jpg")
    val uploadTask = imageStorageRef.putBytes(imageStream.toByteArray())
    uploadTask.addOnFailureListener {
    Log.d(TAG, "onFailure uploadMotionImage")
    }.addOnSuccessListener {
    // save image log to realtime db
    val downloadUrl = imageStorageRef.path
    val ref = FirebaseDatabase.getInstance().getReference(FIREBASE_MOTION_LOGS).push()
    ref.setValue(FirebaseImageLog(System.currentTimeMillis(), downloadUrl))
    }

    View Slide

  75. //data class for saving
    data class FirebaseImageLog(val timestamp: Long? = null, val imageRef: String? = null)
    //upload image
    val storageRef = FirebaseStorage.getInstance().getReference(FIREBASE_MOTION_REF)
    val imageStorageRef = storageRef.child(FIREBASE_IMAGE_PREFIX + System.currentTimeMillis() + ".jpg")
    val uploadTask = imageStorageRef.putBytes(imageStream.toByteArray())
    uploadTask.addOnFailureListener {
    Log.d(TAG, "onFailure uploadMotionImage")
    }.addOnSuccessListener {
    // save image log to realtime db
    val downloadUrl = imageStorageRef.path
    val ref = FirebaseDatabase.getInstance().getReference(FIREBASE_MOTION_LOGS).push()
    ref.setValue(FirebaseImageLog(System.currentTimeMillis(), downloadUrl))
    }

    View Slide

  76. //data class for saving
    data class FirebaseImageLog(val timestamp: Long? = null, val imageRef: String? = null)
    //upload image
    val storageRef = FirebaseStorage.getInstance().getReference(FIREBASE_MOTION_REF)
    val imageStorageRef = storageRef.child(FIREBASE_IMAGE_PREFIX + System.currentTimeMillis() + ".jpg")
    val uploadTask = imageStorageRef.putBytes(imageStream.toByteArray())
    uploadTask.addOnFailureListener {
    Log.d(TAG, "onFailure uploadMotionImage")
    }.addOnSuccessListener {
    // save image log to realtime db
    val downloadUrl = imageStorageRef.path
    val ref = FirebaseDatabase.getInstance().getReference(FIREBASE_MOTION_LOGS).push()
    ref.setValue(FirebaseImageLog(System.currentTimeMillis(), downloadUrl))
    }

    View Slide

  77. View Slide

  78. View Slide

  79. $ firebase login
    $ firebase init functions

    View Slide

  80. var functions = require('firebase-functions');
    var admin = require('firebase-admin');
    admin.initializeApp(functions.config().firebase);
    exports.annotateImage = functions.database.ref('/motion-logs/{id}')
    .onCreate(event => {
    const original = event.data.val();
    var topic = "/topics/intruders";
    var payload = {
    data: {
    title: "Intruder Alert!",
    body: "An intruder has been detected"
    }
    };
    return admin.messaging().sendToTopic(topic, payload)
    .then(function (response) {
    console.log("Successfully sent message:", response);
    }).catch(function (error) {
    console.log("Error sending message:", error);
    });
    }
    );

    View Slide

  81. var functions = require('firebase-functions');
    var admin = require('firebase-admin');
    admin.initializeApp(functions.config().firebase);
    exports.annotateImage = functions.database.ref('/motion-logs/{id}')
    .onCreate(event => {
    const original = event.data.val();
    var topic = "/topics/intruders";
    var payload = {
    data: {
    title: "Intruder Alert!",
    body: "An intruder has been detected"
    }
    };
    return admin.messaging().sendToTopic(topic, payload)
    .then(function (response) {
    console.log("Successfully sent message:", response);
    }).catch(function (error) {
    console.log("Error sending message:", error);
    });
    }
    );

    View Slide

  82. var functions = require('firebase-functions');
    var admin = require('firebase-admin');
    admin.initializeApp(functions.config().firebase);
    exports.annotateImage = functions.database.ref('/motion-logs/{id}')
    .onCreate(event => {
    const original = event.data.val();
    var topic = "/topics/intruders";
    var payload = {
    data: {
    title: "Intruder Alert!",
    body: "An intruder has been detected"
    }
    };
    return admin.messaging().sendToTopic(topic, payload)
    .then(function (response) {
    console.log("Successfully sent message:", response);
    }).catch(function (error) {
    console.log("Error sending message:", error);
    });
    }
    );

    View Slide

  83. $ firebase deploy --only functions

    View Slide

  84. View Slide

  85. View Slide

  86. //Use FirebaseUI dependency
    api "com.firebaseui:firebase-ui-database:3.0.0"
    // Get DB Reference
    val databaseRef = FirebaseDatabase.getInstance().getReference(MOTION_LOGS_FIREBASE_REF)
    // Create adapter that extends FirebaseRecyclerAdapter
    class LogsAdapter(ref: DatabaseReference) : FirebaseRecyclerAdapterLogsViewHolder>(FirebaseImageLog::class.java,
    R.layout.list_item_log, LogsViewHolder::class.java, ref) {
    //.. Do adapter things
    }
    // Set adapter to use reference
    adapter = LogsAdapter(databaseRef.orderByChild(ORDER_BY_TIMESTAMP).ref)

    View Slide

  87. //Use FirebaseUI dependency
    api "com.firebaseui:firebase-ui-database:3.0.0"
    // Get DB Reference
    val databaseRef = FirebaseDatabase.getInstance().getReference(MOTION_LOGS_FIREBASE_REF)
    // Create adapter that extends FirebaseRecyclerAdapter
    class LogsAdapter(ref: DatabaseReference) : FirebaseRecyclerAdapterLogsViewHolder>(FirebaseImageLog::class.java,
    R.layout.list_item_log, LogsViewHolder::class.java, ref) {
    //.. Do adapter things
    }
    // Set adapter to use reference
    adapter = LogsAdapter(databaseRef.orderByChild(ORDER_BY_TIMESTAMP).ref)

    View Slide

  88. //Use FirebaseUI dependency
    api "com.firebaseui:firebase-ui-database:3.0.0"
    // Get DB Reference
    val databaseRef = FirebaseDatabase.getInstance().getReference(MOTION_LOGS_FIREBASE_REF)
    // Create adapter that extends FirebaseRecyclerAdapter
    class LogsAdapter(ref: DatabaseReference) : FirebaseRecyclerAdapterLogsViewHolder>(FirebaseImageLog::class.java,
    R.layout.list_item_log, LogsViewHolder::class.java, ref) {
    //.. Do adapter things
    }
    // Set adapter to use reference
    adapter = LogsAdapter(databaseRef.orderByChild(ORDER_BY_TIMESTAMP).ref)

    View Slide

  89. //Use FirebaseUI dependency
    api "com.firebaseui:firebase-ui-database:3.0.0"
    // Get DB Reference
    val databaseRef = FirebaseDatabase.getInstance().getReference(MOTION_LOGS_FIREBASE_REF)
    // Create adapter that extends FirebaseRecyclerAdapter
    class LogsAdapter(ref: DatabaseReference) : FirebaseRecyclerAdapterLogsViewHolder>(FirebaseImageLog::class.java,
    R.layout.list_item_log, LogsViewHolder::class.java, ref) {
    //.. Do adapter things
    }
    // Set adapter to use reference
    adapter = LogsAdapter(databaseRef.orderByChild(ORDER_BY_TIMESTAMP).ref)

    View Slide

  90. View Slide

  91. View Slide

  92. View Slide

  93. View Slide

  94. Clear all Android Apps on device:
    https://gist.github.com/blundell/7c0c3bb17898b28fe8122b0dc230af50
    Button Example : https://github.com/androidthings/sample-button
    Full MainActivity:
    https://gist.github.com/riggaroo/b4cad83be2f81d54fd854403af029b2d
    Github Project with Steps as branches:
    https://github.com/riggaroo/android-things-workshop-camera

    View Slide