Android Things Workshop - Building a Motion Sensing Camera

In this workshop, attendees built an Android Things motion sensing camera using the Google Android Things libraries. The slides acted as a guide throughout the workshop.
The full hackster project can be found here - https://www.hackster.io/riggaroo/smart-motion-sensing-camera-with-intruder-notifications-b6c613
With the Github Repository for the workshop with the different steps: https://github.com/riggaroo/android-things-workshop-camera

Transcript

1. View Slide

2. View Slide

3. View Slide

4. View Slide

5. View Slide

6. extension
   IoT embedded
   

   View Slide

7. powerful intelligent
   secure
   

   View Slide

8. Android SDK Play Services Firebase
   Android Studio Cloud Platform
   

   View Slide

9. No Play Store Deploy OTAs
   Subset of
   APIs
   Custom
   Hardware
   Single Purpose
   Device
   

   View Slide

10. View Slide

11. View Slide

12. View Slide

13. SoM
    Architecture
    Google
    Managed BSP
    

    View Slide

14. Android Framework
    Hardware Libraries
    Linux Kernel
    Apps
    User Drivers
    

    View Slide

15. ●
    ●
    ●
    ●
    

    View Slide

16. View Slide

17. View Slide

18. View Slide

19. View Slide

20. View Slide

21. Button GPS
    PWM Servo
    RGB LED Strip
    Temperature
    Sensor
    Capacitive
    Touch Buttons
    GPIO PWM
    I2C SPI
    UART I2S
    

    View Slide

22. View Slide

23. View Slide

24. View Slide

25. View Slide

26. View Slide

27. View Slide

28. Electronics Basics
    

    View Slide

29. Components
    Breadboard - Construction base for
    prototyping electronics
    Jumper Wire - Used to interconnect the
    components of a breadboard
    

    View Slide

30. LEDs - Emits visible light when an
    electric current passes through it
    Push Switches - Allows electricity
    to flow between its two contacts
    Components
    

    View Slide

31. Resistor -Used to reduce current flow
    Source: http://www.electronicshub.org/resistor-color-code/
    Components
    and many more...
    

    View Slide

32. Ohm’s Law
    The current through a conductor
    between two points is directly
    proportional to the voltage across the
    two points.
    V = I * R
    V is Voltage (volts)
    I is Current (amps)
    R is Resistance (ohms)
    I = 0.023A
    V = 5V
    R = 5V / 0.023A
    R = ± 217 ohms
    

    View Slide

33. View Slide

34. View Slide

35. Red, Red, Brown = 220 ohm
    Brown, Black, Orange = 1k ohm
    

    View Slide

36. View Slide

37. View Slide

38. View Slide

39. View Slide

40. View Slide

41. Pico Setup: bit.ly/pico-pro-setup
    

    View Slide

42. View Slide

43. //Boot into fastboot mode
    $ adb reboot bootloader
    // Get list of fastboot devices
    $ fastboot devices
    1b2f21d4e1fe0129 fastboot
    //Locate directory with factory image
    $ ./flash_all.sh //windows ./flash_all.bat
    // Get list of devices once flashed
    $ adb devices
    List of devices attached
    1b2f21d4e1fe0129 device
    

    View Slide

44. //Start internet
    $ adb shell am startservice -n com.google.wifisetup/.WifiSetupService -a
    WifiSetupService.Connect -e ssid "Jozihub Events" -e passphrase Jozihub#
    

    View Slide

45. View Slide

46. View Slide

47. View Slide

48. View Slide

49. View Slide

50. //Automatically added in app level build.gradle
    compileOnly 'com.google.android.things:androidthings:...'
    

    View Slide

51. //Automatically added in AndroidManifest.xml
    
    
    
    
    
    
    android:name="android.intent.category.IOT_LAUNCHER"/>
    
    
    
    
    

    View Slide

52. //Automatically added in AndroidManifest.xml
    
    
    
    
    
    
    android:name="android.intent.category.IOT_LAUNCHER"/>
    
    
    
    
    

    View Slide

53. View Slide

54. //Accessing Sensor Data
    private val buttonGpio: Gpio = PeripheralManagerService().openGpio(“GPIO_175”)
    fun start() {
    buttonGpio.setDirection(Gpio.DIRECTION_IN)
    buttonGpio.setActiveType(Gpio.ACTIVE_HIGH)
    buttonGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    buttonGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    ledGpio.value = true
    } else {
    //do something else
    }
    return true
    }
    })
    }
    

    View Slide

55. //Accessing Sensor Data
    private val buttonGpio: Gpio = PeripheralManagerService().openGpio(“GPIO_175”)
    fun start() {
    buttonGpio.setDirection(Gpio.DIRECTION_IN)
    buttonGpio.setActiveType(Gpio.ACTIVE_HIGH)
    buttonGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    buttonGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    ledGpio.value = true
    } else {
    //do something else
    }
    return true
    }
    })
    }
    

    View Slide

56. //Accessing Sensor Data
    private val buttonGpio: Gpio = PeripheralManagerService().openGpio(“GPIO_175”)
    fun start() {
    buttonGpio.setDirection(Gpio.DIRECTION_IN)
    buttonGpio.setActiveType(Gpio.ACTIVE_HIGH)
    buttonGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    buttonGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    ledGpio.value = true
    } else {
    //do something else
    }
    return true
    }
    })
    }
    

    View Slide

57. View Slide

58. //Accessing Sensor Data
    private val motionSensorGpio: Gpio = PeripheralManagerService().openGpio(“GPIO_35”)
    fun start() {
    motionSensorGpio.setDirection(Gpio.DIRECTION_IN)
    motionSensorGpio.setActiveType(Gpio.ACTIVE_HIGH)
    motionSensorGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    motionSensorGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    motionListener.onMotionDetected()
    } else {
    motionListener.onMotionStopped()
    }
    return true
    }
    })
    }
    

    View Slide

59. //Accessing Sensor Data
    private val motionSensorGpio: Gpio = PeripheralManagerService().openGpio(“GPIO_35”)
    fun start() {
    motionSensorGpio.setDirection(Gpio.DIRECTION_IN)
    motionSensorGpio.setActiveType(Gpio.ACTIVE_HIGH)
    motionSensorGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    motionSensorGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    motionListener.onMotionDetected()
    } else {
    motionListener.onMotionStopped()
    }
    return true
    }
    })
    }
    

    View Slide

60. //Accessing Sensor Data
    private val motionSensorGpio: Gpio = PeripheralManagerService().openGpio(“GPIO_35”)
    fun start() {
    motionSensorGpio.setDirection(Gpio.DIRECTION_IN)
    motionSensorGpio.setActiveType(Gpio.ACTIVE_HIGH)
    motionSensorGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    motionSensorGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    camera.takePhoto()
    } else {
    //do something else
    }
    return true
    }
    })
    }
    

    View Slide

61. View Slide

62. val peripheralManagerService = PeripheralManagerService()
    ledGpio = peripheralManagerService.openGpio(“GPIO_174”)
    ledGpio.setDirection(Gpio.DIRECTION_OUT_INITIALLY_LOW)
    ledGpio.value = true
    ledGpio.close()
    

    View Slide

63. val peripheralManagerService = PeripheralManagerService()
    ledGpio = peripheralManagerService.openGpio(“GPIO_174”)
    ledGpio.setDirection(Gpio.DIRECTION_OUT_INITIALLY_LOW)
    ledGpio.value = true
    ledGpio.close()
    

    View Slide

64. val peripheralManagerService = PeripheralManagerService()
    ledGpio = peripheralManagerService.openGpio(“GPIO_174”)
    ledGpio.setDirection(Gpio.DIRECTION_OUT_INITIALLY_LOW)
    ledGpio.value = true
    ledGpio.close()
    

    View Slide

65. View Slide

66. View Slide

67. private fun setupCamera() {
    camera = CustomCamera.getInstance()
    camera.initializeCamera(this, Handler(), imageAvailableListener)
    }
    private val imageAvailableListener = object : CustomCamera.ImageCapturedListener {
    override fun onImageCaptured(bitmap: Bitmap) {
    motionImageView.setImageBitmap(bitmap)
    motionViewModel.uploadMotionImage(bitmap)
    }
    }
    camera.takePicture()
    

    View Slide

68. private fun setupCamera() {
    camera = CustomCamera.getInstance()
    camera.initializeCamera(this, Handler(), imageAvailableListener)
    }
    private val imageAvailableListener = object : CustomCamera.ImageCapturedListener {
    override fun onImageCaptured(bitmap: Bitmap) {
    motionImageView.setImageBitmap(bitmap)
    motionViewModel.uploadMotionImage(bitmap)
    }
    }
    camera.takePicture()
    

    View Slide

69. private fun setupCamera() {
    camera = CustomCamera.getInstance()
    camera.initializeCamera(this, Handler(), imageAvailableListener)
    }
    private val imageAvailableListener = object : CustomCamera.ImageCapturedListener {
    override fun onImageCaptured(bitmap: Bitmap) {
    motionImageView.setImageBitmap(bitmap)
    motionViewModel.uploadMotionImage(bitmap)
    }
    }
    camera.takePicture()
    

    View Slide

70. View Slide

71. View Slide

72. View Slide

73. //data class for saving
    data class FirebaseImageLog(val timestamp: Long? = null, val imageRef: String? = null)
    //upload image
    val storageRef = FirebaseStorage.getInstance().getReference(FIREBASE_MOTION_REF)
    val imageStorageRef = storageRef.child(FIREBASE_IMAGE_PREFIX + System.currentTimeMillis() + ".jpg")
    val uploadTask = imageStorageRef.putBytes(imageStream.toByteArray())
    uploadTask.addOnFailureListener {
    Log.d(TAG, "onFailure uploadMotionImage")
    }.addOnSuccessListener {
    // save image log to realtime db
    val downloadUrl = imageStorageRef.path
    val ref = FirebaseDatabase.getInstance().getReference(FIREBASE_MOTION_LOGS).push()
    ref.setValue(FirebaseImageLog(System.currentTimeMillis(), downloadUrl))
    }
    

    View Slide

74. //data class for saving
    data class FirebaseImageLog(val timestamp: Long? = null, val imageRef: String? = null)
    //upload image
    val storageRef = FirebaseStorage.getInstance().getReference(FIREBASE_MOTION_REF)
    val imageStorageRef = storageRef.child(FIREBASE_IMAGE_PREFIX + System.currentTimeMillis() + ".jpg")
    val uploadTask = imageStorageRef.putBytes(imageStream.toByteArray())
    uploadTask.addOnFailureListener {
    Log.d(TAG, "onFailure uploadMotionImage")
    }.addOnSuccessListener {
    // save image log to realtime db
    val downloadUrl = imageStorageRef.path
    val ref = FirebaseDatabase.getInstance().getReference(FIREBASE_MOTION_LOGS).push()
    ref.setValue(FirebaseImageLog(System.currentTimeMillis(), downloadUrl))
    }
    

    View Slide

75. //data class for saving
    data class FirebaseImageLog(val timestamp: Long? = null, val imageRef: String? = null)
    //upload image
    val storageRef = FirebaseStorage.getInstance().getReference(FIREBASE_MOTION_REF)
    val imageStorageRef = storageRef.child(FIREBASE_IMAGE_PREFIX + System.currentTimeMillis() + ".jpg")
    val uploadTask = imageStorageRef.putBytes(imageStream.toByteArray())
    uploadTask.addOnFailureListener {
    Log.d(TAG, "onFailure uploadMotionImage")
    }.addOnSuccessListener {
    // save image log to realtime db
    val downloadUrl = imageStorageRef.path
    val ref = FirebaseDatabase.getInstance().getReference(FIREBASE_MOTION_LOGS).push()
    ref.setValue(FirebaseImageLog(System.currentTimeMillis(), downloadUrl))
    }
    

    View Slide

76. //data class for saving
    data class FirebaseImageLog(val timestamp: Long? = null, val imageRef: String? = null)
    //upload image
    val storageRef = FirebaseStorage.getInstance().getReference(FIREBASE_MOTION_REF)
    val imageStorageRef = storageRef.child(FIREBASE_IMAGE_PREFIX + System.currentTimeMillis() + ".jpg")
    val uploadTask = imageStorageRef.putBytes(imageStream.toByteArray())
    uploadTask.addOnFailureListener {
    Log.d(TAG, "onFailure uploadMotionImage")
    }.addOnSuccessListener {
    // save image log to realtime db
    val downloadUrl = imageStorageRef.path
    val ref = FirebaseDatabase.getInstance().getReference(FIREBASE_MOTION_LOGS).push()
    ref.setValue(FirebaseImageLog(System.currentTimeMillis(), downloadUrl))
    }
    

    View Slide

77. View Slide

78. View Slide

79. $ firebase login
    $ firebase init functions
    

    View Slide

80. var functions = require('firebase-functions');
    var admin = require('firebase-admin');
    admin.initializeApp(functions.config().firebase);
    exports.annotateImage = functions.database.ref('/motion-logs/{id}')
    .onCreate(event => {
    const original = event.data.val();
    var topic = "/topics/intruders";
    var payload = {
    data: {
    title: "Intruder Alert!",
    body: "An intruder has been detected"
    }
    };
    return admin.messaging().sendToTopic(topic, payload)
    .then(function (response) {
    console.log("Successfully sent message:", response);
    }).catch(function (error) {
    console.log("Error sending message:", error);
    });
    }
    );
    

    View Slide

81. var functions = require('firebase-functions');
    var admin = require('firebase-admin');
    admin.initializeApp(functions.config().firebase);
    exports.annotateImage = functions.database.ref('/motion-logs/{id}')
    .onCreate(event => {
    const original = event.data.val();
    var topic = "/topics/intruders";
    var payload = {
    data: {
    title: "Intruder Alert!",
    body: "An intruder has been detected"
    }
    };
    return admin.messaging().sendToTopic(topic, payload)
    .then(function (response) {
    console.log("Successfully sent message:", response);
    }).catch(function (error) {
    console.log("Error sending message:", error);
    });
    }
    );
    

    View Slide

82. var functions = require('firebase-functions');
    var admin = require('firebase-admin');
    admin.initializeApp(functions.config().firebase);
    exports.annotateImage = functions.database.ref('/motion-logs/{id}')
    .onCreate(event => {
    const original = event.data.val();
    var topic = "/topics/intruders";
    var payload = {
    data: {
    title: "Intruder Alert!",
    body: "An intruder has been detected"
    }
    };
    return admin.messaging().sendToTopic(topic, payload)
    .then(function (response) {
    console.log("Successfully sent message:", response);
    }).catch(function (error) {
    console.log("Error sending message:", error);
    });
    }
    );
    

    View Slide

83. $ firebase deploy --only functions
    

    View Slide

84. View Slide

85. View Slide

86. //Use FirebaseUI dependency
    api "com.firebaseui:firebase-ui-database:3.0.0"
    // Get DB Reference
    val databaseRef = FirebaseDatabase.getInstance().getReference(MOTION_LOGS_FIREBASE_REF)
    // Create adapter that extends FirebaseRecyclerAdapter
    class LogsAdapter(ref: DatabaseReference) : FirebaseRecyclerAdapterLogsViewHolder>(FirebaseImageLog::class.java,
    R.layout.list_item_log, LogsViewHolder::class.java, ref) {
    //.. Do adapter things
    }
    // Set adapter to use reference
    adapter = LogsAdapter(databaseRef.orderByChild(ORDER_BY_TIMESTAMP).ref)
    

    View Slide

87. //Use FirebaseUI dependency
    api "com.firebaseui:firebase-ui-database:3.0.0"
    // Get DB Reference
    val databaseRef = FirebaseDatabase.getInstance().getReference(MOTION_LOGS_FIREBASE_REF)
    // Create adapter that extends FirebaseRecyclerAdapter
    class LogsAdapter(ref: DatabaseReference) : FirebaseRecyclerAdapterLogsViewHolder>(FirebaseImageLog::class.java,
    R.layout.list_item_log, LogsViewHolder::class.java, ref) {
    //.. Do adapter things
    }
    // Set adapter to use reference
    adapter = LogsAdapter(databaseRef.orderByChild(ORDER_BY_TIMESTAMP).ref)
    

    View Slide

88. //Use FirebaseUI dependency
    api "com.firebaseui:firebase-ui-database:3.0.0"
    // Get DB Reference
    val databaseRef = FirebaseDatabase.getInstance().getReference(MOTION_LOGS_FIREBASE_REF)
    // Create adapter that extends FirebaseRecyclerAdapter
    class LogsAdapter(ref: DatabaseReference) : FirebaseRecyclerAdapterLogsViewHolder>(FirebaseImageLog::class.java,
    R.layout.list_item_log, LogsViewHolder::class.java, ref) {
    //.. Do adapter things
    }
    // Set adapter to use reference
    adapter = LogsAdapter(databaseRef.orderByChild(ORDER_BY_TIMESTAMP).ref)
    

    View Slide

89. //Use FirebaseUI dependency
    api "com.firebaseui:firebase-ui-database:3.0.0"
    // Get DB Reference
    val databaseRef = FirebaseDatabase.getInstance().getReference(MOTION_LOGS_FIREBASE_REF)
    // Create adapter that extends FirebaseRecyclerAdapter
    class LogsAdapter(ref: DatabaseReference) : FirebaseRecyclerAdapterLogsViewHolder>(FirebaseImageLog::class.java,
    R.layout.list_item_log, LogsViewHolder::class.java, ref) {
    //.. Do adapter things
    }
    // Set adapter to use reference
    adapter = LogsAdapter(databaseRef.orderByChild(ORDER_BY_TIMESTAMP).ref)
    

    View Slide

90. View Slide

91. View Slide

92. View Slide

93. View Slide

94. Clear all Android Apps on device:
    https://gist.github.com/blundell/7c0c3bb17898b28fe8122b0dc230af50
    Button Example : https://github.com/androidthings/sample-button
    Full MainActivity:
    https://gist.github.com/riggaroo/b4cad83be2f81d54fd854403af029b2d
    Github Project with Steps as branches:
    https://github.com/riggaroo/android-things-workshop-camera
    

    View Slide