DevFest NYC - Ok Google, Build an Android Things Product

Rebecca covered the basics of Android Things, and how you can integrate with the Google Assistant.

Transcript

1. Ok Google, Make an Android
   Things Product
   Rebecca Franks
   Android Engineering Lead at DVT
   Google Developer Expert Android
   @riggaroo
   

   View Slide

2. riggaroo.co.za
   

   View Slide

3. View Slide

4. View Slide

5. View Slide

6. Android Things
   

   View Slide

7. Android Things is an extension of the Android platform for
   IoT and embedded devices.
   

   View Slide

8. Interactive Ads
   Vending Machines
   Point of Sale
   Stock Control
   Retail
   Cameras
   Gateways
   Access Control
   Smart Meters
   Business
   Asset Tracking
   Fleet Management
   Driver Assist
   Predictive Service
   Logistics
   Security Systems
   Smart Doorbells
   Routers
   Energy Monitors
   Home
   Android Things is ideal for powerful, intelligent
   devices that need to be secure.
   

   View Slide

9. Similarities
   Any other Android library...
   Android SDK Play Services Firebase
   Android Studio Cloud Platform
   

   View Slide

10. Differences
    No Play Store Deploy OTAs
    Subset of
    APIs
    Custom
    Hardware
    Single Purpose
    Device
    

    View Slide

11. Applications
    Launcher Phone Messaging Contacts Calendar Browser Settings
    Application Framework
    Activity
    Manager
    Window Manager Power
    Manager
    Resource
    Manager
    XMPP
    Service
    Content
    Providers
    Wallpapers System UI
    Package
    Manager
    Telephony
    Manager
    Location
    Manager
    Connectivity
    Manager
    View
    System
    Runtime
    Permissions
    Soft
    Keyboards Notifications
    Libraries
    Surface
    Manager
    Media
    Framework
    Chromium SSL HAL
    Audio
    Manager
    SQLite Open GL libc
    Core
    Libraries
    Android
    Runtime (ART)
    Android Runtime
    Linux Kernel
    Display Driver Camera Driver Bluetooth Driver
    Binder (IPC)
    Driver
    USB Driver Audio Driver WiFi Driver
    Power
    Management
    

    View Slide

12. Applications
    Launcher Phone Messaging Contacts Calendar Browser Settings
    Application Framework
    Activity
    Manager
    Window Manager Power
    Manager
    Resource
    Manager
    XMPP
    Service
    Content
    Providers
    Wallpapers System UI
    Package
    Manager
    Telephony
    Manager
    Location
    Manager
    Connectivity
    Manager
    View
    System
    Runtime
    Permissions
    Soft
    Keyboards Notifications
    Libraries
    Surface
    Manager
    Media
    Framework
    Chromium SSL HAL
    Audio
    Manager
    SQLite Open GL libc
    Core
    Libraries
    Android
    Runtime (ART)
    Android Runtime
    Linux Kernel
    Display Driver Camera Driver Bluetooth Driver
    Binder (IPC)
    Driver
    USB Driver Audio Driver WiFi Driver
    Power
    Management
    

    View Slide

13. Applications
    Launcher Phone Messaging Contacts Calendar Browser Settings
    Application Framework
    Activity
    Manager
    Window Manager Power
    Manager
    Resource
    Manager
    XMPP
    Service
    Wallpapers System UI
    Package
    Manager
    Telephony
    Manager
    Location
    Manager
    Connectivity
    Manager
    View
    System
    Soft
    Keyboards Notifications
    Libraries
    Surface
    Manager
    Media
    Framework
    Chromium SSL HAL
    Audio
    Manager
    SQLite Open GL libc
    Core
    Libraries
    Android
    Runtime (ART)
    Android Runtime
    Linux Kernel
    Display Driver Camera Driver Bluetooth Driver
    Binder (IPC)
    Driver
    USB Driver Audio Driver WiFi Driver
    Power
    Management
    Things Support Library
    Peripheral
    I/O
    Device
    Management
    User
    Drivers Connectivity
    

    View Slide

14. SoM
    Architecture
    Google
    Managed BSP
    

    View Slide

15. Distribution with Android Things
    Android Framework
    Hardware Libraries
    Linux Kernel
    Managed by Google
    Apps
    User Drivers
    Managed by Developers
    

    View Slide

16. Android Things Console
    ● Manage your Android Things IoT Product
    ● Download and install the latest Android Things system
    image
    ● Build factory images that contain OEM applications along
    with the system image
    ● Push over-the-air (OTA) updates
    partner.android.com/things/console
    

    View Slide

17. View Slide

18. View Slide

19. View Slide

20. View Slide

21. Automatic
    Security Updates
    Signed Images Verified Boot
    A/B Rollback
    Protection
    

    View Slide

22. Developing for Android
    Things
    

    View Slide

23. Developer Kits
    NXP I.MX7D Raspberry Pi 3
    

    View Slide

24. Hardware Integration
    Peripheral Driver Library - Github
    + many more...
    bit.ly/androidthings-github
    Button GPS
    PWM Servo
    RGB LED Strip
    Temperature
    Sensor
    Capacitive
    Touch Buttons
    Peripheral I/O - Low Level Access
    GPIO PWM
    I2C SPI
    UART I2S
    

    View Slide

25. Build a Motion Sensing
    Camera
    

    View Slide

26. Motion Sensing Camera
    

    View Slide

27. Pico i.MX7 Pinout
    * there was a change in pin naming in DP6
    

    View Slide

28. View Slide

29. View Slide

30. View Slide

31. //Automatically added in app level build.gradle
    compileOnly 'com.google.android.things:androidthings:...'
    

    View Slide

32. //Automatically added in AndroidManifest.xml
    
    
    
    
    
    
    android:name="android.intent.category.IOT_LAUNCHER"/>
    
    
    
    
    

    View Slide

33. //Automatically added in AndroidManifest.xml
    
    
    
    
    
    
    android:name="android.intent.category.IOT_LAUNCHER"/>
    
    
    
    
    

    View Slide

34. Working with a Motion
    Sensor
    

    View Slide

35. GPIO
    - General Purpose Input / Output
    - Programmable way to read true / false values (1 or 0) (push
    button, PIR Sensor)
    - Write true/false values (LED)
    - Configurable
    

    View Slide

36. PIR Sensor
    

    View Slide

37. Motion Sensor Control
    

    View Slide

38. //Accessing Sensor Data
    private val motionSensorGpio: Gpio = PeripheralManagerService().openGpio(motionSensorPin)
    fun start() {
    motionSensorGpio.setDirection(Gpio.DIRECTION_IN)
    motionSensorGpio.setActiveType(Gpio.ACTIVE_HIGH)
    motionSensorGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    motionSensorGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    motionListener.onMotionDetected()
    } else {
    motionListener.onMotionStopped()
    }
    return true
    }
    })
    }
    

    View Slide

39. //Accessing Sensor Data
    private val motionSensorGpio: Gpio = PeripheralManagerService().openGpio(motionSensorPin)
    fun start() {
    motionSensorGpio.setDirection(Gpio.DIRECTION_IN)
    motionSensorGpio.setActiveType(Gpio.ACTIVE_HIGH)
    motionSensorGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    motionSensorGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    motionListener.onMotionDetected()
    } else {
    motionListener.onMotionStopped()
    }
    return true
    }
    })
    }
    

    View Slide

40. //Accessing Sensor Data
    private val motionSensorGpio: Gpio = PeripheralManagerService().openGpio(motionSensorPin)
    fun start() {
    motionSensorGpio.setDirection(Gpio.DIRECTION_IN)
    motionSensorGpio.setActiveType(Gpio.ACTIVE_HIGH)
    motionSensorGpio.setEdgeTriggerType(Gpio.EDGE_BOTH)
    motionSensorGpio.registerGpioCallback(object : GpioCallback() {
    override fun onGpioEdge(gpio: Gpio): Boolean {
    if (gpio.value) {
    motionListener.onMotionDetected()
    } else {
    motionListener.onMotionStopped()
    }
    return true
    }
    })
    }
    

    View Slide

41. LED Control
    

    View Slide

42. val peripheralManagerService = PeripheralManagerService()
    ledGpio = peripheralManagerService.openGpio(LED_GPIO_PIN)
    ledGpio.setDirection(Gpio.DIRECTION_OUT_INITIALLY_LOW)
    ledGpio.value = true
    ledGpio.close()
    

    View Slide

43. val peripheralManagerService = PeripheralManagerService()
    ledGpio = peripheralManagerService.openGpio(LED_GPIO_PIN)
    ledGpio.setDirection(Gpio.DIRECTION_OUT_INITIALLY_LOW)
    ledGpio.value = true
    ledGpio.close()
    

    View Slide

44. val peripheralManagerService = PeripheralManagerService()
    ledGpio = peripheralManagerService.openGpio(LED_GPIO_PIN)
    ledGpio.setDirection(Gpio.DIRECTION_OUT_INITIALLY_LOW)
    ledGpio.value = true
    ledGpio.close()
    

    View Slide

45. Google Assistant Integration
    

    View Slide

46. View Slide

47. Actions on Google
    Conversational Actions
    - Two way dialog
    - Your action manages the
    conversation
    “Ok Google, Talk to Spelling Master”
    Direct Actions
    - Google handles the entire user
    interaction
    - You only handle the fufilment
    “Ok Google, Turn off the Motion
    Sensing Alarm”
    

    View Slide

48. Smart Home Apps rely on
    Home Graph
    Living Room
    Kitchen
    Bedroom
    Office
    

    View Slide

49. How do you create a Smart
    Home App?
    

    View Slide

50. 1. Setup OAuth 2.0 Server for
    account linking
    ...or fake it for testing :)
    

    View Slide

51. 2. Create actions on Google
    project
    console.actions.google.com
    

    View Slide

52. View Slide

53. 3. Create an action package,
    declaring support for Smart
    Home intents.
    

    View Slide

54. { "actions": [{
    "name": "actions.devices",
    "deviceControl": {
    },
    "fulfillment": {
    "conversationName": "automation"
    }
    }],
    "conversations": {
    "automation" :
    {
    "name": "automation",
    "url": "https://example.com/google-assistant/endpoint"
    }
    }
    }
    

    View Slide

55. 4. Provide fulfillment of Smart
    Home intents. (Cloud function)
    

    View Slide

56. Smart Home - Setup + SYNC
    

    View Slide

57. Smart Home - EXECUTE Command
    

    View Slide

58. Smart Home - QUERY Command
    

    View Slide

59. 5. Test and submit your app for
    approval.
    

    View Slide

60. View Slide

61. Google Assistant Example
    bit.ly/actions-google
    Hackster Project:
    bit.ly/hack-at-motion-camera
    

    View Slide

62. Examples
    

    View Slide

63. Edison Candle
    Dave Smith
    @devunwired
    

    View Slide

64. BrailleBox
    Joe Birch
    @hitherejoe
    

    View Slide

65. AI Candy Dispenser
    Alvaro Viebrantz
    @alvaroviebrantz
    

    View Slide

66. Wildlife Detector
    Paul Trebilcox-Ruiz
    @PaulTR88
    

    View Slide

67. Automatic and
    Secure
    The Power of
    Android
    Managed by
    Google
    Summary: Why Android Things?
    

    View Slide

68. More info:
    docs bit.ly/androidthings
    codelab bit.ly/at-codelab
    

    View Slide

69. Workshop
    Android Things- Building a Motion Sensing
    Camera - Tomorrow!
    13:00
    

    View Slide

70. Thank you!
    Rebecca Franks
    Android Engineering Lead at DVT
    @riggaroo
    

    View Slide