Deep into the IoT trenches: how to build a connected product

Deep into the IoT trenches: how to build a connected product

Slides from my talk at Devoxx UK, Droidcon UK 2017
Video: https://www.youtube.com/watch?v=rZ4b7UxE920

Don't you love making your app robust against network failures? Now imagine your app talking to a Wi-Fi connected product... Locally when your home or via a server when you’re not. You'll certainly love handling those network switches!... And what if your app doesn't just control one product, but multiple ones? Some locally, some remotely connected?... Or how about having multiple users controlling the same product at the same time?... It goes without question that building apps for connected products is incredibly challenging.

From device discovery, over minimizing network requests to out of the box setup, this talk will cover it all. Packed with code examples, architecture suggestions and tons of best practices, you'll learn five years’ worth of hard-earned experience. This won't just kick start your IoT app development skills, but it'll also teach you how to keep your app scalable and maintainable towards the future.

Starting as early as 2011, Jeroen has built an impressive range of connected products: BT LE coffee machines, Wi-Fi speakers, connected air purifiers and even full blown connectivity libraries. Currently he is the lead Android developer for the official Philips Hue app, the biggest IoT system in the world.

5f57d2d205e77e185986459c1b89a874?s=128

Jeroen Mols

May 12, 2017
Tweet

Transcript

  1. @MOLSJEROEN HOW TO BUILD A CONNECTED PRODUCT DEEP INTO THE

    IOT TRENCHES:
  2. @MOLSJEROEN @MOLSJEROEN www.philips.com/careers-hue

  3. IT’S NOT ENOUGH FOR CODE TO WORK. Robert C. Martin

  4. @MOLSJEROEN BUILDING A CONNECTED PRODUCT

  5. @MOLSJEROEN BUILDING A CONNECTED PRODUCT

  6. @MOLSJEROEN BUILDING A CONNECTED PRODUCT

  7. @MOLSJEROEN BUILDING A CONNECTED PRODUCT

  8. @MOLSJEROEN BUILDING A CONNECTED PRODUCT

  9. @MOLSJEROEN BUILDING A CONNECTED PRODUCT

  10. @MOLSJEROEN BUILDING A CONNECTED PRODUCT

  11. @MOLSJEROEN BUILDING A CONNECTED PRODUCT

  12. @MOLSJEROEN BUILDING A CONNECTED PRODUCT

  13. @MOLSJEROEN BUILDING A CONNECTED PRODUCT ?

  14. @MOLSJEROEN CHALLENGES 1. Easy to use (and test) ▸ Wi-Fi

    setup ▸ Prepare for bad weather 2. Performant ▸ Fast discovery ▸ Maximise control speed 3. Scalable ▸ Lessons learned
  15. @MOLSJEROEN CHALLENGES 1. Easy to use (and test) ▸ Wi-Fi

    setup ▸ Prepare for bad weather 2. Performant ▸ Fast discovery ▸ Maximise control speed 3. Scalable ▸ Lessons learned
  16. WI-FI SETUP PART 1: EASY TO USE (AND TEST)

  17. @MOLSJEROEN WI-FI SETUP - GOAL

  18. @MOLSJEROEN WI-FI SETUP - GOAL ?

  19. @MOLSJEROEN WI-FI SETUP - MECHANISM

  20. @MOLSJEROEN WI-FI SETUP - MECHANISM

  21. @MOLSJEROEN WI-FI SETUP - MECHANISM SSID & PASSWORD

  22. @MOLSJEROEN WI-FI SETUP - MECHANISM

  23. @MOLSJEROEN WI-FI SETUP - MECHANISM

  24. @MOLSJEROEN WI-FI SETUP - CHROMECAST 4 3 2 1

  25. @MOLSJEROEN WI-FI SETUP - CHROMECAST 4 3 2 1

  26. @MOLSJEROEN WI-FI SETUP - CHROMECAST 4 3 2 1

  27. @MOLSJEROEN WI-FI SETUP - CHROMECAST 8 7 6 5

  28. @MOLSJEROEN WI-FI SETUP - CHROMECAST 8 7 6 5

  29. @MOLSJEROEN WI-FI SETUP - CHROMECAST 8 7 6 5

  30. @MOLSJEROEN WI-FI SETUP - CHROMECAST 12 11 10 9

  31. @MOLSJEROEN WI-FI SETUP - CHROMECAST 12 11 10 9

  32. @MOLSJEROEN WI-FI SETUP - CHROMECAST 12 11 10 9

  33. @MOLSJEROEN WI-FI SETUP - CHROMECAST 12 11 10 9

  34. @MOLSJEROEN WI-FI SETUP - CHALLENGES ▸ Typing Wi-Fi passwords ▸

    Network switching is fragile ▸ phone/tablet refuses to connect ▸ OS routes request over 4G ▸ Extremely slow feedback ▸ Users have to leave app on iOS ▸ Many bad weather scenarios
  35. @MOLSJEROEN WI-FI SETUP - CHALLENGES ▸ 1

  36. IF WI-FI SETUP FAILS, YOUR PRODUCT WILL BE RETURNED Yours

    truly
  37. @MOLSJEROEN ANDROID 5.0+ ▸ Behaviour change in Android 5.0 ▸

    Wi-Fi without internet ▸ Before: all requests sent over Wi-Fi ▸ After: all requests routed over 3G/4G ▸ Done for all, no subnet matching
  38. @MOLSJEROEN ANDROID 5.0+ - SINGLE REQUEST OVER WI-FI @RequiresApi(api =

    Build.VERSION_CODES.LOLLIPOP) public void singleRequestOverWifi(Context context) { NetworkRequest.Builder request = new NetworkRequest.Builder(); request.addTransportType(NetworkCapabilities.TRANSPORT_WIFI); ConnectivityManager connMan = getConnectivityManager(); connMan.registerNetworkCallback(request.build(), new NetworkCallback(){ @Override public void onAvailable(Network network) { // Do network request } }); }
  39. @MOLSJEROEN ANDROID 5.0+ - SINGLE REQUEST OVER WI-FI @RequiresApi(api =

    Build.VERSION_CODES.LOLLIPOP) public void singleRequestOverWifi(Context context) { NetworkRequest.Builder request = new NetworkRequest.Builder(); request.addTransportType(NetworkCapabilities.TRANSPORT_WIFI); ConnectivityManager connMan = getConnectivityManager(); connMan.registerNetworkCallback(request.build(), new NetworkCallback(){ @Override public void onAvailable(Network network) { // Do network request } }); }
  40. @MOLSJEROEN ANDROID 5.0+ - SINGLE REQUEST OVER WI-FI @RequiresApi(api =

    Build.VERSION_CODES.LOLLIPOP) public void singleRequestOverWifi(Context context) { NetworkRequest.Builder request = new NetworkRequest.Builder(); request.addTransportType(NetworkCapabilities.TRANSPORT_WIFI); ConnectivityManager connMan = getConnectivityManager(); connMan.registerNetworkCallback(request.build(), new NetworkCallback(){ @Override public void onAvailable(Network network) { // Do network request } }); }
  41. @MOLSJEROEN ANDROID 5.0+ - SINGLE REQUEST OVER WI-FI @RequiresApi(api =

    Build.VERSION_CODES.LOLLIPOP) public void singleRequestOverWifi(Context context) { NetworkRequest.Builder request = new NetworkRequest.Builder(); request.addTransportType(NetworkCapabilities.TRANSPORT_WIFI); ConnectivityManager connMan = getConnectivityManager(); connMan.registerNetworkCallback(request.build(), new NetworkCallback(){ @Override public void onAvailable(Network network) { // Do network request } }); }
  42. @MOLSJEROEN ANDROID 5.0+ - ALL REQUESTS OVER WI-FI @RequiresApi(api =

    Build.VERSION_CODES.LOLLIPOP) public void allRequestsOverWifi(Context context) { NetworkRequest.Builder request = new NetworkRequest.Builder(); request.addTransportType(NetworkCapabilities.TRANSPORT_WIFI); final ConnectivityManager connMan = getConnectivityManager(); connMan.registerNetworkCallback(request.build(), new NetworkCallback() { @Override public void onAvailable(Network network) { connMan.bindProcessToNetwork(network); // Do network requests } }); }
  43. @MOLSJEROEN ANDROID 5.0+ - ALL REQUESTS OVER WI-FI @RequiresApi(api =

    Build.VERSION_CODES.LOLLIPOP) public void allRequestsOverWifi(Context context) { NetworkRequest.Builder request = new NetworkRequest.Builder(); request.addTransportType(NetworkCapabilities.TRANSPORT_WIFI); final ConnectivityManager connMan = getConnectivityManager(); connMan.registerNetworkCallback(request.build(), new NetworkCallback() { @Override public void onAvailable(Network network) { connMan.bindProcessToNetwork(network); // Do network requests } }); }
  44. @MOLSJEROEN ALTERNATIVES ▸ One product as ethernet bridge ▸ WPS

    ▸ Other mechanisms for communication ▸ e.g High frequency sound, … ▸ Still need to enter Wi-Fi password ▸ WAC (Homekit devices) ▸ …
  45. @MOLSJEROEN #PROTIP - RANDOM SETUP NETWORK NAME ssid
 SETUP ssid


    SETUP ssid
 SETUP ssid
 SETUP ????????
  46. @MOLSJEROEN #PROTIP - RANDOM SETUP NETWORK NAME

  47. PREPARE FOR BAD WEATHER PART 1: EASY TO USE (AND

    TEST)
  48. @MOLSJEROEN GOOD WEATHER SCENARIOS

  49. @MOLSJEROEN BAD WEATHER SCENARIOS

  50. @MOLSJEROEN BAD WEATHER SCENARIOS

  51. @MOLSJEROEN BAD WEATHER SCENARIOS

  52. @MOLSJEROEN INTERESTING SCENARIOS ▸ Bridge (almost) full ▸ Network request

    failed for reason X ▸ Resource doesn’t exist ▸ Connection lost ▸ Authentication lost ▸ Dynamic state changes ▸ …. => nearly impossible to reproduce manually
  53. @MOLSJEROEN SIMULATOR ▸ Hardcoded responses ▸ Simulate dynamic behaviour ▸

    Recreate bad weather scenarios
  54. @MOLSJEROEN SIMULATOR COMMUNICATIONSTRATEGY REMOTESTRATEGY LOCALSTRATEGY SIMULATORSTRATEGY PRODUCT 1.. ▸ Hardcoded

    responses ▸ Simulate dynamic behaviour ▸ Recreate bad weather scenarios
  55. @MOLSJEROEN SIMULATOR COMMUNICATIONSTRATEGY REMOTESTRATEGY LOCALSTRATEGY SIMULATORSTRATEGY PRODUCT 1.. ▸ Hardcoded

    responses ▸ Simulate dynamic behaviour ▸ Recreate bad weather scenarios
  56. @MOLSJEROEN #PROTIP - DON'T USE THE WORD “DEVICE” ▸ Use

    product or appliance
  57. @MOLSJEROEN CHALLENGES 1. Easy to use (and test) ▸ Wi-Fi

    setup ▸ Prepare for bad weather 2. Performant ▸ Fast discovery ▸ Maximise control speed 3. Scalable ▸ Lessons learned
  58. DISCOVERY PART 2: PERFORMANT

  59. @MOLSJEROEN LOCAL DISCOVERY ▸ Multicast based ▸ Often Unicast responses

    ▸ SSDP, mDNS,…
  60. @MOLSJEROEN LOCAL DISCOVERY ▸ Multicast based ▸ Often Unicast responses

    ▸ SSDP, mDNS,…
  61. @MOLSJEROEN LOCAL DISCOVERY ▸ Multicast based ▸ Often Unicast responses

    ▸ SSDP, mDNS,…
  62. @MOLSJEROEN LOCAL DISCOVERY ▸ Multicast based ▸ Often Unicast responses

    ▸ SSDP, mDNS,… ▸ Alternatives ▸ IP scan ▸ Manual IP ▸ Proprietary product IP server
  63. @MOLSJEROEN LOCAL DISCOVERY - IP SERVER https://www.meethue.com/api/nupnp [ { "id":"001788fffe100491",

    “internalipaddress":"192.168.2.23" }, { "id":"001788fffe09a168", “internalipaddress":"192.168.88.252" } ]
  64. @MOLSJEROEN REMOTE DISCOVERY ▸ Web service ▸ product announces ▸

    interface to get products ▸ Security ▸ Local setup first required
  65. @MOLSJEROEN OPTIMISING DISCOVERY

  66. @MOLSJEROEN OPTIMISING DISCOVERY 1. Discover local and remote 2. Connect

    local and remote ▸ ~5 - 10 seconds ▸ Empty UI when no products found
  67. @MOLSJEROEN OPTIMISING DISCOVERY - REMEMBER PRODUCTS 1. Products from database

    2. Discover local and remote 3. Connect local and remote ▸ ~5 - 10 seconds ▸ Disconnected device when not found ▸ Need for deleting devices
  68. @MOLSJEROEN OPTIMISING DISCOVERY - REMEMBER STATE 1. Products & last

    state from database 2. Discover local and remote 3. Connect local and remote 4. Update UI with actual state ▸ ~3 - 4 seconds ▸ UI instantly ready ▸ Control possible after discovery
  69. @MOLSJEROEN OPTIMISING DISCOVERY - REMEMBER CONNECTION INFO 1. Products, last

    state & connection info from database 2. Attempt local and remote connection 3. Fallback discovery 4. Update UI with actual state ▸ ~0 seconds ▸ UI & Control instantly ready ▸ Remember IP address and last SSID
  70. @MOLSJEROEN #PROTIP - SIMPLIFY DEVELOPMENT BY LIMITING STATE TRANSITIONS ▸

    Transition local to remote connection always via disconnected
  71. MAXIMISE CONTROL SPEED PART 2: PERFORMANT

  72. @MOLSJEROEN CONTROL SPEED 1. Always prefer local connection 2. Optimise

    local connection 3. Reduce amount of requests 4. Instant UI ▸ Implicit cost savings ▸ Remote connection usually not critical
  73. @MOLSJEROEN 1. PREFER LOCAL CONNECTION ▸ Inherently faster ▸ Cost

    benefit ▸ Force connection
  74. @MOLSJEROEN 1. PREFER LOCAL CONNECTION COMPOSITESTRATEGY PRODUCT LOCALSTRATEGY SIMULATORSTRATEGY REMOTESTRATEGY

    COMMUNICATIONSTRATEGY - boolean isAvailable() ▸ Inherently faster ▸ Cost benefit ▸ Force connection
  75. @MOLSJEROEN 2. OPTIMISE LOCAL CONNECTION ▸ Data compression ▸ Cache

    headers ▸ Socket reuse ▸ => OKHttp ▸ Interference UDP/TCP or other traffic ▸ Wireshark low level traffic
  76. @MOLSJEROEN 2. OPTIMISE LOCAL CONNECTION ▸ Data compression ▸ Cache

    headers ▸ Socket reuse ▸ => OKHttp ▸ Interference UDP/TCP or other traffic ▸ Wireshark low level traffic
  77. @MOLSJEROEN 3. REDUCE # REQUESTS ▸ Toggle sequences ▸ Combine

    requests to same endpoint
  78. @MOLSJEROEN 3. REDUCE # REQUESTS private Map<String, Object> pendingAttr =

    new HashMap<>(); public void setOn(boolean isOn) { synchronized (pendingAttr) { pendingAttr("KEY_ON", isOn); } sendUpdate(); } private void sendUpdate() { strategy.doRequest(updateRequest); }
  79. @MOLSJEROEN 3. REDUCE # REQUESTS private Map<String, Object> pendingAttr =

    new HashMap<>(); public void setOn(boolean isOn) { synchronized (pendingAttr) { pendingAttr("KEY_ON", isOn); } sendUpdate(); } private void sendUpdate() { strategy.doRequest(updateRequest); }
  80. @MOLSJEROEN 3. REDUCE # REQUESTS private Map<String, Object> pendingAttr =

    new HashMap<>(); public void setOn(boolean isOn) { synchronized (pendingAttr) { pendingAttr("KEY_ON", isOn); } sendUpdate(); } private void sendUpdate() { strategy.doRequest(updateRequest); }
  81. @MOLSJEROEN 3. REDUCE # REQUESTS private Map<String, Object> pendingAttr =

    new HashMap<>(); public void setOn(boolean isOn) { synchronized (pendingAttr) { pendingAttr("KEY_ON", isOn); } sendUpdate(); } private void sendUpdate() { strategy.doRequest(updateRequest); }
  82. @MOLSJEROEN 3. REDUCE # REQUESTS private UpdateRequest updateRequest = new

    UpdateRequest() { @Override public String getEndpoint() { return "/lights"; } @Override public Map<String, Object> getAttributes() { synchronized (pendingAttr) { Map<String, Object> copy = new HashMap<>(pendingAttr); pendingAttr.clear(); return copy; } }};
  83. @MOLSJEROEN 3. REDUCE # REQUESTS private UpdateRequest updateRequest = new

    UpdateRequest() { @Override public String getEndpoint() { return "/lights"; } @Override public Map<String, Object> getAttributes() { synchronized (pendingAttr) { Map<String, Object> copy = new HashMap<>(pendingAttr); pendingAttr.clear(); return copy; } }};
  84. @MOLSJEROEN 3. REDUCE # REQUESTS private UpdateRequest updateRequest = new

    UpdateRequest() { @Override public String getEndpoint() { return "/lights"; } @Override public Map<String, Object> getAttributes() { synchronized (pendingAttr) { Map<String, Object> copy = new HashMap<>(pendingAttr); pendingAttr.clear(); return copy; } }};
  85. @MOLSJEROEN 3. REDUCE # REQUESTS private class CommunicationStrategy { private

    Set<UpdateRequest> queue = new LinkedHashSet<>(); public void doRequest(UpdateRequest updateRequest) { queue.add(updateRequest); // logic to trigger requests } }
  86. @MOLSJEROEN 3. REDUCE # REQUESTS private class CommunicationStrategy { private

    Set<UpdateRequest> queue = new LinkedHashSet<>(); public void doRequest(UpdateRequest updateRequest) { queue.add(updateRequest); // logic to trigger requests } }
  87. @MOLSJEROEN 3. REDUCE # REQUESTS private class CommunicationStrategy { private

    Set<UpdateRequest> queue = new LinkedHashSet<>(); public void doRequest(UpdateRequest updateRequest) { queue.add(updateRequest); // logic to trigger requests } }
  88. @MOLSJEROEN 4. INSTANT UI

  89. @MOLSJEROEN #PROTIP - NEVER ASSUME, ALWAYS MEASURE

  90. @MOLSJEROEN CHALLENGES 1. Easy to use (and test) ▸ Wi-Fi

    setup ▸ Prepare for bad weather 2. Performant ▸ Fast discovery ▸ Maximise control speed 3. Scalable ▸ Lessons learned
  91. LESSONS LEARNED PART 3: SCALABLE

  92. @MOLSJEROEN DEVELOPING A SYSTEM 0 0.5 1 1.5 2 2.5

    3 3.5 4 4.5 1 2 3 4 5 6 7 8 9 10 11 12 New Features Maintenance
  93. @MOLSJEROEN DEVELOPING A SYSTEM 0 0.5 1 1.5 2 2.5

    3 3.5 4 4.5 1 2 3 4 5 6 7 8 9 10 11 12 New Features Maintenance
  94. @MOLSJEROEN DEVELOPING A SYSTEM 0 0.5 1 1.5 2 2.5

    3 3.5 4 4.5 1 2 3 4 5 6 7 8 9 10 11 12 New Features Maintenance
  95. @MOLSJEROEN DEVELOPING A SYSTEM 0 0.5 1 1.5 2 2.5

    3 3.5 4 4.5 1 2 3 4 5 6 7 8 9 10 11 12 New Features Maintenance
  96. @MOLSJEROEN DEVELOPING A SYSTEM 0 0.5 1 1.5 2 2.5

    3 3.5 4 4.5 1 2 3 4 5 6 7 8 9 10 11 12 New Features Maintenance
  97. @MOLSJEROEN DEVELOPING A SYSTEM 0 0.5 1 1.5 2 2.5

    3 3.5 4 4.5 1 2 3 4 5 6 7 8 9 10 11 12 New Features Maintenance
  98. THE TRUE COST OF SOFTWARE IS IN ITS MAINTENANCE unknown

  99. @MOLSJEROEN MAINTENANCE ▸ Big focus on automated testing ▸ Need

    for good architecture ▸ Over-engineering has a huge cost ▸ Avoid Hype driven development ▸ Think well before introducing new features
  100. @MOLSJEROEN #FAILUREHALLOFFAME ▸ Detect working internet connection (Wifi, no internet)

    ▸ Instrumentation tests ▸ Lock orientation to portrait ▸ Nested fragments ▸ Broadcasts as callbacks ▸ Post( new Runnable(…) ) ▸ Null checks ▸ …. RELATED BLOGPOSTS
  101. WRAP UP

  102. IN SOFTWARE DESIGN, OFTEN THE CONSEQUENCES OF YOUR DECISIONS DON'T

    BECOME APPARENT FOR YEARS Kent Beck
  103. @MOLSJEROEN CONCLUSION ▸ Developing connected products is hard ▸ Developing

    systems is even harder ▸ Maximise local connections ▸ Keep things simple for users ▸ Strong focus on automated testing ▸ Proper architecture
  104. @MOLSJEROEN IMAGE CREDITS • Welcome image http://www.store.meethue.com/media/catalog/product/cache/1/ feature_img_7/9df78eab33525d08d6e5fb8d27136e95/e/x/expand_your_ecosystem_1.jpg • Wi-fi

    setup https://images.philips.com/is/image/PhilipsConsumer/AW3000_10-MI1-global-001? $jpglarge$&wid=1250 • Good vs bad weather https://images.philips.com/is/image/PhilipsConsumer/FC8830_82-U1P-global-001? $jpglarge$&wid=1250 • Discovery https://s.blogcdn.com/slideshows/images/slides/403/398/9/S4033989/slug/l/philips-hue-motion- sensor-wall-attached-1.jpg • Control speed http://www.philips.com/consumerfiles/newscenter/main/shared/assets/de/Downloadablefile/ press/elektro_hausgeraete/20140716_Philips_Air_AC4072_11_Lifestyle_4.jpg • Developing a system http://www.philips.com/consumerfiles/newscenter/main/standard/resources/corporate/ press/2014/IFA2014/Saeco-GranBaristo-Avanti_image-5.jpg • Warp up http://www.philips.com/consumerfiles/newscenter/main/standard/resources/corporate/press/2014/ IFA2014/Hue-Beyond-lifestyle_all-three.jpg
  105. @MOLSJEROEN MANY THANKS ▸ Jeroen Mols (Belgium) ▸ @MolsJeroen ▸

    http://jeroenmols.com/blog