Deep into the IoT trenches: how to build a connected product

Slides from my talk at Devoxx UK, Droidcon UK 2017
Video: https://www.youtube.com/watch?v=rZ4b7UxE920

Don't you love making your app robust against network failures? Now imagine your app talking to a Wi-Fi connected product... Locally when your home or via a server when you’re not. You'll certainly love handling those network switches!... And what if your app doesn't just control one product, but multiple ones? Some locally, some remotely connected?... Or how about having multiple users controlling the same product at the same time?... It goes without question that building apps for connected products is incredibly challenging.

From device discovery, over minimizing network requests to out of the box setup, this talk will cover it all. Packed with code examples, architecture suggestions and tons of best practices, you'll learn five years’ worth of hard-earned experience. This won't just kick start your IoT app development skills, but it'll also teach you how to keep your app scalable and maintainable towards the future.

Starting as early as 2011, Jeroen has built an impressive range of connected products: BT LE coffee machines, Wi-Fi speakers, connected air purifiers and even full blown connectivity libraries. Currently he is the lead Android developer for the official Philips Hue app, the biggest IoT system in the world.

Transcript

1. @MOLSJEROEN HOW TO BUILD A CONNECTED PRODUCT DEEP INTO THE IOT TRENCHES:
2. @MOLSJEROEN @MOLSJEROEN www.philips.com/careers-hue
3. IT’S NOT ENOUGH FOR CODE TO WORK. Robert C. Martin
4. @MOLSJEROEN BUILDING A CONNECTED PRODUCT
5. @MOLSJEROEN BUILDING A CONNECTED PRODUCT
6. @MOLSJEROEN BUILDING A CONNECTED PRODUCT
7. @MOLSJEROEN BUILDING A CONNECTED PRODUCT
8. @MOLSJEROEN BUILDING A CONNECTED PRODUCT
9. @MOLSJEROEN BUILDING A CONNECTED PRODUCT
10. @MOLSJEROEN BUILDING A CONNECTED PRODUCT
11. @MOLSJEROEN BUILDING A CONNECTED PRODUCT
12. @MOLSJEROEN BUILDING A CONNECTED PRODUCT
13. @MOLSJEROEN BUILDING A CONNECTED PRODUCT ?
14. @MOLSJEROEN CHALLENGES 1. Easy to use (and test) ▸ Wi-Fi setup ▸ Prepare for bad weather 2. Performant ▸ Fast discovery ▸ Maximise control speed 3. Scalable ▸ Lessons learned
15. @MOLSJEROEN CHALLENGES 1. Easy to use (and test) ▸ Wi-Fi setup ▸ Prepare for bad weather 2. Performant ▸ Fast discovery ▸ Maximise control speed 3. Scalable ▸ Lessons learned
16. WI-FI SETUP PART 1: EASY TO USE (AND TEST)
17. @MOLSJEROEN WI-FI SETUP - GOAL
18. @MOLSJEROEN WI-FI SETUP - GOAL ?
19. @MOLSJEROEN WI-FI SETUP - MECHANISM
20. @MOLSJEROEN WI-FI SETUP - MECHANISM
21. @MOLSJEROEN WI-FI SETUP - MECHANISM SSID & PASSWORD
22. @MOLSJEROEN WI-FI SETUP - MECHANISM
23. @MOLSJEROEN WI-FI SETUP - MECHANISM
24. @MOLSJEROEN WI-FI SETUP - CHROMECAST 4 3 2 1
25. @MOLSJEROEN WI-FI SETUP - CHROMECAST 4 3 2 1
26. @MOLSJEROEN WI-FI SETUP - CHROMECAST 4 3 2 1
27. @MOLSJEROEN WI-FI SETUP - CHROMECAST 8 7 6 5
28. @MOLSJEROEN WI-FI SETUP - CHROMECAST 8 7 6 5
29. @MOLSJEROEN WI-FI SETUP - CHROMECAST 8 7 6 5
30. @MOLSJEROEN WI-FI SETUP - CHROMECAST 12 11 10 9
31. @MOLSJEROEN WI-FI SETUP - CHROMECAST 12 11 10 9
32. @MOLSJEROEN WI-FI SETUP - CHROMECAST 12 11 10 9
33. @MOLSJEROEN WI-FI SETUP - CHROMECAST 12 11 10 9
34. @MOLSJEROEN WI-FI SETUP - CHALLENGES ▸ Typing Wi-Fi passwords ▸ Network switching is fragile ▸ phone/tablet refuses to connect ▸ OS routes request over 4G ▸ Extremely slow feedback ▸ Users have to leave app on iOS ▸ Many bad weather scenarios
35. @MOLSJEROEN WI-FI SETUP - CHALLENGES ▸ 1
36. IF WI-FI SETUP FAILS, YOUR PRODUCT WILL BE RETURNED Yours truly
37. @MOLSJEROEN ANDROID 5.0+ ▸ Behaviour change in Android 5.0 ▸ Wi-Fi without internet ▸ Before: all requests sent over Wi-Fi ▸ After: all requests routed over 3G/4G ▸ Done for all, no subnet matching
38. @MOLSJEROEN ANDROID 5.0+ - SINGLE REQUEST OVER WI-FI @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) public void singleRequestOverWifi(Context context) { NetworkRequest.Builder request = new NetworkRequest.Builder(); request.addTransportType(NetworkCapabilities.TRANSPORT_WIFI); ConnectivityManager connMan = getConnectivityManager(); connMan.registerNetworkCallback(request.build(), new NetworkCallback(){ @Override public void onAvailable(Network network) { // Do network request } }); }
39. @MOLSJEROEN ANDROID 5.0+ - SINGLE REQUEST OVER WI-FI @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) public void singleRequestOverWifi(Context context) { NetworkRequest.Builder request = new NetworkRequest.Builder(); request.addTransportType(NetworkCapabilities.TRANSPORT_WIFI); ConnectivityManager connMan = getConnectivityManager(); connMan.registerNetworkCallback(request.build(), new NetworkCallback(){ @Override public void onAvailable(Network network) { // Do network request } }); }
40. @MOLSJEROEN ANDROID 5.0+ - SINGLE REQUEST OVER WI-FI @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) public void singleRequestOverWifi(Context context) { NetworkRequest.Builder request = new NetworkRequest.Builder(); request.addTransportType(NetworkCapabilities.TRANSPORT_WIFI); ConnectivityManager connMan = getConnectivityManager(); connMan.registerNetworkCallback(request.build(), new NetworkCallback(){ @Override public void onAvailable(Network network) { // Do network request } }); }
41. @MOLSJEROEN ANDROID 5.0+ - SINGLE REQUEST OVER WI-FI @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) public void singleRequestOverWifi(Context context) { NetworkRequest.Builder request = new NetworkRequest.Builder(); request.addTransportType(NetworkCapabilities.TRANSPORT_WIFI); ConnectivityManager connMan = getConnectivityManager(); connMan.registerNetworkCallback(request.build(), new NetworkCallback(){ @Override public void onAvailable(Network network) { // Do network request } }); }
42. @MOLSJEROEN ANDROID 5.0+ - ALL REQUESTS OVER WI-FI @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) public void allRequestsOverWifi(Context context) { NetworkRequest.Builder request = new NetworkRequest.Builder(); request.addTransportType(NetworkCapabilities.TRANSPORT_WIFI); final ConnectivityManager connMan = getConnectivityManager(); connMan.registerNetworkCallback(request.build(), new NetworkCallback() { @Override public void onAvailable(Network network) { connMan.bindProcessToNetwork(network); // Do network requests } }); }
43. @MOLSJEROEN ANDROID 5.0+ - ALL REQUESTS OVER WI-FI @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) public void allRequestsOverWifi(Context context) { NetworkRequest.Builder request = new NetworkRequest.Builder(); request.addTransportType(NetworkCapabilities.TRANSPORT_WIFI); final ConnectivityManager connMan = getConnectivityManager(); connMan.registerNetworkCallback(request.build(), new NetworkCallback() { @Override public void onAvailable(Network network) { connMan.bindProcessToNetwork(network); // Do network requests } }); }
44. @MOLSJEROEN ALTERNATIVES ▸ One product as ethernet bridge ▸ WPS ▸ Other mechanisms for communication ▸ e.g High frequency sound, … ▸ Still need to enter Wi-Fi password ▸ WAC (Homekit devices) ▸ …
45. @MOLSJEROEN #PROTIP - RANDOM SETUP NETWORK NAME ssid
 SETUP ssid
 SETUP ssid
 SETUP ssid
 SETUP ????????
46. @MOLSJEROEN #PROTIP - RANDOM SETUP NETWORK NAME
47. PREPARE FOR BAD WEATHER PART 1: EASY TO USE (AND TEST)
48. @MOLSJEROEN GOOD WEATHER SCENARIOS
49. @MOLSJEROEN BAD WEATHER SCENARIOS
50. @MOLSJEROEN BAD WEATHER SCENARIOS
51. @MOLSJEROEN BAD WEATHER SCENARIOS
52. @MOLSJEROEN INTERESTING SCENARIOS ▸ Bridge (almost) full ▸ Network request failed for reason X ▸ Resource doesn’t exist ▸ Connection lost ▸ Authentication lost ▸ Dynamic state changes ▸ …. => nearly impossible to reproduce manually
53. @MOLSJEROEN SIMULATOR ▸ Hardcoded responses ▸ Simulate dynamic behaviour ▸ Recreate bad weather scenarios
54. @MOLSJEROEN SIMULATOR COMMUNICATIONSTRATEGY REMOTESTRATEGY LOCALSTRATEGY SIMULATORSTRATEGY PRODUCT 1.. ▸ Hardcoded responses ▸ Simulate dynamic behaviour ▸ Recreate bad weather scenarios
55. @MOLSJEROEN SIMULATOR COMMUNICATIONSTRATEGY REMOTESTRATEGY LOCALSTRATEGY SIMULATORSTRATEGY PRODUCT 1.. ▸ Hardcoded responses ▸ Simulate dynamic behaviour ▸ Recreate bad weather scenarios
56. @MOLSJEROEN #PROTIP - DON'T USE THE WORD “DEVICE” ▸ Use product or appliance
57. @MOLSJEROEN CHALLENGES 1. Easy to use (and test) ▸ Wi-Fi setup ▸ Prepare for bad weather 2. Performant ▸ Fast discovery ▸ Maximise control speed 3. Scalable ▸ Lessons learned
58. DISCOVERY PART 2: PERFORMANT
59. @MOLSJEROEN LOCAL DISCOVERY ▸ Multicast based ▸ Often Unicast responses ▸ SSDP, mDNS,…
60. @MOLSJEROEN LOCAL DISCOVERY ▸ Multicast based ▸ Often Unicast responses ▸ SSDP, mDNS,…
61. @MOLSJEROEN LOCAL DISCOVERY ▸ Multicast based ▸ Often Unicast responses ▸ SSDP, mDNS,…
62. @MOLSJEROEN LOCAL DISCOVERY ▸ Multicast based ▸ Often Unicast responses ▸ SSDP, mDNS,… ▸ Alternatives ▸ IP scan ▸ Manual IP ▸ Proprietary product IP server
63. @MOLSJEROEN LOCAL DISCOVERY - IP SERVER https://www.meethue.com/api/nupnp [ { "id":"001788fffe100491", “internalipaddress":"192.168.2.23" }, { "id":"001788fffe09a168", “internalipaddress":"192.168.88.252" } ]
64. @MOLSJEROEN REMOTE DISCOVERY ▸ Web service ▸ product announces ▸ interface to get products ▸ Security ▸ Local setup first required
65. @MOLSJEROEN OPTIMISING DISCOVERY
66. @MOLSJEROEN OPTIMISING DISCOVERY 1. Discover local and remote 2. Connect local and remote ▸ ~5 - 10 seconds ▸ Empty UI when no products found
67. @MOLSJEROEN OPTIMISING DISCOVERY - REMEMBER PRODUCTS 1. Products from database 2. Discover local and remote 3. Connect local and remote ▸ ~5 - 10 seconds ▸ Disconnected device when not found ▸ Need for deleting devices
68. @MOLSJEROEN OPTIMISING DISCOVERY - REMEMBER STATE 1. Products & last state from database 2. Discover local and remote 3. Connect local and remote 4. Update UI with actual state ▸ ~3 - 4 seconds ▸ UI instantly ready ▸ Control possible after discovery
69. @MOLSJEROEN OPTIMISING DISCOVERY - REMEMBER CONNECTION INFO 1. Products, last state & connection info from database 2. Attempt local and remote connection 3. Fallback discovery 4. Update UI with actual state ▸ ~0 seconds ▸ UI & Control instantly ready ▸ Remember IP address and last SSID
70. @MOLSJEROEN #PROTIP - SIMPLIFY DEVELOPMENT BY LIMITING STATE TRANSITIONS ▸ Transition local to remote connection always via disconnected
71. MAXIMISE CONTROL SPEED PART 2: PERFORMANT
72. @MOLSJEROEN CONTROL SPEED 1. Always prefer local connection 2. Optimise local connection 3. Reduce amount of requests 4. Instant UI ▸ Implicit cost savings ▸ Remote connection usually not critical
73. @MOLSJEROEN 1. PREFER LOCAL CONNECTION ▸ Inherently faster ▸ Cost benefit ▸ Force connection
74. @MOLSJEROEN 1. PREFER LOCAL CONNECTION COMPOSITESTRATEGY PRODUCT LOCALSTRATEGY SIMULATORSTRATEGY REMOTESTRATEGY COMMUNICATIONSTRATEGY - boolean isAvailable() ▸ Inherently faster ▸ Cost benefit ▸ Force connection
75. @MOLSJEROEN 2. OPTIMISE LOCAL CONNECTION ▸ Data compression ▸ Cache headers ▸ Socket reuse ▸ => OKHttp ▸ Interference UDP/TCP or other traffic ▸ Wireshark low level traffic
76. @MOLSJEROEN 2. OPTIMISE LOCAL CONNECTION ▸ Data compression ▸ Cache headers ▸ Socket reuse ▸ => OKHttp ▸ Interference UDP/TCP or other traffic ▸ Wireshark low level traffic
77. @MOLSJEROEN 3. REDUCE # REQUESTS ▸ Toggle sequences ▸ Combine requests to same endpoint
78. @MOLSJEROEN 3. REDUCE # REQUESTS private Map<String, Object> pendingAttr = new HashMap<>(); public void setOn(boolean isOn) { synchronized (pendingAttr) { pendingAttr("KEY_ON", isOn); } sendUpdate(); } private void sendUpdate() { strategy.doRequest(updateRequest); }
79. @MOLSJEROEN 3. REDUCE # REQUESTS private Map<String, Object> pendingAttr = new HashMap<>(); public void setOn(boolean isOn) { synchronized (pendingAttr) { pendingAttr("KEY_ON", isOn); } sendUpdate(); } private void sendUpdate() { strategy.doRequest(updateRequest); }
80. @MOLSJEROEN 3. REDUCE # REQUESTS private Map<String, Object> pendingAttr = new HashMap<>(); public void setOn(boolean isOn) { synchronized (pendingAttr) { pendingAttr("KEY_ON", isOn); } sendUpdate(); } private void sendUpdate() { strategy.doRequest(updateRequest); }
81. @MOLSJEROEN 3. REDUCE # REQUESTS private Map<String, Object> pendingAttr = new HashMap<>(); public void setOn(boolean isOn) { synchronized (pendingAttr) { pendingAttr("KEY_ON", isOn); } sendUpdate(); } private void sendUpdate() { strategy.doRequest(updateRequest); }
82. @MOLSJEROEN 3. REDUCE # REQUESTS private UpdateRequest updateRequest = new UpdateRequest() { @Override public String getEndpoint() { return "/lights"; } @Override public Map<String, Object> getAttributes() { synchronized (pendingAttr) { Map<String, Object> copy = new HashMap<>(pendingAttr); pendingAttr.clear(); return copy; } }};
83. @MOLSJEROEN 3. REDUCE # REQUESTS private UpdateRequest updateRequest = new UpdateRequest() { @Override public String getEndpoint() { return "/lights"; } @Override public Map<String, Object> getAttributes() { synchronized (pendingAttr) { Map<String, Object> copy = new HashMap<>(pendingAttr); pendingAttr.clear(); return copy; } }};
84. @MOLSJEROEN 3. REDUCE # REQUESTS private UpdateRequest updateRequest = new UpdateRequest() { @Override public String getEndpoint() { return "/lights"; } @Override public Map<String, Object> getAttributes() { synchronized (pendingAttr) { Map<String, Object> copy = new HashMap<>(pendingAttr); pendingAttr.clear(); return copy; } }};
85. @MOLSJEROEN 3. REDUCE # REQUESTS private class CommunicationStrategy { private Set<UpdateRequest> queue = new LinkedHashSet<>(); public void doRequest(UpdateRequest updateRequest) { queue.add(updateRequest); // logic to trigger requests } }
86. @MOLSJEROEN 3. REDUCE # REQUESTS private class CommunicationStrategy { private Set<UpdateRequest> queue = new LinkedHashSet<>(); public void doRequest(UpdateRequest updateRequest) { queue.add(updateRequest); // logic to trigger requests } }
87. @MOLSJEROEN 3. REDUCE # REQUESTS private class CommunicationStrategy { private Set<UpdateRequest> queue = new LinkedHashSet<>(); public void doRequest(UpdateRequest updateRequest) { queue.add(updateRequest); // logic to trigger requests } }
88. @MOLSJEROEN 4. INSTANT UI
89. @MOLSJEROEN #PROTIP - NEVER ASSUME, ALWAYS MEASURE
90. @MOLSJEROEN CHALLENGES 1. Easy to use (and test) ▸ Wi-Fi setup ▸ Prepare for bad weather 2. Performant ▸ Fast discovery ▸ Maximise control speed 3. Scalable ▸ Lessons learned
91. LESSONS LEARNED PART 3: SCALABLE
92. @MOLSJEROEN DEVELOPING A SYSTEM 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 1 2 3 4 5 6 7 8 9 10 11 12 New Features Maintenance
93. @MOLSJEROEN DEVELOPING A SYSTEM 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 1 2 3 4 5 6 7 8 9 10 11 12 New Features Maintenance
94. @MOLSJEROEN DEVELOPING A SYSTEM 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 1 2 3 4 5 6 7 8 9 10 11 12 New Features Maintenance
95. @MOLSJEROEN DEVELOPING A SYSTEM 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 1 2 3 4 5 6 7 8 9 10 11 12 New Features Maintenance
96. @MOLSJEROEN DEVELOPING A SYSTEM 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 1 2 3 4 5 6 7 8 9 10 11 12 New Features Maintenance
97. @MOLSJEROEN DEVELOPING A SYSTEM 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 1 2 3 4 5 6 7 8 9 10 11 12 New Features Maintenance
98. THE TRUE COST OF SOFTWARE IS IN ITS MAINTENANCE unknown
99. @MOLSJEROEN MAINTENANCE ▸ Big focus on automated testing ▸ Need for good architecture ▸ Over-engineering has a huge cost ▸ Avoid Hype driven development ▸ Think well before introducing new features
100. @MOLSJEROEN #FAILUREHALLOFFAME ▸ Detect working internet connection (Wifi, no internet) ▸ Instrumentation tests ▸ Lock orientation to portrait ▸ Nested fragments ▸ Broadcasts as callbacks ▸ Post( new Runnable(…) ) ▸ Null checks ▸ …. RELATED BLOGPOSTS
101. WRAP UP
102. IN SOFTWARE DESIGN, OFTEN THE CONSEQUENCES OF YOUR DECISIONS DON'T BECOME APPARENT FOR YEARS Kent Beck
103. @MOLSJEROEN CONCLUSION ▸ Developing connected products is hard ▸ Developing systems is even harder ▸ Maximise local connections ▸ Keep things simple for users ▸ Strong focus on automated testing ▸ Proper architecture
104. @MOLSJEROEN IMAGE CREDITS • Welcome image http://www.store.meethue.com/media/catalog/product/cache/1/ feature_img_7/9df78eab33525d08d6e5fb8d27136e95/e/x/expand_your_ecosystem_1.jpg • Wi-fi setup https://images.philips.com/is/image/PhilipsConsumer/AW3000_10-MI1-global-001? $jpglarge$&wid=1250 • Good vs bad weather https://images.philips.com/is/image/PhilipsConsumer/FC8830_82-U1P-global-001? $jpglarge$&wid=1250 • Discovery https://s.blogcdn.com/slideshows/images/slides/403/398/9/S4033989/slug/l/philips-hue-motion- sensor-wall-attached-1.jpg • Control speed http://www.philips.com/consumerfiles/newscenter/main/shared/assets/de/Downloadablefile/ press/elektro_hausgeraete/20140716_Philips_Air_AC4072_11_Lifestyle_4.jpg • Developing a system http://www.philips.com/consumerfiles/newscenter/main/standard/resources/corporate/ press/2014/IFA2014/Saeco-GranBaristo-Avanti_image-5.jpg • Warp up http://www.philips.com/consumerfiles/newscenter/main/standard/resources/corporate/press/2014/ IFA2014/Hue-Beyond-lifestyle_all-three.jpg
105. @MOLSJEROEN MANY THANKS ▸ Jeroen Mols (Belgium) ▸ @MolsJeroen ▸ http://jeroenmols.com/blog