[Mark van Straten] The Google Cloud powers your Philips Hue Lightbulbs

Presentation from GDG DevFest Ukraine 2018 - the biggest community-driven Google tech conference in the CEE.

Learn more at: https://devfest.gdg.org.ua

__

Philips Hue is one of the leading global Internet of Things platforms and has integrations with Amazon Echo, Google Home, IFTTT & Nest to make your lights work for you. The platform has grown considerably during the past years and to accommodate this has been migrated towards Google Kubernetes Engine at the end of 2015.

In this session Mark will take you on an adventure into the Philips Hue architecture and show you what it takes to turn on a lightbulb in 2018. While doing so he will touch upon the many different google Cloud products being used and the reasons why. Finally he will explain how his team solved challenges like monitoring and Continuous Delivery with minimal impact for our millions of users.

Transcript

1. Mark van Straten @markvanstraten Senior Engineer @Q42 The Google Cloud powers your Philips Hue Lightbulbs
4. WHERE WE STARTED
6. WILL IT SUCCEED? UNKNOWN (THEN)
7. MINIMAL VIABLE PRODUCT
8. APP ENGINE 2 DEVELOPERS
9. FIRMWARE UPGRADES
12. BACKUP PLAN POLL EVERY 10 SEC
13. 10 USERS ≈ 1 REQ/S
14. 100.000 USERS ≈ 10.000 REQ/S
15. ©KC Green, kcgreendotcom.com - gunshowcomic.com/648
16. WHERE WE ARE TODAY
19. 2012 2014 2016 2012 GAE GAE flex GKE
20. CURRENT ARCHITECTURE Backend notifier Token validation service Websocket API HTTP Polling API Hue Cloud API
21. 1 cluster 25 services 160 nodes 600 pods
22. 10 Developers
23. BUMPS IN THE ROAD Photo credit: Cloudfront.net
24. LOGGING
25. SO MUCH DATA
26. DISTRIBUTED LOGS
29. METRICS
30. i
31. SEE THE BIGGER PICTURE https://www.nasa.gov/content/earth-right-now-image-gallery/
33. MICROSERVICES
34. LESS COMPLEXITY LESS VELOCITY
36. CONTINUOUS DELIVERY
38. TEAMSIZE
39. HOW TO SHARE KNOWLEDGE?
40. Architectural Decision Records
42. PULL REQUESTS
44. AUTOMATE QUALITY CHECKS
46. WHAT WE LEARNED ALONG THE WAY
47. APPENGINE = NO DEVOPS
48. KUBERNETES REQUIRES DEVOPS
49. METRIC = WHAT LOG = WHY
50. MICROSERVICES = NO SILVER BULLET
51. SCALING TEAMS IS HARD
52. WHERE WE ARE GOING https://mars.nasa.gov/mer/gallery/specialEffects/opportunity/
53. MULTI REGION
54. ISTIO
55. “If everything seems under control, you're not going fast enough.” - Mario Andretti
56. THANKS FOR LISTENING @markvanstraten q42.com
57. References • https://cloud.google.com/logging/docs/logs-based-metrics/ • https://medium.com/@brunose/gitlab-ci-cd-kubernetes-65eec29d0555 • https://www.sumologic.com/blog/log-management-analysis/logs-metrics-overview/ • https://martinfowler.com/articles/microservice-trade-offs.html#ops • https://medium.com/google-cloud/istio-why-do-i-need-it-18d122838ee3 • https://adr.github.io/ • https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1/