[Mark van Straten] The Google Cloud powers your Philips Hue Lightbulbs

Presentation from GDG DevFest Ukraine 2018 - the biggest community-driven Google tech conference in the CEE.

Learn more at: https://devfest.gdg.org.ua

__

Philips Hue is one of the leading global Internet of Things platforms and has integrations with Amazon Echo, Google Home, IFTTT & Nest to make your lights work for you. The platform has grown considerably during the past years and to accommodate this has been migrated towards Google Kubernetes Engine at the end of 2015.

In this session Mark will take you on an adventure into the Philips Hue architecture and show you what it takes to turn on a lightbulb in 2018. While doing so he will touch upon the many different google Cloud products being used and the reasons why. Finally he will explain how his team solved challenges like monitoring and Continuous Delivery with minimal impact for our millions of users.

Transcript

1. Mark van Straten @markvanstraten
   Senior Engineer @Q42
   The Google Cloud powers your
   Philips Hue Lightbulbs
   

   View Slide

2. View Slide

3. View Slide

4. WHERE WE
   STARTED
   

   View Slide

5. View Slide

6. WILL IT SUCCEED?
   UNKNOWN (THEN)
   

   View Slide

7. MINIMAL
   VIABLE
   PRODUCT
   

   View Slide

8. APP ENGINE
   2 DEVELOPERS
   

   View Slide

9. FIRMWARE
   UPGRADES
   

   View Slide

10. View Slide

11. View Slide

12. BACKUP PLAN
    POLL EVERY 10 SEC
    

    View Slide

13. 10 USERS
    ≈ 1 REQ/S
    

    View Slide

14. 100.000 USERS
    
    ≈ 10.000 REQ/S
    

    View Slide

15. ©KC Green, kcgreendotcom.com - gunshowcomic.com/648
    

    View Slide

16. WHERE WE
    ARE TODAY
    

    View Slide

17. View Slide

18. View Slide

19. 2012
    2014 2016
    2012
    GAE GAE flex GKE
    

    View Slide

20. CURRENT
    ARCHITECTURE
    Backend notifier
    Token validation
    service
    Websocket
    API
    HTTP Polling API
    Hue
    Cloud
    API
    

    View Slide

21. 1 cluster 25 services
    160 nodes 600 pods
    

    View Slide

22. 10 Developers
    

    View Slide

23. BUMPS IN
    THE ROAD
    Photo credit: Cloudfront.net
    

    View Slide

24. LOGGING
    

    View Slide

25. SO MUCH DATA
    

    View Slide

26. DISTRIBUTED
    LOGS
    

    View Slide

27. View Slide

28. View Slide

29. METRICS
    

    View Slide

30. i
    

    View Slide

31. SEE THE
    BIGGER PICTURE
    https://www.nasa.gov/content/earth-right-now-image-gallery/
    

    View Slide

32. View Slide

33. MICROSERVICES
    

    View Slide

34. LESS COMPLEXITY
    LESS VELOCITY
    

    View Slide

35. View Slide

36. CONTINUOUS DELIVERY
    

    View Slide

37. View Slide

38. TEAMSIZE
    

    View Slide

39. HOW TO SHARE
    KNOWLEDGE?
    

    View Slide

40. Architectural
    Decision Records
    

    View Slide

41. View Slide

42. PULL REQUESTS
    

    View Slide

43. View Slide

44. AUTOMATE QUALITY
    CHECKS
    

    View Slide

45. View Slide

46. WHAT WE LEARNED
    ALONG THE WAY
    

    View Slide

47. APPENGINE =
    NO DEVOPS
    

    View Slide

48. KUBERNETES
    REQUIRES DEVOPS
    

    View Slide

49. METRIC = WHAT
    LOG = WHY
    

    View Slide

50. MICROSERVICES =
    NO SILVER BULLET
    

    View Slide

51. SCALING TEAMS
    IS HARD
    

    View Slide

52. WHERE WE
    ARE GOING
    https://mars.nasa.gov/mer/gallery/specialEffects/opportunity/
    

    View Slide

53. MULTI REGION
    

    View Slide

54. ISTIO
    

    View Slide

55. “If everything seems
    under control, you're not
    going fast enough.” -
    Mario Andretti
    

    View Slide

56. THANKS FOR
    LISTENING
    @markvanstraten
    
    q42.com
    

    View Slide

57. References
    • https://cloud.google.com/logging/docs/logs-based-metrics/
    
    • https://medium.com/@brunose/gitlab-ci-cd-kubernetes-65eec29d0555
    
    • https://www.sumologic.com/blog/log-management-analysis/logs-metrics-overview/
    
    • https://martinfowler.com/articles/microservice-trade-offs.html#ops
    
    • https://medium.com/google-cloud/istio-why-do-i-need-it-18d122838ee3
    
    • https://adr.github.io/
    
    • https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1/
    

    View Slide