From monoliths on AWS to microservices on Google Cloud Platform

An experience story of how we're migrating from AWS to Google Cloud Platform.

Transcript

1. From monoliths on AWS to microservices on GCP Mike Nikles @mikenikles Lead Engineer @ VC Labs
2. Stand Up If you’re able to Choreography by Kent C. Dodds
3. VC Labs • Chatbot platform for market research & customer insights • NFL, Canucks, Warner Brothers, … • 10 engineers and counting
7. Agenda • Business challenges • AWS architecture • GCP architecture
8. Agenda • Business challenges • AWS architecture • GCP architecture
9. Business challenges AWS architecture GCP architecture Traffic Patterns Data Jobs
10. Business challenges 5’ 3,000% AWS architecture GCP architecture
11. Agenda • Business challenges • AWS architecture • GCP architecture
12. 2 monoliths on AWS Business challenges AWS architecture GCP architecture
13. 2 monoliths on AWS Business challenges AWS architecture GCP architecture
14. 3 monoliths on AWS Business challenges AWS architecture GCP architecture
15. Challenges with the monoliths Business challenges AWS architecture GCP architecture • Ownership & accountability • Single points of failure • Development velocity • Scale & release all or nothing
16. Agenda • Business challenges • AWS architecture • GCP architecture
17. Google Cloud Platform Business challenges AWS architecture GCP architecture
18. 18 The Products logos contained in this icon library may be used freely and without permission to accurately reference Google's technology and tools, for instance in books or architecture diagrams. Google Cloud Platform Icons Compute Big Data BigQuery Cloud Dataflow Cloud Dataproc Cloud Datalab Cloud Pub/Sub Genomics Storage & Databases Cloud Storage Cloud Bigtable Cloud Datastore Cloud SQL Cloud Spanner Persistent Disk Machine Learning Cloud Machine Learning Cloud Vision API Cloud Speech API Cloud Natural Language API Cloud Translation API Cloud Jobs API Data Studio Cloud Dataprep Cloud Video Intelligence API Advanced Solutions Lab Compute Engine App Engine Kubernetes Engine GPU Cloud Functions Container- Optimized OS Identity & Security Cloud IAM Cloud Resource Manager Cloud Security Scanner Key Management Service BeyondCorp Data Loss Prevention API Identity-Aware Proxy Security Key Enforcement Internet of Things Cloud IoT Core Transfer Appliance
19. 19 The Products logos contained in this icon library may be used freely and without permission to accurately reference Google's technology and tools, for instance in books or architecture diagrams. Google Cloud Platform Icons Developer Tools Cloud SDK Cloud Deployment Manager Cloud Source Repositories Cloud Tools for Android Studio Cloud Tools for IntelliJ Cloud Tools for PowerShell Cloud Tools for Visual Studio Container Registry Google Plug-in for Eclipse GCP products and services without a unique icon have a generic hexagon. Cloud Test Lab Networking Virtual Private Cloud Cloud Load Balancing Cloud CDN Cloud Interconnect Cloud DNS Cloud Network Cloud External IP Addresses Cloud Firewall Rules Cloud Routes Cloud VPN Management Tools Stackdriver Monitoring Logging Error Reporting Trace Debugger Cloud Deployment Manager Cloud Endpoints Cloud Console Cloud Shell Cloud Mobile App Cloud Billing API Cloud APIs Cloud Router Dedicated Interconnect Container Builder
20. Business challenges AWS architecture GCP architecture We’ve seen it (almost all)
21. Business challenges AWS architecture GCP architecture GCP ? October 19, 2017
22. Business challenges AWS architecture GCP architecture GCP !! October 19, 2017
23. Business challenges AWS architecture GCP architecture GCP !! GKE ! October 19, 2017 October
24. Business challenges AWS architecture GCP architecture GCP !! GKE ! October 19, 2017 GAE ! October November
25. Business challenges AWS architecture GCP architecture GCP !! GKE ! October 19, 2017 GAE ! October November GCF, for real! December
26. Business challenges AWS architecture GCP architecture
27. Business challenges First distributed system: Profile Variables Job AWS architecture GCP architecture
28. 3 monoliths on AWS Business challenges AWS architecture GCP architecture
29. Jobs Service on AWS Business challenges AWS architecture GCP architecture
30. Profile Variables Job Business challenges AWS architecture GCP architecture
31. Business challenges Why the profile variables service? • Low risk • Scheduled, infrequent invocations • Integrations with the outside world • Requires some sort of storage • Tens of thousands of respondents ◦ Each with 10, 15, 20+ individual profile variables In summary, it requires a bit of everything AWS architecture GCP architecture
32. Business challenges AWS architecture GCP architecture
33. Business challenges AWS architecture GCP architecture
34. Business challenges AWS architecture GCP architecture
35. Business challenges AWS architecture GCP architecture
36. Business challenges AWS architecture GCP architecture
37. Business challenges AWS architecture GCP architecture
38. Business challenges AWS architecture GCP architecture
39. Business challenges AWS architecture GCP architecture Why Google Cloud Platform?
40. Built by developers, for developers Business challenges AWS architecture GCP architecture
41. gcloud Business challenges AWS architecture GCP architecture
42. gcloud --quiet Business challenges AWS architecture GCP architecture
43. Sustained Use Discounts Business challenges AWS architecture GCP architecture
44. Business challenges AWS architecture GCP architecture
45. Application Default Credentials Business challenges AWS architecture GCP architecture
46. Stackdriver Centralized Logging Error Reporting Alerting Tracing Business challenges AWS architecture GCP architecture
47. Business challenges AWS architecture GCP architecture Lessons Learned
48. Go slow, one piece at a time Business challenges AWS architecture GCP architecture
49. Make the right thing the easy thing Business challenges AWS architecture GCP architecture
50. Decouple your services, or else ... Business challenges AWS architecture GCP architecture
51. Business challenges AWS architecture GCP architecture … microliths
52. Don’t document what you can automate Business challenges AWS architecture GCP architecture
53. RTFM (and join Slack) Business challenges AWS architecture GCP architecture https://cloud.google.com/docs/
54. It’s been four months... Business challenges AWS architecture GCP architecture We’ve only touched the surface
55. Q&A / Discussions Mike Nikles @mikenikles Lead Engineer @ VC Labs