Microservices - War Stories

Some stories and things to be careful about when moving to microservices

Transcript

1. Shop smart, feel great Microservices War Stories
2. Big Data/Cloud Architect @ vente-exclusive.com Google Developer Expert - follow @alexvb Alex Van Boxel
3. The views and opinions expressed in this talk are those of the authors and do not necessarily reflect the official position of Vente-Exclusive or Google
4. Monolith - ETL Data Lake Production database Reporting database BigQuery Cloud Pub/Sub Nightly table transfers SparkPost ADYEN App Engine App Engine Cloud Storage Channel interactions (CAL events) Payments E-mail .Net Applications Tableau Server Microsoft Excel Cloud Dataflow Cloud Dataflow Cloud Dataflow
5. Why?
6. the people behind the lines People
7. People For a lot of people : new tech stack
8. People Multi Repo
9. People Multi Datastore
10. People Multi Language
11. People Developer Experience
12. People Bootstrapping
13. People Enable developers with Proxied Kubernetes Namespaces
14. People Enable developers with Proxied Local Development
15. People Cool Stackdriver Stuff
16. People
17. do's, don't and the complexity Patterns and Technology
18. Patterns and Technology Kubernetes Engine
19. Patterns and Technology CI/CD pipeline > start early
20. Patterns and Technology Don't take the slogan "It should be rebuild in a few days" as a guideline!
21. Patterns and Technology Embrace eventual consistency, go async
22. Patterns and Technology 99,999 %
23. Patterns and Technology No Hard Dependencies on External services
24. Patterns and Technology Be careful with single point of failure
25. Patterns and Technology Scaling, easy right?
26. Patterns and Technology Kubernetes knots and bolts ( resource limits )
27. let's at least agree that Contracts and Conventions
28. Contracts and Conventions Make convention and stick with it
29. Contracts and Conventions boreporting-api
30. Contracts and Conventions boreporting-api
31. Contracts and Conventions tableauauth-api
32. Contracts and Conventions Contracts First? well...
33. Contracts and Conventions gRPC
34. Contracts and Conventions gRPC + HTTP Gateway
35. my reliable site Operations and SRE
36. Operations and SRE Logging
37. Operations and SRE Logging Tracing
38. Operations and SRE
39. Operations and SRE Logging Tracing Monitoring
40. Operations and SRE
41. Operations and SRE SRE
42. starting over, make it count Opportunities
43. Opportunities Streaming operational data to the data lake
44. Opportunities Angular Front-end MongoDB Databases Angular Back-Office Microservices .Net Event sourcing Cloud Pub/Sub Cloud Bigtable Client events -- clickstream (sessions) -- interesting business events DATA microservices Python / GO DATA microservices Python / GO SparkPost ADYEN External events Cloud Pub/Sub Payments E-mail MS SQL IMS, Area51 Entity builder + KPI calculation DataFlow streaming BigQuery Events processing Dataflow streaming Production data -- not accessible outside IT -- splitted per microservice Reporting data -- real-time! -- full change history -- self-service BI -- excel report generation -- permission based -- GDPR! -- SaaS -- Standard SQL language -- High-availability -- low-latency -- real-time -- PaaS -- backup + history Elasticsearch elastic.io managed -- search (backoffice only atm) -- future: site search Website backoffice PHOTON
45. Opportunities Data Lake
46. Opportunities Data Lake enables Ops
47. Opportunities Data Lake for GDPR compliance Forseti Security
48. Opportunities Security Forseti Security Istio
49. Opportunities Make it Count
50. Shop smart, feel great THANK YOU twitter @alexvb