Micro serviços e dados distribuidos

Palestra na python sudeste sobre dados distribuidos e como resolvemos.

Transcript

1. DATA DICHOTOMY Data system are about exposing data. Services are about hiding it http://vignette2.wikia.nocookie.net/memoryalpha/images/e/e7/Data_makeup_tests.jpg/revision/latest?cb=20120725111417&path-prefix=en
2. 15+ anos Sr System Engineer at Bemobi / Opera github.com/aoqfonseca @aoqfonseca
3. https://speakerdeck.com/aoqfonseca
4. https://en.wikipedia.org/wiki/Data_(Star_Trek) “ Data: is a character from Star Trek , from next generation. A robot with all knowledge…"
5. Hail to the King!!!
6. SERVICES DATA USER
7. https://martinfowler.com/articles/microservices.html “ … In short, the microservice architectural style [1] is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies… "
8. Micro services everywhere
10. Assinatura Medias DRM Atividades Homes Conteúdo
11. • Easy to deploy • Evolve multiples features • Monitoring • Model / Data • Transactions / PROS
12. Assinatura Medias DRM Atividades Homes Conteúdo
13. Usuários Medias Autorização Atividades Homes Conteúdo
14. • Polyglot (best tool for the job) • Scaling • Distributed • Reduced number of LoC (less code, less bug) / PROS
15. AWS
16. DOMAIN
17. CODE DATA
20. Share State
21. First try …
22. A B C BD
23. • Database aren’t easy to scale (shards, shared files, etc) • Single point of failure. • Not polyglot • Hard evolving: if you change some scheme, you need to adapt all services that use it. • Not isolated
24. CODE DATA Borders
25. CODE DATA CODE DATA CODE DATA
27. CODE DATA CODE DATA CODE DATA ADMIN
28. Evolving …
29. Flow State
30. CODE DATA CODE DATA CODE DATA
34. Evolving …
35. https://www.cartoonstock.com/directory/q/queue_jumpers.asp Queues
36. ASYNC
40. CELERY
42. Ben Stopford. December 13, 2016. https://www.confluent.io/blog/data-dichotomy-rethinking-the-way-we-treat-data-and-services/ JR D'Amore. May 5, 2015. https://www.thoughtworks.com/pt/insights/blog/scaling-microservices-event-stream
45. Neha Narkhede : 18/02/2016 https://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines/
46. https://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines/
49. Kafka Stream Stream Stream Emit events