Почему микросервисы не летают и как помочь им взлететь

SECR 2018
Александр Щербаков
Senior Software Engineer, Аурига

В последние время все больше команд начинают использовать микросервисы в своих проектах. Это следствие того, что бизнес прямо указывает в требованиях возможность масштабирования и highload уже при старте любого, даже не очень большого проекта. Однако, команды, которые занимались раньше разработкой классических клиент- серверных решений, бывают не готовы к созданию правильной микросервисной архитектуры. И это порождает столько проблем при эксплуатации разработанной системы, что потенциальные – и несомненные, – преимущества ее использования остаются за бортом. В докладе мы дадим обзор наиболее распространенных сложностей, с которыми сталкиваются команды при реализации микросервисов, на конкретных примерах проанализируем возможные ошибки при разработке и рассмотрим преимущества решения на основе event-driven подхода.

Transcript

1. Why microservices do not fly and how to help them to take off Alexandr Shcherbakov Auriga, Inc Software Engineering Conference Russia 2018 October 12-13 Moscow
2. Quiz • Who has ever developed microservices architecture? • Who has positive experience? • Who has negative experience? 2
3. Client Classic client-server architecture Server 3
4. Client Multi-tier architecture Storage Service 4
5. Client SOA Хранилища Сервисы ESB Storages Сервисы Services 5
6. Nowadays typical case 1. We need a new enterprise system 2. We have the distribution network of clients 3. We have only 6 months for MVP and one year for first release, developing for the end of days. 4. But microservices are strongly required! 6
7. Microservices are just services but... • Domain splitting • Independence • Everyone has own storage 7
8. How to name a microservice? User service Profiles Groups 8
9. How to name a microservice? Authorization service Profiles Groups Authentication Authorization 9
10. How to name a microservice? Authorization service Profiles Groups Authentication Authorization Customers? Business Units? 10
11. How to name a microservice? Gladiolus service Profiles Groups Authentication Authorization Customers Business Units 11
12. How to name a microservice? Gladiolus service Profiles Groups Authentication Authorization Customers Business Units Postcodes States 12
13. How to synchronize data storages? Gladiolus MySQL Order MySQL 13
14. How to synchronize data storages? Gladiolus MySQL Order MySQL 14
15. How to synchronize data storages? Gladiolus MySQL Order MySQL Master-Slave 15
16. How to synchronize data storages? Gladiolus MySQL GIS MySQL Order PostgreSQL 16
17. How to synchronize data storages? Gladiolus MySQL GIS service MySQL Order PostgreSQL ETL 17
18. How to call a service? Order Plan 18
19. How to call a service? Order Plan Store 19 Gladiolus
20. How to call a service? Order Plan Store 20 Gladiolus
21. How to do transactions? Order Store Frontend 21
22. How to do transactions? Order 1. ID 2. Store 3. DB Store Frontend 22
23. How to do transactions? Order 1. ID 2. Store 3. DB Store Frontend 23
24. How to do transactions? Order 1. ID 2. 2PC 1. DB 2. Store Store REST? Locking Frontend 24
25. How to do transactions? Order 1. ID 2. 2PC 1. DB 2. Store Store Duplication Frontend 25
26. How to design software architecture? Gladiolus Postgres Order Store MySQL ETL Neo4J MongoDB MySQL GIS Plan Notification 26
27. How to change software architecture? REUSE A MICROSERVICE ONE DOES NOT SIMPLY 27
28. Top 3 of microservices benefits • Flexible scaling • Flexible deployment and functionality • Flexible development 28
29. The key benefit of microservices architecture Flexibility! Agile! 29
30. Monolith vs Microservices • Clear and hard logic. ACID • Strong contracts and interfaces • Integrity and normalization • Soft and fuzzy logic. Eventually consistent • Soft contracts and dependencies • Data duplication and polymorphing 30
31. Monolith vs Microservices 31
32. Monolith vs Microservices 32
33. Monolith vs Microservices Microservices are weak but they can adapt to circumstances 33
34. How to make components more independent Order Plan get 34
35. The event driven design paradigm Order Plan event ? event 35 Martin Fowler https://www.martinfowler.com/eaaDev/EventCollaboration.html
36. The event driven design for microservices 36 Events Bus Order Plan
37. The problem of distributed storage 37 • We are used to store the final state of data • Microservices are distributed system • But how can we reconcile the data in the distributed system properly? Order { …. } Order { …. } ???
38. The problem of distributed storage 38 • Use the distributed transaction or 2PC (with locking and complexity) • Use the event sourcing as a single source of true
39. The event sourcing Order Created Order Changed state ... Order Changed address Order (aggregate) { …. address: “...”, state: “processing” } 39
40. The event sourcing • Single source of true • Immutable • Atomic • Compensation events 40
41. The event sourcing Order Created Order Changed state ... Order Canceled state Order (aggregate) { …. address: “...”, state: “created” } 41
42. The event sourcing Order Created Order Changed state ... Snapshot Order (aggregate) { …. address: “...”, state: “processing” } 42
43. The event sourcing Event storage • Event Log (file + index) • DB Table + JSON • NoSQL (Elastic, MongoDB) • Kafka (with full persistence) 43
44. The event sourcing • M. Fowler. https://martinfowler.com/eaaDev/EventSourcing.html • Chris Richardson. https://microservices.io/patterns/data/event- sourcing.html • Sebastian Daschner. https://blog.sebastian- daschner.com/entries/event_sourcing_cqrs_video_course • Jonas Bonér. http://jonasboner.com/articles/ 44
45. Eventual consistency Or don’t worry about the Strong Consistency and just take the Eventual Consistency • In distributed system the data can be inconsistent anyway (Frontend - Backend) • If you use the event sourcing you can’t lose any data and all data will be consistent ultimately • For critical transaction use the Soft Consistency(status changing or pending records) 45
46. The event bus as an environment • Simple publisher-subscriber queue • Capacity • Persistency • Performance • Distributed 46
47. The event bus as an environment • Kafka • RabbitMQ (but it’s overuse) • Amazon (SNS+SQS) • NSQ 47
48. What do we have for agile microservices architecture creation? • The event driven design paradigm • The event sourcing storage • The event bus as an environment • The eventual consistency instead of transactions 48
49. How to cook a microservice • CQRS • Stateless • Resources oriented API • Event handling 49
50. How to cook a microservice API Aggregates Queries Commands Events Events Bus Handlers 50
51. How to cook a microservice API Aggregates Queries Commands Events Events Bus Handlers 51
52. How to cook a microservice API Gateway Notification Frontend Service Events Bus Service 52
53. Event driven microservices architecture Plan Order Events Bus Store 1 create order 2 4.1 6 3 update store 4.2 5 update plan 53
54. Event driven microservices architecture Profile Events Bus User management Order Store Plan API Gateway Frontend Authentication Notification Unique Id Integration and validation 54
55. Authentication and user management • Use OAuth • Frameworks: Spring, IndentyServer 4 • PaaS & SaaS: Auth0, Curity https://oauth.net/code/ 55
56. API Gateway The Facade of your microservices • Authentication • Authorization • Data aggregate • Intelligent caching • Discovering and balancing • Frameworks: Spring, Ocelot • SaaS & PaaS: Amazon, Oracle, APIGee • Servers: Nginx, OSS: Tyk, DreamFactory, Kong, Traefik 56
57. Audit • In the box since we have event sourcing • ES for data aggregation 57
58. Integration service • Data aggregation from different microservices • Transfer data from one format to another • Data adaptation • Validation and data integrity • Use the serverless lambdas • SaaS & PaaS: Amazon, Google cloud functions • OSS: OpenLambda, Serverless Framework, Kubeless 58
59. Unique ID • Don’t use auto-increment for ID at all • Don’t generate any ID in the domain services • Use UUIDs • Use sequences • You can your own implementation or any based on Twitter's Snowflake 59
60. DevOps • Full team member • Full time • The right hand of architect 60
61. Contact Me • Alexandr Shcherbakov • Email: alexandr.sherby@gmail.com • https://auriga.com/ 61 Summary • The microservices come to us for a long period • The whole team need to realize the key benefit of the microservices architecture first: flexibility and adaptation • Use best practices like as the event-driven approach, the event sourcing and CQRS, existing services and frameworks • Don’t miss DevOps!