Microservices as an evolutionary architecture: lessons learned

Over the years the architecture of microservices has been widely adopted, since it provides numerous advantages such as: technological heterogeneity, scalability, decoupling and so on.

In this sense the microservices architecture meets the definitions of an evolutionary architecture, that is, an architecture designed for incremental changes even changes of languages.

In this lecture, we will discuss the decisions to adopt frameworks and techniques such as: Spring, Vert.x, gRPC, Event-driven Architecture in an architecture for a payment medium solution in which throughput and response time is crucial for the survival of the business .

Transcript

1. Microservices as an Evolutionary Architecture: Lessons learned Claudio Eduardo de Oliveira & Luram Archanjo
2. Who am I? • Software Engineer @Sensedia • MBA in Java projects • Java and Microservice enthusiast
3. Who am I? • Lead Solutions Architect @Sensedia • Book Author • Spring, Java and Cloud Native enthusiast
4. Agenda • Use Case & As Is • Microservices • Evolutionary Architecture • Domain Driven Design • Communication Patterns & Reactive Programming • Observability & Database Migration • Questions
5. Use case
6. Use case Token Service Provider Tokenization reduces the value of stored payment credentials by removing consumers’ primary account numbers (PANs) from the transaction process. It does this by replacing them with a unique identifier called a payment token. This reduces the appeal of stealing the credentials as they would be largely useless to hackers. Source: https://www.rambus.com/blogs/what-is-a-token-service-provider/
7. Use case
10. In the beginning
11. As is • Single communication pattern • 50-60% of our commits during the sprint were in the same repository • Service with different workload requirements (Crypto and Business) • Traffic east-west in the API Gateway • Java & Spring everywhere • Synchronous process everywhere
12. We were in the wrong way to adopt microservices architecture!
13. Let's recap what is the microservices architecture!
14. Moving to Microservices Feature A Feature B Feature C Monolith Microservice Microservice Microservices Microservice
15. Technological Heterogeneity Microservice Microservice Microservice DB DB DB
16. Evolutionary Architecture
17. Evolutionary Architecture An evolutionary architecture supports guided, incremental change as a first principle across multiple dimensions. Source: https://www.thoughtworks.com/books/building-evolutionary-architectures
18. Response time Microservices as an Evolutionary Architecture: Lessons learned Throughput
19. Before make a architectural decision, we need to be confident! How?
20. Regression Tests
21. Regression Tests REGRESSION TEST SUITE REGRESSION SCRIPTS AUTOMATED REGRESSION TESTING NEW FEATURES SCRIPT UPDATES
22. We have tests coverage and now?
23. Domain Driven Design (DDD)
24. Domain Driven Design(DDD) Domain-driven design (DDD) is an approach to developing software for complex needs by deeply connecting the implementation to an evolving model of the core business concepts. • Place the project’s primary focus on the core domain and domain logic • Base complex designs on a model • Initiate a creative collaboration between technical and domain experts to iteratively cut ever closer to the conceptual heart of the problem. Source: https://dddcommunity.org/learning-ddd/what_is_ddd/
25. We are trying to figure out the technique to divide services, what part of DDD can help us?
26. Domain Driven Design - Bounded Context Bounded Context is a central pattern in Domain-Driven Design. It is the focus of DDD's strategic design section which is all about dealing with large models and teams. DDD deals with large models by dividing them into different Bounded Contexts and being explicit about their interrelationships. Source: https://martinfowler.com/bliki/BoundedContext.html
27. We separated the responsibilities and now?
28. Communication Patterns
29. REST API REST API, EVERYWHERE
30. Advanced Message Queuing Protocol (AMQP)
31. Advanced Message Queuing Protocol (AMQP) • Introducing Message-Driven Programming with Low Level Events • Fire & Forget principle • Durable Messages (avoid Circuit Breakers & Retries) • Better Scale strategies & Application Decoupling • Fits well for 3rd partners integrations
32. Advanced Message Queuing Protocol (AMQP) - Thoughts • Keep track of events otherwise you will have problems • We used retries to put message in the queue • Take care of the data into the message (security concerns)
33. HTTP / gRPC
34. HTTP / gRPC • HTTP Connections are stateful (avoid open & close) • Static Typed & Well Defined Contracts • Reduce serialization complexity & improve efficiency • Easy Peasy to integrate (there is code generation based in .proto) • Traffic is binary reduce bandwidth usage
35. HTTP / gRPC - Thoughts • Load Balancing and Client Load Balancing • We used in the high throughput solution • By definition in general we use for internal communication (a.k.a East-West) • Not for humans, designed by machine communications
36. HTTP / REST
37. HTTP / REST • Well documented Style • “De-Facto” Pattern for Microservices Architecture • “Toolability” • Security
38. HTTP / REST - Thoughts • Well documented Style • For Humans, fits very well for External APIs ( API Strategy ) • API Management provided by Sensedia offers interesting features to troubleshoot
39. We have tests, responsibilities and communication! It is possible to improve more?
40. Reactive Programming
41. Reactive Programming Reactive programming is an asynchronous programming paradigm concerned with data streams and the propagation of change. • Easier to read, maintain and evolute • Low memory footprint • Scalability Source: https://en.wikipedia.org/wiki/Reactive_programming
42. Reactive Programming Source: https://craftsmen.nl/memory-usage-6-popular-rest-server-frameworks-compared/
43. Language
44. Language Kotlin is fully compatible with all Java-based frameworks, which lets you stay on your familiar technology stack while reaping the benefits of a more modern language. • 40% less code = Easier to read, maintain and evolute • Fail fast principle = Time to Market Source: https://kotlinlang.org/docs/reference/server-overview.html
45. Everything have been possible, because we formed a great Team Mindset
46. Team Mindset In a fixed mindset, people believe their basic qualities, like their intelligence or talent, are simply fixed traits. Contrarily, in a growth mindset, people believe that their most basic abilities can be developed through dedication and hard work. Source: https://mindsetonline.com/whatisit/about/
47. Team Mindset - Thoughts • Hands-on • Tech talks • Lightning talks • Events ◦ The Developer’s Conference (TDC) ◦ APIX ◦ Meetups
48. We achieved the challenge! And now?
50. sensedia.com Monitoring & Observability
51. Monitoring is the practice of collecting signals, telemetry, traces, etc and aggregating them, and matching them against some pre-defined criteria of system states we should carefully watch. When we find that one of our signals has crossed a threshold and may be heading toward a known bad state, we take action to remedy the system - Christian Posta
52. Monitoring is a subset of Observability
53. Observability on the other hand supposes up front that our systems are highly unpredictable and we cannot know all of the possible failure modes up front
54. We need to collect much more data, even high-cardinality data like userIDs, requestIDs, source IPs, etc where the entire set could be exponentially large
55. With observability, we need fine-grained data
56. A user goes to pay for the items in their cart and experiences a 10s delay choosing a payment option. All of the pre-defined metric thresholds (disk usage, queue depth, machine health, etc) might be at acceptable levels
57. sensedia.com
58. Database Migration
59. Database Migration SQL databases scale vertically, meaning you’ll need to increase the capacity of a single server (increasing CPU, RAM, or SSD) to scale your database. NoSQL databases scale horizontally, meaning you can add more servers to power your growing database. Source: https://www.thorntech.com/2019/03/sql-vs-nosql/
60. Database Migration Microservice DB DB Synchronous, legacy Asynchronous, new Event
61. Before & After
62. Before
63. After
64. Summary 2º Place 1º Place 3º Place Last responsible moment • Decisions • Experience • Maturity Team mindset • Commitment • Opportunity to grow and learn • Feedbacks Technologies • Understand the technologies concepts • Business value
65. Thanks a million! Questions? /larchanjo /luram-archanjo /claudioed /claudioed