Commit 2018 - A microservices experience in banking

This talk presents a real case of a brand new digital bank based on the principles of delivering maximum value in the least possible time, being centered around customer needs and not sacrificing quality.
A culture of quality, purpose and commitment has been promoted and system, teams and processes have been split into autonomous parts using agile methodologies, DevOps, microservices and automated testing.
A decision log will be exposed during the talk, explaining the reasons behind the decisions and the results.
Then, future actions and plans will be presented.

Transcript

1. 1 A microservices experience in the banking industry MAD · NOV 23-24 · 2018
2. About us David García Gil @davgarcia5 2 Rubén Aguilera @raguilera82 “Senior Technical Consultant at Autentia, developing software solutions for more than 15 years in a wide variety of technologies and platforms” “Tech Coach at Autentia, training several courses about several technologies around Agile, Front, Back and DevOps areas”
3. Services 3 SOFTWARE DEVELOPMENT SUPPORT AGILE FACILITATION CUSTOM SOFTWARE DEVELOPMENT TECHNICAL AUDITS TRAINING PRODUCT DESIGN AND UX
4. ⸺ Context ⸺ Decision log ⸺ The future Contents 4
5. Why another bank? Because of changes in ⸺ Society ⸺ Economics ⸺ Technology ⇒ New regulations: GDPR, PSD2 — Open Banking Context - Environment 5
6. Context - Business Shared vision and objectives ⸺ Produce maximum value to business — Minimize time to market — Reduce costs ⸺ Customer centric, not process centric — Digital omnichannel — Never disrupt service ⸺ Don’t sacrifice quality! 6
7. How to deliver maximum value to customers in the minimum time with expected quality? ⸺ Encourage principles, purpose and mastery ⸺ Split system, teams and process in independent manageable parts ⇒ Autonomy — Every decision must support and improve it Context - Delivery 7
8. Autonomy vs reusability Reusability ⸺ Sharing through components/libraries — Sharing == Coupling ⸺ Important software design principle? — Coupling goes against autonomy — How much value does it add? ‒ Starts to pay off after 3rd reuse (Fred Brooks) ⇒ Relegated in favour of autonomy 8
9. Organizational Functional Technical Deployment Operation Autonomy coverage 9
10. Agile methodologies DevOps Achieving autonomy 10 Microservices Automated testing
11. 7. Microservices architecture 8. Tracing & metrics 9. Technology stack 10. CI/CD pipelines 11. Cloud 12. Kubernetes Decision log 1. Agility 2. DevOps 3. Team organization 4. API first 5. Testing and quality 6. Branching policy 11
12. Decision log - Agility Agile organization ⸺ SAFe, Scrum & Kanban Not all rainbows and unicorns ⸺ Agile coaches & scrum masters — Time suckers vs team assistants — Too many meetings — Team policies enforced 12
13. Decision log - DevOps CI & (ready for) CD ⸺ Containers everywhere ⸺ Fully automated pipelines & deployments Combat tendency to create silos ⸺ Communication, collaboration and participation 13
14. Spotify model ⸺ Squads — Vertical == product/feature — Scrum, 2 week sprints ⸺ Chapters — Horizontal == layer, technology, concern — Kanban ⸺ Psychological safety Decision log - Teams 14
15. Decision log - Teams Fighting capacity/experience/knowledge disparity ⸺ Pair programming — Senior/junior ⸺ Code review & live refactoring sessions ⸺ Technical workshops ⇒ Excellent results 15
16. API Blueprint definition ⇒ Implement afterwards Decision log - API first 16 Contract testing Mock server Human docs
17. Cumbersome ⸺ Examples ⸺ Tooling — Dredd, Drakov, Aglio, custom ⇒ Experimenting with Cucumber + OpenAPI 3.0 Decision log - API first 17
18. Decision log - Testing and quality 18 Golden rules ⸺ Quality is not negotiable ⸺ Software craftsmanship ⸺ Agile manifesto ⸺ Extreme programming ⸺ KISS & YAGNI
19. Decision log - Testing and quality 19 ⸺ Dev techniques — SOLID — TDD — Refactoring ⸺ Quality assurance — SonarQube with security & custom rules ⸺ Testing strategy — Testing pyramid — Working to fix overlaps and gaps
20. Decision log - Branching policy ⸺ Don’t branch! — Master based development FTW! — Feature toggles for unfinished functionality ⸺ Branches are opposed to CI & CD — With branches ⇒ Integration problems hidden for days or weeks — Without branches ⇒ Integration problems revealed quickly 20
21. Decision log - Microservices architecture Starting straight with microservices?!?!?! ⸺ Domain known & understood ⸺ Autonomy is #1 priority ⸺ Aware of the perils (hopefully) ⇒ Not regretting it — It can be done even if some microservices are thrown away along the road 21
22. Decision log - Microservices architecture 22
23. Decision log - Tracing & metrics ⸺ Collection — Custom interceptors, handlers... — Execution context + correlation id (Sleuth) ⸺ Forwarding — K8S daemonset Filebeat & Metricbeat ⸺ Aggregation & usage — ElasticSearch + Kibana ⇒ Very successful 23
24. Decision log - Tech stack ⸺ Spring Boot 2.0 recommended but not enforced — Shared starters and libs created on-demand ⸺ Hybrid and native for mobile — Now moving to native ‒ Same effort & cost than hybrid ‒ Higher quality ⸺ Angular 6 for the web 24
25. Decision log - CI/CD ⸺ GitLab — Source code control — CI/CD pipeline ‒ Own runners, GitLab’s hosted ones are very slow ‒ No Jenkins to avoid another tool to manage ⸺ Nexus — All artifacts repository & Docker registry ⸺ SonarQube — Quality gate 25
26. Decision log - Cloud ⸺ AWS for commodity services — RDS — ElastiCache — Cognito ⸺ EC2 instances for everything else ⇒ Cognito not ok (for us) — Proprietary authn flow with AWS-provided libraries 26
27. Decision log - Kubernetes K8S fully manages runtime ⇒ The new OS ⸺ Config & secrets ⸺ Network & comms — Service discovery — Load balancing — Circuit breaking — Security ⸺ Scaling & availability ⸺ Cloud/infra agnostic 27
28. ⸺ Improved onboarding ⸺ DDD — Clearer boundaries for greater autonomy ⸺ Cucumber for E2E testing, ATDD and CDCT — Unified tooling and less overlap ⸺ Mountebank for mock servers ⸺ Moving from GitLab to BitBucket + Jenkins ⸺ Bank in a box The future 28
29. ⸺ No cloud vendor lock-in — Universal/commodity vendor services — Multi-cloud ready with full IaC ⸺ More container-native — Jib Maven Plugin The future 29
30. CQRS + Event sourcing The future 30
31. ⸺ API governance — No decission yet — Events are part of the API! ‒ Producer vs consumer ‒ Owner ⸺ API specification — REST: OpenAPI 3.0 + ReDoc — Events: AsyncAPI + DocGen The future 31
32. 32 Thanks! MAD · NOV 23-24 · 2018