Experiences on building a modular reactive architecture CIR-BCC 

About Me @anestodta [email protected] https://www.linkedin.com/in/anesto-del-toro • Head of Production, DATYS VC • MSc Computer Sciences • Wearing multiple hats oTechnical Lead oArchitecture oSoftware Development (mostly backend) oProject Management oBusiness Analysis oLean-Agile enthusiast oEager learner • 17 years of experience o~6 Researcher at UCLV (BE, PR & CV) Anesto del Toro Almenares 

This talk and Devops… 

•CIR-BCC: Scope •Building the solution • Challenges • Architecture/design decisions • Dev/IT support services •Conclusions Outline 

SCOPE 

No content

What is the Risk Information Center (CIR)? Free public service offered by the Superintendence of the Central Bank of Cuba (BCC). Directed to the National Banking and Financial System and to legal and natural persons who have credit from the country's financial institutions and operate current account. 

What is the objective? Provide information to: -financial institutions, in order to contribute to the mitigation of their risks -debtors about their credit situation -clients on the measures applied in their accounts for breaches of the banking rules. 

Software Delivery System Achieve faster development and deployment cycles of quality products, in a reliable and sustainable way Enterprise Agility 

Communication matters… 

(c4 diagrams) 

•What are we building? •Who is using it? (users, actors, roles, personas, etc.) •How does it fit to the existing IT environment? Context diagram 

Risk Information Center - BCC Supervisor Financial Institution Legal person Admin [sign-in user + psw] Users Accesses Apps authorizations Legislations Natural person Credit info Operational info Exposure Level Risk concentration Credit info Operational info Credit info Operational info Credit info Operational info Ext. App 1 … Ext App N [sign-in AppId + token] Analysis requests SIB (legacy system) 

SIB (legacy system) Risk Information Center - BCC Supervisor Financial Institution Legal person Admin [sign-in user + psw] Users Accesses Apps authorizations Legislations Natural person Credit info Operational info Exposure Level Risk concentration Credit info Operational info Credit info Operational info Credit info Operational info Ext. App 1 … Ext App N [sign-in AppId + token] Analysis requests NTP SMTP Monitoring Register events of interest Email responses Time synch 

What are the high-level technology decisions? 

Challenges 

CONCURRENT ACCESS 

DEPLOYMENT FLEXIBILITY 

LEGACY SYSTEM INTEGRATION Data lifecycle 

Decoupling frontend and backend teams 

Frontend WebApp Backend Services • Non blocking / Async • Event Sourcing (ES) • Command Query Responsibility Segregation (CQRS) • SPA Stack Clear API contract 

STACK SELECTION 

Frontend stack 

Backend stack 

Backend stack • Seamless JAVA interop • Type inference • Concurrency and distribution • Traits • Pattern matching • High-order functions • Design patterns 

Backend stack • MVC Web development framework • Stateless, asynchronous and non- blocking • Deliver all the benefits of RS, including high degree of responsiveness, elasticity and resiliency • Hot reloading, interoperability with DI frameworks, modularity and extensibility,.. 

Backend stack Toolkit for building highly concurrent, distributed, and resilient message-driven applications for Java and Scala • Simpler Concurrent & Distributed Systems • Resilient by Design • High Performance • Elastic & Decentralized • Reactive Streaming Data 

Supporting services • CVS • Dependencies repository • Docker repository • Continuous Integration Automation scripts • build.sbt • Testing • Code metrics, Check style • gitlab-ci.yml 

No content

- Reactive stores - Kafka (durable message bus…) - net.cakesolutions.scala-kafka-client - net.cakesolutions.scala-kafka-client-akka Storage services - Reactive DB drivers and tools (Scala) - reactive-mongodb - rediscala - play-ws 

docker-compose.yml Sandboxing 

Modularity & Composability 

Modularity 

build.sbt mainApp [Play] JWT Auth, Users Management, Credit and Operational Analysis Modularity 

mainApp/application.conf Modularity 

d2AppCOInfo [Play] Detailed Credit and Operational Info d2AppCore [Play] Authentication (JWT), User management, Customers, Roles, 3rd party Apps, d2AppRCEL [Play] Exposure Level Risk Concentration Gross Loss Stockholders' Equity API Gateway [NGinX ] build.sbt Modularity 

.../d2AppRCEL/application.conf .../d2AppCore/application.conf ... Modularity 

Messaging System [Kafka] NoSQL Data Store [MongoDB 3.4.x] Cache Data Store [Redis 3.2.x] d2AppCOInfo [Play] Detailed Credit and Operational Info d2AppCore [Play] Authentication (JWT), User management, Customers, Roles, 3rd party Apps, d2AppRCEL [Play] Exposure Level Risk Concentration Gross Loss Stockholders' Equity API Gateway [NGinX ] Modularity 

Messaging System [Kafka] NoSQL Data Store [MongoDB 3.4.x] Cache Data Store [Redis 3.2.x] d2AppCOInfo [Play] Detailed Credit and Operational Info d2AppCore [Play] Authentication (JWT), User management, Customers, Roles, 3rd party Apps, d2AppRCEL [Play] Exposure Level Risk Concentration Gross Loss Stockholders' Equity API Gateway [NGinX ] NoSQL Data Store [MongoDB 3.4.x] Cache Data Store [Redis 3.2.x] NoSQL Data Store [MongoDB 3.4.x] Cache Data Store [Redis 3.2.x] Modularity 

Deploy project 

•Modularity / App Composability •Horizontal scalability •Deployment flexibility Different scalability levels per App, including data services ad-hoc, incremental Small apps working coordinately (Bounded Contexts) Apps evolving in parallel (different teams) Benefits 

The ability to behave correctly under load and to scale on demand can not be a last time though 

Legacy system integration 

SIB (legacy system) Ingestion Aggregation Query/Retrieval Event Journal Snapshots Topics subscription Topics publisher Read Model 

read model aggr-apps SIB Credit Info AggApp Risk Concentration AggApp Exposure Level AggApp M300Event SIB Ingester M301Event M910Event M918Event MCMEvent … M920Event topics web-app mainApp [Play] JWT Auth, Users Management, Credit and Operational Analysis 

M300Event SIB Ingester M301Event M910Event M918Event MCMEvent … M920Event SIB topics Risk Concentration AggApp Credit Info AggApp Exposure Level AggApp aggr-apps read model web-app d2AppRCEL [Play] Exposure Level Risk Concentration Gross Loss Stockholders' Equity d2AppCOInfo [Play] Detailed Credit and Operational Info 

- Event Sourcing (ES) ES-CQRS - Command Query Responsibility Segregation (CQRS) 

"com.hootsuite" %% "akka-persistence-redis" % 0.6.0 "com.github.scullxbones" %% "akka-persistence-mongo-rxmongo" % 1.4.1 Persistent Actor 

SIB (legacy system) Ingestion Aggregation Query/Retrieval Data lifecycle Change Data Capture (CDC) to the rescue… Extracting any inserts, updates, and deletes into a stream of data change events 

• What is the shape / general appearance of the system? • How the main responsibilities are distributed in the system? • How do components communicate? Containers diagram 

[https, WS] SIB [External System] Web browser Frontend / Proxy [NGinX ] [https] Risk Information Center - BCC 

[https, WS] SIB [External System] Web Application [Play, Stateless] JWT Auth, Management of Users & Legislations, Credit and Operational Analysis Rest Services. UI (SPA) [NGinX] API Gateway [NGinX ] Authentication (JWT), Request throttling, Load balancing. Web browser Frontend / Proxy [NGinX ] [https] 

[https, WS] SIB [External System] Web Application [Play, Stateless] JWT Auth, Management of Users & Legislations, Credit and Operational Analysis Rest Services. UI (SPA) [NGinX] NoSQL Data Store [MongoDB 3.4.x] Store CI preaggregated docs (CQRS) Cache Data Store [Redis 3.2.x] Caches IFs, Entities, NE, CB, CC, pre- computes, etc. API Gateway [NGinX ] Authentication (JWT), Request throttling, Load balancing. Web browser Frontend / Proxy [NGinX ] [https] 

[https, WS] SIB [External System] Content Updater [Standalone, Akka] ES, Updates Models, Customers, Financial Institutions, etc. Messaging System [Kafka] Durable message bus, integration, etc. Publish messages [Avro] Web Application [Play, Stateless] JWT Auth, Management of Users & Legislations, Credit and Operational Analysis Rest Services. UI (SPA) [NGinX] Reads and write messages [Avro] NoSQL Data Store [MongoDB 3.4.x] Store CI preaggregated docs (CQRS) Cache Data Store [Redis 3.2.x] Caches IFs, Entities, NE, CB, CC, pre- computes, etc. API Gateway [NGinX ] Authentication (JWT), Request throttling, Load balancing. Web browser Frontend / Proxy [NGinX ] [https] 

[https, WS] SIB [External System] Content Updater [Standalone, Akka] ES, Updates Models, Customers, Financial Institutions, etc. Messaging System [Kafka] Durable message bus, integration, etc. Publish messages [Avro] Web Application [Play, Stateless] JWT Auth, Management of Users & Legislations, Credit and Operational Analysis Rest Services. UI (SPA) [NGinX] Reads and write messages [Avro] NoSQL Data Store [MongoDB 3.4.x] Store CI preaggregated docs (CQRS) Cache Data Store [Redis 3.2.x] Caches IFs, Entities, NE, CB, CC, pre- computes, etc. API Gateway [NGinX ] Authentication (JWT), Request throttling, Load balancing. Web browser Changes Tracking [SQLServer feature] Track in SQL Server changes to records. Frontend / Proxy [NGinX ] [https] 

[https, WS] SIB [External System] Content Updater [Standalone, Akka] ES, Updates Models, Customers, Financial Institutions, etc. Messaging System [Kafka] Durable message bus, integration, etc. Publish messages [Avro] Web Application [Play, Stateless] JWT Auth, Management of Users & Legislations, Credit and Operational Analysis Rest Services. UI (SPA) [NGinX] Reads and write messages [Avro] NoSQL Data Store [MongoDB 3.4.x] Store CI preaggregated docs (CQRS) Cache Data Store [Redis 3.2.x] Caches IFs, Entities, NE, CB, CC, pre- computes, etc. API Gateway [NGinX ] Authentication (JWT), Request throttling, Load balancing. Web browser Changes Tracking [SQLServer feature] Track in SQL Server changes to records. Frontend / Proxy [NGinX ] [https] Schema Registry 

[https, WS] SIB [External System] Content Updater [Standalone, Akka] ES, Updates Models, Customers, Financial Institutions, etc. Messaging System [Kafka] Durable message bus, integration, etc. Publish messages [Avro] Web Application [Play, Stateless] JWT Auth, Management of Users & Legislations, Credit and Operational Analysis Rest Services. UI (SPA) [NGinX] Reads and write messages [Avro] NoSQL Data Store [MongoDB 3.4.x] Store CI preaggregated docs (CQRS) Cache Data Store [Redis 3.2.x] Caches IFs, Entities, NE, CB, CC, pre- computes, etc. API Gateway [NGinX ] Authentication (JWT), Request throttling, Load balancing. Web browser Changes Tracking [SQLServer feature] Track in SQL Server changes to records. Frontend / Proxy [NGinX ] [https] Schema Registry vcMon vcAlert [External System] [External System] Gets metrics from [Http] Metrics Collector [cAdvisor] Gathers metrics from Hosts and Containers Metrics Engine [Prometheus] Stores and serves aggregated metrics and emits alerts 

What do we need? 

Production Staging Security / NFR Minutes Weeks Each cycle targets a different concern Feedback loops UAT Increment deploy Code Reviews Integration tests Acceptance tests Unit Testing 

Infrastructure supporting services 

Bring Ops tools closer to Developers 

Load and Performance tests automation 

UAT Deploy Production like Deploy Unit tests Integration tests Code & Vulnerabilities Analysis Acceptance tests Load, Smoke, Security tests … Exploratory Production deploy Deployment pipeline Up to 8 deploys/releases* per day 

Increment Review Increment Review User Acceptance Testing User Acceptance Testing Documentation Documentation Development & Operations Development & Operations Management & Planning Management & Planning Knowledge sharing/retention Knowledge sharing/retention Automation Self-services Automation Self-services Code repository Code repository Platform as a Service Platform as a Service Artifacts and dependencies repositories Artifacts and dependencies repositories Stakeholders Stakeholders Internet Production System 

No content

No content

• Communication/collaboration amplified • Shared mindset/vision • Automation of supporting services • Results are not the solely point, but the development of People and Systems that allows us to achieve the results we expect Conclusions 

[email protected] @anestodta https://www.linkedin.com/in/anesto-del-toro Anesto del Toro Almenares Experiences on building a modular reactive architecture