Building distributed applications with NServiceBus Software Architect at ServiziCGN @manuelscapolan Manuel Scapolan ServiziCGN – a case study 

Servizi CGN • Fiscal and tax advice • E-learning and class learning • Dedicated software to fill tax return statements and to run the companies accounting Leader company in offering services to accountants 

Servizi CGN • More than 1.260.000 tax return statements sent only this year • Thanks to a network of 35.000 professionals that use its software free to fill and send the tax return statements of their clients 1° CAF in Italy as private business 

“Status quo” • Many applications to be integrated (> 20) • Different platforms (COBOL, .NET) • Different architectures (client-server, n-tier, …) • Different technologies • Many databases (semantic duplication) 

Where we live • High number of requests in a short interval of time (data-entry activity) • High number of concurrent users • Short and long running operations 

Where we started Billing/Accou nting system Tax return applications Management system Revenue agency Two-way channel uses custom protocol built over TCP/IP (on-demand communication started by the client) Standalone and off-line applications built on cobol platform Data-sharing between file transfer 

Synchronize data Challenge #1 

Synchronize data • Every system has its data stores to store customer related information • When a customer is created (or updated) in one system all the others need to change their copy of data Data synchronization 

Solving at the database level • ETL process (run nightly) using bulk data modification techniques – Pro: • High control over data manipulation • Easy to load huge amounts of data – Cons: • Strong dependence on data stores availability • The data is not updated in real-time 

Asynchronous messaging • One application publishes a message (with the updated information) to a common channel • Other systems can read the message from the channel at a later time • The systems must agree on a channel as well as on the format of the message • The communication is asynchronous 

Benefits of async messaging • No temporal coupling, not require both systems to be up and ready at the same time • “Real-time” data synchronization • Reliability of store-and-forward • “Fine-grained” log out of the box 

What we need • A Message Bus: NServiceBus • A transport: MSMQ • Producer/consumer components: endpoint • And obviously, a message: event end-point 

NServiceBus • The most developer-friendly service bus for .NET • Supporting one-way, full-duplex, publish/subscribe patterns • Easy to extend and configure • Long running processes definition and management capabilities using Sagas • Out of the box transports (MSMQ, RabbitMQ, SQL Server, Windows Azure Queues,Windows Azure Service Bus) • Easy to integrate with existing system 

MSMQ • Max 4MB message size • Persists messages to file • Supports transactional queues • Supports store and forward 

Message endpoint • A client of the messaging system that the application can then use to send or receive messages • NServiceBus.Host hosted in a Windows Service 

Message-based architecture • Application development model: Front-end (Composite UI) Web application / API gateway CommandHandlers EventHandlers Message bus 

Communication mechanisms • RESTful API – In the ApiController of the API gateway the call is translated to a command which is sent to the commandhandlers endpoint synchronously • Messages (Commands or Events) – Asynchronous communication 

Anti-Corruption Layer (ACL) • Layer built to integrate the bounded context with an existing system where we don’t have control (i.e. legacy or external system) • Translate conceptual objects or actions from one model to another • It can be developed as a façade or an adapter Legacy Application Message- based system 

Communicating with the client • The ACL is a file adapter that picks a file from a file system and turn into a message to deliver onto a channel and vice versa Message bus Membership Service ACL CreateUser() UserCreated UserCreated Client applications Management system 

Integrate applications Challenge #2 

Integrate applications • The tax payment application (F24 model in Italy) is “interested” to the tax amount • The tax amount is a calculated value based on the data of the tax return statement Tax payment service Tax return statement applications Tax return statement applications Revenue agency amount total 

Publish/subscribe • The tax payment application subscribes to the event TaxAmountChanged • When in a tax return statement application a tax amount is changed a new event of type TaxAmountChanged is published to the message bus • The handler in the tax payment service endpoint receives the message and updates the total amount 

Scalability Challenge #3 

Scalability • The life of the accountant will often be on the cutting edge • The unavailability of the software or a high latency in those moments can bring serious harm to our clients • Not all the operations are equals… 

Scaling out with NServiceBus • In our case the print of a tax return statement is a good candidate to scale out Frontend / web application CommandHandlers endpoint Print Queue NServiceBus Host #1 NServiceBus Host #2 Competing consumer 

Long-running “transactions” Challenge #4 

Long-running “transactions” • Avoid locks on non-local resources • Use compensation to handle failures • Potentially aggregate smaller ACID transactions (also referred to as atomic transactions) • Typically use a coordinator to complete or abort the transaction 

Long-running “transactions” • There are a few different types of statements that must be delivered together • We are in a “all-or-none” situation Revenue agency 

Long-running “transactions” • Implemented with an NServiceBus saga 

Lessons Learned 

Lessons learned • Thinking asynchronously • You can’t have functional scalability without bounded contexts • For integration prefer asynchronous messaging over synchronous http calls (loose coupling) • Keep handlers small and idempotent (single responsability principle) 

To learn more about Bounded context, Messaging system, DevOps,… 

Resources • NServiceBus introduction to a message based distributed architecture [http://www.slideshare.net/mauroservienti/nservicebus-introduction-to-a-message-based-distributed-architecture?related=6] • Message Oriented Architecture using NServiceBus [http://www.slideshare.net/kindblad/2013-november-message-oriented-architecture-using-n-servicebus?related=1] • Distributed Application Development with NServiceBus [http://www.slideshare.net/davidboike/distributed-application-development-with-nservicebus] • Introduction to NServiceBus [http://www.slideshare.net/danielmarbach50/introduction-to-nservicebus-14752472?related=1] • Slide5 [https://www.flickr.com/photos/archi-bald/3780921448/] 

Thank you @manuelscapolan