"Principals of System Integration" by Matthew French

Getting different IT systems to talk to each other doesn't need to be hard, but we often find ways to make it much more difficult than it needs to be.

This talk will discuss some of the common mistakes made when trying to move data between different systems, and ways to avoid them. The focus will be on the most common use case: real time communication in a single direction.

The talk will have examples using Python and RabbitMQ, although the principals we will discuss could apply to many technologies and communication paradigms.

Transcript

1. PRINCIPALS OF SYSTEM INTEGRATION Matthew French (FIS) October 2019
2. About FIS 2
3. What is System Integration? 3 System integration is defined in engineering as the process of bringing together the component sub-systems into one system (an aggregation of subsystems cooperating so that the system is able to deliver the overarching functionality) and ensuring that the subsystems function together as a system… Wikipedia 2019-10 For our purposes: system integration is the process of transferring data between two unrelated computer applications. And for this talk we will only consider the transfer of discrete entities in one direction.
4. How hard can it be? 4 It isn’t hard… …but give it time
5. Unstated requirements of System Integration 5 •Transfer must be reliable •Data must be reliable
6. Peter’s Parcels 6 For this talk we will use a fictitious logistics company called Peter’s Parcels. Their business is delivering actual physical packages, not packets of data. Like most business users, they don’t care about the technology needed to get this done. We are responsible for the application that tracks parcels. The business has many other applications: • The customer web app • The driver web app • The fleet management system
7. Parcel Tracker An ideal world 7 Customer Web Driver Web Fleet Repository RESTful web service
8. 8 ENEMY #1 REALITY
9. Store and Forward How about adding some reliability? 9 Parcel Tracker Customer Web Driver Web Fleet Repository Subscriber
10. 10 ENEMY #2 COMPLEXITY
11. But the data isn’t the same? 11 Store and Forward Parcel Tracker Customer Web Driver Web Fleet Repository Driver Subscriber Fleet Subscriber Customer Subscriber
12. 12 ENEMY #3 CHANGE
13. We now offer parcel insurance… 13 Store and Forward Parcel Tracker Customer Web++ Driver Web Fleet Repository Driver Subscriber Fleet Subscriber Customer Subscriber Insurance: 95.00
14. 14 ENEMY #4 SEMANTICS
15. We’ve gone National 15 Store and Forward Parcel Tracker Customer Web Driver Web Fleet Repository Fleet Subscriber Customer Subscriber Accounting Airline Driver Subscriber Insurance Subscriber Location: -26.1357184,28.0508693 Location: JNB
16. 16 ENEMY #5 PEOPLE
17. Store and Forward W… T… F…?!? 17 Parcel Tracker Driver Web Repository Fleet Subscriber Customer Subscriber Accounting Airline Driver Subscriber Fleet Customer Web
18. Principals of System Integration 18 • Technology is only a small part of the solution • Focus on recovery not availability • Keep it simple, don’t be clever • Source sends everything they can in a format convenient for them • Recipient is responsible for interpreting data, they own the problem • Work with the other teams, not against them • Use a general purpose programming language. Python works well!
19. 19 AND FINALLY: Remember there are no silver bullets. Use common sense and break the rules when you have to.
20. ©2019 FIS and/or its subsidiaries. All Rights Reserved. FIS confidential and proprietary information. matthew.french@fisglobal.com