"Principals of System Integration" by Matthew French

7b0645f018c0bddc8ce3900ccc3ba70c?s=47 Pycon ZA
October 11, 2019

"Principals of System Integration" by Matthew French

Getting different IT systems to talk to each other doesn't need to be hard, but we often find ways to make it much more difficult than it needs to be.

This talk will discuss some of the common mistakes made when trying to move data between different systems, and ways to avoid them. The focus will be on the most common use case: real time communication in a single direction.

The talk will have examples using Python and RabbitMQ, although the principals we will discuss could apply to many technologies and communication paradigms.

7b0645f018c0bddc8ce3900ccc3ba70c?s=128

Pycon ZA

October 11, 2019
Tweet

Transcript

  1. PRINCIPALS OF SYSTEM INTEGRATION Matthew French (FIS) October 2019

  2. About FIS 2

  3. What is System Integration? 3 System integration is defined in

    engineering as the process of bringing together the component sub-systems into one system (an aggregation of subsystems cooperating so that the system is able to deliver the overarching functionality) and ensuring that the subsystems function together as a system… Wikipedia 2019-10 For our purposes: system integration is the process of transferring data between two unrelated computer applications. And for this talk we will only consider the transfer of discrete entities in one direction.
  4. How hard can it be? 4 It isn’t hard… …but

    give it time
  5. Unstated requirements of System Integration 5 •Transfer must be reliable

    •Data must be reliable
  6. Peter’s Parcels 6 For this talk we will use a

    fictitious logistics company called Peter’s Parcels. Their business is delivering actual physical packages, not packets of data. Like most business users, they don’t care about the technology needed to get this done. We are responsible for the application that tracks parcels. The business has many other applications: • The customer web app • The driver web app • The fleet management system
  7. Parcel Tracker An ideal world 7 Customer Web Driver Web

    Fleet Repository RESTful web service
  8. 8 ENEMY #1 REALITY

  9. Store and Forward How about adding some reliability? 9 Parcel

    Tracker Customer Web Driver Web Fleet Repository Subscriber
  10. 10 ENEMY #2 COMPLEXITY

  11. But the data isn’t the same? 11 Store and Forward

    Parcel Tracker Customer Web Driver Web Fleet Repository Driver Subscriber Fleet Subscriber Customer Subscriber
  12. 12 ENEMY #3 CHANGE

  13. We now offer parcel insurance… 13 Store and Forward Parcel

    Tracker Customer Web++ Driver Web Fleet Repository Driver Subscriber Fleet Subscriber Customer Subscriber Insurance: 95.00
  14. 14 ENEMY #4 SEMANTICS

  15. We’ve gone National 15 Store and Forward Parcel Tracker Customer

    Web Driver Web Fleet Repository Fleet Subscriber Customer Subscriber Accounting Airline Driver Subscriber Insurance Subscriber Location: -26.1357184,28.0508693 Location: JNB
  16. 16 ENEMY #5 PEOPLE

  17. Store and Forward W… T… F…?!? 17 Parcel Tracker Driver

    Web Repository Fleet Subscriber Customer Subscriber Accounting Airline Driver Subscriber Fleet Customer Web
  18. Principals of System Integration 18 • Technology is only a

    small part of the solution • Focus on recovery not availability • Keep it simple, don’t be clever • Source sends everything they can in a format convenient for them • Recipient is responsible for interpreting data, they own the problem • Work with the other teams, not against them • Use a general purpose programming language. Python works well!
  19. 19 AND FINALLY: Remember there are no silver bullets. Use

    common sense and break the rules when you have to.
  20. ©2019 FIS and/or its subsidiaries. All Rights Reserved. FIS confidential

    and proprietary information. matthew.french@fisglobal.com