Mainframe Modernization: From COBOL to Event Sourcing

Mainframe Modernization: From COBOL to Event Sourcing

This talk will explore modern programming techniques such as reactive programming, event storming, and domain-driven design to demonstrate how leading organizations in banking are modernizing their legacy systems -- including COBOL! -- to solve scaling, cost, and business continuity issues such as recruiting. We'll cover a mix of real-world use cases and our collective experience to help practitioners and technology leaders get started.

Anyone interested in reactive programming or how banks *actually work* should get a lot out of this talk. There are a host of business opportunities waiting to be unleashed through the power of reactive programming! For the most part, companies have stopped short of full modernization: bringing book of record systems into a modern era of technology. Meanwhile, customer adoption of these new platforms has led to an explosion of traffic to underlying legacy systems such as mainframes, resulting in huge cost increases, degraded performance and sub-par customer experiences.

We'll showcase the power of modern technologies such as Play, Akka, and Kafka, and how they are being used to break the chains of the past. We will demonstrate techniques such as event sourcing and CQRS for improving latency from mobile to mainframe from 10+ seconds to sub-second. In this always-on, always-connected era, consumers expect real-time insights. We'll show you a path towards real-time.

665a7ca82af87606f4fc83b3d94b5fd5?s=128

Kevin Webber

May 10, 2018
Tweet

Transcript

  1. 2.

    Topics Part 1: An Introduction to Legacy Systems Part 2:

    The Old World and New World Collide Part 3: A Path to Full Modernization Part 4: The Read Only Bank Part 5: Conclusion
  2. 3.
  3. 4.

    Part 1 An Introduction to Legacy Systems Part 1: An

    introduction to legacy systems Part 2: The old world and new world collide Part 3: A path to full modernization Part 4: The Read Only Bank Part 5: Conclusion
  4. 6.
  5. 8.
  6. 9.
  7. 10.
  8. 11.
  9. 12.
  10. 15.
  11. 17.

    Customer expectations As the market evolves, customer expectations go up

    They expect higher quality data that is up to date Customers want an instant view of their world Customers compare to all apps on their phone
  12. 18.

    All companies are technology companies The Wealthsimple effect Robo advisors

    were the first wave of competition Amazon launching chequing accounts The second wave of competition will be fierce Uber is a bank? Uber is one of the largest originators of small business loans in the U.S
  13. 19.

    2020 and beyond? Core mainframe systems struggling to adapt to

    a mobile-first, always connected world Legacy systems designed decades ago Near-real time, API-driven usage was unanticipated How long is this sustainable?
  14. 20.

    Ch-ch-changes What happens when a core system needs to change

    with urgency? Regulators are forcing improvements in many areas ◦ PSD2 Data Requirements (Open Banking) ◦ Real Time Payments ◦ GDPR
  15. 23.

    Limited size for transaction descriptions Means they are ı̖͇̳̰̰͍̲̻͕̩͗̒̊̈́̈́̀͆ ̃̀͘ň̢

    ̤͔̙͙̖̙͙͍͉͛̈͋͑̄͒ ̃ ͗̚ ç̧ ̲͍̯̝̻͉͎̦͛͛̒̀̀̎̍̌ ̕ ͠ ǒ̧ ̙̞̭̺̘͈͇̤̀̐̑͗́̃ ͂͂͂͜ m̨ ̩̙̝̹̥̟͚̼̟͌͆̓̏̈́̆͐ ͘̕͝p̛̛̠̲̙̰͚̥͙̳͈̦͐̇̑̈̊̈́̉͠ r̡ ̼͎͕̗͈͍̘̗͚̋͌̈́͋̌̒̓́̈̚ę̢ ̩̜̱̖͍̮̜͖̿̿́̑̓̉̓̀̋̚ ḥ ͇̳̟̪͖̜̙̳̙́͊̆̂͊̈̎̿̓͠ e ̖̙̪͖̹̫͇̱͉̹͆́̽͂̍͑̓̍͝͝ n ̰̞̳̭̭͙̮̥̤̖͌͂̌̂̀̈̍͛͒͠ s̢ ̨̛͖̗̰̤͚̞̙͙̆̆͐̆̈́̃ ̂͆̈́ï̧̟̜̪̩̠̜̥͖̥̂̒̏̂̅̀͑̿ ͘b̧ ̫̙̘̥̯̖̞͚͕̉̃ ͋̀̀̀̎͌ ͘͘l̢̨̢͚͉̜͕̪̙͉͑͐̌̓̆̽͂̂̆̀ë̛ ̟̯ ̣ ̙̫̰̼̭̫͖́̌̎̑̈̅̾̅͝ Technical limitations
  16. 24.

    Technical limitations Deposit systems are running out of account numbers

    Accounts have balance and transaction size limits due to lack of integer space Support for only one currency
  17. 27.

    Batch Risk Batch based systems have failure modes that can

    quickly cascade and are quite fragile Due to tight time windows, failures can take days to rectify Despite the jobs being rock solid, they are only as good as the scheduling system
  18. 28.

    Getting Already very expensive Mainframes, and most software running on

    them (OS, databases, etc), are priced on capacity measured in MIPS Testing requires an army of people, and the expense of the underlying platform limits the number of testing environments
  19. 29.

    Getting Already very expensive The business have large back office

    teams who perform tasks solely to overcome system limitations
  20. 30.

    Recap of goals ➢ Improve customer experience ➢ Create a

    path for recruiting and retaining top talent ➢ Reduce operational costs ➢ Reduce risk ➢ Increase responsiveness to business needs ➢ What’s in our way?
  21. 31.

    Part 2 The Old World and New World Collide Part

    1: An introduction to legacy systems Part 2: The old world and new world collide Part 3: A path to full modernization Part 4: The Read Only Bank Part 5: Conclusion
  22. 32.

    COBOL: The Good Parts COBOL and JCL ◦ Immutable datasets

    written to disk, read by next step ◦ Can handle massive throughput ◦ Resilient, can be rerun in the event of failure (e.g, bad data) ◦ Referential transparency What do mainframe systems sound similar to?
  23. 33.

    Now is the time for full modernization Modern tools and

    techniques are ready for mainframe workloads
  24. 34.

    Steps Step 1: Understand current state Step 2: Model future

    state Step 3: Isolate legacy components Step 4: API enablement Step 5: Conclusion
  25. 35.

    Tactic 1: Understand current state Event storming: Model interesting business

    processes in your organization Use a well defined set of operations (commands, events)
  26. 37.

    Tactic 2: Model future state Domain Driven Design: ◦ Domains

    ◦ Subdomains ◦ Cores ◦ Ubiquitous language
  27. 38.

    Domains and events - Domain driven design helps us identify

    boundaries - Event sourcing helps us define interactions via aggregate events and domain events
  28. 39.

    Tactic 3: Isolate Ensure responsiveness to change: ◦ Isolate business

    functionality within well defined domains ◦ Model interaction between domains with domain events ◦ Unbounded complexity is what prevents change!
  29. 41.

    Tactic 4: Enable Microservices and gateways ◦ Expose data and

    functionality through service interfaces ◦ REST, pub/sub, or streams: anything with a contract ◦ Bezos’ tenets ▪ No direct linking ▪ No direct reads of another team’s data store ▪ No shared-memory model ▪ No backdoors whatsoever
  30. 42.

    CQRS and Event Sourcing - CQRS within a single domain

    - Commands, events, queries, and views form the contract - No relational database required
  31. 43.

    Tactic 5: Modernize Slowly replace the old with the new:

    ◦ Stabilize legacy integrations ◦ Replace legacy where it brings value ◦ Leave legacy where replacement costs outweighs benefits
  32. 44.

    ➢ Named after the stranger vine that grows up and

    around trees, effectively “replacing” them ➢ We can use a similar pattern to replace functionality in a legacy system with new functionality ➢ Keeps the tree intact until completely replaced “Strangler Pattern”: Decompose without destroying!
  33. 45.

    Decompose without destroying! ◦ The legacy system is never modified

    ◦ Transaction scripts can be thousands of lines long and have many side effects ◦ Worst case scenario: revert to an unmodified, functioning legacy system
  34. 46.
  35. 47.
  36. 48.
  37. 50.

    Part 3 A Path to Full Modernization Part 1: An

    introduction to legacy systems Part 2: The old world and new world collide Part 3: A path to full modernization Part 4: The Read Only Bank Part 5: Conclusion
  38. 51.

    Commonwealth Bank of Australia Core Banking Replacement $1.3Bn core banking

    replacement system and still replacing components ”They have been developing it for 18 months and they are still adding modules.” ”The upgrade started in 2007 and was finished by April of this year [2013] , with 1500 people working on the project full time for six years.”
  39. 54.
  40. 57.
  41. 58.
  42. 59.

    Part 4 The Read Only Bank Part 1: An introduction

    to legacy systems Part 2: The old world and new world collide Part 3: A path to full modernization Part 4: The Read Only Bank Part 5: Conclusion
  43. 60.
  44. 68.

    Part 5 Conclusion Part 1: An introduction to legacy systems

    Part 2: The old world and new world collide Part 3: A path to full modernization Part 4: The Read Only Bank Part 5: Conclusion
  45. 69.

    Conclusion Current core systems are under immense strain with an

    explosion in traffic and heightened customer expectations The implementation is new but the concepts are proven Urgency to act - regulators, competitors and talent There is a path forward to break the chains of the past
  46. 70.

    Let’s chat! Kevin Webber, RedElastic Principal Consultant kevin.webber@redelastic.com Dominic Wallace,

    CGI Director, Emerging Technology Engineering dominic.wallace@cgi.com Adrian Maurer, CGI Director, Emerging Technology Engineering adriano.maurer@cgi.com