Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mainframe Modernization: From COBOL to Event Sourcing

Mainframe Modernization: From COBOL to Event Sourcing

This talk will explore modern programming techniques such as reactive programming, event storming, and domain-driven design to demonstrate how leading organizations in banking are modernizing their legacy systems -- including COBOL! -- to solve scaling, cost, and business continuity issues such as recruiting. We'll cover a mix of real-world use cases and our collective experience to help practitioners and technology leaders get started.

Anyone interested in reactive programming or how banks *actually work* should get a lot out of this talk. There are a host of business opportunities waiting to be unleashed through the power of reactive programming! For the most part, companies have stopped short of full modernization: bringing book of record systems into a modern era of technology. Meanwhile, customer adoption of these new platforms has led to an explosion of traffic to underlying legacy systems such as mainframes, resulting in huge cost increases, degraded performance and sub-par customer experiences.

We'll showcase the power of modern technologies such as Play, Akka, and Kafka, and how they are being used to break the chains of the past. We will demonstrate techniques such as event sourcing and CQRS for improving latency from mobile to mainframe from 10+ seconds to sub-second. In this always-on, always-connected era, consumers expect real-time insights. We'll show you a path towards real-time.

665a7ca82af87606f4fc83b3d94b5fd5?s=128

Kevin Webber

May 10, 2018
Tweet

More Decks by Kevin Webber

Other Decks in Programming

Transcript

  1. Mainframe Modernization From COBOL to Event Sourcing PRESENTED BY:

  2. Topics Part 1: An Introduction to Legacy Systems Part 2:

    The Old World and New World Collide Part 3: A Path to Full Modernization Part 4: The Read Only Bank Part 5: Conclusion
  3. None
  4. Part 1 An Introduction to Legacy Systems Part 1: An

    introduction to legacy systems Part 2: The old world and new world collide Part 3: A path to full modernization Part 4: The Read Only Bank Part 5: Conclusion
  5. Batches, batches everywhere!

  6. None
  7. Need a new feature? Add a Layer!

  8. None
  9. None
  10. None
  11. None
  12. None
  13. Let’s go!

  14. How long do you think it takes for this call

    to return?
  15. None
  16. It’s not just the performance... Customer Expectations Risk Cost +

    +
  17. Customer expectations As the market evolves, customer expectations go up

    They expect higher quality data that is up to date Customers want an instant view of their world Customers compare to all apps on their phone
  18. All companies are technology companies The Wealthsimple effect Robo advisors

    were the first wave of competition Amazon launching chequing accounts The second wave of competition will be fierce Uber is a bank? Uber is one of the largest originators of small business loans in the U.S
  19. 2020 and beyond? Core mainframe systems struggling to adapt to

    a mobile-first, always connected world Legacy systems designed decades ago Near-real time, API-driven usage was unanticipated How long is this sustainable?
  20. Ch-ch-changes What happens when a core system needs to change

    with urgency? Regulators are forcing improvements in many areas ◦ PSD2 Data Requirements (Open Banking) ◦ Real Time Payments ◦ GDPR
  21. Technical limitations Different views across systems

  22. Batch-based processing Multi-day processing times for simple transactions Technical limitations

  23. Limited size for transaction descriptions Means they are ı̖͇̳̰̰͍̲̻͕̩͗̒̊̈́̈́̀͆ ̃̀͘ň̢

    ̤͔̙͙̖̙͙͍͉͛̈͋͑̄͒ ̃ ͗̚ ç̧ ̲͍̯̝̻͉͎̦͛͛̒̀̀̎̍̌ ̕ ͠ ǒ̧ ̙̞̭̺̘͈͇̤̀̐̑͗́̃ ͂͂͂͜ m̨ ̩̙̝̹̥̟͚̼̟͌͆̓̏̈́̆͐ ͘̕͝p̛̛̠̲̙̰͚̥͙̳͈̦͐̇̑̈̊̈́̉͠ r̡ ̼͎͕̗͈͍̘̗͚̋͌̈́͋̌̒̓́̈̚ę̢ ̩̜̱̖͍̮̜͖̿̿́̑̓̉̓̀̋̚ ḥ ͇̳̟̪͖̜̙̳̙́͊̆̂͊̈̎̿̓͠ e ̖̙̪͖̹̫͇̱͉̹͆́̽͂̍͑̓̍͝͝ n ̰̞̳̭̭͙̮̥̤̖͌͂̌̂̀̈̍͛͒͠ s̢ ̨̛͖̗̰̤͚̞̙͙̆̆͐̆̈́̃ ̂͆̈́ï̧̟̜̪̩̠̜̥͖̥̂̒̏̂̅̀͑̿ ͘b̧ ̫̙̘̥̯̖̞͚͕̉̃ ͋̀̀̀̎͌ ͘͘l̢̨̢͚͉̜͕̪̙͉͑͐̌̓̆̽͂̂̆̀ë̛ ̟̯ ̣ ̙̫̰̼̭̫͖́̌̎̑̈̅̾̅͝ Technical limitations
  24. Technical limitations Deposit systems are running out of account numbers

    Accounts have balance and transaction size limits due to lack of integer space Support for only one currency
  25. Increasing risk SMEs are retiring Technical expertise No Documentation

  26. Regulators demand more

  27. Batch Risk Batch based systems have failure modes that can

    quickly cascade and are quite fragile Due to tight time windows, failures can take days to rectify Despite the jobs being rock solid, they are only as good as the scheduling system
  28. Getting Already very expensive Mainframes, and most software running on

    them (OS, databases, etc), are priced on capacity measured in MIPS Testing requires an army of people, and the expense of the underlying platform limits the number of testing environments
  29. Getting Already very expensive The business have large back office

    teams who perform tasks solely to overcome system limitations
  30. Recap of goals ➢ Improve customer experience ➢ Create a

    path for recruiting and retaining top talent ➢ Reduce operational costs ➢ Reduce risk ➢ Increase responsiveness to business needs ➢ What’s in our way?
  31. Part 2 The Old World and New World Collide Part

    1: An introduction to legacy systems Part 2: The old world and new world collide Part 3: A path to full modernization Part 4: The Read Only Bank Part 5: Conclusion
  32. COBOL: The Good Parts COBOL and JCL ◦ Immutable datasets

    written to disk, read by next step ◦ Can handle massive throughput ◦ Resilient, can be rerun in the event of failure (e.g, bad data) ◦ Referential transparency What do mainframe systems sound similar to?
  33. Now is the time for full modernization Modern tools and

    techniques are ready for mainframe workloads
  34. Steps Step 1: Understand current state Step 2: Model future

    state Step 3: Isolate legacy components Step 4: API enablement Step 5: Conclusion
  35. Tactic 1: Understand current state Event storming: Model interesting business

    processes in your organization Use a well defined set of operations (commands, events)
  36. Credit evaluation policy

  37. Tactic 2: Model future state Domain Driven Design: ◦ Domains

    ◦ Subdomains ◦ Cores ◦ Ubiquitous language
  38. Domains and events - Domain driven design helps us identify

    boundaries - Event sourcing helps us define interactions via aggregate events and domain events
  39. Tactic 3: Isolate Ensure responsiveness to change: ◦ Isolate business

    functionality within well defined domains ◦ Model interaction between domains with domain events ◦ Unbounded complexity is what prevents change!
  40. Interaction between domains with domain events

  41. Tactic 4: Enable Microservices and gateways ◦ Expose data and

    functionality through service interfaces ◦ REST, pub/sub, or streams: anything with a contract ◦ Bezos’ tenets ▪ No direct linking ▪ No direct reads of another team’s data store ▪ No shared-memory model ▪ No backdoors whatsoever
  42. CQRS and Event Sourcing - CQRS within a single domain

    - Commands, events, queries, and views form the contract - No relational database required
  43. Tactic 5: Modernize Slowly replace the old with the new:

    ◦ Stabilize legacy integrations ◦ Replace legacy where it brings value ◦ Leave legacy where replacement costs outweighs benefits
  44. ➢ Named after the stranger vine that grows up and

    around trees, effectively “replacing” them ➢ We can use a similar pattern to replace functionality in a legacy system with new functionality ➢ Keeps the tree intact until completely replaced “Strangler Pattern”: Decompose without destroying!
  45. Decompose without destroying! ◦ The legacy system is never modified

    ◦ Transaction scripts can be thousands of lines long and have many side effects ◦ Worst case scenario: revert to an unmodified, functioning legacy system
  46. None
  47. None
  48. None
  49. Legacy integration with anti- corruption layer

  50. Part 3 A Path to Full Modernization Part 1: An

    introduction to legacy systems Part 2: The old world and new world collide Part 3: A path to full modernization Part 4: The Read Only Bank Part 5: Conclusion
  51. Commonwealth Bank of Australia Core Banking Replacement $1.3Bn core banking

    replacement system and still replacing components ”They have been developing it for 18 months and they are still adding modules.” ”The upgrade started in 2007 and was finished by April of this year [2013] , with 1500 people working on the project full time for six years.”
  52. There is a another way...

  53. 70% read-only traffic

  54. None
  55. Facts Relevant Facts

  56. Change Data Capture (CDC)

  57. None
  58. None
  59. Part 4 The Read Only Bank Part 1: An introduction

    to legacy systems Part 2: The old world and new world collide Part 3: A path to full modernization Part 4: The Read Only Bank Part 5: Conclusion
  60. None
  61. Why is this so important?

  62. Lower cost & create new possibilities

  63. What about speed and innovation?

  64. What about organization structure?

  65. What about architecture?

  66. What about talent?

  67. What about DevOps?

  68. Part 5 Conclusion Part 1: An introduction to legacy systems

    Part 2: The old world and new world collide Part 3: A path to full modernization Part 4: The Read Only Bank Part 5: Conclusion
  69. Conclusion Current core systems are under immense strain with an

    explosion in traffic and heightened customer expectations The implementation is new but the concepts are proven Urgency to act - regulators, competitors and talent There is a path forward to break the chains of the past
  70. Let’s chat! Kevin Webber, RedElastic Principal Consultant kevin.webber@redelastic.com Dominic Wallace,

    CGI Director, Emerging Technology Engineering dominic.wallace@cgi.com Adrian Maurer, CGI Director, Emerging Technology Engineering adriano.maurer@cgi.com