Running a Fintech company with Ruby

Devengo (https://devengo.com/) is a European fintech providing instant transfers and A2A (Account to Account) banking infrastructure across the SEPA space.

For the last four years, we have been using Ruby/Rails as the core of our tech stack, and it has taken us through a global pandemic, a few funding rounds and even a pivot. As CTO, I've been lucky to have a first-row seat in this journey and would love to share what we have learned about the industry and using Ruby on it.

Ruby and almost indissolubly Rails have had for a very long time the reputation of being a slow, too-unpredictable language and, as a consequence, frown upon industries that prime performance and security. Although some of the heaviest hitters in the world of transacting money -Stripe, Shopify- depend on Ruby, we still get sceptical looks from time to time when we describe our stack.

However, in our daily practice, we have not only not found the language a source of problems but have come to appreciate it as a source of joy, especially the devs that have come in contact with Ruby for the first time when incorporated in Devengo. Nonetheless, there are some tough challenges to overcome when building a scalable service in one of the most tightly regulated and demanding spaces: Finance.

This is a working outline of the talk:

- Our engineering principles.
- Fast learners over stack-specific experts.
- Building tech in a heavily regulated industry.
- Focusing on product over infra.
- Boring technology is good technology.
- Deviating from Rails doctrine.
- Pros/Cons of using Ruby to build a modular monolith.
- Leveraging Ruby's ability to create DSLs for fun and profit.
- Providing excellent documentation as a competitive advantage.

Transcript

1. Running a Fintech with Ruby Learnings from the trenches Aitor García Rey - @_aitor - aitor.is
2. The Banking Sector
3. What they told you banking was about… BEGIN; UPDATE accounts SET balance = balance - 100.00 WHERE name = 'Alice'; UPDATE accounts SET balance = balance + 100.00 WHERE name = 'Bob'; COMMIT; Balkan Ruby 2024 Running a Fintech with Ruby 3 / 31
4. What it really is about… ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ╮ ╭ ─ ─ ┴ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ Retail Banking ┌────────────┐ │ │ │ │ ┌─────────────────────▷ IT ◁───────────┐ │ └──────□───△─┘ │ │ │ │ │ │ ┌─────────────────┐ │ │ │ │ │ Internal Audit □──┼───┘ ┌──────────□───────────┐ │ │ │ │ │ └────────────□────┘ │ │ Credit & Lending │ │ │ │ └────△─────□───────────┘ │ │ │ ╭┴─────□─────┐ ┌───────────┼───────┘ │ │ │ Compliance □──┼───────────┼───────────────────┘ │ │ │ │ ╰┬─────△─────┘ │ ┌─────▽───────────┐ │ │ │ │ Risk Management □────┐ │ │ │ ╰ ─ ─ ┼ ─ ─ ┼ ─ ─ ─ ─│─ ─ ─└──────────□──────┘─ ─ ┼ ─ ─ ─ ─│─ ─ ─ ─ ─ ─ ─ └────────┼────────────────┼───────────┼────────┘ │ │ ┌───────────▽┐ ┌────────▽───┐ ┌─▽────────────────────┐ │ Operations │ │ Treasury □─────▷ Investment & Markets │ │ │ └────────────┘ └────────────┘ └──────────────────────┘ Investment Banking│ ╰ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ Balkan Ruby 2024 Running a Fintech with Ruby 4 / 31
5. What it really is about… A transnational distributed ecosystem with an incredibly heterogeneous group of players, all of which have complex internal structures, in one of the most heavily regulated industries, with strict reliability constraints and ZERO-tolerance for downtime. Balkan Ruby 2024 Running a Fintech with Ruby 5 / 31
6. Step 0: KYB • We MUST conduct a detailed KYB (Know Your Business) process with any new customer • This is akin to the KYC you go through when opening an account in a Neo-bank but for corporates. • Nature of business • UBOs (Ultimate Beneficial Owners) & PSCs ◦ What does your business do? (Persons with Significant Control) ◦ Who are the company's main ◦ Structure Rationale customers? ◦ UBOs: Who stands to benefit economically ◦ Who are the company's main from the business? suppliers? ◦ PSCs: Who controls the business? ◦ Describe the money flows, both inflows and outflows. ◦ Does your company provide • Directors and account/s signatories services for buying, selling or holding crypto? ◦ Does your company offer credit • Velocity checks & Expected activity or loan services to consumers? ◦ Expected monthly payouts/payins (volume & value) ◦ Max amount of a single payout/payin? • Customers ◦ What type of customers do you serve? • Contacts ◦ Which countries of the EU do ◦ OPS you have exposure greater than ◦ Dev 10%? ◦ Finance manager Balkan Ruby 2024 Running a Fintech with Ruby 6 / 31
7. Step 1: Customer wants to make a payment ╭─────────────────╮ │ │ │ Originator │ │ │ ╰────────□────────╯ │ │ │ ╭ ─ ─ ─ ─▽─ ─ ─ ─ ╮ 1. SCT Inst │ Instruction │ ─ ─ ─ ─ □ ─ ─ ─ ─ │ │ ╭────────▽────────╮ │ │ │ Originator PSP │ │ │ ╰─────────────────╯ Balkan Ruby 2024 Running a Fintech with Ruby 7 / 31
8. Step 2: Balance check • The obvious first step to make a transfer is to check customer's balance. • Due to the multistep, asynchronous nature of interbank communication, we must block/secure the money that is in-flight, even if it's only for a few seconds. • These blockings produced a "variety" of balances: ◦ The real balance (the money that is ACTUALLY in the account) ◦ The available balance (the real balance - the outgoing payments that are being processed) ◦ The pending balance (the available balance + the incoming payments that are being processed) • We use the available balance for most operations but keep track of all of them for different reasons. Balkan Ruby 2024 Running a Fintech with Ruby 8 / 31
9. Step 3: Sanction lists check • Sanction lists are published by public institutions around the world to prevent money flows to/from certain actors. • Of course, no central API, published on all kinds of formats (from PDF to JSONs, from XMLs to Excels) • If the beneficiary of the transfer "matches" any of these entities we MUST block the payment AND investigate. | Entity_Type | First_Name | Family_Name | DOB | Place_of_birth | Notes | | ----------- | ---------- | ----------- | ---------- | ------------------- | --------- | | Човек | Абделгани | Селмани | 14.06.1974 | Алжир, Алжир | Член ... | | Човек | Софиан | Сенуци | 15.04.1971 | Хуссеин Дей, Алжир | Член ... | | Човек | Жозе | Сисон | 08.02.1939 | Габугао, Филипините | | | Човек | Мохаммед | Тингуали | 21.04.1964 | Блида, Алжир | Член ... | | Човек | Итзиар | Уранга | 07.10.1963 | Дуранго, Баския | активи... | | ... | ... | ... | ... | ... | ... | Source: State Agency for National Security Balkan Ruby 2024 Running a Fintech with Ruby 9 / 31
10. Step 4: PEP check • PEP (Politically Exposed Persons) lists include people that hold public or relevant positions in society (therefore exposed to bribery, corruption, etc.) • Again, published on all kinds of formats and frequencies. • And again, even partial matches MUST be BLOCKED and INVESTIGATED. | fullName | country | politicalGroup | nationalPoliticalGroup | | ------------------------ | -------- | ---------------------- | -------------------------- | | Adam BIELAN | Poland | European Conservatives | Prawo i Sprawiedliwość | | Stéphane BIJOUX | France | Renew Europe | La République en marche | | Izaskun BILBAO BARANDICA | Spain | Renew Europe | Partido Nacionalista Vasco | | Vladimír BILČÍK | Slovakia | Christian Democrats | Independent | | Dominique BILDE | France | Identity and Democracy | Rassemblement national | | ... | ... | ... | ... | Source: European Parliament Balkan Ruby 2024 Running a Fintech with Ruby 10 / 31
11. Step 5: Starting communication - ISO20022, A standard to rule them all <?xml version="1.0" encoding="UTF-8"?> • XML based <Document xmlns="urn:iso:std:iso:20022"> <CstmrCdtTrfInitn> <GrpHdr> • Provides message definitions for <MsgId>1649186242</MsgId> everything you can imagine in <CreDtTm>2024-04-05T00:00:00</CreDtTm> banking: <NbOfTxs>7</NbOfTxs> <CtrlSum>188552.18</CtrlSum> <InitgPty> ◦ Payments Initiation, Clearing & <Nm>Gig Delivery SL</Nm> Settlement <Id> ◦ Direct Debit <OrgId> ◦ Card Transactions <Othr> ◦ Forex <Id>B65874893000</Id> ◦ ATM <SchmeNm> ◦ Fraud Reporting <Prtry>SEPA</Prtry> </SchmeNm> </Othr> • Used as backbone by most countries' </OrgId> initiatives </Id> </InitgPty> </GrpHdr> <PmtInf> <PmtInfId>1649186242-0</PmtInfId> <PmtMtd>TRF</PmtMtd> <BtchBookg>false</BtchBookg>
12. Step 6: The instant transfer scheme, SCT-INST ╭─────────────────╮ ╭─────────────────╮ │ │ │ │ │ Originator │ │ Beneficiary │ │ │ │ │ ╰────────□────────╯ ╰────────△────────╯ │ │ │ │ │ │ ╭ ─ ─ ─ ─▽─ ─ ─ ─ ╮ ╭ ─ ─ ─ ─ ─ ─ ─ ─ ╮ ╭ ─ ─ ─ ─ ─ ─ ─ ─ ╮ ╭ ─ ─ ─ ─□─ ─ ─ ─ ╮ 1. SCT Inst 2. SCT Inst 3. Relay 5. Funds │ Instruction │ │ Transaction │ │ Transaction │ │ available │ ─ ─ ─ ─ □ ─ ─ ─ ─ ─ ─△─ ─ ─ ─ ─ ─□─ ─ △ ─ ─ ─ ─ ─ □ ─ ─ ─ ─ ─ △ ─ ─ ─ ─ │ │ │ │ │ │ │ │ │ │ │ │ ╭────────▽────────╮ │ ╭───▽─────────□───╮ │ ╭────────□────────╮ │ │ │ │ │ │ │ │ │ Originator PSP □──────────┘ │ CSM │ └────────▷ Beneficiary PSP │ │ │ │ │ │ │ ╰────────△────────╯ ╰──□─────□─────△──╯ ╰──────□──────△───╯ │ │ │ │ │ │ │ │ │ │ ╭ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─▽─ ─ │ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ╮ │ │ └──□ 4a. Communicates OK / KO │ │ └──□│ 6. Relay Result ◁─────┘ │ ╰ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ╯ │ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ╮ │ └────────▷│ 4b. ACK Communication □──┘ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ╯ Balkan Ruby 2024 Running a Fintech with Ruby 12 / 31
13. SCT-Inst Constraints • 10s execution limit E2E • 100.000€ max per transfer • 24x7x365 Balkan Ruby 2024 Running a Fintech with Ruby 13 / 31
14. And that is just ONE scheme • Regular Transfers ◦ They are not settled in real-time (time windows during the day) • Request to Pay ◦ The "pull" version of payments. Candidate to destroy a significant portion of the debit card sector. • Verification of Payee ◦ A new addition to prevent fraud/errors in a real-time context. Balkan Ruby 2024 Running a Fintech with Ruby 14 / 31
15. Learnings from the trenches
16. Some context on our journey - Devengo 1.0 • Started as a Salary Advance solution to fight payday loans / predatory options. • First commit on Jul 21, 2019 • Rails API + Native apps (iOS & Android) + Custom integrations with HR/Payroll software • Got smacked in the face by COVID-19 but experienced constant growth from 2021 onwards. • Spent 2 ½ years building the product... • ...just to discover that it took us in the wrong direction Balkan Ruby 2024 Running a Fintech with Ruby 16 / 31
17. Some context on our journey - Devengo 2.0 • We reflected on how we were doing all the heavy lifting of moving money... • ...and deploying it (too early) in a very narrow niche. • Pivoted to Payments Provider on 2022 Q2 • Special focus on EU and Instant/Real-time A2A payments • Still an Early-Stage startup, just 7 developers (including me) running everything • But already moving a volume of hundreds of thousands of transfers per month for a value of dozens of million euros (even from old competitors) • We'll (hopefully!) get a PSP license by the Central Bank of Spain this summer Balkan Ruby 2024 Running a Fintech with Ruby 17 / 31
18. How to model the business logic? • We were very conscious from the very beginning that we will need to implement tons of business logic that may change due to external factors (e.g. regulation) and that must be "tweakable" according to customers' specifics (market, size, etc.) • Other than implementing the MVC pattern, Rails is mostly agnostic about how to model business logic ◦ Hard to determine if that is a good or a bad thing ;) • We knew from previous experience in Rails and many other frameworks/paradigms of ideas like Service Objects, Use-cases (mostly borrowed from Clean Architecture) • We saw too other ruby specific ideas like Interactors (Hanami.rb), Operations (Trailblazer), etc. • But none of these strategies were included in the Rails Doctrine Balkan Ruby 2024 Running a Fintech with Ruby 18 / 31
19. Modular Monolith & Domain Driven Design • Around the beginning of the project we came in contact with the idea of a "Modular Monolith" introduced by Shopify circa 2019: ◦ Splitting business logic on multiple components, packages or domains. ◦ Introducing clear boundaries between them ◦ Interacting through simple interfaces ◦ Forcing us to think about internal dependencies • It made a lot of sense to us! • Thanks to that approach from the get go the "Salary Advance" domain was NEVER touching money, and just asked the "Banking" domain to "create a payment" through a very specific "client" interface. Something that was invaluable when we had to pivot. Balkan Ruby 2024 Running a Fintech with Ruby 19 / 31
20. Modular Monolith & Microservices • It's in a way similar to microservices when it comes to splitting the business logic BUT… • Everything happens with in-process communication: no serialization, no HTTP connection, no network latency, no de-serialization, no deploy coordination issues, etc. | Operation | ns | µs | ms | note | | --------------------- | -------------: | ---------: | -----: | --------------------------- | | L1 cache reference | 0.5 ns | | | | | Mutex lock/unlock | 25 ns | | | | | Main memory reference | 100 ns | | | 20x L2 cache, 200x L1 cache | | Send 1K 1Gbps network | 10,000 ns | 10 µs | | | | Read 4K SSD* | 150,000 ns | 150 µs | | ~1GB/sec SSD | | Round trip within DC | 500,000 ns | 500 µs | | | | Read 1 MB SSD* | 1,000,000 ns | 1,000 µs | 1 ms | ~1GB/sec SSD, 4X memory | | Disk seek | 10,000,000 ns | 10,000 µs | 10 ms | 20x datacenter roundtrip | | Read 1 MB Disk | 20,000,000 ns | 20,000 µs | 20 ms | 80x memory, 20X SSD | | TCP packet continents | 150,000,000 ns | 150,000 µs | 150 ms | | • We wanted more modularity WITHOUT the logistical complexity and overhead (Kubernetes & Friends) Balkan Ruby 2024 Running a Fintech with Ruby 20 / 31
21. Defensive/Offensive programming Working with PSPs and providers means constantly interacting with them through APIs, webhooks, etc. A pretty trivial but common scenario is the integration of business entities state changes based on documented enums, eg. ACCP, BLCK and RJCT. 1 case psp_response.code 1 case psp_response.code 2 when "ACCP" 2 when "ACCP" 3 payment.confirm! 3 payment.confirm! 4 when "BLCK" 4 when "BLCK" 5 payment.block! 5 payment.block! 6 when "RJCT" 6 when "RJCT" 7 payment.reject! 7 payment.reject! 8 end 8 else 9 report(:ops_team, psp_response) 10 end Lesson: always be irrationally defensive on your interpretation of what is happening. If anything unexpected happens DO NOTHING & RAISE THE HAND Balkan Ruby 2024 Running a Fintech with Ruby 21 / 31
22. Idempotency is not a good practice, is mandatory • A common scenario in A2A Fintech: ◦ A customer sends a payment request ◦ Our service sends the request to the beneficiary PSP ◦ Response is NOT received in the usual time-span. ◦ Our customer retries the same payment (probably triggered by its customer impatience) ◦ Both payments end up being confirmed, and therefore paid twice. • One of the parts that help to solve that is idempotency keys. ◦ Unique IDs provided by the customer that are assured to be accepted/processed only ONCE in a given timeframe Balkan Ruby 2024 Running a Fintech with Ruby 22 / 31
23. Audit everything, all the time We are LEGALLY bound to audit every change on our system, who made it and in many cases WHY it made it. --- Transfers & Ledger ------------------------------------------------------------------ AccounHolder#35 ACTIVE Linking Paths Origin#91 ACTIVE Linking Paths Payments - ES74683 Payment#1261918 CONFIRMED 2024-04-03 11:12:30 pyo_7KVDQmk7ClxIgahy23Q - €0.04 Transfer#1801849 CONFIRMED 2024-04-03 11:12:30 tra_72ZHAAl9laW5N1Juva6 - INSTAN Log#4213804 SANC 2024-04-03 11:12:24 NO REASON Log#4213806 C 2024-04-03 11:12:30 ACCP Log#4213805 X 2024-04-03 11:12:24 NO REASON LdgTxn#1134740 CONFIRMED 2024-04-03 11:12:31 Linking Paths Ledger LdgEntry#2269316 2024-04-03 07:56:50 DEBIT - LdgAccount#4 - Linking LdgEntry#2269317 2024-04-03 07:56:50 CREDIT - LdgAccount#32 - Aitor L --- Webhooks --------------------------------------------------------------------------- Event#51581939 2024-04-03 07:56:50 outgoing_payment.created WhkReq#4973549 200 2024-04-03 07:56:54 OK Event#51581942 2024-04-03 07:56:50 outgoing_payment.validating WhkReq#4973551 200 2024-04-03 07:56:54 OK Event#51678768 2024-04-03 11:12:18 outgoing_payment.processing WhkReq#4993158 200 2024-04-03 11:12:18 OK Event#51678810 2024-04-03 11:12:30 outgoing_payment.confirmed WhkReq#4993163 200 2024-04-03 11:12:34 OK Event#51678812 2024-04-03 11:12:31 outgoing_payment_receipt.created --- TMS -------------------------------------------------------------------------------- Evaluation#340839 2024-04-03 07:56:50 MRM#7 NOOP 2024-04-03 07:56:50 (TESTING) - [Company linking-pat MRM#8 BLOCK 2024-04-03 07:56:50 (ACTIVE) - [Company linking-path ------------------------------------------------------------------------------------------
24. Audit YOURSELF Not even our own developers are allowed to touch data without the change being audited: ❯ heroku run rails console -a dv-REDACTED-prod Running rails console on ⬢ dv-REDACTED-prod... up, run.1708 (Standard-1X) I, [2024-04-26T08:17:07.230660 #2] INFO -- : Registering subscribers for REDACTED domain I, [2024-04-26T08:17:07.422579 #2] INFO -- : Registering subscribers for REDACTED domain I, [2024-04-26T08:17:07.447965 #2] INFO -- : Registering subscribers for REDACTED domain I, [2024-04-26T08:17:07.463217 #2] INFO -- : Registering subscribers for REDACTED domain I, [2024-04-26T08:17:07.466086 #2] INFO -- : Registering subscribers for REDACTED domain I, [2024-04-26T08:17:07.575645 #2] INFO -- : Registering subscribers for REDACTED domain I, [2024-04-26T08:17:07.598012 #2] INFO -- : Registering subscribers for REDACTED integration Loading production environment (Rails 6.1.7.7) Enter your email: REDACTED@devengo.com Enter your password: I, [2024-04-26T08:17:24.080141 #2] INFO -- : Logged in as REDACTED@devengo.com. All changes will be audited to this manager. irb(main):001:0> Balkan Ruby 2024 Running a Fintech with Ruby 24 / 31
25. DSLs: Retry System • +95% of transfers are processed without problem, but the other 5%… • …is a big problem if you are making dozens of thousands of transfers per day. • We leverage Ruby's ability to create beautiful DSLs to build a retry system that ◦ automatically deals with those errors on behalf of our customers ◦ based on declarative policies that are intuitive a clear 1 module Banking 2 module Models 3 module RetryPolicies 4 class SepaAbxxError < RetriableBase 5 error_codes %w[AB05 AB06 AB07 AB08 AB09 AB10] 6 max_attempts 5 7 retry_interval [30.minutes, 3.hours, 5.hours, 7.hours, 9.hours] 8 end 9 end 10 end 11 end Balkan Ruby 2024 Running a Fintech with Ruby 25 / 31
26. DSLs: Transaction Monitoring System • As part of processing EACH payment we run it through something that in the industry is call "Transaction Monitoring" • This process is aimed at detecting anomalous pattern in the use of our service: ◦ Is the amount of the transfer way lower or higher than usual? ◦ Is this the N time we are sending money to the same IBAN in a M period of time? ◦ Is this company receiving more than X money from this counterparty in a Y timeframe? 1 enum timeframe: { 2 daily: "daily", 3 monthly: "monthly", 4 quarterly: "quaterly", 5 }, _default: "daily" 6 7 enum dimension: { 8 volume: "volume", 9 value: "value", 10 }, _default: "volume" 11 12 enum entity: { 13 payment: "payment", 14 incoming: "incoming", 15 }, _default: "payment" 16 17 enum actor: { 18 company: "company", 19 … Balkan Ruby 2024 Running a Fintech with Ruby 26 / 31
27. Documentation is critical • As an API-First service having a top-notch documentation is critical for us. • In a way for the companies integrating us on their workflows the documentation IS the product. • Hat-tip to Stripe for teaching us that lesson more than a decade ago. • We are now at the point where CI pipelines integrates multiple documentation testing procedures • We have a home-grown documentation building pipeline that leverages Ruby's flexibility to integrate business entities: 1 ### Metadata limits and behavior 2 3 There are some limits to the information you can store as metadata: 4 5 - A maximum of <%= Banking::Values::Shared::Metadata::MAX_KEYS_LIMIT %> keys for each entity. 6 - A maximum of <%= Banking::Values::Shared::Metadata::KEY_SIZE_LIMIT %> bytes for each metadata key. 7 - A maximum of <%= Banking::Values::Shared::Metadata::VALUE_SIZE_LIMIT %> bytes for each metadata value. 8 - All the values and keys will be coerced to strings. Balkan Ruby 2024 Running a Fintech with Ruby 27 / 31
28. So in the end… what should our engineering principles be? • Bias for Action (Amazon) • Make Big, Bold Bets (Uber) • Move fast and Break things (Facebook) Balkan Ruby 2024 Running a Fintech with Ruby 28 / 31
29. Our Engineering Principles • Finesse: Everything is taken care of down to the smallest detail • Zen Pragmatism: Making tradeoffs is part of the game • Do the right thing: Period • Managers of one: Hire people with resolution and gives them freedom and autonomy. • Taking Care of Business: Understand our craft is a "people problem". Balkan Ruby 2024 Running a Fintech with Ruby 29 / 31
30. And last but not least: Trust, but verify • The financial world is a sector where trust is EVERYTHING • Доверяй, но проверяй is probably the one principle we try to hammer in the attitude of the whole company, specially in the engineering team • What does it mean? ◦ Expansive testing ◦ PR Reviews ◦ Exhaustive monitoring ◦ Automatic and manual checks ◦ Feature flags Balkan Ruby 2024 Running a Fintech with Ruby 30 / 31
31. 𝕏 @_aitor / aitor.is Thank you! 𝕏 @devengoapi / devengo.com