“API?” – How LendingHome Approaches “Legacy” Technologies

LendingHome is on a mission to simplify the mortgage process, but that’s not always easy in a world where we hear “API? What’s that?” from our third-party vendors. For the ones that do have technology solutions the phrases “FTP” and “Fixed-Width” are frequently thrown around.

This talk will dive into the solutions we’ve built that abstract away some of these concepts and how we provide clean interfaces for these services to the rest of the organization. If you’ve ever wanted to learn how modern companies interface with the old, this is the talk for you.

This is a sponsored talk by LendingHome.

Transcript

1. “API?” — How LendingHome Approaches “Legacy” Technologies RailsConf 2018 / April 19, 2018 / Sam Aarons — Senior Software Engineer For External Use
2. LendingHome is rethinking mortgage from the ground up with technology. INTRODUCTION For External Use - 2
3. INTRODUCTION For External Use - 3
4. LendingHome has hundreds of vendor integrations. We’ve seen everything. INTRODUCTION For External Use - 4
5. API? We have a server… Anonymous, Vendor Scaring Product Manager “ ” INTRODUCTION For External Use - 5
6. Build the Interface You Want Plan for Failure Be Efficient When Reading & Writing Lessons We’ve Learned From “Legacy APIs” For External Use - 6
7. Build the Interface You Want
8. • Clean interfaces benefit people, machines don’t care. • Focus on building an internal API (either in code or through a microservice) that you wish your vendor had built. • Make it easier for your teammates to understand how to interact with the vendor without needing to understand the file/message format. • Make sure to use technologies that match with the vendor’s synchronous / asynchronous interfaces. Build an Abstraction BUILD THE INTERFACE YOU WANT For External Use - 8
9. Initial Architecture BUILD THE INTERFACE YOU WANT For External Use - 9 Platform Rainmaker Bank SQS FTP SQS FTP
10. Final Architecture BUILD THE INTERFACE YOU WANT For External Use - 10 Platform Rainmaker Grand Central FTP Vendor 1 FTP Vendor 2 FTP Vendor N S3 S3 SQS
11. UI/UX is Interface Too BUILD THE INTERFACE YOU WANT For External Use - 11
12. Plan for Failure
13. PLAN FOR FAILURE For External Use - 13
14. They shut down the server every night before they leave. Anonymous, LendingHome engineer lamenting “ ” PLAN FOR FAILURE For External Use - 14
15. • Use good libraries. Verify they actually work. • Remove files you’ve read, or let your vendor do it. • Don’t read files into memory. • Log everything. • Save a copy of every file/message you send and receive. • A static IP almost always is required. • Learn the exact transport: FTP, FTPS, SFTP, Proprietary. Everything Will Go Wrong, So Make Your System Robust PLAN FOR FAILURE For External Use - 15
16. It’s not all technical ability. Soft skills will save you. PLAN FOR FAILURE For External Use - 16
17. Be Efficient When Reading & Writing
18. XML Streaming Parsers (Nokogiri) Streaming Writers (libxml-ruby) Computers Are Pretty Good at Files Fixed Width Streaming Parsers (#each_line) Streaming Writers (#puts) BE EFFICIENT WHEN READING & WRITING For External Use - 18 CSV Streaming Parsers (#shift) Streaming Writers (#<<)
19. lines = IO.readlines("testfile") lines.map do |line| magic(line) end Simple IO BE EFFICIENT WHEN READING & WRITING For External Use - 19
20. def each_row(input) input.each_line do |line| row = magic(line) yield row end end IO & Iterators BE EFFICIENT WHEN READING & WRITING For External Use - 20
21. Build the Interface You Want Plan for Failure Be Efficient When Reading & Writing In Conclusion For External Use - 21
22. sam@lendinghome.com Sam Aarons We’re Hiring. Questions? For External Use