Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a highly scalable exchange - To react or not to react

Building a highly scalable exchange - To react or not to react

A non-engineering and non-technical deep dive into building a highly scalable exchange. A self-learning and a retrospection into what we did and what we did wrong.

4cfacd21141350a366eea3152667bf49?s=128

Sreekanth G S

March 02, 2019
Tweet

Transcript

  1. Building a highly scalable exchange To react or not to

    react Sreekanth G S Chief Technology Officer Hatio Innovations Private Limited A BillDesk Company
  2. Me Thinker Blogger Technologist Programmer Traveler Writer Crypto Evangelist Ideologist

    2
  3. What this presentation is not A coding exercise A programming

    tutorial A technology recommendation Or a demo presentation </> 3
  4. Instead it is about Our self-learning And a retrospection on

    What we did And what we did wrong 4
  5. We built a crypto currency exchange 5

  6. Our initial stack Ruby on Rails PostgreSQL Dockerised Deployments Orchestration

    using Rancher 6
  7. The best part Very easy to build Shortest time to

    market Handles things pretty well Until… 7
  8. The woes Scaling Experience Availability 8

  9. What went wrong The stack couldn’t scale Rails view rendering

    wasn’t efficient Nothing but jQuery in Frontend And Rails for Backend wasn’t the solution 9
  10. RCA – The three letter horror Heavy long running SQL

    queries Lack of efficient caching 502 Bad Gateways on Upstream unavailability No way to let customers know that we are down 10
  11. Data, data, data Almost a single page app. Most time

    spent on Trading. Data is extremely important. Realtime updates. Customers make decisions based on data. 11
  12. Realtime updates WebSocket updates Charts & Graphs XHR Requests Events,

    alerts and notifications 12
  13. Scaling – The mammoth task Scaling is a multistep solution

    Split into - Fixing the SQL queries - Caching of responses - Decoupling the frontend 13
  14. Fixing the backend issues SQL Query Optimizations – Moving to

    Aurora Caching of requests & data – Memcached & Redis Archiving of stale data – Adopting the ELK Stack Taking care of complex weightlifting – C# & .NET 14
  15. Initial JavaScript days For simple DOM manipulations Handling of event

    life cycles – WebSocket updates using jQuery bundled with Rails and Vue.js for simple data bindings 15
  16. Limitations of this setup Cluttered and spaghetti JavaScript code Never

    ending conflicts across multiple framework Frontend was still tightly coupled with backend View rendering was a toll on App Server 16
  17. Decoupling the frontend Deciding on a single frontend framework Multiple

    contenders – Vue, React, Backbone and Angular Roadmap for Mobile Apps – Android & iPhone 17
  18. The difficult choice 18

  19. Vue React Lightweight Flexible Vue Specific Components Pure JavaScript Better

    Documentation Larger Community HTML Templating Optional JSX Templating Very easy to Integrate Higher learning curve Limited support for Mobile Apps React Native Difficult to hire engineers Very good talent pool 19
  20. The shortlist React it is. Rich Library Support Very good

    talent pool React Native for Apps 20
  21. The learning curve Too minimalistic Concept of a state store

    was new One-way data binding was challenging Not so good documentation 21
  22. JSX as a pain point Developers used to write HTML/Erb

    Difficulty and inertia learning JSX 22
  23. The good part - Components Code became components - Reusable

    code, with Single Responsibility - Easier to manage - Without any or very low side effects 23
  24. Components & Actions let handleMarketTradeHistory = activeMarket => { return

    dispatch => { const marketTradeHistoryChannel = pusher.subscribe( `market-${activeMarket}-trade-history` ); marketTradeHistoryChannel.bind("market_trade_history", data => { data.trade_history.map(item => dispatch(processTradeHistoryData(item))); }); }; }; 24
  25. Dispatchers to do the work export function processTradeHistoryData(data) { return

    dispatch => { dispatch({ type: ActionType.ADD_TRADE_ORDER, payload: { amount: data.a, created_at: data.T, order_type: data.M, price: data.p, vol: data.q } }); }; } 25
  26. The good part – Clean Data Data is better handled

    - Single storage of data and state - One-way data binding, with clean hierarchy - You cannot accidentally pollute data 26
  27. The good part – Invisible DOM The declarative approach -You

    do not directly touch DOM -Updates are batched and automatic -Think about what and not how 27
  28. 100s of lines of code to a few Componentization made

    code smaller Legacy JavaScript code wasn’t reusable No way to debug or test Now it’s decoupled and self handling 28
  29. MVC to Pure API Functionality interwoven with View Single codebase,

    hard to maintain Transition to Pure API, everything returns data View can be anything, Web or Mobile! 29
  30. React Native Future Apps in React Native Pure API model

    complements this approach Frontend developers can collaborate UX and Apps coming under an umbrella 30
  31. Benchmarking Load times between page loads and data population: Category

    Load Time RoR with No Optimization 2.80s RoR with Backend Optimization 2.03s RoR with React Frontend 1.80s RoR with Backend Optimization & React Frontend 1.07s 31
  32. Unsolved Issues Virtual DOM is at times slow Upgrading and

    updating is not easy as expected High memory consumption – 16.3 MB Slow startup times – 219 ms 32
  33. The learnings Scale is not a frontend/backend only problem Customers

    tolerate informed issues, not 502s Developer happiness is extremely important Initial learning curve can be future productivity 33
  34. The takeaways Rewriting entire code base isn’t easy. When you

    are forced to do it, decide wisely. React is not a one stop solution, it was a fit for us. At the end of the day, its about delivering. 34
  35. Thank You @sreekanthgs 35