Tweet
Tweet

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Political Revolutions • ActBlue is a nonprofit that builds fundraising software for: • Democratic campaigns and committees • advocacy groups • nonprofit organization • Bernie most popular, but 17,500 organizations since 2004 • 3,000 organizations Q1 2017 alone • using Rails since 2005

Slide 3

Slide 3 text

• process donations (payments) • technology + compliance • fundraise tools, stats, A/B tests • 3.8 million ActBlue Express users • $1.6 billion in 31 million contributions ($50 avg) • empower small-dollar donors

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

New Hampshire primary night February 9, 2016

Slide 8

Slide 8 text

Optimize What? • contribution form: • load contribution form • process a contribution

Slide 9

Slide 9 text

Process a Contribution

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

Scaling Challenge • approach • implementation • cost / solution because lots of small-dollar donations Presenting

Slide 12

Slide 12 text

Metrics

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Metrics # statsd-ruby statsd = Statsd.new(‘chicago-1’) statsd.gauge(‘pending.auths’, pending_auths.count) statsd.timing(‘latency.auths’, Time.now - auth.created_at) • Graphite renders the graphs • collectd has plugins for PostgreSQL, Postfix, … • logs!

Slide 15

Slide 15 text

Metrics Multiple Servers

Slide 16

Slide 16 text

Multiple Servers

Slide 17

Slide 17 text

Multiple Servers # NGINX load balancer configuration http { upstream backend { server 192.168.1.101; server 192.168.1.102; server 192.168.1.103; } server { listen 80; location / { proxy_pass http://backend; }}}

Slide 18

Slide 18 text

Multiple Servers Cost • no file system • handle persistence: Rails sessions, Redis, … • increases required DB connections • LB is single point of failure

Slide 19

Slide 19 text

Metrics Multiple Servers Caching

Slide 20

Slide 20 text

Caching • general term, exists at multiple levels • highest value for the effort • Fastly CDN - huge! • stress tested with DDoS

Slide 21

Slide 21 text

Caching

Slide 22

Slide 22 text

CDN Dashboard

Slide 23

Slide 23 text

Caching Control • HTTP headers • Varnish Configuration Language (VCL) • API Cache-Control: max-age=400 Surrogate-Control: max-age=3600 Vary: Accept-Encoding Surrogate-Key: key1 key2 if (req.url ~ "^/videos") { set req.backend = F_VIDEO_LB; }

Slide 24

Slide 24 text

Caching Cost • expensive • complexity if you want flexibility • intricate SSL termination • contribution form filled by JS

Slide 25

Slide 25 text

Metrics Multiple Servers Caching Separation of Concerns

Slide 26

Slide 26 text

Separation of Concerns • main app in Rails • credit card tokenizer in Node • multiple DB instances • separation useful for compliance as well

Slide 27

Slide 27 text

Separation of Concerns Cost • complex developing and testing

Slide 28

Slide 28 text

Metrics Multiple Servers Caching Separation of Concerns Deferred Tasks

Slide 29

Slide 29 text

Deferred Tasks • CC processing = multiple calls to web services • fault isolation • increased reliability • deferred batch settlements - huge!

Slide 30

Slide 30 text

Queueing System execute authorizations execute settlements send email web server 1 web server 2 web server 3

Slide 31

Slide 31 text

Sidekiq class SettlementProcessor include Sidekiq::Worker def perform(id) Settlement.find(id).settle! end end SettlementProcessor.perform_async(settlement.id) ActiveJob ReceiptMailer.receipt(contribution).deliver_later

Slide 32

Slide 32 text

Deferred Tasks Cost • queuing system is unreliable • coordination • difficult to debug

Slide 33

Slide 33 text

Metrics Multiple Servers Caching Separation of Concerns Deferred Tasks Scalable Architecture

Slide 34

Slide 34 text

Scalable Architecture • mindset, i.e. most important • being fast + being scalable • most difficult to fix if you get it wrong • concurrency instead of central process • avoid premature optimization

Slide 35

Slide 35 text

Scalable Architecture Metrics Multiple Servers Caching Separation of Concerns Deferred Tasks

Slide 36

Slide 36 text

• interested in what we do? • come work with us • talk to me directly • slides on speakerdeck.com • stickers! Braulio Carreño @bcarreno