Redis at Heroku

Redis at Heroku

Given at RedisConf 2015

53be54e5db4dc58e4980db5a8255621b?s=128

Harold Giménez

March 05, 2015
Tweet

Transcript

  1. Redis at Heroku Harold Giménez @hgmnz redisconf 2015 +

  2. Redis at Heroku git deployment

  3. proxy git push heroku master worker worker worker build service

    repos git push user
  4. proxy git push heroku master worker worker worker build service

    repos build app breezing-swiftly-152 user
  5. proxy git push heroku master worker worker worker build service

    repos I got this user
  6. proxy git push heroku master worker worker worker build service

    repos user
  7. proxy git push heroku master worker worker worker build service

    repos here’s my hostname and port user
  8. proxy git push heroku master worker worker worker build service

    repos user
  9. Redis at Heroku Logs

  10. heroku logs --tail -a hgmnz 2010-09-16T15:13:46.677020+00:00 app[web.1]: Processing PostController#list (for

    208.39.138.12 at 2010-09-16 15:13:46) [GET] 2010-09-16T15:13:46.677023+00:00 app[web.1]: Rendering template within layouts/application 2010-09-16T15:13:46.698234+00:00 app[web.1]: Completed in 74ms (View: 31, DB: 40) | 200 OK [http:// myapp.heroku.com/] 2010-09-16T15:13:46.723498+00:00 heroku[router]: at=info method=GET path="/posts" host=myapp.herokuapp.com" fwd="204.204.204.204" dyno=web.1 connect=1ms service=18ms status=200 bytes=975 2010-09-16T15:13:47.893472+00:00 app[worker.1]: 2 jobs processed at 16.6761 j/s, 0 failed ... $
  11. app dyno 1 dyno 2 dyno 3 postgres primary postgres

    follower redis logplex config heroku logs --tail Logplex app log messages
  12. Logplex canaries msg producer msg producer msg producer msg producer

    logging pipeline msg set message loss? latency?
  13. Redis at Heroku Orchestration

  14. Process orchestration Scaling web dynos... done, now running 10 Scaling

    worker dynos... done, now running 5 $ $ heroku ps:scale web=10 worker=5 -a demo-app process manager Messages sent to any Redis to boot,
 kill, restart processes dynos
  15. Web dyno idling heroku router cluster keys with a 1h

    expiry HTTP traffic process manager which apps have had
 HTTP traffic
 in the last 24h?
  16. Redis at Heroku Database Lifecycle

  17. observe environment finite state machine

  18. available creating uncertain unavailable deprovisioning deprovisioned

  19. class Resource def feel observations.create( Feeler.new(self).current_environment ) end end class

    Feeler def current_environment { services_available?: service_available?, connections: connections, row_count: row_count, table_count: table_count, seq_scans: seq_scans, index_scans: index_scans } end end
  20. class Resource include Stateful state :available do unless service_available? transition

    :uncertain end end end resource = Resource.find resource.transition :available resource.fetch_observation resource.tick puts resource.state # => :uncertain
  21. module Stateful def self.included(base) base.extend ClassMethods end module ClassMethods def

    state(name, &block) state[name] = block end def states; @states ||={}; end end def tick self.instance_eval( &self.class.states[self.state.to_sym] ) end def transition(state); end end
  22. resource.fetch_observation resource.tick Need to do this all the time

  23. db1 db2 db3 db4 db5 db6 db7 db8 dbn W

    orkers db1.fetch_observation
 db1.tick
  24. db db3 db2 db1 dbn Enqueue

  25. Redis at Heroku Database Monitoring

  26. Database Observations and Monitoring supervisor worker 1 worker 2 worker

    3 worker 4 worker N I’m available for work
  27. supervisor worker 1 worker 2 worker 3 worker 4 worker

    N Worker 1 monitor this DB Database Observations and Monitoring
  28. supervisor worker 1 worker 2 worker 3 worker 4 worker

    N Database Observations and Monitoring monitor
  29. supervisor worker 1 worker 2 worker 3 worker 4 worker

    N Database Observations and Monitoring
  30. supervisor worker 1 worker 2 worker 3 worker 4 worker

    N I can’t reach this DB Database Observations and Monitoring
  31. supervisor worker 1 worker 2 worker 3 worker 4 worker

    N Worker 4 monitor this DB Database Observations and Monitoring
  32. supervisor worker 1 worker 2 worker 3 worker 4 worker

    N Database Observations and Monitoring monitor
  33. worker Database Monitoring Last Observation cache postgres Heroku Postgres automation

    service GET /last_observation HSET db1 db_size 4213 HSET db1 cache_hit_rate 98
  34. Redis at Heroku Other use cases

  35. Router stats router node router node router node router node

    router node router node router node requests errors
  36. pubsub + websockets dyno 1 dyno 2 dyno 3 pubsub

    browser websockets
  37. I asked: what’s good and bad about Redis?

  38. good bad •Fast •Simple •Powerful primitives •Expires on any data

    type •Many use cases, single infra •replication protocol is “surprisingly elegant” •No SSL support •Rotating creds is a pain •SPOF •Single threaded(ish)
  39. Thanks! @hgmnz +