Redis at Heroku

Given at RedisConf 2015

Transcript

1. Redis at Heroku Harold Giménez @hgmnz redisconf 2015 +
2. Redis at Heroku git deployment
3. proxy git push heroku master worker worker worker build service repos git push user
4. proxy git push heroku master worker worker worker build service repos build app breezing-swiftly-152 user
5. proxy git push heroku master worker worker worker build service repos I got this user
6. proxy git push heroku master worker worker worker build service repos user
7. proxy git push heroku master worker worker worker build service repos here’s my hostname and port user
8. proxy git push heroku master worker worker worker build service repos user
9. Redis at Heroku Logs
10. heroku logs --tail -a hgmnz 2010-09-16T15:13:46.677020+00:00 app[web.1]: Processing PostController#list (for 208.39.138.12 at 2010-09-16 15:13:46) [GET] 2010-09-16T15:13:46.677023+00:00 app[web.1]: Rendering template within layouts/application 2010-09-16T15:13:46.698234+00:00 app[web.1]: Completed in 74ms (View: 31, DB: 40) | 200 OK [http:// myapp.heroku.com/] 2010-09-16T15:13:46.723498+00:00 heroku[router]: at=info method=GET path="/posts" host=myapp.herokuapp.com" fwd="204.204.204.204" dyno=web.1 connect=1ms service=18ms status=200 bytes=975 2010-09-16T15:13:47.893472+00:00 app[worker.1]: 2 jobs processed at 16.6761 j/s, 0 failed ... $
11. app dyno 1 dyno 2 dyno 3 postgres primary postgres follower redis logplex config heroku logs --tail Logplex app log messages
12. Logplex canaries msg producer msg producer msg producer msg producer logging pipeline msg set message loss? latency?
13. Redis at Heroku Orchestration
14. Process orchestration Scaling web dynos... done, now running 10 Scaling worker dynos... done, now running 5 $ $ heroku ps:scale web=10 worker=5 -a demo-app process manager Messages sent to any Redis to boot,
 kill, restart processes dynos
15. Web dyno idling heroku router cluster keys with a 1h expiry HTTP traffic process manager which apps have had
 HTTP traffic
 in the last 24h?
16. Redis at Heroku Database Lifecycle
17. observe environment finite state machine
18. available creating uncertain unavailable deprovisioning deprovisioned
19. class Resource def feel observations.create( Feeler.new(self).current_environment ) end end class Feeler def current_environment { services_available?: service_available?, connections: connections, row_count: row_count, table_count: table_count, seq_scans: seq_scans, index_scans: index_scans } end end
20. class Resource include Stateful state :available do unless service_available? transition :uncertain end end end resource = Resource.find resource.transition :available resource.fetch_observation resource.tick puts resource.state # => :uncertain
21. module Stateful def self.included(base) base.extend ClassMethods end module ClassMethods def state(name, &block) state[name] = block end def states; @states ||={}; end end def tick self.instance_eval( &self.class.states[self.state.to_sym] ) end def transition(state); end end
22. resource.fetch_observation resource.tick Need to do this all the time
23. db1 db2 db3 db4 db5 db6 db7 db8 dbn W orkers db1.fetch_observation
 db1.tick
24. db db3 db2 db1 dbn Enqueue
25. Redis at Heroku Database Monitoring
26. Database Observations and Monitoring supervisor worker 1 worker 2 worker 3 worker 4 worker N I’m available for work
27. supervisor worker 1 worker 2 worker 3 worker 4 worker N Worker 1 monitor this DB Database Observations and Monitoring
28. supervisor worker 1 worker 2 worker 3 worker 4 worker N Database Observations and Monitoring monitor
29. supervisor worker 1 worker 2 worker 3 worker 4 worker N Database Observations and Monitoring
30. supervisor worker 1 worker 2 worker 3 worker 4 worker N I can’t reach this DB Database Observations and Monitoring
31. supervisor worker 1 worker 2 worker 3 worker 4 worker N Worker 4 monitor this DB Database Observations and Monitoring
32. supervisor worker 1 worker 2 worker 3 worker 4 worker N Database Observations and Monitoring monitor
33. worker Database Monitoring Last Observation cache postgres Heroku Postgres automation service GET /last_observation HSET db1 db_size 4213 HSET db1 cache_hit_rate 98
34. Redis at Heroku Other use cases
35. Router stats router node router node router node router node router node router node router node requests errors
36. pubsub + websockets dyno 1 dyno 2 dyno 3 pubsub browser websockets
37. I asked: what’s good and bad about Redis?
38. good bad •Fast •Simple •Powerful primitives •Expires on any data type •Many use cases, single infra •replication protocol is “surprisingly elegant” •No SSL support •Rotating creds is a pain •SPOF •Single threaded(ish)
39. Thanks! @hgmnz +