Leaderboards - A Practical Use-Case for Redis

In this talk, I'll present a practical use-case for Redis, leaderboards (aka scoreboards aka high score tables). The pain and frustration of having to engineer leaderboards using a traditional database had me constantly looking for a more practical, simpler and more scalable solution. I will cover the commands in Redis to support leaderboards, a high-level library wrapping the Redis commands, use-cases for leaderboards, performance metrics, and a very interesting developer thought experiment that arose from developing the library.

Transcript

1. Leaderboards A Practical Use-Case for Redis
2. @czarneckid
3. @agoragames Guitar Hero, Mortal Kombat, Brink, and more
4. @mlg U MAD?
5. Leaderboards
6. Data structure server http://redis.io
7. New Years Eve 2010 the impetus
8. MySQL leaderboards the disease
9. Redis sorted sets the cure
10. Leaderboard gem https://github.com/agoragames/leaderboard/
11. Leaderboard gem • Semantic wrapper around Redis sorted sets • Overall data • Total Members, Total Pages • Individual data • Rank, Find Rank, Find Score, Change Score, Check Member, Remove Member • Group data • Leaders, “Around Me”, Arbitrary List
12. ruby-1.9.2-p180 :002 > highscore_lb = Leaderboard.new('highscores') => #<Leaderboard:0x0000010307b530 @leaderboard_name="highscores", @page_size=25, @redis_connection=#<Redis client v2.2.2 connected to redis:// localhost:6379/0 (Redis v2.2.5)>> Create a leaderboard
13. ruby-1.9.2-p180 :005 > 1.upto(10) do |index| ruby-1.9.2-p180 :006 > highscore_lb.rank_member("member_#{index}", index) ruby-1.9.2-p180 :007?> end => 1 Add players to the leaderboard
14. ruby-1.9.2-p180 :008 > highscore_lb.total_members => 10 ruby-1.9.2-p180 :009 > highscore_lb.total_pages => 1 Get information about the leaderboard
15. ruby-1.9.2-p180 :010 > highscore_lb.score_for('member_4') => 4.0 ruby-1.9.2-p180 :011 > highscore_lb.rank_for('member_4') => 7 ruby-1.9.2-p180 :012 > highscore_lb.rank_for('member_10') => 1 Get some information about specific members
16. ruby-1.9.2-p180 :013 > highscore_lb.leaders(1) => [{:member=>"member_10", :rank=>1, :score=>10.0}, {:member=>"member_9", :rank=>2, :score=>9.0}, {:member=>"member_8", :rank=>3, :score=>8.0}, {:member=>"member_7", :rank=>4, :score=>7.0}, {:member=>"member_6", :rank=>5, :score=>6.0}, {:member=>"member_5", :rank=>6, :score=>5.0}, {:member=>"member_4", :rank=>7, :score=>4.0}, {:member=>"member_3", :rank=>8, :score=>3.0}, {:member=>"member_2", :rank=>9, :score=>2.0}, {:member=>"member_1", :rank=>10, :score=>1.0}] Get page number 1 from the leaderboard
17. ruby-1.9.2-p180 :014 > highscore_lb.leaders (1, :with_scores => false, :with_rank => false) => [{:member=>"member_10"}, {:member=>"member_9"}, {:member=>"member_8"}, {:member=>"member_7"}, {:member=>"member_6"}, {:member=>"member_5"}, {:member=>"member_4"}, {:member=>"member_3"}, {:member=>"member_2"}, {:member=>"member_1"}] Get page number 1 from the leaderboard (without scores or ranks)
18. ruby-1.9.2-p180 :015 > 50.upto(95) do |index| ruby-1.9.2-p180 :016 > highscore_lb.rank_member("member_#{index}", index) ruby-1.9.2-p180 :017?> end => 50 ruby-1.9.2-p180 :018 > highscore_lb.total_pages => 3 Add more members to the leaderboard
19. ruby-1.9.2-p180 :019 > highscore_lb.around_me('member_53') => [{:member=>"member_65", :rank=>31, :score=>65.0}, {:member=>"member_64", :rank=>32, :score=>64.0}, {:member=>"member_63", :rank=>33, :score=>63.0}, {:member=>"member_62", :rank=>34, :score=>62.0}, {:member=>"member_61", :rank=>35, :score=>61.0}, {:member=>"member_60", :rank=>36, :score=>60.0}, {:member=>"member_59", :rank=>37, :score=>59.0}, {:member=>"member_58", :rank=>38, :score=>58.0}, {:member=>"member_57", :rank=>39, :score=>57.0}, {:member=>"member_56", :rank=>40, :score=>56.0}, {:member=>"member_55", :rank=>41, :score=>55.0}, {:member=>"member_54", :rank=>42, :score=>54.0}, {:member=>"member_53", :rank=>43, :score=>53.0}, {:member=>"member_52", :rank=>44, :score=>52.0}, {:member=>"member_51", :rank=>45, :score=>51.0}, {:member=>"member_50", :rank=>46, :score=>50.0}, {:member=>"member_10", :rank=>47, :score=>10.0}, {:member=>"member_9", :rank=>48, :score=>9.0}, {:member=>"member_8", :rank=>49, :score=>8.0}, {:member=>"member_7", :rank=>50, :score=>7.0}, {:member=>"member_6", :rank=>51, :score=>6.0}, {:member=>"member_5", :rank=>52, :score=>5.0}, {:member=>"member_4", :rank=>53, :score=>4.0}, {:member=>"member_3", :rank=>54, :score=>3.0}, {:member=>"member_2", :rank=>55, :score=>2.0}] Get an “Around Me” leaderboard
20. Get rank and score information for an arbitrary list (e.g. friends) ruby-1.9.2-p180 :020 > highscore_lb.ranked_in_list(['member_1', 'member_62', 'member_67']) => [{:member=>"member_1", :rank=>56, :score=>1.0}, {:member=>"member_62", :rank=>34, :score=>62.0}, {:member=>"member_67", :rank=>29, :score=>67.0}]
21. Performance Metrics • Rank 10 million players with sequential scores • 15 minutes or ~10,600 writes/sec* • Request an arbitrary page from the leaderboard (50,000 x) • 0.001513 seconds on average • Rank 10 million players with random scores • 36 minutes or ~4,500 writes/sec* • Request an arbitrary page from the leaderboard (50,000 x) • 0.001461 seconds on average *Numbers obtained while working so YMMV
22. Leaderboard Usage • Agora Games middleware platform • GameBattles • http://gamebattles.majorleaguegaming.com/ leaderboards • activity_feed gem • https://github.com/agoragames/ activity_feed
23. Summary • Redis • Data structure server • Sorted sets are useful • Leaderboard gem • Wraps low-level Redis commands • Sorted sets are more useful • Performance • Redis is fast • Usage • Real-world and not an “experiment”
24. Developer Experiment on the cheap
25. Porting Leaderboard • php-leaderboard • https://github.com/agoragames/php-leaderboard • java-leaderboard • https://github.com/agoragames/java-leaderboard • scala-leaderboard • https://github.com/agoragames/scala-leaderboard • python-leaderboard • https://github.com/agoragames/python-leaderboard • erlang-leaderboard • https://github.com/agoragames/erlang-leaderboard • node-leaderboard • https://github.com/omork/node-leaderboard client library, data structures, iteration, TDD, packaging
26. @czarneckid