Rebuilding Twitter with Cassandra and Ruby

Presented at WIDS 2014 in Hong Kong by Matthew Rudy Jacobs

Check out my Twitter implementation https://github.com/matthewrudy/twissandra-rb

Transcript

1. Rebuilding Twitter with Cassandra Friday 28th March @ WIDS 2014, Hong Kong Matthew Rudy Jacobs
2. @matthewrudy
3. Let’s rebuild twitter!
4. matthewrudy/ twissandra-rb
5. a Ruby re- implementation of Twissandra
6. Fork it!
7. What is Twitter?
8. People
9. Tweets
10. Friends
11. Followers
12. Timeline
13. A whole load of other complex stuff
14. Complex stuff • @mentions • replies • retweets • favourites • sources • the streaming API
15. What’s so hard about that?
16. Nothing
17. Let’s do it in SQL
18. Relational Approach class User has_many :friendships has_many :friends, through: :friendships has_many :tweets has_many :timeline_tweets, through: :friends end ! me = User.find_by(username: “me”) me.timeline_tweets.order(“created_at DESC”).limit(20)
19. BOOM!
20. We’re done!
21. Thanks
22. Just kidding
23. When stuff gets big?
24. We have 2 problems
25. Throughput
26. Add Read Slaves Tweet slave #1 Tweet master Tweet slave #2 Client
27. That’s fine
28. Data size
29. 10 TB of data? • Heroku Postgres has a 1TB limit • RDS Postgres has a 3TB limit • Amazon I2 8xlarge has a 6TB limit • buy a massive 12TB disk array?
30. Shard it? Tweets#1 Tweets#2 Tweets#3 Client id % 3 == 0 id % 3 == 2 id % 3 == 1 Tweet id generator
31. Joins?
32. NO!
33. Joins don’t scale!
34. You can scale SQL
35. But there’s a whole load of caveats
36. “On the shoulders of giants”
37. Cassandra
38. NoSQL?
39. No, CQL!
40. CREATE TABLE CREATE TABLE tweets ( id timeuuid, user_id uuid, body text, mentions set<uuid>, ! PRIMARY KEY (id) ); ! CREATE INDEX ON tweets (user_id);
41. INSERT INTO INSERT INTO tweets (id, user_id, body) VALUES (1, 1337, ‘This is my first tweet!’);
42. SELECT FROM SELECT * FROM tweets WHERE user_id = 1337;
43. It’s awesome for timelines
44. Timelines CREATE TABLE timeline ( user_id uuid, tweet_id timeuuid, ! PRIMARY KEY (user_id, tweet_id) ) WITH CLUSTERING ORDER BY (tweet_id DESC); ! SELECT * FROM timeline WHERE user_id = 1337 AND tweet_id < minTimeuuid('2013-02-02 10:00+0000’);
45. “it’s like SQL, but without all the stuff that doesn’t scale”
46. Types
47. Types • integer - 1234567 • long - 123.456 • text - ‘abc123’ • uuid - {756716f7-2e54-4715-9f00-91dcbea6cf50} • timeuuid - NOW() • timestamp - ‘2013-06-13 11:00:00'
48. TimeUUID is awesome!
49. Collections • Sets - {‘dog’, ‘cat’, ‘elephant’} • Lists - [‘AMZN’, ‘AAPL’, ‘FB’] • Maps - { ‘GOOG': 1200, ‘AAPL’: 512}
50. Sets CREATE TABLE friends ( user_id uuid, friend_ids set<uuid>, ! PRIMARY KEY (user_id) ); ! UPDATE users SET friend_ids = friend_ids + {8007} WHERE user_id = 1337;
51. Counters CREATE TABLE user_tweet_counts ( user_id INT, tweet_count COUNTER, ! PRIMARY KEY (user_id) ); ! UPDATE user_tweet_counts SET count = count + 1 WHERE user_id = 1337; ! INSERT INTO tweets (id, user_id, body) VALUES (NOW(), 1337, ‘some tweet’);
52. Distribution Model
53. Consistent Hashing Tweet A B C D E F G H Murmur3 Partitioner
54. Distributed A B C D E F G H A B C D G H A B C D E F A B C D E F G H E F G H replication factor =3
55. Throughput, Redundancy, Availability
56. Write - Quorum Node 1 Node 2 Node 3 Client
57. Read - Quorum Node 1 Node 2 Node 3 Client
58. Write - ALL Node 1 Node 2 Node 3 Client
59. Read - ONE Node 1 Node 2 Node 3 Client
60. The Accidental Win
61. Singapore Multi-region master master A B C D E F G H Virginia A B C D E F G H HK user SF user
62. BOOM!
63. Finally, let’s code!
64. NoSQL needs planning first!
65. P.E.Q.S.
66. P.E.Q.S. • Processes • Entities • Queries • Schema
67. Processes • Create a User • Send a Tweet • Read my Timeline • Follow a User • Unfollow a User
68. Create a User • store User details
69. Send a Tweet • create a Tweet for Me • add Tweet to my Followers Timelines • add Tweet to *mentioned* Users' Timeline
70. Read my Timeline • find Tweets from my Timeline • load the Tweet details • load the User details
71. Follow a User • add User to my Friends • add Me to the User's Followers • add User's Tweets to my Timeline
72. Unfollow a User • remove User from my Friends • remove Me from User's Followers • remove User's Tweets from my Timeline
73. Entities • Users • Tweets • Userline - tweets from the user • Timeline - tweets for the user to see • Friends - people the user follows • Followers - people who follow the user
74. Users • id - used for references • username - used for display - UNIQUE • metadata - location, photos and stuff
75. Tweets • id - used for references • user_id - the user who tweeted • body - the text of the tweet • mentions - a list of users mentioned
76. Userline • user_id - the user who tweeted • tweet_id - id of the tweet - UNIQUE • timestamp - when it was tweeted
77. Timeline • user_id - the user who sees this • tweet_id - id of the tweet - UNIQUE • timestamp - when it was tweeted
78. Friends • user_id - the user • friend_id - the person they follow • timestamp - when they were followed
79. Followers • user_id - the user • follower_id - the person who follows them • timestamp - when they were followed
80. Queries
81. Users • create - insert into database • find_by_username - find @matthewrudy • find_all_by_id - find all users for a set of tweets • UNIQUE by username
82. Tweets • create • find_ by_id • UNIQUE by id
83. Userline • add_tweet_for_user • find_all_by_user • ORDER BY timestamp DESC • UNIQUE by {user, tweet}
84. Timeline • add_tweet_for_user • find_all_by_user • ORDER BY timestamp DESC • UNIQUE by {user, tweet} • delete_all_by_tweet_user
85. Friends • find_all_by_user • create • UNIQUE by {user, friend}
86. Followers • find_all_by_user • create • UNIQUE by {user, follower}
87. Schema
88. Keyspace CREATE KEYSPACE twissandra WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '1' };
89. Users CREATE TABLE users ( id uuid, username text, location text, ! PRIMARY KEY (id) ); ! CREATE INDEX ON users (username);
90. Tweets CREATE TABLE tweets ( id timeuuid, // unique id with timestamp user_id uuid, body text, mentions set<uuid>, ! PRIMARY KEY (id) );
91. Userline CREATE TABLE userline ( user_id uuid, tweet_id timeuuid, ! PRIMARY KEY (user_id, tweet_id) ) ! WITH CLUSTERING ORDER BY (tweet_id DESC);
92. Timeline CREATE TABLE timeline ( user_id uuid, tweet_id timeuuid, tweet_user_id uuid, ! PRIMARY KEY (user_id, tweet_id) ) ! WITH CLUSTERING ORDER BY (tweet_id DESC);
93. Friends CREATE TABLE friends ( user_id uuid, friend_id uuid, timestamp timestamp, ! PRIMARY KEY (user_id, friend_id) );
94. Followers CREATE TABLE followers ( user_id uuid, follower_id uuid, timestamp timestamp, ! PRIMARY KEY (user_id, follower_id) );
95. the code writes itself, right?
96. user.rb
97. tweet.rb
98. userline.rb
99. timeline.rb
100. friend.rb
101. follower.rb
102. create_user.rb
103. send_tweet.rb
104. read_timeline.rb
105. follow_user.rb
106. unfollow_user.rb
107. Thanks
108. @matthewrudy