Data Modeling in RethinkDB

Transcript

1. RethinkDB Meetup San Francisco, California June 29, 2015 Data Modeling in RethinkDB
2. Jorge Silva @thejsj Developer Evangelist @ RethinkDB
3. Introduction Data modeling in other databases
4. SQL • Very basic data types: strings, numbers, and date/time • All data is saved in rows under a predefined column (schema) • Complex data is modeled through relations (foreign keys)
5. SQL • Relations favor data spread across multiple tables • Limited data types favor very normalized data
6. SQL Family Tree People id name 1 Darth 2 Luke 3 Leia ancestor descendant 1 1 1 2 1 3 TreePaths
7. MongoDB • JSON support reduces the need for spreading data across tables • Joins happen at the application layer • Lack of relations and joins favors storing more data on a single document
8. MongoDB Family Tree { "id": 1, "name": "Darth", "children": [ { "name": "Luke" }, { "name": "Leia" } ] }, { "id": 2, "name": "Luke" }, { "id": 3, "name": "Leia", }
9. MongoDB Family Tree { "id": 1, "name": "Darth", "children": [ 2, 3 ] }, { "id": 2, "name": "Luke" }, { "id": 3, "name": "Leia", } • Array of ids, joined at application level
10. RethinkDB • RethinkDB allows for both: SQL style data modeling through joins and MongoDB style modeling through subdocuments
11. Data Types • The introduction of arrays and objects completely changes how you structure your data • Arrays reduce the need for intersection tables • Objects allow for nested subdocuments and easy lookup
12. RethinkDB vs SQL • Even though RethinkDB has joins, it has no relations. • There is no concept of a foreign key or cascading queries at a database-level • Schema enforcement has to be done manually
13. Data Modeling in RethinkDB • Data modeling in RethinkDB is incredibly flexible. This flexibility may seem daunting at first • What you already know is applicable
14. Data Modeling in RethinkDB You still need to think about your data
15. Thinking about your database Tradeoffs
16. Main Tradeoff • Do I store this related piece of data as a subdocument or do I save it in a different table? • What you do depends on your use case
17. Structure and Performance Considerations
18. How to decide what to do? • The format and structure of your data is not the only consideration • RethinkDB's flexibility allows you to optimize for ease-of-use and performance
19. Considerations • Structure of your data • Joins in query • Atomic updates
20. Structure of your data • Models should still be simple, faithful and avoid redundancy • Data should still be normalized and follow the three normal forms • Only break these rules for denormalization, but don't optimize prematurely
21. Joins in query • How you read your data is important • If you always read your data in conjunction, you should store data in the same document • If data is read separately, it should be store in different tables
22. Joins in query: Example • Example: You have 'cities' and 'states' and you need to store the relationships between them
23. Joins in query: Example • Options: • Have an intersection table • Store the whole city as a subdocument • Store the city IDs in the document
24. Joins in query: Example // States { "id": 1, "name": "California" }, { "id": 2, "name": "Oregon" } // Cities { "id": 1, "name": "Portland" }, { "id": 2, "name": "Salem" }, { "id": 3, "name": "San Francisco" }, { "id": 4, "name": "Mountain View" } // State-to-city { "state": 1, "city": 3 }, { "state": 1, "city": 4 }, { "state": 2, "city": 1 }, { "state": 2, "city": 2 }
25. Joins in query: Example # Reading all cities in state r.table("states") .get(stateId).merge({ "cities": r.table("cities") .get_all(r.args(r.table("state-to-city") .get_all(stateId, index="state")["city"] )) }) # Reading one city r.table("cities") .get_all("Mountain View", index="name")
26. Joins in query: Example # Adding a state r.table("states") .insert({ "name": "Washington" }) # Adding a city r.table("cities") .insert({ "name": "San Jose" }) r.table("state-to-city") .insert({ "state": stateId, "city": cityId })
27. Joins in query: Example // States { "id": 1, "name": "California", "cities": [ { "name": "San Francisco" }, { "name": "Mountain View" } ] }, { "id": 2, "name": "Oregon", "cities": [ { "name": "Portland" }, { "name": "Salem" } ] }
28. Joins in query: Example # Getting a state with cities r.table("states") .get(stateId) # Getting a city r.table("states") .filter(lambda row: row["cities"]["name"] == "San Jose" )[0] .map(lambda row: row.filter(lambda city: city["name"] == "San Jose") )
29. Joins in query: Example # Adding a state r.table("states") .insert({ "name": "Washington", "cities": [] }) # Adding a city r.table("states") .get(stateId) .update({ "cities": r.row["cities"].append({ "name": "San Jose" }) })
30. Joins in query: Example // States { "id": 1, "name": "California", "cities": [ 3, 4 ] }, { "id": 2, "name": "Oregon", "cities": [ 1, 2 ] } // Cities { "id": 1, "name": "Portland" }, { "id": 2, "name": "Salem" }, { "id": 3, "name": "San Francisco" }, { "id": 4, "name": "Mountain View" }
31. Joins in query: Example # Getting a state with cities r.table("states") .get(stateId).merge({ "cities": r.table("cities") .get_all(r.row["cities"]) }) # Getting a city r.table("cities") .get_all("Mountain View", index="name")
32. Joins in query: Example # Adding a state r.table("states") .insert({ "name": "Washington", "cities": [] }) # Adding a city r.table("cities") .insert({ "name": "San Jose" }) r.table("states") .get(stateId).update({ "cities": r.row["cities"].append(cityId) })
33. Joins in query: Example # Creating an index for cities r.table("states") .index_create("city", lambda row: row["cities"]["name"], multi=True) # Query city using index r.table("states") .get_all("San Francisco", index="city") Using an index for subdocument
34. Atomic Updates • Because RethinkDB is distributed, atomic updates can only be guaranteed on a per-document basis • If you need atomic updates, you must keep everything in one document
35. • Example: You have an 'orders' table with an order id and an array of products Atomic Updates: Example
36. • Options: • Store everything in one document to make the update atomic • Store it in separate documents and give up atomic guarantees Atomic Updates: Example
37. Atomic Updates: Example # Update total and products in order r.table("orders").get(order_id).update({ "products": r.row["products"].append(product['id']), "total_price": r.row["total_price"] + product["price"]) }) • The complete query is guaranteed to succeed or fail
38. Atomic Updates: Example # Update total and products in order r.table("orders").get(order_id).update({ "products": r.row["products"].append(product['id']), "total_price": r.row["total_price"] + product["price"]) }) • The complete query is guaranteed to succeed or fail
39. Atomic Updates: Example # Update total and products in order r.table("orders").get(order_id).update({ "products": r.row["products"].append(product['id']), "total_price": r.row["total_price"] + product["price"]) }) • The complete query is guaranteed to succeed or fail
40. Atomic Updates: Example # Update total and products in order r.table("orders").get(order_id).update({ "products": r.row["products"].append(product['id']), "total_price": r.row["total_price"] + product["price"]) }) • The complete query is guaranteed to succeed or fail
41. Atomic Updates: Example # Update total in order r.table('orders').get(order_id).update({ "total_amount": r.row["total_amount"] + product["price"] }) .do(lambda result: # Update products in intersection table # If this fails, the `update` won't be unapplied r.table("orders_products").insert({ "order": order_id, "product": product["id"] }) ) • The complete query is not guaranteed to succeed or fail
42. Atomic Updates: Example # Update total in order r.table('orders').get(order_id).update({ "total_amount": r.row["total_amount"] + product["price"] }) .do(lambda result: # Update products in intersection table # If this fails, the `update` won't be unapplied r.table("orders_products").insert({ "order": order_id, "product": product["id"] }) ) • The complete query is not guaranteed to succeed or fail
43. Atomic Updates: Example # Update total in order r.table('orders').get(order_id).update({ "total_amount": r.row["total_amount"] + product["price"] }) .do(lambda result: # Update products in intersection table # If this fails, the `update` won't be unapplied r.table("orders_products").insert({ "order": order_id, "product": product["id"] }) ) • The complete query is not guaranteed to succeed or fail
44. Atomic Updates: Example # Update total in order r.table('orders').get(order_id).update({ "total_amount": r.row["total_amount"] + product["price"] }) .do(lambda result: # Update products in intersection table # If this fails, the `update` won't be unapplied r.table("orders_products").insert({ "order": order_id, "product": product["id"] }) ) • The complete query is not guaranteed to succeed or fail
45. Summary • Data modeling in RethinkDB is very flexible • The main tradeoff is subdocuments vs joins • How you model your data will depend heavily on how you interact with the database
46. Summary Think about your data!
47. Questions?