SpringOne 2gx 2012 : MongoDB and Spring Data

Spring and and MonogDB

Transcript

1. MongoDB and Spring Data Chris Harris Twitter : cj_harris5 Email : charris@10gen.com © 2012 SpringOne 2GX. All rights reserved. Do not distribute without permission. Tuesday, 16 October 12
2. 22 Solution Architect at 10gen (the company behind MongoDB) - Previous Roles: • EMEA Architect at SpringSource • EMEA Principal JBoss Consultant at RedHat About Me Tuesday, 16 October 12
3. http://images.mirror.co.uk/upl/m4/mar2009/0/2/satnav-cliff-hanger-pic-ross-parry- image-3-213726463.jpg Tuesday, 16 October 12
4. http://www.simpleadhdexpert.com/wp-content/uploads/2009/08/frustration.jpg Tuesday, 16 October 12
5. http://daddytypes.com/archive/duct_tape_baby_mianro.jpg Tuesday, 16 October 12
6. Is there a better way? Tuesday, 16 October 12
7. Terminology RDBMS MongoDB Table Collection Row(s) JSON Document Index Index Join Embedding & Linking Partition Shard Partition Key Shard Key Tuesday, 16 October 12
8. As simple as possible, but no simpler Depth of functionality Scalability & Performance Memcached Key / Value RDBMS MongoDB Tuesday, 16 October 12
9. Lets build a Spring Data MongoDB App! Tuesday, 16 October 12
10. Welcome to Dev News • User Story 1: – As a user I should be able to create “News Articles” Tuesday, 16 October 12
11. What is a News Article? { title: “What is new in 2.2 MongoDB”, date : new Date(), author : “Fred” text : “.......” } Tuesday, 16 October 12
12. Example 1 - Create Tuesday, 16 October 12
13. Welcome to MongoDB News Dev • User Story 1: – As a user I should be able to create “News Articles” • User Story 2: – As a user I should be able to find a “News Articles” by Name Tuesday, 16 October 12
14. Example 2 - Query Tuesday, 16 October 12
15. Example 3 - Repo Tuesday, 16 October 12
16. Welcome to MongoDB News Dev • User Story 1: – As a user I should be able to create “News Articles” • User Story 2: – As a user I should be able to find a “News Articles” by Name • User Story 3: – As a user I should be able to comment on “News Articles” Tuesday, 16 October 12
17. What is a Comment? { title: “What is new in 2.2 MongoDB”, date : new Date(), author : “Fred” text : “.......” tags : [“mongodb”, “nosql”] comments : [ {author : “chris”, text: “this is great” , date : new Date()}, {author : “tom”, text: “rock on”, date : new Date()} ] } Tuesday, 16 October 12
18. Example 4 - Comments Tuesday, 16 October 12
19. Welcome to MongoDB News Dev • User Story 1: – As a user I should be able to create “News Articles” • User Story 2: – As a user I should be able to find a “News Articles” by Name • User Story 3: – As a user I should be able to comment on “News Articles” • User Story 4: –As a user I want to know how many comments are on a “Articles” Tuesday, 16 October 12
20. What is a Comment? { title: “What is new in 2.2 MongoDB”, .... comments : [ {author : “chris”, text: “this is great” , date : new Date()}, {author : “tom”, text: “rock on”, date : new Date()} ] , comments: count : 2 } Tuesday, 16 October 12
21. Example 5 - Inc Tuesday, 16 October 12
22. “MongoDB lost my data!!” Tuesday, 16 October 12
23. Data: 125647383885969795743 Tuesday, 16 October 12
24. Least durability - Don't use! Driver MongoDB apply in memory write Tuesday, 16 October 12
25. Wait for Journal Sync - Same as RDBMS Driver MongoDB apply in memory write j:true Write to journal Tuesday, 16 October 12
26. Replication http://www.flickr.com/photos/10335017@N07/4570943043 Tuesday, 16 October 12
27. Replica Set • Data Protection • Multiple copies of the data • Spread across Data Centers, AZs • High Availability • Automated Failover • Automated Recovery Tuesday, 16 October 12
28. Primary Secondary Secondary Read Write Java App Asynchronous Replication Replica Sets Tuesday, 16 October 12
29. Replica Sets Primary Secondary Secondary Read Write Java App Tuesday, 16 October 12
30. Replica Sets Primary Primary Secondary Read Write Automatic Election of new Primary Java App Tuesday, 16 October 12
31. Replica Sets Recovering Primary Secondary Read Write New primary serves data Java App Tuesday, 16 October 12
32. Replica Sets Secondary Primary Secondary Read Write Java App Tuesday, 16 October 12
33. Durability Summary Memory Journal Secondary Other Data Center RDBMS Default "Fire & Forget" w=1 w=1 j=true w="majority" w=n w="myTag" Less More Tuesday, 16 October 12
34. Example 6 - Write Concern Tuesday, 16 October 12
35. Eventual Consistency • http://www.flickr.com/photos/26095468@N04/3779692985 Tuesday, 16 October 12
36. Understanding Eventual Consistency Primary Secondary v1 Thread #1 Insert Tuesday, 16 October 12
37. Understanding Eventual Consistency Primary Secondary Thread #1 Insert v1 v1 Tuesday, 16 October 12
38. Understanding Eventual Consistency Primary Secondary Thread #1 Insert Read v1 v1 Tuesday, 16 October 12
39. Understanding Eventual Consistency Primary Secondary Thread #1 Insert Read reads v1 v1 v1 Tuesday, 16 October 12
40. Understanding Eventual Consistency Primary Secondary Thread #1 Insert v2 Update Read reads v1 v1 v1 Tuesday, 16 October 12
41. Understanding Eventual Consistency Primary Secondary Thread #1 Insert Update Read v2 reads v1 v2 v1 v1 Tuesday, 16 October 12
42. Understanding Eventual Consistency Primary Secondary Thread #1 Insert Update Read Read reads v1 v2 v2 v1 v1 Tuesday, 16 October 12
43. Understanding Eventual Consistency Primary Secondary Thread #1 Insert Update Read Read reads v2 reads v1 v2 v2 v1 v1 Tuesday, 16 October 12
44. Understanding Eventual Consistency Primary Secondary Thread #1 Insert Update Read Read Thread #2 v2 v2 v1 v1 Tuesday, 16 October 12
45. Understanding Eventual Consistency Primary Secondary Thread #1 Insert Update Read Read Thread #2 v2 v2 v1 v1 Tuesday, 16 October 12
46. Understanding Eventual Consistency Primary Secondary Thread #1 Insert Update Read Read Thread #2 v2 v2 v1 v1 v1 does not exist on Secondary Tuesday, 16 October 12
47. Understanding Eventual Consistency Primary Secondary Thread #1 Insert Update Read Read Thread #2 v2 v2 v1 v1 v1 does not exist on Secondary Reads v1 but Primary at v2 Read v2 Tuesday, 16 October 12
48. Example 7 - Eventual Consistency Tuesday, 16 October 12
49. http://community.qlikview.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/theqlikviewblog/Cutting-Grass-with-Scissors-_2D00_-2.jpg Tuesday, 16 October 12
50. http://www.bitquill.net/blog/wp-content/uploads/2008/07/pack_of_harvesters.jpg Tuesday, 16 October 12
51. MongoDB Sharding • Automatic partitioning and management • Range based • Convert to sharded system with no downtime • Fully consistent Tuesday, 16 October 12
52. How mongoDB Sharding works • Range keys from -∞ to +∞ • Ranges are stored as "chunks" -∞ +∞ > db.runCommand({addshard: "shard1"}); > db.runCommand({shardCollection: "mydb.users", key: {age: 1}}) Tuesday, 16 October 12
53. How mongoDB Sharding works • Data in inserted • Ranges are split into more "chunks" -∞ +∞ -∞ 40 41 +∞ > db.users.save({age: 40}) Tuesday, 16 October 12
54. How mongoDB Sharding works -∞ +∞ -∞ 40 41 +∞ 41 50 51 +∞ 61 +∞ 51 60 > db.users.save({age: 40}) > db.users.save({age: 50}) > db.users.save({age: 60}) Tuesday, 16 October 12
55. -∞ 40 41 50 61 +∞ 51 60 > db.runCommand({addshard: "shard2"}); > db.runCommand({addshard: "shard3"}); How mongoDB Sharding works Tuesday, 16 October 12
56. -∞ 40 41 50 61 +∞ 51 60 shard1 > db.runCommand({addshard: "shard2"}); > db.runCommand({addshard: "shard3"}); How mongoDB Sharding works Tuesday, 16 October 12
57. -∞ 40 41 50 61 +∞ 51 60 shard1 shard2 shard3 > db.runCommand({addshard: "shard2"}); > db.runCommand({addshard: "shard3"}); How mongoDB Sharding works Tuesday, 16 October 12
58. Architecture C1 C2 C3 Config Servers mongos mongos app app mongod Shard 1 mongod mongod mongod Shard 2 mongod mongod Shard 4 mongod Shard 3 mongod mongod Replica Set mongod mongod mongod Tuesday, 16 October 12
59. Balancing config config config Shard 1 Shard 2 Shard 4 Shard 3 mongos balancer 5 9 1 6 10 2 7 11 3 8 12 4 13 14 15 16 17 18 19 20 21 22 23 24 Imbalance Data inserted all in a range mongos Tuesday, 16 October 12
60. Balancing config config config Shard 1 Shard 2 Shard 4 Shard 3 mongos balancer 5 9 1 6 10 2 7 11 3 8 12 4 13 14 15 16 17 18 19 20 21 22 23 24 Move chunk 1 to Shard 2 mongos Tuesday, 16 October 12
61. Balancing config config config Shard 1 Shard 2 Shard 4 Shard 3 balancer 5 9 6 10 2 7 11 3 8 12 4 13 14 15 16 17 18 19 20 21 22 23 24 1 mongos Tuesday, 16 October 12
62. Balancing config config config Shard 1 Shard 2 Shard 4 Shard 3 balancer 5 9 6 10 2 7 11 3 8 12 4 13 14 15 16 17 18 19 20 21 22 23 24 1 mongos Tuesday, 16 October 12
63. Balancing config config config Shard 1 Shard 2 Shard 4 Shard 3 mongos balancer 5 9 1 6 10 2 7 11 3 8 12 4 13 14 15 16 17 18 19 20 21 22 23 24 mongos Tuesday, 16 October 12
64. Example 8 - Sharding Tuesday, 16 October 12
65. • MongoDB 2.2 –Improved concurrency –Tag-aware sharding –Aggregation framework –TTL collections • Free online MongoDB training –Develop –Deploy –Classes start Oct. 2012 Questions Tuesday, 16 October 12