Why Do Databases Exist?

An overview of where databases came from, why they exist and why databases designed when the Internet had 20 nodes struggle to handle modern database workloads.

Latency is not 0 :-)

Transcript

1. Why Do Databases Exist? And what's wrong with them? Joe Drumgoole Senior Director of Developer Advocacy
2. Way Back in The Day
3. Reel To Reel Tape Drives
4. Original IBM RAMAC Disk Drive -1956
6. 6 But Speeds? Operation 1ns = 1sec Minute Hours Days Weeks Months execute typical instruction 1 fetch from L1 cache memory 0.5 branch misprediction 5 fetch from L2 cache memory 7 Mutex lock/unlock 25 fetch from main memory 1.67 send 2K bytes over 1Gbps network 6 read 1MB sequentially from memory 7 fetch from new disk location (seek) 3 read 1MB sequentially from disk 8
7. IBM 360 - 1964 -Multiprogramming Arrives
8. The IBM IMS Database Arrives - 1966 1966 - IBM delivers IMS, the world's first commercial database
9. A Simpler Time
10. A New World Order - Relational Databases - 1970
11. Tables
12. Indexes
13. Where does data come from - 1970?
14. The Other Source Of Data - 1971
15. The Internet - 1971
16. The 70's and 80's
17. Tabular Schema for Storing Ledgers
18. The World Turned Upside Down - 1990
19. Data is No Longer Textual
20. Big Data Problems
21. It’s a sensor world
22. Single Node Databases?
23. Modern Data Demands
24. NoSQL Volume Velocity Variety
25. Replica Sets
26. Typical Database Application Driver mongod /data
27. 27 Replica Set
28. 28 Normal Operation
29. 29 Primary Failure electionTimeoutMilli s
30. 30 Failover
31. 31 Recovery
32. 32 Recovery Complete
33. Sharding
34. Replica Set with MongoDB Application Driver Primary /data Secondary /data Secondary /data
35. Replica Set Bottlenecks Application Driver Primary /data Secondary /data Secondary /data RAM Limits on single server CPU Limits on single server Network Bandwidth Disk I/O
36. What Is Sharding? Application mongos mongos mongos Driver
37. But There is More Application mongos mongos mongos Driver Config Server
38. Types of NoSQL Database
39. 39 Key Value Stores • An associative array • Single key lookup • Very fast single key lookup • Not so hot for “reverse lookups” Key Value 12345 4567.3456787 12346 { addr1 : “The Grange”, addr2: “Dublin” } 12347 “top secret password” 12358 “Shopping basket value : 24560” 12787 12345
40. 40 Revision : Row Stores (RDBMS) ID Name Salary Start Date 1 Joe D $24000 1/Jun/1970 2 Peter J $28000 1/Feb/1972 3 Phil G $23000 1/Jan/1973 1 Joe D $24000 1/Jun/1970 2 Peter J $28000 1/Feb/1972 3 Phil G $23000 1/Jan/1973
41. 41 How a Column Store Does it 1 2 3 ID Name Salary Start Date 1 Joe D $24000 1/Jun/1970 2 Peter J $28000 1/Feb/1972 3 Phil G $23000 1/Jan/1973 Joe D Peter J Phil G $24000 $28000 $23000 1/Jun/1970 1/Feb/1972 1/Jan/1973
42. 42 Document Store { name : “Joe Drumgoole”, title : “Director of Developer Advocacy”, Address : { address1 : “Latin Hall”, address2 : “Golden Lane”, eircode : “D09 N623”, } expertise: [ “MongoDB”, “Python”, “Javascript” ], employee_number : 320, location : [ 53.34, -6.26 ] }
43. Q & A