Houston.js - October 2016 - I Have a NoSQL Toaster

My toaster stores data without SQL and without tables. But making a choice based on what something doesn’t have isn’t terribly useful. “NoSQL” is an increasingly inaccurate catch-all term that covers a lot of different types of data storage. Let’s make more sense of this new breed of database management systems and go beyond the buzzword. In this session, the four main data models that make up the NoSQL movement will be covered: key-value, document, columnar and graph. How they differ and when you might want to use each one will be discussed.

This session will be looking at the whole ecosystem, with a more detailed focus on Couchbase, Cassandra, Riak KV, and Neo4j.

Transcript

1. I have a NoSQL Toaster. Why would I want a NoSQL database? Matthew D. Groves @mgroves Original slides by Matthew Revell - @matthewrevell
2. ©2016 Couchbase Inc. 2 Who am I? • Matthew D. Groves • Developer Advocate for Couchbase • @mgroves on Twitter • Podcast and blog: http://crosscuttingconcerns.com • “I am not an expert, but I am an enthusiast.” – Alan Stevens
3. ©2016 Couchbase Inc. 3 Tweet something, get a sticker! Use #Couchbase Tweet something interesting you’ve learned. Tweet a picture. I’ve got a few stickers! (greedy Austin devs took most of them)
4. ©2016 Couchbase Inc. 4 Couchbase and CouchDB Couchbase is to CouchDB as MySQL is to SQL Server
5. ©2016 Couchbase Inc. 5 SQL is the norm, right? https://commons.wikimedia.org/wiki/File:George_Wendt_at_the_41st_Emmy_Awards_cropped.jpg
6. ~3200 BC: invention of writing in Mesopotamia ~300 BC: Library of Alexandria ~1041 AD: Movable type invented in China 1450: Movable type invented in Europe 1822: Babbage’s paper on the application of difference engines 1943: Colossus, the first programmable digital computer 1957-1960: IBM SABRE, the first commercial use of a database 1970: EF Codd proposes the relational database model 1974: Ingres released 1979: Commercial launch of Oracle database 1988: Object databases appear 1989: Microsoft SQL Server version 1 1991: HTTP v0.9 1995: MySQL’s initial release 2005: CouchDB released 2006: Google’s Big Table paper 2007: Amazon’s Dynamo paper 2008-2009: The NoSQL Cambrian explosion: Riak, MongoDB, Cassandra, Redis 2010: Couchbase arrives on the scene
7. NoSQL isn’t a very useful term https://commons.wikimedia.org/wiki/File:Toaster.jpg
8. ©2016 Couchbase Inc. 8 Easy?
9. ©2016 Couchbase Inc. 9 Scalable? https://commons.wikimedia.org/wiki/File:Dagwood_sandwich.jpg
10. ©2016 Couchbase Inc. 10 Commodity Hardware https://commons.wikimedia.org/wiki/File:Rear_of_rack_at_NERSC_data_center_-_closeup.jpg
11. ©2016 Couchbase Inc. 11 ACID vs BASE? https://commons.wikimedia.org/wiki/File:PH_Scale.svg Atomic Consistent Isolated Durable Basic Availability Soft-state Eventual consistency
12. ©2016 Couchbase Inc. 12 Schema Schemaless https://en.wikipedia.org/wiki/File:Star-schema-example.png
13. ©2016 Couchbase Inc. 13 Normalized Denormalized https://commons.wikimedia.org/wiki/File:Frown.JPG
14. ©2016 Couchbase Inc. 14 First: Why NoSQL? https://www.flickr.com/photos/wonderlane/13067014944
15. ©2016 Couchbase Inc. 15 Round pegs, square holes http://www.timesofbook.com/2013/09/blobfish-image-gallery.html#.V6iXwbgrJhE
16. ©2016 Couchbase Inc. 16 Perhaps SQL is right for this project
17. ©2016 Couchbase Inc. 17 Scaling can be hard https://commons.wikimedia.org/wiki/File:Sandwiches_Vienna.jpg
18. ©2016 Couchbase Inc. 18 Availability is hard https://www.flickr.com/photos/jeepersmedia/14562425273/ C A P
19. ©2016 Couchbase Inc. 19 Schemas can be hard https://commons.wikimedia.org/wiki/File:MediaWiki_database_schema_1-17_(r82044).svg
20. ©2016 Couchbase Inc. 20 Types of NoSQL databases Document Key/Value Columnar Graph
21. ©2016 Couchbase Inc. 21 Convergence!
22. ©2016 Couchbase Inc. 22 Document
23. ???
24. ©2016 Couchbase Inc. 24 What makes it a document database?
25. JSON XML BSON
27. ©2016 Couchbase Inc. 27 The document database understands the format of the document
28. ©2016 Couchbase Inc. 28 It can do server-side stuff
29. Buckets Documents Views N1QL
30. "ClientProductCode","Title","Quantity" "TShirtCBS","Couchbase logo t-shirt (small)",29 "TShirtCBM","Couchbase logo t-shirt (medium)",72 "TShirtCBL","Couchbase logo t-shirt (large)",34 "TShirtCBXL","Couchbase logo t-shirt (extra large)",12 "StickerCBLogo","Couchbase logo sticker",256 "PenCBLogo","Couchbase logo pen",365
31. class Program { private static IBucket _bucket; static void Main(string[] args) { SetupCouchbase(); LoadStockList(); ClusterHelper.Close(); } private static void SetupCouchbase() { ClientConfiguration config = new ClientConfiguration(); config.Servers = new List<Uri> {new Uri("couchbase://192.168.1.5") }; ClusterHelper.Initialize(config); _bucket = ClusterHelper.GetBucket("default"); } private static void LoadStockList() { var csv = new CsvReader(File.OpenText("swag.csv")); while (csv.Read()) { var record = csv.GetRecord<dynamic>(); var document = new Document<dynamic> { Id = record.ClientProductCode, Content = new { record.Title, record.Quantity } }; _bucket.Insert(document); Console.WriteLine($"Added document '{record.ClientProductCode}'"); } } }
32. private static void SetupCouchbase() { ClientConfiguration config = new ClientConfiguration(); config.Servers = new List<Uri> {new Uri("couchbase://192.168.1.5") }; ClusterHelper.Initialize(config); _bucket = ClusterHelper.GetBucket("default"); }
33. private static void LoadStockList() { var csv = new CsvReader(File.OpenText("swag.csv")); while (csv.Read()) { var record = csv.GetRecord<dynamic>(); var document = new Document<dynamic> { Id = record.ClientProductCode, Content = new { record.Title, record.Quantity } }; _bucket.Insert(document); Console.WriteLine($"Added document '{record.ClientProductCode}'"); } }
35. private static void GetDocument() { var doc = _bucket.Get<dynamic>("StickerCBLogo"); Console.WriteLine($"Document Key: {doc.Id}"); Console.WriteLine($"Title: {doc.Value.title}"); Console.WriteLine($"Quantity: {doc.Value.quantity}"); }
37. ©2016 Couchbase Inc. 37 Typical Document Database Operations • Upsert (aka set) • Inserts document if key doesn’t exist, replaces otherwise • Insert (aka add) • Inserts document, error if it already exists • Update (aka replace) • Updates document, error if it doesn’t exist • Delete • Get
38. ©2016 Couchbase Inc. 38 N1QL SELECT m.* FROM `default` m WHERE META(m).id = ‘StickerCBLogo’ SELECT m.* FROM `default` m WHERE m.quantity > 10
39. Great for Dev Scales Easily Consistently Fast
40. ©2016 Couchbase Inc. 40 Use cases for Document Databases • User profiles • Session stores • Content management • General relational replacement • http://info.couchbase.com/15Q1TopTenUC.html
41. ©2016 Couchbase Inc. 41 Key/Value Database
43. ©2016 Couchbase Inc. 43 Key-Value Databases In-memory caches and distributed data stores
44. ©2016 Couchbase Inc. 44 Doesn’t care what your data is (mostly) KEY1 { type: “json” } KEY2 <data type=“xml”></data> KEY3 Lorem ipsum … 00100101000100101 KEYN https://www.flickr.com/photos/dcmot/23168680069
45. Buckets Key-Value Pairs
46. ©2016 Couchbase Inc. 46 C# Riak private static void DoStuffWithRiak() { var cluster = RiakCluster.FromConfig("riakConfig"); var client = cluster.CreateClient(); var actor = new RiakObject("actors", "James Bond","Sean Connery"); client.Put(actor); var actorBackOut = client.Get("actors", "James Bond"); var str = System.Text.Encoding.UTF8.GetString(actorBackOut.Value.Value); Console.WriteLine($"Value: {str}"); }
48. ©2016 Couchbase Inc. 48 Introducing: Eventual Consistency!
49. ©2016 Couchbase Inc. 49 Eventual Consistency https://www.flickr.com/photos/scottchene/7046992749 https://commons.wikimedia.org/wiki/File:Timothy_Dalton_1987.jpg https://commons.wikimedia.org/wiki/File:Sir_Roger_Moore_crop.jpg
50. ©2016 Couchbase Inc. 50 There can be only one! https://en.wikipedia.org/wiki/File:Highlander_film_Connor_MacLeod.jpg
51. ©2016 Couchbase Inc. 51 So, fix it https://www.flickr.com/photos/94086507@N00/84368105
52. Scales Well Highly Available Data Agnostic Maybe some extra work
53. ©2016 Couchbase Inc. 53 Use Cases for K/V • Data variability • Object caching • Session storage • Large object storage
54. ©2016 Couchbase Inc. 54 Columnar
56. Keyspaces Column Families Rows Columns
57. ©2016 Couchbase Inc. 57 Keyspace
58. ©2016 Couchbase Inc. 58 Column Family
59. CREATE KEYSPACE mykeyspace WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '1'}; CREATE TABLE users (firstname text, lastname text, age int, email text, city text, PRIMARY KEY (lastname));
60. private static void DoStuffWithDataStax() { var cluster = Cluster.Builder().AddContactPoint("127.0.0.1").Build(); var session = cluster.Connect("mykeyspace"); session.Execute("insert into users (lastname, age, city, email, firstname) values ('Jones', 35, 'Austin', 'bob@example.com', 'Bob')"); var userBackOut = session.Execute("select * from users where lastname='Jones'").First(); Console.WriteLine($"{userBackOut["firstname"]} {userBackOut["age"]}"); }
62. Impedance Mismatch Scales Well (Ops Headache) Suited to Analytics
63. ©2016 Couchbase Inc. 63 Use cases for Columnar •Metering data •Really big data •Analytics
64. ©2016 Couchbase Inc. 64 Graph https://commons.wikimedia.org/wiki/File:The_protein_interaction_network_of_Treponema_pallidum.png
65. ©2016 Couchbase Inc. 65 Leonhard Euler
66. ©2016 Couchbase Inc. 66 7 Bridges of Königsberg
68. ©2016 Couchbase Inc. 68 Use cases for Graph databases • Social networks / dating sites • Fraud detection • Parcel routing • Shopping recommendations
69. ©2016 Couchbase Inc. 69 Say what? Relational Document Key/Value Columnar Graph Object others…
70. ©2016 Couchbase Inc. 70 Box arrow box arrow Web Logic SQL Riak Couchbase More Logic Mobile
71. ©2016 Couchbase Inc. 71 Couchbase, everybody! http://couchbase.com/connect http://info.couchbase.com/Connect16_Livestream_Registration.html
72. ©2016 Couchbase Inc. 72 Where do you find us? • developer.couchbase.com • blog.couchbase.com • forums.couchbase.com • @couchbasedev • #couchbase • @mgroves • @couchbase