Houston.js - October 2016 - I Have a NoSQL Toaster

My toaster stores data without SQL and without tables. But making a choice based on what something doesn’t have isn’t terribly useful. “NoSQL” is an increasingly inaccurate catch-all term that covers a lot of different types of data storage. Let’s make more sense of this new breed of database management systems and go beyond the buzzword. In this session, the four main data models that make up the NoSQL movement will be covered: key-value, document, columnar and graph. How they differ and when you might want to use each one will be discussed.

This session will be looking at the whole ecosystem, with a more detailed focus on Couchbase, Cassandra, Riak KV, and Neo4j.

Transcript

1. I have a NoSQL Toaster.
   Why would I want a NoSQL database?
   Matthew D. Groves @mgroves
   Original slides by Matthew Revell - @matthewrevell
   

   View Slide

2. ©2016 Couchbase Inc. 2
   Who am I?
   • Matthew D. Groves
   • Developer Advocate for Couchbase
   • @mgroves on Twitter
   • Podcast and blog: http://crosscuttingconcerns.com
   • “I am not an expert, but I am an enthusiast.” – Alan Stevens
   

   View Slide

3. ©2016 Couchbase Inc. 3
   Tweet something, get a sticker!
   Use #Couchbase
   Tweet something interesting you’ve learned.
   Tweet a picture.
   I’ve got a few stickers!
   (greedy Austin devs took most of them)
   

   View Slide

4. ©2016 Couchbase Inc. 4
   Couchbase and CouchDB
   Couchbase is to CouchDB
   as
   MySQL is to SQL Server
   

   View Slide

5. ©2016 Couchbase Inc. 5
   SQL is the norm, right?
   https://commons.wikimedia.org/wiki/File:George_Wendt_at_the_41st_Emmy_Awards_cropped.jpg
   

   View Slide

6. ~3200 BC: invention of writing in Mesopotamia
   ~300 BC: Library of Alexandria
   ~1041 AD: Movable type invented in China
   1450: Movable type invented in Europe
   1822: Babbage’s paper on the application of difference engines
   1943: Colossus, the first programmable digital computer
   1957-1960: IBM SABRE, the first commercial use of a database
   1970: EF Codd proposes the relational database model
   1974: Ingres released
   1979: Commercial launch of Oracle database
   1988: Object databases appear
   1989: Microsoft SQL Server version 1
   1991: HTTP v0.9
   1995: MySQL’s initial release
   2005: CouchDB released
   2006: Google’s Big Table paper
   2007: Amazon’s Dynamo paper
   2008-2009: The NoSQL Cambrian explosion: Riak, MongoDB, Cassandra, Redis
   2010: Couchbase arrives on the scene
   

   View Slide

7. NoSQL isn’t a very useful term
   https://commons.wikimedia.org/wiki/File:Toaster.jpg
   

   View Slide

8. ©2016 Couchbase Inc. 8
   Easy?
   

   View Slide

9. ©2016 Couchbase Inc. 9
   Scalable?
   https://commons.wikimedia.org/wiki/File:Dagwood_sandwich.jpg
   

   View Slide

10. ©2016 Couchbase Inc. 10
    Commodity Hardware
    https://commons.wikimedia.org/wiki/File:Rear_of_rack_at_NERSC_data_center_-_closeup.jpg
    

    View Slide

11. ©2016 Couchbase Inc. 11
    ACID vs BASE?
    https://commons.wikimedia.org/wiki/File:PH_Scale.svg
    Atomic
    Consistent
    Isolated
    Durable
    Basic Availability
    Soft-state
    Eventual consistency
    

    View Slide

12. ©2016 Couchbase Inc. 12
    Schema Schemaless
    https://en.wikipedia.org/wiki/File:Star-schema-example.png
    

    View Slide

13. ©2016 Couchbase Inc. 13
    Normalized Denormalized
    https://commons.wikimedia.org/wiki/File:Frown.JPG
    

    View Slide

14. ©2016 Couchbase Inc. 14
    First: Why NoSQL?
    https://www.flickr.com/photos/wonderlane/13067014944
    

    View Slide

15. ©2016 Couchbase Inc. 15
    Round pegs, square holes
    http://www.timesofbook.com/2013/09/blobfish-image-gallery.html#.V6iXwbgrJhE
    

    View Slide

16. ©2016 Couchbase Inc. 16
    Perhaps SQL is right for this project
    

    View Slide

17. ©2016 Couchbase Inc. 17
    Scaling can be hard
    https://commons.wikimedia.org/wiki/File:Sandwiches_Vienna.jpg
    

    View Slide

18. ©2016 Couchbase Inc. 18
    Availability is hard
    https://www.flickr.com/photos/jeepersmedia/14562425273/
    C
    A
    P
    

    View Slide

19. ©2016 Couchbase Inc. 19
    Schemas can be hard
    https://commons.wikimedia.org/wiki/File:MediaWiki_database_schema_1-17_(r82044).svg
    

    View Slide

20. ©2016 Couchbase Inc. 20
    Types of NoSQL databases
    Document
    Key/Value
    Columnar
    Graph
    

    View Slide

21. ©2016 Couchbase Inc. 21
    Convergence!
    

    View Slide

22. ©2016 Couchbase Inc. 22
    Document
    

    View Slide

23. ???
    

    View Slide

24. ©2016 Couchbase Inc. 24
    What makes it a document database?
    

    View Slide

25. JSON
    XML
    BSON
    

    View Slide

26. View Slide

27. ©2016 Couchbase Inc. 27
    The document database
    understands the
    format of the document
    

    View Slide

28. ©2016 Couchbase Inc. 28
    It can do
    server-side stuff
    

    View Slide

29. Buckets
    Documents
    Views
    N1QL
    

    View Slide

30. "ClientProductCode","Title","Quantity"
    "TShirtCBS","Couchbase logo t-shirt (small)",29
    "TShirtCBM","Couchbase logo t-shirt (medium)",72
    "TShirtCBL","Couchbase logo t-shirt (large)",34
    "TShirtCBXL","Couchbase logo t-shirt (extra large)",12
    "StickerCBLogo","Couchbase logo sticker",256
    "PenCBLogo","Couchbase logo pen",365
    

    View Slide

31. class Program
    {
    private static IBucket _bucket;
    static void Main(string[] args)
    {
    SetupCouchbase();
    LoadStockList();
    ClusterHelper.Close();
    }
    private static void SetupCouchbase()
    {
    ClientConfiguration config = new ClientConfiguration();
    config.Servers = new List {new Uri("couchbase://192.168.1.5") };
    ClusterHelper.Initialize(config);
    _bucket = ClusterHelper.GetBucket("default");
    }
    private static void LoadStockList()
    {
    var csv = new CsvReader(File.OpenText("swag.csv"));
    while (csv.Read())
    {
    var record = csv.GetRecord();
    var document = new Document
    {
    Id = record.ClientProductCode,
    Content = new
    {
    record.Title,
    record.Quantity
    }
    };
    _bucket.Insert(document);
    Console.WriteLine($"Added document '{record.ClientProductCode}'");
    }
    }
    }
    

    View Slide

32. private static void SetupCouchbase()
    {
    ClientConfiguration config = new ClientConfiguration();
    config.Servers = new List {new Uri("couchbase://192.168.1.5") };
    ClusterHelper.Initialize(config);
    _bucket = ClusterHelper.GetBucket("default");
    }
    

    View Slide

33. private static void LoadStockList() {
    var csv = new CsvReader(File.OpenText("swag.csv"));
    while (csv.Read()) {
    var record = csv.GetRecord();
    var document = new Document {
    Id = record.ClientProductCode,
    Content = new {
    record.Title,
    record.Quantity
    }
    };
    _bucket.Insert(document);
    Console.WriteLine($"Added document '{record.ClientProductCode}'");
    }
    }
    

    View Slide

34. View Slide

35. private static void GetDocument()
    {
    var doc = _bucket.Get("StickerCBLogo");
    Console.WriteLine($"Document Key: {doc.Id}");
    Console.WriteLine($"Title: {doc.Value.title}");
    Console.WriteLine($"Quantity: {doc.Value.quantity}");
    }
    

    View Slide

36. View Slide

37. ©2016 Couchbase Inc. 37
    Typical Document Database Operations
    • Upsert (aka set)
    • Inserts document if key doesn’t exist, replaces otherwise
    • Insert (aka add)
    • Inserts document, error if it already exists
    • Update (aka replace)
    • Updates document, error if it doesn’t exist
    • Delete
    • Get
    

    View Slide

38. ©2016 Couchbase Inc. 38
    N1QL
    SELECT m.*
    FROM `default` m
    WHERE META(m).id = ‘StickerCBLogo’
    SELECT m.*
    FROM `default` m
    WHERE m.quantity > 10
    

    View Slide

39. Great for Dev
    Scales Easily
    Consistently Fast
    

    View Slide

40. ©2016 Couchbase Inc. 40
    Use cases for Document Databases
    • User profiles
    • Session stores
    • Content management
    • General relational replacement
    • http://info.couchbase.com/15Q1TopTenUC.html
    

    View Slide

41. ©2016 Couchbase Inc. 41
    Key/Value Database
    

    View Slide

42. View Slide

43. ©2016 Couchbase Inc. 43
    Key-Value Databases
    In-memory caches
    and
    distributed data stores
    

    View Slide

44. ©2016 Couchbase Inc. 44
    Doesn’t care what your data is (mostly)
    KEY1 { type: “json” }
    KEY2
    KEY3 Lorem ipsum
    … 00100101000100101
    KEYN
    https://www.flickr.com/photos/dcmot/23168680069
    

    View Slide

45. Buckets
    Key-Value Pairs
    

    View Slide

46. ©2016 Couchbase Inc. 46
    C# Riak
    private static void DoStuffWithRiak()
    {
    var cluster = RiakCluster.FromConfig("riakConfig");
    var client = cluster.CreateClient();
    var actor = new RiakObject("actors", "James Bond","Sean Connery");
    client.Put(actor);
    var actorBackOut = client.Get("actors", "James Bond");
    var str = System.Text.Encoding.UTF8.GetString(actorBackOut.Value.Value);
    Console.WriteLine($"Value: {str}");
    }
    

    View Slide

47. View Slide

48. ©2016 Couchbase Inc. 48
    Introducing: Eventual Consistency!
    

    View Slide

49. ©2016 Couchbase Inc. 49
    Eventual Consistency
    https://www.flickr.com/photos/scottchene/7046992749
    https://commons.wikimedia.org/wiki/File:Timothy_Dalton_1987.jpg
    https://commons.wikimedia.org/wiki/File:Sir_Roger_Moore_crop.jpg
    

    View Slide

50. ©2016 Couchbase Inc. 50
    There can be only one!
    https://en.wikipedia.org/wiki/File:Highlander_film_Connor_MacLeod.jpg
    

    View Slide

51. ©2016 Couchbase Inc. 51
    So, fix it
    https://www.flickr.com/photos/94086507@N00/84368105
    

    View Slide

52. Scales Well
    Highly Available
    Data Agnostic
    Maybe some extra work
    

    View Slide

53. ©2016 Couchbase Inc. 53
    Use Cases for K/V
    • Data variability
    • Object caching
    • Session storage
    • Large object storage
    

    View Slide

54. ©2016 Couchbase Inc. 54
    Columnar
    

    View Slide

55. View Slide

56. Keyspaces
    Column Families
    Rows
    Columns
    

    View Slide

57. ©2016 Couchbase Inc. 57
    Keyspace
    

    View Slide

58. ©2016 Couchbase Inc. 58
    Column Family
    

    View Slide

59. CREATE KEYSPACE mykeyspace
    WITH replication = { 'class': 'SimpleStrategy',
    'replication_factor': '1'};
    CREATE TABLE users (firstname text, lastname text, age int,
    email text, city text, PRIMARY KEY (lastname));
    

    View Slide

60. private static void DoStuffWithDataStax()
    {
    var cluster = Cluster.Builder().AddContactPoint("127.0.0.1").Build();
    var session = cluster.Connect("mykeyspace");
    session.Execute("insert into users (lastname, age, city, email, firstname)
    values ('Jones', 35, 'Austin', 'bob@example.com', 'Bob')");
    var userBackOut = session.Execute("select * from users where
    lastname='Jones'").First();
    Console.WriteLine($"{userBackOut["firstname"]} {userBackOut["age"]}");
    }
    

    View Slide

61. View Slide

62. Impedance Mismatch
    Scales Well (Ops Headache)
    Suited to Analytics
    

    View Slide

63. ©2016 Couchbase Inc. 63
    Use cases for Columnar
    •Metering data
    •Really big data
    •Analytics
    

    View Slide

64. ©2016 Couchbase Inc. 64
    Graph
    https://commons.wikimedia.org/wiki/File:The_protein_interaction_network_of_Treponema_pallidum.png
    

    View Slide

65. ©2016 Couchbase Inc. 65
    Leonhard Euler
    

    View Slide

66. ©2016 Couchbase Inc. 66
    7 Bridges of Königsberg
    

    View Slide

67. View Slide

68. ©2016 Couchbase Inc. 68
    Use cases for Graph databases
    • Social networks / dating sites
    • Fraud detection
    • Parcel routing
    • Shopping recommendations
    

    View Slide

69. ©2016 Couchbase Inc. 69
    Say what?
    Relational
    Document
    Key/Value
    Columnar
    Graph
    Object
    others…
    

    View Slide

70. ©2016 Couchbase Inc. 70
    Box arrow box arrow
    Web Logic
    SQL
    Riak
    Couchbase
    More Logic
    Mobile
    

    View Slide

71. ©2016 Couchbase Inc. 71
    Couchbase, everybody!
    http://couchbase.com/connect
    http://info.couchbase.com/Connect16_Livestream_Registration.html
    

    View Slide

72. ©2016 Couchbase Inc. 72
    Where do you find us?
    • developer.couchbase.com
    • blog.couchbase.com
    • forums.couchbase.com
    • @couchbasedev
    • #couchbase
    • @mgroves
    • @couchbase
    

    View Slide