Introduction 2 Documents Document-Oriented Data Model Updates and Concurrency Design Documents 3 Distributed System 4 Queries and Views Views Creating Views with Map Reduce Example Kerstin Puschke NoSQL
Introduction 2 Documents Document-Oriented Data Model Updates and Concurrency Design Documents 3 Distributed System 4 Queries and Views Views Creating Views with Map Reduce Example 5 CouchApps Kerstin Puschke NoSQL
Of Unreliable Commodity Hardware DataBase NoSQL database, non-relational data store schemaless distributed computing - robust replication scaling up and down Kerstin Puschke NoSQL
Of Unreliable Commodity Hardware DataBase NoSQL database, non-relational data store schemaless distributed computing - robust replication scaling up and down open web standards Kerstin Puschke NoSQL
Of Unreliable Commodity Hardware DataBase NoSQL database, non-relational data store schemaless distributed computing - robust replication scaling up and down open web standards decentralized (web)applications - available offline Kerstin Puschke NoSQL
RESTful HTTP/JSON API Webserver included http://localhost:5984 http://localhost:5984/_all_dbs create a database PUT http://localhost:5984/foo Kerstin Puschke NoSQL
RESTful HTTP/JSON API Webserver included http://localhost:5984 http://localhost:5984/_all_dbs create a database PUT http://localhost:5984/foo delete a a database DELETE http://localhost:5984/foo Kerstin Puschke NoSQL
RESTful HTTP/JSON API Webserver included http://localhost:5984 http://localhost:5984/_all_dbs create a database PUT http://localhost:5984/foo delete a a database DELETE http://localhost:5984/foo Webinterface Futon http://localhost:5984/_utils Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Document-Oriented Data Model abstraction layer above key-value pairs document: self-contained data, reasonable unit of information (invoice, business card,. . . ) Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Document-Oriented Data Model abstraction layer above key-value pairs document: self-contained data, reasonable unit of information (invoice, business card,. . . ) schemaless, no referential integrity flexible, but dangerous Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Document-Oriented Data Model abstraction layer above key-value pairs document: self-contained data, reasonable unit of information (invoice, business card,. . . ) schemaless, no referential integrity flexible, but dangerous JSON-object values can be scalar, array, objects, nested data structures,. . . Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Document-Oriented Data Model abstraction layer above key-value pairs document: self-contained data, reasonable unit of information (invoice, business card,. . . ) schemaless, no referential integrity flexible, but dangerous JSON-object values can be scalar, array, objects, nested data structures,. . . mandatory fields: Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Document-Oriented Data Model abstraction layer above key-value pairs document: self-contained data, reasonable unit of information (invoice, business card,. . . ) schemaless, no referential integrity flexible, but dangerous JSON-object values can be scalar, array, objects, nested data structures,. . . mandatory fields: unique document identifier _id (UUID) Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Document-Oriented Data Model abstraction layer above key-value pairs document: self-contained data, reasonable unit of information (invoice, business card,. . . ) schemaless, no referential integrity flexible, but dangerous JSON-object values can be scalar, array, objects, nested data structures,. . . mandatory fields: unique document identifier _id (UUID) revision identifier _rev Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Document-Oriented Data Model abstraction layer above key-value pairs document: self-contained data, reasonable unit of information (invoice, business card,. . . ) schemaless, no referential integrity flexible, but dangerous JSON-object values can be scalar, array, objects, nested data structures,. . . mandatory fields: unique document identifier _id (UUID) revision identifier _rev files can be attached to documents Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Updates client sends full document append-only - new version appended to database no diffs, no partial updates Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Updates client sends full document append-only - new version appended to database no diffs, no partial updates robust and fast Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Multi Version Concurrency Control optimistic locking client sends new version of document along with old _rev Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Multi Version Concurrency Control optimistic locking client sends new version of document along with old _rev server checks if _rev matches the one stored in the database Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Multi Version Concurrency Control optimistic locking client sends new version of document along with old _rev server checks if _rev matches the one stored in the database yes: new version saved along with new _rev Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Multi Version Concurrency Control optimistic locking client sends new version of document along with old _rev server checks if _rev matches the one stored in the database yes: new version saved along with new _rev no: conflict - update refused with 409 Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Multi Version Concurrency Control optimistic locking client sends new version of document along with old _rev server checks if _rev matches the one stored in the database yes: new version saved along with new _rev no: conflict - update refused with 409 no version control some versions unavailable (at least on some nodes) Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Design Documents _id starts with _design create, update, replicate,. . . like any other document Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Design Documents _id starts with _design create, update, replicate,. . . like any other document no data but code Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Design Documents _id starts with _design create, update, replicate,. . . like any other document no data but code validate_doc_update: validation of updates Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Design Documents _id starts with _design create, update, replicate,. . . like any other document no data but code validate_doc_update: validation of updates update: server-side logic for document creation or update Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Design Documents _id starts with _design create, update, replicate,. . . like any other document no data but code validate_doc_update: validation of updates update: server-side logic for document creation or update show, list: rendering output Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Design Documents _id starts with _design create, update, replicate,. . . like any other document no data but code validate_doc_update: validation of updates update: server-side logic for document creation or update show, list: rendering output map, reduce: creation of views/indices Kerstin Puschke NoSQL
Model Updates and Concurrency Design Documents Design Documents _id starts with _design create, update, replicate,. . . like any other document no data but code validate_doc_update: validation of updates update: server-side logic for document creation or update show, list: rendering output map, reduce: creation of views/indices default language: JavaScript plugins for PHP, Ruby, Python, Perl, Common Lisp, Erlang,. . . Kerstin Puschke NoSQL
nothing cluster incremental latest revision only - no transfer of previous versions N-Master, Master-Slave,. . . very robust filtered Kerstin Puschke NoSQL
nothing cluster incremental latest revision only - no transfer of previous versions N-Master, Master-Slave,. . . very robust filtered ’non-destructive’ conflict handling Kerstin Puschke NoSQL
Consistency, Availability, Partition Tolerance Consistency, Availability, Partition Tolerance - you can’t have it all. . . CouchDB provides eventual consistency inconsistency window BASE - basically available, soft-state, eventually consistent ACID on a single node, BASE in a distributed system Kerstin Puschke NoSQL
Views with Map Reduce Example Queries fetch document by _id no dynamic queries, no ad hoc reporting need to create views (indices) to query Kerstin Puschke NoSQL
Views with Map Reduce Example Views (secondary) index key-value-pairs keys and values are values from documents Example: creation date as key, title of post as value Kerstin Puschke NoSQL
Views with Map Reduce Example Views (secondary) index key-value-pairs keys and values are values from documents Example: creation date as key, title of post as value composite keys and values possible Kerstin Puschke NoSQL
Views with Map Reduce Example Views (secondary) index key-value-pairs keys and values are values from documents Example: creation date as key, title of post as value composite keys and values possible sorted by key Kerstin Puschke NoSQL
Views with Map Reduce Example Views (secondary) index key-value-pairs keys and values are values from documents Example: creation date as key, title of post as value composite keys and values possible sorted by key efficient lookup of keys or key ranges ’Posts created in May 2009’ Kerstin Puschke NoSQL
Views with Map Reduce Example Views (secondary) index key-value-pairs keys and values are values from documents Example: creation date as key, title of post as value composite keys and values possible sorted by key efficient lookup of keys or key ranges ’Posts created in May 2009’ values can be aggregated Kerstin Puschke NoSQL
Views with Map Reduce Example Views (secondary) index key-value-pairs keys and values are values from documents Example: creation date as key, title of post as value composite keys and values possible sorted by key efficient lookup of keys or key ranges ’Posts created in May 2009’ values can be aggregated created and updated when accessed Kerstin Puschke NoSQL
Views with Map Reduce Example Map Reduce map and reduce functions common in functional programming MapReduce: framework for concurrent, distributed processing of large data sets (Google) Kerstin Puschke NoSQL
Views with Map Reduce Example Map Reduce map and reduce functions common in functional programming MapReduce: framework for concurrent, distributed processing of large data sets (Google) map and reduce defined in design documents Kerstin Puschke NoSQL
Views with Map Reduce Example Map Reduce map and reduce functions common in functional programming MapReduce: framework for concurrent, distributed processing of large data sets (Google) map and reduce defined in design documents map Kerstin Puschke NoSQL
Views with Map Reduce Example Map Reduce map and reduce functions common in functional programming MapReduce: framework for concurrent, distributed processing of large data sets (Google) map and reduce defined in design documents map input: one document Kerstin Puschke NoSQL
Views with Map Reduce Example Map Reduce map and reduce functions common in functional programming MapReduce: framework for concurrent, distributed processing of large data sets (Google) map and reduce defined in design documents map input: one document computes key-value pairs for view Kerstin Puschke NoSQL
Views with Map Reduce Example Map Reduce map and reduce functions common in functional programming MapReduce: framework for concurrent, distributed processing of large data sets (Google) map and reduce defined in design documents map input: one document computes key-value pairs for view sorted by key Kerstin Puschke NoSQL
Views with Map Reduce Example Map Reduce map and reduce functions common in functional programming MapReduce: framework for concurrent, distributed processing of large data sets (Google) map and reduce defined in design documents map input: one document computes key-value pairs for view sorted by key reduce (optional) Kerstin Puschke NoSQL
Views with Map Reduce Example Map Reduce map and reduce functions common in functional programming MapReduce: framework for concurrent, distributed processing of large data sets (Google) map and reduce defined in design documents map input: one document computes key-value pairs for view sorted by key reduce (optional) input: (partial) results of map or of previous calls to reduce Kerstin Puschke NoSQL
Views with Map Reduce Example Map Reduce map and reduce functions common in functional programming MapReduce: framework for concurrent, distributed processing of large data sets (Google) map and reduce defined in design documents map input: one document computes key-value pairs for view sorted by key reduce (optional) input: (partial) results of map or of previous calls to reduce returns aggregated values Kerstin Puschke NoSQL
Views with Map Reduce Example Example View to query for posts and their comments map-function with composite key function(doc) { if (doc.type == "post") { emit([doc._id, 0], doc); } else if (doc.type == "comment") { emit([doc.post_id, 1], doc); } } Kerstin Puschke NoSQL
living in the database no middleware data and code are documents: update, fork, backup etc. via replication show/list functions, direct access to attachments (HTML,JavaScript,. . . ) Kerstin Puschke NoSQL
living in the database no middleware data and code are documents: update, fork, backup etc. via replication show/list functions, direct access to attachments (HTML,JavaScript,. . . ) load data via AJAX Kerstin Puschke NoSQL
titanoboa
afilina
chocoyama
javiergs
prof18
nashiusagi
niftycorp
yusukebe
kyoheig3
manfredsteyer
syumai
tomohisa
yosuke_furukawa
bkeepers
jakevdp
philhawksworth
skipperchong
tammyeverts
csswizardry
chrislema
trishagee
chriscoyier
addyosmani
eileencodes
mclloyd
