Tweet
Tweet

Slide 1

Slide 1 text

Round Pegs and Square Holes Django and MongoDB by Daniel Greenfeld and Audrey Roy Monday, June 4, 12

Slide 2

Slide 2 text

@pydanny / @audreyr cartwheelweb.com Audrey / Danny • Principals at Cartwheel Web • Co-founders of consumer.io • Met at PyCon 2010 • Now affianced Photo credit: Christopher Neugebauer Monday, June 4, 12

Slide 3

Slide 3 text

@pydanny / @audreyr cartwheelweb.com What is MongoDB? • NoSQL • Fast, Scalable, Indexable, Shardable • Schema-less Monday, June 4, 12

Slide 4

Slide 4 text

@pydanny / @audreyr cartwheelweb.com What is MongoDB? • Written in C++ • Stores data in BSON (kinda like JSON) • Uses JavaScript internally for scripting • Has Python, Ruby, PHP, C, other drivers • Huge community Monday, June 4, 12

Slide 5

Slide 5 text

@pydanny / @audreyr cartwheelweb.com MongoDB: SQL Analogy • Collections are like tables • Documents are like records (rows) Monday, June 4, 12

Slide 6

Slide 6 text

@pydanny / @audreyr cartwheelweb.com What is a MongoDB queryset? Minimalist view As served by pymongo Monday, June 4, 12

Slide 7

Slide 7 text

@pydanny / @audreyr cartwheelweb.com What is a MongoDB queryset? A list of dictionaries Minimalist view As served by pymongo Monday, June 4, 12

Slide 8

Slide 8 text

@pydanny / @audreyr cartwheelweb.com What it looks like Minimalist view collection = [] document = { '_objectId': ObjectId('4f844e916c97c1000c000003'), 'username': 'pydanny', 'fiancee': { 'username': 'audreyr', 'location': 'Zurich' } } collection = [document, ] list Monday, June 4, 12

Slide 9

Slide 9 text

@pydanny / @audreyr cartwheelweb.com What it looks like Minimalist view collection = [] document = { '_objectId': ObjectId('4f844e916c97c1000c000003'), 'username': 'pydanny', 'fiancee': { 'username': 'audreyr', 'location': 'Zurich' } } collection = [document, ] list dictionary Monday, June 4, 12

Slide 10

Slide 10 text

@pydanny / @audreyr cartwheelweb.com What it looks like Minimalist view collection = [] document = { '_objectId': ObjectId('4f844e916c97c1000c000003'), 'username': 'pydanny', 'fiancee': { 'username': 'audreyr', 'location': 'Zurich' } } collection = [document, ] list dictionary list of dictionaries! Monday, June 4, 12

Slide 11

Slide 11 text

Tools for using MongoDB with Django (actually, most of these are not Django-dependent) Monday, June 4, 12

Slide 12

Slide 12 text

@pydanny / @audreyr cartwheelweb.com “MongoDB connectors” • pymongo • mongoengine • mongokit • Django-nonrel • the only one here requiring Django Monday, June 4, 12

Slide 13

Slide 13 text

Option 1 pymongo Monday, June 4, 12

Slide 14

Slide 14 text

@pydanny / @audreyr cartwheelweb.com pymongo http://api.mongodb.org/python/current/ Official Python binding for MongoDB >>> from pymongo import Connection >>> connection = Connection() >>> my_data = {'rating': 3, 'title': 'I like ice cream'} >>> connection.reviews.insert(my_data) >>> your_data = {'rating': 3, 'subject': 'You like ice cream'} >>> connection.reviews.insert(your_data) Monday, June 4, 12

Slide 15

Slide 15 text

@pydanny / @audreyr cartwheelweb.com pymongo http://api.mongodb.org/python/current/ Official Python binding for MongoDB >>> from pymongo import Connection >>> connection = Connection() >>> my_data = {'rating': 3, 'title': 'I like ice cream'} >>> connection.reviews.insert(my_data) >>> your_data = {'rating': 3, 'subject': 'You like ice cream'} >>> connection.reviews.insert(your_data) my_data with title Monday, June 4, 12

Slide 16

Slide 16 text

@pydanny / @audreyr cartwheelweb.com pymongo http://api.mongodb.org/python/current/ Official Python binding for MongoDB >>> from pymongo import Connection >>> connection = Connection() >>> my_data = {'rating': 3, 'title': 'I like ice cream'} >>> connection.reviews.insert(my_data) >>> your_data = {'rating': 3, 'subject': 'You like ice cream'} >>> connection.reviews.insert(your_data) my_data with title your_data with subject Monday, June 4, 12

Slide 17

Slide 17 text

@pydanny / @audreyr cartwheelweb.com pymongo >>> connection = pymongo.Connection() >>> db = connection.db >>> for review in db.reviews.find({'rating': 3}): ... review['title'] >>> for review in db.reviews.find( ... {"title": {"$regex": "ice cream"} } ... ): ... review['title'] [ {'rating': 3,'title': 'I like ice cream'}, {'rating': 3, 'subject': 'You like ice cream'} ] Monday, June 4, 12

Slide 18

Slide 18 text

@pydanny / @audreyr cartwheelweb.com pymongo >>> connection = pymongo.Connection() >>> db = connection.db >>> for review in db.reviews.find({'rating': 3}): ... review['title'] >>> for review in db.reviews.find( ... {"title": {"$regex": "ice cream"} } ... ): ... review['title'] [ {'rating': 3,'title': 'I like ice cream'}, {'rating': 3, 'subject': 'You like ice cream'} ] Finds all reviews with a rating of 3 Monday, June 4, 12

Slide 19

Slide 19 text

@pydanny / @audreyr cartwheelweb.com pymongo >>> connection = pymongo.Connection() >>> db = connection.db >>> for review in db.reviews.find({'rating': 3}): ... review['title'] >>> for review in db.reviews.find( ... {"title": {"$regex": "ice cream"} } ... ): ... review['title'] [ {'rating': 3,'title': 'I like ice cream'}, {'rating': 3, 'subject': 'You like ice cream'} ] Monday, June 4, 12

Slide 20

Slide 20 text

@pydanny / @audreyr cartwheelweb.com pymongo >>> connection = pymongo.Connection() >>> db = connection.db >>> for review in db.reviews.find({'rating': 3}): ... review['title'] >>> for review in db.reviews.find( ... {"title": {"$regex": "ice cream"} } ... ): ... review['title'] [ {'rating': 3,'title': 'I like ice cream'}, {'rating': 3, 'subject': 'You like ice cream'} ] Only finds the document with ‘title’ in it. Monday, June 4, 12

Slide 21

Slide 21 text

@pydanny / @audreyr cartwheelweb.com pymongo • Really fast, bare metal • Lets you go schema-crazy • Supported directly by 10gen • They say “PyMongo is the recommended way to work with MongoDB from Python.” Pros Monday, June 4, 12

Slide 22

Slide 22 text

@pydanny / @audreyr cartwheelweb.com pymongo • “PyMongo introspection” is an oxymoron • Very low-level • Lose out on ModelForms, Auth, Admin, etc • Syntax not as clean as with an object mapper Cons Monday, June 4, 12

Slide 23

Slide 23 text

Option 2 MongoEngine Monday, June 4, 12

Slide 24

Slide 24 text

@pydanny / @audreyr cartwheelweb.com MongoEngine import mongoengine as me class Review(me.Document): title = me.StringField() body = me.StringField() author = me.StringField() created = me.DateTimeField(default=datetime.utcnow) rating = me.IntField() http://mongoengine.org/ Doesn’t this look like the Django ORM? Monday, June 4, 12

Slide 25

Slide 25 text

@pydanny / @audreyr cartwheelweb.com MongoEngine Doesn’t this look like a Django query? >>> from reviews.models import Review >>> for review in Review.objects.all(): ... review.title Monday, June 4, 12

Slide 26

Slide 26 text

@pydanny / @audreyr cartwheelweb.com • Looks similar to Django ORM code • You can develop SUPER-QUICKLY • Can use with django-mongonaut for introspection • Light schema, unenforced by the db • Supports some inter-document connections MongoEngine Pros Monday, June 4, 12

Slide 27

Slide 27 text

@pydanny / @audreyr cartwheelweb.com • Some feel there’s too much structure • Validation messages sometimes unclear • Using it with Django, you lose out on the Django Admin’s introspection* and ModelForms MongoEngine Cons * django-mongonaut addresses this. Monday, June 4, 12

Slide 28

Slide 28 text

@pydanny / @audreyr cartwheelweb.com mongoengine and django-mongonaut http://bit.ly/django-mongonaut Monday, June 4, 12

Slide 29

Slide 29 text

Option 3 MongoKit Monday, June 4, 12

Slide 30

Slide 30 text

@pydanny / @audreyr cartwheelweb.com MongoKit http://namlook.github.com/mongokit/ from mongokit import Document, Connection connection = Connection() @connection.register class Review(Document): structure = { 'title':unicode, 'body':unicode, 'author':unicode, 'created':datetime.datetime, 'rating':int } required_fields = ['title', 'author', 'created'] default_values = {'rating': 0, 'created': datetime.utcnow} Monday, June 4, 12

Slide 31

Slide 31 text

@pydanny / @audreyr cartwheelweb.com MongoKit http://namlook.github.com/mongokit/ from mongokit import Document, Connection connection = Connection() @connection.register class Review(Document): structure = { 'title':unicode, 'body':unicode, 'author':unicode, 'created':datetime.datetime, 'rating':int } required_fields = ['title', 'author', 'created'] default_values = {'rating': 0, 'created': datetime.utcnow} Review model representing MongoDB collection Monday, June 4, 12

Slide 32

Slide 32 text

@pydanny / @audreyr cartwheelweb.com MongoKit http://namlook.github.com/mongokit/ from mongokit import Document, Connection connection = Connection() @connection.register class Review(Document): structure = { 'title':unicode, 'body':unicode, 'author':unicode, 'created':datetime.datetime, 'rating':int } required_fields = ['title', 'author', 'created'] default_values = {'rating': 0, 'created': datetime.utcnow} Review model representing MongoDB collection Expected fields Monday, June 4, 12

Slide 33

Slide 33 text

@pydanny / @audreyr cartwheelweb.com MongoKit http://namlook.github.com/mongokit/ from mongokit import Document, Connection connection = Connection() @connection.register class Review(Document): structure = { 'title':unicode, 'body':unicode, 'author':unicode, 'created':datetime.datetime, 'rating':int } required_fields = ['title', 'author', 'created'] default_values = {'rating': 0, 'created': datetime.utcnow} Review model representing MongoDB collection Expected fields Connect to reviews collection Monday, June 4, 12

Slide 34

Slide 34 text

@pydanny / @audreyr cartwheelweb.com MongoKit http://namlook.github.com/mongokit/ >>> from mongokit import Connection >>> connection = Connection() >>> for review in connection.Review.find({'rating': 3}): ... review['title'] Identical to pymongo queries Monday, June 4, 12

Slide 35

Slide 35 text

@pydanny / @audreyr cartwheelweb.com • Light schema, unenforced by the db • Or can go all out schemaless! • Speed • Types are a mix of Python & MongoDB • Uses pymongo-style queries MongoKit Pros Monday, June 4, 12

Slide 36

Slide 36 text

@pydanny / @audreyr cartwheelweb.com • Using it with Django, you lose out on the Django Admin’s introspection, ModelForms, auth, etc • Introspection is hard MongoKit Cons Monday, June 4, 12

Slide 37

Slide 37 text

Option 4 Django-nonrel + mongodb-engine Monday, June 4, 12

Slide 38

Slide 38 text

@pydanny / @audreyr cartwheelweb.com • A patch to Django that adds NoSQL support to the Django ORM • Works with GAE, MongoDB, even SQL DBs http://docs.django-nonrel.org Django-nonrel + mongodb-engine Monday, June 4, 12

Slide 39

Slide 39 text

@pydanny / @audreyr cartwheelweb.com Django-nonrel + mongodb-engine • Can use Django as you normally would • Mirrors the ORM functionality • Introspection via djangotoolbox Pros Monday, June 4, 12

Slide 40

Slide 40 text

@pydanny / @audreyr cartwheelweb.com • Fork of the whole Django project. • Dependent on others to maintain parity with Django core (still on Django 1.3). • Multi-db usage is confusing Cons Django-nonrel + mongodb-engine Monday, June 4, 12

Slide 41

Slide 41 text

@pydanny / @audreyr cartwheelweb.com Summary • pymongo is low-level and well-supported by 10gen. • MongoEngine is like schemaless Django models. • MongoKit is like pymongo with extra structure. • Django-nonrel is a fork of Django 1.3. Monday, June 4, 12

Slide 42

Slide 42 text

The Take Away? Monday, June 4, 12

Slide 43

Slide 43 text

@pydanny / @audreyr cartwheelweb.com What about a single third-party app that lets you combine critical Django apps and MongoDB? Danny’s Thoughts Can we build a “simple” bridge? • django.contrib.auth • django.forms • django-social-auth / registration • others... Monday, June 4, 12

Slide 44

Slide 44 text

@pydanny / @audreyr cartwheelweb.com Danny’s Thoughts I wonder, why add schemas to schema-less when: * cache-machine or johnny-cache Relational Databases South High level Caching tools* allow you to develop fast moving datastores with transactions and built-in Django support? Monday, June 4, 12

Slide 45

Slide 45 text

@pydanny / @audreyr cartwheelweb.com Danny’s Thoughts Introspection Tool Idea Treat introspection like MongoDB Queries Immediate introspection tool (no ‘title’ then don’t show title) Monday, June 4, 12

Slide 46

Slide 46 text

@pydanny / @audreyr cartwheelweb.com Audrey’s Thoughts • Schemaless dbs promise performance advantages • Especially for distributed systems • Tradeoff: ACID compliance http://stackoverflow.com/questions/3856222/whats-the-attraction-of-schemaless-database-systems Monday, June 4, 12

Slide 47

Slide 47 text

@pydanny / @audreyr cartwheelweb.com Audrey’s Thoughts “Schemaless database” == ACID-noncompliant database OK when performance is more important than being consistent 100% of the time. Monday, June 4, 12

Slide 48

Slide 48 text

@pydanny / @audreyr cartwheelweb.com Audrey’s Thoughts Schemaless Python models != Schemaless MongoDB collections I think MongoEngine is best unless your use case calls for schema anarchy. Monday, June 4, 12

Slide 49

Slide 49 text

Going Forward Monday, June 4, 12

Slide 50

Slide 50 text

@pydanny / @audreyr cartwheelweb.com • Big hurdles: ORM, Admin, ModelForms, Auth • Were built for relational data • But the situation is improving rapidly Using Django With MongoDB Monday, June 4, 12

Slide 51

Slide 51 text

@pydanny / @audreyr cartwheelweb.com • New introspection tools (working on it) • django.forms bridge. • django.contrib.admin bridge. • Drop-in replacement for Django Auth (Django 1.5?) • Creation of best practices document for use of MongoDB with Django. What needs to be done Monday, June 4, 12

Slide 52

Slide 52 text

@pydanny / @audreyr cartwheelweb.com Django Mongonaut • Introspection tool for MongoEngine • Works well so far • Integrate graphing tools • Make independent from mongoengine • Contributors wanted: • https://github.com/pydanny/django-mongonaut Monday, June 4, 12

Slide 53

Slide 53 text

@pydanny / @audreyr cartwheelweb.com Django Mongonaut Mockup Based off of immediate introspection - not definition Monday, June 4, 12

Slide 54

Slide 54 text

@pydanny / @audreyr cartwheelweb.com Django Mongonaut Mockup Based off of immediate introspection - not definition SVG w/links for examining nested structures Monday, June 4, 12

Slide 55

Slide 55 text

@pydanny / @audreyr cartwheelweb.com Final Summary pymongo mongoengine mongokit django-nonrel django-mongodb django-mongonaut Python/MongoDB tools to consider { Monday, June 4, 12