Stupid ORM Tricks

SFPython, November 2011

Transcript

1. Stupid
   ORM
   Tricks
   

   View Slide

2. "[Django has] an ORM with a beautiful
   interface - for very basic things. But
   once you try to do the extraordinary
   with it… it's frustrating to say the least."
   — Adam Gomaa
   

   View Slide

3. CUSTOM MODEL FIELDS
   (1)
   

   View Slide

4. AD HOC MODELS
   (2)
   

   View Slide

5. USING VIEWS AS MODELS
   (3)
   

   View Slide

6. Warning:
   

   View Slide

7. When it breaks,
   you get to keep
   all the pieces.
   

   View Slide

8. Hi, Internets!
   If you were here in person, you'd hear me say that most code in this talk
   is not production-worthy. But since you're not here, this'll have to do.
   The code presented here is untested, and uses internal, undocumented
   interfaces. At best, it's a hack; at worst, it'll bring your site down.
   It's not the kind of code you can just copy and paste!
   Please don't reuse code from this talk in production without considering
   it carefully! Only once you complete understand how it can fail — it will
   — should you consider using this code in your own projects.
   Thank you!
   

   View Slide

9. CUSTOM MODEL FIELDS
   (1)
   

   View Slide

10. class Contact(models.Model):
    name = models.CharField(max_length=200)
    phone = models.CharField(max_length=30)
    

    View Slide

11. what am I supposed to type here?
    what am I supposed to type here?
    

    View Slide

12. >>> c = Contact.objects.get(...)
    >>> c.phone
    u'yo, call me, bro!'
    

    View Slide

13. PROGRAMMER
    Y U NO USE PROPER TYPES?
    

    View Slide

14. from internationalphone.models import \
    InternationalPhoneNumberField
    class Contact(models.Model):
    name = models.CharField(max_length=200)
    phone = InternationalPhoneNumberField(
    default_region='US')
    

    View Slide

15. View Slide

16. >>> Contact.objects.get(name='Jacob').phone
    PhoneNumber(country_code=1,
    national_number=2125551212L,
    extension=None,
    italian_leading_zero=False,
    country_code_source=None,
    preferred_domestic_carrier_code=None)
    

    View Slide

17. Requirements:
    phonenumbers
    iso3166
    

    View Slide

18. from django.db import models
    class InternationalPhoneNumberField(models.Field):
    __metaclass__ = models.SubfieldBase
    description = "An international phone number."
    def __init__(self, *args, **kwargs):
    ...
    def to_python(self, value):
    ...
    def get_prep_value(self, value):
    ...
    def formfield(self, **kwargs):
    ...
    def get_internal_type(self):
    ...
    

    View Slide

19. Required:
    

    View Slide

20. class InternationalPhoneNumberField(models.Field):
    ...
    def to_python(self, value):
    if not value:
    return None
    return utils.parse_number(value,
    self.default_region)
    

    View Slide

21. class InternationalPhoneNumberField(models.Field):
    ...
    def get_prep_value(self, value):
    if not isinstance(value, phonenumbers.PhoneNumber):
    value = self.to_python(value)
    if value:
    return utils.format_number(value)
    else:
    return None
    

    View Slide

22. class InternationalPhoneNumberField
    (models.Field):
    ...
    def get_internal_type(self):
    return 'CharField'
    

    View Slide

23. Optional:
    

    View Slide

24. class InternationalPhoneNumberField(models.Field):
    ...
    def formfield(self, **kwargs):
    kwargs.setdefault('form_class', forms.InternationalPhoneNumberField)
    kwargs.setdefault('default_region', self.default_region)
    return super(InternationalPhoneNumberField, self).formfield(**kwargs)
    

    View Slide

25. try:
    import south.modelsinspector
    rules = [
    # Field classes this rule applies to
    [InternationalPhoneNumberField],
    # Positional arguments (none).
    [],
    # Keyword arguments.
    {
    'max_length': ["max_length", {"default": 20}],
    'default_region': [
    "default_region",
    {"default": utils.guess_default_region()}
    ],
    },
    ]
    south.modelsinspector.add_introspection_rules(
    [rules], ["^internationalphone\.models"]
    )
    except ImportError:
    pass
    

    View Slide

26. MORE:
    django.me/custom-model-fields
    

    View Slide

27. AUTO-GENERATED
    MODELS
    (2)
    

    View Slide

28. Background:
    

    View Slide

29. In Python, classes are just
    syntactic sugar!
    

    View Slide

30. >>> class Foo(object):
    ... pass
    ...
    >>> f = Foo()
    >>> type(f)
    
    >>> type(Foo)
    
    >>> isinstance(Foo, type)
    True
    

    View Slide

31. >>> help(type)
    Help on class type in module __builtin__:
    class type(object)
    | type(object) -> the object's type
    | type(name, bases, dict) -> a new type
    

    View Slide

32. >>> help(type)
    Help on class type in module __builtin__:
    class type(object)
    | type(object) -> the object's type
    | type(name, bases, dict) -> a new type
    

    View Slide

33. class Foo(object):
    x = 1
    

    View Slide

34. Foo = type('Foo', (object,), {'x': 1})
    

    View Slide

35. class Person(models.Model):
    first_name = models.CharField(...)
    last_name = models.CharField(...)
    def __unicode__(self):
    return self.first_name
    

    View Slide

36. Person = type("Person", (models.Model), {
    'first_name' : models.CharField(...),
    'last_name': models.CharField(...),
    '__unicode__': lambda self: self.first_name,
    })
    

    View Slide

37. What
    does this
    mean?
    

    View Slide

38. settings.DATABASES
    "A dictionary containing the settings for
    all databases to be used with Django."
    

    View Slide

39. HARDCODE ALL
    THE SETTINGS!
    

    View Slide

40. But…
    what if…
    

    View Slide

41. Ad hoc
    models:
    

    View Slide

42. >>> from adhocdb.models import DB
    >>> lightroom = DB(alias='lightroom',
    ... path='.../Lightroom 3 Catalog.lrcat')
    >>> lightroom.model_names()
    ['Adobe_AdditionalMetadata',
    'Adobe_imageDevelopBeforeSettings',
    'Adobe_imageDevelopSettings',
    'Adobe_imageProperties',
    'Adobe_images',
    ...
    ]
    >>> Image = lightroom.model('Adobe_images')
    >>> Image.objects.count()
    4775
    

    View Slide

43. from django import db
    from django.conf import settings
    from django.db import models
    class DB(object):
    def __init__(self, alias, path):
    self.alias = alias
    settings.DATABASES[self.alias] = {
    'ENGINE': 'django.db.backends.sqlite3',
    'NAME': path,
    }
    if self.alias in db.connections._connections:
    del db.connections._connections[self.alias]
    self._model_cache = None
    

    View Slide

44. from django import db
    from django.conf import settings
    from django.db import models
    class DB(object):
    ...
    def model_names(self):
    conn = db.connections[self.alias]
    introspection = conn.introspection
    return map(str, introspection.table_names())
    

    View Slide

45. from django import db
    from django.conf import settings
    from django.db import models
    class DB(object):
    ...
    def model_for_table(self, table_name, **namespace):
    introspection = db.connections[self.alias].introspection
    cursor = db.connections[self.alias].cursor()
    namespace.update({
    '__module__': __name__,
    'Meta': type("Meta", (object,), {'db_table': table_name}),
    '_adhocdb_alias': self.alias,
    })
    for row in introspection.get_table_description(cursor, table_name):
    rname, rtype, _, _, _, _, nullok = row
    try:
    classname = introspection.DATA_TYPES_REVERSE[rtype]
    fieldclass = getattr(models, classname)
    except (KeyError, AttributeError):
    fieldclass = models.TextField
    namespace[rname] = fieldclass(null=nullok)
    return type(table_name, (models.Model,), namespace)
    

    View Slide

46. class AdHocRouter(object):
    def db_for_read(self, model, **hints):
    return getattr(model, '_adhocdb_alias', None)
    db_for_write = db_for_read
    

    View Slide

47. MORE:
    gist.github.com/1337664
    Pro Django by Marty Alchin
    code.djangoproject.com/wiki/DynamicModels
    

    View Slide

48. USING VIEWS AS MODELS
    (3)
    

    View Slide

49. View Slide

50. #  \d  wiki
               Table  "public.wiki"
       Column    |    Type      |  Modifiers  
    -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐+-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐+-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐
     name          |  text        |  not  null
     version    |  integer  |  not  null
     time          |  bigint    |  
     author      |  text        |  
     ipnr          |  text        |  
     text          |  text        |  
     comment    |  text        |  
     readonly  |  integer  |  
    Indexes:
           "wiki_pk"  PRIMARY  KEY,  btree  (name,  version)
           "wiki_time_idx"  btree  ("time")
    

    View Slide

51. View Slide

52. FFFFFFF
    UUUUUU
    UUUUUU
    

    View Slide

53. There's
    gotta be
    a way…
    

    View Slide

54. CREATE VIEW "wiki_django_view" AS
    SELECT "name" || '.' || "version" AS "django_id", *
    FROM wiki;
    

    View Slide

55. #  select  django_id,  author  from  wiki_django_view  limit  5;
                 django_id              |              author                
    -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐+-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐
     CMSAppsComparison.73    |  whardeman@gmail.com
     DevelopersForHire.694  |  knlg2011
     Tutorials.60                    |  anonymous
     DevelopersForHire.725  |  jwilk
     SummerOfCode2011.25      |  ramiro
    

    View Slide

56. class Wiki(models.Model):
    django_id = models.TextField(primary_key=True)
    ...
    class Meta:
    db_table = u'wiki_django_view'
    managed = False
    

    View Slide

57. class Wiki(models.Model):
    django_id = models.TextField(primary_key=True)
    ...
    class Meta:
    db_table = u'wiki_django_view'
    managed = False
    

    View Slide

58. MORE:
    django.me/managed
    github.com/django/djangoproject.com
    

    View Slide

59. Would you do
    this in the
    real world?
    

    View Slide

60. "Django is a high-level Python
    Web framework that encourages
    rapid development and
    clean, pragmatic design."
    

    View Slide

61. rapid development
    clean, pragmatic design
    

    View Slide

62. rapid development
    clean, pragmatic design
    

    View Slide

63. Sometimes "do it right now"
    has to win over "do it right."
    

    View Slide

64. (but please be careful!)
    

    View Slide

65. jacob@jacobian.org
    Thank you!
    

    View Slide