Stupid ORM Tricks

SFPython, November 2011

Transcript

1. Stupid ORM Tricks
2. "[Django has] an ORM with a beautiful interface - for very basic things. But once you try to do the extraordinary with it… it's frustrating to say the least." — Adam Gomaa
3. CUSTOM MODEL FIELDS (1)
4. AD HOC MODELS (2)
5. USING VIEWS AS MODELS (3)
6. Warning:
7. When it breaks, you get to keep all the pieces.
8. Hi, Internets! If you were here in person, you'd hear me say that most code in this talk is not production-worthy. But since you're not here, this'll have to do. The code presented here is untested, and uses internal, undocumented interfaces. At best, it's a hack; at worst, it'll bring your site down. It's not the kind of code you can just copy and paste! Please don't reuse code from this talk in production without considering it carefully! Only once you complete understand how it can fail — it will — should you consider using this code in your own projects. Thank you!
9. CUSTOM MODEL FIELDS (1)
10. class Contact(models.Model): name = models.CharField(max_length=200) phone = models.CharField(max_length=30)
11. what am I supposed to type here? what am I supposed to type here?
12. >>> c = Contact.objects.get(...) >>> c.phone u'yo, call me, bro!'
13. PROGRAMMER Y U NO USE PROPER TYPES?
14. from internationalphone.models import \ InternationalPhoneNumberField class Contact(models.Model): name = models.CharField(max_length=200) phone = InternationalPhoneNumberField( default_region='US')
16. >>> Contact.objects.get(name='Jacob').phone PhoneNumber(country_code=1, national_number=2125551212L, extension=None, italian_leading_zero=False, country_code_source=None, preferred_domestic_carrier_code=None)
17. Requirements: phonenumbers iso3166
18. from django.db import models class InternationalPhoneNumberField(models.Field): __metaclass__ = models.SubfieldBase description = "An international phone number." def __init__(self, *args, **kwargs): ... def to_python(self, value): ... def get_prep_value(self, value): ... def formfield(self, **kwargs): ... def get_internal_type(self): ...
19. Required:
20. class InternationalPhoneNumberField(models.Field): ... def to_python(self, value): if not value: return None return utils.parse_number(value, self.default_region)
21. class InternationalPhoneNumberField(models.Field): ... def get_prep_value(self, value): if not isinstance(value, phonenumbers.PhoneNumber): value = self.to_python(value) if value: return utils.format_number(value) else: return None
22. class InternationalPhoneNumberField (models.Field): ... def get_internal_type(self): return 'CharField'
23. Optional:
24. class InternationalPhoneNumberField(models.Field): ... def formfield(self, **kwargs): kwargs.setdefault('form_class', forms.InternationalPhoneNumberField) kwargs.setdefault('default_region', self.default_region) return super(InternationalPhoneNumberField, self).formfield(**kwargs)
25. try: import south.modelsinspector rules = [ # Field classes this rule applies to [InternationalPhoneNumberField], # Positional arguments (none). [], # Keyword arguments. { 'max_length': ["max_length", {"default": 20}], 'default_region': [ "default_region", {"default": utils.guess_default_region()} ], }, ] south.modelsinspector.add_introspection_rules( [rules], ["^internationalphone\.models"] ) except ImportError: pass
26. MORE: django.me/custom-model-fields
27. AUTO-GENERATED MODELS (2)
28. Background:
29. In Python, classes are just syntactic sugar!
30. >>> class Foo(object): ... pass ... >>> f = Foo() >>> type(f) <class '__main__.Foo'> >>> type(Foo) <type 'type'> >>> isinstance(Foo, type) True
31. >>> help(type) Help on class type in module __builtin__: class type(object) | type(object) -> the object's type | type(name, bases, dict) -> a new type
32. >>> help(type) Help on class type in module __builtin__: class type(object) | type(object) -> the object's type | type(name, bases, dict) -> a new type
33. class Foo(object): x = 1
34. Foo = type('Foo', (object,), {'x': 1})
35. class Person(models.Model): first_name = models.CharField(...) last_name = models.CharField(...) def __unicode__(self): return self.first_name
36. Person = type("Person", (models.Model), { 'first_name' : models.CharField(...), 'last_name': models.CharField(...), '__unicode__': lambda self: self.first_name, })
37. What does this mean?
38. settings.DATABASES "A dictionary containing the settings for all databases to be used with Django."
39. HARDCODE ALL THE SETTINGS!
40. But… what if…
41. Ad hoc models:
42. >>> from adhocdb.models import DB >>> lightroom = DB(alias='lightroom', ... path='.../Lightroom 3 Catalog.lrcat') >>> lightroom.model_names() ['Adobe_AdditionalMetadata', 'Adobe_imageDevelopBeforeSettings', 'Adobe_imageDevelopSettings', 'Adobe_imageProperties', 'Adobe_images', ... ] >>> Image = lightroom.model('Adobe_images') >>> Image.objects.count() 4775
43. from django import db from django.conf import settings from django.db import models class DB(object): def __init__(self, alias, path): self.alias = alias settings.DATABASES[self.alias] = { 'ENGINE': 'django.db.backends.sqlite3', 'NAME': path, } if self.alias in db.connections._connections: del db.connections._connections[self.alias] self._model_cache = None
44. from django import db from django.conf import settings from django.db import models class DB(object): ... def model_names(self): conn = db.connections[self.alias] introspection = conn.introspection return map(str, introspection.table_names())
45. from django import db from django.conf import settings from django.db import models class DB(object): ... def model_for_table(self, table_name, **namespace): introspection = db.connections[self.alias].introspection cursor = db.connections[self.alias].cursor() namespace.update({ '__module__': __name__, 'Meta': type("Meta", (object,), {'db_table': table_name}), '_adhocdb_alias': self.alias, }) for row in introspection.get_table_description(cursor, table_name): rname, rtype, _, _, _, _, nullok = row try: classname = introspection.DATA_TYPES_REVERSE[rtype] fieldclass = getattr(models, classname) except (KeyError, AttributeError): fieldclass = models.TextField namespace[rname] = fieldclass(null=nullok) return type(table_name, (models.Model,), namespace)
46. class AdHocRouter(object): def db_for_read(self, model, **hints): return getattr(model, '_adhocdb_alias', None) db_for_write = db_for_read
47. MORE: gist.github.com/1337664 Pro Django by Marty Alchin code.djangoproject.com/wiki/DynamicModels
48. USING VIEWS AS MODELS (3)
50. #  \d  wiki            Table  "public.wiki"    Column    |    Type      |  Modifiers   -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐+-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐+-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐  name          |  text        |  not  null  version    |  integer  |  not  null  time          |  bigint    |    author      |  text        |    ipnr          |  text        |    text          |  text        |    comment    |  text        |    readonly  |  integer  |   Indexes:        "wiki_pk"  PRIMARY  KEY,  btree  (name,  version)        "wiki_time_idx"  btree  ("time")
52. FFFFFFF UUUUUU UUUUUU
53. There's gotta be a way…
54. CREATE VIEW "wiki_django_view" AS SELECT "name" || '.' || "version" AS "django_id", * FROM wiki;
55. #  select  django_id,  author  from  wiki_django_view  limit  5;              django_id              |              author                 -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐+-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐  CMSAppsComparison.73    |  whardeman@gmail.com  DevelopersForHire.694  |  knlg2011  Tutorials.60                    |  anonymous  DevelopersForHire.725  |  jwilk  SummerOfCode2011.25      |  ramiro
56. class Wiki(models.Model): django_id = models.TextField(primary_key=True) ... class Meta: db_table = u'wiki_django_view' managed = False
57. class Wiki(models.Model): django_id = models.TextField(primary_key=True) ... class Meta: db_table = u'wiki_django_view' managed = False
58. MORE: django.me/managed github.com/django/djangoproject.com
59. Would you do this in the real world?
60. "Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design."
61. rapid development clean, pragmatic design
62. rapid development clean, pragmatic design
63. Sometimes "do it right now" has to win over "do it right."
64. (but please be careful!)
65. jacob@jacobian.org Thank you!