The best (and worst) of Django

The best (and worst) of Django

Given at OSCON 2011.

2f5463832ccb768ccb4a1ca3607c27ef?s=128

Jacob Kaplan-Moss

March 22, 2012
Tweet

Transcript

  1. The best (and worst) of Django jacob@jacobian.org http://lanyrd.com/sgbwt/

  2. None
  3. None
  4. None
  5. 1,215,616

  6. Public Enemy #1: Over-engineering.

  7. ˑ ˑ “Do the simplest thing that could possibly work.”

    — Simon Willison
  8. Small is beautiful

  9. The “application” is the whole site — no apps, no

    components, no modules. Makes heavy use of site-wide logic: middleware, context processors and custom plugin-like concepts. The monolith monster
  10. (I blame Rails.)

  11. “It’s a pretty simple site; about fifteen or twenty thousand

    lines of code…”
  12. site_of_unspeakable_horrors !""  settings.py !""  urls.py !""  views.py #""  models.py

  13. models.py — 2,800 lines views.py — 11,900 lines

  14. None
  15. ˑ Big is bad; Small is beautiful.

  16. The Django mindset: Application: some bit of functionality. Site: several

    applications. Spin off new apps liberally. A suite of apps ready for use.
  17.          

                                                                                                                                                                                                 !                 !                         !      "                   #                   !           $        !                                                   "                     %          &   &                             '         !    (                                       )      *  +      !                                       %+ ,             &  - 
  18. There’s no such thing as “content.”

  19. XXX screenshot: “content” box

  20. ˑ Django is an un-CMS.

  21. The Django mindset: A great and powerful respect for data.

    Model the data correctly and the rest of the site will just fall out of that. Denormalization is all well and good, but never throw data away.
  22. The “everything-is-a…” anti-pattern

  23. “Everything needs a creation date.”

  24. class  BaseObject(models.Model):        creation_date  =  models.DateField()    

       … class  Animal(BaseObject):        … class  Vegetable(BaseObject):        … class  Mineral(BaseObject):        …
  25. None
  26. Without a concrete base class: >>>  Animal.objects.all()

  27. Without a concrete base class: >>>  Animal.objects.all() SELECT  ...  FROM

     animal;
  28. Without a concrete base class: >>>  Animal.objects.all() SELECT  ...  FROM

     animal; With a concrete base class: >>>  Animal.objects.all()
  29. Without a concrete base class: >>>  Animal.objects.all() SELECT  ...  FROM

     animal; With a concrete base class: >>>  Animal.objects.all() SELECT  ...  FROM  "animal"  INNER  JOIN  "baseobject"   ON  ("animal"."baseobject_ptr_id"  =  "baseobject"."id")
  30. None
  31. What you want: >>>  BaseObject.objects.all() [<Animal:  Frog>,  <Vegetable:  Carrot>,  <Mineral:

     Gold>,  ...]
  32. What you want: >>>  BaseObject.objects.all() [<Animal:  Frog>,  <Vegetable:  Carrot>,  <Mineral:

     Gold>,  ...] What you get: >>>  BaseObject.objects.all() [<BaseObject:  1>,  <BaseObject:  2>,  <BaseObject:  3>,  ...]
  33. So maybe you try something like: def  get_some_of_everything():    

       qs  =  BaseObject.objects.all()        for  obj  in  qs:                for  cls  in  BaseObject.__subclasses__():                        try:                                obj  =  cls.objects.get(baseobject_ptr=obj)                                break                        except  cls.DoesNotExist:                                continue                yield  obj
  34. “Our site worked fine in development and testing, and was

    working wonderfully for the first few months. “But we just added a bunch more data, and now our homepage takes 27 seconds to load.”
  35. 1,800 queries

  36. None
  37. “But… but… everything really does need a creation date!”

  38. “But… but… everything really does need a creation date!” So

    give everything a creation date.
  39. ˑ ˑ “Do the simplest thing that could possibly work.”

    — Simon Willison
  40. Abstract base classes don’t suffer from these performance problems. Denormalize

    into a UI-ordered auxiliary model. Non-relational databases work particular well here (I like SOLR). If you must get fancy:
  41. Those who do not understand PYTHONPATH are doomed to failure.

  42. None
  43. >>>  import  sys >>>  sys.path ['',  '/Library/Python/2.6/site-­‐packages/pip-­‐0.8-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/pdbpp-­‐0.7-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/Pygments-­‐1.4-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/wmctrl-­‐0.1-­‐py2.6.egg',

     '/Library/Python/2.6/site-­‐packages/pyrepl-­‐0.8.2-­‐py2.6.egg',  ...  '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6',  ...  '/Users/jacob/.local/lib/python2.6/site-­‐packages',  '/Library/Python/2.6/site-­‐packages',  ...]
  44. >>>  import  sys >>>  sys.path ['',  '/Library/Python/2.6/site-­‐packages/pip-­‐0.8-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/pdbpp-­‐0.7-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/Pygments-­‐1.4-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/wmctrl-­‐0.1-­‐py2.6.egg',

     '/Library/Python/2.6/site-­‐packages/pyrepl-­‐0.8.2-­‐py2.6.egg',  ...  '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6',  ...  '/Users/jacob/.local/lib/python2.6/site-­‐packages',  '/Library/Python/2.6/site-­‐packages',  ...] >>>  import  re >>>  re <module  're'  from  '/System/Library/Frameworks/Python.framework/Versions/ 2.6/lib/python2.6/re.pyc'>
  45. ~/Projects/djdash !""  __init__.py !""  dashboard $      !""  __init__.py

    $      !""  admin.py $      !""  models.py $      #""  views.py #""  manage.py (https://github.com/jacobian/django-dev-dashboard)
  46. None
  47. $  ./manage.py  shell

  48. $  ./manage.py  shell >>>  import  dashboard.models

  49. $  ./manage.py  shell >>>  import  dashboard.models >>>  dashboard.models

  50. $  ./manage.py  shell >>>  import  dashboard.models >>>  dashboard.models <module  'dashboard.models'

       from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'>
  51. $  ./manage.py  shell >>>  import  dashboard.models >>>  dashboard.models <module  'dashboard.models'

       from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'> >>>  import  djdash.dashboard.models
  52. $  ./manage.py  shell >>>  import  dashboard.models >>>  dashboard.models <module  'dashboard.models'

       from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'> >>>  import  djdash.dashboard.models >>>  djdash.dashboard.models <module  'djdash.dashboard.models'  from  '/Users/jacob/Projects/djdash/../djdash/dashboard/models.pyc'>
  53. $  ./manage.py  shell >>>  import  dashboard.models >>>  dashboard.models <module  'dashboard.models'

       from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'> >>>  import  djdash.dashboard.models >>>  djdash.dashboard.models <module  'djdash.dashboard.models'  from  '/Users/jacob/Projects/djdash/../djdash/dashboard/models.pyc'> >>>  djdash.dashboard.models.Metric  is  dashboard.models.Metric
  54. $  ./manage.py  shell >>>  import  dashboard.models >>>  dashboard.models <module  'dashboard.models'

       from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'> >>>  import  djdash.dashboard.models >>>  djdash.dashboard.models <module  'djdash.dashboard.models'  from  '/Users/jacob/Projects/djdash/../djdash/dashboard/models.pyc'> >>>  djdash.dashboard.models.Metric  is  dashboard.models.Metric False
  55. “Hey, many-to-many relations don’t show up in the admin.” “What’s

    up with these import errors when I deploy under mod_wsgi?” “Grrr… assertRaises doesn’t work!” You might have an import issue if…
  56. Django is wrong! (I’m sorry.)

  57. Fixing import madness 1. Use non-project-relative imports (import  app.models, not

    import project.app.models). 2. Use relative imports (from  .  import  x) where possible (see http://bit.ly/pep328). 3. Stop using manage.py.
  58. Delete manage.py?

  59. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local Delete manage.py?

  60. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local >>>  import  dashboard.models Delete manage.py?

  61. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local >>>  import  dashboard.models >>>  import

     djdash.dashboard.models Delete manage.py?
  62. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local >>>  import  dashboard.models >>>  import

     djdash.dashboard.models Traceback  (most  recent  call  last) ... ImportError:  No  module  named  djdash.dashboard.models Delete manage.py?
  63. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local >>>  import  dashboard.models >>>  import

     djdash.dashboard.models Traceback  (most  recent  call  last) ... ImportError:  No  module  named  djdash.dashboard.models Delete manage.py?
  64. $  add2virtualenv  ~/Projects/djdash $  echo  "export  DJANGO_SETTINGS_MODULE=settings.local"  \    >>

     $VIRTUAL_ENV/bin/postactivate $  echo  "unset  DJANGO_SETTINGS_MODULE"  \    >>  $VIRTUAL_ENV/bin/postdeactivate $  django-­‐admin.py  shell For virtualenv users:
  65. Keep (your settings) simple.

  66. INSTALLED_APPS  +=  [p  for  p  in  os.listdir(BASE)      

                                   if  os.path.isdir(p)] Don’t do this …
  67. urlpatterns  =  patterns('',  ...) for  app  in  settings.INSTALLED_APPS:    

       if  not  app.startswith('django'):                p  =  url('^%s/'  %  app,  include('%s.urls')  %  app)                urlpatterns  +=  patterns('',  p) … or this …
  68. MIDDLEWARE_CLASSES  =  [...] def  callback(arg,  dirname,  fnames):      

     if  'middleware.py'  in  fnames:                m  =  '%s.middleware'  %  os.path.split(dirname)[-­‐1])                MIDDLEWARE_CLASSES.append(m) os.path.walk(BASE,  callback,  None) … or this.
  69. ˑ Python’s design is predicated on the proposition that code

    is more often read than written.
  70. INSTALLED_APPS  =  (        'django.contrib.auth',      

     'django.contrib.contenttypes',        'django.contrib.sessions',        'django.contrib.sites',        'django.contrib.messages',        'django.contrib.staticfiles',        'django.contrib.admin',        'django.contrib.flatpages',          'django_extensions',          'debug_toolbar',          'south',          'rs.users',          'rs.orgs',          'rs.signup',        'rs.clients',        'rs.timezones',        'rs.caregivers',        'rs.dashboard',        'rs.scripts',        'rs.reminders',        'rs.billing',        'rs.calls',        'chunks',        'contact_form', )
  71. INSTALLED_APPS  =  (        'django.contrib.auth',      

     'django.contrib.contenttypes',        'django.contrib.sessions',        'django.contrib.sites',        'django.contrib.messages',        'django.contrib.staticfiles',        'django.contrib.admin',        'django.contrib.flatpages',          'django_extensions',          'debug_toolbar',          'south',          'rs.users',          'rs.orgs',          'rs.signup',        'rs.clients',        'rs.timezones',        'rs.caregivers',        'rs.dashboard',        'rs.scripts',        'rs.reminders',        'rs.billing',        'rs.calls',        'chunks',        'contact_form', ) urlpatterns  =  patterns('',        url(r'^admin/',  include(admin.site.urls)),        url(r'^signup/',  include('rs.signup.urls')),        url(r'^org/',  include('rs.orgs.urls')),          url(r'^clients/',  include('rs.clients.urls')),        url(r'^caregivers/',  include('rs.caregivers.urls')),        url(r'^account/',  include('rs.users.urls')),        url(r'^dashboard/',  include('rs.dashboard.urls')),        url(r'^reminders/',  include('rs.reminders.urls')),        url(r'^calls/',  include('rs.calls.urls')),        url(r'^scripts/',  include('rs.scripts.urls')),        url(r'^contact/',  include('contact_form.urls')),          url(r'^login/',  'django.contrib.auth.views.login',  {},  'login'),          url(r'^logout/$',  'django.contrib.auth.views.logout',  {},  'logout',),        url(r'^changepassword/$',  'django.contrib.auth.views.password_change') )
  72. INSTALLED_APPS  =  (        'django.contrib.auth',      

     'django.contrib.contenttypes',        'django.contrib.sessions',        'django.contrib.sites',        'django.contrib.messages',        'django.contrib.staticfiles',        'django.contrib.admin',        'django.contrib.flatpages',          'django_extensions',          'debug_toolbar',          'south',          'rs.users',          'rs.orgs',          'rs.signup',        'rs.clients',        'rs.timezones',        'rs.caregivers',        'rs.dashboard',        'rs.scripts',        'rs.reminders',        'rs.billing',        'rs.calls',        'chunks',        'contact_form', ) urlpatterns  =  patterns('',        url(r'^admin/',  include(admin.site.urls)),        url(r'^signup/',  include('rs.signup.urls')),        url(r'^org/',  include('rs.orgs.urls')),          url(r'^clients/',  include('rs.clients.urls')),        url(r'^caregivers/',  include('rs.caregivers.urls')),        url(r'^account/',  include('rs.users.urls')),        url(r'^dashboard/',  include('rs.dashboard.urls')),        url(r'^reminders/',  include('rs.reminders.urls')),        url(r'^calls/',  include('rs.calls.urls')),        url(r'^scripts/',  include('rs.scripts.urls')),        url(r'^contact/',  include('contact_form.urls')),          url(r'^login/',  'django.contrib.auth.views.login',  {},  'login'),          url(r'^logout/$',  'django.contrib.auth.views.logout',  {},  'logout',),        url(r'^changepassword/$',  'django.contrib.auth.views.password_change') ) MIDDLEWARE_CLASSES  =  (        'django.middleware.common.CommonMiddleware',        'django.contrib.sessions.middleware.SessionMiddleware',        'django.middleware.csrf.CsrfViewMiddleware',        'django.contrib.auth.middleware.AuthenticationMiddleware',        'django.contrib.messages.middleware.MessageMiddleware',        'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware',        'debug_toolbar.middleware.DebugToolbarMiddleware', )
  73. Multiple settings files

  74. At the bottom of your settings file: try:    

       from  local_settings  import  * except  ImportError:        pass The localsettings anti-pattern
  75. “It’s simple: just create a local_settings.py, throw overridden settings in

    there, and then never check the file into source control.”
  76. “It’s simple: just create a local_settings.py, throw overridden settings in

    there, and then never check the file into source control.”
  77. Handling multiple settings files

  78. 1. Don’t. Why is your staging environment different from production?

    Handling multiple settings files
  79. 1. Don’t. Why is your staging environment different from production?

    2. Use DJANGO_SETTINGS_MODULE. Handling multiple settings files
  80. settings !""  __init__.py !""  base.py !""  staging.py !""  production.py #""

     local.py The one true way
  81. settings !""  __init__.py !""  base.py !""  staging.py !""  production.py #""

     local.py The one true way #  base.py INSTALLED_APPS  =  [...] #  local.py from  settings.base  import  * INSTALLED_APPS  +=  ['debug_toolbar']
  82. settings !""  __init__.py !""  base.py !""  staging.py !""  production.py #""

     local.py The one true way #  base.py INSTALLED_APPS  =  [...] #  local.py from  settings.base  import  * INSTALLED_APPS  +=  ['debug_toolbar'] $  django-­‐admin.py  shell  -­‐-­‐settings=settings.local
  83. settings !""  __init__.py !""  base.py !""  staging.py !""  production.py #""

     local.py The one true way #  base.py INSTALLED_APPS  =  [...] #  local.py from  settings.base  import  * INSTALLED_APPS  +=  ['debug_toolbar'] $  django-­‐admin.py  shell  -­‐-­‐settings=settings.local #  deploy.wsgi os.environ['DJANGO_SETTINGS_MODULE']  =  'settings.deploy'
  84. ˑ ˑ “Do the simplest thing that could possibly work.”

    — Simon Willison
  85. Thank you! jacob@jacobian.org http://lanyrd.com/sgbwt/