The best (and worst) of Django

Given at OSCON 2011.

1. The best (and worst) of Django jacob@jacobian.org http://lanyrd.com/sgbwt/
5. 1,215,616
6. Public Enemy #1: Over-engineering.
7. ˑ ˑ “Do the simplest thing that could possibly work.” — Simon Willison
8. Small is beautiful
9. The “application” is the whole site — no apps, no components, no modules. Makes heavy use of site-wide logic: middleware, context processors and custom plugin-like concepts. The monolith monster
10. (I blame Rails.)
11. “It’s a pretty simple site; about fifteen or twenty thousand lines of code…”
12. site_of_unspeakable_horrors !""  settings.py !""  urls.py !""  views.py #""  models.py
13. models.py — 2,800 lines views.py — 11,900 lines
15. ˑ Big is bad; Small is beautiful.
16. The Django mindset: Application: some bit of functionality. Site: several applications. Spin off new apps liberally. A suite of apps ready for use.
17. 
     
           
      
    
      
         
     
    
          
     
      
     
    
        
      
      
            
     
              
       
      
    
        
       
       
        
      
       
        
            
               
      
      
     
    
               
     
     
     
      
        
    
     
       
       
      
       ! 
         
      
      
       ! 
            
              
      ! 
        "  
        
     
     
     
      #  
     
             
    
     ! 
       
     
      $    
      ! 
          
        
         
        
          
           
            
      "   
     
    
         
          %      
       &   &   
          
              
          '     
       !    ( 
    
        
       
     
    
              
       
     
       )
     
      *  +  
       !
     
       
     
    
        
    
       
                  %+ , 
     
     
        &  -
    
18. There’s no such thing as “content.”
19. XXX screenshot: “content” box
20. ˑ Django is an un-CMS.
21. The Django mindset: A great and powerful respect for data. Model the data correctly and the rest of the site will just fall out of that. Denormalization is all well and good, but never throw data away.
22. The “everything-is-a…” anti-pattern
23. “Everything needs a creation date.”
24. class  BaseObject(models.Model):        creation_date  =  models.DateField()        … class  Animal(BaseObject):        … class  Vegetable(BaseObject):        … class  Mineral(BaseObject):        …
26. Without a concrete base class: >>>  Animal.objects.all()
27. Without a concrete base class: >>>  Animal.objects.all() SELECT  ...  FROM  animal;
28. Without a concrete base class: >>>  Animal.objects.all() SELECT  ...  FROM  animal; With a concrete base class: >>>  Animal.objects.all()
29. Without a concrete base class: >>>  Animal.objects.all() SELECT  ...  FROM  animal; With a concrete base class: >>>  Animal.objects.all() SELECT  ...  FROM  "animal"  INNER  JOIN  "baseobject"   ON  ("animal"."baseobject_ptr_id"  =  "baseobject"."id")
31. What you want: >>>  BaseObject.objects.all() [<Animal:  Frog>,  <Vegetable:  Carrot>,  <Mineral:  Gold>,  ...]
32. What you want: >>>  BaseObject.objects.all() [<Animal:  Frog>,  <Vegetable:  Carrot>,  <Mineral:  Gold>,  ...] What you get: >>>  BaseObject.objects.all() [<BaseObject:  1>,  <BaseObject:  2>,  <BaseObject:  3>,  ...]
33. So maybe you try something like: def  get_some_of_everything():        qs  =  BaseObject.objects.all()        for  obj  in  qs:                for  cls  in  BaseObject.__subclasses__():                        try:                                obj  =  cls.objects.get(baseobject_ptr=obj)                                break                        except  cls.DoesNotExist:                                continue                yield  obj
34. “Our site worked fine in development and testing, and was working wonderfully for the first few months. “But we just added a bunch more data, and now our homepage takes 27 seconds to load.”
35. 1,800 queries
37. “But… but… everything really does need a creation date!”
38. “But… but… everything really does need a creation date!” So give everything a creation date.
39. ˑ ˑ “Do the simplest thing that could possibly work.” — Simon Willison
40. Abstract base classes don’t suffer from these performance problems. Denormalize into a UI-ordered auxiliary model. Non-relational databases work particular well here (I like SOLR). If you must get fancy:
41. Those who do not understand PYTHONPATH are doomed to failure.
43. >>>  import  sys >>>  sys.path ['',  '/Library/Python/2.6/site-­‐packages/pip-­‐0.8-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/pdbpp-­‐0.7-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/Pygments-­‐1.4-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/wmctrl-­‐0.1-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/pyrepl-­‐0.8.2-­‐py2.6.egg',  ...  '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6',  ...  '/Users/jacob/.local/lib/python2.6/site-­‐packages',  '/Library/Python/2.6/site-­‐packages',  ...]
44. >>>  import  sys >>>  sys.path ['',  '/Library/Python/2.6/site-­‐packages/pip-­‐0.8-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/pdbpp-­‐0.7-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/Pygments-­‐1.4-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/wmctrl-­‐0.1-­‐py2.6.egg',  '/Library/Python/2.6/site-­‐packages/pyrepl-­‐0.8.2-­‐py2.6.egg',  ...  '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6',  ...  '/Users/jacob/.local/lib/python2.6/site-­‐packages',  '/Library/Python/2.6/site-­‐packages',  ...] >>>  import  re >>>  re <module  're'  from  '/System/Library/Frameworks/Python.framework/Versions/ 2.6/lib/python2.6/re.pyc'>
45. ~/Projects/djdash !""  __init__.py !""  dashboard $      !""  __init__.py $      !""  admin.py $      !""  models.py $      #""  views.py #""  manage.py (https://github.com/jacobian/django-dev-dashboard)
47. $  ./manage.py  shell
48. $  ./manage.py  shell >>>  import  dashboard.models
49. $  ./manage.py  shell >>>  import  dashboard.models >>>  dashboard.models
50. $  ./manage.py  shell >>>  import  dashboard.models >>>  dashboard.models <module  'dashboard.models'    from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'>
51. $  ./manage.py  shell >>>  import  dashboard.models >>>  dashboard.models <module  'dashboard.models'    from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'> >>>  import  djdash.dashboard.models
52. $  ./manage.py  shell >>>  import  dashboard.models >>>  dashboard.models <module  'dashboard.models'    from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'> >>>  import  djdash.dashboard.models >>>  djdash.dashboard.models <module  'djdash.dashboard.models'  from  '/Users/jacob/Projects/djdash/../djdash/dashboard/models.pyc'>
53. $  ./manage.py  shell >>>  import  dashboard.models >>>  dashboard.models <module  'dashboard.models'    from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'> >>>  import  djdash.dashboard.models >>>  djdash.dashboard.models <module  'djdash.dashboard.models'  from  '/Users/jacob/Projects/djdash/../djdash/dashboard/models.pyc'> >>>  djdash.dashboard.models.Metric  is  dashboard.models.Metric
54. $  ./manage.py  shell >>>  import  dashboard.models >>>  dashboard.models <module  'dashboard.models'    from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'> >>>  import  djdash.dashboard.models >>>  djdash.dashboard.models <module  'djdash.dashboard.models'  from  '/Users/jacob/Projects/djdash/../djdash/dashboard/models.pyc'> >>>  djdash.dashboard.models.Metric  is  dashboard.models.Metric False
55. “Hey, many-to-many relations don’t show up in the admin.” “What’s up with these import errors when I deploy under mod_wsgi?” “Grrr… assertRaises doesn’t work!” You might have an import issue if…
56. Django is wrong! (I’m sorry.)
57. Fixing import madness 1. Use non-project-relative imports (import  app.models, not import project.app.models). 2. Use relative imports (from  .  import  x) where possible (see http://bit.ly/pep328). 3. Stop using manage.py.
58. Delete manage.py?
59. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local Delete manage.py?
60. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local >>>  import  dashboard.models Delete manage.py?
61. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local >>>  import  dashboard.models >>>  import  djdash.dashboard.models Delete manage.py?
62. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local >>>  import  dashboard.models >>>  import  djdash.dashboard.models Traceback  (most  recent  call  last) ... ImportError:  No  module  named  djdash.dashboard.models Delete manage.py?
63. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local >>>  import  dashboard.models >>>  import  djdash.dashboard.models Traceback  (most  recent  call  last) ... ImportError:  No  module  named  djdash.dashboard.models Delete manage.py?
64. $  add2virtualenv  ~/Projects/djdash $  echo  "export  DJANGO_SETTINGS_MODULE=settings.local"  \    >>  $VIRTUAL_ENV/bin/postactivate $  echo  "unset  DJANGO_SETTINGS_MODULE"  \    >>  $VIRTUAL_ENV/bin/postdeactivate $  django-­‐admin.py  shell For virtualenv users:
65. Keep (your settings) simple.
66. INSTALLED_APPS  +=  [p  for  p  in  os.listdir(BASE)                                      if  os.path.isdir(p)] Don’t do this …
67. urlpatterns  =  patterns('',  ...) for  app  in  settings.INSTALLED_APPS:        if  not  app.startswith('django'):                p  =  url('^%s/'  %  app,  include('%s.urls')  %  app)                urlpatterns  +=  patterns('',  p) … or this …
68. MIDDLEWARE_CLASSES  =  [...] def  callback(arg,  dirname,  fnames):        if  'middleware.py'  in  fnames:                m  =  '%s.middleware'  %  os.path.split(dirname)[-­‐1])                MIDDLEWARE_CLASSES.append(m) os.path.walk(BASE,  callback,  None) … or this.
69. ˑ Python’s design is predicated on the proposition that code is more often read than written.
70. INSTALLED_APPS  =  (        'django.contrib.auth',        'django.contrib.contenttypes',        'django.contrib.sessions',        'django.contrib.sites',        'django.contrib.messages',        'django.contrib.staticfiles',        'django.contrib.admin',        'django.contrib.flatpages',          'django_extensions',          'debug_toolbar',          'south',          'rs.users',          'rs.orgs',          'rs.signup',        'rs.clients',        'rs.timezones',        'rs.caregivers',        'rs.dashboard',        'rs.scripts',        'rs.reminders',        'rs.billing',        'rs.calls',        'chunks',        'contact_form', )
71. INSTALLED_APPS  =  (        'django.contrib.auth',        'django.contrib.contenttypes',        'django.contrib.sessions',        'django.contrib.sites',        'django.contrib.messages',        'django.contrib.staticfiles',        'django.contrib.admin',        'django.contrib.flatpages',          'django_extensions',          'debug_toolbar',          'south',          'rs.users',          'rs.orgs',          'rs.signup',        'rs.clients',        'rs.timezones',        'rs.caregivers',        'rs.dashboard',        'rs.scripts',        'rs.reminders',        'rs.billing',        'rs.calls',        'chunks',        'contact_form', ) urlpatterns  =  patterns('',        url(r'^admin/',  include(admin.site.urls)),        url(r'^signup/',  include('rs.signup.urls')),        url(r'^org/',  include('rs.orgs.urls')),          url(r'^clients/',  include('rs.clients.urls')),        url(r'^caregivers/',  include('rs.caregivers.urls')),        url(r'^account/',  include('rs.users.urls')),        url(r'^dashboard/',  include('rs.dashboard.urls')),        url(r'^reminders/',  include('rs.reminders.urls')),        url(r'^calls/',  include('rs.calls.urls')),        url(r'^scripts/',  include('rs.scripts.urls')),        url(r'^contact/',  include('contact_form.urls')),          url(r'^login/',  'django.contrib.auth.views.login',  {},  'login'),          url(r'^logout/$',  'django.contrib.auth.views.logout',  {},  'logout',),        url(r'^changepassword/$',  'django.contrib.auth.views.password_change') )
72. INSTALLED_APPS  =  (        'django.contrib.auth',        'django.contrib.contenttypes',        'django.contrib.sessions',        'django.contrib.sites',        'django.contrib.messages',        'django.contrib.staticfiles',        'django.contrib.admin',        'django.contrib.flatpages',          'django_extensions',          'debug_toolbar',          'south',          'rs.users',          'rs.orgs',          'rs.signup',        'rs.clients',        'rs.timezones',        'rs.caregivers',        'rs.dashboard',        'rs.scripts',        'rs.reminders',        'rs.billing',        'rs.calls',        'chunks',        'contact_form', ) urlpatterns  =  patterns('',        url(r'^admin/',  include(admin.site.urls)),        url(r'^signup/',  include('rs.signup.urls')),        url(r'^org/',  include('rs.orgs.urls')),          url(r'^clients/',  include('rs.clients.urls')),        url(r'^caregivers/',  include('rs.caregivers.urls')),        url(r'^account/',  include('rs.users.urls')),        url(r'^dashboard/',  include('rs.dashboard.urls')),        url(r'^reminders/',  include('rs.reminders.urls')),        url(r'^calls/',  include('rs.calls.urls')),        url(r'^scripts/',  include('rs.scripts.urls')),        url(r'^contact/',  include('contact_form.urls')),          url(r'^login/',  'django.contrib.auth.views.login',  {},  'login'),          url(r'^logout/$',  'django.contrib.auth.views.logout',  {},  'logout',),        url(r'^changepassword/$',  'django.contrib.auth.views.password_change') ) MIDDLEWARE_CLASSES  =  (        'django.middleware.common.CommonMiddleware',        'django.contrib.sessions.middleware.SessionMiddleware',        'django.middleware.csrf.CsrfViewMiddleware',        'django.contrib.auth.middleware.AuthenticationMiddleware',        'django.contrib.messages.middleware.MessageMiddleware',        'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware',        'debug_toolbar.middleware.DebugToolbarMiddleware', )
73. Multiple settings files
74. At the bottom of your settings file: try:        from  local_settings  import  * except  ImportError:        pass The localsettings anti-pattern
75. “It’s simple: just create a local_settings.py, throw overridden settings in there, and then never check the file into source control.”
76. “It’s simple: just create a local_settings.py, throw overridden settings in there, and then never check the file into source control.”
77. Handling multiple settings files
78. 1. Don’t. Why is your staging environment different from production? Handling multiple settings files
79. 1. Don’t. Why is your staging environment different from production? 2. Use DJANGO_SETTINGS_MODULE. Handling multiple settings files
80. settings !""  __init__.py !""  base.py !""  staging.py !""  production.py #""  local.py The one true way
81. settings !""  __init__.py !""  base.py !""  staging.py !""  production.py #""  local.py The one true way #  base.py INSTALLED_APPS  =  [...] #  local.py from  settings.base  import  * INSTALLED_APPS  +=  ['debug_toolbar']
82. settings !""  __init__.py !""  base.py !""  staging.py !""  production.py #""  local.py The one true way #  base.py INSTALLED_APPS  =  [...] #  local.py from  settings.base  import  * INSTALLED_APPS  +=  ['debug_toolbar'] $  django-­‐admin.py  shell  -­‐-­‐settings=settings.local
83. settings !""  __init__.py !""  base.py !""  staging.py !""  production.py #""  local.py The one true way #  base.py INSTALLED_APPS  =  [...] #  local.py from  settings.base  import  * INSTALLED_APPS  +=  ['debug_toolbar'] $  django-­‐admin.py  shell  -­‐-­‐settings=settings.local #  deploy.wsgi os.environ['DJANGO_SETTINGS_MODULE']  =  'settings.deploy'
84. ˑ ˑ “Do the simplest thing that could possibly work.” — Simon Willison
85. Thank you! jacob@jacobian.org http://lanyrd.com/sgbwt/