Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The best (and worst) of Django

The best (and worst) of Django

Given at OSCON 2011.

Jacob Kaplan-Moss

March 22, 2012
Tweet

More Decks by Jacob Kaplan-Moss

Other Decks in Technology

Transcript

  1. The best
    (and worst)
    of Django
    [email protected]
    http://lanyrd.com/sgbwt/

    View full-size slide

  2. Public Enemy #1:
    Over-engineering.

    View full-size slide

  3. ˑ
    ˑ
    “Do the simplest
    thing that could
    possibly work.”
    — Simon Willison

    View full-size slide

  4. Small is beautiful

    View full-size slide

  5. The “application” is the whole site — no
    apps, no components, no modules.
    Makes heavy use of site-wide logic:
    middleware, context processors and
    custom plugin-like concepts.
    The monolith monster

    View full-size slide

  6. (I blame Rails.)

    View full-size slide

  7. “It’s a pretty simple site; about fifteen or
    twenty thousand lines of code…”

    View full-size slide

  8. site_of_unspeakable_horrors
    !""  settings.py
    !""  urls.py
    !""  views.py
    #""  models.py

    View full-size slide

  9. models.py — 2,800 lines
    views.py — 11,900 lines

    View full-size slide

  10. ˑ
    Big is bad;
    Small is beautiful.

    View full-size slide

  11. The Django mindset:
    Application: some bit of functionality.
    Site: several applications.
    Spin off new apps liberally.
    A suite of apps ready for use.

    View full-size slide




















































  12. !


    !









    !

    "


    #




    !


    $



    !










    "





    %




    & &









    '


    !
    (









    )

    *
    +
    !











    %+ ,



    &

    -

    View full-size slide

  13. There’s no such thing
    as “content.”

    View full-size slide

  14. XXX screenshot:
    “content” box

    View full-size slide

  15. ˑ
    Django is an
    un-CMS.

    View full-size slide

  16. The Django mindset:
    A great and powerful respect for data.
    Model the data correctly and the rest
    of the site will just fall out of that.
    Denormalization is all well and good, but
    never throw data away.

    View full-size slide

  17. The
    “everything-is-a…”
    anti-pattern

    View full-size slide

  18. “Everything needs a creation date.”

    View full-size slide

  19. class  BaseObject(models.Model):
           creation_date  =  models.DateField()
           …
    class  Animal(BaseObject):
           …
    class  Vegetable(BaseObject):
           …
    class  Mineral(BaseObject):
           …

    View full-size slide

  20. Without a concrete base class:
    >>>  Animal.objects.all()

    View full-size slide

  21. Without a concrete base class:
    >>>  Animal.objects.all()
    SELECT  ...  FROM  animal;

    View full-size slide

  22. Without a concrete base class:
    >>>  Animal.objects.all()
    SELECT  ...  FROM  animal;
    With a concrete base class:
    >>>  Animal.objects.all()

    View full-size slide

  23. Without a concrete base class:
    >>>  Animal.objects.all()
    SELECT  ...  FROM  animal;
    With a concrete base class:
    >>>  Animal.objects.all()
    SELECT  ...  FROM  "animal"  INNER  JOIN  "baseobject"  
    ON  ("animal"."baseobject_ptr_id"  =  "baseobject"."id")

    View full-size slide

  24. What you want:
    >>>  BaseObject.objects.all()
    [,  ,  ,  ...]

    View full-size slide

  25. What you want:
    >>>  BaseObject.objects.all()
    [,  ,  ,  ...]
    What you get:
    >>>  BaseObject.objects.all()
    [,  ,  ,  ...]

    View full-size slide

  26. So maybe you try something like:
    def  get_some_of_everything():
           qs  =  BaseObject.objects.all()
           for  obj  in  qs:
                   for  cls  in  BaseObject.__subclasses__():
                           try:
                                   obj  =  cls.objects.get(baseobject_ptr=obj)
                                   break
                           except  cls.DoesNotExist:
                                   continue
                   yield  obj

    View full-size slide

  27. “Our site worked fine in development
    and testing, and was working
    wonderfully for the first few months.
    “But we just added a bunch more data,
    and now our homepage takes 27
    seconds to load.”

    View full-size slide

  28. 1,800 queries

    View full-size slide

  29. “But… but… everything really does need
    a creation date!”

    View full-size slide

  30. “But… but… everything really does need
    a creation date!”
    So give everything a creation date.

    View full-size slide

  31. ˑ
    ˑ
    “Do the simplest
    thing that could
    possibly work.”
    — Simon Willison

    View full-size slide

  32. Abstract base classes don’t suffer from
    these performance problems.
    Denormalize into a UI-ordered
    auxiliary model.
    Non-relational databases work
    particular well here (I like SOLR).
    If you must get fancy:

    View full-size slide

  33. Those who do not
    understand PYTHONPATH
    are doomed to failure.

    View full-size slide

  34. >>>  import  sys
    >>>  sys.path
    ['',
     '/Library/Python/2.6/site-­‐packages/pip-­‐0.8-­‐py2.6.egg',
     '/Library/Python/2.6/site-­‐packages/pdbpp-­‐0.7-­‐py2.6.egg',
     '/Library/Python/2.6/site-­‐packages/Pygments-­‐1.4-­‐py2.6.egg',
     '/Library/Python/2.6/site-­‐packages/wmctrl-­‐0.1-­‐py2.6.egg',
     '/Library/Python/2.6/site-­‐packages/pyrepl-­‐0.8.2-­‐py2.6.egg',
     ...
     '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6',
     ...
     '/Users/jacob/.local/lib/python2.6/site-­‐packages',
     '/Library/Python/2.6/site-­‐packages',
     ...]

    View full-size slide

  35. >>>  import  sys
    >>>  sys.path
    ['',
     '/Library/Python/2.6/site-­‐packages/pip-­‐0.8-­‐py2.6.egg',
     '/Library/Python/2.6/site-­‐packages/pdbpp-­‐0.7-­‐py2.6.egg',
     '/Library/Python/2.6/site-­‐packages/Pygments-­‐1.4-­‐py2.6.egg',
     '/Library/Python/2.6/site-­‐packages/wmctrl-­‐0.1-­‐py2.6.egg',
     '/Library/Python/2.6/site-­‐packages/pyrepl-­‐0.8.2-­‐py2.6.egg',
     ...
     '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6',
     ...
     '/Users/jacob/.local/lib/python2.6/site-­‐packages',
     '/Library/Python/2.6/site-­‐packages',
     ...]
    >>>  import  re
    >>>  re
    2.6/lib/python2.6/re.pyc'>

    View full-size slide

  36. ~/Projects/djdash
    !""  __init__.py
    !""  dashboard
    $      !""  __init__.py
    $      !""  admin.py
    $      !""  models.py
    $      #""  views.py
    #""  manage.py
    (https://github.com/jacobian/django-dev-dashboard)

    View full-size slide

  37. $  ./manage.py  shell

    View full-size slide

  38. $  ./manage.py  shell
    >>>  import  dashboard.models

    View full-size slide

  39. $  ./manage.py  shell
    >>>  import  dashboard.models
    >>>  dashboard.models

    View full-size slide

  40. $  ./manage.py  shell
    >>>  import  dashboard.models
    >>>  dashboard.models
     from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'>

    View full-size slide

  41. $  ./manage.py  shell
    >>>  import  dashboard.models
    >>>  dashboard.models
     from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'>
    >>>  import  djdash.dashboard.models

    View full-size slide

  42. $  ./manage.py  shell
    >>>  import  dashboard.models
    >>>  dashboard.models
     from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'>
    >>>  import  djdash.dashboard.models
    >>>  djdash.dashboard.models
     from  '/Users/jacob/Projects/djdash/../djdash/dashboard/models.pyc'>

    View full-size slide

  43. $  ./manage.py  shell
    >>>  import  dashboard.models
    >>>  dashboard.models
     from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'>
    >>>  import  djdash.dashboard.models
    >>>  djdash.dashboard.models
     from  '/Users/jacob/Projects/djdash/../djdash/dashboard/models.pyc'>
    >>>  djdash.dashboard.models.Metric  is  dashboard.models.Metric

    View full-size slide

  44. $  ./manage.py  shell
    >>>  import  dashboard.models
    >>>  dashboard.models
     from  '/Users/jacob/Projects/djdash/dashboard/models.pyc'>
    >>>  import  djdash.dashboard.models
    >>>  djdash.dashboard.models
     from  '/Users/jacob/Projects/djdash/../djdash/dashboard/models.pyc'>
    >>>  djdash.dashboard.models.Metric  is  dashboard.models.Metric
    False

    View full-size slide

  45. “Hey, many-to-many relations don’t
    show up in the admin.”
    “What’s up with these import errors
    when I deploy under mod_wsgi?”
    “Grrr… assertRaises doesn’t work!”
    You might have an
    import issue if…

    View full-size slide

  46. Django is wrong!
    (I’m sorry.)

    View full-size slide

  47. Fixing import madness
    1. Use non-project-relative imports
    (import  app.models, not import
    project.app.models).
    2. Use relative imports (from  .  import  x)
    where possible (see http://bit.ly/pep328).
    3. Stop using manage.py.

    View full-size slide

  48. Delete manage.py?

    View full-size slide

  49. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local
    Delete manage.py?

    View full-size slide

  50. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local
    >>>  import  dashboard.models
    Delete manage.py?

    View full-size slide

  51. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local
    >>>  import  dashboard.models
    >>>  import  djdash.dashboard.models
    Delete manage.py?

    View full-size slide

  52. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local
    >>>  import  dashboard.models
    >>>  import  djdash.dashboard.models
    Traceback  (most  recent  call  last)
    ...
    ImportError:  No  module  named  djdash.dashboard.models
    Delete manage.py?

    View full-size slide

  53. $  django-­‐admin.py  shell  -­‐-­‐pythonpath=`pwd`  -­‐-­‐settings=settings.local
    >>>  import  dashboard.models
    >>>  import  djdash.dashboard.models
    Traceback  (most  recent  call  last)
    ...
    ImportError:  No  module  named  djdash.dashboard.models
    Delete manage.py?

    View full-size slide

  54. $  add2virtualenv  ~/Projects/djdash
    $  echo  "export  DJANGO_SETTINGS_MODULE=settings.local"  \
       >>  $VIRTUAL_ENV/bin/postactivate
    $  echo  "unset  DJANGO_SETTINGS_MODULE"  \
       >>  $VIRTUAL_ENV/bin/postdeactivate
    $  django-­‐admin.py  shell
    For virtualenv users:

    View full-size slide

  55. Keep
    (your settings)
    simple.

    View full-size slide

  56. INSTALLED_APPS  +=  [p  for  p  in  os.listdir(BASE)
                                         if  os.path.isdir(p)]
    Don’t do this …

    View full-size slide

  57. urlpatterns  =  patterns('',  ...)
    for  app  in  settings.INSTALLED_APPS:
           if  not  app.startswith('django'):
                   p  =  url('^%s/'  %  app,  include('%s.urls')  %  app)
                   urlpatterns  +=  patterns('',  p)
    … or this …

    View full-size slide

  58. MIDDLEWARE_CLASSES  =  [...]
    def  callback(arg,  dirname,  fnames):
           if  'middleware.py'  in  fnames:
                   m  =  '%s.middleware'  %  os.path.split(dirname)[-­‐1])
                   MIDDLEWARE_CLASSES.append(m)
    os.path.walk(BASE,  callback,  None)
    … or this.

    View full-size slide

  59. ˑ
    Python’s design is predicated
    on the proposition that
    code is more often
    read than written.

    View full-size slide

  60. INSTALLED_APPS  =  (
           'django.contrib.auth',
           'django.contrib.contenttypes',
           'django.contrib.sessions',
           'django.contrib.sites',
           'django.contrib.messages',
           'django.contrib.staticfiles',
           'django.contrib.admin',
           'django.contrib.flatpages',  
           'django_extensions',  
           'debug_toolbar',  
           'south',  
           'rs.users',  
           'rs.orgs',  
           'rs.signup',
           'rs.clients',
           'rs.timezones',
           'rs.caregivers',
           'rs.dashboard',
           'rs.scripts',
           'rs.reminders',
           'rs.billing',
           'rs.calls',
           'chunks',
           'contact_form',
    )

    View full-size slide

  61. INSTALLED_APPS  =  (
           'django.contrib.auth',
           'django.contrib.contenttypes',
           'django.contrib.sessions',
           'django.contrib.sites',
           'django.contrib.messages',
           'django.contrib.staticfiles',
           'django.contrib.admin',
           'django.contrib.flatpages',  
           'django_extensions',  
           'debug_toolbar',  
           'south',  
           'rs.users',  
           'rs.orgs',  
           'rs.signup',
           'rs.clients',
           'rs.timezones',
           'rs.caregivers',
           'rs.dashboard',
           'rs.scripts',
           'rs.reminders',
           'rs.billing',
           'rs.calls',
           'chunks',
           'contact_form',
    )
    urlpatterns  =  patterns('',
           url(r'^admin/',  include(admin.site.urls)),
           url(r'^signup/',  include('rs.signup.urls')),
           url(r'^org/',  include('rs.orgs.urls')),  
           url(r'^clients/',  include('rs.clients.urls')),
           url(r'^caregivers/',  include('rs.caregivers.urls')),
           url(r'^account/',  include('rs.users.urls')),
           url(r'^dashboard/',  include('rs.dashboard.urls')),
           url(r'^reminders/',  include('rs.reminders.urls')),
           url(r'^calls/',  include('rs.calls.urls')),
           url(r'^scripts/',  include('rs.scripts.urls')),
           url(r'^contact/',  include('contact_form.urls')),  
           url(r'^login/',  'django.contrib.auth.views.login',  {},  'login'),  
           url(r'^logout/$',  'django.contrib.auth.views.logout',  {},  'logout',),
           url(r'^changepassword/$',  'django.contrib.auth.views.password_change')
    )

    View full-size slide

  62. INSTALLED_APPS  =  (
           'django.contrib.auth',
           'django.contrib.contenttypes',
           'django.contrib.sessions',
           'django.contrib.sites',
           'django.contrib.messages',
           'django.contrib.staticfiles',
           'django.contrib.admin',
           'django.contrib.flatpages',  
           'django_extensions',  
           'debug_toolbar',  
           'south',  
           'rs.users',  
           'rs.orgs',  
           'rs.signup',
           'rs.clients',
           'rs.timezones',
           'rs.caregivers',
           'rs.dashboard',
           'rs.scripts',
           'rs.reminders',
           'rs.billing',
           'rs.calls',
           'chunks',
           'contact_form',
    )
    urlpatterns  =  patterns('',
           url(r'^admin/',  include(admin.site.urls)),
           url(r'^signup/',  include('rs.signup.urls')),
           url(r'^org/',  include('rs.orgs.urls')),  
           url(r'^clients/',  include('rs.clients.urls')),
           url(r'^caregivers/',  include('rs.caregivers.urls')),
           url(r'^account/',  include('rs.users.urls')),
           url(r'^dashboard/',  include('rs.dashboard.urls')),
           url(r'^reminders/',  include('rs.reminders.urls')),
           url(r'^calls/',  include('rs.calls.urls')),
           url(r'^scripts/',  include('rs.scripts.urls')),
           url(r'^contact/',  include('contact_form.urls')),  
           url(r'^login/',  'django.contrib.auth.views.login',  {},  'login'),  
           url(r'^logout/$',  'django.contrib.auth.views.logout',  {},  'logout',),
           url(r'^changepassword/$',  'django.contrib.auth.views.password_change')
    )
    MIDDLEWARE_CLASSES  =  (
           'django.middleware.common.CommonMiddleware',
           'django.contrib.sessions.middleware.SessionMiddleware',
           'django.middleware.csrf.CsrfViewMiddleware',
           'django.contrib.auth.middleware.AuthenticationMiddleware',
           'django.contrib.messages.middleware.MessageMiddleware',
           'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware',
           'debug_toolbar.middleware.DebugToolbarMiddleware',
    )

    View full-size slide

  63. Multiple settings files

    View full-size slide

  64. At the bottom of your settings file:
    try:
           from  local_settings  import  *
    except  ImportError:
           pass
    The localsettings
    anti-pattern

    View full-size slide

  65. “It’s simple: just create a
    local_settings.py, throw overridden
    settings in there, and then never check
    the file into source control.”

    View full-size slide

  66. “It’s simple: just create a
    local_settings.py, throw overridden
    settings in there, and then never check
    the file into source control.”

    View full-size slide

  67. Handling multiple
    settings files

    View full-size slide

  68. 1. Don’t.
    Why is your staging environment
    different from production?
    Handling multiple
    settings files

    View full-size slide

  69. 1. Don’t.
    Why is your staging environment
    different from production?
    2. Use DJANGO_SETTINGS_MODULE.
    Handling multiple
    settings files

    View full-size slide

  70. settings
    !""  __init__.py
    !""  base.py
    !""  staging.py
    !""  production.py
    #""  local.py
    The one true way

    View full-size slide

  71. settings
    !""  __init__.py
    !""  base.py
    !""  staging.py
    !""  production.py
    #""  local.py
    The one true way
    #  base.py
    INSTALLED_APPS  =  [...]
    #  local.py
    from  settings.base  import  *
    INSTALLED_APPS  +=  ['debug_toolbar']

    View full-size slide

  72. settings
    !""  __init__.py
    !""  base.py
    !""  staging.py
    !""  production.py
    #""  local.py
    The one true way
    #  base.py
    INSTALLED_APPS  =  [...]
    #  local.py
    from  settings.base  import  *
    INSTALLED_APPS  +=  ['debug_toolbar']
    $  django-­‐admin.py  shell  -­‐-­‐settings=settings.local

    View full-size slide

  73. settings
    !""  __init__.py
    !""  base.py
    !""  staging.py
    !""  production.py
    #""  local.py
    The one true way
    #  base.py
    INSTALLED_APPS  =  [...]
    #  local.py
    from  settings.base  import  *
    INSTALLED_APPS  +=  ['debug_toolbar']
    $  django-­‐admin.py  shell  -­‐-­‐settings=settings.local
    #  deploy.wsgi
    os.environ['DJANGO_SETTINGS_MODULE']  =  'settings.deploy'

    View full-size slide

  74. ˑ
    ˑ
    “Do the simplest
    thing that could
    possibly work.”
    — Simon Willison

    View full-size slide

  75. Thank you!
    [email protected]
    http://lanyrd.com/sgbwt/

    View full-size slide