Full-Stack Monitoring with django-watchman

My Pycon Canada 2015 presentation about the open-source project django-watchman I wrote to make it easier to monitor your application + backing services (caches, databases, etc.)

Transcript

1. FULL-STACK MONITORING WITH DJANGO-WATCHMAN https://github.com/mwarkentin/django-watchman
2. WHO AM I? Michael Warkentin Operations Engineer, Wave @mwarkentin https://www.waveapps.com
3. WHAT IS DJANGO- WATCHMAN?
9. WHO'S USING IT?
10. WAVE VM FARMS (AND CUSTOMERS) NYC PARKS
12. USING DJANGO-WATCHMAN
13. # requirements.txt pip install django-watchman # settings.py INSTALLED_APPS = ( ... 'watchman', ) # urls.py url(r'^watchman/', include('watchman.urls')),
14. HTTP://127.0.0.1:8000/WATCHMAN/ { "databases": [ { "default": { "ok": true } } ], "caches": [ { "default": { "ok": true } } ], "storage": { "ok": true } }
15. DB Config Error { "databases": [ { "default": { "ok": false, "error": "django.db.utils.OperationalError: (1045, "Access denied for user ...")", "traceback": "<full traceback>" } } ] }
16. http://127.0.0.1:8000/watchman/?check=watchman.checks.databases { "databases": [ { "default": { "ok": true } } ] }
17. HTTP://127.0.0.1:8000/WATCHMAN/DASHBOARD
19. $ python manage.py watchman -v 2 {"storage": {"ok": true}} {"caches": [{"default": {"ok": true}}]} {"databases": [{"default": {"ok": true}}]} $ echo $? 0
20. BUILT-IN: CACHES DATABASES EMAIL STORAGE
21. CUSTOM CHECKS
22. NO ARGUMENTS RETURN A DICT CATCH AND RETURN EXCEPTIONS
23. def rabbitmq_connection_check(): try: conn_params = pika.ConnectionParameters( settings.PIKA_CONNECTION_PARAMS['hostname'], settings.PIKA_CONNECTION_PARAMS['port'], settings.PIKA_CONNECTION_PARAMS['vhost'], pika.PlainCredentials( settings.PIKA_CONNECTION_PARAMS['username'], settings.PIKA_CONNECTION_PARAMS['password'] ) ) connection = pika.BlockingConnection(conn_params) connection.close() return {'ok': True} except Exception as e: return { 'ok': False, 'error': unicode(e), 'stacktrace': traceback.format_exc(), }
24. AUTHENTICATION? WE GOT IT!
25. TOKEN-BASED AUTH Built in Simple to use Works with tools like Pingdom
26. # settings.py WATCHMAN_TOKEN = 'foo' http://127.0.0.1:8000/watchman/?watchman-token=foo
27. CUSTOM AUTH Use an existing auth decorator or write your own auth decorator
28. RESTRICT TO DJANGO "STAFF" # settings.py WATCHMAN_AUTH_DECORATOR = 'django.contrib.admin.views.decorators.staff_member_required'
29. RESTRICT BY USER ID # core/watchman_auth.py def userid_is_one(view_func): @csrf_exempt @wraps(view_func) def _wrapped_view(request, *args, **kwargs): if request.user.id == 1: return view_func(request, *args, **kwargs) return HttpResponseForbidden() return _wrapped_view # settings.py WATCHMAN_AUTH_DECORATOR = 'core.watchman_auth.userid_is_one'
30. CAVEATS
31. CAN'T CATCH EXCEPTIONS THROWN IN CUSTOM MIDDLEWARE
32. 3RD-PARTY SERVICES
33. SPRINTS Come hack on django-watchman during sprints, or get help configuring for your own site https://github.com/mwarkentin/django-watchman @mwarkentin