Full-Stack Monitoring with django-watchman

Slide 1

Slide 1 text

FULL-STACK MONITORING WITH DJANGO-WATCHMAN https://github.com/mwarkentin/django-watchman

Slide 2

Slide 2 text

WHO AM I? Michael Warkentin Operations Engineer, Wave @mwarkentin https://www.waveapps.com

Slide 3

Slide 3 text

WHAT IS DJANGO- WATCHMAN?

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

WHO'S USING IT?

Slide 10

Slide 10 text

WAVE VM FARMS (AND CUSTOMERS) NYC PARKS

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

USING DJANGO-WATCHMAN

Slide 13

Slide 13 text

# requirements.txt pip install django-watchman # settings.py INSTALLED_APPS = ( ... 'watchman', ) # urls.py url(r'^watchman/', include('watchman.urls')),

Slide 14

Slide 14 text

HTTP://127.0.0.1:8000/WATCHMAN/ { "databases": [ { "default": { "ok": true } } ], "caches": [ { "default": { "ok": true } } ], "storage": { "ok": true } }

Slide 15

Slide 15 text

DB Config Error { "databases": [ { "default": { "ok": false, "error": "django.db.utils.OperationalError: (1045, "Access denied for user ...")", "traceback": "" } } ] }

Slide 16

Slide 16 text

http://127.0.0.1:8000/watchman/?check=watchman.checks.databases { "databases": [ { "default": { "ok": true } } ] }

Slide 17

Slide 17 text

HTTP://127.0.0.1:8000/WATCHMAN/DASHBOARD

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

$ python manage.py watchman -v 2 {"storage": {"ok": true}} {"caches": [{"default": {"ok": true}}]} {"databases": [{"default": {"ok": true}}]} $ echo $? 0

Slide 20

Slide 20 text

BUILT-IN: CACHES DATABASES EMAIL STORAGE

Slide 21

Slide 21 text

CUSTOM CHECKS

Slide 22

Slide 22 text

NO ARGUMENTS RETURN A DICT CATCH AND RETURN EXCEPTIONS

Slide 23

Slide 23 text

def rabbitmq_connection_check(): try: conn_params = pika.ConnectionParameters( settings.PIKA_CONNECTION_PARAMS['hostname'], settings.PIKA_CONNECTION_PARAMS['port'], settings.PIKA_CONNECTION_PARAMS['vhost'], pika.PlainCredentials( settings.PIKA_CONNECTION_PARAMS['username'], settings.PIKA_CONNECTION_PARAMS['password'] ) ) connection = pika.BlockingConnection(conn_params) connection.close() return {'ok': True} except Exception as e: return { 'ok': False, 'error': unicode(e), 'stacktrace': traceback.format_exc(), }

Slide 24

Slide 24 text

AUTHENTICATION? WE GOT IT!

Slide 25

Slide 25 text

TOKEN-BASED AUTH Built in Simple to use Works with tools like Pingdom

Slide 26

Slide 26 text

# settings.py WATCHMAN_TOKEN = 'foo' http://127.0.0.1:8000/watchman/?watchman-token=foo

Slide 27

Slide 27 text

CUSTOM AUTH Use an existing auth decorator or write your own auth decorator

Slide 28

Slide 28 text

RESTRICT TO DJANGO "STAFF" # settings.py WATCHMAN_AUTH_DECORATOR = 'django.contrib.admin.views.decorators.staff_member_required'

Slide 29

Slide 29 text

RESTRICT BY USER ID # core/watchman_auth.py def userid_is_one(view_func): @csrf_exempt @wraps(view_func) def _wrapped_view(request, *args, **kwargs): if request.user.id == 1: return view_func(request, *args, **kwargs) return HttpResponseForbidden() return _wrapped_view # settings.py WATCHMAN_AUTH_DECORATOR = 'core.watchman_auth.userid_is_one'

Slide 30

Slide 30 text

CAVEATS

Slide 31

Slide 31 text

CAN'T CATCH EXCEPTIONS THROWN IN CUSTOM MIDDLEWARE

Slide 32

Slide 32 text

3RD-PARTY SERVICES

Slide 33

Slide 33 text

SPRINTS Come hack on django-watchman during sprints, or get help configuring for your own site https://github.com/mwarkentin/django-watchman @mwarkentin