Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tracing Error Like A Boss

Tracing Error Like A Boss

Isman Firmansyah

December 21, 2013
Tweet

More Decks by Isman Firmansyah

Other Decks in Programming

Transcript

  1. Tracing Error Like A Boss (with Sentry and Docker) A

    presentation by Isman Firmansyah
  2. Life's Good New app is finished — developed, tested, and

    deployed to production without errors.
  3. The Naive Way 1. SSH to remote server 2. Finding

    the needle in a haystack (a.k.a. log file) 3. Update the code in dev env and re-deploy
  4. The Lazy Way 1. Wait for email sent by app

    for each error (with full traceback) 2. Update the code in dev env and re-deploy
  5. Error Tracing For Humans™ 1. Wait for email sent by

    app for unique error (with less traceback) 2. See the full traceback in web UI 3. Update the code in dev env and re-deploy
  6. Why Sentry It's opensource — transparent and active development Battle-tested

    at Trusted by notable shops ( , , etc) Host your own or pay the service Errors are groupped, email sent per error group Nice UI Disqus Pinterest Instagram
  7. The Easiest Way Paid Sentry Service Case is closed ...

    but then the next sections are pointless
  8. The D.I.Y. Way Host Your Own Sentry Instance What could

    be wrong? You don't familiar with Sentry/Django/Python You might have Rock Lee's syndromes
  9. Docker http://docker.io/ From its official documentation: docker, the Linux Container

    Runtime, runs Unix processes with strong guarantees of isolation across servers. Your software runs repeatably everywhere because its Container includes any dependencies.
  10. Docker: From My Own Perspective Strong foundation — built by

    people who run DotCloud A Linux container with steroids: lightweight batteries-included — git-like experience, registry, etc. opensource community awesomeness — 7696 GH stars, 1006 forks A handy tool to tackle dev-prod parity PaaS
  11. Learn Docker The Hard Way Official documentation Guidebook Videos http://docs.docker.io/

    https://github.com/kencochrane/docker-guidebook http://www.youtube.com/results? search_query=docker+dotcloud
  12. Talk Coverage Simulation of a broken app Use Sentry to

    trace the errors Use Docker container to ship Sentry No email integration (leave it as practice for others)
  13. Goal(s) At the end of this talk — hopefully —

    you'll know how to: Trace and fix errors as soon as possible — thanks to Sentry Deploy Sentry to elsewhere painlessly — thanks to Docker Get enough sleep
  14. "Hello World" App # Taken from ``dockerfiles/flask-demo/app.py``. # This snippet

    is running under Docker container. @app.route("/") def hello(): return "Hello World from Flask" @app.route("/about") def about(): a = 1 / 0 # NOQA return "Hello World from Flask, again?" @app.route("/help") def info(): func_a() return "Hello World from Flask, SRSLY?"
  15. App Dockerfile # Python env RUN apt-get install -y python-pip

    RUN pip install flask RUN pip install raven RUN pip install blinker EXPOSE 5000 ENV SENTRY_PROTO tcp ENV SENTRY_AUTH None ENV SENTRY_PROJECT 0 ADD app.py /opt/app.py CMD ["python", "/opt/app.py"]
  16. MySQL Dockerfile # boring instruction is omitted RUN apt-get -y

    install mysql-server # make the server listen to 0.0.0.0 RUN sed -i -e"s/^bind-address\s*=\s*127.0.0.1/bind-address = 0.0.0.0/" /etc/mysql/my.cnf ENV DB_USER admin ENV DB_PASSWD password ADD init-mysql.sh /opt/init-mysql.sh EXPOSE 3306 CMD ["/bin/bash", "/opt/init-mysql.sh"]
  17. MySQL & Docker # step 1: build the container sudo

    docker build -rm -t $USER/$MYSQLREPO . # step 2: run the container sudo docker run -v /opt/mysql:/var/lib/mysql -e DB_USER=admin -e DB_PASSWD=password \ -name mysql $USER/$MYSQLREPO # step 3: create a database for Sentry mysql -u $DB_USER -p -h $CONTAINER_IP
  18. Sentry Dockerfile # boring instruction is omitted RUN apt-get install

    python-dev -y RUN apt-get install python-pip -y RUN easy_install -U distribute RUN pip install sentry RUN pip install gevent RUN pip install eventlet RUN pip install django-bcrypt RUN apt-get install libmysqlclient-dev -y RUN pip install MySQL-python ADD sentry.conf.py /opt/sentry.conf.py EXPOSE 9000/tcp 9001/udp ENTRYPOINT ["sentry", "--config=/opt/sentry.conf.py"] CMD ["help"]
  19. Sentry Configuration Pt. 1 DATABASES = { 'default': { 'ENGINE':

    'django.db.backends.mysql', 'NAME': os.environ.get("SENTRY_DB_NAME", "sentry"), 'USER': os.environ.get("SENTRY_DB_USER", "admin"), 'PASSWORD': os.environ.get("SENTRY_DB_PASS", "password"), 'HOST': os.environ.get("DB_PORT_3306_TCP_ADDR", "localhost"), 'PORT': os.environ.get("DB_PORT_3306_TCP_PORT", 3306), } }
  20. Sentry Configuration Pt. 2 SENTRY_URL_PREFIX = os.environ.get( "SENTRY_URL_PREFIX", "http://0.0.0.0:9000") SENTRY_WEB_HOST

    = '0.0.0.0' SENTRY_WEB_PORT = 9000 SENTRY_WEB_OPTIONS = { 'workers': 3, # the number of gunicorn workers 'limit_request_line': 0, # required for raven-js 'secure_scheme_headers': {'X-FORWARDED-PROTO': 'https'}, "worker_class": "gevent", } SENTRY_UDP_HOST = "0.0.0.0" SENTRY_UDP_PORT = 9001 # for demo only ALLOWED_HOSTS = ["*"]
  21. Sentry & Docker # step 1: build the container sudo

    docker build -rm -t $USER/$SENTRYREPO . # step 2.1: run Sentry service sudo docker run -p 9000:9000 -name sentry -link mysql:db $USER/$SENTRYREPO start http # (optional) step 2.2: create first user if there's no existing user yet sudo docker run -i -t -link mysql:db $USER/$SENTRYREPO createsuperuser
  22. Linked Containers # MySQL container sudo docker run -v /opt/mysql:/var/lib/mysql

    -e DB_USER=admin -e DB_PASSWD=password \ -name mysql $USER/$MYSQLREPO # Sentry container sudo docker run -p 9000:9000 -name sentry -link mysql:db $USER/$SENTRYREPO start http # App container sudo docker run -p 5000:5000 -link sentry:sentry \ -e SENTRY_PROTO=tcp -e SENTRY_PROJECT=2 -e SENTRY_AUTH=$AUTH \ $USER/$APPREPO
  23. Advanced Setup Recommended for high-traffic Sentry instance: for background jobs

    for caching for buffers Rate-limit using Celery Memcache Redis Cyclops
  24. Recap Tracing errors using Sentry is relaxing Installing & deploying

    Sentry elsewhere is trivial — thanks to Docker Advanced Sentry & Docker usage might be your next adventure