Share
Embed
Transcript
46 slides
-
Tracing Error Like A Boss (with Sentry and Docker) A
presentation by Isman Firmansyah -
$ whoami a backend engineer // on GitHub // on
Twitter iromli @iromli -
Life's Good New app is finished — developed, tested, and
deployed to production without errors. -
Time Goes On ... and On More features, dependencies, typos
Missed testcases, sloppy deploys -
Life's (not) Good My app is broken; it was running
fine few days ago. -
The Naive Way 1. SSH to remote server 2. Finding
the needle in a haystack (a.k.a. log file) 3. Update the code in dev env and re-deploy -
The Lazy Way 1. Wait for email sent by app
for each error (with full traceback) 2. Update the code in dev env and re-deploy -
Error Tracing For Humans™ 1. Wait for email sent by
app for unique error (with less traceback) 2. See the full traceback in web UI 3. Update the code in dev env and re-deploy -
Sentry http://getsentry.com/ Sentry is a realtime, platform-agnostic error logging and
aggregation platform. -
Why Sentry It's opensource — transparent and active development Battle-tested
at Trusted by notable shops ( , , etc) Host your own or pay the service Errors are groupped, email sent per error group Nice UI Disqus Pinterest Instagram -
The Easiest Way Paid Sentry Service Case is closed ...
but then the next sections are pointless -
The D.I.Y. Way Host Your Own Sentry Instance What could
be wrong? You don't familiar with Sentry/Django/Python You might have Rock Lee's syndromes -
Life's (not) Good pt. 2 Sentry doesn't work on my
CentOS production server -
You Have Dev-Prod Parity Problem TL;DR http://12factor.net/dev-prod-parity A state where
dev != production environment. -
Docker http://docker.io/ From its official documentation: docker, the Linux Container
Runtime, runs Unix processes with strong guarantees of isolation across servers. Your software runs repeatably everywhere because its Container includes any dependencies. -
Docker: From My Own Perspective Strong foundation — built by
people who run DotCloud A Linux container with steroids: lightweight batteries-included — git-like experience, registry, etc. opensource community awesomeness — 7696 GH stars, 1006 forks A handy tool to tackle dev-prod parity PaaS -
Learn Docker The Hard Way Official documentation Guidebook Videos http://docs.docker.io/
https://github.com/kencochrane/docker-guidebook http://www.youtube.com/results? search_query=docker+dotcloud -
Talk Coverage Simulation of a broken app Use Sentry to
trace the errors Use Docker container to ship Sentry No email integration (leave it as practice for others) -
Goal(s) At the end of this talk — hopefully —
you'll know how to: Trace and fix errors as soon as possible — thanks to Sentry Deploy Sentry to elsewhere painlessly — thanks to Docker Get enough sleep -
"Hello World" App # Taken from ``dockerfiles/flask-demo/app.py``. # This snippet
is running under Docker container. @app.route("/") def hello(): return "Hello World from Flask" @app.route("/about") def about(): a = 1 / 0 # NOQA return "Hello World from Flask, again?" @app.route("/help") def info(): func_a() return "Hello World from Flask, SRSLY?" -
App Dockerfile # Python env RUN apt-get install -y python-pip
RUN pip install flask RUN pip install raven RUN pip install blinker EXPOSE 5000 ENV SENTRY_PROTO tcp ENV SENTRY_AUTH None ENV SENTRY_PROJECT 0 ADD app.py /opt/app.py CMD ["python", "/opt/app.py"] -
MySQL Dockerfile # boring instruction is omitted RUN apt-get -y
install mysql-server # make the server listen to 0.0.0.0 RUN sed -i -e"s/^bind-address\s*=\s*127.0.0.1/bind-address = 0.0.0.0/" /etc/mysql/my.cnf ENV DB_USER admin ENV DB_PASSWD password ADD init-mysql.sh /opt/init-mysql.sh EXPOSE 3306 CMD ["/bin/bash", "/opt/init-mysql.sh"] -
MySQL & Docker # step 1: build the container sudo
docker build -rm -t $USER/$MYSQLREPO . # step 2: run the container sudo docker run -v /opt/mysql:/var/lib/mysql -e DB_USER=admin -e DB_PASSWD=password \ -name mysql $USER/$MYSQLREPO # step 3: create a database for Sentry mysql -u $DB_USER -p -h $CONTAINER_IP -
Sentry Dockerfile # boring instruction is omitted RUN apt-get install
python-dev -y RUN apt-get install python-pip -y RUN easy_install -U distribute RUN pip install sentry RUN pip install gevent RUN pip install eventlet RUN pip install django-bcrypt RUN apt-get install libmysqlclient-dev -y RUN pip install MySQL-python ADD sentry.conf.py /opt/sentry.conf.py EXPOSE 9000/tcp 9001/udp ENTRYPOINT ["sentry", "--config=/opt/sentry.conf.py"] CMD ["help"] -
Sentry Configuration Pt. 1 DATABASES = { 'default': { 'ENGINE':
'django.db.backends.mysql', 'NAME': os.environ.get("SENTRY_DB_NAME", "sentry"), 'USER': os.environ.get("SENTRY_DB_USER", "admin"), 'PASSWORD': os.environ.get("SENTRY_DB_PASS", "password"), 'HOST': os.environ.get("DB_PORT_3306_TCP_ADDR", "localhost"), 'PORT': os.environ.get("DB_PORT_3306_TCP_PORT", 3306), } } -
Sentry Configuration Pt. 2 SENTRY_URL_PREFIX = os.environ.get( "SENTRY_URL_PREFIX", "http://0.0.0.0:9000") SENTRY_WEB_HOST
= '0.0.0.0' SENTRY_WEB_PORT = 9000 SENTRY_WEB_OPTIONS = { 'workers': 3, # the number of gunicorn workers 'limit_request_line': 0, # required for raven-js 'secure_scheme_headers': {'X-FORWARDED-PROTO': 'https'}, "worker_class": "gevent", } SENTRY_UDP_HOST = "0.0.0.0" SENTRY_UDP_PORT = 9001 # for demo only ALLOWED_HOSTS = ["*"] -
Sentry & Docker # step 1: build the container sudo
docker build -rm -t $USER/$SENTRYREPO . # step 2.1: run Sentry service sudo docker run -p 9000:9000 -name sentry -link mysql:db $USER/$SENTRYREPO start http # (optional) step 2.2: create first user if there's no existing user yet sudo docker run -i -t -link mysql:db $USER/$SENTRYREPO createsuperuser -
Linked Containers # MySQL container sudo docker run -v /opt/mysql:/var/lib/mysql
-e DB_USER=admin -e DB_PASSWD=password \ -name mysql $USER/$MYSQLREPO # Sentry container sudo docker run -p 9000:9000 -name sentry -link mysql:db $USER/$SENTRYREPO start http # App container sudo docker run -p 5000:5000 -link sentry:sentry \ -e SENTRY_PROTO=tcp -e SENTRY_PROJECT=2 -e SENTRY_AUTH=$AUTH \ $USER/$APPREPO -
Advanced Setup Recommended for high-traffic Sentry instance: for background jobs
for caching for buffers Rate-limit using Celery Memcache Redis Cyclops -
Recap Tracing errors using Sentry is relaxing Installing & deploying
Sentry elsewhere is trivial — thanks to Docker Advanced Sentry & Docker usage might be your next adventure