Reed Wade: Bottle + uWSGI: simple web app configuration and fun hidden features

Reed Wade: Bottle + uWSGI: simple web app configuration and fun hidden features

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Reed Wade:
Bottle + uWSGI: simple web app configuration and fun hidden features
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
@ Kiwi PyCon 2014 - Sunday, 14 Sep 2014 - Track 2
http://kiwi.pycon.org/

**Audience level**

Intermediate

**Description**

I will introduce the Bottle Python Web Framework and show how simple it is to configure a python web application to run under Nginx using uWSGI. I'll also demonstrate some lesser known features of uWSGI that let your web app work a little smarter.

**Abstract**

The Bottle Python Web Framework makes it easy to create a little web application. Nginx and uWSGI make it easy for your web app to scale and perform well. In this talk I'll describe how to create and configure an API focused web application.

I'll demonstrate some features of uWSGI which allow your web app to share data amongst separate processes and delegate work to non-client facing processes.

I'll show how all this can fit together with a full stack of: bottle+uWSGI+Nginx+Bootstrap+AngularJS

**YouTube**

https://www.youtube.com/watch?v=wGPGRW1wXdk

6b880a0b67fac54c42c77fe70d97334d?s=128

New Zealand Python User Group

September 14, 2014
Tweet

Transcript

  1. 1.

    Your Little Web API+app bottle + uWSGI + nginx +

    angularJS typist.geek.nz Reed Wade Kiwi Pycon 2014
  2. 2.

    Agenda - make a web API - make a web

    API+App - add some uWSGI trickery - try it at home
  3. 4.

    Stack - bottle +a little of YOUR python code -

    uWSGI +a little boilerplate config - Nginx +a little boilerplate config - AngularJS +a little of YOUR html+javascript
  4. 7.

    bottle (reed ♥s) import bottle @bottle.route('/hello') def hello(): return "Hi\n"

    @bottle.route('/hello/<name>') def hello_name(name): return "Hello {name}\n".format(name=name) @bottle.route('/api/fun') def hello_fun(): return { 'fun': 123, } if __name__ == '__main__': bottle.run(host='0.0.0.0', port=8000, debug=True) URL routing return a string to the browser return a dict and it’s magically converted to JSON! sweet convenient dev web server
  5. 8.

    more like real life import bottle @bottle.route('/api/fun') def hello_fun(): return

    { 'fun': 123, } @bottle.route('/api/slow') def slow(): time.sleep(8) return "you didn't plan this well" if __name__ == '__main__': bottle.run(host='0.0.0.0', port=8000, debug=True) I wish we could tell the user not to worry. I wish the entire server wasn’t blocked right now.
  6. 9.

    you need a proper application server uWSGI provides an operating

    environment between the web server and your WSGI compliant application. your python code Bottle uWSGI Nginx web browser AngularJS (and friends) your html+javascript
  7. 10.

    /etc/uwsgi/apps-enabled/example-uwsgi.ini [uwsgi] socket = localhost:8002 chdir = /opt/example file =

    example.py master = true plugins = python uid = www-data gid = www-data workers = 6 Your code web server will find us here (not HTTP)
  8. 11.

    you need a fab web server Nginx is a simple

    high performance web server which is flexible and easy to configure. And it knows how to talk to uWSGI. your python code Bottle uWSGI Nginx web browser AngularJS (and friends) your html+javascript
  9. 12.

    /etc/nginx/sites-enabled/example-nginx.conf upstream example { server localhost:8002; } server { server_name

    example-server; root /opt/example/www; index index.html; location / { try_files $uri $uri/ =404; } location /api { include uwsgi_params; uwsgi_pass example; } } tells Nginx to send any URL starting /api to your uWSGI applications uWSGI protocol socket (not HTTP) let Nginx serve static files directly
  10. 15.

    Let’s play with AngularJS an md5 hash calculator you type

    here it updates here and shows a new md5 value here
  11. 16.

    import bottle import hashlib app = application = bottle.Bottle() @app.route('/compute_md5')

    def do_the_trick(): return { 'result': hashlib.md5( bottle.request.query.v ).hexdigest() } if __name__ == '__main__': app.run(application=app, host='0.0.0.0', port=8000, debug=True)
  12. 17.

    <!DOCTYPE html> <html lang="en" ng-app="md5"> <head> <script src="http://ajax.googleapis.com/ajax/libs/angularjs/1.2.23/angular.min. js"></script> <script

    type="text/javascript"> var app = angular.module('md5', []); app.controller('c', function($scope,$http){ $scope.update = function() { $http.get('/compute_md5?v='+$scope.i) .success(function(data){ $scope.result = data['result']; }); } }); </script> </head> <body ng-controller="c"> <input ng-model="i" ng-change="update()" /> {{i}} is {{result}} </body> </html>
  13. 18.

    done? It works, go do something else now. Too late.

    I’d started reading the uWSGI docs. - mules! - shared memory! - signals! - other stuff not covered today! J2EE flashbacks
  14. 19.

    mules and workers - workers service client requests - mules

    don’t number of each is set in uWSGI config for your app - also ‘programmed mule’
  15. 20.

    Caches and Queues (not the names I’d have used but

    they’re not insane) - caches are a key/value store - queues are arrays in your uWSGI config file-- cache2 = name=mycache,items=100 queue = 100
  16. 23.

    Signals 1 byte message sent to listeners: - all workers

    - all mules - worker or mule by id - one “available” worker or mule sent explicitly or by timer (or file modification)
  17. 24.

    Sending Signals Receive a signal: def thanks(signal): print "something happened",

    uwsgi.mule_id() uwsgi.register_signal(111, "mule", thanks) Triggers: uwsgi.add_timer(111, 60) uwsgi.signal(111) uwsgi.add_file_monitor(111, "/tmp/cheesecake")
  18. 26.
  19. 27.

    <!DOCTYPE html> <html lang="en" ng-app="bakery"> <head> <script src="http://ajax.googleapis.com/ajax/libs/angularjs/1.2.23/angular.min. js"></script> <script

    type="text/javascript"> var app = angular.module('bakery', []); app.controller('c', function($scope,$http){ $scope.place_order = function(item) { $http.get('/api/place_order/'+item).success(function(data){ $scope.order_id = data['order_id']; }); }; $scope.check_inventory = function(item) { $http.get('/api/check_inventory').success(function(data){ $scope.inventory = data['result']; }); }; }); </script> </head>
  20. 28.

    <body ng-controller="c"> <input type="button" ng-click="place_order('cake')" value="Order a Cake" /> <input

    type="button" ng-click="place_order('pie')" value="Order a Pie" /> {{order_id}} <hr/> <input type="button" ng-click="check_inventory()" value="Check Inventory" /> <hr/> Pies: <ul> <li ng-repeat="item in inventory" ng-show="item.item=='pie'"> {{item.order_id}} : {{item.notes}} </li> </ul> Cakes: <ul> <li ng-repeat="item in inventory" ng-show="item.item=='cake'"> {{item.order_id}} : {{item.notes}} </li> </ul> </body> </html>
  21. 29.

    NEW_ORDER_SIGNAL = 10 @app.route('/api/place_order/<item>') def place_order(item): order_id = str(uuid.uuid4())[-6:] uwsgi.queue_push("{order_id}\t{item}\tnew\tthanks".format(

    order_id=order_id, item=item)) uwsgi.signal(NEW_ORDER_SIGNAL) return {'order_id': order_id} def mule_new_order(signum): print "mule {mule_id} sees a new order".format( mule_id=uwsgi.mule_id()) item = uwsgi.queue_pop() item = item.split('\t') item[2] = 'baking' item[3] = 'mule {mule_id} is baking you a {thing}'.format( mule_id=uwsgi.mule_id(), thing=item[1]) uwsgi.queue_push('\t'.join(item)) uwsgi.register_signal(NEW_ORDER_SIGNAL, "mule", mule_new_order)
  22. 30.

    @app.route('/api/check_inventory') def check_inventory(): out = [] for i in range(0,

    uwsgi.queue_size): item = uwsgi.queue_get(i) if item: item = item.split('\t') out.append({ 'order_id':item[0], 'item': item[1], 'state': item[2], 'notes': item[3] }) return {'result': out} @app.route('/api/reload') def reload(): uwsgi.reload() return "reloading"
  23. 31.

    opportunities for disaster Sat Sep 13 10:41:06 2014 - ***

    Operational MODE: preforking *** Sat Sep 13 10:41:06 2014 - *** no app loaded. going in full dynamic mode *** Sat Sep 13 10:41:06 2014 - *** uWSGI is running in multiple interpreter mode *** Sat Sep 13 10:41:06 2014 - !!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!! Sat Sep 13 10:41:06 2014 - no request plugin is loaded, you will not be able to manage requests. Sat Sep 13 10:41:06 2014 - you may need to install the package for your language of choice, or simply load it with --plugin. Sat Sep 13 10:41:06 2014 - !!!!!!!!!!! END OF WARNING !!!!!!!!!! Sat Sep 13 10:41:06 2014 - spawned uWSGI master process (pid: 14552) Sat Sep 13 10:41:06 2014 - spawned uWSGI worker 1 (pid: 14553, cores: 1) Sat Sep 13 10:41:06 2014 - spawned uWSGI worker 2 (pid: 14554, cores: 1) (ALSO -- Application Zero means python) Sat Sep 13 11:17:08 2014 - WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x20c0f80 pid: 15721 (default app)
  24. 32.

    opportunities for disaster [uwsgi] socket = localhost:8005 chdir = /opt/example

    master = true plugins = python file = example.py uid = www-data gid = www-data workers = 6 order matters? no request plugin?
  25. 33.

    nginx didn’t attempt to route to uWSGI -- didn’t think

    it was supposed to the nginx config must not be configured to route /booger to a uWSGI upstream server
  26. 34.

    this means nginx can’t reach uWSGI service - but it

    tried 2014/09/13 20:59:24 [error] 20743#0: *353 connect() failed (111: Connection refused) while connecting to upstream, client: 111.69.102.38, server: typist.geek.nz, request: "GET /compute_md5?v=jkkjknjk HTTP/1.1", upstream: "uwsgi: //127.0.0.1:8002", host: "typist.geek.nz"
  27. 35.

    nginx passed request to uWSGI but your application hasn’t made

    a route for the URL nginx sent /api/booger to a uWSGI upstream server but it wasn’t one that had a route for that URL
  28. 36.

    - at import time * Restarting app server(s) uwsgi [

    OK ] Sat Sep 13 21:23:08 2014 - *** Operational MODE: preforking *** Traceback (most recent call last): File "compute_md5.py", line 3, in <module> assert False AssertionError Sat Sep 13 21:23:08 2014 - unable to load app 0 (mountpoint='') (callable not found or import error) Sat Sep 13 21:23:08 2014 - *** no app loaded. going in full dynamic mode *** Sat Sep 13 21:23:08 2014 - *** uWSGI is running in multiple interpreter mode *** Sat Sep 13 21:23:08 2014 - spawned uWSGI master process (pid: 25219) Sat Sep 13 21:23:08 2014 - spawned uWSGI worker 1 (pid: 25226, cores: 1) Sat Sep 13 21:23:08 2014 - spawned uWSGI worker 2 (pid: 25229, cores: 1) Sat Sep 13 21:23:08 2014 - spawned uWSGI worker 3 (pid: 25230, cores: 1) Sat Sep 13 21:23:08 2014 - spawned uWSGI worker 4 (pid: 25231, cores: 1) Sat Sep 13 21:23:08 2014 - spawned uWSGI worker 5 (pid: 25232, cores: 1) Sat Sep 13 21:23:08 2014 - spawned uWSGI worker 6 (pid: 25233, cores: 1)
  29. 37.

    your uWSGI app broke - at request time Traceback (most

    recent call last): File "/usr/lib/python2.7/dist-packages/bottle.py", line 856, in _handle return route.call(**args) File "/usr/lib/python2.7/dist-packages/bottle.py", line 1721, in wrapper rv = callback(*a, **ka) File "compute_md5.py", line 10, in hello assert False AssertionError [pid: 24351|app: 0|req: 1/1] 111.69.102.38 () {42 vars in 783 bytes} [Sat Sep 13 21:07: 18 2014] GET /compute_md5?v=jkkjknjk => generated 764 bytes in 19 msecs (HTTP/1.1 500) 2 headers in 99 bytes (1 switches on core 0)
  30. 38.

    tips - printing to stdout goes to uWSGI log for

    app - touch-reload - read the uWSGI docs
  31. 39.

    References - http://bottlepy.org/ - http://uwsgi-docs.readthedocs.org/ - http://uwsgi-docs.readthedocs.org/en/latest/PythonModule.html - http://uwsgi-docs.readthedocs.org/en/latest/Signals.html -

    http://uwsgi-docs.readthedocs.org/en/latest/Caching.html - http://uwsgi-docs.readthedocs.org/en/latest/Queue.html - http://michael.lustfield.net/nginx/bottle-uwsgi-nginx-quickstart - http://campus.codeschool.com/courses/shaping-up-with-angular-js/intro - https://angularjs.org/ - https://www.youtube.com/user/dwahlin/videos More details and examples at: - https://typist.geek.nz/
  32. 40.