Your Little
Web API+app
bottle + uWSGI
+ nginx
+ angularJS
typist.geek.nz
Reed Wade
Kiwi Pycon 2014
Slide 2
Slide 2 text
Agenda
- make a web API
- make a web API+App
- add some uWSGI trickery
- try it at home
Slide 3
Slide 3 text
Stack
- bottle
- uWSGI
- Nginx
- AngularJS
Slide 4
Slide 4 text
Stack
- bottle +a little of YOUR python code
- uWSGI +a little boilerplate config
- Nginx +a little boilerplate config
- AngularJS +a little of YOUR html+javascript
Slide 5
Slide 5 text
your python code
Bottle
uWSGI
Nginx
web browser
AngularJS (and friends)
your html+javascript
Slide 6
Slide 6 text
your python code
web API
your html+javascript
you have 2 different kinds of customer
Slide 7
Slide 7 text
bottle (reed ♥s)
import bottle
@bottle.route('/hello')
def hello():
return "Hi\n"
@bottle.route('/hello/')
def hello_name(name):
return "Hello {name}\n".format(name=name)
@bottle.route('/api/fun')
def hello_fun():
return {
'fun': 123,
}
if __name__ == '__main__':
bottle.run(host='0.0.0.0', port=8000, debug=True)
URL routing
return a string to
the browser
return a dict and it’s
magically converted to
JSON! sweet
convenient dev
web server
Slide 8
Slide 8 text
more like real life
import bottle
@bottle.route('/api/fun')
def hello_fun():
return {
'fun': 123,
}
@bottle.route('/api/slow')
def slow():
time.sleep(8)
return "you didn't plan this well"
if __name__ == '__main__':
bottle.run(host='0.0.0.0', port=8000, debug=True)
I wish we could tell the user not to worry.
I wish the entire server wasn’t blocked right now.
Slide 9
Slide 9 text
you need a proper application
server
uWSGI provides an operating
environment between the web server
and your WSGI compliant
application.
your python code
Bottle
uWSGI
Nginx
web browser
AngularJS (and friends)
your html+javascript
Slide 10
Slide 10 text
/etc/uwsgi/apps-enabled/example-uwsgi.ini
[uwsgi]
socket = localhost:8002
chdir = /opt/example
file = example.py
master = true
plugins = python
uid = www-data
gid = www-data
workers = 6
Your code
web server will find us here
(not HTTP)
Slide 11
Slide 11 text
you need a fab web server
Nginx is a simple high performance
web server which is flexible and
easy to configure.
And it knows how to talk to uWSGI.
your python code
Bottle
uWSGI
Nginx
web browser
AngularJS (and friends)
your html+javascript
Slide 12
Slide 12 text
/etc/nginx/sites-enabled/example-nginx.conf
upstream example {
server localhost:8002;
}
server {
server_name example-server;
root /opt/example/www;
index index.html;
location / {
try_files $uri $uri/ =404;
}
location /api {
include uwsgi_params;
uwsgi_pass example;
}
}
tells Nginx to send any URL
starting /api to your uWSGI
applications
uWSGI protocol socket
(not HTTP)
let Nginx serve static files directly
Slide 13
Slide 13 text
install the things
sudo apt-get install \
python-bottle \
uwsgi uwsgi-plugin-python \
nginx
done?
It works, go do something else now.
Too late. I’d started reading the
uWSGI docs.
- mules!
- shared memory!
- signals!
- other stuff not covered today!
J2EE flashbacks
Slide 19
Slide 19 text
mules and workers
- workers service client requests
- mules don’t
number of each is set in uWSGI
config for your app
- also ‘programmed mule’
Slide 20
Slide 20 text
Caches and Queues
(not the names I’d have used but they’re not insane)
- caches are a key/value store
- queues are arrays
in your uWSGI config file--
cache2 = name=mycache,items=100
queue = 100
Signals
1 byte message sent to listeners:
- all workers
- all mules
- worker or mule by id
- one “available” worker or mule
sent explicitly or by timer (or
file modification)
NEW_ORDER_SIGNAL = 10
@app.route('/api/place_order/')
def place_order(item):
order_id = str(uuid.uuid4())[-6:]
uwsgi.queue_push("{order_id}\t{item}\tnew\tthanks".format(
order_id=order_id, item=item))
uwsgi.signal(NEW_ORDER_SIGNAL)
return {'order_id': order_id}
def mule_new_order(signum):
print "mule {mule_id} sees a new order".format(
mule_id=uwsgi.mule_id())
item = uwsgi.queue_pop()
item = item.split('\t')
item[2] = 'baking'
item[3] = 'mule {mule_id} is baking you a {thing}'.format(
mule_id=uwsgi.mule_id(), thing=item[1])
uwsgi.queue_push('\t'.join(item))
uwsgi.register_signal(NEW_ORDER_SIGNAL, "mule", mule_new_order)
Slide 30
Slide 30 text
@app.route('/api/check_inventory')
def check_inventory():
out = []
for i in range(0, uwsgi.queue_size):
item = uwsgi.queue_get(i)
if item:
item = item.split('\t')
out.append({
'order_id':item[0],
'item': item[1],
'state': item[2],
'notes': item[3]
})
return {'result': out}
@app.route('/api/reload')
def reload():
uwsgi.reload()
return "reloading"
Slide 31
Slide 31 text
opportunities for disaster
Sat Sep 13 10:41:06 2014 - *** Operational MODE: preforking ***
Sat Sep 13 10:41:06 2014 - *** no app loaded. going in full dynamic mode
***
Sat Sep 13 10:41:06 2014 - *** uWSGI is running in multiple interpreter
mode ***
Sat Sep 13 10:41:06 2014 - !!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!
Sat Sep 13 10:41:06 2014 - no request plugin is loaded, you will not be
able to manage requests.
Sat Sep 13 10:41:06 2014 - you may need to install the package for your
language of choice, or simply load it with --plugin.
Sat Sep 13 10:41:06 2014 - !!!!!!!!!!! END OF WARNING !!!!!!!!!!
Sat Sep 13 10:41:06 2014 - spawned uWSGI master process (pid: 14552)
Sat Sep 13 10:41:06 2014 - spawned uWSGI worker 1 (pid: 14553, cores: 1)
Sat Sep 13 10:41:06 2014 - spawned uWSGI worker 2 (pid: 14554, cores: 1)
(ALSO -- Application Zero means python)
Sat Sep 13 11:17:08 2014 - WSGI app 0 (mountpoint='') ready in 0 seconds on
interpreter 0x20c0f80 pid: 15721 (default app)
nginx didn’t attempt to route
to uWSGI -- didn’t think it
was supposed to
the nginx config must not be configured to route /booger
to a uWSGI upstream server
Slide 34
Slide 34 text
this means nginx can’t reach
uWSGI service - but it tried
2014/09/13 20:59:24 [error] 20743#0: *353 connect() failed
(111: Connection refused) while connecting to upstream,
client: 111.69.102.38, server: typist.geek.nz, request:
"GET /compute_md5?v=jkkjknjk HTTP/1.1", upstream: "uwsgi:
//127.0.0.1:8002", host: "typist.geek.nz"
Slide 35
Slide 35 text
nginx passed request to uWSGI
but your application hasn’t
made a route for the URL
nginx sent /api/booger to a uWSGI upstream server but it
wasn’t one that had a route for that URL
Slide 36
Slide 36 text
- at import time
* Restarting app server(s) uwsgi
[ OK ]
Sat Sep 13 21:23:08 2014 - *** Operational MODE: preforking ***
Traceback (most recent call last):
File "compute_md5.py", line 3, in
assert False
AssertionError
Sat Sep 13 21:23:08 2014 - unable to load app 0 (mountpoint='') (callable not found or
import error)
Sat Sep 13 21:23:08 2014 - *** no app loaded. going in full dynamic mode ***
Sat Sep 13 21:23:08 2014 - *** uWSGI is running in multiple interpreter mode ***
Sat Sep 13 21:23:08 2014 - spawned uWSGI master process (pid: 25219)
Sat Sep 13 21:23:08 2014 - spawned uWSGI worker 1 (pid: 25226, cores: 1)
Sat Sep 13 21:23:08 2014 - spawned uWSGI worker 2 (pid: 25229, cores: 1)
Sat Sep 13 21:23:08 2014 - spawned uWSGI worker 3 (pid: 25230, cores: 1)
Sat Sep 13 21:23:08 2014 - spawned uWSGI worker 4 (pid: 25231, cores: 1)
Sat Sep 13 21:23:08 2014 - spawned uWSGI worker 5 (pid: 25232, cores: 1)
Sat Sep 13 21:23:08 2014 - spawned uWSGI worker 6 (pid: 25233, cores: 1)
Slide 37
Slide 37 text
your uWSGI app broke
- at request time
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/bottle.py", line 856, in _handle
return route.call(**args)
File "/usr/lib/python2.7/dist-packages/bottle.py", line 1721, in wrapper
rv = callback(*a, **ka)
File "compute_md5.py", line 10, in hello
assert False
AssertionError
[pid: 24351|app: 0|req: 1/1] 111.69.102.38 () {42 vars in 783 bytes} [Sat Sep 13 21:07:
18 2014] GET /compute_md5?v=jkkjknjk => generated 764 bytes in 19 msecs (HTTP/1.1 500) 2
headers in 99 bytes (1 switches on core 0)
Slide 38
Slide 38 text
tips
- printing to stdout goes to uWSGI
log for app
- touch-reload
- read the uWSGI docs