Lessons learned from running aiohttp web application in production for 15+ months. Reasons why to choose asyncio stack for web development over standard Django / Flask approaches.
Python project My last to date Python project Wri en on aioh p Works at production for 15+ months Project for signing and sharing legal documents in Ukraine Sign process done in JavaScript mostly All other necessary stuff done in Python
asyncio itself is not suitable for web development You need to: have web framework built on top of it: aiohttp.web communicate with data sources: aiopg, asyncpg, aioredis, aio* communicate with external API: aiohttp.client
execution Concurrent communication with data sources Concurrent fetching data from external API Deployment without application server async / await all around your code
A empt to use as low resources as possible A lot of concurrent requests from users A lot of data to receive (uploaded documents) from users A lot of data to send to external sources Communicate with external APIs
rocket science Very similar to sync frameworks Init app, setup routes, run the app from aiohttp import web async def hello_world(request: web.Request) -> web.Response: return web.json_response({'data': 'Hello, world!'}) app = web.Application() app.router.add_get('/', hello_world) web.run_app(app)
This is not a production-ready code! This is not a production-ready code! async def upload_document(request: web.Request) -> web.Response: data = await request.post() files = {} for file in data.values(): files[file.filename] = file.body await upload_to_s3(file.filename, file.body) async with request.app['db'].acquire() as conn: for file in files: await insert_document(conn, file) return web.json_response(status=201)
transform request data with trafaret Process safe data Return JSON response (mostly empty one) For public API we're used Swagger for docs Maybe there is Django REST Framework for aioh p, but we didn't aware of
for managing data We were fine with querying data without ORM We were fine with omiting REST API framework We chose aioh p cause of concurrency and let see how it payed off
push your app structure The freedom is great But for one dev project More devs -> your project became a mess You need to enforce not only coding style, but to conform on structure of your code as well
need to agree on how to manage se ings for your app by yourself Our setup: Store se ings in *.yaml Provide trafaret schema to validate se ings Use trafaret-config for reading se ings Error in se ings? App even didn't start
as well Type annotations indirectly enforce you to write more understandable code Instead of dict you might want to start using namedtuple or dataclass, or at least StructedDict Documentation for your code Your teammates with IDE thank you everyday
Run it in executor The power of asyncio in await statement No await – asyncio is not your choice But there is run_in_executor async def gzip_buffer(buffer: io.BytesIO, *, loop: asyncio.AbstractEventLoop) -> io.BytesIO: """Non-blocking way of compressing prepared bytes IO with gzip.""" return await loop.run_in_executor(None, sync_gzip_buffer, buffer)
aiohttp.web sill in semi-active development New feature are coming for sure (like @route decorator) aio-* libs might not cover your case You might need to payback to open-source Like fixing bugs, that blocks you from update to new aioh p version
p devs? Your project grows – you need more devs Where to find them? The market offers much more Django / Flask devs, then aioh p devs Especially it is hard to substitute senior / lead dev
p dev? Start with basics: how asyncio works, tests, etc… Asyncio requires time for diving in Continue with basics: read & discuss aioh p code aiohttp is not a rocket science It may pay dividends later
Igor Davydenko
playpauseandstop
iamctodd
reverentgeek
rmw
skipperchong
carmenhchung
keathley
productmarketing
clairelew
thoeni
