Distributed Applications in Python: Celery vs. Crossbar (I couldn’t find a single “cute” image for Crossbar ) By Adam Jorgensen, https://github.com/oopman/ 

• Provide online bookings for medical appointments • Integration with PMA software via our API • Our tech stack includes: 

Distributed Computing… And that’s just the tip of the iceberg, it’s an extremely broad topic! 

Distributed Computing in Python… 

More Distributed Computing in Python! 

Enough content for a million talks… …so let’s skip most of it and just talk about two players in this game: Celery Crossbar 

• Celery Applications consist primarily of Tasks from myapp.celery import app @app.task def add(x, y): return x + y Celery: A Distributed Task Queue • Tasks can be executed synchronously or asynchronously from myapp.tasks import add # Synchronous calls occur in the current thread/process add(1,2) # Asynchronous calls result in execution by a Celery Worker add.delay(1,2) # Short form for basic usage add.apply_async(args=[1], kwargs={'y': 2}) # Long form allows for additional execution options • Celery Deployment • 1 or more Worker instances • Optional: 1 Beat instance • Optional: Flower 

Celery: The Crunchy Parts • Workflow system • Signatures • Callbacks • Chains • Groups • Chords • Map • Chunks • Task routing • Signals • Django integration 

Celery: The Good Parts • Fairly mature • Highly configurable • Easy to scale • Celery Beat • Documentation 

Celery: The Bad Parts • Highly configurable • Prone to peculiar issues • Debugging 

Crossbar: Networking for Applications • Crossbar is a WAMP router • WAMP? WebSockets? Autobahn? from autobahn.twisted.util import sleep from twisted.internet.defer import inlineCallbacks from autobahn import wamp from autobahn.twisted.wamp import ApplicationSession class ExamplePython2Component(ApplicationSession): @wamp.register(u'com.demo.add') def add(x, y) return x + y @inlineCallbacks def onJoin(self, details): yield self.register(self) counter = 0 while True: self.publish('com.demo.tick', counter) counter += 1 yield sleep(1) from asyncio import sleep from autobahn import wamp from autobahn.asyncio.wamp import ApplicationSession class ExamplePython3Component(ApplicationSession): @wamp.register(u'com.demo.add') def add(x, y) return x + y async def onJoin(self, details): await self.register(self) counter = 0 while True: self.publish('com.demo.tick', counter) counter += 1 await sleep(1) 

Crossbar: The Good Parts • Impressive connection handling abilities • Components are location agnostic • Component hosting • Extensible security system • Shared registrations • Polyglot friendly • Embedded HTTP server 

Crossbar: The Bad Parts • Twisted requires usage of alternate ecosystem of libraries • No Clustering support. Yet… • Documentation is out of date 

Celery vs. Crossbar Reasons to use Celery: • Horizontal/Vertical Scalability • Scheduling • Workflows Reasons to use Crossbar: • Non-blocking asynchronous paradigm • Authentication/Authorization • WAMP