in Python for 10 years - Using Celery for 2 years - Contributed Celery helper projects - Celery Mutex - zkcelery - I have screwed up with Celery. - A lot.
computing - I focus on Celery, because its what I know - But I know its not the only game in town. - This talk does assume, at times, you know Celery - If you don't, stick around. You still may learn something
computing - I focus on Celery, because its what I know - But I know its not the only game in town. - This talk does assume, at times, you know Celery - If you don't, stick around. You still may learn something - I'd love to talk with you about Celery
with a message broker. (e.g., RabbitMQ, Redis) - You build tasks that can then be run locally or across a group of computers that can communicate with the message broker.
with a message broker. (e.g., RabbitMQ, Redis) - You build tasks that can then be run locally or across a group of computers that can communicate with the message broker. - Tasks can be reactive or scheduled
is reliable - Latency is zero - Bandwidth is infinite - The network is secure - Topology doesn't change - There is one administrator - Transport cost is zero
- Sometimes Calling Outside Your Environment - Watch Their Memory Usage - Incrementally read output - Segregate to its own queue - Utilize Soft / Hard Timeouts (Celery)
- Sometimes Calling Outside Your Environment - Watch Their Memory Usage - Incrementally read output - Segregate to its own queue - Utilize Soft / Hard Timeouts (Celery) - Soft Timeout lets you "clean up" afterwards
- Sometimes Calling Outside Your Environment - Watch Their Memory Usage - Incrementally read output - Segregate to its own queue - Utilize Soft / Hard Timeouts (Celery) - Soft Timeout lets you "clean up" afterwards - Hard Timeout kills without remorse
'{:%H}0000'.format(datetime.datetime.now())! select_stmt = 'SELECT id FROM reports WHERE time = ?'! for report_id in db.execute(select_stmt, (time,)):! run_report.delay(report_id)! ! ! CELERYBEAT_SCHEDULE = {! 'report_finder': {! 'task': 'scheduled_report_finder',! 'schedule': celery.schedules.crontab(minute=0)! }! }
'{:%H}0000'.format(datetime.datetime.now())! select_stmt = 'SELECT id FROM reports WHERE time = ?'! for report_id in db.execute(select_stmt, (time,)):! run_report.delay(report_id)! ! ! CELERYBEAT_SCHEDULE = {! 'report_finder': {! 'task': 'scheduled_report_finder',! 'schedule': celery.schedules.crontab(minute=0)! }! }
'{:%H}0000'.format(datetime.datetime.now())! select_stmt = 'SELECT id FROM reports WHERE time = ?'! for report_id in db.execute(select_stmt, (time,)):! run_report.delay(report_id)! ! ! CELERYBEAT_SCHEDULE = {! 'report_finder': {! 'task': 'scheduled_report_finder',! 'schedule': celery.schedules.crontab(minute=0)! }! }
'{:%H}0000'.format(datetime.datetime.now())! select_stmt = 'SELECT id FROM reports WHERE time = ?'! for report_id in db.execute(select_stmt, (time,)):! run_report.delay(report_id)! ! ! CELERYBEAT_SCHEDULE = {! 'report_finder': {! 'task': 'scheduled_report_finder',! 'schedule': celery.schedules.crontab(minute=0)! }! }
and not self.called_directly: semaphore = client.Semaphore(self.dms_code, max_leases=3) if not semaphore.acquire(blocking=False): raise celery.task.current.retry() try: yield finally: if semaphore: semaphore.release()
and not self.called_directly: semaphore = client.Semaphore(self.dms_code, max_leases=3) if not semaphore.acquire(blocking=False): raise celery.task.current.retry() try: yield finally: if semaphore: semaphore.release()
and not self.called_directly: semaphore = client.Semaphore(self.dms_code, max_leases=3) if not semaphore.acquire(blocking=False): raise celery.task.current.retry() try: yield finally: if semaphore: semaphore.release()
and not self.called_directly: semaphore = client.Semaphore(self.dms_code, max_leases=3) if not semaphore.acquire(blocking=False): raise celery.task.current.retry() try: yield finally: if semaphore: semaphore.release()
waiting for an event are awoken when that event occurs, but only one process is able to proceed at a time." - Add jitter to retries - random.randint(30, 60)
waiting for an event are awoken when that event occurs, but only one process is able to proceed at a time." - Add jitter to retries - random.randint(30, 60) - Simple, but effective
waiting for an event are awoken when that event occurs, but only one process is able to proceed at a time." - Add jitter to retries - random.randint(30, 60) - Simple, but effective - ZooKeeper Locks
waiting for an event are awoken when that event occurs, but only one process is able to proceed at a time." - Add jitter to retries - random.randint(30, 60) - Simple, but effective - ZooKeeper Locks - Adds complexity, but arguably "more correct"
waiting for an event are awoken when that event occurs, but only one process is able to proceed at a time." - Add jitter to retries - random.randint(30, 60) - Simple, but effective - ZooKeeper Locks - Adds complexity, but arguably "more correct" - Locks are held in a list
I got careless and introduced a deadlock Friday - Ask Questions - Now - In the Hallway - At lunch - Over drinks (My time can be bought with Cream Sodas)
pycon2014
bkeepers
codingconduct
axbom
tessaabrams
mkilby
tamaranovitovic
bluesmoon
mclloyd
nikkihalliwell
lindahogenes
pwiseman
brettharned
![Fin - Get In Touch! - @brolewis - [email protected] -](https://files.speakerdeck.com/presentations/b9e50340a3ec013145083607ee153eb8/slide_192.jpg){kind=link}