The overnight failure

The overnight failure

this talk is based on a true horror story.

Imagine your work week ends after releasing a set of features to production. Your team is happy and you feel good about yourself. A call about a problem with payments wakes you up the next morning. You find out your most valuable users were charged hundreds of times, consuming their credit card limits, leaving others in overdraft. They're angry because they can't even buy milk at the store.

Learn how a "perfect bug storm" caused the problem, how our processes failed to catch it and how hard it was to gain our users trust back.

Presented at: RubyConf Taiwan 2016


Sebastian Sogamoso

December 02, 2016