Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Distributed Domain Destruction - Adventures in ...

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →

Distributed Domain Destruction - Adventures in building distributed systems

The project has grown enough that it requires refactoring, and extracting pieces of it to be distributed to different machines and places. Of course it should work like a well oiled mechanism, but it may not... at first. For the last three years I have been building, maintaining and fighting a few projects that make use of distributed computing, parallel processing, message brokers, queues and workers. This is one of those "from the trenches" talks, where I will regale you with tales about the series of unfortunate events that may happen as your application grows in complexity, it will try to convey some of the pitfalls and solutions you will encounter along the way, in hope that you will have an easier time circumventing them when it is your turn to battle them. Come hear about the agony you will experience when it starts falling apart, and the thrill you will feel when everything is running juuust right.

Vranac Srdjan

July 19, 2016

Transcript

27 slides
  1. business owner, developer, consultant, mercenary, writing terrible code that performs

    exceptionally, wrangling elePHPants and Pythons, obsessed with process automation, interested in continuous integration and delivery, clean code, testing, best practices and distributed systems
  2. THE SIMPLE DEFINITON It is a system where you can

    distribute processing of tasks to other workers. By costly taks I mean anything ranging to heavy computation and cpu utilization to long running processes. If we are talking in the context of web applications, basically anything that can't be done within a request is a candidate for distribution/background processing.
  3. A MORE REALISTIC DEFINITION Leslie B. Lamport, 1987: A distributed

    system is one in which the failure of a computer you didn't even know existed can render your own computer unusable.
  4. /**
 * Handle DynamoDb BatchWriter erors
 *
 * Because Batchwriter

    is a special little snowflake that ignores error, drops the write batch and
 * just keeps chugging along, giving a nice false positive causing 4 days worth of data to be lost.
 * You need to actually specify an error handler function and provide it in constructor (see getWriteRequestBatch())
 * Any and all post exception processing should happen here.
 *
 * @param \Aws\Exception\AwsException $exception
 */
 public function handleDynamoDbBatchWriteError(\Aws\Exception\AwsException $exception)
 {
 throw $exception;
 }
  5. THIS INSTANCE IS FAST… THAT INSTANCE IS SLOW… WHICH ONE

    WILL CRASH THE DATABASE? NOBODY KNOWS!

More Decks by Vranac Srdjan

Other Decks in Programming