Serverless chaos: Disk space failure Do we need to worry about disk space in serverless?

Serverless Chaos Demo – Last video • Looked at the new failure modes in Adrian’s failure injection layer: • Status code injection • Exception injection • Tested the functionality for rate control of failure • Demoed the new Serverless Chaos demo app

Serverless Chaos Demo – Last video

Serverless Chaos Demo – Last video

AWS Lambda Storage • 512 MB /tmp directory storage https://docs.aws.amazon.com/lambda/latest/dg/limits.html • Ephemeral storage shared between multiple invocations • Used until recycled or another Lambda instance is deployed • We don’t control if new instances are started or existing ones are re-used • Yes, we need to think about storage in serverless

Disk function demo

Disk corruption injection • Using disk duplicator command as a subprocess • The dd command used in this example: dd if=/dev/zero of=/tmp/corrupt-diskspace-1234567890.tmp count=1024 bs=1024 • Is there a faster way of filling disk space?

Things to consider • Disabling corrupt_diskspace doesn’t clean /tmp • Cleanup /tmp within your functions if possible • Handle errors thrown if disk is full • Could other storage solution (S3 or DynamoDB) be used instead? • As always: Graceful degradation!

@serverlesschaos #chaosengineering