Hate organising your photos? Try it with 5 Terabytes

My partner is a freelancer photographer. During the years she’ve gathered a ~5 terabyte collection of photos from over hundred different occasions. All the photos have to be backed up in two physical locations and organised for browsing. This talk is about how I solved this for her in serverless way

My partner is a freelancer photographer. During the years she has gathered a ~5 terabyte collection of photos from over hundred different occasions. All the photos have to be backed up in at least two physical locations and organised for browsing. This talk is about how I built the solution for her in serverless way, costing less than 10€ per month. We’ll walk through my solution starting from local network & Raspberry Pi to AWS S3 & Rekognition.

Transcript

1. HATE ORGANIZING YOUR PHOTOS? Try it with 5 terabytes Hi, I’m Arto
10. ARTO LIUKKONEN Full-stack developer Serverless (AWS) for ~4 years Live in Jyväskylä, Finland Work at Nordcloud (recruiting in 10 countries) “
11. Cheap & scaling backup solution? Consider it done! “
12. Copy files to home NAS CLOUD BACKUP Cloud backup
13. Let’s just use Google Drive!
15. ... Dropbox!
17. ... S3 Bucket!
18. 0.023$/Gb/month Total ~115e/month AWS S3
19. No thumbnails from RAWs Galleries are hard to browse Not magical enough “
20. Copy files to home NAS CLOUD BACKUP 2.0 Magic Magic Photo browser
21. 0.00099$/Gb/month Total ~5$/month AWS S3 GLACIER DEEP ARCHIVE
22. AWS S3 Glacier Deep Archive + Cheap + Unlimited storage - Retrieval price 0.02€/Gb - Retrieval time +12 hours “
23. Magic Copy files to home NAS CLOUD BACKUP 2.0 Raw files in S3 Magic Photo browser ✅
25. Upload RAW to S3 EXTRACTING JPG FROM RAW Extract PreviewImage with exiftool Copy EXIF with exiftool Resize to 2000px with ImageMagick
26. Previews from RAW require native binaries “
27. Raspberry Pi to the rescue! “
30. 0.023$/Gb/month Total ~1$/month AWS S3 (2000px previews)
31. Raspberry Pi resize & sync Raw files in S3 Copy files to home NAS CLOUD BACKUP 2.0 Magic Photo browser ✅✅
33. AWS Rekognition • Face recognition ◦ Find same person from different photoshoots • Object & scene recognition ◦ Browse photos by occasion ◦ Weddings, cooking, animals, etc “
36. 1$/1000 images ...quite a lot, but luckily only once AWS REKOGNITION (free tier 5000 per month)
37. Copy files to home NAS SO HOW DOES THE SOLUTION LOOK? Raspberry Pi resize & sync AWS Rekognition AWS Lambda AWS DynamoDB Photo browser
38. RPi with HDD Rekognition & storage Gallery browser
39. DEMO TIME!
40. 5Tb of raws: 5$/month Rekognition: 0-5$/month 2000px previews: 1$/month DynamoDB, traffic, Lambda: ~0.5$/month Time spent: priceless TOTAL COST
41. THANKS! @artoliukkonen