Amazon S3
Portland Java User Group
2024-09-17
Sean Sullivan
Slide 2
Slide 2 text
Agenda
Amazon S3
AWS SDK
Code
Bonus topics
Slide 3
Slide 3 text
Amazon S3
“Simple Storage Service”
Launched on March 14, 2006
Slide 4
Slide 4 text
Amazon S3
S3 is an object storage service with an HTTP REST API
https://www.allthingsdistributed.com/2023/07/building-and-operating-a-pretty-big-storage-system.html
Slide 5
Slide 5 text
Amazon S3
“There is a frontend
fl
eet with a REST API, a namespace
service, a storage
fl
eet that’s full of hard disks, and a
fl
eet that
does background operations.”
https://www.allthingsdistributed.com/2023/07/building-and-operating-a-pretty-big-storage-system.html
Slide 6
Slide 6 text
S3 core concepts
Buckets
Objects
Slide 7
Slide 7 text
S3 core concepts
An Amazon S3 object represents a
fi
le or collection of data
Every object must reside within a bucket
Slide 8
Slide 8 text
S3 pricing
https://aws.amazon.com/s3/pricing/
“You pay for storing objects in your S3 buckets.
The rate you’re charged depends on your
objects' size, how long you stored the objects
during the month, and the storage class”
Slide 9
Slide 9 text
S3 storage classes
https://aws.amazon.com/s3/storage-classes/
“Amazon S3 o
ff
ers a range of storage classes that you can
choose from based on the performance, data access, resiliency,
and cost requirements of your workloads.”
Slide 10
Slide 10 text
S3 storage classes
https://aws.amazon.com/s3/storage-classes/
Standard
Intelligent Tiering
Express One Zone
… and many others
Slide 11
Slide 11 text
S3 storage classes
https://aws.amazon.com/s3/storage-classes/
You can con
fi
gure S3 storage classes at the
object level, and a single general purpose bucket
can contain objects stored across all storage
classes except S3 Express One Zone
Slide 12
Slide 12 text
Storage class choice matters
https://www.youtube.com/watch?v=RxgYNrXPOLw
Zero Disk Architecture
November 2023
September 2024
Slide 52
Slide 52 text
S3 Conditional Writes
https://aws.amazon.com/about-aws/whats-new/2024/08/amazon-s3-conditional-writes/
Conditional writes can ensure there is no existing object
with the same key name in you bucket
during PUT operations
Slide 53
Slide 53 text
S3 Storage Lens
https://aws.amazon.com/s3/storage-lens/
“S3 Storage Lens delivers organization-wide
visibility into object storage usage, activity
trends, and makes actionable
recommendations to optimize costs ”
Slide 54
Slide 54 text
S3 bucket permissions
https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketLifecycleCon
fi
guration.html
By default, all Amazon S3 resources are private,
including buckets, objects, and related subresources
Slide 55
Slide 55 text
S3 performance
considerations?
Slide 56
Slide 56 text
S3 performance
https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html
"your application can achieve at least
3,500 PUT/COPY/POST/DELETE
or 5,500 GET/HEAD requests per second
per partitioned Amazon S3 pre
fi
x”
Slide 57
Slide 57 text
S3 performance
https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html
"There are no limits to the number of pre
fi
xes in a bucket.
You can increase your read or write performance by using
parallelization”
Slide 58
Slide 58 text
S3 performance
https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html
“While Amazon S3 is scaling to your new higher request rate,
you may see some 503 (Slow Down) errors. These errors will
dissipate when the scaling is complete.”