Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Everything I know about S3 pre-signed URLs

Everything I know about S3 pre-signed URLs

Almost every web application at some point needs a way to upload or download files… and no one seems to enjoy building reliable and scalable upload/download servers… and for good reasons too! In fact, you’ll probably need to manage long-running connections and handle files that can be quite large (i.e videos). If you are running a fully serverless backend using API Gateway and Lambda, you probably know that you are limited in terms of payload size and execution time, so things get even more complicated there. In all these cases you should consider offloading this problem to S3 by using S3 pre-signed URLs. Pre-signed URLs are a fantastic tool to handle file download and upload directly in S3 in a managed and scalable fashion. But all that glitters is not gold and S3 pre-signed URLs come with quite a few gotchas… So in this talk, we will explore some use cases, see some potential implementations of S3 pre-signed URLs and uncover some of the gotchas that I discovered while using them. By the end of this talk, you should know exactly when to use pre-signed URLs and how to avoid most of the many mistakes I made with them!

Luciano Mammino

November 17, 2022
Tweet

More Decks by Luciano Mammino

Other Decks in Technology

Transcript

  1. AWS User Group: Dublin Meetup / 2022-11-17
    1

    View full-size slide

  2. META_SLIDE!
    fth.link/presign
    loige 2

    View full-size slide

  3. 😎 "I have a startup idea..."
    loige
    fth.link/presign
    3

    View full-size slide

  4. loige
    meower
    fth.link/presign
    4

    View full-size slide

  5. Yes, I still have to implement the
    profile picture upload feature...
    😅
    loige 9

    View full-size slide

  6. $ ~ whoami
    👋 I'm Luciano (
    🍕🍝)
    Senior Architect @ fourTheorem (Dublin )
    nodejsdp.link
    📔 Co-Author of Node.js Design Patterns
    👉
    Let's connect!
    (blog)
    (twitter)
    (twitch)
    (github)
    loige.co
    @loige
    loige
    lmammino
    10

    View full-size slide

  7. Always re-imagining
    We are a pioneering technology consultancy
    focused on AWS and serverless
    | |
    Accelerated Serverless AI as a Service Platform Modernisation
    loige
    ✉ Reach out to us at
    😇 We are always looking for talent:
    [email protected]
    fth.link/careers
    11

    View full-size slide

  8. We host a weekly podcast about AWS
    awsbites.com
    loige 12

    View full-size slide

  9. 🤔
    "upload" feature
    in a web app...
    loige 13

    View full-size slide

  10. What's an upload, really?
    loige 14

    View full-size slide

  11. OK, what protocol?
    loige 15

    View full-size slide

  12. loige
    Structure of an HTTP request
    POST /profilepic/upload HTTP/1.1
    Host: api.meower.com
    Content-Type: text/plain
    Content-Length: 9
    Some data
    Method
    Path Version
    Headers
    Body
    16

    View full-size slide

  13. loige
    What if it's a binary (like a picture)?
    PUT /profilepic/upload HTTP/1.1
    Host: api.meower.com
    Content-Type: image/jpeg
    Content-Length: 2097852
    ����JFIFHH������"��
    ���Dl��FW�'6N�()H�'p��FD3 [...]
    read 2097852
    bytes
    17

    View full-size slide

  14. loige
    Using Lambda
    Lambda proxy integration*
    * JSON Based protocol mapping to HTTP
    (JSON Request / JSON Response)
    18

    View full-size slide

  15. {
    "resource": "/profilepic/upload",
    "path": "/profilepic/upload",
    "httpMethod": "POST",
    "headers": {
    "Content-Type": "text/plain",
    "Content-Length": "9",
    "Host": "api.meower.com",
    },
    "body": "Some data"
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    "resource": "/profilepic/upload",
    "path": "/profilepic/upload",
    {
    1
    2
    3
    "httpMethod": "POST",
    4
    "headers": {
    5
    "Content-Type": "text/plain",
    6
    "Content-Length": "9",
    7
    "Host": "api.meower.com",
    8
    },
    9
    "body": "Some data"
    10
    }
    11
    "httpMethod": "POST",
    {
    1
    "resource": "/profilepic/upload",
    2
    "path": "/profilepic/upload",
    3
    4
    "headers": {
    5
    "Content-Type": "text/plain",
    6
    "Content-Length": "9",
    7
    "Host": "api.meower.com",
    8
    },
    9
    "body": "Some data"
    10
    }
    11
    "headers": {
    "Content-Type": "text/plain",
    "Content-Length": "9",
    "Host": "api.meower.com",
    },
    {
    1
    "resource": "/profilepic/upload",
    2
    "path": "/profilepic/upload",
    3
    "httpMethod": "POST",
    4
    5
    6
    7
    8
    9
    "body": "Some data"
    10
    }
    11
    "body": "Some data"
    {
    1
    "resource": "/profilepic/upload",
    2
    "path": "/profilepic/upload",
    3
    "httpMethod": "POST",
    4
    "headers": {
    5
    "Content-Type": "text/plain",
    6
    "Content-Length": "9",
    7
    "Host": "api.meower.com",
    8
    },
    9
    10
    }
    11
    {
    "resource": "/profilepic/upload",
    "path": "/profilepic/upload",
    "httpMethod": "POST",
    "headers": {
    "Content-Type": "text/plain",
    "Content-Length": "9",
    "Host": "api.meower.com",
    },
    "body": "Some data"
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    loige
    Lambda proxy integration (request)
    19

    View full-size slide

  16. {
    "resource": "/profilepic/upload",
    "path": "/profilepic/upload",
    "httpMethod": "PUT",
    "headers": {
    "Content-Type": "image/jpeg",
    "Content-Length": "2097852",
    "Host": "api.meower.com",
    },
    "body": "????????"
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    "Content-Type": "image/jpeg",
    "Content-Length": "2097852",
    {
    1
    "resource": "/profilepic/upload",
    2
    "path": "/profilepic/upload",
    3
    "httpMethod": "PUT",
    4
    "headers": {
    5
    6
    7
    "Host": "api.meower.com",
    8
    },
    9
    "body": "????????"
    10
    }
    11
    "body": "????????"
    {
    1
    "resource": "/profilepic/upload",
    2
    "path": "/profilepic/upload",
    3
    "httpMethod": "PUT",
    4
    "headers": {
    5
    "Content-Type": "image/jpeg",
    6
    "Content-Length": "2097852",
    7
    "Host": "api.meower.com",
    8
    },
    9
    10
    }
    11
    {
    "resource": "/profilepic/upload",
    "path": "/profilepic/upload",
    "httpMethod": "PUT",
    "headers": {
    "Content-Type": "image/jpeg",
    "Content-Length": "2097852",
    "Host": "api.meower.com",
    },
    "body": "????????"
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    loige
    Lambda proxy integration (request - picture)
    20

    View full-size slide

  17. "isBase64Encoded": true,
    "body": "/9j/4AAQSkZJRgABAQEASABIAAD/2w[...]"
    {
    1
    "resource": "/profilepic/upload",
    2
    "path": "/profilepic/upload",
    3
    "httpMethod": "PUT",
    4
    "headers": {
    5
    "Content-Type": "image/jpeg",
    6
    "Content-Length": "2097852",
    7
    "Host": "api.meower.com",
    8
    },
    9
    10
    11
    }
    12
    {
    "resource": "/profilepic/upload",
    "path": "/profilepic/upload",
    "httpMethod": "PUT",
    "headers": {
    "Content-Type": "image/jpeg",
    "Content-Length": "2097852",
    "Host": "api.meower.com",
    },
    "isBase64Encoded": true,
    "body": "/9j/4AAQSkZJRgABAQEASABIAAD/2w[...]"
    }
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    loige
    Lambda proxy integration (request - picture)
    21

    View full-size slide

  18. loige
    1. Parse request (JSON)
    2. Decode body (Base64)
    3. Validation / resize
    4. ...
    Lambda proxy integration request
    /profilepic/upload
    22

    View full-size slide

  19. loige
    1. Parse request (JSON)
    2. Decode body (Base64)
    3. Validation / resize
    4. ...
    Lambda proxy integration request
    /profilepic/upload
    😎
    23

    View full-size slide

  20. loige
    Is this a good solution?
    🙄
    24

    View full-size slide

  21. loige
    Limitations...
    API Gateway requests timeout: 30 sec
    API Gateway payload: max 10 MB
    Lambda timeout: max 15 mins
    Lambda payload size: max 6 MB
    Upload: 6 MB / 30 sec
    25

    View full-size slide

  22. loige
    Is this a good solution?
    🙄
    ... not really!
    What about supporting big images or even videos?
    26

    View full-size slide

  23. loige
    An alternative approach
    ✅ Long lived connection
    ✅ No size limit
    27

    View full-size slide

  24. loige
    🤔 SERVERS...
    28

    View full-size slide

  25. loige
    S3 pre-signed URLs
    😱
    An S3 built-in feature
    to authorize operations (download, upload, etc) on a
    bucket / object
    using time-limited authenticated URLs
    29

    View full-size slide

  26. loige
    Using S3 pre-signed URLs for upload
    * yeah, this can be a Lambda as well
    😇
    *
    30

    View full-size slide

  27. loige
    Using S3 pre-signed URLs for upload
    * yeah, this can be a Lambda as well
    😇
    *
    31

    View full-size slide

  28. loige
    Using S3 pre-signed URLs for upload
    * yeah, this can be a Lambda as well
    😇
    *
    32

    View full-size slide

  29. loige
    Using S3 pre-signed URLs for upload
    * yeah, this can be a Lambda as well
    😇
    *
    33

    View full-size slide

  30. loige
    Using S3 pre-signed URLs for upload

    * yeah, this can be a Lambda as well
    😇
    *
    34

    View full-size slide

  31. loige
    ... and we can also use it for downloads!
    🤩
    35

    View full-size slide

  32. loige
    Using S3 pre-signed URLs for download
    36

    View full-size slide

  33. loige
    Using S3 pre-signed URLs for download
    37

    View full-size slide

  34. loige
    Using S3 pre-signed URLs for download
    38

    View full-size slide

  35. loige
    Using S3 pre-signed URLs for download
    39

    View full-size slide

  36. loige
    Using S3 pre-signed URLs for download

    40

    View full-size slide

  37. loige
    ⚠ VERY important details!
    I lied to you a little in those diagrams...
    🤥
    It's a decent mental model, but it's not accurate
    😅
    The server never really talks with S3!
    The server actually creates the signed URL by itself!
    We will see later what's the security model around this idea!
    41

    View full-size slide

  38. loige
    Is this a good solution?
    🙄
    ✅ It's a managed feature (a.k.a. no servers to manage)
    ✅ We can upload and download arbitrarily big files with no practical limits*
    ✅ Reasonably simple and secure
    👍 Seems good to me!
    * objects in S3 are "limited" to 5TB (when using multi-part upload), 5 GB otherwise.
    42

    View full-size slide

  39. loige
    Generating our first pre-signed URL
    $ aws s3 presign \
    s3://finance-department-bucket/2022/tax-certificate.pdf
    https://s3.amazonaws.com/finance-department-bucket/2022/tax-
    certificate.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-
    Credential=AKIA3SGQVQG7FGA6KKA6%2F20221104%2Fus-east-
    1%2Fs3%2Faws4_request&X-Amz-Date=20221104T140227Z&X-Amz-
    Expires=3600&X-Amz-SignedHeaders=host&X-Amz-
    Signature=b228dbec8c1008c80c162e1210e4503dceead1e4d4751b4d9787
    314fd6da4d55
    Whoever has this URL can
    download the tax certificate!
    43

    View full-size slide

  40. loige
    What's in a pre-signed URL
    https://s3.amazonaws.com/finance-department-bucket/2022/tax-
    certificate.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-
    Credential=AKIA3SGQVQG7FGA6KKA6%2F20221104%2Fus-east-
    1%2Fs3%2Faws4_request&X-Amz-Date=20221104T140227Z&X-Amz-
    Expires=3600&X-Amz-SignedHeaders=host&X-Amz-
    Signature=b228dbec8c1008c80c162e1210e4503dceead1e4d4751b4d9787
    314fd6da4d55
    44

    View full-size slide

  41. loige
    What's in a pre-signed URL
    https://s3.amazonaws.com
    /finance-department-bucket
    /2022/tax-certificate.pdf
    ?X-Amz-Algorithm=AWS4-HMAC-SHA256
    &X-Amz-Credential=AKIA3SGQXQG7XXXYKKA6%2F20221104...
    &X-Amz-Date=20221104T140227Z
    &X-Amz-Expires=3600
    &X-Amz-SignedHeaders=host
    &X-Amz-Signature=b228dbec8c1008c80c162e1210e4503dceead1e4d4...
    What if I change this to
    /passwords.txt?
    45

    View full-size slide

  42. loige
    Pre-signed URLs validation
    https://s3.amazonaws.com
    /finance-department-bucket
    /2022/tax-certificate.pdf
    ?X-Amz-Algorithm=AWS4-HMAC-SHA256
    &X-Amz-Credential=AKIA3SGQXQG7XXXYKKA6%2F20221104...
    &X-Amz-Date=20221104T140227Z
    &X-Amz-Expires=3600
    &X-Amz-SignedHeaders=host
    &X-Amz-Signature=b228dbec8c1008c80c162e1210e4503dceead1e4d4...
    47

    View full-size slide

  43. loige
    🤓
    Once a pre-signed URL is generated you
    cannot edit it without breaking it
    Photo by on
    CHUTTERSNAP Unsplash
    ⚠ Also note that you can use a pre-signed URL as many
    times as you want until it expires
    48

    View full-size slide

  44. loige
    🔐 Permissions
    Anyone with valid credentials can create a pre-signed URL (client side)
    valid credentials = Role, User, or Security Token
    The generated URL inherits the permissions of the credentials used to generate it
    This means you can generate pre-signed URLs for things you don't have access to
    😅
    49

    View full-size slide

  45. loige
    $ aws s3 presign s3://ireland/i-love-you
    https://ireland.s3.eu-west-1.amazonaws.com/i-love-you?X-Amz-
    Algorithm=AWS4-HMAC-SHA256&X-Amz-
    Credential=AKIA3ABCVQG7FGA6KKA6%2F20221115%2Feu-west-
    1%2Fs3%2Faws4_request&X-Amz-Date=20221115T182036Z&X-Amz-
    Expires=3600&X-Amz-SignedHeaders=host&X-Amz-
    Signature=75749c92d94d03e411e7bbf64419f2af09301d1791b0df54c639
    137c715f7888
    😱
    I swear I don't even know if this
    bucket exists or who owns it!
    50

    View full-size slide

  46. loige
    Pre-signed URLs are validated at request time
    51

    View full-size slide

  47. loige
    Creating a pre-signed URL
    programmatically
    AWS SDK for JavaScript v3
    52

    View full-size slide

  48. import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3'
    import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
    const s3Client = new S3Client()
    const command = new GetObjectCommand({
    Bucket: "some-bucket",
    Key: "some-object"
    })
    const preSignedUrl = await getSignedUrl(s3Client, command, {
    expiresIn: 3600
    })
    console.log(preSignedUrl)
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3'
    import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
    1
    2
    3
    const s3Client = new S3Client()
    4
    5
    const command = new GetObjectCommand({
    6
    Bucket: "some-bucket",
    7
    Key: "some-object"
    8
    })
    9
    10
    const preSignedUrl = await getSignedUrl(s3Client, command, {
    11
    expiresIn: 3600
    12
    })
    13
    14
    console.log(preSignedUrl)
    15
    const s3Client = new S3Client()
    import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3'
    1
    import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
    2
    3
    4
    5
    const command = new GetObjectCommand({
    6
    Bucket: "some-bucket",
    7
    Key: "some-object"
    8
    })
    9
    10
    const preSignedUrl = await getSignedUrl(s3Client, command, {
    11
    expiresIn: 3600
    12
    })
    13
    14
    console.log(preSignedUrl)
    15
    const command = new GetObjectCommand({
    Bucket: "some-bucket",
    Key: "some-object"
    })
    import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3'
    1
    import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
    2
    3
    const s3Client = new S3Client()
    4
    5
    6
    7
    8
    9
    10
    const preSignedUrl = await getSignedUrl(s3Client, command, {
    11
    expiresIn: 3600
    12
    })
    13
    14
    console.log(preSignedUrl)
    15
    const preSignedUrl = await getSignedUrl(s3Client, command, {
    expiresIn: 3600
    })
    import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3'
    1
    import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
    2
    3
    const s3Client = new S3Client()
    4
    5
    const command = new GetObjectCommand({
    6
    Bucket: "some-bucket",
    7
    Key: "some-object"
    8
    })
    9
    10
    11
    12
    13
    14
    console.log(preSignedUrl)
    15 console.log(preSignedUrl)
    import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3'
    1
    import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
    2
    3
    const s3Client = new S3Client()
    4
    5
    const command = new GetObjectCommand({
    6
    Bucket: "some-bucket",
    7
    Key: "some-object"
    8
    })
    9
    10
    const preSignedUrl = await getSignedUrl(s3Client, command, {
    11
    expiresIn: 3600
    12
    })
    13
    14
    15
    import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3'
    import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
    const s3Client = new S3Client()
    const command = new GetObjectCommand({
    Bucket: "some-bucket",
    Key: "some-object"
    })
    const preSignedUrl = await getSignedUrl(s3Client, command, {
    expiresIn: 3600
    })
    console.log(preSignedUrl)
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    loige 53

    View full-size slide

  49. loige
    📦
    Uploading a file
    using pre-signed URLs
    54

    View full-size slide

  50. loige
    2 Options: PUT & POST
    🤨
    55

    View full-size slide

  51. loige
    PUT Method
    PUT HTTP/1.1
    Host: .s3..amazonaws.com
    Content-Length: 2097852
    ����JFIFHH������"��
    ���Dl��FW�'6N�()H�'p��FD3 [...]
    56

    View full-size slide

  52. import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3'
    import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
    const s3Client = new S3Client()
    const command = new PutObjectCommand({
    Bucket: "some-bucket",
    Key: "some-object"
    })
    const preSignedUrl = await getSignedUrl(s3Client, command, {
    expiresIn: 3600
    })
    console.log(preSignedUrl)
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    loige
    Only difference with the
    previous example
    57

    View full-size slide

  53. loige
    PUT Method - Limitations
    You cannot set a limit on the upload size (max of 5 GB)! *
    You can limit the Content-Type but you can specify exactly one
    * Unless you know the exact size in advance
    58

    View full-size slide

  54. loige
    POST method
    It uses the multipart/form-data encoding (form upload)
    Gives more freedom to the client to shape the request (Content-Type, file name, etc)
    It uses a policy mechanism to define the "rules" of what can be uploaded
    E.g. you can limit the supported mime types and provide a maximum file size
    You can use it to upload from a web form and even configure the redirect URL
    It's not really a URL but more of a pre-signed form!
    59

    View full-size slide

  55. POST / HTTP/1.1
    Host: .s3.amazonaws.com
    Content-Type: multipart/form-data; boundary=9431149156168
    Content-Length: 2097852
    --9431149156168
    Content-Disposition: form-data; name="key"
    picture.jpg
    --9431149156168
    Content-Disposition: form-data; name="X-Amz-Credential"
    AKIA3SGABCDXXXA6KKA6/20221115/eu-west-1/s3/aws4_request
    --9431149156168
    Content-Disposition: form-data; name="Policy"
    eyJleHBpcmF0aW9uIjoiMjAyMi0xMS0xNVQyMDo0NjozN1oiLCJjb25kaXRpb25zIjpbWyJj[...]
    --9431149156168
    Content-Disposition: form-data; name="X-Amz-Signature"
    2c1da0001dfec7caea1c9fb80c7bc8847f515a9e4483d2942464f48d2f827de7
    --9431149156168
    Content-Disposition: form-data; name="file"; filename="MyFilename.jpg"
    Content-Type: image/jpeg
    ����JFIFHH������"��
    ���Dl��FW�'6N�()H�'p��FD3[...]
    --9431149156168--
    loige
    60

    View full-size slide

  56. loige
    POST method Policy
    A JSON object (Base64 encoded) that defines the upload rules (conditions) and the
    expiration date
    This is what gets signed: you cannot alter the policy without breaking the signature
    {
    "expiration": "2022-11-15T20:46:37Z",
    "conditions": [
    ["content-length-range", 0, 5242880],
    ["starts-with", "$Content-Type", "image/"],
    {"bucket": "somebucket"},
    {"X-Amz-Algorithm": "AWS4-HMAC-SHA256"},
    {"X-Amz-Credential": "AKIA3SGABCDXXXA6KKA6/20221115/eu-west-1/s3/aws4_request"},
    {"X-Amz-Date": "20221115T194637Z"},
    {"key": "picture.jpg"}
    ]
    }
    61

    View full-size slide

  57. import { S3Client } from '@aws-sdk/client-s3'
    import { createPresignedPost } from '@aws-sdk/s3-presigned-post'
    const { BUCKET_NAME, OBJECT_KEY } = process.env
    const s3Client = new S3Client()
    const { url, fields } = await createPresignedPost(s3Client, {
    Bucket: 'somebucket',
    Key: 'someobject',
    Conditions: [
    ['content-length-range', 0, 5 * 1024 * 1024] // 5 MB max
    ],
    Fields: {
    success_action_status: '201',
    'Content-Type': 'image/png'
    },
    Expires: 3600
    })
    console.log({ url, fields })
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    import { createPresignedPost } from '@aws-sdk/s3-presigned-post'
    import { S3Client } from '@aws-sdk/client-s3'
    1
    2
    3
    const { BUCKET_NAME, OBJECT_KEY } = process.env
    4
    const s3Client = new S3Client()
    5
    6
    const { url, fields } = await createPresignedPost(s3Client, {
    7
    Bucket: 'somebucket',
    8
    Key: 'someobject',
    9
    Conditions: [
    10
    ['content-length-range', 0, 5 * 1024 * 1024] // 5 MB max
    11
    ],
    12
    Fields: {
    13
    success_action_status: '201',
    14
    'Content-Type': 'image/png'
    15
    },
    16
    Expires: 3600
    17
    })
    18
    19
    console.log({ url, fields })
    20
    const { url, fields } = await createPresignedPost(s3Client, {
    })
    import { S3Client } from '@aws-sdk/client-s3'
    1
    import { createPresignedPost } from '@aws-sdk/s3-presigned-post'
    2
    3
    const { BUCKET_NAME, OBJECT_KEY } = process.env
    4
    const s3Client = new S3Client()
    5
    6
    7
    Bucket: 'somebucket',
    8
    Key: 'someobject',
    9
    Conditions: [
    10
    ['content-length-range', 0, 5 * 1024 * 1024] // 5 MB max
    11
    ],
    12
    Fields: {
    13
    success_action_status: '201',
    14
    'Content-Type': 'image/png'
    15
    },
    16
    Expires: 3600
    17
    18
    19
    console.log({ url, fields })
    20
    Bucket: 'somebucket',
    import { S3Client } from '@aws-sdk/client-s3'
    1
    import { createPresignedPost } from '@aws-sdk/s3-presigned-post'
    2
    3
    const { BUCKET_NAME, OBJECT_KEY } = process.env
    4
    const s3Client = new S3Client()
    5
    6
    const { url, fields } = await createPresignedPost(s3Client, {
    7
    8
    Key: 'someobject',
    9
    Conditions: [
    10
    ['content-length-range', 0, 5 * 1024 * 1024] // 5 MB max
    11
    ],
    12
    Fields: {
    13
    success_action_status: '201',
    14
    'Content-Type': 'image/png'
    15
    },
    16
    Expires: 3600
    17
    })
    18
    19
    console.log({ url, fields })
    20
    Key: 'someobject',
    import { S3Client } from '@aws-sdk/client-s3'
    1
    import { createPresignedPost } from '@aws-sdk/s3-presigned-post'
    2
    3
    const { BUCKET_NAME, OBJECT_KEY } = process.env
    4
    const s3Client = new S3Client()
    5
    6
    const { url, fields } = await createPresignedPost(s3Client, {
    7
    Bucket: 'somebucket',
    8
    9
    Conditions: [
    10
    ['content-length-range', 0, 5 * 1024 * 1024] // 5 MB max
    11
    ],
    12
    Fields: {
    13
    success_action_status: '201',
    14
    'Content-Type': 'image/png'
    15
    },
    16
    Expires: 3600
    17
    })
    18
    19
    console.log({ url, fields })
    20
    Conditions: [
    ['content-length-range', 0, 5 * 1024 * 1024] // 5 MB max
    ],
    import { S3Client } from '@aws-sdk/client-s3'
    1
    import { createPresignedPost } from '@aws-sdk/s3-presigned-post'
    2
    3
    const { BUCKET_NAME, OBJECT_KEY } = process.env
    4
    const s3Client = new S3Client()
    5
    6
    const { url, fields } = await createPresignedPost(s3Client, {
    7
    Bucket: 'somebucket',
    8
    Key: 'someobject',
    9
    10
    11
    12
    Fields: {
    13
    success_action_status: '201',
    14
    'Content-Type': 'image/png'
    15
    },
    16
    Expires: 3600
    17
    })
    18
    19
    console.log({ url, fields })
    20
    Fields: {
    success_action_status: '201',
    'Content-Type': 'image/png'
    },
    import { S3Client } from '@aws-sdk/client-s3'
    1
    import { createPresignedPost } from '@aws-sdk/s3-presigned-post'
    2
    3
    const { BUCKET_NAME, OBJECT_KEY } = process.env
    4
    const s3Client = new S3Client()
    5
    6
    const { url, fields } = await createPresignedPost(s3Client, {
    7
    Bucket: 'somebucket',
    8
    Key: 'someobject',
    9
    Conditions: [
    10
    ['content-length-range', 0, 5 * 1024 * 1024] // 5 MB max
    11
    ],
    12
    13
    14
    15
    16
    Expires: 3600
    17
    })
    18
    19
    console.log({ url, fields })
    20
    Expires: 3600
    import { S3Client } from '@aws-sdk/client-s3'
    1
    import { createPresignedPost } from '@aws-sdk/s3-presigned-post'
    2
    3
    const { BUCKET_NAME, OBJECT_KEY } = process.env
    4
    const s3Client = new S3Client()
    5
    6
    const { url, fields } = await createPresignedPost(s3Client, {
    7
    Bucket: 'somebucket',
    8
    Key: 'someobject',
    9
    Conditions: [
    10
    ['content-length-range', 0, 5 * 1024 * 1024] // 5 MB max
    11
    ],
    12
    Fields: {
    13
    success_action_status: '201',
    14
    'Content-Type': 'image/png'
    15
    },
    16
    17
    })
    18
    19
    console.log({ url, fields })
    20 console.log({ url, fields })
    import { S3Client } from '@aws-sdk/client-s3'
    1
    import { createPresignedPost } from '@aws-sdk/s3-presigned-post'
    2
    3
    const { BUCKET_NAME, OBJECT_KEY } = process.env
    4
    const s3Client = new S3Client()
    5
    6
    const { url, fields } = await createPresignedPost(s3Client, {
    7
    Bucket: 'somebucket',
    8
    Key: 'someobject',
    9
    Conditions: [
    10
    ['content-length-range', 0, 5 * 1024 * 1024] // 5 MB max
    11
    ],
    12
    Fields: {
    13
    success_action_status: '201',
    14
    'Content-Type': 'image/png'
    15
    },
    16
    Expires: 3600
    17
    })
    18
    19
    20
    import { S3Client } from '@aws-sdk/client-s3'
    import { createPresignedPost } from '@aws-sdk/s3-presigned-post'
    const { BUCKET_NAME, OBJECT_KEY } = process.env
    const s3Client = new S3Client()
    const { url, fields } = await createPresignedPost(s3Client, {
    Bucket: 'somebucket',
    Key: 'someobject',
    Conditions: [
    ['content-length-range', 0, 5 * 1024 * 1024] // 5 MB max
    ],
    Fields: {
    success_action_status: '201',
    'Content-Type': 'image/png'
    },
    Expires: 3600
    })
    console.log({ url, fields })
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    loige 62

    View full-size slide

  58. // you can use `url` and `fields` to generate an HTML form
    const code = `Upload an image to S3

    ${Object.entries(fields).map(([key, value]) => {
    return ``
    }).join('\n')}


    `
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    loige 63

    View full-size slide

  59. loige
    Limitations and quirks
    It supports only 1 file (cannot upload multiple files in one go)
    The file field must be the last entry in the form
    (S3 will ignore every other field after the file)
    From the browser (AJAX) you need to enable CORS on the bucket
    64

    View full-size slide

  60. loige
    Should I use PUT or POST?
    🧐
    PUT is simpler but definitely more limited
    POST is slightly more complicated (and less adopted) but it's more flexible
    You should probably put some time into learning POST and use that!
    65

    View full-size slide

  61. loige
    Pre-signed URLs for other operations
    S3 pre-signed URLs are not limited to GET, PUT or POST operations
    You can literally create pre-signed URLs for any command
    (DeleteObject, ListBuckets, MultiPartUpload, etc...)
    66

    View full-size slide

  62. loige
    Do you need moar examples?
    😼
    github.com/lmammino/s3-presigned-urls-examples
    67

    View full-size slide

  63. loige
    ... In summary
    S3 pre-signed URLs are a great way to authorise operations on S3
    They are generally used to implement upload/download features
    The signature is created client-side so you can sign anything. Access is validated at
    request time
    This is not the only solution, you can also use the JavaScript SDK from the frontend
    and get limited credentials from Cognito (Amplify makes that process simpler)
    For upload you can use PUT and POST, but POST is much more flexible
    💬 PS: Meower.com doesn't really exist... but... do you want to invest?! It's a great
    idea, trust me!
    68

    View full-size slide

  64. Cover photo by on
    Kelly Sikkema Unsplash
    fourtheorem.com
    THANKS!
    🙌
    fth.link/presign
    loige
    It's a wrap!
    69

    View full-size slide