Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ECSとSQSでスケーラブルなバッチを作った

 ECSとSQSでスケーラブルなバッチを作った

Takafumi Yoshida

August 17, 2019
Tweet

More Decks by Takafumi Yoshida

Other Decks in Technology

Transcript

  1. &$4ͱ424Ͱ
    εέʔϥϒϧͳόονΛ࡞ͬͨ
    ٢ాوจ ![FQIJSBOTBT

    Ϋϥεϝιουגࣜձࣾ

    View Slide

  2. εϥΠυ͸ޙͰೖख͢Δ͜ͱ͕ग़དྷ·͢ͷͰ
    ൃදதͷ಺༰ΛϝϞ͢Δඞཁ͸͋Γ·ͤΜɻ
    ࣸਅࡱӨΛ͢Δ৔߹͸
    ϑϥογϡɾγϟολʔԻ͕ग़ͳ͍Α͏ʹ͝഑ྀ͍ͩ͘͞
    Attention

    View Slide



  3. #jawsug
    #jawsoka
    #soracomug

    View Slide

  4. ࣗݾ঺հ

    ٢ాوจ ![FQIJSBOTBT

    wΫϥεϝιουגࣜձࣾ
    w$9ࣄۀຊ෦αʔόαΠυΤϯδχΞ
    wԬࢁ+BWBϢʔβձ୅දΦʔϓϯη
    ϛφʔԬࢁ࣮ߦҕһ
    w޷͖ͳ"84ͷαʔϏε
    w&$4 %ZOBNP%#

    View Slide

  5. ΞδΣϯμ

    wΞʔΩςΫνϟ֓ཁ
    w4XJUDI3PMFʹ͍ͭͯ
    w424Ͱ΍ͬͨ͜ͱ
    w&$4Ͱ΍ͬͨ͜ͱ
    w΍ͬͯΈͨ

    View Slide



  6. ΞʔΩςΫνϟ֓ཁ

    View Slide

  7. ΍Γ͍ͨ͜ͱ

    wσʔλͷҰׅߋ৽
    wݩσʔλ͸$47
    wσʔλྔ͸े਺ສ݅ఔ౓
    w*%ͱɺߋ৽಺༰͕ೖ͍ͬͯΔ
    wߋ৽ʹ͸֎෦ͷ"1*Λୟ͘

    View Slide



  8. View Slide



  9. Switch RoleͰ
    ΍ͬͨ͜ͱ

    View Slide



  10. w424΁ͷσʔλૹ৴ॲཧͰ4XJUDI3PMF͍ͨ͠
    w4XJUDI3PMF͢Δʹ͸.'"ඞਢ
    wBXTDMJͰ͋Ε͹్தͰτʔΫϯΛೖྗͰ͖Δ
    w4%,ͩͱࣗલͰΫϨσϯγϟϧΛऔಘͯ͠΍
    Δඞཁ͕͋Δ

    View Slide

  11. BXTDPOpH

    [default]
    region = ap-northeast-1
    output = json
    [profile hoge]
    region = ap-northeast-1
    source_profile = default
    role_arn = arn:aws:iam::ACCOUNT_ID:role/john-doe
    mfa_serial = arn:aws:iam::ACCOUNT_ID:mfa/john-doe

    View Slide

  12. BXTDMJͰ4XJUDI3PMF͢Δ৔߹

    $ AWS_PROFILE=hoge aws s3 ls
    Enter MFA code for arn:aws:iam::ACCOUNT_ID:mfa/john-doe
    [MFAτʔΫϯΛೖྗ͢Δ]

    View Slide

  13. "3/ɺ.'"τʔΫϯɺTUTΫϥΠΞϯτ

    sts_client =
    Aws::STS::Client.new(region: 'ap-northeast-1')
    role_arn =
    `aws configure get role_arn --profile hoge`.chomp
    serial_number =
    `aws configure get mfa_serial --profile hoge`.chomp
    puts "Input MFA token code..."
    token_code = gets.chomp

    View Slide

  14. ΫϨσϯγϟϧੜ੒͠424ΫϥΠΞϯτΛ࡞੒

    role_credentials = Aws::AssumeRoleCredentials.new(
    client: sts_client,
    role_arn: role_arn,
    role_session_name: "hoge_session",
    serial_number: serial_number,
    token_code: token_code)
    Aws::SQS::Client.new(credentials: role_credentials)

    View Slide



  15. w؀ڥม਺"84@130'*-&͸ར༻͠ͳ͍
    wBXTDPOpHVSFHFUͰඞཁͳ"3/Λऔಘ͢Δ
    w.'"τʔΫϯ͸ผ్ɺೖྗͤ͞Δ
    wTUTΫϥΠΞϯτΛ࡞੒͠ɺ
    "TTVNF3PMF$SFEFOUJBMTͰɺΫϨσϯγϟϧ
    Λऔಘ͢Δ

    View Slide



  16. SQSͰ΍ͬͨ͜ͱ

    View Slide



  17. wෳ਺ͷλεΫ͔ΒΞΫηε͞ΕΔͷͰɺ͜ΕΛ
    ͍͍ײ͡ʹॲཧͰ͖Δ
    wॲཧͰ͖ͳ͔ͬͨσʔλΛɺผΩϡʔʹҠͯ͠
    ϦτϥΠ͠΍͘͢͢Δ
    w424ͷ%FBE-FUUFS2VFVFͷ࢓૊ΈΛ࢖͏

    View Slide



  18. View Slide



  19. #PEZ 3FDFJWF$PVOU 7JTJCMF
    536&

    View Slide



  20. #PEZ 3FDFJWF$PVOU 7JTJCMF
    '"-4&

    View Slide



  21. #PEZ 3FDFJWF$PVOU 7JTJCMF
    536&
    VisibilityTimeoutΛա͗ͯ΋Delete͞Εͳ͔ͬͨ৔߹

    View Slide



  22. #PEZ 3FDFJWF$PVOU 7JTJCMF
    Receive Count͕࠷େReceive CountΛ௒͑ͨ৔߹
    #PEZ 3FDFJWF$PVOU 7JTJCMF
    536&
    DLQ΁Ҡಈ

    View Slide



  23. ECSͰ΍ͬͨ͜ͱ

    View Slide



  24. w'BSHBUFͰϦιʔε؅ཧͷखؒΛݮΒ͍ͨ͠
    wฒྻͰ࣮ߦͰ͖ΔΑ͏ʹ͍ͨ͠
    wঢ়گʹԠͯ͡ɺλεΫͷ਺Λௐ੔͍ͨ͠
    w$MJFOU4FDSFUͳͲΛ҆શʹѻ͍͍ͨ

    View Slide



  25. View Slide

  26. ύϥϝʔλετΞʹઃఆ஋Λ֨ೲ

    aws ssm put-parameter \
    --name /ClientId \
    --value CLIENT_ID_XXXX \
    --type String

    View Slide

  27. λεΫఆ͔ٛΒࢀর

    ContainerDefinitions:
    -
    Name: app
    ...
    Secrets:
    - Name: CLIENT_ID
    ValueFrom:
    !Sub "arn:aws:ssm:ap-northeast-1:${AWS::AccountId}:parameter/ClientId"
    - Name: CLIENT_SECRET
    ValueFrom:
    !Sub "arn:aws:ssm:ap-northeast-1:${AWS::AccountId}:parameter/ClientSecret"
    λεΫ಺ͷ؀ڥม਺Ͱ஋ΛऔಘͰ͖Δ

    View Slide



  28. ΍ͬͯΈͨ

    View Slide



  29. wର৅σʔλສ݅
    w424΁ͷσʔλૹ৴ʹ࣌ؒ
    wʢͳΜ͔վળ͍ͨ͠ؾ͕͢Δ
    w&$4ͷόονॲཧ͕࣌ؒະຬͰऴྃ

    View Slide



  30. w4%,Ͱ4XJUDI3PMF͢Δʹ͸ͻͱखؒඞཁ
    w424͸࢓૊ΈΛཧղ͔ͯͭ͑͠͹ɺ͘͢͝ศ

    w&$4ͷฒྻλεΫΛ࢖ͬͯɺεέʔϥϒϧʹ͠
    Α͏

    View Slide

  31. View Slide