ECSとSQSでスケーラブルなバッチを作った

Transcript

1. &$4ͱ424Ͱ
   εέʔϥϒϧͳόονΛ࡞ͬͨ
   ٢ాوจ ![FQIJSBOTBT
   
   Ϋϥεϝιουגࣜձࣾ
   

   View Slide

2. εϥΠυ͸ޙͰೖख͢Δ͜ͱ͕ग़དྷ·͢ͷͰ
   ൃදதͷ಺༰ΛϝϞ͢Δඞཁ͸͋Γ·ͤΜɻ
   ࣸਅࡱӨΛ͢Δ৔߹͸
   ϑϥογϡɾγϟολʔԻ͕ग़ͳ͍Α͏ʹ͝഑ྀ͍ͩ͘͞
   Attention
   

   View Slide

3. 
   
   #jawsug
   #jawsoka
   #soracomug
   

   View Slide

4. ࣗݾ঺հ
   
   ٢ాوจ ![FQIJSBOTBT
   
   wΫϥεϝιουגࣜձࣾ
   w$9ࣄۀຊ෦αʔόαΠυΤϯδχΞ
   wԬࢁ+BWBϢʔβձ୅දΦʔϓϯη
   ϛφʔԬࢁ࣮ߦҕһ
   w޷͖ͳ"84ͷαʔϏε
   w&$4 %ZOBNP%#
   

   View Slide

5. ΞδΣϯμ
   
   wΞʔΩςΫνϟ֓ཁ
   w4XJUDI3PMFʹ͍ͭͯ
   w424Ͱ΍ͬͨ͜ͱ
   w&$4Ͱ΍ͬͨ͜ͱ
   w΍ͬͯΈͨ
   

   View Slide

6. 
   
   ΞʔΩςΫνϟ֓ཁ
   

   View Slide

7. ΍Γ͍ͨ͜ͱ
   
   wσʔλͷҰׅߋ৽
   wݩσʔλ͸$47
   wσʔλྔ͸े਺ສ݅ఔ౓
   w*%ͱɺߋ৽಺༰͕ೖ͍ͬͯΔ
   wߋ৽ʹ͸֎෦ͷ"1*Λୟ͘
   

   View Slide

8. 
   
   

   View Slide

9. 
   
   Switch RoleͰ
   ΍ͬͨ͜ͱ
   

   View Slide

10. 
    
    w424΁ͷσʔλૹ৴ॲཧͰ4XJUDI3PMF͍ͨ͠
    w4XJUDI3PMF͢Δʹ͸.'"ඞਢ
    wBXTDMJͰ͋Ε͹్தͰτʔΫϯΛೖྗͰ͖Δ
    w4%,ͩͱࣗલͰΫϨσϯγϟϧΛऔಘͯ͠΍
    Δඞཁ͕͋Δ
    

    View Slide

11. BXTDPOpH
    
    [default]
    region = ap-northeast-1
    output = json
    [profile hoge]
    region = ap-northeast-1
    source_profile = default
    role_arn = arn:aws:iam::ACCOUNT_ID:role/john-doe
    mfa_serial = arn:aws:iam::ACCOUNT_ID:mfa/john-doe
    

    View Slide

12. BXTDMJͰ4XJUDI3PMF͢Δ৔߹
    
    $ AWS_PROFILE=hoge aws s3 ls
    Enter MFA code for arn:aws:iam::ACCOUNT_ID:mfa/john-doe
    [MFAτʔΫϯΛೖྗ͢Δ]
    

    View Slide

13. "3/ɺ.'"τʔΫϯɺTUTΫϥΠΞϯτ
    
    sts_client =
    Aws::STS::Client.new(region: 'ap-northeast-1')
    role_arn =
    `aws configure get role_arn --profile hoge`.chomp
    serial_number =
    `aws configure get mfa_serial --profile hoge`.chomp
    puts "Input MFA token code..."
    token_code = gets.chomp
    

    View Slide

14. ΫϨσϯγϟϧੜ੒͠424ΫϥΠΞϯτΛ࡞੒
    
    role_credentials = Aws::AssumeRoleCredentials.new(
    client: sts_client,
    role_arn: role_arn,
    role_session_name: "hoge_session",
    serial_number: serial_number,
    token_code: token_code)
    Aws::SQS::Client.new(credentials: role_credentials)
    

    View Slide

15. 
    
    w؀ڥม਺"84@130'*-&͸ར༻͠ͳ͍
    wBXTDPOpHVSFHFUͰඞཁͳ"3/Λऔಘ͢Δ
    w.'"τʔΫϯ͸ผ్ɺೖྗͤ͞Δ
    wTUTΫϥΠΞϯτΛ࡞੒͠ɺ
    "TTVNF3PMF$SFEFOUJBMTͰɺΫϨσϯγϟϧ
    Λऔಘ͢Δ
    

    View Slide

16. 
    
    SQSͰ΍ͬͨ͜ͱ
    

    View Slide

17. 
    
    wෳ਺ͷλεΫ͔ΒΞΫηε͞ΕΔͷͰɺ͜ΕΛ
    ͍͍ײ͡ʹॲཧͰ͖Δ
    wॲཧͰ͖ͳ͔ͬͨσʔλΛɺผΩϡʔʹҠͯ͠
    ϦτϥΠ͠΍͘͢͢Δ
    w424ͷ%FBE-FUUFS2VFVFͷ࢓૊ΈΛ࢖͏
    

    View Slide

18. 
    
    

    View Slide

19. 
    
    #PEZ 3FDFJWF$PVOU 7JTJCMF
    536&
    

    View Slide

20. 
    
    #PEZ 3FDFJWF$PVOU 7JTJCMF
    '"-4&
    

    View Slide

21. 
    
    #PEZ 3FDFJWF$PVOU 7JTJCMF
    536&
    VisibilityTimeoutΛա͗ͯ΋Delete͞Εͳ͔ͬͨ৔߹
    

    View Slide

22. 
    
    #PEZ 3FDFJWF$PVOU 7JTJCMF
    Receive Count͕࠷େReceive CountΛ௒͑ͨ৔߹
    #PEZ 3FDFJWF$PVOU 7JTJCMF
    536&
    DLQ΁Ҡಈ
    

    View Slide

23. 
    
    ECSͰ΍ͬͨ͜ͱ
    

    View Slide

24. 
    
    w'BSHBUFͰϦιʔε؅ཧͷखؒΛݮΒ͍ͨ͠
    wฒྻͰ࣮ߦͰ͖ΔΑ͏ʹ͍ͨ͠
    wঢ়گʹԠͯ͡ɺλεΫͷ਺Λௐ੔͍ͨ͠
    w$MJFOU4FDSFUͳͲΛ҆શʹѻ͍͍ͨ
    

    View Slide

25. 
    
    

    View Slide

26. ύϥϝʔλετΞʹઃఆ஋Λ֨ೲ
    
    aws ssm put-parameter \
    --name /ClientId \
    --value CLIENT_ID_XXXX \
    --type String
    

    View Slide

27. λεΫఆ͔ٛΒࢀর
    
    ContainerDefinitions:
    -
    Name: app
    ...
    Secrets:
    - Name: CLIENT_ID
    ValueFrom:
    !Sub "arn:aws:ssm:ap-northeast-1:${AWS::AccountId}:parameter/ClientId"
    - Name: CLIENT_SECRET
    ValueFrom:
    !Sub "arn:aws:ssm:ap-northeast-1:${AWS::AccountId}:parameter/ClientSecret"
    λεΫ಺ͷ؀ڥม਺Ͱ஋ΛऔಘͰ͖Δ
    

    View Slide

28. 
    
    ΍ͬͯΈͨ
    

    View Slide

29. 
    
    wର৅σʔλສ݅
    w424΁ͷσʔλૹ৴ʹ࣌ؒ
    wʢͳΜ͔վળ͍ͨ͠ؾ͕͢Δ
    w&$4ͷόονॲཧ͕࣌ؒະຬͰऴྃ
    

    View Slide

30. 
    
    w4%,Ͱ4XJUDI3PMF͢Δʹ͸ͻͱखؒඞཁ
    w424͸࢓૊ΈΛཧղ͔ͯͭ͑͠͹ɺ͘͢͝ศ
    ར
    w&$4ͷฒྻλεΫΛ࢖ͬͯɺεέʔϥϒϧʹ͠
    Α͏
    

    View Slide

31. View Slide