AWSではてなブログの常時HTTPS配信をバーンとやる話 / The Epic of migration from HTTP to HTTPS on Hatena Blog with AWS

Hatena Engineer Seminar #10 (https://hatena.connpass.com/event/87909/) で発表した資料です。

Transcript

1. AWSͰ͸ͯͳϒϩάͷ

   ৗ࣌HTTPS഑৴Λ

   όʔϯͱ΍Δ࿩
   Hatena Engineer Seminar #10 @ Tokyo
   
   גࣜձࣾ͸ͯͳ id:aereal
   

   View Slide

2. ࣗݾ঺հ
   • id:aereal
   
   • GitHub: aereal
   
   • Twitter: aereal
   
   • ϒϩάϢʔβʔνʔϜ

   ΞϓϦέʔγϣϯΤϯδχΞ

   ςοΫϦʔυ
   

   View Slide

3. ࿩͢͜ͱ
   • ͸ͯͳϒϩάͷৗ࣌HTTPS഑৴ͷٕज़తͳৄࡉ
   
   • ূ໌ॻͷಈతಡΈࠐΈ
   
   • ূ໌ॻͷࣗಈߋ৽
   
   • ͓ΑͼϓϩδΣΫτͷਐΊํ
   

   View Slide

4. എܠ
   • ͸ͯͳϒϩάͰ͸ɺ͸ͯͳఏڙυϝΠϯͷ

   ͍ͣΕ͔͔Βબ΂·͢
   
   • *.hatenablog.com, *.hatenadiary.jp, etc.
   
   • ͞Βʹɺ͸ͯͳϒϩάPro (༗ྉΦϓγϣϯ) ʹਃ͠ࠐΉͱ

   ಠࣗυϝΠϯ͕࢖͑·͢
   
   • ಠࣗυϝΠϯͰ΋ৗ࣌HTTPS഑৴Λ࣮ݱ͍ͨ͠
   

   View Slide

5. Let's Encrypt
   • https://letsencrypt.org/
   
   • ISRG = Internet Security Research Group͕ఏڙ͢Δ

   ϓϩάϥϚϒϧʹΞΫηεՄೳͳೝূہ (CA)
   
   • ෆಛఆଟ਺ͷυϝΠϯʹର͢Δ

   ূ໌ॻൃߦͷࣗಈԽ͕Մೳʹͳͬͨ
   

   View Slide

6. Let's Encrypt
   • ಠࣗυϝΠϯͷৗ࣌HTTPS഑৴ʹ͔ܽͤͳ͍ଘࡏɺ౰વ
   ར༻͠·͢
   
   • Let's EncryptΛར༻͢Δاۀͱͯ͠ɺ

   ·ͨܝ͛Δࢥ૝ʹڞײ͢ΔWebαʔϏεࣄۀऀͱͯ͠ɺ

   ͸ͯͳ͸Let's Encryptʹد෇Λ͠·͢
   

   View Slide

7. ಠࣗυϝΠϯͱূ໌ॻ
   • DONE: ͸ͯͳఏڙυϝΠϯ (*.hatenablog.com, etc.)
   
   • ਺͕஌Ε͍ͯΔͷͰূ໌ॻ1ͭͷ഑ஔͰࡁΉ
   
   • SAN (= Subject Alternative Names) Λ࢖͏
   
   • ϫΠϧυΧʔυূ໌ॻΛ࢖͏
   

   View Slide

8. ಠࣗυϝΠϯͱূ໌ॻ
   • ಠࣗυϝΠϯ
   
   • ਺͕ଟ͍ͷͰূ໌ॻͷൃߦ΋ಡΈࠐΈ΋େม
   
   • LE = Let's Encrypt͸ূ໌ॻ͋ͨΓ

   100υϝΠϯͷ੍໿͕͋Δ
   
   • Ұ౓ʹಡΈࠐΉͱproxyͷϝϞϦ࢖༻ྔ͕ਹΉ
   

   View Slide

9. ΰʔϧ(1): ূ໌ॻͷಡΈࠐΈ
   • ؆୯ͷͨΊূ໌ॻ͸1υϝΠϯ1ͭɺSAN͸࢖Θͳ͍
   
   • ΦϯσϚϯυͰಡΈࠐΜͰϝϞϦઅ໿
   
   • ϘτϧωοΫʹͳΓ͏ΔͷͰ

   ϥ΢ϯυτϦοϓɺϨΠςϯγΛ཈͍͑ͨ
   

   View Slide

10. ΰʔϧ (2): ఆظߋ৽
    • ϦΞϧλΠϜੑ͸௿͍
    
    • ظݶΛܴ͑Δ·Ͱͷ೚ҙͷλΠϛϯάͰ

    ࣮ߦ͢Ε͹Α͍
    
    • ҰํɺσʔλҰ؏ੑʹର͢Δཁٻ͕ߴ͍
    
    • ࣦഊ͢ΔͳͲߋ৽࿙Ε͕͋Δͱ·͍ͣ
    
    • ֎෦API (LE) Λར༻͢ΔͨΊࣦഊՄೳੑ͕ߴ͍

    →ద੾ͳϦτϥΠॲཧ͕ඞཁ
    

    View Slide

11. γεςϜͷߏ੒
    

    View Slide

12. cert-dispatcher
    cert-cache-gw
    cert-updater-state
    cert-updater-function
    cert-update-notifier
    cert-update-trigger
    Let's Encrypt
    cert-store
    cert-cache cert-lifecycle-store
    User
    Blog
    HTTP
    ssl_handshake_handler
    HTTP
    Get/Set
    Get
    HTTP
    HTTP
    ࣮ߦ
    ࣮ߦ ࣮ߦ
    UpdateItem
    UpdateItem
    TTL trigger
    ࣮ߦ
    ূ໌ॻൃߦ
    

    View Slide

13. ূ໌ॻͷಡΈࠐΈ
    

    View Slide

14. cert-dispatcher
    cert-cache-gw
    cert-updater-state
    cert-updater-function
    cert-update-notifier
    cert-update-trigger
    Let's Encrypt
    cert-store
    cert-cache cert-lifecycle-store
    User
    Blog
    HTTPS
    ssl_handshake_handler
    HTTP
    Get/Set
    Get
    HTTP
    HTTP
    ࣮ߦ
    ࣮ߦ ࣮ߦ
    UpdateItem
    UpdateItem
    TTL trigger
    ࣮ߦ
    ূ໌ॻൃߦ
    

    View Slide

15. cert-dispatcher
    cert-cache-gw
    cert-updater-state
    cert-updater-function
    cert-update-notifier
    cert-update-trigger
    Let's Encrypt
    cert-store
    cert-cache cert-lifecycle-store
    User
    Blog
    HTTP
    ssl_handshake_handler
    HTTP
    Get/Set
    Get
    HTTP
    HTTP
    ࣮ߦ
    ࣮ߦ ࣮ߦ
    UpdateItem
    UpdateItem
    TTL trigger
    ࣮ߦ
    ূ໌ॻൃߦ
    

    View Slide

16. cert-dispatcher
    cert-cache-gw
    cert-updater-state
    cert-updater-function
    cert-update-notifier
    cert-update-trigger
    Let's Encrypt
    cert-store
    cert-cache cert-lifecycle-store
    User
    Blog
    HTTP
    ssl_handshake_handler
    HTTP
    Get/Set
    Get
    HTTP
    HTTP
    ࣮ߦ
    ࣮ߦ ࣮ߦ
    UpdateItem
    UpdateItem
    TTL trigger
    ࣮ߦ
    ূ໌ॻൃߦ
    

    View Slide

17. ূ໌ॻͷಈతಡΈࠐΈ
    • cert-dispatcher: ngx_mruby
    
    • TLS handshake࣌ʹϋϯυϥ͕ݺ͹ΕΔ
    
    • cert-cache-gwʹHTTP GETͯ͠ূ໌ॻΛऔಘ͢Δ
    
    • cert-cache-gw: GoͰॻ͍ͨHTTP API
    
    • υϝΠϯʹରԠ͢Δূ໌ॻΛcert-store (DynamoDB) ͔
    Βऔಘͯ͠ฦ͢
    
    • cert-cache (memcached) ʹ΋อଘ͢Δ
    

    View Slide

18. ূ໌ॻͷऔಘ
    

    View Slide

19. cert-dispatcher
    cert-cache-gw
    cert-updater-state
    cert-updater-function
    cert-update-notifier
    cert-update-trigger
    Let's Encrypt
    cert-store
    cert-cache cert-lifecycle-store
    User
    Blog
    HTTP
    ssl_handshake_handler
    HTTP
    Get/Set
    Get
    HTTP
    HTTP
    ࣮ߦ
    ࣮ߦ ࣮ߦ
    UpdateItem
    UpdateItem
    TTL trigger
    ࣮ߦ
    ূ໌ॻൃߦ
    

    View Slide

20. cert-dispatcher
    cert-cache-gw
    cert-updater-state
    cert-updater-function
    cert-update-notifier
    cert-update-trigger
    Let's Encrypt
    cert-store
    cert-cache cert-lifecycle-store
    User
    Blog
    HTTP
    ssl_handshake_handler
    HTTP
    Get/Set
    Get
    HTTP
    HTTP
    ࣮ߦ
    ࣮ߦ ࣮ߦ
    UpdateItem
    UpdateItem
    TTL trigger
    ࣮ߦ
    ূ໌ॻൃߦ
    

    View Slide

21. cert-dispatcher
    cert-cache-gw
    cert-updater-state
    cert-updater-function
    cert-update-notifier
    cert-update-trigger
    Let's Encrypt
    cert-store
    cert-cache cert-lifecycle-store
    User
    Blog
    HTTP
    ssl_handshake_handler
    HTTP
    Get/Set
    Get
    HTTP
    HTTP
    ࣮ߦ
    ࣮ߦ ࣮ߦ
    UpdateItem
    UpdateItem
    TTL trigger
    ࣮ߦ
    ূ໌ॻൃߦ
    

    View Slide

22. cert-dispatcher
    cert-cache-gw
    cert-updater-state
    cert-updater-function
    cert-update-notifier
    cert-update-trigger
    Let's Encrypt
    cert-store
    cert-cache cert-lifecycle-store
    User
    Blog
    HTTP
    ssl_handshake_handler
    HTTP
    Get/Set
    Get
    HTTP
    HTTP
    ࣮ߦ
    ࣮ߦ ࣮ߦ
    UpdateItem
    UpdateItem
    TTL trigger
    ࣮ߦ
    ূ໌ॻൃߦ
    

    View Slide

23. cert-dispatcher
    cert-cache-gw
    cert-updater-state
    cert-updater-function
    cert-update-notifier
    cert-update-trigger
    Let's Encrypt
    cert-store
    cert-cache cert-lifecycle-store
    User
    Blog
    HTTP
    ssl_handshake_handler
    HTTP
    Get/Set
    Get
    HTTP
    HTTP
    ࣮ߦ
    ࣮ߦ ࣮ߦ
    UpdateItem
    UpdateItem
    TTL trigger
    ࣮ߦ
    ূ໌ॻൃߦ
    

    View Slide

24. cert-dispatcher
    cert-cache-gw
    cert-updater-state
    cert-updater-function
    cert-update-notifier
    cert-update-trigger
    Let's Encrypt
    cert-store
    cert-cache cert-lifecycle-store
    User
    Blog
    HTTP
    ssl_handshake_handler
    HTTP
    Get/Set
    Get
    HTTP
    HTTP
    ࣮ߦ
    ࣮ߦ ࣮ߦ
    UpdateItem
    UpdateItem
    TTL trigger
    ࣮ߦ
    ূ໌ॻൃߦ
    

    View Slide

25. ূ໌ॻͷൃߦ
    • cert-updater-state: AWS Step Functions
    
    • JSONͰεςʔτϚγϯΛ࣮ߦͯ͘͠ΕΔαʔϏε
    
    • ॊೈͳϦτϥΠॲཧ΍ঢ়ଶભҠΛ؅ཧͰ͖Δ
    
    • cert-updater-function: AWS Lambda
    
    • LEͱ௨৴͠ূ໌ॻΛऔಘ͢Δ
    
    • ൃߦͨ͠ূ໌ॻ͸DynamoDBʹॻ͖ࠐΉ
    
    • cert-update-notifier: AWS Lambda
    
    • ূ໌ॻͷൃߦঢ়گΛ͸ͯͳϒϩάຊମʹ఻͑Δ
    

    View Slide

26. ূ໌ॻͷఆظߋ৽
    

    View Slide

27. cert-dispatcher
    cert-cache-gw
    cert-updater-state
    cert-updater-function
    cert-update-notifier
    cert-update-trigger
    Let's Encrypt
    cert-store
    cert-cache cert-lifecycle-store
    User
    Blog
    HTTP
    ssl_handshake_handler
    HTTP
    Get/Set
    Get
    HTTP
    HTTP
    ࣮ߦ
    ࣮ߦ ࣮ߦ
    UpdateItem
    UpdateItem
    TTL trigger
    ࣮ߦ
    ূ໌ॻൃߦ
    

    View Slide

28. cert-dispatcher
    cert-cache-gw
    cert-updater-state
    cert-updater-function
    cert-update-notifier
    cert-update-trigger
    Let's Encrypt
    cert-store
    cert-cache cert-lifecycle-store
    User
    Blog
    HTTP
    ssl_handshake_handler
    HTTP
    Get/Set
    Get
    HTTP
    HTTP
    ࣮ߦ
    ࣮ߦ ࣮ߦ
    UpdateItem
    UpdateItem
    TTL trigger
    ࣮ߦ
    ূ໌ॻൃߦ
    

    View Slide

29. ূ໌ॻͷఆظߋ৽
    • cert-lifecycle-store: DynamoDB
    
    • ূ໌ॻͷऔಘ࣌ʹ͜ͷςʔϒϧʹ΋ॻ͖ࠐΉ
    
    • TTL triggerΛൃߦ͠ɺcert-update-triggerΛىಈ͢Δ
    
    • cert-update-trigger: AWS Lambda
    
    • TTL͕੾Εͯ࡟আ͞ΕͨΞΠςϜΛड͚औΔ
    
    • cert-updater-stateΛ࣮ߦ͠ɺূ໌ॻऔಘϑϩʔΛ։࢝
    

    View Slide

30. DynamoDB TTL
    

    View Slide

31. cert-lifecycle-store

    (DynamoDB)
    
    Domain: ex1.example.com
    ExpiresAt: 2018-05-23T02:00:00
    Domain: ex2.example.com
    ExpiresAt: 2018-05-23T03:00:00
    Domain: ex2.example.com
    ExpiresAt: 2018-05-23T04:00:00
    Domain: ex2.example.com
    ExpiresAt: 2018-05-23T05:00:00
    

    View Slide

32. cert-lifecycle-store

    (DynamoDB)
    
    Domain: ex2.example.com
    ExpiresAt: 2018-05-23T03:00:00
    Domain: ex2.example.com
    ExpiresAt: 2018-05-23T04:00:00
    Domain: ex2.example.com
    ExpiresAt: 2018-05-23T05:00:00
    

    View Slide

33. cert-lifecycle-store

    (DynamoDB)
    Domain: ex2.example.com
    ExpiresAt: 2018-05-23T04:00:00
    Domain: ex2.example.com
    ExpiresAt: 2018-05-23T05:00:00
    

    View Slide

34. cert-lifecycle-store

    (DynamoDB)
    
    Domain: ex2.example.com
    ExpiresAt: 2018-05-23T05:00:00
    

    View Slide

35. ͳͥAWS͔
    • ෳࡶ͔ͭߴ౓ͳόονΛߏஙʹඞཁͳαʔϏε͕

    ἧ͍ͬͯΔ͔Β
    
    • ෳࡶ: ূ໌ॻͷऔಘɾ݁Ռͷ௨஌ͳͲෳ਺εςοϓ͔Β
    ͳΔ
    
    • ߴ౓: ෼ࢄΞϓϦέʔγϣϯʹ΋ؔΘΒͣ

    σʔλͷҰ؏ੑ͕ߴ͍ϨϕϧͰٻΊΒΕΔ
    
    • = Lambda, Step Functions, etc.
    

    View Slide

36. Step Functions͸࠷ߴ
    • ग़ྗ಺༰ʹԠͯ͡ঢ়ଶભҠΛ෼ذͰ͖Δ
    
    • άϥϑΟΧϧʹग़ྗͯ͘͠ΕΔ (͍͢͝!)
    
    • Τϥʔग़ྗ಺༰ʹԠ্ͨ͡ݶ෇͖ϦτϥΠॲཧ
    
    • ΊͪΌͪ͘Ό͔ͬ͜Α͘ͳ͍Ͱ͔͢?
    

    View Slide

37. View Slide

38. View Slide

39. Go
    • Lambda function͸͢΂ͯGo, cert-cache-gw΋Go
    
    • ίϯύΠϥʹΑΔܕݕࠪͰ҆৺
    • ೖग़ྗͷܕΛLambda functionؒͰڞ༗Ͱ͖ɺ

    ᴥᴪ͕ੜͨ͡ΒίϯύΠϧΤϥʔʹͳΔ
    
    • ΤίγεςϜ͕੒ख़͍ͯ͠Δ
    
    • ΫϩείϯύΠϧ
    
    • ςετϥϯφʔɺςετϑϨʔϜϫʔΫ
    

    View Slide

40. ϓϩδΣΫτͷਐΊํ
    • ΞʔΩςΫνϟΛݕ౼
    
    • AWSΛۦ࢖ͨ࣍͠ੈ୅TSDBͷઃܭʹؔΘͬͨ

    id:y_uukiʹڠྗͯ͠΋Βͬͨ
    
    • http://blog.yuuk.io/entry/the-rebuild-of-tsdb-on-cloud
    
    • ϓϩτλΠϐϯά (1िؒ)
    
    • ࣮ࡍʹखΛಈ͔͢͜ͱͰෆ໌ྎͩͬͨ఺ͷݟੵ΋Γ͕ਖ਼֬ʹͳͬͨ
    
    • (LambdaͷσϓϩΠͳͲ)
    
    • Goॳֶऀͩͬͨϝϯόʔ΋צΛ௫Ίͯɺຊ࣮૷Ͱ૝ఆҎ্ʹ

    ϕϩγςΟ͕҆ఆͨ͠
    

    View Slide

41. ϓϩδΣΫτͷਐΊํ
    • ·ͣূ໌ॻಡΈࠐΈ෦෼ (cert-loader) Λ։ൃ
    
    • ͜ͷ࣌఺Ͱ͸·ͩূ໌ॻΛऔಘ͠ͳ͍ͷͰɺӨڹ͸ͳ
    ͍
    
    • ࣍ʹূ໌ॻऔಘ෦෼ (cert-updater) Λ։ൃ
    
    • ͜ͷ෦෼ΛϦϦʔε͢Δ͜ͱͰ͸͡Ίͯ

    ಠࣗυϝΠϯͰHTTPS഑৴͕ར༻ՄೳʹͳΔ
    

    View Slide

42. ϓϩδΣΫτͷਐΊํ
    • ϦϦʔε୯ҐͷCQS = ίϚϯυΫΤϦ෼ׂ͕ͳ͞Εͨ
    
    • command: cert-updater
    
    • query: cert-loader
    
    • CQS = Command-query Separation:
    
    • มߋܥ (command) ͱಡऔܥ (query) Λ

    ෼ׂ͢ΔΞʔΩςΫνϟ
    
    • େ͖ͳϦϦʔεͰ͋Δ͕ɺগͣͭ͠ग़͍ͯ͘͠ͱ͍͏

    ීஈͷελΠϧΛऔΓೖΕΒΕ͍ͯΔ
    

    View Slide

43. ·ͱΊ
    • ͸ͯͳϒϩάͷৗ࣌HTTPS഑৴ͷཪଆΛ͝঺հ͠·ͨ͠
    
    • ։ൃ͸ॱௐͰɺࠂ஌௨ΓͷεέδϡʔϧͰ

    ఏڙ։࢝Ͱ͖ΔݟࠐΈͰ͢
    
    • ࣮͸Perl͚ͩ͡Όͳ͍͠ɺAWS΋׆༻͍ͯ͠·͢!
    

    View Slide

44. એ఻: αϚʔΠϯλʔϯ2018
    • http://developer.hatenastaff.com/entry/intern-
    preentry-2018
    
    • ࠓ೥΋΍Γ·͢
    
    • લ൒ߨٛͷݴޠ͸GoͰ͢
    

    View Slide

45. ׬
    

    View Slide