Replace a batch application to ECS

E74600a3064ae510f783a9bb8214e8ff?s=47 Yukiyan
September 09, 2016

Replace a batch application to ECS

E74600a3064ae510f783a9bb8214e8ff?s=128

Yukiyan

September 09, 2016
Tweet

Transcript

  1. όονΞϓϦͷӡ༻ج൫Λ&$4΁Ҡߦ͢Δ࿩ 2016/09/09 M3 tech meetup! #3 #m3sushi @yukiyan_w

  2. Who? ʙ4*FS ʙGFFEGPSDF*OD "QQMJDBUJPO&OHJOFFS όονɾ$47ɾ3VCZΛओʹ৮Δɻ Ώ͖΍Μ @yukiyan_w http://yukiyan.hatenablog.jp/

  3. Introduction લஔ͖ ׬શҠߦ͸·ͩͰ͖͍ͯͳ͍

  4. Introduction લஔ͖ %PDLFSJ[F΍&$4Խ͸ɺ݄͘Β͍ʹ࢝Ίͨ

  5. Introduction લஔ͖ ͜͜ϲ݄Ͱ΍ͬͨ͜ͱΛ࿩͠·͢

  6. Introduction લஔ͖ ࣌ؒͷ౎߹্ɺ͍ͩͿ಺༰Λ࡟ͬͨͷͰ ͋ͱͰิ଍هࣄॻ͖·͢

  7. Introduction લఏ஌ࣝ ͲΜͳόονΛ૸Β͍ͤͯΔͷ͔ʁ

  8. Introduction લఏ஌ࣝ $47Λड͚औΓɺม׵ॲཧΛࢪ͢όον

  9. Introduction લఏ஌ࣝ &$4ͱ͸ʁ

  10. Introduction લఏ஌ࣝ ߴੑೳͳίϯςφ؅ཧαʔϏε

  11. Introduction લఏ஌ࣝ &$ΠϯελϯεΛऔΓ·ͱΊͯ؅ཧ্ͨ͠Ͱɺ ͋Β͔͡ΊఆΊͨ௨ΓʹίϯςφΛάϦάϦಈ͔ͨ͢Ίͷ ϚωʔδυαʔϏε

  12. Table Of Contents Agenda ݱঢ় 1 2 ਐΊํ 3 ؾΛ͚ͭͨ͜ͱ

    4 ͭΒ͔ͬͨ͜ͱ 5 ·ͱΊ 6 ໨ࢦ͢΋ͷ
  13. ݱঢ়

  14. ݱঢ় αʔόߏ੒ਤ ˞Ұ෦ׂѪ 3BJMTΞϓϦ %#ແ͠

  15. ݱঢ় εέʔϧ͢ΔͱͭΒ͘ͳΔ ɾDSPOTFSWFS͕410' ɾෛՙ͕ෆۉߧ Ջͳαʔό͕͋Δ  ɾखಈεέʔϧ

  16. ໨ࢦ͢΋ͷ

  17. ໨ࢦ͢΋ͷ ΄ͬͱ͍͓͚ͯΔঢ়ଶ ˞ఆ࣮࣌ߦ͸MBNCEBʹ೚ͤΔ

  18. ໨ࢦ͢΋ͷ ྑ͘ͳΔ͜ͱ ɾδϣϒ͕Ͳ͜Ͱಈ͔͘ؾʹ͠ͳͯ͘Α͘ͳΔ ɾϦιʔε͕༨ͬͯΔΠϯελϯεͰ࣮ߦ͞ΕΔ ɾ଍Γͳ͘ͳͬͨΒΦʔτεέʔϧ

  19. ਐΊํ

  20. ਐΊํ ։ൃମ੍ Engineer Leader Engineer Engineer  !  %PDLFSৄ͍͠

    %PDLFSॳ৺ऀ ೥৽ଔೖࣾ %PDLFSॳ৺ऀ શһ"QQMJDBUJPO&OHJOFFS
  21. %PDLFSॳ৺ऀूஂʜ

  22. ৽ਓ΋͍Δ͚Ͳʜ

  23. ڭҭίετ͸੎͍ͰΧόʔ

  24. ࢿྉ΋๛෋ͩ͠վΊͯڭҭ͢Δඞཁͳ͍Ͱ͠ΐ

  25. ڧؾͷ࢟੎

  26. ڧؾͷ࢟੎

  27. ੎͍ͷ͋Δ৽ਓ IUUQCJUMZD3L:J

  28. ۀ຿࣌ؒ֎ͳͷʹͳ͔ͥଈϨε

  29. ؾ͍ͮͨΒΈΜͳৄ͘͠ͳͬͯͨ

  30. ೔ʹ೔ʹৄ͘͠ͳ͍ͬͯ͘ਤ

  31. ਐΊํ &$4ԽΛͲͷΑ͏ʹਐΊ͍͔ͯͬͨ ։ൃ؀ڥͷDockerԽ ɾDockerize ɾdocker-compose ECS؀ڥͷߏங ɾrun-taskͰδϣϒ࣮ߦ ɾCircleCI੔උ ӡ༻४උ ɾAutoscaling

    ruleߟ͑Δ ɾmackerel ɾͦͷଞʢlambdaͱ͔ʣ ׬શҠߦ ɾঃʑʹҠߦ ↑ ΠϚίίͷ։ൃத ↑ ίίϚσ࿩͢
  32. ਐΊํ։ൃ؀ڥͷ%PDLFSԽ %PDLFSJ[F FROM centos:6.8 FROM fluent/fluentd:v0.12.26-onbuild FROM centos:6.8

  33. ਐΊํ։ൃ؀ڥͷ%PDLFSԽ %PDLFSJ[F %PDLFS*NBHFͷμΠΤοτ͸͋ͱ·Θ͠

  34. ਐΊํ։ൃ؀ڥͷ%PDLFSԽ %PDLFSJ[F ૫ͤΑ͏ͱ͢Δҙࢥ͸͋Δ

  35. ਐΊํ։ൃ؀ڥͷ%PDLFSԽ EPDLFSDPNQPTFΛ࢖͏ ϩʔΧϧͰಈ࡞֬ೝ͕͠΍͍͢

  36. ਐΊํ&$4؀ڥͷߏங &$4Ϋϥελ͸։ൃϝϯόʔ෼༻ҙ ։ൃϝϯόʔ༻ QSPEVDUJPO༻ͭ ಈ࡞֬ೝ͕͠΍͍͢ɻؾܰʹࢼͤΔͷͰཧղ΋ਂ·Δɻ &$4ʹؔ͢Δݕূ͸֤ࣗΑΖ͘͠΍ͬͯ΋Β͏ɻ

  37. ਐΊํ&$4؀ڥͷߏங &$4Ϋϥελ͸։ൃϝϯόʔ෼༻ҙ 6TFSEBUB͸͜Μͳײ͡ lib/tasks/aws/production.rake lib/tasks/aws/sandbox.rake

  38. ਐΊํ&$4؀ڥͷߏங 5BTLEFGJOJUJPOͷߏ੒ CBUDI༻

  39. ਐΊํ&$4؀ڥͷߏங 5BTLEFGJOJUJPOͷߏ੒ GMVFOUE༻

  40. ਐΊํ&$4؀ڥͷߏங 5BTLͷߏ੒ ͖ͬ͞ͷ5BTLEFGJOJUJPO͕͜͏ͳΔ Docker container Docker container Docker container batch༻

    Task fluentd༻ Task
  41. ਐΊํ&$4؀ڥͷߏங &$4্Ͱ͸͜͏ͳΔ GMVFOUE༻5BTL͕4FSWJDFͱͯ͠ৗ࣌Քಇ͍ͯ͠Δ ECS Cluster Internal CLB EC2 ECS Agent

    Task Service EC2 ECS Agent Task Service EC2 ECS Agent EC2 ECS Agent :24224 :24224 "desiredCount": 2, "deploymentConfiguration": { "maximumPercent": 200, "minimumHealthyPercent": 100 } http://bit.ly/2clQnv5
  42. ਐΊํ&$4؀ڥͷߏங &$4Ͱͷߏ੒ SVOUBTL͢ΔͱCBUDI͕૸Δ ECS Cluster Internal CLB EC2 ECS Agent

    Task Service EC2 ECS Agent Task Service EC2 ECS Agent EC2 ECS Agent :24224 :24224 ECS Scheduler batch Task
  43. ਐΊํ&$4؀ڥͷߏங &$4Ͱͷߏ੒ &-#Λ࢖ͬͨ4FSWJDFEJTDPWFSZ ECS Cluster Internal CLB EC2 ECS Agent

    Task Service EC2 ECS Agent Task Service EC2 ECS Agent EC2 ECS Agent :24224 :24224 ECS Scheduler batch Task sandbox-wakayama.vpclocal:24224 "logDriver": "fluentd", "options": { "tag": “docker.batch-app", "fluentd-address": “sandbox-wakayama.vpclocal:24224" }
  44. ਐΊํ&$4؀ڥͷߏங $JSDMF$*ܦ༝ͰͷϗοτσϓϩΠ ϚʔδΛϑοΫʹίϯςφ͕σϓϩΠ͞ΕΔ deploy aws ecs update-service Serverspec bundle exec

    rake serverspec RSpec bundle exec rake spec ! git merge topic branchΛϚʔδ CircleCI workflow hot deploy via CircleCI " ECR docker push
  45. ؾΛ͚ͭͨ͜ͱ

  46. ؾΛ͚ͭͨ͜ͱ %PDLFSGJMF͸Ͱ͖Δ͚ͩಡΈ΍͘͢͢Δ ൿ఻ͷλϨԽΛ๷͙ɻ

  47. lazͰฒ΂Δ΍ͭ গ͠Ͱ΋ಡΈ΍͘͢ͳΔΑ͏ʹɻ

  48. ґଘੑͷ͋ΔQBDLBHF͸&/7ʹࢦఆͯ͠͠·͏ ʮ͜ΕͬͯͳΜͷͨΊʹೖΕͯΜͷʁʯͷ๷ࢭ

  49. ؾΛ͚ͭͨ͜ͱ ࢖͍͍ͨϩάυϥΠό͸6TFSEBUBʹॻ͔ͳ͍ͱ࢖͑ͳ͍ &$4"HFOUͷઃఆ

  50. σϑΥϧτ͸[“json-file"]ͷΈɻ໘౗ͳͷͰશ෦ॻ͍ͨɻ https://docs.aws.amazon.com/ja_jp/AmazonECS/latest/developerguide/ecs-agent-config.html

  51. ͭΒ͔ͬͨ͜ͱ

  52. ஌ݟ ͭΒ͔ͬͨ͜ͱ ίϯςφͷ؀ڥม਺ͷ੾Γସ͑

  53. EPDLFSGJMFTGMVFOUEFYQPSU@FOWGJMFTI EPDLFSGJMFTGMVFOUE%PDLFSGJMF &OUSZLJUͷTXJUDIͰؤுΔελΠϧ ൿಗ஋ΛؚΜͰ͍ΔͷͰ4͔Βऔ͖͍ͬͯͯΔ 44&ࡁΈ

  54. ஌ݟ ͭΒ͔ͬͨ͜ͱ $JSDMF$*ͰΩϟογϡ͕ޮ͔ͳ͍ dockerfileͷfull build͕ຖճ૸Δɻ load/saveํࣜͰͰ͖Δͱॻ͍ͯ͋Δ͕ɺͦΕͰ΋Ωϟογϡ͞Εͳ͍ɻ ɾhttps://circleci.com/docs/docker/#caching-docker-layers

  55. ஌ݟ ͭΒ͔ͬͨ͜ͱ "-#࢖͑ͳ͔ͬͨ Service discoveryʹCLBΛ࢖ͬͯΔ͕ALBʹ͍ͨ͠ɻ ͔͠͠ɺCLB͸fluentd༻ʹ24224 (TCP)ΛListen͍ͯ͠ΔͷͰ ALBͷTCPରԠ଴ͪɻ

  56. ·ͱΊ

  57. ·ͱΊ Θ͔ͬͨ͜ͱ ɾDockerfile͸ಡΈ΍͘͢ॻ͍ͨ΄͏͕͍͍ɻ ɾόονͳΒECSԽ͸ͦΜͳʹϋϚΒͣεοͱͰ͖Δɻ ɾECSԽʹ൐͏Dockerize΍Autoscalingͷ෭࣍ޮՌ͕େ͖͍ɻ

  58. ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ ɾDockerfile͸ಡΈ΍͘͢ॻ͍ͨ΄͏͕͍͍ɻ ɾόονͳΒECSԽ͸ͦΜͳʹϋϚΒͣεοͱͰ͖Δɻ ɾECSԽʹ൐͏Dockerize΍Autoscalingͷ෭࣍ޮՌ͕େ͖͍ɻ 5)"/,:06 -FU`TUSZ&$4