wantedly発表会１

Transcript

1. AWS ECS Λ࢖ͬͯ batchΛಈ͔ͦ͏ʂ גࣜձࣾNOVASTO ۙ౻େᠳ
2. ·ͣࣗݾ঺հ • େࡕେֶجૅ޻ֶ෦ࡾճੜ • גࣜձࣾNOVASTOʹjoin • ຊ͕޷͖
3. ͋Δ೔ݴΘΕͨ͜ͱ ʰ֎෦ͷAPI͔Βσʔλʔऔ͖ͬͯͯɻఆظతʹ࣮ߦ ͍͔ͨ͠ΒόονͰɺαʔόʔ͸awsͰυοΧʔͰί ϯςφ্ཱͪ͛Α͏ɻ࠷ۙaws͕γʔΤϧΞΠͰΠʔ γʔΤεઃఆͰ͖ΔΑ͏ʹͳ͔ͬͨΒͦΕ΋࢖ͬͯΈ ͯɻʱ
4. ୯ޠͷҙຯ͕Θ͔Βͳ͍ • όον • υοΧʔ • ΠʔγʔΤε উखʹಈ͘΍ͭ Ծ૝؀ڥʁ ҙຯෆ໌୯ޠ
5. ࠓ೔ͷΰʔϧ 1. ढจΛݴΘΕͨ࣌ͷࣗ෼ʢࢀՃऀʣ͕ཧղͰ͖Δ Α͏ʹઆ໌͢Δ 2. batch࣮ߦํ๏
6. ࠓ೔ͷΰʔϧ 1. ढจΛݴΘΕͨ࣌ͷࣗ෼ʢࢀՃऀʣ͕ཧղͰ͖Δ Α͏ʹઆ໌͢Δ 2. batch࣮ߦํ๏
7. ढจͷղઆ 1. Batch 2. Docker 3. Ecs
8. Batch େྔͷσʔλΛҰׅͰॲཧ͢Δ
9. Batch ޠݯɿbatch - 1ଋɺ1܈ɺ1ஂ ୹࣌ؒͰॲཧΛ͍ͨ͠΋ͷ ʢεϧʔϓοτʣ ͙͢ʹ࣮ߦ͍ͨ͠΋ͷ ʢԠ౴ੑʣ όονॲཧ ΦϯϥΠϯॲཧ
10. -Docker- ௒͍ܰԾ૝ίϯςφٕज़
11. -Docker- ϝϦοτ̍ - ܰྔ ʰDocker࣮ઓ׆༻ΨΠυʯp̓ΑΓ ίϯςφͷ࢓૊ΈʹΑΓɺԾ૝Ϛγϯ΍ήετOS͸ଘࡏ͠ͳ͍ ىಈఀࢭ͕ඇৗʹૣ͍
12. -Docker- ϝϦοτ̎ - ΞϓϦͷ࠶ར༻͕༰қ ࡞੒ͨ͠؀ڥΛimageͱͯ͠ύοέʔδԽՄೳ ଞऀͷimage΋DockerHub͔Βར༻Ͱ͖Δ
13. -Docker- ϝϦοτ̏ - ։ൃ؀ڥͷߋ৽ʹڧ͍ (http://www.itmedia.co.jp/enterprise/articles/1701/23/news036.html)
14. Ecs (Elastic Container Service) Docker Λ AWS্Ͱ௒؆୯ʹ࢖͏ͨΊͷαʔϏε
15. Ecs (Elastic Container Service) ΦʔέετϨʔγϣϯ ίϯςφΛͲ͏ӡ༻͢Δ͔ʁ
16. Ecs (Elastic Container Service) ίϯςφӡ༻શൠΛ೚ͤΔ (https://thinkit.co.jp/article/9701)
17. Ecs (Elastic Container Service) ΦʔέετϨʔγϣϯπʔϧ͕͋Ε͹ɺɺɺ ——શͯઃఆ௨Γʹߦͬͯ͘ΕΔ 1. ϓϩϏδϣχϯά 2. ߏ੒εΫϦϓτ 3. ؂ࢹ 4. …
18. ๯಄ͷґཔΛ຋༁ ʰεΫϨΠϐϯάͯ͠঎඼σʔλऔ͖ͬͯͯɻఆظతʹ࣮ߦ͍͔ͨ͠Βόο νͰɺαʔόʔ͸awsͰυοΧʔͰίϯςφ্ཱͪ͛Α͏ɻ࠷ۙaws ΠʔγʔΤεͰ࣮ߦͷ͕͍͍ͱࢥ͏ɻʱ ఆظతʹ·ͱΊͯσʔλΛॲཧɺ࣮ߦ ίϯςφΛ αʔόʔΛҙࣝͤͣʹ؅ཧ͢ΔαʔϏε γϯϓϧͳԾ૝؀ڥ ҰͭͷԾ૝؀ڥ
19. ࠓ೔ͷΰʔϧ 1. ढจΛݴΘΕͨ࣌ͷࣗ෼ʢࢀՃऀʣ͕ཧղͰ͖Δ Α͏ʹઆ໌͢Δ 2. batch࣮ߦํ๏
20. ίϯςφͷϥΠϑαΠΫϧ Docker Engine Docker Engine ίϯςφ Docker Engine ίϯςφ Docker Engine ίϯςφىಈ ίϯςφফڈ batch࣮ߦ Batch
21. શମͷΠϝʔδ (https://stackoverflow.com/questions/23735149/what-is-the-difference-between-a-docker-image-and-a-container)
22. શମͷΠϝʔδ Docker-Compose.yml Batchfile(~~.rb) DockerFile imageͷอଘ Taskͷىಈ imageͷఏڙ
23. Ecsͷ࣮ߦํ๏
24. DockerίϯςφΛ࡞੒ Docker-file FROM ruby:2.5.1 RUN mkdir -p $APP_DIR WORKDIR $APP_DIR COPY src/Gemfile $APP_DIR RUN gem install bundler & bundle install RUN git clone git@github.com:novasto/~~~ WORKDIR /usr/share/batch/ RUN git checkout $branch Dockerfileʹίϯςφ ʹඞཁͳ৘ใΛهࡌ
25. BatchfileΛ ίϯςφ಺ʹೖΕΔํ๏ Docker-file FROM ruby:2.5.1 RUN mkdir -p $APP_DIR WORKDIR $APP_DIR COPY src/Gemfile $APP_DIR RUN gem install bundler & bundle install RUN git clone git@github.com:novasto/~~~ WORKDIR /usr/share/batch/ RUN git checkout $branch Git cloneͰϦϙδτϦ͔Β batchϓϩάϥϜΛίϯςφ಺ʹ
26. DockerHub΁ϓογϡ VERSION=1.0 MINOR_VERSION=1 # ϩʔΧϧʹίϯςφΠϝʔδͷ࡞੒ docker build -t novasto/leaves.batch:$VERSION.$MINOR_VERSION # VERSIONʹlatestλά௥Ճ docker tag novasto/leaves.batch:$VERSION.$MINOR_VERSION novasto/ leaves.batch:latest # DockerHubʹpush VERSIONࢦఆ docker push novasto/leaves.batch:$VERSION.$MINOR_VERSION
27. Docker-compose.ymlͷ࡞੒ Docker-compose.yml version: '3' services: ruby: image: novasto/batch container_name: scraping_batch tty: true command: ruby /usr/share/batch/app/batch/run.rb env_file: - ./ruby/app.env DockerHub͔Βϓϧ ECSͰ࣮ߦͤ͞ΔίϚϯυͷઃఆ ʢbatch։࢝ʣ
28. λεΫͷొ࿥ # ecs-cli compose --file docker-compose.yml --ecs-params ecs-params.yml --project-name ruby create --launch-type EC2 # ecs-cli compose --file docker-compose.yml --ecs-params ecs-params.yml --project-name ruby create --launch-type version: 1 task_definition: ecs_network_mode: awsvpc task_execution_role: secretsTaskExecutionRole3 task_size: cpu_limit: 256 mem_limit: 512 services: fluentd: essential: true - sg-0d7a8b34dd3cfee84 ecs-params
29. λεΫͷ࣮ߦ ecs-cli compose --file ./$VERSION/docker-compose.yml --task-role-arn secretsTaskExecutionRole3 --ecs-params ./$VERSION/ecs-params.yml --project-name ruby start --launch-type EC2 -c leaves-batch ecs-params
30. OK!!! ecs-params
31. Cloud Watch ecs-params ΞϓϦΛϦΞϧλΠϜͰϞχλϦϯά
32. Cloud WatchΛ༻͍ͯఆظ࣮ߦ ecs-params ݄ ೔ ࣌ؒ 1 11 16 24 2 12 1 24
33. ٧·ͬͨͱ͜Ζ̍ ecs-params ECS ಛ༗ͷ୯ޠ͕ଟ͍ • Ϋϥελʔ • λεΫ • λεΫఆٛ
34. ٧·ͬͨͱ͜Ζ̎ ecs-params ECS͸AWSͷ஌͕ࣝඞཁ • Subnet • VPC • SecurityGroup • I AM Role
35. ײ૝ ecs-params • ·ͩ·ͩػೳͷษڧ͕ඞཁ • ษڧͷίεύ͸ྑͦ͞͏ʢଞͰ΋࢖͏஌ࣝʣ • ͋·Γίʔυ͕Webʹస͕͍ͬͯͳ͍ • ଞͷιϦϡʔγϣϯͱൺ΂ͯͲ͏༏Ε͍ͯΔͷ͔͸ෆ໌
36. ऴΘΓ