Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
wantedly発表会1
Search
Sponsored
·
Your Podcast. Everywhere. Effortlessly.
Share. Educate. Inspire. Entertain. You do you. We'll handle the rest.
→
tekonfo
November 24, 2018
130
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
wantedly発表会1
tekonfo
November 24, 2018
Transcript
36 slides
AWS ECS Λͬͯ batchΛಈ͔ͦ͏ʂ גࣜձࣾNOVASTO ۙ౻େᠳ
·ͣࣗݾհ • େࡕେֶجૅֶ෦ࡾճੜ • גࣜձࣾNOVASTOʹjoin • ຊ͕͖
͋ΔݴΘΕͨ͜ͱ ʰ֎෦ͷAPI͔Βσʔλʔऔ͖ͬͯͯɻఆظతʹ࣮ߦ ͍͔ͨ͠ΒόονͰɺαʔόʔawsͰυοΧʔͰί ϯςφ্ཱͪ͛Α͏ɻ࠷ۙaws͕γʔΤϧΞΠͰΠʔ γʔΤεઃఆͰ͖ΔΑ͏ʹͳ͔ͬͨΒͦΕͬͯΈ ͯɻʱ
୯ޠͷҙຯ͕Θ͔Βͳ͍ • όον • υοΧʔ • ΠʔγʔΤε উखʹಈͭ͘ Ծڥʁ ҙຯෆ໌୯ޠ
ࠓͷΰʔϧ 1. ढจΛݴΘΕͨ࣌ͷࣗʢࢀՃऀʣ͕ཧղͰ͖Δ Α͏ʹઆ໌͢Δ 2. batch࣮ߦํ๏
ࠓͷΰʔϧ 1. ढจΛݴΘΕͨ࣌ͷࣗʢࢀՃऀʣ͕ཧղͰ͖Δ Α͏ʹઆ໌͢Δ 2. batch࣮ߦํ๏
ढจͷղઆ 1. Batch 2. Docker 3. Ecs
Batch େྔͷσʔλΛҰׅͰॲཧ͢Δ
Batch ޠݯɿbatch - 1ଋɺ1܈ɺ1ஂ ࣌ؒͰॲཧΛ͍ͨ͠ͷ ʢεϧʔϓοτʣ ͙͢ʹ࣮ߦ͍ͨ͠ͷ ʢԠੑʣ όονॲཧ ΦϯϥΠϯॲཧ
-Docker- ͍ܰԾίϯςφٕज़
-Docker- ϝϦοτ̍ - ܰྔ ʰDocker࣮ઓ׆༻ΨΠυʯp̓ΑΓ ίϯςφͷΈʹΑΓɺԾϚγϯήετOSଘࡏ͠ͳ͍ ىಈఀࢭ͕ඇৗʹૣ͍
-Docker- ϝϦοτ̎ - ΞϓϦͷ࠶ར༻͕༰қ ࡞ͨ͠ڥΛimageͱͯ͠ύοέʔδԽՄೳ ଞऀͷimageDockerHub͔Βར༻Ͱ͖Δ
-Docker- ϝϦοτ̏ - ։ൃڥͷߋ৽ʹڧ͍ (http://www.itmedia.co.jp/enterprise/articles/1701/23/news036.html)
Ecs (Elastic Container Service) Docker Λ AWS্Ͱ؆୯ʹ͏ͨΊͷαʔϏε
Ecs (Elastic Container Service) ΦʔέετϨʔγϣϯ ίϯςφΛͲ͏ӡ༻͢Δ͔ʁ
Ecs (Elastic Container Service) ίϯςφӡ༻શൠΛͤΔ (https://thinkit.co.jp/article/9701)
Ecs (Elastic Container Service) ΦʔέετϨʔγϣϯπʔϧ͕͋Εɺɺɺ ——શͯઃఆ௨Γʹߦͬͯ͘ΕΔ 1. ϓϩϏδϣχϯά 2. ߏεΫϦϓτ
3. ࢹ 4. …
಄ͷґཔΛ༁ ʰεΫϨΠϐϯάͯ͠σʔλऔ͖ͬͯͯɻఆظతʹ࣮ߦ͍͔ͨ͠Βόο νͰɺαʔόʔawsͰυοΧʔͰίϯςφ্ཱͪ͛Α͏ɻ࠷ۙaws ΠʔγʔΤεͰ࣮ߦͷ͕͍͍ͱࢥ͏ɻʱ ఆظతʹ·ͱΊͯσʔλΛॲཧɺ࣮ߦ ίϯςφΛ αʔόʔΛҙࣝͤͣʹཧ͢ΔαʔϏε γϯϓϧͳԾڥ ҰͭͷԾڥ
ࠓͷΰʔϧ 1. ढจΛݴΘΕͨ࣌ͷࣗʢࢀՃऀʣ͕ཧղͰ͖Δ Α͏ʹઆ໌͢Δ 2. batch࣮ߦํ๏
ίϯςφͷϥΠϑαΠΫϧ Docker Engine Docker Engine ίϯςφ Docker Engine ίϯςφ Docker
Engine ίϯςφىಈ ίϯςφফڈ batch࣮ߦ Batch
શମͷΠϝʔδ (https://stackoverflow.com/questions/23735149/what-is-the-difference-between-a-docker-image-and-a-container)
શମͷΠϝʔδ Docker-Compose.yml Batchfile(~~.rb) DockerFile imageͷอଘ Taskͷىಈ imageͷఏڙ
Ecsͷ࣮ߦํ๏
DockerίϯςφΛ࡞ Docker-file FROM ruby:2.5.1 RUN mkdir -p $APP_DIR WORKDIR $APP_DIR
COPY src/Gemfile $APP_DIR RUN gem install bundler & bundle install RUN git clone
[email protected]
:novasto/~~~ WORKDIR /usr/share/batch/ RUN git checkout $branch Dockerfileʹίϯςφ ʹඞཁͳใΛهࡌ
BatchfileΛ ίϯςφʹೖΕΔํ๏ Docker-file FROM ruby:2.5.1 RUN mkdir -p $APP_DIR WORKDIR
$APP_DIR COPY src/Gemfile $APP_DIR RUN gem install bundler & bundle install RUN git clone
[email protected]
:novasto/~~~ WORKDIR /usr/share/batch/ RUN git checkout $branch Git cloneͰϦϙδτϦ͔Β batchϓϩάϥϜΛίϯςφʹ
DockerHubϓογϡ VERSION=1.0 MINOR_VERSION=1 # ϩʔΧϧʹίϯςφΠϝʔδͷ࡞ docker build -t novasto/leaves.batch:$VERSION.$MINOR_VERSION #
VERSIONʹlatestλάՃ docker tag novasto/leaves.batch:$VERSION.$MINOR_VERSION novasto/ leaves.batch:latest # DockerHubʹpush VERSIONࢦఆ docker push novasto/leaves.batch:$VERSION.$MINOR_VERSION
Docker-compose.ymlͷ࡞ Docker-compose.yml version: '3' services: ruby: image: novasto/batch container_name: scraping_batch
tty: true command: ruby /usr/share/batch/app/batch/run.rb env_file: - ./ruby/app.env DockerHub͔Βϓϧ ECSͰ࣮ߦͤ͞ΔίϚϯυͷઃఆ ʢbatch։࢝ʣ
λεΫͷొ # ecs-cli compose --file docker-compose.yml --ecs-params ecs-params.yml --project-name ruby
create --launch-type EC2 # ecs-cli compose --file docker-compose.yml --ecs-params ecs-params.yml --project-name ruby create --launch-type version: 1 task_definition: ecs_network_mode: awsvpc task_execution_role: secretsTaskExecutionRole3 task_size: cpu_limit: 256 mem_limit: 512 services: fluentd: essential: true - sg-0d7a8b34dd3cfee84 ecs-params
λεΫͷ࣮ߦ ecs-cli compose --file ./$VERSION/docker-compose.yml --task-role-arn secretsTaskExecutionRole3 --ecs-params ./$VERSION/ecs-params.yml --project-name
ruby start --launch-type EC2 -c leaves-batch ecs-params
OK!!! ecs-params
Cloud Watch ecs-params ΞϓϦΛϦΞϧλΠϜͰϞχλϦϯά
Cloud WatchΛ༻͍ͯఆظ࣮ߦ ecs-params ݄ ࣌ؒ 1 11 16 24
2 12 1 24
٧·ͬͨͱ͜Ζ̍ ecs-params ECS ಛ༗ͷ୯ޠ͕ଟ͍ • Ϋϥελʔ • λεΫ • λεΫఆٛ
٧·ͬͨͱ͜Ζ̎ ecs-params ECSAWSͷ͕ࣝඞཁ • Subnet • VPC • SecurityGroup •
I AM Role
ײ ecs-params • ·ͩ·ͩػೳͷษڧ͕ඞཁ • ษڧͷίεύྑͦ͞͏ʢଞͰ͏ࣝʣ • ͋·Γίʔυ͕Webʹస͕͍ͬͯͳ͍ • ଞͷιϦϡʔγϣϯͱൺͯͲ͏༏Ε͍ͯΔͷ͔ෆ໌
ऴΘΓ
Featured
See All Featured
The Spectacular Lies of Maps
axbom
PRO
1
1.1k
A Tale of Four Properties
chriscoyier
163
24k
The World Runs on Bad Software
bkeepers
PRO
72
12k
Unlocking the hidden potential of vector embeddings in international SEO
frankvandijk
0
1k
Building a Scalable Design System with Sketch
lauravandoore
464
34k
Java REST API Framework Comparison - PWX 2021
mraible
34
9.8k
RailsConf 2023
tenderlove
30
1.6k
Designing Powerful Visuals for Engaging Learning
tmiket
1
660
Tips & Tricks on How to Get Your First Job In Tech
honzajavorek
1
850
Claude Code どこまでも/ Claude Code Everywhere
nwiizo
68
60k
Bootstrapping a Software Product
garrettdimon
PRO
306
120k
Max Prin - Stacking Signals: How International SEO Comes Together (And Falls Apart)
techseoconnect
PRO
0
530