Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
無限にスケールする上に自律的なJenkinsに見る夢~AWS篇~ / Dream of aut...
Search
moznion
August 18, 2018
Technology
7.4k
21
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
無限にスケールする上に自律的なJenkinsに見る夢~AWS篇~ / Dream of autonomous and infinite scalable Jenkins with AWS
Kyotoなんか #4の資料です
moznion
August 18, 2018
Transcript
56 slides
ແݶʹεέʔϧ͢Δ্ʹ ࣗతͳJenkinsʹݟΔເ ~AWSἫ~ KyotoͳΜ͔ #4 @moznion
@moznion Software engineer Hatena Intern 2013
Jenkins
Jenkins - CI (Continuous Integration) Platform - Workflow - ศརͳcron
Why Jenkins?
Why Jenkins? - ͱͱ (Ұ෦ࠓ) CircleCIΛ͍ͬͯͨ - Job͕૿͑ΔʹͭΕ٧·Δqueue - ϓϥϯ্͛Δͱߴ͍ΜεΑ……
- ͬͱϚονϣͳϚγϯͰϏϧυ͍ͨ͠ͱ͍͏ཉ - ͦΖͦΖͪΌΜͱͨ͠σϓϩΠϑϩʔͱ͔Λ͍͑ͨ -
Why Jenkins? - ͱͱ (Ұ෦ࠓ) CircleCIΛ͍ͬͯͨ - Job͕૿͑ΔʹͭΕ٧·Δqueue - ϓϥϯ্͛Δͱߴ͍ΜεΑ……
- ͬͱϚονϣͳϚγϯͰϏϧυ͍ͨ͠ͱ͍͏ཉ - ͦΖͦΖͪΌΜͱͨ͠σϓϩΠϑϩʔͱ͔Λ͍͑ͨ - ͱͱJenkinsΛεέʔϧͤ͞Δํ๏Λ͍ͬͯͨ
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢ - ฒྻ࣮ߦJobͷ૿Ճʹ͏ੑೳྼԽ - ߜΔͱbuildͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ - Master
InstanceࢮͶͳ͍ - ͓ئ͍͠·͢ͷڪා (™ @kyanny)
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢ - ฒྻ࣮ߦJobͷ૿Ճʹ͏ੑೳྼԽ - ߜΔͱbuildͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ - Master
InstanceࢮͶͳ͍ - ͓ئ͍͠·͢ͷڪා (™ @kyanny)
ฒྻjob্͛ΔͷϚδَʂ
ฒྻjob্͛ΔͷϚδَ SlaveΛ͢ͷ؆୯ʂʂʂʂ
ฒྻjob্͛ΔͷϚδَ - slave͢ͷ؆୯ͳΜ͚ͩͲ…… - ୯७ʹཧίετ͕૿͑Δͧʂ - ʮ͜ͷslaveͩͱͳ͔ͥςετ͕ίέ·͢ʯ
ฒྻjob্͛ΔͷϚδَ - slave͢ͷ؆୯ͳΜ͚ͩͲ…… - ΦʔτεέʔϦϯάΛͪ·ͪ·͍ͬͯ͘ඞཁ͕͋Γ…… - ΠϯελϯεΛ༡͓ͤͯ࣌ؒ͘ແ͍ - CIΑ΄ͲϓϩδΣΫτཱ͕ͯࠐΜͰͳ͍ݶΓ৸ͯΔ͕࣌ؒ ࢧత
- ׂͱઃఆ͕͍͠ (ίπ͕͍Δ)
͏͏……
զʑ΄ͬͱ͍ͯແݶʹϏϧυ͕εέʔϧ͢Δ Jenkins͕΄͍͚ͩ͠ͳΜ͡Ό……
ͦΜͳͷ…… ͋Δʂʂʂ
ͦΜͳͷ…… ͋Δʂʂʂ
AWS CodeBuild
AWS CodeBuild - ୯Ґͷ࣌ؒି͠ - ҙͷίϯςφΛ࣮ߦͰ͖Δ - ͦͷίϯςφ্ͰϏϧυΒςετΒ͢Δ - (LambdaΑΓ͕͔͔࣌ؒΔλεΫͷ࣮ߦڥʹ……)
AWS CodeBuild - Jenkins Plugin͕͋Δ (AWSۘ) - https://github.com/awslabs/aws-codebuild-jenkins-plugin - JenkinsͰड͚ͨτϦΨͰCodeBuildΛൃՐͰ͖Δ
https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/
https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/
https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/ ࣮࣭ແݶʂʂʂʂ
ͨΒ͞ΕͨԸܙ - slave nodeͷϝϯςΒͣ - ΨϯΨϯฒྻϏϧυ͢Δͧʂʂʂ - ࣌ؒି͠ (= ͚ͬͨͩ)
ͷྉۚઃఆͳͷͰ ࣗͰΦʔτεέʔϧॲཧΛ͠ͳͯ͘Α͍
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢ - ฒྻ࣮ߦJobͷ૿Ճʹ͏ੑೳྼԽ - ߜΔͱbuildͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ - Master
InstanceࢮͶͳ͍ - ͓ئ͍͠·͢ͷڪා (™ @kyanny)
Master Instance͕ਆ֨Խ͢Δ
Master Instanceਆ֨Խ - Jenkinsͷmaster datamasterϚγϯͷ*ϩʔΧϧετϨʔδ*ʹ อଘ͞ΕΔ - Πϯελϯε͕ࢮΜͩΒऴΘΓ - ͔͠͠ࢮආ͚ΒΕͳ͍
- ͔ͩΒόοΫΞοϓΛऔͬͨΓ͢Δ - e.g. https://papix.hatenablog.com/entry/2016/02/09/101918 - όοΫΞοϓ͔ΒͷϦετΞ࣮ࡍ໘ - ָ͍͠ࣄͰͳ͍…… - μϯλΠϜग़Δ͠……
Master Instanceਆ֨Խ - ݁Ռͱͯ͠ຖଉࡂΛفΔʑ…… - ӡѱ͘Πϯελϯε͕ࢮΜͩΒσΠϦʔόοΫΞοϓ ͔ΒϦετΞ - σΠϦʔόοΫΞοϓͱࢮ͵ؒࡍͱͷࠩʹ͍ͭͯ ఘΊΔ͔͠ແ͍……
͏͏……
զʑৎͰεέʔϧ͢Δ Jenkins͕΄͍͚ͩ͠ͳΜ͡Ό……
ͦΜͳͷ…… ͋Δʂʂʂ
ͦΜͳͷ…… ͋Δʂʂʂ
Amazon Elastic File System
AWS Elastic File System (EFS) - AmazonͷnfsΈ͍ͨͳͭ - Instanceʹmountͯ͠͏ -
EBS + nfsతͳ…… - ϘϦϡʔϜαΠζ͕ࣗಈͰ֦ுɾॖখ͢Δ - Disk fullΒͣͷੜ׆
AWS Elastic File System (EFS) - $JENKINS_HOMEΛEFSϘϦϡʔϜ্ʹ͢Δ - Πϯελϯε͕ࢮΜͰ৽͍͠Πϯελϯεʹ ͦͷϘϦϡʔϜΛϚϯτ͢Δ
- => ΠϯελϯεલੈͷهԱΛҾ͖ܧ͗෮׆ʂʂʂ
mount Working...
❌mount Instance down...
❌ New instance coming up...
❌ mount Works fine!!
AWS Elastic File System (EFS) - EBSͰྑ͍ͷͰʁ - ͍ʂɹEBSͷ΄͏͕͍҆͠ʂʂ -
ͨͩEFSϘϦϡʔϜࣗಈ֦ு͕خ͍͠ - (͏͔ͬΓdisk fullʹͳΒͳ͍) - ͋ͱEFSͩͱmulti masterΈ͍ͨͳ͜ͱͰ͖Δ (ޙड़)
Jenkins Multi Master - ଟͰ͖Δ (࣌ؒӡ༻ͯ͠ͳ͍) - ͔͠͠CodeBuildͱڞଘ͍ͯ͠ΔڥͰMulti Masterʹ͢Δ ϝϦοτ΄΅ͳ͘ͳ͍Ͱ͔͢ʁ
- σϓϩΠ࣌ʹҰॠ2masterΛཱ͓͍ͯͯͯɼݹ͍ํΛ ࡴ͢Έ͍ͨͳμϯλΠϜσϓϩΠ͚ͷ͍ํՄೳ
ͨΒ͞ΕͨԸܙ - ΧδϡΞϧʹΠϯελϯε͕ࢮΜͰ҆৺ʂ - ͱ͍͑όοΫΞοϓऔͬͯΔ - ݁Ռͱͯ͠ΠϯελϯεͷೖΕସָ͕͑ʹͳͬͨ - ετϨʔδࣗಈ֦ு͢ΔͷͰDisk fullΒͣ
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢ - ฒྻ࣮ߦJobͷ૿Ճʹ͏ੑೳྼԽ - ߜΔͱbuildͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ - Master
InstanceࢮͶͳ͍ - ͓ئ͍͠·͢ͷڪා (™ @kyanny)
ࣾձ: Jenkins͓͡͞ΜͷͳΓखෆ
Jenkinsͷӡ༻Λ୭͔ʹԡ͚͠ΔͷΛΊΔ - ୭Ͱӡ༻Ͱ͖ΔΑ͏ʹ͢Δ - શਓຽJenkins͓͡͞Μ/͓͞ΜԽ - σϓϩΠϝϯςۃྗলྗԽ͢Δ
Jenkinsͷӡ༻Λ୭͔ʹԡ͚͠ΔͷΛΊΔ - Amazon Elastic Beanstalk + Docker - Dockerfile +
Dockerrun.aws.jsonʹΑΔߏͷίʔυԽ - .ebextensionsʹΑΔࣗಈԽ - EFS mountͳͲ - https://github.com/moznion/jenkins_eb_efs_codebuild_example
- Elastic Beanstalkͷmanaged updateͰࣗಈతʹύον͕ͨΔ - ͍ͭͰʹΠϯελϯεͷೖΕସ͑ߦΘΕΔ - EFSڥԼͳͷͰΠϯελϯεೖΕସָ͑ʑࣗಈԽ - جຊతʹӡ༻์ஔͯ͠·͢
- EFSͷϘϦϡʔϜࣗಈ֦ுͷύϫʔͰ͋Δ ͨΒ͞ΕͨԸܙ
·ͱΊ
·ͱΊ - JenkinsͷslaveϊʔυΛAWS CodeBuildʹҕৡ͢Δ͜ͱͰ࣮࣭ ແݶʹฒྻϏϧυ͕εέʔϧ͢ΔΑ͏ʹͳΓ·ͨ͠ - JenkinsͷετϨʔδΛEFSʹ͢Δ͜ͱͰϚελʔσʔλͷ ϩόετԽΛߦ͍·ͨ͠ - Elastic
BeanstalkΛ༻͍ͯJenkinsΛӡ༻͢Δ͜ͱͰӡ༻ͷ ίετԽΛ͠·ͨ͠ - Α͔ͬͨͰ͢Ͷ
Q?
Wait...
͜ͷJenkinsʹ͕͋Δ - EFS͍ʂʂʂʂʂ - Jenkins͕git clone͢ΔͱϚδ͔͔࣌ؒΓ·͢ - git cloneΛճආ͢Δํ๏͍·ͷͱ͜Ζແ͍…… -
shallow cloneΛ༗ޮʹ͏ - sparse checkoutΛ༗ޮʹ͏ - ͳͲͷ͕ඞཁ - ͦͦCodeBuild͕git clone͢ΔΜͩ͠jenkins͠ͳͯ͘ ྑ͍Μ͚ͩͲ…… - ͋Δ͍EBSΛ͏……
ຊʹJenkinsඞཁʁ - ͭ͡AWS CodeBuild࠷ۙΊͬͪΌͰ͖͕Αͯ͘ɼ ୯ମͰे͑Δ - GitHubͷhookड͚ΕΔ͠ - Commit statusΛม͑Δػೳ·Ͱ͍͍ͭͯΔ
- ϏϧυཤྺҰཡͰ͖Δ͠ - CloudWatch LogsʹϏϧυͷϩάग़Δ - JenkinsෆཁͰʁʁʁʁɹ͜ΕͰेͰʁʁʁʁ - ͍
ຊʹJenkinsඞཁʁ - ࣗ༝ͳWebhookΛड͚Δඞཁ͕͋Δͱ͔…… - ͋ͱ௨ͱ͔Ͷʂ - ͋Δ͍ͬͱଞʹϏϧυύΠϓϥΠϯΛܨ ͛Δඞཁ͕͋Δͱ͔…… - ͦ͏͍͏ͱ͖ΜΓͰ͢
- (͋ͱ·͋Ұཡੑ͕Α͍) (ॾઆ͋Γ·͢)
Q?
More Decks by moznion
See All by moznion
go-sqletch
moznion
0
160
ブラウザアプリの継続的パフォーマンスモニタリング (序) / Continuous Browser Application Performance Monitoring; Act 1
moznion
0
150
reFACToring
moznion
1
1.4k
cccccc
moznion
1
3.1k
履歴テーブル、今回はこう作りました 〜 Delegated Types編 〜 / How We Built Our History Table This Time — With Delegated Types
moznion
16
16k
「データ無い! 腹立つ! 推論する!」から 「データ無い! 腹立つ! データを作る」へ チームでデータを作り、育てられるようにするまで / How can we create, use, and maintain data ourselves?
moznion
11
7.9k
避けられないI/O待ちに対処する: Rails アプリにおけるSSEとasync gemの活用 / Tackling Inevitable I/O Latency in Rails Apps with SSE and the async gem
moznion
4
10k
RubyKaigi Hack Space in Tokyo & 函館最速 "予習" 会 / RubyKaigi Hack Space in Tokyo & The Fastest Briefing of RubyKaigi 2026 in Hakodate
moznion
1
500
地に足の付いた現実的な技術選定から魔力のある体験を得る『AIレシート読み取り機能』のケーススタディ / From Grounded Tech Choices to Magical UX: A Case Study of AI Receipt Scanning
moznion
7
5.3k
Other Decks in Technology
See All in Technology
Skill とカスタムエージェント (手順と役割を渡す) | 目指せ Kiro マスター 08
inariku
PRO
2
110
Context parameters で作る!はじめての型クラス
kworkdev
PRO
1
130
文書画像からの情報抽出を行うエージェントを、軌跡の正解データ無しで作れるか
sansantech
PRO
2
160
ラクスル会社紹介資料|エンジニア向け / RAKSUL Introduction for Engineers
raksulrecruiting
0
120
Shared Signals Framework 仕様解説
oidfj
0
460
20261113_JAWS_Osaka
takuyay0ne
0
570
異夢同船〜異なる夢でも同じ船でなんとかする〜
fortegp05
1
450
農家は秘伝のタレを知らない - NOBODY KNOWS THE HIDDEN SOURCE - Kaggriculture 3rd Place Solution
masspeaks
2
480
Where Should Platform Complexity Live? Lessons from a Cross-Team PR Preview Platform
taiki45
0
120
Context != Context: One window, many lifecycles (AI Native DevCon, Nov 2026)
birgitta410
0
160
Claude Codeで新規事業を伸ばせるか? Zennの試行錯誤の記録
wadayusuke
2
220
さようならSSH鍵、こんにちはゼロトラスト ― AIフレンドリーで快適な自宅鯖SSH
atalie2m
14
6.5k
Featured
See All Featured
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
12
1.4k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
35
3.9k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
25
2.2k
Prompt Engineering for Job Search
mfonobong
0
590
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
28
3.8k
Believing is Seeing
oripsolob
1
280
Lightning talk: Run Django tests with GitHub Actions
sabderemane
0
320
Fireside Chat
paigeccino
43
4.1k
Practical Orchestrator
shlominoach
192
12k
Git: the NoSQL Database
bkeepers
PRO
433
67k
How People are Using Generative and Agentic AI to Supercharge Their Products, Projects, Services and Value Streams Today
helenjbeal
2
400
State of Search Keynote: SEO is Dead Long Live SEO
ryanjones
0
350