Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
無限にスケールする上に自律的なJenkinsに見る夢~AWS篇~ / Dream of aut...
Search
moznion
August 18, 2018
Technology
21
7.1k
無限にスケールする上に自律的なJenkinsに見る夢~AWS篇~ / Dream of autonomous and infinite scalable Jenkins with AWS
Kyotoなんか #4の資料です
moznion
August 18, 2018
Tweet
Share
More Decks by moznion
See All by moznion
Simple組み合わせ村から大都会Railsにやってきた俺は / Coming to Rails from the Simple
moznion
3
2.1k
AIレシート読み取り機能をRuby on Rails on AWSで実現するLLMにまつわるアレコレ / AI-based receipt reading function powered by LLM on Ruby on Rails on AWS
moznion
3
130
Develop to Survive - YAPC::Hakodate 2024 Keynote
moznion
10
15k
これはPerl? それともRuby? クイズ〜〜〜〜〜!!!- Perl or Ruby Quiz
moznion
3
2.5k
Perl 5 OOP機構30年史 - Perl 5's OOP Mechanism over the past 30 years
moznion
1
960
RuboCopのカスタムCopを書いてContributionしてみる - Contributing a Custom Cop to RuboCop: A Hands-on Experience
moznion
0
73
AWS上に構築する メンテ容易なElasticsearch System / Maintainable Elasticsearch system on AWS
moznion
2
3.9k
PROXY Protocol aware Proxy Server on Node.js
moznion
2
2.5k
Perl meets AWS Lambda
moznion
0
4.7k
Other Decks in Technology
See All in Technology
AWSの生成AIサービス Amazon Bedrock入門!(2025年1月版)
minorun365
PRO
7
460
.NET 最新アップデート ~ AI とクラウド時代のアプリモダナイゼーション
chack411
0
200
Reactフレームワークプロダクトを モバイルアプリにして、もっと便利に。 ユーザに価値を届けよう。/React Framework with Capacitor
rdlabo
0
110
デジタルアイデンティティ人材育成推進ワーキンググループ 翻訳サブワーキンググループ 活動報告 / 20250114-OIDF-J-EduWG-TranslationSWG
oidfj
0
460
Copilotの力を実感!3ヶ月間の生成AI研修の試行錯誤&成功事例をご紹介。果たして得たものとは・・?
ktc_shiori
0
340
深層学習と3Dキャプチャ・3Dモデル生成(土木学会応用力学委員会 応用数理・AIセミナー)
pfn
PRO
0
450
re:Invent 2024のふりかえり
beli68
0
110
シフトライトなテスト活動を適切に行うことで、無理な開発をせず、過剰にテストせず、顧客をビックリさせないプロダクトを作り上げているお話 #RSGT2025 / Shift Right
nihonbuson
3
2.1k
SpiderPlus & Co. エンジニア向け会社紹介資料
spiderplus_cb
0
860
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
6
54k
The future we create with our own MVV
matsukurou
0
2k
Cloudflareで実現する AIエージェント ワークフロー基盤
kmd09
0
280
Featured
See All Featured
Fashionably flexible responsive web design (full day workshop)
malarkey
406
66k
Designing on Purpose - Digital PM Summit 2013
jponch
116
7.1k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
33
2.7k
YesSQL, Process and Tooling at Scale
rocio
170
14k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
365
25k
Writing Fast Ruby
sferik
628
61k
Adopting Sorbet at Scale
ufuk
74
9.2k
Building Better People: How to give real-time feedback that sticks.
wjessup
366
19k
Designing for Performance
lara
604
68k
Fantastic passwords and where to find them - at NoRuKo
philnash
50
2.9k
A better future with KSS
kneath
238
17k
Transcript
ແݶʹεέʔϧ͢Δ্ʹ ࣗతͳJenkinsʹݟΔເ ~AWSἫ~ KyotoͳΜ͔ #4 @moznion
@moznion Software engineer Hatena Intern 2013
Jenkins
Jenkins - CI (Continuous Integration) Platform - Workflow - ศརͳcron
Why Jenkins?
Why Jenkins? - ͱͱ (Ұ෦ࠓ) CircleCIΛ͍ͬͯͨ - Job͕૿͑ΔʹͭΕ٧·Δqueue - ϓϥϯ্͛Δͱߴ͍ΜεΑ……
- ͬͱϚονϣͳϚγϯͰϏϧυ͍ͨ͠ͱ͍͏ཉ - ͦΖͦΖͪΌΜͱͨ͠σϓϩΠϑϩʔͱ͔Λ͍͑ͨ -
Why Jenkins? - ͱͱ (Ұ෦ࠓ) CircleCIΛ͍ͬͯͨ - Job͕૿͑ΔʹͭΕ٧·Δqueue - ϓϥϯ্͛Δͱߴ͍ΜεΑ……
- ͬͱϚονϣͳϚγϯͰϏϧυ͍ͨ͠ͱ͍͏ཉ - ͦΖͦΖͪΌΜͱͨ͠σϓϩΠϑϩʔͱ͔Λ͍͑ͨ - ͱͱJenkinsΛεέʔϧͤ͞Δํ๏Λ͍ͬͯͨ
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢ - ฒྻ࣮ߦJobͷ૿Ճʹ͏ੑೳྼԽ - ߜΔͱbuildͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ - Master
InstanceࢮͶͳ͍ - ͓ئ͍͠·͢ͷڪා (™ @kyanny)
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢ - ฒྻ࣮ߦJobͷ૿Ճʹ͏ੑೳྼԽ - ߜΔͱbuildͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ - Master
InstanceࢮͶͳ͍ - ͓ئ͍͠·͢ͷڪා (™ @kyanny)
ฒྻjob্͛ΔͷϚδَʂ
ฒྻjob্͛ΔͷϚδَ SlaveΛ͢ͷ؆୯ʂʂʂʂ
ฒྻjob্͛ΔͷϚδَ - slave͢ͷ؆୯ͳΜ͚ͩͲ…… - ୯७ʹཧίετ͕૿͑Δͧʂ - ʮ͜ͷslaveͩͱͳ͔ͥςετ͕ίέ·͢ʯ
ฒྻjob্͛ΔͷϚδَ - slave͢ͷ؆୯ͳΜ͚ͩͲ…… - ΦʔτεέʔϦϯάΛͪ·ͪ·͍ͬͯ͘ඞཁ͕͋Γ…… - ΠϯελϯεΛ༡͓ͤͯ࣌ؒ͘ແ͍ - CIΑ΄ͲϓϩδΣΫτཱ͕ͯࠐΜͰͳ͍ݶΓ৸ͯΔ͕࣌ؒ ࢧత
- ׂͱઃఆ͕͍͠ (ίπ͕͍Δ)
͏͏……
զʑ΄ͬͱ͍ͯແݶʹϏϧυ͕εέʔϧ͢Δ Jenkins͕΄͍͚ͩ͠ͳΜ͡Ό……
ͦΜͳͷ…… ͋Δʂʂʂ
ͦΜͳͷ…… ͋Δʂʂʂ
AWS CodeBuild
AWS CodeBuild - ୯Ґͷ࣌ؒି͠ - ҙͷίϯςφΛ࣮ߦͰ͖Δ - ͦͷίϯςφ্ͰϏϧυΒςετΒ͢Δ - (LambdaΑΓ͕͔͔࣌ؒΔλεΫͷ࣮ߦڥʹ……)
AWS CodeBuild - Jenkins Plugin͕͋Δ (AWSۘ) - https://github.com/awslabs/aws-codebuild-jenkins-plugin - JenkinsͰड͚ͨτϦΨͰCodeBuildΛൃՐͰ͖Δ
https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/
https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/
https://aws.amazon.com/jp/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/ ࣮࣭ແݶʂʂʂʂ
ͨΒ͞ΕͨԸܙ - slave nodeͷϝϯςΒͣ - ΨϯΨϯฒྻϏϧυ͢Δͧʂʂʂ - ࣌ؒି͠ (= ͚ͬͨͩ)
ͷྉۚઃఆͳͷͰ ࣗͰΦʔτεέʔϧॲཧΛ͠ͳͯ͘Α͍
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢ - ฒྻ࣮ߦJobͷ૿Ճʹ͏ੑೳྼԽ - ߜΔͱbuildͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ - Master
InstanceࢮͶͳ͍ - ͓ئ͍͠·͢ͷڪා (™ @kyanny)
Master Instance͕ਆ֨Խ͢Δ
Master Instanceਆ֨Խ - Jenkinsͷmaster datamasterϚγϯͷ*ϩʔΧϧετϨʔδ*ʹ อଘ͞ΕΔ - Πϯελϯε͕ࢮΜͩΒऴΘΓ - ͔͠͠ࢮආ͚ΒΕͳ͍
- ͔ͩΒόοΫΞοϓΛऔͬͨΓ͢Δ - e.g. https://papix.hatenablog.com/entry/2016/02/09/101918 - όοΫΞοϓ͔ΒͷϦετΞ࣮ࡍ໘ - ָ͍͠ࣄͰͳ͍…… - μϯλΠϜग़Δ͠……
Master Instanceਆ֨Խ - ݁Ռͱͯ͠ຖଉࡂΛفΔʑ…… - ӡѱ͘Πϯελϯε͕ࢮΜͩΒσΠϦʔόοΫΞοϓ ͔ΒϦετΞ - σΠϦʔόοΫΞοϓͱࢮ͵ؒࡍͱͷࠩʹ͍ͭͯ ఘΊΔ͔͠ແ͍……
͏͏……
զʑৎͰεέʔϧ͢Δ Jenkins͕΄͍͚ͩ͠ͳΜ͡Ό……
ͦΜͳͷ…… ͋Δʂʂʂ
ͦΜͳͷ…… ͋Δʂʂʂ
Amazon Elastic File System
AWS Elastic File System (EFS) - AmazonͷnfsΈ͍ͨͳͭ - Instanceʹmountͯ͠͏ -
EBS + nfsతͳ…… - ϘϦϡʔϜαΠζ͕ࣗಈͰ֦ுɾॖখ͢Δ - Disk fullΒͣͷੜ׆
AWS Elastic File System (EFS) - $JENKINS_HOMEΛEFSϘϦϡʔϜ্ʹ͢Δ - Πϯελϯε͕ࢮΜͰ৽͍͠Πϯελϯεʹ ͦͷϘϦϡʔϜΛϚϯτ͢Δ
- => ΠϯελϯεલੈͷهԱΛҾ͖ܧ͗෮׆ʂʂʂ
mount Working...
❌mount Instance down...
❌ New instance coming up...
❌ mount Works fine!!
AWS Elastic File System (EFS) - EBSͰྑ͍ͷͰʁ - ͍ʂɹEBSͷ΄͏͕͍҆͠ʂʂ -
ͨͩEFSϘϦϡʔϜࣗಈ֦ு͕خ͍͠ - (͏͔ͬΓdisk fullʹͳΒͳ͍) - ͋ͱEFSͩͱmulti masterΈ͍ͨͳ͜ͱͰ͖Δ (ޙड़)
Jenkins Multi Master - ଟͰ͖Δ (࣌ؒӡ༻ͯ͠ͳ͍) - ͔͠͠CodeBuildͱڞଘ͍ͯ͠ΔڥͰMulti Masterʹ͢Δ ϝϦοτ΄΅ͳ͘ͳ͍Ͱ͔͢ʁ
- σϓϩΠ࣌ʹҰॠ2masterΛཱ͓͍ͯͯͯɼݹ͍ํΛ ࡴ͢Έ͍ͨͳμϯλΠϜσϓϩΠ͚ͷ͍ํՄೳ
ͨΒ͞ΕͨԸܙ - ΧδϡΞϧʹΠϯελϯε͕ࢮΜͰ҆৺ʂ - ͱ͍͑όοΫΞοϓऔͬͯΔ - ݁Ռͱͯ͠ΠϯελϯεͷೖΕସָ͕͑ʹͳͬͨ - ετϨʔδࣗಈ֦ு͢ΔͷͰDisk fullΒͣ
Jenkinsӡ༻ʹࡍͯ͠ग़͘Θ͢ - ฒྻ࣮ߦJobͷ૿Ճʹ͏ੑೳྼԽ - ߜΔͱbuildͪͷΩϡʔ͕٧·Δ͠…… - Slave nodeͷӡ༻େม͗͢Δ - Master
InstanceࢮͶͳ͍ - ͓ئ͍͠·͢ͷڪා (™ @kyanny)
ࣾձ: Jenkins͓͡͞ΜͷͳΓखෆ
Jenkinsͷӡ༻Λ୭͔ʹԡ͚͠ΔͷΛΊΔ - ୭Ͱӡ༻Ͱ͖ΔΑ͏ʹ͢Δ - શਓຽJenkins͓͡͞Μ/͓͞ΜԽ - σϓϩΠϝϯςۃྗলྗԽ͢Δ
Jenkinsͷӡ༻Λ୭͔ʹԡ͚͠ΔͷΛΊΔ - Amazon Elastic Beanstalk + Docker - Dockerfile +
Dockerrun.aws.jsonʹΑΔߏͷίʔυԽ - .ebextensionsʹΑΔࣗಈԽ - EFS mountͳͲ - https://github.com/moznion/jenkins_eb_efs_codebuild_example
- Elastic Beanstalkͷmanaged updateͰࣗಈతʹύον͕ͨΔ - ͍ͭͰʹΠϯελϯεͷೖΕସ͑ߦΘΕΔ - EFSڥԼͳͷͰΠϯελϯεೖΕସָ͑ʑࣗಈԽ - جຊతʹӡ༻์ஔͯ͠·͢
- EFSͷϘϦϡʔϜࣗಈ֦ுͷύϫʔͰ͋Δ ͨΒ͞ΕͨԸܙ
·ͱΊ
·ͱΊ - JenkinsͷslaveϊʔυΛAWS CodeBuildʹҕৡ͢Δ͜ͱͰ࣮࣭ ແݶʹฒྻϏϧυ͕εέʔϧ͢ΔΑ͏ʹͳΓ·ͨ͠ - JenkinsͷετϨʔδΛEFSʹ͢Δ͜ͱͰϚελʔσʔλͷ ϩόετԽΛߦ͍·ͨ͠ - Elastic
BeanstalkΛ༻͍ͯJenkinsΛӡ༻͢Δ͜ͱͰӡ༻ͷ ίετԽΛ͠·ͨ͠ - Α͔ͬͨͰ͢Ͷ
Q?
Wait...
͜ͷJenkinsʹ͕͋Δ - EFS͍ʂʂʂʂʂ - Jenkins͕git clone͢ΔͱϚδ͔͔࣌ؒΓ·͢ - git cloneΛճආ͢Δํ๏͍·ͷͱ͜Ζແ͍…… -
shallow cloneΛ༗ޮʹ͏ - sparse checkoutΛ༗ޮʹ͏ - ͳͲͷ͕ඞཁ - ͦͦCodeBuild͕git clone͢ΔΜͩ͠jenkins͠ͳͯ͘ ྑ͍Μ͚ͩͲ…… - ͋Δ͍EBSΛ͏……
ຊʹJenkinsඞཁʁ - ͭ͡AWS CodeBuild࠷ۙΊͬͪΌͰ͖͕Αͯ͘ɼ ୯ମͰे͑Δ - GitHubͷhookड͚ΕΔ͠ - Commit statusΛม͑Δػೳ·Ͱ͍͍ͭͯΔ
- ϏϧυཤྺҰཡͰ͖Δ͠ - CloudWatch LogsʹϏϧυͷϩάग़Δ - JenkinsෆཁͰʁʁʁʁɹ͜ΕͰेͰʁʁʁʁ - ͍
ຊʹJenkinsඞཁʁ - ࣗ༝ͳWebhookΛड͚Δඞཁ͕͋Δͱ͔…… - ͋ͱ௨ͱ͔Ͷʂ - ͋Δ͍ͬͱଞʹϏϧυύΠϓϥΠϯΛܨ ͛Δඞཁ͕͋Δͱ͔…… - ͦ͏͍͏ͱ͖ΜΓͰ͢
- (͋ͱ·͋Ұཡੑ͕Α͍) (ॾઆ͋Γ·͢)
Q?