Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Goで作る大量プロセス管理機構
Search
Keita Mohri
July 13, 2019
Technology
2
3.4k
Goで作る大量プロセス管理機構
Go Conference'19 Summer in Fukuoka
Keita Mohri
July 13, 2019
Tweet
Share
More Decks by Keita Mohri
See All by Keita Mohri
Excelを扱うRubyGemまとめ 2022
ktam1219
0
570
モクえもんのお時間です
ktam1219
0
170
在宅ワーク中だけど会社にしかGPSマルチユニットがない?でも大丈夫!そう、mockmockがあればね。
ktam1219
0
390
IoTデバイスの疑似データ送信システムにおける サーバーレスなログ処理機構の試行錯誤
ktam1219
0
560
実写版モクえもん in Explorer ~愛・おぼえていますか~
ktam1219
0
300
エンジニアのおしごと
ktam1219
0
130
mockmockの大量のログをいい感じに捌きたい
ktam1219
0
1k
わりとゴツいKubernetesハンズオン そのあとに
ktam1219
0
610
明太子とEndorseと私
ktam1219
0
660
Other Decks in Technology
See All in Technology
偏光画像処理ライブラリを作った話
elerac
1
170
RemoveだらけのPHPUnit 12に備えよう
cocoeyes02
0
290
Fraxinus00tw assembly manual
fukumay
0
110
JAWS FESTA 2024「バスロケ」GPS×サーバーレスの開発と運用の舞台裏/jawsfesta2024-bus-gps-serverless
ma2shita
3
210
ESXi で仮想化した ARM 環境で LLM を動作させてみるぞ
unnowataru
0
180
EDRの検知の仕組みと検知回避について
chayakonanaika
12
4.9k
4th place solution Eedi - Mining Misconceptions in Mathematics
rist
0
150
LINEギフトにおけるバックエンド開発
lycorptech_jp
PRO
0
290
エンジニアリング価値を黒字化する バリューベース戦略を用いた 技術戦略策定の道のり
kzkmaeda
6
2.7k
ウォンテッドリーのデータパイプラインを支える ETL のための analytics, rds-exporter / analytics, rds-exporter for ETL to support Wantedly's data pipeline
unblee
0
130
依存パッケージの更新はコツコツが勝つコツ! / phpcon_nagoya2025
blue_goheimochi
3
220
Aurora PostgreSQLがCloudWatch Logsに 出力するログの課金を削減してみる #jawsdays2025
non97
1
210
Featured
See All Featured
Agile that works and the tools we love
rasmusluckow
328
21k
Designing for humans not robots
tammielis
250
25k
Mobile First: as difficult as doing things right
swwweet
223
9.4k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
175
52k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
193
16k
The Cost Of JavaScript in 2023
addyosmani
47
7.4k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
27
1.6k
Gamification - CAS2011
davidbonilla
80
5.2k
Writing Fast Ruby
sferik
628
61k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
45
9.4k
StorybookのUI Testing Handbookを読んだ
zakiyama
28
5.5k
Thoughts on Productivity
jonyablonski
69
4.5k
Transcript
(PͰ࡞Δ େྔϓϩηεཧػߏ ໟརܒଠ
‣ 'VTJD$P -UE ‣ ൃҊऀϓϩμΫτΦʔφʔ ‣ *P5ܥͷडୗҊ݅ ‣
(P 3VCZ "84 403"$0. ໟརܒଠʢ͏Γ͚͍ͨʣ
ࣗࣾαʔϏε
ࣗࣾαʔϏε डୗ։ൃ 8FCαΠτ &$αΠτ FUD ۀΞϓϦ ίϯαϧ "3
Ϋϥυ *P5 'JO5FDI "* ΠϯλϥΫςΟϒίϯςϯπ εϚϗΞϓϦ
ࣗࣾαʔϏε डୗ։ൃ 8FCαΠτ &$αΠτ FUD ۀΞϓϦ ίϯαϧ "3 Ϋϥυ *P5
'JO5FDI "* ΠϯλϥΫςΟϒίϯςϯπ εϚϗΞϓϦ ύʔτφʔ
ࠓ͓͍ͨ͜͠ͱ
ͷલʹʜ
ͬͯͳʹʁ
*P5։ൃऀ͚ ٙࣅσʔλੜαʔϏε
*P5ͷ
Ϟϊ͕ Thing
Πϯλʔωοτܦ༝Ͱ Thing
αʔόʹσʔλΛૹΓ Backend Thing D a t a
ղੳͨ͠Γͯ͠ Backend Thing D a t a
Կ͔͢Δ Backend Thing D a t a
*P5ͷ ͓ΘΓ
*P5 ։ൃ͠·͢
ςετ͍ͨ͠ Backend Thing D a t a
σʔλૹͬͯ΄͍͠ Backend Thing D a t a
Ͱ
Ϟϊʹͩͬͯ ߹͋Δ
͑͐ͬɺ ։ൃʹ͔͔Δͷʂʁ
͑͐ͬɺ ϗϯτʹյ͞ͳ͍ͱ Τϥʔσʔλग़ͤͳͷʂʁ
ෛՙࢼݧ༻ʹ ཉ͍͚͠Ͳʜ ݸສԁʂʁ
ෛՙࢼݧ༻ʹ ༻ҙ͚ͨ͠Ͳʜ ୭͕ૢ࡞͢Δͷ͜Εʂʁ
ςετʹࠔΔ
ͦ͏ͩʂ
γϛϡϨʔλʔΛ࡞Ζ͏ʂ Backend Simulator D a t a
͍ɺ ϦϦʔεʹؒʹ߹Θͳ͍ʂ
͗ͯ͢͠ ʓʓ͞Μ ࡞ͬͨਓ ͔͠ ಈ͔ͤͳ͍ʜ
ٱ͠ͿΓʹಈ͔ͨ͠Β શવಈ͔ͳ͍ʜ
ͦ͜Ͱ
ࢥ͍Ͳ͓ΓʹͳΔϞϊ Backend D a t a
͍ͭͰ ཉ͍͠σʔλΛ ཉ͍͠ྔ͚ͩ ૹͬͯ͘ΕΔʂ
͏Ε͍͠
ը໘Πϝʔδ ࢥ͍௨ΓͷϑΥʔϚοτͰ+40/σʔλΛੜ ԾσόΠεͷεςʔλεͱͦͷมԽΛҙʹઃఆ ͞·͟·ͳ࣌ܥྻσʔλੜػೳ
ͷͬ͘͟Γͨ͠Έ
PublicSubnet PrivateSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3
Bucket (Lot) S3 Bucket (Binary) Web Web Worker Worker Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ mockૢ࡞JobΛ ൃߦ PrivateSubnet Mock Mock Mock Mock Mock
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ PrivateSubnet Mock Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker ίϯιʔϧͰ mockΛૢ࡞ mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock mockىಈ Mock Mock [ mock ] GoόΠφϦ͔Βཱͯͨ ԾσόΠεϓϩηε [ tower ] GoόΠφϦ͔Βཱͯͨ mockཧϓϩηε
ࠓ͓͍ͨ͜͠ͱ վΊͯʂ
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ mockͷࢮ׆ࢹ Ͳ͏ͯ͠Δͷʁ ઃఆϑΝΠϧ όΠφϦΛDL
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ mockͷࢮ׆ࢹ Ͳ͏ͯ͠Δͷʁ όΠφϦͷ ߋ৽ Ͳ͏͢Δͷʁ
ͳΜͰϓϩηεʁ ͳΜͰ(P
‣ NPDLJT αʔόʔ߽՚͗͢Δʜ ίϯςφएׯ໘ͩ͠ɺίϯςφԽ͢Δඞཁͳ͍ ϓϩηεͪΐ͏Ͳ͍͍ʂ εϨουσόοάେมͦ͏ɺڞΕͦ͠͏
ͳΜͰϓϩηεʁ
‣ αʔόʔʹԿΠϯετʔϧͨ͘͠ͳ͍ ͨͱ͑3VCZͩͱ3VCZΛΠϯετʔϧ͢Δඞཁ͕ʜ ‣ γϯάϧόΠφϦʹ͢Εஔ͚ͩ͘Ͱىಈ͕Ͱ͖Δ ‣ γϯάϧόΠφϦ͕Ͱ͖ΔͭͰָͦ͠͏ͳͷ ‣ ͱ͍͑(Pʂ
ओ؍ ͳΜͰ(Pʁ
‣ ίʔυͷن ςετআ͘ NPDLߦ UPXFSߦ ͪͳΈʹ
େྔNPDLͷૢ࡞
‣ ϑΝΠϧϕʔεͰͷૢ࡞ ಛఆͷσΟϨΫτϦʹϑΝΠϧΛஔ͔ΕͨͷΛݕͯ͠ॲཧΛߦ͏ ૢ࡞͕ඞཁͳ͚ͩϑΝΠϧΛஔ͚͍͍ NPDLͷૢ࡞ NPDLىಈ
NPDLఀࢭ ىಈτϦΨʔ ϑΝΠϧઃஔ tower͕ݕ͠ mockΛىಈ ఀࢭτϦΨʔ ϑΝΠϧઃஔ ֤mock༻ ఀࢭτϦΨʔ ϑΝΠϧઃஔ mock͕ݕ͠ ࣗݾফ໓
‣ ݪ࢝త͚ͩͲγϯϓϧͰ৴པੑߴ͘Γ͍͢ ԿΠϯετʔϧ͠ͳ͍͍ͯ͘ ॲཧ͕ྃͨ͠ޙʹϑΝΠϧΛফ͢Α͏ʹ͓͚ͯ͠ɺ ͠UPXFS్͕தͰ࠶ىಈͯ͠ϑΝΠϧΛݟΕϨδϡʔϜͰ͖Δ ‣ ͦͷଞͷ͔ͬ͜Α͛͞ͳΓํΊΜͲ͍͘͞ؾ͕͢Δ
)551αʔόʔ 1VC4VC2VFVF NPDLͷૢ࡞ ૹ৴ࣦഊͨ͠ͱ͖ͷ࠶ૹʜ ૢ࡞ྃલʹUPXFS͕ࢮΜͩ߹ʜ
‣ https://github.com/fsnotify/fsnotify ‣ ϑΝΠϧมߋݕϥΠϒϥϦ GTOPUJGZ
GTOPUJGZ watcher, err := fsnotify.NewWatcher() defer watcher.Close() err
= watcher.Add("path/to/trigger/directory") // ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } }
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ }
GTOPUJGZ watcher, err := fsnotify.NewWatcher() defer watcher.Close() err
= watcher.Add("path/to/trigger/directory") // ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error ev.OpʹϏοτϚεΫΛ͔͚Δ͜ͱͰૢ࡞छผΛ֬ೝ type Op uint32 const ( Create Op = 1 << iota Write Remove Rename Chmod ) type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ }
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error ev.OpʹϏοτϚεΫΛ͔͚Δ͜ͱͰૢ࡞छผΛ֬ೝ type Op uint32 const ( Create Op = 1 << iota Write Remove Rename Chmod ) type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ } sleepೖΕͯͪΌΜͱશ෦௨ͯ͘͠ΕΔͷͰ QueueͬΆ͍ײ͡ʹͳΔ
NPDLͷࢮ׆ࢹ
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ ϓϩηεͷ ঢ়ଶมԽΛࢹ
NPDLϓϩηεͷঢ়ଶมԽΛࢹ processExitChannel chan int func (t *Tower) mockExec(mock
MockCode, version string, ...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid)
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ)
NPDLϓϩηεͷঢ়ଶมԽΛࢹ processExitChannel chan int func (t *Tower) mockExec(mock
MockCode, version string, ...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ status.ExitStatus()Ͱҟৗऴྃ֬ೝͨ͠Γ͢Δ
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PIDΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) ऴྃͨ͠ΒprocessExitChannelʹ௨ WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ status.ExitStatus()Ͱҟৗऴྃ֬ೝͨ͠Γ͢Δ
NPDLϓϩηεͷঢ়ଶมԽΛࢹ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ case pid := <-processExitChannel: mock := mockProcessTable[pid] // ऴྃޙͷ͓ย͚ΖΖ delete(mockProcessTable, pid) } }
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ ϓϩηεͷ ঢ়ଶมԽΛࢹ ఀࢭτϦΨʔϑΝΠϧͱ ಈ࡞தͷϓϩηεΛൺֱ
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } }
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ ઃஔޙҰఆ͕࣌ؒܦաͨ͠ͷͷΈ͕֬ೝର
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ ઃஔޙҰఆ͕࣌ؒܦաͨ͠ͷͷΈ͕֬ೝର ϓϩηεڧ੍ऴྃ
όΠφϦͷߋ৽
‣ NPDL Քಇதߋ৽ෆཁ ಉ͡ઃఆͰಈ͖ଓ͚ΔͷͰ ىಈ࣌ʹ࠷৽൛ͷόΠφϦΛ࣋ͬͯ͘Ε0, ‣ UPXFS
NPDL͕ՔಇதͰߋ৽͍ͨ͠ ࠶ىಈ چUPXFSΛఀࢭ৽UPXFSΛىಈ Ͱ͖ΔΑ͏ʹ͓ͯ͘͠ NPDLࢹঢ়گͷϨδϡʔϜΛ͕ΜΔʂ όΠφϦͷߋ৽
‣ ࠶ىಈखॱ <چ>UPXFSͷ࠶ىಈτϦΨʔϑΝΠϧͷݕ <چ>τϦΨʔϑΝΠϧʹॻ͔Ε͍ͯΔόʔδϣϯΛ4͔Β%-ͯ͠ىಈ <چ>ऴྃ <৽>NPDLͷࢹঢ়گΛϨδϡʔϜ
‣ ϨδϡʔϜ͠ͳ͍ͱ͍͚ͳ͍ͷ NPDL$PEFͱ1*%Λؔ࿈͚ΔNBQ NPDLϓϩηεΛࢹ͢ΔHPSPVUJOF UPXFSͷ࠶ىಈ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } }
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ τϦΨʔϑΝΠϧͷதΛݺΜͰόʔδϣϯΛऔಘ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ τϦΨʔϑΝΠϧͷதΛݺΜͰόʔδϣϯΛऔಘ ϝΠϯϧʔϓ͔Βൈ͚Δͱͦͷ··ऴྃ
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } }
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } } ֤mockىಈ࣌ʹPIDϑΝΠϧΛ࡞Δ
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } } ֤mockىಈ࣌ʹPIDϑΝΠϧΛ࡞Δ লུͯ͠Δ͚Ͳɺ ϓϩηε͕ݟ͔ͭΒͳ͔ͬͨΒPIDϑΝΠϧΛআ
·ͱΊ
‣ ɹɹʹ͓͍ͯɺ(PΛͬͯ େྔͷϓϩηεΛͲͷΑ͏ʹཧ͍ͯ͠Δ͔Λհ ͦͦͳΜͰϓϩηεʁͳΜͰ(P ϑΝΠϧϕʔεͷNPDLૢ࡞ NPDLͷࢮ׆ࢹ
όΠφϦͷߋ৽ ‣ ʮ໌͔Β͑ΔʯΑ͏ͳ༰Ͱͳ͍͚ͣͩͲɺ ָ͠ΜͰΒ͑ͨͳΒ͍Ͱ͢ ‣ (Pͨͷ͍͠ʂ ·ͱΊ
ΛҰॹʹ ։ൃͯ͘͠ΕΔؒΛืूதʂ ͝ڵຯ͋Δํໟར·Ͱʂ Go Ruby AWS IoT ͬͱ͏·͘ΕΔʂ ͓͠Ζͦ͏ʂ @Fukuoka