Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Goで作る大量プロセス管理機構
Search
Keita Mohri
July 13, 2019
Technology
2
3.6k
Goで作る大量プロセス管理機構
Go Conference'19 Summer in Fukuoka
Keita Mohri
July 13, 2019
Tweet
Share
More Decks by Keita Mohri
See All by Keita Mohri
Excelを扱うRubyGemまとめ 2022
ktam1219
0
610
モクえもんのお時間です
ktam1219
0
190
在宅ワーク中だけど会社にしかGPSマルチユニットがない?でも大丈夫!そう、mockmockがあればね。
ktam1219
0
410
IoTデバイスの疑似データ送信システムにおける サーバーレスなログ処理機構の試行錯誤
ktam1219
0
590
実写版モクえもん in Explorer ~愛・おぼえていますか~
ktam1219
0
330
エンジニアのおしごと
ktam1219
0
150
mockmockの大量のログをいい感じに捌きたい
ktam1219
0
1.1k
わりとゴツいKubernetesハンズオン そのあとに
ktam1219
0
650
明太子とEndorseと私
ktam1219
0
710
Other Decks in Technology
See All in Technology
"複雑なデータ処理 × 静的サイト" を両立させる、楽をするRails運用 / A low-effort Rails workflow that combines “Complex Data Processing × Static Sites”
hogelog
3
1.7k
about #74462 go/token#FileSet
tomtwinkle
1
270
PythonとLLMで挑む、 4コマ漫画の構造化データ化
esuji5
1
130
Tomorrow graphlib, Let us use everybody
hayaosuzuki
0
160
組織観点からIAM Identity CenterとIAMの設計を考える
nrinetcom
PRO
1
150
非同期処理実行基盤 Delayed脱出 → Solid Queue完全移行への旅路。
srockstyle
3
1.6k
ACA でMAGI システムを社内で展開しようとした話
mappie_kochi
0
160
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
11
77k
stupid jj tricks
indirect
0
7.8k
ユニットテストに対する考え方の変遷 / Everyone should watch his live coding
mdstoy
0
120
SOC2取得の全体像
shonansurvivors
1
350
BirdCLEF+2025 Noir 5位解法紹介
myso
0
180
Featured
See All Featured
Into the Great Unknown - MozCon
thekraken
40
2.1k
Agile that works and the tools we love
rasmusluckow
331
21k
Rebuilding a faster, lazier Slack
samanthasiow
84
9.2k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
890
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
9
570
Mobile First: as difficult as doing things right
swwweet
224
9.9k
The Invisible Side of Design
smashingmag
301
51k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
Imperfection Machines: The Place of Print at Facebook
scottboms
269
13k
Making the Leap to Tech Lead
cromwellryan
135
9.5k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
Transcript
(PͰ࡞Δ େྔϓϩηεཧػߏ ໟརܒଠ
‣ 'VTJD$P -UE ‣ ൃҊऀϓϩμΫτΦʔφʔ ‣ *P5ܥͷडୗҊ݅ ‣
(P 3VCZ "84 403"$0. ໟརܒଠʢ͏Γ͚͍ͨʣ
ࣗࣾαʔϏε
ࣗࣾαʔϏε डୗ։ൃ 8FCαΠτ &$αΠτ FUD ۀΞϓϦ ίϯαϧ "3
Ϋϥυ *P5 'JO5FDI "* ΠϯλϥΫςΟϒίϯςϯπ εϚϗΞϓϦ
ࣗࣾαʔϏε डୗ։ൃ 8FCαΠτ &$αΠτ FUD ۀΞϓϦ ίϯαϧ "3 Ϋϥυ *P5
'JO5FDI "* ΠϯλϥΫςΟϒίϯςϯπ εϚϗΞϓϦ ύʔτφʔ
ࠓ͓͍ͨ͜͠ͱ
ͷલʹʜ
ͬͯͳʹʁ
*P5։ൃऀ͚ ٙࣅσʔλੜαʔϏε
*P5ͷ
Ϟϊ͕ Thing
Πϯλʔωοτܦ༝Ͱ Thing
αʔόʹσʔλΛૹΓ Backend Thing D a t a
ղੳͨ͠Γͯ͠ Backend Thing D a t a
Կ͔͢Δ Backend Thing D a t a
*P5ͷ ͓ΘΓ
*P5 ։ൃ͠·͢
ςετ͍ͨ͠ Backend Thing D a t a
σʔλૹͬͯ΄͍͠ Backend Thing D a t a
Ͱ
Ϟϊʹͩͬͯ ߹͋Δ
͑͐ͬɺ ։ൃʹ͔͔Δͷʂʁ
͑͐ͬɺ ϗϯτʹյ͞ͳ͍ͱ Τϥʔσʔλग़ͤͳͷʂʁ
ෛՙࢼݧ༻ʹ ཉ͍͚͠Ͳʜ ݸສԁʂʁ
ෛՙࢼݧ༻ʹ ༻ҙ͚ͨ͠Ͳʜ ୭͕ૢ࡞͢Δͷ͜Εʂʁ
ςετʹࠔΔ
ͦ͏ͩʂ
γϛϡϨʔλʔΛ࡞Ζ͏ʂ Backend Simulator D a t a
͍ɺ ϦϦʔεʹؒʹ߹Θͳ͍ʂ
͗ͯ͢͠ ʓʓ͞Μ ࡞ͬͨਓ ͔͠ ಈ͔ͤͳ͍ʜ
ٱ͠ͿΓʹಈ͔ͨ͠Β શવಈ͔ͳ͍ʜ
ͦ͜Ͱ
ࢥ͍Ͳ͓ΓʹͳΔϞϊ Backend D a t a
͍ͭͰ ཉ͍͠σʔλΛ ཉ͍͠ྔ͚ͩ ૹͬͯ͘ΕΔʂ
͏Ε͍͠
ը໘Πϝʔδ ࢥ͍௨ΓͷϑΥʔϚοτͰ+40/σʔλΛੜ ԾσόΠεͷεςʔλεͱͦͷมԽΛҙʹઃఆ ͞·͟·ͳ࣌ܥྻσʔλੜػೳ
ͷͬ͘͟Γͨ͠Έ
PublicSubnet PrivateSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3
Bucket (Lot) S3 Bucket (Binary) Web Web Worker Worker Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ mockૢ࡞JobΛ ൃߦ PrivateSubnet Mock Mock Mock Mock Mock
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ PrivateSubnet Mock Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker ίϯιʔϧͰ mockΛૢ࡞ mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock mockىಈ Mock Mock [ mock ] GoόΠφϦ͔Βཱͯͨ ԾσόΠεϓϩηε [ tower ] GoόΠφϦ͔Βཱͯͨ mockཧϓϩηε
ࠓ͓͍ͨ͜͠ͱ վΊͯʂ
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock mockىಈ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ mockαʔόʔʹ ૢ࡞ࢦࣔ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ ઃఆϑΝΠϧ όΠφϦΛDL PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ mockͷࢮ׆ࢹ Ͳ͏ͯ͠Δͷʁ ઃఆϑΝΠϧ όΠφϦΛDL
PublicSubnet PrivateSubnet DynamoDB Lambda SNS SQS S3 Bucket
(Lot) S3 Bucket (Binary) Web Web Worker Worker mockૢ࡞JobΛ ൃߦ ઃఆϑΝΠϧΛ Ξοϓϩʔυ Քಇঢ়گɾϩάૹ৴ PrivateSubnet Mock Mock Mock Mock Mock Mock ͳΜͰϓϩηεʁ ͳΜͰGoʁ WebίϯιʔϧͰ ಈ࡞Λઃఆ mockΛૢ࡞ Ͳ͏ͬͯ େྔͷϓϩηεΛૢ࡞ʁ mockͷࢮ׆ࢹ Ͳ͏ͯ͠Δͷʁ όΠφϦͷ ߋ৽ Ͳ͏͢Δͷʁ
ͳΜͰϓϩηεʁ ͳΜͰ(P
‣ NPDLJT αʔόʔ߽՚͗͢Δʜ ίϯςφएׯ໘ͩ͠ɺίϯςφԽ͢Δඞཁͳ͍ ϓϩηεͪΐ͏Ͳ͍͍ʂ εϨουσόοάେมͦ͏ɺڞΕͦ͠͏
ͳΜͰϓϩηεʁ
‣ αʔόʔʹԿΠϯετʔϧͨ͘͠ͳ͍ ͨͱ͑3VCZͩͱ3VCZΛΠϯετʔϧ͢Δඞཁ͕ʜ ‣ γϯάϧόΠφϦʹ͢Εஔ͚ͩ͘Ͱىಈ͕Ͱ͖Δ ‣ γϯάϧόΠφϦ͕Ͱ͖ΔͭͰָͦ͠͏ͳͷ ‣ ͱ͍͑(Pʂ
ओ؍ ͳΜͰ(Pʁ
‣ ίʔυͷن ςετআ͘ NPDLߦ UPXFSߦ ͪͳΈʹ
େྔNPDLͷૢ࡞
‣ ϑΝΠϧϕʔεͰͷૢ࡞ ಛఆͷσΟϨΫτϦʹϑΝΠϧΛஔ͔ΕͨͷΛݕͯ͠ॲཧΛߦ͏ ૢ࡞͕ඞཁͳ͚ͩϑΝΠϧΛஔ͚͍͍ NPDLͷૢ࡞ NPDLىಈ
NPDLఀࢭ ىಈτϦΨʔ ϑΝΠϧઃஔ tower͕ݕ͠ mockΛىಈ ఀࢭτϦΨʔ ϑΝΠϧઃஔ ֤mock༻ ఀࢭτϦΨʔ ϑΝΠϧઃஔ mock͕ݕ͠ ࣗݾফ໓
‣ ݪ࢝త͚ͩͲγϯϓϧͰ৴པੑߴ͘Γ͍͢ ԿΠϯετʔϧ͠ͳ͍͍ͯ͘ ॲཧ͕ྃͨ͠ޙʹϑΝΠϧΛফ͢Α͏ʹ͓͚ͯ͠ɺ ͠UPXFS్͕தͰ࠶ىಈͯ͠ϑΝΠϧΛݟΕϨδϡʔϜͰ͖Δ ‣ ͦͷଞͷ͔ͬ͜Α͛͞ͳΓํΊΜͲ͍͘͞ؾ͕͢Δ
)551αʔόʔ 1VC4VC2VFVF NPDLͷૢ࡞ ૹ৴ࣦഊͨ͠ͱ͖ͷ࠶ૹʜ ૢ࡞ྃલʹUPXFS͕ࢮΜͩ߹ʜ
‣ https://github.com/fsnotify/fsnotify ‣ ϑΝΠϧมߋݕϥΠϒϥϦ GTOPUJGZ
GTOPUJGZ watcher, err := fsnotify.NewWatcher() defer watcher.Close() err
= watcher.Add("path/to/trigger/directory") // ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } }
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ }
GTOPUJGZ watcher, err := fsnotify.NewWatcher() defer watcher.Close() err
= watcher.Add("path/to/trigger/directory") // ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error ev.OpʹϏοτϚεΫΛ͔͚Δ͜ͱͰૢ࡞छผΛ֬ೝ type Op uint32 const ( Create Op = 1 << iota Write Remove Rename Chmod ) type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ }
watcher, err := fsnotify.NewWatcher() defer watcher.Close() err = watcher.Add("path/to/trigger/directory") //
ϝΠϯϧʔϓ for { select { case ev := <-watcher.Events: // ϑΝΠϧ࡞Ҏ֎ແࢹ if (ev.Op & fsnotify.Create) == 0 { continue } // τϦΨʔϑΝΠϧͷॲཧ runTriggerProcessor(ev.Name) // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰదʹsleep // (ུ) case err := <-watcher.Errors: // Τϥʔॲཧ case //ͦͷଞॾʑͷॲཧ } } GTOPUJGZ ࢹରͷσΟϨΫτϦΛઃఆ func (w *Watcher) Add(name string) error ev.OpʹϏοτϚεΫΛ͔͚Δ͜ͱͰૢ࡞छผΛ֬ೝ type Op uint32 const ( Create Op = 1 << iota Write Remove Rename Chmod ) type Event struct { Name string // Πϕϯτ͕͋ͬͨ file/directory Op Op // ૢ࡞छผΛද͢Ϗοτྻ } sleepೖΕͯͪΌΜͱશ෦௨ͯ͘͠ΕΔͷͰ QueueͬΆ͍ײ͡ʹͳΔ
NPDLͷࢮ׆ࢹ
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ ϓϩηεͷ ঢ়ଶมԽΛࢹ
NPDLϓϩηεͷঢ়ଶมԽΛࢹ processExitChannel chan int func (t *Tower) mockExec(mock
MockCode, version string, ...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid)
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ)
NPDLϓϩηεͷঢ়ଶมԽΛࢹ processExitChannel chan int func (t *Tower) mockExec(mock
MockCode, version string, ...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PidΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ status.ExitStatus()Ͱҟৗऴྃ֬ೝͨ͠Γ͢Δ
processExitChannel chan int func (t *Tower) mockExec(mock MockCode, version string,
...) { // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ) // ϓϩηεىಈ var proc *os.Process proc, err = os.StartProcess(binPath, args, attr) // PIDΛอଘ mockProcessTable[proc.Pid] = mock // goroutineͰऴྃࢹ go func(t *Tower, mock MockCode, pid int) { for { var status syscall.WaitStatus var rusage syscall.Rusage wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage) if err != nil || (wpid != 0 && status.Exited()) { processExitChannel <- pid return } time.Sleep(intervalMsec) } }(t, mock, proc.Pid) NPDLϓϩηεͷঢ়ଶมԽΛࢹ ϓϩηεىಈޙɺgoroutineΛΒͤͯࢹ͢Δ (ىಈ͍ͯ͠Δmockͷ͚ͩgoroutineΛΒ͍ͤͯΔ) ऴྃͨ͠ΒprocessExitChannelʹ௨ WNOHANGΦϓγϣϯ: ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍߹ʹ͙͢ʹ෮ؼ status.ExitStatus()Ͱҟৗऴྃ֬ೝͨ͠Γ͢Δ
NPDLϓϩηεͷঢ়ଶมԽΛࢹ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ case pid := <-processExitChannel: mock := mockProcessTable[pid] // ऴྃޙͷ͓ย͚ΖΖ delete(mockProcessTable, pid) } }
‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ ࣗવఀࢭ w ࠷େՔಇ࣌ؒܦա w ఀࢭεςʔλεͷભҠ
‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔ NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑθϩͰͳ͍ దʹݕɾ௨͠ͳ͚Ε NPDLͷࢮ׆ࢹ ϓϩηεͷ ঢ়ଶมԽΛࢹ ఀࢭτϦΨʔϑΝΠϧͱ ಈ࡞தͷϓϩηεΛൺֱ
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } }
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ ઃஔޙҰఆ͕࣌ؒܦաͨ͠ͷͷΈ͕֬ೝର
ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ ticker := time.NewTicker(tickerIntervalMsec) defer ticker.Stop() // ϝΠϯϧʔϓ
for { select { case // τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ case tickTime := <-ticker.C: // τϦΨʔϑΝΠϧ͕͋Δͷʹ͍ͬͯΔϓϩηεΛ֬ೝ t.processLeftMock(tickTime) } } func (t *Tower) processLeftMock(cur time.Time) { for mock, term := range t.terminateTable { if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) { continue } pid, processExist := t.mockTable[mock] proc, err := os.FindProcess(pid) err = proc.Signal(os.Kill) delete(t.terminateTable, mock) } } tickerΛͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ ઃஔޙҰఆ͕࣌ؒܦաͨ͠ͷͷΈ͕֬ೝର ϓϩηεڧ੍ऴྃ
όΠφϦͷߋ৽
‣ NPDL Քಇதߋ৽ෆཁ ಉ͡ઃఆͰಈ͖ଓ͚ΔͷͰ ىಈ࣌ʹ࠷৽൛ͷόΠφϦΛ࣋ͬͯ͘Ε0, ‣ UPXFS
NPDL͕ՔಇதͰߋ৽͍ͨ͠ ࠶ىಈ چUPXFSΛఀࢭ৽UPXFSΛىಈ Ͱ͖ΔΑ͏ʹ͓ͯ͘͠ NPDLࢹঢ়گͷϨδϡʔϜΛ͕ΜΔʂ όΠφϦͷߋ৽
‣ ࠶ىಈखॱ <چ>UPXFSͷ࠶ىಈτϦΨʔϑΝΠϧͷݕ <چ>τϦΨʔϑΝΠϧʹॻ͔Ε͍ͯΔόʔδϣϯΛ4͔Β%-ͯ͠ىಈ <چ>ऴྃ <৽>NPDLͷࢹঢ়گΛϨδϡʔϜ
‣ ϨδϡʔϜ͠ͳ͍ͱ͍͚ͳ͍ͷ NPDL$PEFͱ1*%Λؔ࿈͚ΔNBQ NPDLϓϩηεΛࢹ͢ΔHPSPVUJOF UPXFSͷ࠶ىಈ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } }
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ τϦΨʔϑΝΠϧͷதΛݺΜͰόʔδϣϯΛऔಘ
࠶ىಈτϦΨʔϑΝΠϧݕdऴྃ // ϝΠϯϧʔϓ for { select { case
// τϦΨʔϑΝΠϧͷݕॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔ case ev := <-t.towerReboot.Events: if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) { continue } rebootStrs, err := ioutil.ReadFile(ev.Name) reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ") // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍߹μϯϩʔυͯ͘͠Δ(ུ) // ৽towerىಈ _, err = os.StartProcess(binPath, args, attr) // ϝΠϯϧʔϓ͔Βൈ͚Δ return } } ࠶ىಈτϦΨʔϑΝΠϧͷ࡞ΠϕϯτҎ֎ແࢹ τϦΨʔϑΝΠϧͷதΛݺΜͰόʔδϣϯΛऔಘ ϝΠϯϧʔϓ͔Βൈ͚Δͱͦͷ··ऴྃ
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } }
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } } ֤mockىಈ࣌ʹPIDϑΝΠϧΛ࡞Δ
NPDLࢹঢ়گͷϨδϡʔϜ func NewTower() *Tower { // ͍Ζ͍ΖॳظԽ(ུ) //
PIDϑΝΠϧΛऔಘ pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*")) for _, pidFile := range pidFiles { mock := MockCode(filepath.Base(pidFile)) // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ pidStr, err := ioutil.ReadFile(pidFile) pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr))) // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ proc, err := os.FindProcess(pid) t.mockProcessTable[pid] = mock // ϓϩηεऴྃ௨goroutineΛՃ go func(t *Tower, mock MockCode, pid int) { // (ུ) }(&t, mock, pid) } } ֤mockىಈ࣌ʹPIDϑΝΠϧΛ࡞Δ লུͯ͠Δ͚Ͳɺ ϓϩηε͕ݟ͔ͭΒͳ͔ͬͨΒPIDϑΝΠϧΛআ
·ͱΊ
‣ ɹɹʹ͓͍ͯɺ(PΛͬͯ େྔͷϓϩηεΛͲͷΑ͏ʹཧ͍ͯ͠Δ͔Λհ ͦͦͳΜͰϓϩηεʁͳΜͰ(P ϑΝΠϧϕʔεͷNPDLૢ࡞ NPDLͷࢮ׆ࢹ
όΠφϦͷߋ৽ ‣ ʮ໌͔Β͑ΔʯΑ͏ͳ༰Ͱͳ͍͚ͣͩͲɺ ָ͠ΜͰΒ͑ͨͳΒ͍Ͱ͢ ‣ (Pͨͷ͍͠ʂ ·ͱΊ
ΛҰॹʹ ։ൃͯ͘͠ΕΔؒΛืूதʂ ͝ڵຯ͋Δํໟར·Ͱʂ Go Ruby AWS IoT ͬͱ͏·͘ΕΔʂ ͓͠Ζͦ͏ʂ @Fukuoka