Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Goで作る大量プロセス管理機構

 Goで作る大量プロセス管理機構

Go Conference'19 Summer in Fukuoka

Keita Mohri

July 13, 2019
Tweet

More Decks by Keita Mohri

Other Decks in Technology

Transcript

  1. (PͰ࡞Δ
    େྔϓϩηε؅ཧػߏ

    ໟརܒଠ

    View full-size slide



  2. ‣ 'VTJD$P -UE
    ‣ ൃҊऀϓϩμΫτΦʔφʔ
    ‣ *P5ܥͷडୗҊ݅΋
    ‣ (P 3VCZ "84 403"$0.
    ໟརܒଠʢ΋͏Γ͚͍ͨʣ

    View full-size slide



  3. ࣗࣾαʔϏε

    View full-size slide



  4. ࣗࣾαʔϏε
    डୗ։ൃ
    8FCαΠτ &$αΠτ
    FUD
    ۀ຿ΞϓϦ
    ίϯαϧ "3
    Ϋϥ΢υ
    *P5 'JO5FDI
    "*
    ΠϯλϥΫςΟϒίϯςϯπ εϚϗΞϓϦ

    View full-size slide

  5. ࣗࣾαʔϏε
    डୗ։ൃ
    8FCαΠτ &$αΠτ
    FUD
    ۀ຿ΞϓϦ
    ίϯαϧ "3
    Ϋϥ΢υ
    *P5 'JO5FDI
    "*
    ΠϯλϥΫςΟϒίϯςϯπ εϚϗΞϓϦ
    ύʔτφʔ

    View full-size slide

  6. ࠓ೔͓࿩͍ͨ͜͠ͱ

    View full-size slide



  7. *P5։ൃऀ޲͚
    ٙࣅσʔλੜ੒αʔϏε

    View full-size slide



  8. Πϯλʔωοτܦ༝Ͱ
    Thing

    View full-size slide



  9. αʔόʹσʔλΛૹΓ
    Backend
    Thing
    D a t a

    View full-size slide



  10. ղੳͨ͠Γͯ͠
    Backend
    Thing
    D a t a

    View full-size slide



  11. Կ͔͢Δ
    Backend
    Thing
    D a t a

    View full-size slide



  12. *P5ͷ࿩
    ͓ΘΓ

    View full-size slide



  13. *P5
    ։ൃ͠·͢

    View full-size slide



  14. ςετ͍ͨ͠
    Backend
    Thing
    D a t a

    View full-size slide



  15. σʔλૹͬͯ΄͍͠
    Backend
    Thing
    D a t a

    View full-size slide



  16. Ϟϊʹͩͬͯ
    ౎߹͸͋Δ

    View full-size slide



  17. ͑͐ͬɺ
    ։ൃʹ൒೥΋͔͔Δͷʂʁ

    View full-size slide



  18. ͑͐ͬɺ
    ϗϯτʹյ͞ͳ͍ͱ
    Τϥʔσʔλग़ͤͳͷʂʁ

    View full-size slide



  19. ෛՙࢼݧ༻ʹ୆
    ཉ͍͚͠Ͳʜ
    ݸສԁʂʁ

    View full-size slide



  20. ෛՙࢼݧ༻ʹ୆
    ༻ҙ͚ͨ͠Ͳʜ
    ୭͕ૢ࡞͢Δͷ͜Εʂʁ

    View full-size slide



  21. ςετʹࠔΔ

    View full-size slide



  22. γϛϡϨʔλʔΛ࡞Ζ͏ʂ
    Backend
    Simulator
    D a t a

    View full-size slide



  23. ΍͹͍ɺ
    ϦϦʔεʹؒʹ߹Θͳ͍ʂ

    View full-size slide



  24. ೉͗ͯ͢͠
    ʓʓ͞Μ ࡞ͬͨਓ
    ͔͠
    ಈ͔ͤͳ͍ʜ

    View full-size slide



  25. ٱ͠ͿΓʹಈ͔ͨ͠Β
    શવಈ͔ͳ͍ʜ

    View full-size slide



  26. ࢥ͍Ͳ͓ΓʹͳΔϞϊ
    Backend
    D a t a

    View full-size slide



  27. ͍ͭͰ΋
    ཉ͍͠σʔλΛ
    ཉ͍͠ྔ͚ͩ
    ૹͬͯ͘ΕΔʂ

    View full-size slide

  28. ը໘Πϝʔδ


    ࢥ͍௨ΓͷϑΥʔϚοτͰ+40/σʔλΛੜ੒ Ծ૝σόΠεͷεςʔλεͱͦͷมԽΛ೚ҙʹઃఆ
    ͞·͟·ͳ࣌ܥྻσʔλੜ੒ػೳ

    View full-size slide

  29. ͷͬ͘͟Γͨ͠࢓૊Έ

    View full-size slide



  30. PublicSubnet PrivateSubnet PrivateSubnet
    DynamoDB
    Lambda
    SNS
    SQS
    S3 Bucket
    (Lot)
    S3 Bucket
    (Binary)
    Web
    Web Worker
    Worker
    Mock
    Mock
    Mock
    Mock
    Mock
    WebίϯιʔϧͰ
    ಈ࡞Λઃఆ
    mockΛૢ࡞

    View full-size slide



  31. PublicSubnet PrivateSubnet
    DynamoDB
    Lambda
    SNS
    SQS
    S3 Bucket
    (Lot)
    S3 Bucket
    (Binary)
    Web
    Web Worker
    Worker
    WebίϯιʔϧͰ
    ಈ࡞Λઃఆ
    mockΛૢ࡞
    mockૢ࡞JobΛ
    ൃߦ
    PrivateSubnet
    Mock
    Mock
    Mock
    Mock
    Mock

    View full-size slide



  32. PublicSubnet PrivateSubnet
    DynamoDB
    Lambda
    SNS
    SQS
    S3 Bucket
    (Lot)
    S3 Bucket
    (Binary)
    Web
    Web Worker
    Worker
    mockૢ࡞JobΛ
    ൃߦ
    ઃఆϑΝΠϧΛ
    Ξοϓϩʔυ
    mockαʔόʔʹ
    ૢ࡞ࢦࣔ
    PrivateSubnet
    Mock
    Mock
    Mock
    Mock
    Mock
    Mock
    WebίϯιʔϧͰ
    ಈ࡞Λઃఆ
    mockΛૢ࡞

    View full-size slide



  33. PublicSubnet PrivateSubnet
    DynamoDB
    Lambda
    SNS
    SQS
    S3 Bucket
    (Lot)
    S3 Bucket
    (Binary)
    Web
    Web Worker
    Worker
    mockૢ࡞JobΛ
    ൃߦ
    ઃఆϑΝΠϧΛ
    Ξοϓϩʔυ
    mockαʔόʔʹ
    ૢ࡞ࢦࣔ
    ઃఆϑΝΠϧ
    όΠφϦΛDL
    PrivateSubnet
    Mock
    Mock
    Mock
    Mock
    Mock
    Mock
    WebίϯιʔϧͰ
    ಈ࡞Λઃఆ
    mockΛૢ࡞

    View full-size slide



  34. PublicSubnet PrivateSubnet
    DynamoDB
    Lambda
    SNS
    SQS
    S3 Bucket
    (Lot)
    S3 Bucket
    (Binary)
    Web
    Web Worker
    Worker
    mockૢ࡞JobΛ
    ൃߦ
    ઃఆϑΝΠϧΛ
    Ξοϓϩʔυ
    mockαʔόʔʹ
    ૢ࡞ࢦࣔ
    ઃఆϑΝΠϧ
    όΠφϦΛDL
    PrivateSubnet
    Mock
    Mock
    Mock
    Mock
    Mock
    Mock
    mockىಈ
    WebίϯιʔϧͰ
    ಈ࡞Λઃఆ
    mockΛૢ࡞

    View full-size slide



  35. PublicSubnet PrivateSubnet
    DynamoDB
    Lambda
    SNS
    SQS
    S3 Bucket
    (Lot)
    S3 Bucket
    (Binary)
    Web
    Web Worker
    Worker
    mockૢ࡞JobΛ
    ൃߦ
    ઃఆϑΝΠϧΛ
    Ξοϓϩʔυ
    mockαʔόʔʹ
    ૢ࡞ࢦࣔ
    Քಇঢ়گɾϩάૹ৴
    ઃఆϑΝΠϧ
    όΠφϦΛDL
    PrivateSubnet
    Mock
    Mock
    Mock
    Mock
    Mock
    Mock
    mockىಈ
    WebίϯιʔϧͰ
    ಈ࡞Λઃఆ
    mockΛૢ࡞

    View full-size slide



  36. PublicSubnet PrivateSubnet
    DynamoDB
    Lambda
    SNS
    SQS
    S3 Bucket
    (Lot)
    S3 Bucket
    (Binary)
    Web
    Web Worker
    Worker
    ίϯιʔϧͰ
    mockΛૢ࡞
    mockૢ࡞JobΛ
    ൃߦ
    ઃఆϑΝΠϧΛ
    Ξοϓϩʔυ
    mockαʔόʔʹ
    ૢ࡞ࢦࣔ
    Քಇঢ়گɾϩάૹ৴
    ઃఆϑΝΠϧ
    όΠφϦΛDL
    PrivateSubnet
    Mock
    Mock
    Mock
    Mock
    Mock
    mockىಈ
    Mock
    Mock
    [ mock ]
    Go੡όΠφϦ͔Βཱͯͨ
    Ծ૝σόΠε໾ϓϩηε
    [ tower ]
    Go੡όΠφϦ͔Βཱͯͨ
    mock؅ཧϓϩηε

    View full-size slide

  37. ࠓ೔͓࿩͍ͨ͜͠ͱ
    վΊͯʂ

    View full-size slide



  38. PublicSubnet PrivateSubnet
    DynamoDB
    Lambda
    SNS
    SQS
    S3 Bucket
    (Lot)
    S3 Bucket
    (Binary)
    Web
    Web Worker
    Worker
    mockૢ࡞JobΛ
    ൃߦ
    ઃఆϑΝΠϧΛ
    Ξοϓϩʔυ
    mockαʔόʔʹ
    ૢ࡞ࢦࣔ
    Քಇঢ়گɾϩάૹ৴
    ઃఆϑΝΠϧ
    όΠφϦΛDL
    PrivateSubnet
    Mock
    Mock
    Mock
    Mock
    Mock
    Mock
    mockىಈ
    WebίϯιʔϧͰ
    ಈ࡞Λઃఆ
    mockΛૢ࡞

    View full-size slide



  39. PublicSubnet PrivateSubnet
    DynamoDB
    Lambda
    SNS
    SQS
    S3 Bucket
    (Lot)
    S3 Bucket
    (Binary)
    Web
    Web Worker
    Worker
    mockૢ࡞JobΛ
    ൃߦ
    ઃఆϑΝΠϧΛ
    Ξοϓϩʔυ
    mockαʔόʔʹ
    ૢ࡞ࢦࣔ
    Քಇঢ়گɾϩάૹ৴
    ઃఆϑΝΠϧ
    όΠφϦΛDL
    PrivateSubnet
    Mock
    Mock
    Mock
    Mock
    Mock
    Mock
    ͳΜͰϓϩηεʁ
    ͳΜͰGoʁ
    WebίϯιʔϧͰ
    ಈ࡞Λઃఆ
    mockΛૢ࡞

    View full-size slide



  40. PublicSubnet PrivateSubnet
    DynamoDB
    Lambda
    SNS
    SQS
    S3 Bucket
    (Lot)
    S3 Bucket
    (Binary)
    Web
    Web Worker
    Worker
    mockૢ࡞JobΛ
    ൃߦ
    ઃఆϑΝΠϧΛ
    Ξοϓϩʔυ
    Քಇঢ়گɾϩάૹ৴
    ઃఆϑΝΠϧ
    όΠφϦΛDL
    PrivateSubnet
    Mock
    Mock
    Mock
    Mock
    Mock
    Mock
    ͳΜͰϓϩηεʁ
    ͳΜͰGoʁ
    WebίϯιʔϧͰ
    ಈ࡞Λઃఆ
    mockΛૢ࡞
    Ͳ͏΍ͬͯ
    େྔͷϓϩηεΛૢ࡞ʁ

    View full-size slide



  41. PublicSubnet PrivateSubnet
    DynamoDB
    Lambda
    SNS
    SQS
    S3 Bucket
    (Lot)
    S3 Bucket
    (Binary)
    Web
    Web Worker
    Worker
    mockૢ࡞JobΛ
    ൃߦ
    ઃఆϑΝΠϧΛ
    Ξοϓϩʔυ
    Քಇঢ়گɾϩάૹ৴
    PrivateSubnet
    Mock
    Mock
    Mock
    Mock
    Mock
    Mock
    ͳΜͰϓϩηεʁ
    ͳΜͰGoʁ
    WebίϯιʔϧͰ
    ಈ࡞Λઃఆ
    mockΛૢ࡞
    Ͳ͏΍ͬͯ
    େྔͷϓϩηεΛૢ࡞ʁ
    mockͷࢮ׆؂ࢹ͸
    Ͳ͏ͯ͠Δͷʁ
    ઃఆϑΝΠϧ
    όΠφϦΛDL

    View full-size slide



  42. PublicSubnet PrivateSubnet
    DynamoDB
    Lambda
    SNS
    SQS
    S3 Bucket
    (Lot)
    S3 Bucket
    (Binary)
    Web
    Web Worker
    Worker
    mockૢ࡞JobΛ
    ൃߦ
    ઃఆϑΝΠϧΛ
    Ξοϓϩʔυ
    Քಇঢ়گɾϩάૹ৴
    PrivateSubnet
    Mock
    Mock
    Mock
    Mock
    Mock
    Mock
    ͳΜͰϓϩηεʁ
    ͳΜͰGoʁ
    WebίϯιʔϧͰ
    ಈ࡞Λઃఆ
    mockΛૢ࡞
    Ͳ͏΍ͬͯ
    େྔͷϓϩηεΛૢ࡞ʁ
    mockͷࢮ׆؂ࢹ͸
    Ͳ͏ͯ͠Δͷʁ
    όΠφϦͷ
    ߋ৽͸
    Ͳ͏͢Δͷʁ

    View full-size slide

  43. ͳΜͰϓϩηεʁ
    ͳΜͰ(P

    View full-size slide

  44. ‣ NPDLJT
    αʔόʔ߽՚͗͢Δʜ
    ίϯςφएׯ໘౗ͩ͠ɺίϯςφԽ͢Δඞཁ΋ͳ͍
    ϓϩηεͪΐ͏Ͳ͍͍ʂ
    εϨουσόοάେมͦ͏ɺڞ౗Εͦ͠͏
    ͳΜͰϓϩηεʁ


    View full-size slide

  45. ‣ αʔόʔʹԿ΋Πϯετʔϧͨ͘͠ͳ͍
    ͨͱ͑͹3VCZͩͱ3VCZΛΠϯετʔϧ͢Δඞཁ͕ʜ
    ‣ γϯάϧόΠφϦʹ͢Ε͹ஔ͚ͩ͘Ͱىಈ͕Ͱ͖Δ
    ‣ γϯάϧόΠφϦ͕Ͱ͖Δ΍ͭͰָͦ͠͏ͳͷ
    ‣ ͱ͍͑͹(Pʂ ओ؍

    ͳΜͰ(Pʁ


    View full-size slide

  46. ‣ ίʔυͷن໛ ςετআ͘

    NPDL໿ߦ
    UPXFS໿ߦ
    ͪͳΈʹ


    View full-size slide

  47. େྔNPDLͷૢ࡞

    View full-size slide

  48. ‣ ϑΝΠϧϕʔεͰͷૢ࡞
    ಛఆͷσΟϨΫτϦʹϑΝΠϧΛஔ͔ΕͨͷΛݕ஌ͯ͠ॲཧΛߦ͏
    ૢ࡞͕ඞཁͳ෼͚ͩϑΝΠϧΛஔ͚͹͍͍
    NPDLͷૢ࡞


    NPDLىಈ NPDLఀࢭ
    ىಈτϦΨʔ

    ϑΝΠϧઃஔ
    tower͕ݕ஌͠
    mockΛىಈ
    ఀࢭτϦΨʔ

    ϑΝΠϧઃஔ
    ֤mock༻
    ఀࢭτϦΨʔ

    ϑΝΠϧઃஔ
    mock͕ݕ஌͠
    ࣗݾফ໓

    View full-size slide

  49. ‣ ݪ࢝త͚ͩͲγϯϓϧͰ৴པੑ΋ߴ͘΍Γ΍͍͢
    Կ΋Πϯετʔϧ͠ͳ͍͍ͯ͘
    ॲཧ͕׬ྃͨ͠ޙʹϑΝΠϧΛফ͢Α͏ʹ͓͚ͯ͠͹ɺ

    ΋͠UPXFS్͕தͰ࠶ىಈͯ͠΋ϑΝΠϧΛݟΕ͹ϨδϡʔϜͰ͖Δ
    ‣ ͦͷଞͷ͔ͬ͜Α͛͞ͳ΍Γํ͸ΊΜͲ͍͘͞ؾ͕͢Δ
    )551αʔόʔ
    1VC4VC2VFVF
    NPDLͷૢ࡞


    ૹ৴ࣦഊͨ͠ͱ͖ͷ࠶ૹʜ
    ૢ࡞׬ྃલʹUPXFS͕ࢮΜͩ৔߹ʜ

    View full-size slide

  50. ‣ https://github.com/fsnotify/fsnotify
    ‣ ϑΝΠϧมߋݕ஌ϥΠϒϥϦ
    GTOPUJGZ


    View full-size slide

  51. GTOPUJGZ


    watcher, err := fsnotify.NewWatcher()
    defer watcher.Close()
    err = watcher.Add("path/to/trigger/directory")
    // ϝΠϯϧʔϓ
    for {
    select {
    case ev := <-watcher.Events:
    // ϑΝΠϧ࡞੒Ҏ֎͸ແࢹ
    if (ev.Op & fsnotify.Create) == 0 {
    continue
    }
    // τϦΨʔϑΝΠϧͷॲཧ
    runTriggerProcessor(ev.Name)
    // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰద੾ʹsleep
    // (ུ)
    case err := <-watcher.Errors:
    // Τϥʔॲཧ
    case //ͦͷଞॾʑͷॲཧ
    }
    }

    View full-size slide

  52. watcher, err := fsnotify.NewWatcher()
    defer watcher.Close()
    err = watcher.Add("path/to/trigger/directory")
    // ϝΠϯϧʔϓ
    for {
    select {
    case ev := <-watcher.Events:
    // ϑΝΠϧ࡞੒Ҏ֎͸ແࢹ
    if (ev.Op & fsnotify.Create) == 0 {
    continue
    }
    // τϦΨʔϑΝΠϧͷॲཧ
    runTriggerProcessor(ev.Name)
    // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰద੾ʹsleep
    // (ུ)
    case err := <-watcher.Errors:
    // Τϥʔॲཧ
    case //ͦͷଞॾʑͷॲཧ
    }
    }
    GTOPUJGZ


    ؂ࢹର৅ͷσΟϨΫτϦΛઃఆ
    func (w *Watcher) Add(name string) error

    View full-size slide

  53. watcher, err := fsnotify.NewWatcher()
    defer watcher.Close()
    err = watcher.Add("path/to/trigger/directory")
    // ϝΠϯϧʔϓ
    for {
    select {
    case ev := <-watcher.Events:
    // ϑΝΠϧ࡞੒Ҏ֎͸ແࢹ
    if (ev.Op & fsnotify.Create) == 0 {
    continue
    }
    // τϦΨʔϑΝΠϧͷॲཧ
    runTriggerProcessor(ev.Name)
    // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰద੾ʹsleep
    // (ུ)
    case err := <-watcher.Errors:
    // Τϥʔॲཧ
    case //ͦͷଞॾʑͷॲཧ
    }
    }
    GTOPUJGZ


    ؂ࢹର৅ͷσΟϨΫτϦΛઃఆ
    func (w *Watcher) Add(name string) error
    type Event struct {
    Name string // Πϕϯτ͕͋ͬͨ file/directory
    Op Op // ૢ࡞छผΛද͢Ϗοτྻ
    }

    View full-size slide

  54. GTOPUJGZ


    watcher, err := fsnotify.NewWatcher()
    defer watcher.Close()
    err = watcher.Add("path/to/trigger/directory")
    // ϝΠϯϧʔϓ
    for {
    select {
    case ev := <-watcher.Events:
    // ϑΝΠϧ࡞੒Ҏ֎͸ແࢹ
    if (ev.Op & fsnotify.Create) == 0 {
    continue
    }
    // τϦΨʔϑΝΠϧͷॲཧ
    runTriggerProcessor(ev.Name)
    // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰద੾ʹsleep
    // (ུ)
    case err := <-watcher.Errors:
    // Τϥʔॲཧ
    case //ͦͷଞॾʑͷॲཧ
    }
    }
    ؂ࢹର৅ͷσΟϨΫτϦΛઃఆ
    func (w *Watcher) Add(name string) error
    ev.OpʹϏοτϚεΫΛ͔͚Δ͜ͱͰૢ࡞छผΛ֬ೝ
    type Op uint32
    const (
    Create Op = 1 << iota
    Write
    Remove
    Rename
    Chmod
    )
    type Event struct {
    Name string // Πϕϯτ͕͋ͬͨ file/directory
    Op Op // ૢ࡞छผΛද͢Ϗοτྻ
    }

    View full-size slide

  55. watcher, err := fsnotify.NewWatcher()
    defer watcher.Close()
    err = watcher.Add("path/to/trigger/directory")
    // ϝΠϯϧʔϓ
    for {
    select {
    case ev := <-watcher.Events:
    // ϑΝΠϧ࡞੒Ҏ֎͸ແࢹ
    if (ev.Op & fsnotify.Create) == 0 {
    continue
    }
    // τϦΨʔϑΝΠϧͷॲཧ
    runTriggerProcessor(ev.Name)
    // Ұ੪ʹmock͕ىಈ͢Δͱड͚ଆ͕େมͳͷͰద੾ʹsleep
    // (ུ)
    case err := <-watcher.Errors:
    // Τϥʔॲཧ
    case //ͦͷଞॾʑͷॲཧ
    }
    }
    GTOPUJGZ


    ؂ࢹର৅ͷσΟϨΫτϦΛઃఆ
    func (w *Watcher) Add(name string) error
    ev.OpʹϏοτϚεΫΛ͔͚Δ͜ͱͰૢ࡞छผΛ֬ೝ
    type Op uint32
    const (
    Create Op = 1 << iota
    Write
    Remove
    Rename
    Chmod
    )
    type Event struct {
    Name string // Πϕϯτ͕͋ͬͨ file/directory
    Op Op // ૢ࡞छผΛද͢Ϗοτྻ
    }
    sleepೖΕͯ΋ͪΌΜͱશ෦௨஌ͯ͘͠ΕΔͷͰ
    QueueͬΆ͍ײ͡ʹͳΔ

    View full-size slide

  56. NPDLͷࢮ׆؂ࢹ

    View full-size slide

  57. ‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ
    NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ
    ࣗવఀࢭ
    w ࠷େՔಇ࣌ؒܦա
    w ఀࢭεςʔλε΁ͷભҠ
    ‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔
    NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑ΋θϩͰ͸ͳ͍
    ద੾ʹݕ஌ɾ௨஌͠ͳ͚Ε͹
    NPDLͷࢮ׆؂ࢹ


    View full-size slide

  58. ‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ
    NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ
    ࣗવఀࢭ
    w ࠷େՔಇ࣌ؒܦա
    w ఀࢭεςʔλε΁ͷભҠ
    ‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔
    NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑ΋θϩͰ͸ͳ͍
    ద੾ʹݕ஌ɾ௨஌͠ͳ͚Ε͹
    NPDLͷࢮ׆؂ࢹ


    ϓϩηεͷ
    ঢ়ଶมԽΛ؂ࢹ

    View full-size slide

  59. NPDLϓϩηεͷঢ়ଶมԽΛ؂ࢹ


    processExitChannel chan int
    func (t *Tower) mockExec(mock MockCode, version string, ...) {
    // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ)
    // ϓϩηεىಈ
    var proc *os.Process
    proc, err = os.StartProcess(binPath, args, attr)
    // PidΛอଘ
    mockProcessTable[proc.Pid] = mock
    // goroutineͰऴྃ؂ࢹ
    go func(t *Tower, mock MockCode, pid int) {
    for {
    var status syscall.WaitStatus
    var rusage syscall.Rusage
    wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage)
    if err != nil || (wpid != 0 && status.Exited()) {
    processExitChannel <- pid
    return
    }
    time.Sleep(intervalMsec)
    }
    }(t, mock, proc.Pid)

    View full-size slide

  60. processExitChannel chan int
    func (t *Tower) mockExec(mock MockCode, version string, ...) {
    // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ)
    // ϓϩηεىಈ
    var proc *os.Process
    proc, err = os.StartProcess(binPath, args, attr)
    // PidΛอଘ
    mockProcessTable[proc.Pid] = mock
    // goroutineͰऴྃ؂ࢹ
    go func(t *Tower, mock MockCode, pid int) {
    for {
    var status syscall.WaitStatus
    var rusage syscall.Rusage
    wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage)
    if err != nil || (wpid != 0 && status.Exited()) {
    processExitChannel <- pid
    return
    }
    time.Sleep(intervalMsec)
    }
    }(t, mock, proc.Pid)
    NPDLϓϩηεͷঢ়ଶมԽΛ؂ࢹ


    ϓϩηεىಈޙɺgoroutineΛ૸Βͤͯ؂ࢹ͢Δ
    (ىಈ͍ͯ͠Δmockͷ਺͚ͩgoroutineΛ૸Β͍ͤͯΔ)

    View full-size slide

  61. NPDLϓϩηεͷঢ়ଶมԽΛ؂ࢹ


    processExitChannel chan int
    func (t *Tower) mockExec(mock MockCode, version string, ...) {
    // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ)
    // ϓϩηεىಈ
    var proc *os.Process
    proc, err = os.StartProcess(binPath, args, attr)
    // PidΛอଘ
    mockProcessTable[proc.Pid] = mock
    // goroutineͰऴྃ؂ࢹ
    go func(t *Tower, mock MockCode, pid int) {
    for {
    var status syscall.WaitStatus
    var rusage syscall.Rusage
    wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage)
    if err != nil || (wpid != 0 && status.Exited()) {
    processExitChannel <- pid
    return
    }
    time.Sleep(intervalMsec)
    }
    }(t, mock, proc.Pid)
    ϓϩηεىಈޙɺgoroutineΛ૸Βͤͯ؂ࢹ͢Δ
    (ىಈ͍ͯ͠Δmockͷ਺͚ͩgoroutineΛ૸Β͍ͤͯΔ)
    WNOHANGΦϓγϣϯ:
    ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍৔߹ʹ͙͢ʹ෮ؼ

    View full-size slide

  62. processExitChannel chan int
    func (t *Tower) mockExec(mock MockCode, version string, ...) {
    // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ)
    // ϓϩηεىಈ
    var proc *os.Process
    proc, err = os.StartProcess(binPath, args, attr)
    // PidΛอଘ
    mockProcessTable[proc.Pid] = mock
    // goroutineͰऴྃ؂ࢹ
    go func(t *Tower, mock MockCode, pid int) {
    for {
    var status syscall.WaitStatus
    var rusage syscall.Rusage
    wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage)
    if err != nil || (wpid != 0 && status.Exited()) {
    processExitChannel <- pid
    return
    }
    time.Sleep(intervalMsec)
    }
    }(t, mock, proc.Pid)
    NPDLϓϩηεͷঢ়ଶมԽΛ؂ࢹ


    ϓϩηεىಈޙɺgoroutineΛ૸Βͤͯ؂ࢹ͢Δ
    (ىಈ͍ͯ͠Δmockͷ਺͚ͩgoroutineΛ૸Β͍ͤͯΔ)
    WNOHANGΦϓγϣϯ:
    ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍৔߹ʹ͙͢ʹ෮ؼ
    status.ExitStatus()Ͱҟৗऴྃ֬ೝ΋ͨ͠Γ͢Δ

    View full-size slide

  63. processExitChannel chan int
    func (t *Tower) mockExec(mock MockCode, version string, ...) {
    // ىಈઃఆϑΝΠϧͷऔಘͱ͔(ུ)
    // ϓϩηεىಈ
    var proc *os.Process
    proc, err = os.StartProcess(binPath, args, attr)
    // PIDΛอଘ
    mockProcessTable[proc.Pid] = mock
    // goroutineͰऴྃ؂ࢹ
    go func(t *Tower, mock MockCode, pid int) {
    for {
    var status syscall.WaitStatus
    var rusage syscall.Rusage
    wpid, err := syscall.Wait4(pid, &status, syscall.WNOHANG, &rusage)
    if err != nil || (wpid != 0 && status.Exited()) {
    processExitChannel <- pid
    return
    }
    time.Sleep(intervalMsec)
    }
    }(t, mock, proc.Pid)
    NPDLϓϩηεͷঢ়ଶมԽΛ؂ࢹ


    ϓϩηεىಈޙɺgoroutineΛ૸Βͤͯ؂ࢹ͢Δ
    (ىಈ͍ͯ͠Δmockͷ਺͚ͩgoroutineΛ૸Β͍ͤͯΔ)
    ऴྃͨ͠ΒprocessExitChannelʹ௨஌
    WNOHANGΦϓγϣϯ:
    ঢ়ଶมԽ͕ىͬͨ͜ࢠϓϩηε͕ͳ͍৔߹ʹ͙͢ʹ෮ؼ
    status.ExitStatus()Ͱҟৗऴྃ֬ೝ΋ͨ͠Γ͢Δ

    View full-size slide

  64. NPDLϓϩηεͷঢ়ଶมԽΛ؂ࢹ


    // ϝΠϯϧʔϓ
    for {
    select {
    case // τϦΨʔϑΝΠϧͷݕ஌ॲཧͱ͔
    case pid := <-processExitChannel:
    mock := mockProcessTable[pid]
    // ऴྃޙͷ͓ย෇͚΋Ζ΋Ζ
    delete(mockProcessTable, pid)
    }
    }

    View full-size slide

  65. ‣ NPDLఀࢭॲཧͰਖ਼ৗʹఀࢭͰ͖͔ͨ
    NPDL༻ͷఀࢭτϦΨʔϑΝΠϧΛઃஔ
    ࣗવఀࢭ
    w ࠷େՔಇ࣌ؒܦա
    w ఀࢭεςʔλε΁ͷભҠ
    ‣ NPDL͕ҟৗऴ͍ྃͯ͠ͳ͍͔
    NPDL͕ಥવͷࢮΛܴ͑ΔՄೳੑ΋θϩͰ͸ͳ͍
    ద੾ʹݕ஌ɾ௨஌͠ͳ͚Ε͹
    NPDLͷࢮ׆؂ࢹ


    ϓϩηεͷ
    ঢ়ଶมԽΛ؂ࢹ
    ఀࢭτϦΨʔϑΝΠϧͱ
    ಈ࡞தͷϓϩηεΛൺֱ

    View full-size slide

  66. ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ


    ticker := time.NewTicker(tickerIntervalMsec)
    defer ticker.Stop()
    // ϝΠϯϧʔϓ
    for {
    select {
    case // τϦΨʔϑΝΠϧͷݕ஌ॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔
    case tickTime := <-ticker.C:
    // τϦΨʔϑΝΠϧ͕͋Δͷʹ࢒͍ͬͯΔϓϩηεΛ֬ೝ
    t.processLeftMock(tickTime)
    }
    }
    func (t *Tower) processLeftMock(cur time.Time) {
    for mock, term := range t.terminateTable {
    if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) {
    continue
    }
    pid, processExist := t.mockTable[mock]
    proc, err := os.FindProcess(pid)
    err = proc.Signal(os.Kill)
    delete(t.terminateTable, mock)
    }
    }

    View full-size slide

  67. ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ


    ticker := time.NewTicker(tickerIntervalMsec)
    defer ticker.Stop()
    // ϝΠϯϧʔϓ
    for {
    select {
    case // τϦΨʔϑΝΠϧͷݕ஌ॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔
    case tickTime := <-ticker.C:
    // τϦΨʔϑΝΠϧ͕͋Δͷʹ࢒͍ͬͯΔϓϩηεΛ֬ೝ
    t.processLeftMock(tickTime)
    }
    }
    func (t *Tower) processLeftMock(cur time.Time) {
    for mock, term := range t.terminateTable {
    if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) {
    continue
    }
    pid, processExist := t.mockTable[mock]
    proc, err := os.FindProcess(pid)
    err = proc.Signal(os.Kill)
    delete(t.terminateTable, mock)
    }
    }
    tickerΛ࢖ͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ

    View full-size slide

  68. ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ


    ticker := time.NewTicker(tickerIntervalMsec)
    defer ticker.Stop()
    // ϝΠϯϧʔϓ
    for {
    select {
    case // τϦΨʔϑΝΠϧͷݕ஌ॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔
    case tickTime := <-ticker.C:
    // τϦΨʔϑΝΠϧ͕͋Δͷʹ࢒͍ͬͯΔϓϩηεΛ֬ೝ
    t.processLeftMock(tickTime)
    }
    }
    func (t *Tower) processLeftMock(cur time.Time) {
    for mock, term := range t.terminateTable {
    if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) {
    continue
    }
    pid, processExist := t.mockTable[mock]
    proc, err := os.FindProcess(pid)
    err = proc.Signal(os.Kill)
    delete(t.terminateTable, mock)
    }
    }
    tickerΛ࢖ͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ
    ઃஔޙҰఆ͕࣌ؒܦաͨ͠΋ͷͷΈ͕֬ೝର৅

    View full-size slide

  69. ఀࢭτϦΨʔϑΝΠϧͷফԽ֬ೝ


    ticker := time.NewTicker(tickerIntervalMsec)
    defer ticker.Stop()
    // ϝΠϯϧʔϓ
    for {
    select {
    case // τϦΨʔϑΝΠϧͷݕ஌ॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔
    case tickTime := <-ticker.C:
    // τϦΨʔϑΝΠϧ͕͋Δͷʹ࢒͍ͬͯΔϓϩηεΛ֬ೝ
    t.processLeftMock(tickTime)
    }
    }
    func (t *Tower) processLeftMock(cur time.Time) {
    for mock, term := range t.terminateTable {
    if cur.Before(term.triggeredAt.Add(ForceTerminateTime)) {
    continue
    }
    pid, processExist := t.mockTable[mock]
    proc, err := os.FindProcess(pid)
    err = proc.Signal(os.Kill)
    delete(t.terminateTable, mock)
    }
    }
    tickerΛ࢖ͬͯఆظతʹະফԽτϦΨʔϑΝΠϧΛ֬ೝ
    ઃஔޙҰఆ͕࣌ؒܦաͨ͠΋ͷͷΈ͕֬ೝର৅
    ϓϩηεڧ੍ऴྃ

    View full-size slide

  70. όΠφϦͷߋ৽

    View full-size slide

  71. ‣ NPDL
    Քಇத͸ߋ৽ෆཁ ಉ͡ઃఆͰಈ͖ଓ͚ΔͷͰ

    ىಈ࣌ʹ࠷৽൛ͷόΠφϦΛ࣋ͬͯ͘Ε͹0,
    ‣ UPXFS
    NPDL͕ՔಇதͰ΋ߋ৽͍ͨ͠
    ࠶ىಈ چUPXFSΛఀࢭ৽UPXFSΛىಈ
    Ͱ͖ΔΑ͏ʹ͓ͯ͘͠
    NPDL؂ࢹঢ়گͷϨδϡʔϜΛ͕Μ͹Δʂ
    όΠφϦͷߋ৽


    View full-size slide

  72. ‣ ࠶ىಈखॱ
    <چ>UPXFSͷ࠶ىಈτϦΨʔϑΝΠϧͷݕ஌
    <چ>τϦΨʔϑΝΠϧʹॻ͔Ε͍ͯΔόʔδϣϯΛ4͔Β%-ͯ͠ىಈ
    <چ>ऴྃ
    <৽>NPDLͷ؂ࢹঢ়گΛϨδϡʔϜ
    ‣ ϨδϡʔϜ͠ͳ͍ͱ͍͚ͳ͍΋ͷ
    NPDL$PEFͱ1*%Λؔ࿈෇͚ΔNBQ
    NPDLϓϩηεΛ؂ࢹ͢ΔHPSPVUJOF
    UPXFSͷ࠶ىಈ


    View full-size slide

  73. ࠶ىಈτϦΨʔϑΝΠϧݕ஌dऴྃ


    // ϝΠϯϧʔϓ
    for {
    select {
    case // τϦΨʔϑΝΠϧͷݕ஌ॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔
    case ev := <-t.towerReboot.Events:
    if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) {
    continue
    }
    rebootStrs, err := ioutil.ReadFile(ev.Name)
    reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ")
    // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍৔߹͸μ΢ϯϩʔυͯ͘͠Δ(ུ)
    // ৽towerىಈ
    _, err = os.StartProcess(binPath, args, attr)
    // ϝΠϯϧʔϓ͔Βൈ͚Δ
    return
    }
    }

    View full-size slide

  74. ࠶ىಈτϦΨʔϑΝΠϧݕ஌dऴྃ


    // ϝΠϯϧʔϓ
    for {
    select {
    case // τϦΨʔϑΝΠϧͷݕ஌ॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔
    case ev := <-t.towerReboot.Events:
    if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) {
    continue
    }
    rebootStrs, err := ioutil.ReadFile(ev.Name)
    reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ")
    // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍৔߹͸μ΢ϯϩʔυͯ͘͠Δ(ུ)
    // ৽towerىಈ
    _, err = os.StartProcess(binPath, args, attr)
    // ϝΠϯϧʔϓ͔Βൈ͚Δ
    return
    }
    }
    ࠶ىಈτϦΨʔϑΝΠϧͷ࡞੒ΠϕϯτҎ֎͸ແࢹ

    View full-size slide

  75. ࠶ىಈτϦΨʔϑΝΠϧݕ஌dऴྃ


    // ϝΠϯϧʔϓ
    for {
    select {
    case // τϦΨʔϑΝΠϧͷݕ஌ॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔
    case ev := <-t.towerReboot.Events:
    if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) {
    continue
    }
    rebootStrs, err := ioutil.ReadFile(ev.Name)
    reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ")
    // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍৔߹͸μ΢ϯϩʔυͯ͘͠Δ(ུ)
    // ৽towerىಈ
    _, err = os.StartProcess(binPath, args, attr)
    // ϝΠϯϧʔϓ͔Βൈ͚Δ
    return
    }
    }
    ࠶ىಈτϦΨʔϑΝΠϧͷ࡞੒ΠϕϯτҎ֎͸ແࢹ
    τϦΨʔϑΝΠϧͷத਎ΛݺΜͰόʔδϣϯΛऔಘ

    View full-size slide

  76. ࠶ىಈτϦΨʔϑΝΠϧݕ஌dऴྃ


    // ϝΠϯϧʔϓ
    for {
    select {
    case // τϦΨʔϑΝΠϧͷݕ஌ॲཧͱ͔ϓϩηεऴྃ࣌ͷॲཧͱ͔ఆظॲཧͱ͔
    case ev := <-t.towerReboot.Events:
    if (ev.Op & fsnotify.Create) == 0 || (base != towerRebootName) {
    continue
    }
    rebootStrs, err := ioutil.ReadFile(ev.Name)
    reboot := strings.Split(strings.TrimSpace(string(rebootStrs)), " ")
    // ࠶ىಈ࣌ʹόΠφϦ͕ଘࡏ͠ͳ͍৔߹͸μ΢ϯϩʔυͯ͘͠Δ(ུ)
    // ৽towerىಈ
    _, err = os.StartProcess(binPath, args, attr)
    // ϝΠϯϧʔϓ͔Βൈ͚Δ
    return
    }
    }
    ࠶ىಈτϦΨʔϑΝΠϧͷ࡞੒ΠϕϯτҎ֎͸ແࢹ
    τϦΨʔϑΝΠϧͷத਎ΛݺΜͰόʔδϣϯΛऔಘ
    ϝΠϯϧʔϓ͔Βൈ͚Δͱͦͷ··ऴྃ

    View full-size slide

  77. NPDL؂ࢹঢ়گͷϨδϡʔϜ


    func NewTower() *Tower {
    // ͍Ζ͍ΖॳظԽ(ུ)
    // PIDϑΝΠϧΛऔಘ
    pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*"))
    for _, pidFile := range pidFiles {
    mock := MockCode(filepath.Base(pidFile))
    // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ
    pidStr, err := ioutil.ReadFile(pidFile)
    pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr)))
    // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ
    proc, err := os.FindProcess(pid)
    t.mockProcessTable[pid] = mock
    // ϓϩηεऴྃ௨஌goroutineΛ௥Ճ
    go func(t *Tower, mock MockCode, pid int) {
    // (ུ)
    }(&t, mock, pid)
    }
    }

    View full-size slide

  78. NPDL؂ࢹঢ়گͷϨδϡʔϜ


    func NewTower() *Tower {
    // ͍Ζ͍ΖॳظԽ(ུ)
    // PIDϑΝΠϧΛऔಘ
    pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*"))
    for _, pidFile := range pidFiles {
    mock := MockCode(filepath.Base(pidFile))
    // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ
    pidStr, err := ioutil.ReadFile(pidFile)
    pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr)))
    // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ
    proc, err := os.FindProcess(pid)
    t.mockProcessTable[pid] = mock
    // ϓϩηεऴྃ௨஌goroutineΛ௥Ճ
    go func(t *Tower, mock MockCode, pid int) {
    // (ུ)
    }(&t, mock, pid)
    }
    }
    ֤mock͸ىಈ࣌ʹPIDϑΝΠϧΛ࡞Δ

    View full-size slide

  79. NPDL؂ࢹঢ়گͷϨδϡʔϜ


    func NewTower() *Tower {
    // ͍Ζ͍ΖॳظԽ(ུ)
    // PIDϑΝΠϧΛऔಘ
    pidFiles, err := filepath.Glob(filepath.Join(t.PidsDir(), "mk-*"))
    for _, pidFile := range pidFiles {
    mock := MockCode(filepath.Base(pidFile))
    // PIDϑΝΠϧ͔ΒPIDΛಡΈऔΓ
    pidStr, err := ioutil.ReadFile(pidFile)
    pid, err := strconv.Atoi(strings.TrimSpace(string(pidStr)))
    // PID͔Β࣮ࡍͷϓϩηεΛݟ͚ͭͯอଘ
    proc, err := os.FindProcess(pid)
    t.mockProcessTable[pid] = mock
    // ϓϩηεऴྃ௨஌goroutineΛ௥Ճ
    go func(t *Tower, mock MockCode, pid int) {
    // (ུ)
    }(&t, mock, pid)
    }
    }
    ֤mock͸ىಈ࣌ʹPIDϑΝΠϧΛ࡞Δ
    লུͯ͠Δ͚Ͳɺ
    ϓϩηε͕ݟ͔ͭΒͳ͔ͬͨΒPIDϑΝΠϧΛ࡟আ

    View full-size slide

  80. ‣ ɹɹʹ͓͍ͯɺ(PΛ࢖ͬͯ

    େྔͷϓϩηεΛͲͷΑ͏ʹ؅ཧ͍ͯ͠Δ͔Λ঺հ
    ͦ΋ͦ΋ͳΜͰϓϩηεʁͳΜͰ(P
    ϑΝΠϧϕʔεͷNPDLૢ࡞
    NPDLͷࢮ׆؂ࢹ
    όΠφϦͷߋ৽
    ‣ ʮ໌೔͔Β࢖͑ΔʯΑ͏ͳ಺༰Ͱ͸ͳ͍͸͚ͣͩͲɺ

    ָ͠ΜͰ΋Β͑ͨͳΒ޾͍Ͱ͢
    ‣ (Pͨͷ͍͠ʂ
    ·ͱΊ


    View full-size slide

  81. ΛҰॹʹ
    ։ൃͯ͘͠ΕΔ஥ؒΛืूதʂ
    ͝ڵຯ͋Δํ͸ໟར·Ͱʂ
    Go Ruby AWS IoT
    ΋ͬͱ͏·͘΍ΕΔʂ
    ͓΋͠Ζͦ͏ʂ
    @Fukuoka

    View full-size slide