Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[読み会資料] Federated Learning for Vision-and-Language Grounding Problems

catla
March 10, 2020

[読み会資料] Federated Learning for Vision-and-Language Grounding Problems

ゼミ資料.
Federated Learning for Vision-and-Language Grounding Problems(AAAI'20)の紹介.

catla

March 10, 2020
Tweet

More Decks by catla

Other Decks in Science

Transcript

  1. Naoki Katsura
    Federated Learning
    for Vision-and-Language
    Grounding Problems
    2020-03-10
    http://web.pkusz.edu.cn/adsp/files/2019/11/AAAI-FenglinL.1027.pdf

    View Slide

  2. ࿦จ৘ใ
    ʲஶऀʳ
    Fenglin Liu(Penking Univ), Xian Wu(Tencent), Shen Ge(Tencent),
    Wei Fan(Tencent), Yuexian Zou(Penking Univ & Peng Cheng Laboratory)
    ʲग़యʳ
    AAAI 2020
    ʲͳͥɼ͜ͷ࿦จΛબΜͩʁʳ
    ɹ'-ͷϑϨʔϜϫʔΫΛผͷλεΫʹҠ২ͨ͜͠ͱʹ໘നΈΛײ͔ͨ͡Βɽ
    ʲࡶஊʳ
    ୈҰஶऀ'-JVͷ೥ͷઓ੷ʢୈҰஶऀͷ࿦จʣ
    w /FVS*14 "DDFQUFE
    w *+$"* "DDFQUFE
    w *$%. "DDFQUFE



    View Slide

  3. ༻ޠ
    7JTJPOBOE-BOHVBHF(SPVOEJOH1SPCMFN
    ɹࣗવݴޠॲཧͱίϯϐϡʔλϏδϣϯʹ·͕ͨΔ໰୊Λಉ࣌ʹॲཧ͢Δ໰୊ɽ

    image captioning, visual question answer(VQA), image caption retrieval, etc…
    )PSJ[POUBM'FEFSBUFE-FBSOJOH )'-

    ɹTBNQMFCBTFEGFEFSBUFEMFBSOJOHͱ΋஌ΒΕ͍ͯΔɽಛ௃ۭؒ͸ಉ͕ͩ͡ɼα
    ϯϓϧʢϢʔβʣू߹͸ҟͳΔΑ͏ͳσʔλΛ༻͍Δ'-໰୊ɽ
    7FSUJDBM'FEFSBUFE-FBSOJOH 7'-

    ɹGFBUVSFCBTFEGFEFSBUFEMFBSOJOHͱ΋஌ΒΕ͍ͯΔɽαϯϓϧʢϢʔβʣू߹
    ͸ಉ͕ͩ͡ɼಛ௃ۭؒ͸ҟͳΔΑ͏ͳσʔλΛ༻͍Δ'-໰୊ɽ
    'FEFSBUFE5SBOTGFS-FBSOJOH '5-

    ɹಛ௃ۭؒ΋αϯϓϧʢϢʔβʣू߹΋ҟͳΔΑ͏ͳσʔλΛ༻͍Δ'-໰୊ɽ
    )'- 7'- '5-ͳͲͷৄࡉ͸ɼҎԼͷ࿦จʹॻ͍ͯ͋Δɽ
    “Federated Learning machine learning: Concept and applications”

    View Slide

  4. image caption retrieval
    ग़యɿDual Attention Networks for Multimodal Reasoning and Matching

    View Slide

  5. ͲΜͳ࿦จʁ
    ɹ7JTJPOBOE-BOHVBHF(SPVOEJOH1SPCMFNʹ͓͍ͯɼ
    NVMUJUBTLMFBSOJOHʢ.5-ʣΛద༻ͨ͠৔߹ɼEBUBMFBLBHF
    ͱ͍͏໰୊͕ੜ͡Δɽ
    ɹ'FEFSBUFE-FBSOJOHͷ࿮૊ΈΛ࢖͍ɼ֤λεΫΛ'-ʹ͓͚
    ΔݸʑͷΫϥΠΞϯτͱΈͳ͢ͱɼλεΫ͝ͱͷσʔλΛ׬શ
    ʹ෼཭͠ͳ͕ΒϞσϧΛֶशͰ͖ΔΜ͡Όͳ͍͔ʁɹ
    ͱ͍͏ఏҊɽ

    View Slide

  6. എܠ
    ɹJNBHFDBQUJPOJOHͱ72"͸ɼͲͪΒ΋ը૾ͱࣗવݴޠΛѻ͏ͱ͍͏఺ͰλεΫ
    ͕ࣅ͍ͯΔɽ

    View Slide

  7. എܠ
    ɹJNBHFDBQUJPOJOHͱ72"͸ɼͲͪΒ΋ը૾ͱࣗવݴޠΛѻ͏ͱ͍͏఺ͰλεΫ
    ͕ࣅ͍ͯΔɽ
    ɹλεΫΛ߹ΘͤΕ͹ɼҟͳΔλεΫ͔ΒҟͳΔ஌ࣝΛ֫ಘͰ͖ΔͷͰੑೳ޲্͕ݟ
    ࠐΊΔΜ͡Όͳ͍ʁ
    .VMUJUBTLMFBSOJOHΛద༻ɹ -JFUBM

    ͔͠͠ɾɾɾ

    View Slide

  8. എܠ
    ɹJNBHFDBQUJPOJOHͱ72"͸ɼͲͪΒ΋ը૾ͱࣗવݴޠΛѻ͏ͱ͍͏఺ͰλεΫ
    ͕ࣅ͍ͯΔɽ
    ɹλεΫΛ߹ΘͤΕ͹ɼҟͳΔλεΫ͔ΒҟͳΔ஌ࣝΛ֫ಘͰ͖ΔͷͰੑೳ޲্͕ݟ
    ࠐΊΔΜ͡Όͳ͍ʁ
    .VMUJUBTLMFBSOJOHΛద༻ɹ -JFUBM

    ͔͠͠ɾɾɾ
    ɹҟͳΔλεΫؒͰ΋Ϟσϧޙ൒෦·Ͱ͸ॲཧΛڞ༗͍ͯ͠Δɽ
    ɹ ݁ՌɼEBUBMFBLBHF໰୊ͷൃੜ

    View Slide

  9. [Li et al. , 2018]
    ʲ໰୊఺ʳ
    ɹ͜ͷ৚݅ԼͰ͸ɼλεΫؒͰσʔληοτ͕ҟͳΔͱద༻Ͱ͖ͳ͍ɽʢ7'-ʣ
    ɹશλεΫʹ͓͍ͯೖྗΛ౷Ұ͠ͳ͍ͱ͍͚ͳ͍ɽ
    .5-༻ͷಛผͳσʔληοτΛ࡞Δඞཁ͕͋Δɽ

    View Slide

  10. [Nguyen & Okatani , 2019]
    ɹλεΫؒͰσʔληοτΛ
    ౷Ұ͠ͳ͍ͱ͍͚ͳ͍໰୊Λ
    ؇࿨ͨ͠΋ͷɽ
    ɹ72"ͳͲʹ͸ೖྗʹςΩετ
    ͕ඞਢɽ
    ɹ*NBHFDBQUJPOJOHͷΑ͏ͳೖ
    ྗ͕ը૾ͷΈͷͱ͖ɼ͋·Γྑ͍
    ݁Ռ͕ಘΒΕͳ͍ɽ
    ʢEBUBMFBLBHFೖྗςΩετ

    ͔͠͠
    ͕ͨͬͯ͠

    View Slide

  11. ఏҊ
    ɹEBUBMFBLBHFΛ๷͗ͳ͕ΒλεΫؒͰಛ௃Λڞ༗͢ΔͨΊʹɼ'-ͷ࿮૊ΈΛར༻
    ͨ͠ɹ"MJHOJOH *OUFHSBUJOHBOE.BQQJOH/FUXPSLʢBJN/FUʣͷఏҊɽ
    ɹ༷ʑͳ৚݅Λ)'- 7'- '5-ͷ࿮ʹ౰ͯ͸ΊΔ͜ͱͰॊೈʹରԠՄೳɽ
    ͜ͷ࿦จʹ͓͚Δߩݙ͸ɼҎԼͷ௨Γɽ
    ᶃ '-ͷ࿮૊ΈΛ࢖ͬͯɼҟͳΔλεΫ΍σʔληοτΛEBUBMFBLBHFΛ๷͗ͳ͕Β
    ར༻͢Δ͜ͱͰɼ୯ମͷλεΫΑΓ΋ྑ͍ಛ௃ΛಘΔ͜ͱʹ੒ޭͨ͜͠ͱɽ
    ᶄ '-ʹ͓͚ΔDFOUSBMJ[FENPEFMͱͯ͠ɼೖྗը૾͔ΒWJTVBMͱUFYUVBMಛ௃Λநग़
    ͢ΔBJN/FUΛఏҊͨ͜͠ͱɽ
    ᶅ )PSJ[POUBM'FEFSBUFE-FBSOJOH 7FSUJDBM'FEFSBUFE-FBSOJOH 'FEFSBUFE
    5SBOTGFS-FBSOJOHͷ৚݅Լͷ্Ͱɼ༗ޮੑ͕ࣔ͞Εͨ͜ͱɽ

    View Slide

  12. ఏҊख๏
    ʲఏҊख๏ͷߏ੒ʳ
    ᶃ UIFWJTVBMBOEUFYUVBMGFBUVSFTFYUSBDUPS
    ೖྗը૾͔ΒɼΑΓϦονͳදݱΛಘΔಛ௃ྗநग़ثʢ৽نੑ͸ͳ͍ʣ
    ᶄ "MJHOJOH *OUFHSBUJOHBOE.BQQJOH/FUXPSLBJN/FU
    WJTVBMUFYUVBMGFBUVSFΛΑΓϦονͳදݱʹม׵͢ΔωοτϫʔΫɽλεΫ͓
    ΑͼσʔληοτؒͰڞ༗ɽ
    ᶅ UIFJNQMFNFOUBUJPOJOUISFFGFEFSBUFEMFBSOJOHTFUUJOHT
    )'- 7'- '5-ͷঢ়گʹ౰ͯ͸ΊΔ͜ͱͰɼλεΫ΍σʔληοτʹؔ͢Δঢ়گ
    ʹఏҊख๏͕ࠨӈ͞Εͳ͍͜ͱΛࣔ͢ɽ

    View Slide

  13. ఏҊख๏
    ʲఏҊख๏ͷߏ੒ʳ
    ᶃ UIFWJTVBMBOEUFYUVBMGFBUVSFTFYUSBDUPS
    ೖྗը૾͔ΒɼΑΓϦονͳදݱΛಘΔಛ௃ྗநग़ثʢ৽نੑ͸ͳ͍ʣ
    ᶄ "MJHOJOH *OUFHSBUJOHBOE.BQQJOH/FUXPSLBJN/FU
    WJTVBMUFYUVBMGFBUVSFΛΑΓϦονͳදݱʹม׵͢ΔωοτϫʔΫɽλεΫ͓
    ΑͼσʔληοτؒͰڞ༗ɽ
    ᶅ UIFJNQMFNFOUBUJPOJOUISFFGFEFSBUFEMFBSOJOHTFUUJOHT
    )'- 7'- '5-ͷঢ়گʹ౰ͯ͸ΊΔ͜ͱͰɼλεΫ΍σʔληοτʹؔ͢Δঢ়گ
    ʹఏҊख๏͕ࠨӈ͞Εͳ͍͜ͱΛࣔ͢ɽ

    View Slide

  14. ɹWJTJPOBOEMBOHVBHFHSPVOEJOHUBTLʹ͓͍ͯɼ$//Λ༻͍ͯը૾ͷಛ௃Λநग़
    ͢Δ͜ͱ͸޿͘࢖ΘΕ͍ͯΔɽຊ࿦จͰ͸ɼ'BTUFS3$//Λ࢖༻ͯ͠ɼ
    ಛ௃ʢWJTVBMGFBUVSFʣͷநग़Λߦͬͨɽ
    ͔͠͠
    ɹWJTVBMGFBUVSFͷΈͩͱɼ͜ͷλεΫʹ༗ޮͳಛ௃ͷநग़ʹݶք͕͋Δɽ
    ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹʢ8VFUBM ʣ
    ɹUFYUVBMGFBUVSFΛ࢖͏ͱɼΑΓ༗ޮͳಛ௃Λը૾͔Β໌֬ʹநग़Ͱ͖Δɽ
    ɹ'BTUFS3$//ʹΑΔಛ௃ྔநग़ʹՃ͑ͯɼFUBM >ʹैͬͯɼ
    .VMUJQMF*OTUBODF-FBSOJOHʢ;IBOHFUBM ʣΛద༻͠ɼUFYUVBMGFBUVSFΛ
    நग़ͨ͠ɽ
    Visual and Textual Feature

    View Slide

  15. Visual and Textual Feature
    ʲΠϝʔδʳ
    ೖྗը૾
    .VMUJQMF*OTUBODF
    -FBSOJOH
    'BTUFS3$//
    WJTVBMGFBUVSFFYUSBDUPS
    UFYUVBMGFBUVSFFYUSBDUPS
    &NCFEEJOH

    I = { ⃗
    i1
    , ⃗
    i2
    , …, ⃗
    iN
    } ∈ ℝN×d

    T = { ⃗
    w1
    , ⃗
    w2
    , …, ⃗
    wM
    } ∈ ℝM×d
    ɹࠨͷΑ͏ͳಘΒΕͨ୯ޠ܈ΛTFNBOUJDDPODFQU
    ͱ͍͍ɼ෺ମʢEPH΍GSJTCFFʣɼঢ়ଶʢP⒎΍
    FMFDUSJDʣɼؔ܎ੑʢIPMEJOH qZJOHʣؚ͕·ΕΔɽ
    QSPWJEFECZ"OEFSTPOFUBM

    QSPWJEFECZ'BOHFUBM

    ͜ͷFNCFEEJOH૚͸ɼ
    DBQUJPORVFTUJPOͱڞ༗ɽ
    Experiment Setting
    ɹXPSEFNCFEEJOH͸ɼDBQUJPORVFTUJPOͱڞ༗ɽ ͸Ҏ
    ߱Ͱ঺հ͢Δ#BTFMJOF طଘݚڀͷϞσϧ
    ͷσίʔμͷೖྗ
    ࣍ݩʹ͢Δɽ
    d
    N = M = 36.

    View Slide

  16. ఏҊख๏
    ʲఏҊख๏ͷߏ੒ʳ
    ᶃ UIFWJTVBMBOEUFYUVBMGFBUVSFTFYUSBDUPS
    ೖྗը૾͔ΒɼΑΓϦονͳදݱΛಘΔಛ௃ྗநग़ثʢ৽نੑ͸ͳ͍ʣ
    ᶄ "MJHOJOH *OUFHSBUJOHBOE.BQQJOH/FUXPSLBJN/FU
    WJTVBMUFYUVBMGFBUVSFΛΑΓϦονͳදݱʹม׵͢ΔωοτϫʔΫɽλεΫ͓
    ΑͼσʔληοτؒͰڞ༗ɽ
    ᶅ UIFJNQMFNFOUBUJPOJOUISFFGFEFSBUFEMFBSOJOHTFUUJOHT
    )'- 7'- '5-ͷঢ়گʹ౰ͯ͸ΊΔ͜ͱͰɼλεΫ΍σʔληοτʹؔ͢Δঢ়گ
    ʹఏҊख๏͕ࠨӈ͞Εͳ͍͜ͱΛࣔ͢ɽ

    View Slide

  17. Aligning, Integrating and Mapping Network : aimNet
    ɹBJN/FU͸ɼҎԼͷͭͷϞδϡʔϧͰߏ੒͞ΕΔɽ͜Ε͸ɼ֤λεΫʹ͓͚Δը૾
    ͷΤϯίʔμͷ෦෼ʹ૬౰͢Δɽ
    w"MJHOJOHNPEVMF
    ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ
    w*OUFHSBUJOHNPEVMF
    ɹWJTVBMGFBUVSF಺ɼ͓ΑͼUFYUVBMGFBUVSF಺Ͱͷؔ܎ੑΛରԠ͚ͮΔɽ
    w.BQQJOH.PEVMF
    ɹλεΫ͝ͱͷೖྗαΠζʹม׵͢Δɽ

    View Slide

  18. Aligning, Integrating and Mapping Network : aimNet
    "MJHOJOH.PEVMFͱ*OUFHSBUJOH.PEVMF͸ɼ࣍ͷ#BTJD.PEVMFΛϕʔεͱ͍ͯ͠
    Δɽ
    ʲ#BTJD.PEVMFʳ
    ɹ֤ಛ௃ؒͷؔ܎ੑΛଊ͑ΔͨΊʹɼ.VMUJ)FBE"UUFOUJPOʢ.)"ʣͱ'FFE
    'PSXBSE/FUXPSLʢ''/ʣΛద༻ɽ͜ΕΛ࢖͏͜ͱͰɼಛ௃ΛҰରҰͰ͸ͳ͘ɼ
    ଟରଟͰಛ௃ؒͷؔ܎ੑͷڧ͞ΛଌΔ͜ͱ͕Ͱ͖Δɽ
    ɹ
    ͜ͷΞΠσΞΛɼ
    WJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛଊ͑Δ
    ɹ "MJHOJOH.PEVMF
    WJTVBMGFBUVSFಉ࢜ɼ·ͨ͸ɼUFYUVBMGFBUVSFಉ࢜ͷؔ܎ੑΛଊ͑Δ
    ɹ *OUFHSBUJOH.PEVMF


    View Slide

  19. Aligning, Integrating and Mapping Network : aimNet
    w"MJHOJOHNPEVMF
    ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ
    w*OUFHSBUJOHNPEVMF
    ɹWJTVBMGFBUVSF಺ɼ͓ΑͼUFYUVBMGFBUVSF಺Ͱͷؔ܎ੑΛରԠ͚ͮΔɽ
    w.BQQJOH.PEVMF
    ɹλεΫ͝ͱͷೖྗαΠζʹม׵͢Δɽ

    View Slide

  20. Aligning, Integrating and Mapping Network : aimNet
    ʲ"MJHOJOH.PEVMFʳ
    ɹ΍͍ͬͯΔ͜ͱ͸ɼ
    ɹɹ4PVSDF5BSHFU"UUFOUJPOͱಉ͡ɽ
    ɹɹ
    ɹɹ
    ɹ͜Ε͸72"Ͱ༗ޮͱ͍͏͜ͱ͕ࣔ͞Ε͍ͯΔɽʢ4VFUBMʣ
    ɹը૾ͱΞϥΠϝϯτΛऔΔ͜ͱͰɼಉ͡εϖϧͰ΋ෳ਺ͷҙຯΛ࣋ͭΑ͏ͳᐆດͳ
    දݱΛ཈͑ࠐΉ͜ͱ͕Ͱ͖ΔɽʢྫɿNPVTFͶͣΈɼిࢠػثͷϚ΢εʣ

    Ia
    = FFN(MHA( ⃗
    I, ⃗
    T , ⃗
    T ))

    T a
    = FFN(MHA( ⃗
    T , ⃗
    I, ⃗
    I))
    Experiment Setting
    ɹBUUFOUJPOIFBEͷݸ਺͸ɼGFFEGPSXBSEOFUXPSLͷ࣍
    ݩ਺͸

    View Slide

  21. Aligning, Integrating and Mapping Network : aimNet
    w"MJHOJOHNPEVMF
    ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ
    w*OUFHSBUJOHNPEVMF
    ɹWJTVBMGFBUVSF಺ɼ͓ΑͼUFYUVBMGFBUVSF಺Ͱͷؔ܎ੑΛରԠ͚ͮΔɽ
    w.BQQJOH.PEVMF
    ɹλεΫ͝ͱͷೖྗαΠζʹม׵͢Δɽ

    View Slide

  22. Aligning, Integrating and Mapping Network : aimNet
    ʲ*OUFHSBUJOH.PEVMFʳ
    ɹ΍͍ͬͯΔ͜ͱ͸ɼ
    ɹɹ4FMG"UUFOUJPOͱಉ͡ɽ
    ɹɹ
    ɹɹ
    ɹ͜Ε͸*NBHF$BQUJPOJOHͰ༗ޮͱ͍͏͜ͱ͕ࣔ͞Ε͍ͯΔɽʢ:BPFUBMʣ

    Ii
    = FFN(MHA( ⃗
    Ia
    , ⃗
    Ia
    , ⃗
    Ia
    ))

    T i
    = FFN(MHA( ⃗
    T a
    , ⃗
    T a
    , ⃗
    T a
    ))
    Experiment Setting
    ɹBUUFOUJPOIFBEͷݸ਺͸ɼGFFEGPSXBSEOFUXPSLͷ࣍
    ݩ਺͸

    View Slide

  23. Aligning, Integrating and Mapping Network : aimNet
    w"MJHOJOHNPEVMF
    ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ
    w*OUFHSBUJOHNPEVMF
    ɹWJTVBMGFBUVSF಺ɼ͓ΑͼUFYUVBMGFBUVSF಺Ͱͷؔ܎ੑΛରԠ͚ͮΔɽ
    w.BQQJOH.PEVMF
    ɹλεΫ͝ͱͷೖྗαΠζʹม׵͢Δɽ

    View Slide

  24. Aligning, Integrating and Mapping Network : aimNet
    ʲ.BQQJOH.PEVMFʳ
    ɹҎԼͷΑ͏ʹɼ.BQQJOHؔ਺Λ׆ੑԽؔ਺͕UBOIͷ૚ͷχϡʔϥϧωοτͱ͠
    ͯఆٛ͢Δɽ
    ɹɹɹɹɹɹɹɹɹɹ
    ɹ.BQQJOH.PEVMFͰ͸ɼλεΫ͝ͱͷσίʔμͷೖྗۭؒʹલͭͷϞδϡʔϧ͔
    ΒಘΒΕͨಛ௃Λ߹ΘͤΔͨΊʹҎԼͷΑ͏ͳม׵Λߦ͏ɽ
    ɹɹɹɹɹɹɹɹɹɹ
    ɹ͜ͷϞδϡʔϧ͸ɼҰൠతʹطଘख๏ͷଟ͘Ͱ࢖༻͞Ε͍ͯΔɽ#BTFMJOFͰ͸ɼ
    $//ͳͲʢFH'BTUFS3$//ʣ͔Βͷಛ௃ϚοϓΛ͜ͷϞδϡʔϧʹద༻͢Δɽ
    Mapping(x) = tanh(xWm
    + bm
    )Wmm
    + bmm
    LayerNorm(Mapping( ⃗
    It
    ) + Mapping( ⃗
    T t
    ))

    View Slide

  25. ఏҊख๏
    ʲఏҊख๏ͷߏ੒ʳ
    ᶃ UIFWJTVBMBOEUFYUVBMGFBUVSFTFYUSBDUPS
    ೖྗը૾͔ΒɼΑΓϦονͳදݱΛಘΔಛ௃ྗநग़ثʢ৽نੑ͸ͳ͍ʣ
    ᶄ "MJHOJOH *OUFHSBUJOHBOE.BQQJOH/FUXPSLBJN/FU
    WJTVBMUFYUVBMGFBUVSFΛΑΓϦονͳදݱʹม׵͢ΔωοτϫʔΫɽλεΫ͓
    ΑͼσʔληοτؒͰڞ༗ɽ
    ᶅ UIFJNQMFNFOUBUJPOJOUISFFGFEFSBUFEMFBSOJOHTFUUJOHT
    )'- 7'- '5-ͷঢ়گʹ౰ͯ͸ΊΔ͜ͱͰɼλεΫ΍σʔληοτʹؔ͢Δঢ়گ
    ʹఏҊख๏͕ࠨӈ͞Εͳ͍͜ͱΛࣔ͢ɽ

    View Slide

  26. Implementation
    ɹ͞·͟·ͳλεΫ΍σʔληοτͷঢ়گԼͰ૊Έ߹ΘֶͤͨशΛߦ͏ɽ'-ʹ͓͚
    Δͭͷঢ়گΛ.5-తͳղऍΛ͢Δɽ'-ʹ͓͚ΔΫϥΠΞϯτΛλεΫɼಛ௃ۭؒΛ
    σʔληοτͱΈͳͨ͠ɽ
    ɹධՁࢦඪ͸ɼ
    *NBHF$BQUJPOJOH41*$& $*%&S .&5&03 #-&6
    72"UFTUTUBOEBSETFU
    '-ʹ͓͚Δঢ়گ λεΫ σʔληοτ
    )'-
    ಉ͡
    Image Captioning)
    ҟͳΔ
    MSCOCO & Flickr30k

    7'-
    ҟͳΔ
    Image Captioning & VQA

    ಉ͡
    MSCOCO & VQA v2.0

    '5-
    ҟͳΔ
    Image Captioning & VQA

    ҟͳΔ
    Flickr30k & VQA v2.0

    ˞72"Wͷೖྗը૾͸.4$0$0ͷσʔλͰߏ੒͞Ε͍ͯΔ

    View Slide

  27. Implementation : Horizontal Federated Learning
    Experiment Setting
    Baseline(Image Captioning Decoder)
    • Spatial ( Lu et al. 2017 )
    • NBT ( Lu et al. 2017 )

    View Slide

  28. Implementation : Vertical Federated Learning
    Experiment Setting
    Baseline(Image Captioning Decoder)
    • Spatial ( Lu et al. 2017 )
    • NBT ( Lu et al. 2017 )
    Baseline(Visual Question Answering)
    • BUTB ( Anderson et al. 2018 )
    • NBT ( Kim, Jun and Zhang 2018 )

    View Slide

  29. Implementation : Federated Transfer Learning
    Experiment Setting
    Baseline(Image Captioning Decoder)
    • Spatial ( Lu et al. 2017 )
    • NBT ( Lu et al. 2017 )
    Baseline(Visual Question Answering)
    • BUTB ( Anderson et al. 2018 )
    • NBT ( Kim, Jun and Zhang 2018 )

    View Slide

  30. Result : Horizontal Federated Learning

    View Slide

  31. Result : Vertical Federated Learning

    View Slide

  32. Result : Federated Transfer Learning

    View Slide

  33. Result : Ablation study
    'BTUFS3$//.BQQJOH.PEVMF4QBUJBM#65%
    ೋͭͷλεΫΛ૊Έ߹Θֶͤͨश
    ɹ*NBHF$BQUJPOJOHʹ͸*OUFHSBUJOHɼ
    72"ʹ͸"MJHOJOH͕ޮ͍͍ͯΔ͜ͱ͕Θ
    ͔Δɽ

    View Slide

  34. ·ͱΊ
    wEBUBMFBLBHFͷ໰୊Λ๊͍͑ͯͨWJTJPOBOEMBOHVBHFHSPVOEJOHQSPCMFNʹ͓
    ͚Δ.5-ʹ'-ͷϑϨʔϜϫʔΫΛద༻͢Δ͜ͱͰɼ໰୊ͷղܾͱͭͷλεΫɼ·
    ͨ͸σʔληοτΑΓ΋ੑೳ޲্Λୡ੒ͨ͠ɽ
    w'-ͷ৚݅ )'- 7-' '5-
    ΛλεΫ΍σʔληοτʹରͯ͠ߟ͑Δ͜ͱʹΑΓɼॊ
    ೈʹఏҊख๏͸ରԠͰ͖Δɽ
    wେ͖ͳσʔληοτΑΓ΋খ͞ͳσʔληοτ͸ɼΑΓޮՌతͩͱ͍͏݁Ռ͕ݱΕ
    ͨɽ
    ʲײ૝ʳ
    wͦ΋ͦ΋.5-ʹ͓͚ΔEBUBMFBLBHF͸WJTJPOBOEMBOHVBHFHSPVOEJOH
    QSPCMFNͷΑ͏ͳɼ΍΍ಛघͳλεΫʹͷΈଘࡏ͢ΔͷͰɼ͍͢͝൚༻తͳख๏Ͱ
    ͸ͳ͘ɼγϯϓϧͳઃఆͳΒλεΫΛ෼ׂ͠ͳ͍ํ͕͍͍ͩΖ͏ɽ
    w'-ͷΞϧΰϦζϜΛผͷλεΫʹԠ༻ΑΓ΋ɼ७ਮͳ'-ͷݚڀͷ΄͏͕ࣗ෼͸ڵ
    ຯ͕͋Δͱࢥͬͨɽ
    w'FEFSBUFE-FBSOJOH͕λΠτϧͷओޠ͸͓͔͍͠

    View Slide