[読み会資料] Federated Learning for Vision-and-Language Grounding Problems

ゼミ資料．
Federated Learning for Vision-and-Language Grounding Problems(AAAI'20)の紹介．

Transcript

1. Naoki Katsura
   Federated Learning
   for Vision-and-Language
   Grounding Problems
   2020-03-10
   http://web.pkusz.edu.cn/adsp/files/2019/11/AAAI-FenglinL.1027.pdf
   

   View Slide

2. ࿦จ৘ใ
   ʲஶऀʳ
   Fenglin Liu(Penking Univ), Xian Wu(Tencent), Shen Ge(Tencent),
   Wei Fan(Tencent), Yuexian Zou(Penking Univ & Peng Cheng Laboratory)
   ʲग़యʳ
   AAAI 2020
   ʲͳͥɼ͜ͷ࿦จΛબΜͩʁʳ
   ɹ'-ͷϑϨʔϜϫʔΫΛผͷλεΫʹҠ২ͨ͜͠ͱʹ໘നΈΛײ͔ͨ͡Βɽ
   ʲࡶஊʳ
   ୈҰஶऀ'-JVͷ೥ͷઓ੷ʢୈҰஶऀͷ࿦จʣ
   w /FVS*14 "DDFQUFE
   w *+$"* "DDFQUFE
   w *$%. "DDFQUFE
   ⟵
   ⟵
   ⟵
   

   View Slide

3. ༻ޠ
   7JTJPOBOE-BOHVBHF(SPVOEJOH1SPCMFN
   ɹࣗવݴޠॲཧͱίϯϐϡʔλϏδϣϯʹ·͕ͨΔ໰୊Λಉ࣌ʹॲཧ͢Δ໰୊ɽ
   ྫ
   image captioning, visual question answer(VQA), image caption retrieval, etc…
   )PSJ[POUBM'FEFSBUFE-FBSOJOH )'-
   
   ɹTBNQMFCBTFEGFEFSBUFEMFBSOJOHͱ΋஌ΒΕ͍ͯΔɽಛ௃ۭؒ͸ಉ͕ͩ͡ɼα
   ϯϓϧʢϢʔβʣू߹͸ҟͳΔΑ͏ͳσʔλΛ༻͍Δ'-໰୊ɽ
   7FSUJDBM'FEFSBUFE-FBSOJOH 7'-
   
   ɹGFBUVSFCBTFEGFEFSBUFEMFBSOJOHͱ΋஌ΒΕ͍ͯΔɽαϯϓϧʢϢʔβʣू߹
   ͸ಉ͕ͩ͡ɼಛ௃ۭؒ͸ҟͳΔΑ͏ͳσʔλΛ༻͍Δ'-໰୊ɽ
   'FEFSBUFE5SBOTGFS-FBSOJOH '5-
   
   ɹಛ௃ۭؒ΋αϯϓϧʢϢʔβʣू߹΋ҟͳΔΑ͏ͳσʔλΛ༻͍Δ'-໰୊ɽ
   )'- 7'- '5-ͳͲͷৄࡉ͸ɼҎԼͷ࿦จʹॻ͍ͯ͋Δɽ
   “Federated Learning machine learning: Concept and applications”
   

   View Slide

4. image caption retrieval
   ग़యɿDual Attention Networks for Multimodal Reasoning and Matching
   

   View Slide

5. ͲΜͳ࿦จʁ
   ɹ7JTJPOBOE-BOHVBHF(SPVOEJOH1SPCMFNʹ͓͍ͯɼ
   NVMUJUBTLMFBSOJOHʢ.5-ʣΛద༻ͨ͠৔߹ɼEBUBMFBLBHF
   ͱ͍͏໰୊͕ੜ͡Δɽ
   ɹ'FEFSBUFE-FBSOJOHͷ࿮૊ΈΛ࢖͍ɼ֤λεΫΛ'-ʹ͓͚
   ΔݸʑͷΫϥΠΞϯτͱΈͳ͢ͱɼλεΫ͝ͱͷσʔλΛ׬શ
   ʹ෼཭͠ͳ͕ΒϞσϧΛֶशͰ͖ΔΜ͡Όͳ͍͔ʁɹ
   ͱ͍͏ఏҊɽ
   

   View Slide

6. എܠ
   ɹJNBHFDBQUJPOJOHͱ72"͸ɼͲͪΒ΋ը૾ͱࣗવݴޠΛѻ͏ͱ͍͏఺ͰλεΫ
   ͕ࣅ͍ͯΔɽ
   

   View Slide

7. എܠ
   ɹJNBHFDBQUJPOJOHͱ72"͸ɼͲͪΒ΋ը૾ͱࣗવݴޠΛѻ͏ͱ͍͏఺ͰλεΫ
   ͕ࣅ͍ͯΔɽ
   ɹλεΫΛ߹ΘͤΕ͹ɼҟͳΔλεΫ͔ΒҟͳΔ஌ࣝΛ֫ಘͰ͖ΔͷͰੑೳ޲্͕ݟ
   ࠐΊΔΜ͡Όͳ͍ʁ
   .VMUJUBTLMFBSOJOHΛద༻ɹ -JFUBM
   
   ͔͠͠ɾɾɾ
   

   View Slide

8. എܠ
   ɹJNBHFDBQUJPOJOHͱ72"͸ɼͲͪΒ΋ը૾ͱࣗવݴޠΛѻ͏ͱ͍͏఺ͰλεΫ
   ͕ࣅ͍ͯΔɽ
   ɹλεΫΛ߹ΘͤΕ͹ɼҟͳΔλεΫ͔ΒҟͳΔ஌ࣝΛ֫ಘͰ͖ΔͷͰੑೳ޲্͕ݟ
   ࠐΊΔΜ͡Όͳ͍ʁ
   .VMUJUBTLMFBSOJOHΛద༻ɹ -JFUBM
   
   ͔͠͠ɾɾɾ
   ɹҟͳΔλεΫؒͰ΋Ϟσϧޙ൒෦·Ͱ͸ॲཧΛڞ༗͍ͯ͠Δɽ
   ɹ ݁ՌɼEBUBMFBLBHF໰୊ͷൃੜ
   ⟹
   

   View Slide

9. [Li et al. , 2018]
   ʲ໰୊఺ʳ
   ɹ͜ͷ৚݅ԼͰ͸ɼλεΫؒͰσʔληοτ͕ҟͳΔͱద༻Ͱ͖ͳ͍ɽʢ7'-ʣ
   ɹશλεΫʹ͓͍ͯೖྗΛ౷Ұ͠ͳ͍ͱ͍͚ͳ͍ɽ
   .5-༻ͷಛผͳσʔληοτΛ࡞Δඞཁ͕͋Δɽ
   ⟹
   

   View Slide

10. [Nguyen & Okatani , 2019]
    ɹλεΫؒͰσʔληοτΛ
    ౷Ұ͠ͳ͍ͱ͍͚ͳ͍໰୊Λ
    ؇࿨ͨ͠΋ͷɽ
    ɹ72"ͳͲʹ͸ೖྗʹςΩετ
    ͕ඞਢɽ
    ɹ*NBHFDBQUJPOJOHͷΑ͏ͳೖ
    ྗ͕ը૾ͷΈͷͱ͖ɼ͋·Γྑ͍
    ݁Ռ͕ಘΒΕͳ͍ɽ
    ʢEBUBMFBLBHFೖྗςΩετ
    
    ͔͠͠
    ͕ͨͬͯ͠
    

    View Slide

11. ఏҊ
    ɹEBUBMFBLBHFΛ๷͗ͳ͕ΒλεΫؒͰಛ௃Λڞ༗͢ΔͨΊʹɼ'-ͷ࿮૊ΈΛར༻
    ͨ͠ɹ"MJHOJOH *OUFHSBUJOHBOE.BQQJOH/FUXPSLʢBJN/FUʣͷఏҊɽ
    ɹ༷ʑͳ৚݅Λ)'- 7'- '5-ͷ࿮ʹ౰ͯ͸ΊΔ͜ͱͰॊೈʹରԠՄೳɽ
    ͜ͷ࿦จʹ͓͚Δߩݙ͸ɼҎԼͷ௨Γɽ
    ᶃ '-ͷ࿮૊ΈΛ࢖ͬͯɼҟͳΔλεΫ΍σʔληοτΛEBUBMFBLBHFΛ๷͗ͳ͕Β
    ར༻͢Δ͜ͱͰɼ୯ମͷλεΫΑΓ΋ྑ͍ಛ௃ΛಘΔ͜ͱʹ੒ޭͨ͜͠ͱɽ
    ᶄ '-ʹ͓͚ΔDFOUSBMJ[FENPEFMͱͯ͠ɼೖྗը૾͔ΒWJTVBMͱUFYUVBMಛ௃Λநग़
    ͢ΔBJN/FUΛఏҊͨ͜͠ͱɽ
    ᶅ )PSJ[POUBM'FEFSBUFE-FBSOJOH 7FSUJDBM'FEFSBUFE-FBSOJOH 'FEFSBUFE
    5SBOTGFS-FBSOJOHͷ৚݅Լͷ্Ͱɼ༗ޮੑ͕ࣔ͞Εͨ͜ͱɽ
    

    View Slide

12. ఏҊख๏
    ʲఏҊख๏ͷߏ੒ʳ
    ᶃ UIFWJTVBMBOEUFYUVBMGFBUVSFTFYUSBDUPS
    ೖྗը૾͔ΒɼΑΓϦονͳදݱΛಘΔಛ௃ྗநग़ثʢ৽نੑ͸ͳ͍ʣ
    ᶄ "MJHOJOH *OUFHSBUJOHBOE.BQQJOH/FUXPSLBJN/FU
    WJTVBMUFYUVBMGFBUVSFΛΑΓϦονͳදݱʹม׵͢ΔωοτϫʔΫɽλεΫ͓
    ΑͼσʔληοτؒͰڞ༗ɽ
    ᶅ UIFJNQMFNFOUBUJPOJOUISFFGFEFSBUFEMFBSOJOHTFUUJOHT
    )'- 7'- '5-ͷঢ়گʹ౰ͯ͸ΊΔ͜ͱͰɼλεΫ΍σʔληοτʹؔ͢Δঢ়گ
    ʹఏҊख๏͕ࠨӈ͞Εͳ͍͜ͱΛࣔ͢ɽ
    

    View Slide

13. ఏҊख๏
    ʲఏҊख๏ͷߏ੒ʳ
    ᶃ UIFWJTVBMBOEUFYUVBMGFBUVSFTFYUSBDUPS
    ೖྗը૾͔ΒɼΑΓϦονͳදݱΛಘΔಛ௃ྗநग़ثʢ৽نੑ͸ͳ͍ʣ
    ᶄ "MJHOJOH *OUFHSBUJOHBOE.BQQJOH/FUXPSLBJN/FU
    WJTVBMUFYUVBMGFBUVSFΛΑΓϦονͳදݱʹม׵͢ΔωοτϫʔΫɽλεΫ͓
    ΑͼσʔληοτؒͰڞ༗ɽ
    ᶅ UIFJNQMFNFOUBUJPOJOUISFFGFEFSBUFEMFBSOJOHTFUUJOHT
    )'- 7'- '5-ͷঢ়گʹ౰ͯ͸ΊΔ͜ͱͰɼλεΫ΍σʔληοτʹؔ͢Δঢ়گ
    ʹఏҊख๏͕ࠨӈ͞Εͳ͍͜ͱΛࣔ͢ɽ
    

    View Slide

14. ɹWJTJPOBOEMBOHVBHFHSPVOEJOHUBTLʹ͓͍ͯɼ$//Λ༻͍ͯը૾ͷಛ௃Λநग़
    ͢Δ͜ͱ͸޿͘࢖ΘΕ͍ͯΔɽຊ࿦จͰ͸ɼ'BTUFS3$//Λ࢖༻ͯ͠ɼ
    ಛ௃ʢWJTVBMGFBUVSFʣͷநग़Λߦͬͨɽ
    ͔͠͠
    ɹWJTVBMGFBUVSFͷΈͩͱɼ͜ͷλεΫʹ༗ޮͳಛ௃ͷநग़ʹݶք͕͋Δɽ
    ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹʢ8VFUBM ʣ
    ɹUFYUVBMGFBUVSFΛ࢖͏ͱɼΑΓ༗ޮͳಛ௃Λը૾͔Β໌֬ʹநग़Ͱ͖Δɽ
    ɹ'BTUFS3$//ʹΑΔಛ௃ྔநग़ʹՃ͑ͯɼFUBM >ʹैͬͯɼ
    .VMUJQMF*OTUBODF-FBSOJOHʢ;IBOHFUBM ʣΛద༻͠ɼUFYUVBMGFBUVSFΛ
    நग़ͨ͠ɽ
    Visual and Textual Feature
    

    View Slide

15. Visual and Textual Feature
    ʲΠϝʔδʳ
    ೖྗը૾
    .VMUJQMF*OTUBODF
    -FBSOJOH
    'BTUFS3$//
    WJTVBMGFBUVSFFYUSBDUPS
    UFYUVBMGFBUVSFFYUSBDUPS
    &NCFEEJOH
    ⃗
    I = { ⃗
    i1
    , ⃗
    i2
    , …, ⃗
    iN
    } ∈ ℝN×d
    ⃗
    T = { ⃗
    w1
    , ⃗
    w2
    , …, ⃗
    wM
    } ∈ ℝM×d
    ɹࠨͷΑ͏ͳಘΒΕͨ୯ޠ܈ΛTFNBOUJDDPODFQU
    ͱ͍͍ɼ෺ମʢEPH΍GSJTCFFʣɼঢ়ଶʢP⒎΍
    FMFDUSJDʣɼؔ܎ੑʢIPMEJOH qZJOHʣؚ͕·ΕΔɽ
    QSPWJEFECZ"OEFSTPOFUBM
    
    QSPWJEFECZ'BOHFUBM
    
    ͜ͷFNCFEEJOH૚͸ɼ
    DBQUJPORVFTUJPOͱڞ༗ɽ
    Experiment Setting
    ɹXPSEFNCFEEJOH͸ɼDBQUJPORVFTUJPOͱڞ༗ɽ ͸Ҏ
    ߱Ͱ঺հ͢Δ#BTFMJOF طଘݚڀͷϞσϧ
    ͷσίʔμͷೖྗ
    ࣍ݩʹ͢Δɽ
    d
    N = M = 36.
    

    View Slide

16. ఏҊख๏
    ʲఏҊख๏ͷߏ੒ʳ
    ᶃ UIFWJTVBMBOEUFYUVBMGFBUVSFTFYUSBDUPS
    ೖྗը૾͔ΒɼΑΓϦονͳදݱΛಘΔಛ௃ྗநग़ثʢ৽نੑ͸ͳ͍ʣ
    ᶄ "MJHOJOH *OUFHSBUJOHBOE.BQQJOH/FUXPSLBJN/FU
    WJTVBMUFYUVBMGFBUVSFΛΑΓϦονͳදݱʹม׵͢ΔωοτϫʔΫɽλεΫ͓
    ΑͼσʔληοτؒͰڞ༗ɽ
    ᶅ UIFJNQMFNFOUBUJPOJOUISFFGFEFSBUFEMFBSOJOHTFUUJOHT
    )'- 7'- '5-ͷঢ়گʹ౰ͯ͸ΊΔ͜ͱͰɼλεΫ΍σʔληοτʹؔ͢Δঢ়گ
    ʹఏҊख๏͕ࠨӈ͞Εͳ͍͜ͱΛࣔ͢ɽ
    

    View Slide

17. Aligning, Integrating and Mapping Network : aimNet
    ɹBJN/FU͸ɼҎԼͷͭͷϞδϡʔϧͰߏ੒͞ΕΔɽ͜Ε͸ɼ֤λεΫʹ͓͚Δը૾
    ͷΤϯίʔμͷ෦෼ʹ૬౰͢Δɽ
    w"MJHOJOHNPEVMF
    ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ
    w*OUFHSBUJOHNPEVMF
    ɹWJTVBMGFBUVSF಺ɼ͓ΑͼUFYUVBMGFBUVSF಺Ͱͷؔ܎ੑΛରԠ͚ͮΔɽ
    w.BQQJOH.PEVMF
    ɹλεΫ͝ͱͷೖྗαΠζʹม׵͢Δɽ
    

    View Slide

18. Aligning, Integrating and Mapping Network : aimNet
    "MJHOJOH.PEVMFͱ*OUFHSBUJOH.PEVMF͸ɼ࣍ͷ#BTJD.PEVMFΛϕʔεͱ͍ͯ͠
    Δɽ
    ʲ#BTJD.PEVMFʳ
    ɹ֤ಛ௃ؒͷؔ܎ੑΛଊ͑ΔͨΊʹɼ.VMUJ)FBE"UUFOUJPOʢ.)"ʣͱ'FFE
    'PSXBSE/FUXPSLʢ''/ʣΛద༻ɽ͜ΕΛ࢖͏͜ͱͰɼಛ௃ΛҰରҰͰ͸ͳ͘ɼ
    ଟରଟͰಛ௃ؒͷؔ܎ੑͷڧ͞ΛଌΔ͜ͱ͕Ͱ͖Δɽ
    ɹ
    ͜ͷΞΠσΞΛɼ
    WJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛଊ͑Δ
    ɹ "MJHOJOH.PEVMF
    WJTVBMGFBUVSFಉ࢜ɼ·ͨ͸ɼUFYUVBMGFBUVSFಉ࢜ͷؔ܎ੑΛଊ͑Δ
    ɹ *OUFHSBUJOH.PEVMF
    ⟹
    ⟹
    

    View Slide

19. Aligning, Integrating and Mapping Network : aimNet
    w"MJHOJOHNPEVMF
    ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ
    w*OUFHSBUJOHNPEVMF
    ɹWJTVBMGFBUVSF಺ɼ͓ΑͼUFYUVBMGFBUVSF಺Ͱͷؔ܎ੑΛରԠ͚ͮΔɽ
    w.BQQJOH.PEVMF
    ɹλεΫ͝ͱͷೖྗαΠζʹม׵͢Δɽ
    

    View Slide

20. Aligning, Integrating and Mapping Network : aimNet
    ʲ"MJHOJOH.PEVMFʳ
    ɹ΍͍ͬͯΔ͜ͱ͸ɼ
    ɹɹ4PVSDF5BSHFU"UUFOUJPOͱಉ͡ɽ
    ɹɹ
    ɹɹ
    ɹ͜Ε͸72"Ͱ༗ޮͱ͍͏͜ͱ͕ࣔ͞Ε͍ͯΔɽʢ4VFUBMʣ
    ɹը૾ͱΞϥΠϝϯτΛऔΔ͜ͱͰɼಉ͡εϖϧͰ΋ෳ਺ͷҙຯΛ࣋ͭΑ͏ͳᐆດͳ
    දݱΛ཈͑ࠐΉ͜ͱ͕Ͱ͖ΔɽʢྫɿNPVTFͶͣΈɼిࢠػثͷϚ΢εʣ
    ⃗
    Ia
    = FFN(MHA( ⃗
    I, ⃗
    T , ⃗
    T ))
    ⃗
    T a
    = FFN(MHA( ⃗
    T , ⃗
    I, ⃗
    I))
    Experiment Setting
    ɹBUUFOUJPOIFBEͷݸ਺͸ɼGFFEGPSXBSEOFUXPSLͷ࣍
    ݩ਺͸
    

    View Slide

21. Aligning, Integrating and Mapping Network : aimNet
    w"MJHOJOHNPEVMF
    ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ
    w*OUFHSBUJOHNPEVMF
    ɹWJTVBMGFBUVSF಺ɼ͓ΑͼUFYUVBMGFBUVSF಺Ͱͷؔ܎ੑΛରԠ͚ͮΔɽ
    w.BQQJOH.PEVMF
    ɹλεΫ͝ͱͷೖྗαΠζʹม׵͢Δɽ
    

    View Slide

22. Aligning, Integrating and Mapping Network : aimNet
    ʲ*OUFHSBUJOH.PEVMFʳ
    ɹ΍͍ͬͯΔ͜ͱ͸ɼ
    ɹɹ4FMG"UUFOUJPOͱಉ͡ɽ
    ɹɹ
    ɹɹ
    ɹ͜Ε͸*NBHF$BQUJPOJOHͰ༗ޮͱ͍͏͜ͱ͕ࣔ͞Ε͍ͯΔɽʢ:BPFUBMʣ
    ⃗
    Ii
    = FFN(MHA( ⃗
    Ia
    , ⃗
    Ia
    , ⃗
    Ia
    ))
    ⃗
    T i
    = FFN(MHA( ⃗
    T a
    , ⃗
    T a
    , ⃗
    T a
    ))
    Experiment Setting
    ɹBUUFOUJPOIFBEͷݸ਺͸ɼGFFEGPSXBSEOFUXPSLͷ࣍
    ݩ਺͸
    

    View Slide

23. Aligning, Integrating and Mapping Network : aimNet
    w"MJHOJOHNPEVMF
    ɹWJTVBMGFBUVSFͱUFYUVBMGFBUVSFؒͷؔ܎ੑΛରԠ͚ͮΔɽ
    w*OUFHSBUJOHNPEVMF
    ɹWJTVBMGFBUVSF಺ɼ͓ΑͼUFYUVBMGFBUVSF಺Ͱͷؔ܎ੑΛରԠ͚ͮΔɽ
    w.BQQJOH.PEVMF
    ɹλεΫ͝ͱͷೖྗαΠζʹม׵͢Δɽ
    

    View Slide

24. Aligning, Integrating and Mapping Network : aimNet
    ʲ.BQQJOH.PEVMFʳ
    ɹҎԼͷΑ͏ʹɼ.BQQJOHؔ਺Λ׆ੑԽؔ਺͕UBOIͷ૚ͷχϡʔϥϧωοτͱ͠
    ͯఆٛ͢Δɽ
    ɹɹɹɹɹɹɹɹɹɹ
    ɹ.BQQJOH.PEVMFͰ͸ɼλεΫ͝ͱͷσίʔμͷೖྗۭؒʹલͭͷϞδϡʔϧ͔
    ΒಘΒΕͨಛ௃Λ߹ΘͤΔͨΊʹҎԼͷΑ͏ͳม׵Λߦ͏ɽ
    ɹɹɹɹɹɹɹɹɹɹ
    ɹ͜ͷϞδϡʔϧ͸ɼҰൠతʹطଘख๏ͷଟ͘Ͱ࢖༻͞Ε͍ͯΔɽ#BTFMJOFͰ͸ɼ
    $//ͳͲʢFH'BTUFS3$//ʣ͔Βͷಛ௃ϚοϓΛ͜ͷϞδϡʔϧʹద༻͢Δɽ
    Mapping(x) = tanh(xWm
    + bm
    )Wmm
    + bmm
    LayerNorm(Mapping( ⃗
    It
    ) + Mapping( ⃗
    T t
    ))
    

    View Slide

25. ఏҊख๏
    ʲఏҊख๏ͷߏ੒ʳ
    ᶃ UIFWJTVBMBOEUFYUVBMGFBUVSFTFYUSBDUPS
    ೖྗը૾͔ΒɼΑΓϦονͳදݱΛಘΔಛ௃ྗநग़ثʢ৽نੑ͸ͳ͍ʣ
    ᶄ "MJHOJOH *OUFHSBUJOHBOE.BQQJOH/FUXPSLBJN/FU
    WJTVBMUFYUVBMGFBUVSFΛΑΓϦονͳදݱʹม׵͢ΔωοτϫʔΫɽλεΫ͓
    ΑͼσʔληοτؒͰڞ༗ɽ
    ᶅ UIFJNQMFNFOUBUJPOJOUISFFGFEFSBUFEMFBSOJOHTFUUJOHT
    )'- 7'- '5-ͷঢ়گʹ౰ͯ͸ΊΔ͜ͱͰɼλεΫ΍σʔληοτʹؔ͢Δঢ়گ
    ʹఏҊख๏͕ࠨӈ͞Εͳ͍͜ͱΛࣔ͢ɽ
    

    View Slide

26. Implementation
    ɹ͞·͟·ͳλεΫ΍σʔληοτͷঢ়گԼͰ૊Έ߹ΘֶͤͨशΛߦ͏ɽ'-ʹ͓͚
    Δͭͷঢ়گΛ.5-తͳղऍΛ͢Δɽ'-ʹ͓͚ΔΫϥΠΞϯτΛλεΫɼಛ௃ۭؒΛ
    σʔληοτͱΈͳͨ͠ɽ
    ɹධՁࢦඪ͸ɼ
    *NBHF$BQUJPOJOH41*$& $*%&S .&5&03 #-&6
    72"UFTUTUBOEBSETFU
    '-ʹ͓͚Δঢ়گ λεΫ σʔληοτ
    )'-
    ಉ͡
    Image Captioning)
    ҟͳΔ
    MSCOCO & Flickr30k
    
    7'-
    ҟͳΔ
    Image Captioning & VQA
    
    ಉ͡
    MSCOCO & VQA v2.0
    
    '5-
    ҟͳΔ
    Image Captioning & VQA
    
    ҟͳΔ
    Flickr30k & VQA v2.0
    
    ˞72"Wͷೖྗը૾͸.4$0$0ͷσʔλͰߏ੒͞Ε͍ͯΔ
    

    View Slide

27. Implementation : Horizontal Federated Learning
    Experiment Setting
    Baseline(Image Captioning Decoder)
    • Spatial ( Lu et al. 2017 )
    • NBT ( Lu et al. 2017 )
    

    View Slide

28. Implementation : Vertical Federated Learning
    Experiment Setting
    Baseline(Image Captioning Decoder)
    • Spatial ( Lu et al. 2017 )
    • NBT ( Lu et al. 2017 )
    Baseline(Visual Question Answering)
    • BUTB ( Anderson et al. 2018 )
    • NBT ( Kim, Jun and Zhang 2018 )
    

    View Slide

29. Implementation : Federated Transfer Learning
    Experiment Setting
    Baseline(Image Captioning Decoder)
    • Spatial ( Lu et al. 2017 )
    • NBT ( Lu et al. 2017 )
    Baseline(Visual Question Answering)
    • BUTB ( Anderson et al. 2018 )
    • NBT ( Kim, Jun and Zhang 2018 )
    

    View Slide

30. Result : Horizontal Federated Learning
    

    View Slide

31. Result : Vertical Federated Learning
    

    View Slide

32. Result : Federated Transfer Learning
    

    View Slide

33. Result : Ablation study
    'BTUFS3$//.BQQJOH.PEVMF4QBUJBM#65%
    ೋͭͷλεΫΛ૊Έ߹Θֶͤͨश
    ɹ*NBHF$BQUJPOJOHʹ͸*OUFHSBUJOHɼ
    72"ʹ͸"MJHOJOH͕ޮ͍͍ͯΔ͜ͱ͕Θ
    ͔Δɽ
    

    View Slide

34. ·ͱΊ
    wEBUBMFBLBHFͷ໰୊Λ๊͍͑ͯͨWJTJPOBOEMBOHVBHFHSPVOEJOHQSPCMFNʹ͓
    ͚Δ.5-ʹ'-ͷϑϨʔϜϫʔΫΛద༻͢Δ͜ͱͰɼ໰୊ͷղܾͱͭͷλεΫɼ·
    ͨ͸σʔληοτΑΓ΋ੑೳ޲্Λୡ੒ͨ͠ɽ
    w'-ͷ৚݅ )'- 7-' '5-
    ΛλεΫ΍σʔληοτʹରͯ͠ߟ͑Δ͜ͱʹΑΓɼॊ
    ೈʹఏҊख๏͸ରԠͰ͖Δɽ
    wେ͖ͳσʔληοτΑΓ΋খ͞ͳσʔληοτ͸ɼΑΓޮՌతͩͱ͍͏݁Ռ͕ݱΕ
    ͨɽ
    ʲײ૝ʳ
    wͦ΋ͦ΋.5-ʹ͓͚ΔEBUBMFBLBHF͸WJTJPOBOEMBOHVBHFHSPVOEJOH
    QSPCMFNͷΑ͏ͳɼ΍΍ಛघͳλεΫʹͷΈଘࡏ͢ΔͷͰɼ͍͢͝൚༻తͳख๏Ͱ
    ͸ͳ͘ɼγϯϓϧͳઃఆͳΒλεΫΛ෼ׂ͠ͳ͍ํ͕͍͍ͩΖ͏ɽ
    w'-ͷΞϧΰϦζϜΛผͷλεΫʹԠ༻ΑΓ΋ɼ७ਮͳ'-ͷݚڀͷ΄͏͕ࣗ෼͸ڵ
    ຯ͕͋Δͱࢥͬͨɽ
    w'FEFSBUFE-FBSOJOH͕λΠτϧͷओޠ͸͓͔͍͠
    

    View Slide