Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deep Neural Networkの共同学習

Deep Neural Networkの共同学習

2023年2月2日
知識転移グラフによる深層共同学習
知識転移グラフによる最適な半教師あり学習の探索
素人発想玄人実行2.0

Hironobu Fujiyoshi

February 02, 2023
Tweet

More Decks by Hironobu Fujiyoshi

Other Decks in Technology

Transcript

  1. %FFQ/FVSBM/FUXPSLͷڞಉֶश
    ౻٢߂࿱ʢத෦େֶɾػց஌֮ϩϘςΟΫεάϧʔϓʣ
    IUUQNQSHKQ

    View full-size slide

  2. ೥݄ɿۚग़෢༤ઌੜ͔Β௖͍͓ͨݴ༿
    2
    த෦େֶϩΰ
    த෦େֶϩΰ

    View full-size slide

  3. w ʮൃ૝͸୯७ɼૉ௚ɼࣗ༝ɼ؆୯
    で
    ͳ͚Ε
    ば
    ͳΒͳ͍ɽ͔͠͠ɼൃ૝Λ࣮ߦʹҠ͢ʹ͸஌ࣝ
    が
    ͍Δɼख़࿅͞Εٕͨ
    が
    ͍Δʯʢۚग़෢༤ʣ
    ʮૉਓൃ૝ݰਓ࣮ߦʯͱ͸ʁ
    3
    த෦େֶϩΰ
    த෦େֶϩΰ
    IUUQTXXXBNB[PODPKQEQ#11428/
    IUUQNQSHKQ5,CPPL

    View full-size slide

  4. w ʮݴ͏͸қ͘ɼߦ͏͸೉͠ʯͷయܕ
    w ೥͔Β໿೥Λܦͯʮૉਓൃ૝ݰਓ࣮ߦʯ͚ۙͮͨݚڀ
    ඇ౳ํੑ-P(ϑΟϧλʹΑΔෳ਺ͷΞϑΟϯྖҬͷਪఆ<)BTFHBXB *$$7>
    ஌ࣝసҠ
    グ
    ϥϑʹΑΔڞಉֶश<.JOBNJ "$$7> <0LBNPUP &$$7>
    ݚڀϞοτʔʮૉਓൃ૝ݰਓ࣮ߦʯ
    4
    த෦େֶϩΰ
    த෦େֶϩΰ
    ࿦จ"
    ࿦จ#
    ࿦จ$
    ࿦จ%
    ࿦จ"
    ࿦จ#
    ࿦จ$
    ࿦จ%
    ࿦จ&
    ଟ͘ͷ࿦จΛਂ͘ಡΜͰ͍͘ͱ
    ຊ࣭తͳݚڀ
    ຊ࣭Ͱͳ͘খ͞ͳ͜ͱ
    ʹண໨ͨ͠ݚڀ
    ݚڀۭؒ

    View full-size slide

  5. ਂ૚ֶशͷωοτϫʔΫߏ଄
    த෦େֶϩΰ
    த෦େֶϩΰ
    w *-473$ʹ͓͚ΔωοτϫʔΫߏ଄ͷมભ
    2012
    SuperVision GoogLeNet
    Konvolüzasyon
    Pooling
    Softmax
    Diğer
    [Krizhevsky NIPS 2012] [Szegedy arxiv 2014]-22 [Sim
    "MFY/FU MBZFST
    *-473$

    2014
    GoogLeNet
    Konvolüzasyon
    Pooling
    Softmax
    Diğer
    VGG MSRA
    [Szegedy arxiv 2014]-22 [Simonyan arxiv 2014] -19 [He arxiv 2014]
    n
    2014
    GoogLeNet
    Konvolüzasyon
    Pooling
    Softmax
    Diğer
    VGG MSRA
    35/36 t
    derin öğ
    kullanm
    20/36 t
    açık-kay
    Caffe
    uygula
    kullanm
    012] [Szegedy arxiv 2014]-22 [Simonyan arxiv 2014] -19 [He arxiv 2014]
    7(( MBZFST
    *-473$

    (PPHMF/FU MBZFST
    *-473$

    14
    3FT/FU MBZFST
    *-473$


    ˠ໿ ສύϥϝʔλ
    ˠ໿ԯύϥϝʔλ
    ˠ໿ ສύϥϝʔλ

    View full-size slide

  6. ,OPXMFEHF%JTUJMMBUJPO ,%
    <()JOUPO >
    த෦େֶϩΰ
    த෦େֶϩΰ
    w ஌ࣝৠཹ
    ֶशࡁΈ-BSHFωοτϫʔΫ͔Β4NBMMωοτϫʔΫʹ஌ࣝసҠ
    ੑೳΛอͪͭͭɺύϥϝʔλ਺ͱܭࢉίετΛ࡟ݮՄೳ

    5FBDIFS
    /FUXPSL
    4UVEFOU
    /FUXPSL
    -BSHF

    QSFUSBJOFE
    4NBMM
    ,OPXMFEHFUSBOTGFS
    ஌ࣝৠཹ
    ஌ࣝ

    View full-size slide

  7. ,OPXMFEHF%JTUJMMBUJPO ,%
    <()JOUPO >
    த෦େֶϩΰ
    த෦େֶϩΰ
    w ஌ࣝৠཹɿ5FBDIFSˠ4UVEFOU
    ֶशࡁΈ-BSHFωοτϫʔΫ͔Β4NBMMωοτϫʔΫʹ஌ࣝసҠ
    ੑೳΛอͪͭͭɺύϥϝʔλ਺ͱܭࢉίετΛ࡟ݮՄೳ
    ֶशํ๏ɿ)BSEUBSHFUͱ4PGUUBSHFUͰ4UVEFOUωοτϫʔΫΛֶश

    %BSL,OPXMFEHFʢӅΕͨ஌ࣝʣ
    5FBDIFS
    4UVEFOU
    $SPTT&OUSPQZ
    $SPTT&OUSPQZ
    MBCFM
    QSFUSBJOFE

    4PGUUBSHFU
    )BSEUBSHFU
    ʢਖ਼ղϥϕϧʣ
    p1
    p2
    ʢ֬཰෼෍ʣ
    #BDLQSPQ

    View full-size slide

  8. %FFQ.VUVBM-FBSOJOH %.-
    <:;IBOH >
    த෦େֶϩΰ
    த෦େֶϩΰ
    w 4UVEFOUωοτϫʔΫͷ૬ޓֶशɿ4UVEFOU⁶4UVEFOU
    ,VMMCBDL-FJCMFS ,-
    %JWFSHFODFͰωοτϫʔΫͷग़ྗ ͱ Λ͚ۙͮ߹͏
    p1
    p2

    MBCFM
    4UVEFOU
    4UVEFOU
    p1
    p2
    KL(p1
    ||p2
    )
    KL(p2
    ||p1
    )
    ˠྠߨͷΑ͏ͳֶशܗଶͰ͋Γɺ஌ࣝৠཹΑΓਫ਼౓͕޲্
    )BSE5BSHFU
    $SPTT&OUSPQZ

    4PGU5BSHFU
    ,-EJWFSHFODF

    View full-size slide

  9. %FFQ.VUVBM-FBSOJOH %.-
    <:;IBOH >
    த෦େֶϩΰ
    த෦େֶϩΰ
    w Կނ%.-͸͏·͍͘͘ͷ͔ʁ

    *OEFQFOEFOU
    Y Zd Y Zd
    %.-
    7*46"-*;*/(5)&-044-"/%4$"1&0'/&63"-/&54
    ˠ%.-͸޿͍୩ʹམ͍ͪͯΔ

    View full-size slide

  10. %FFQ.VUVBM-FBSOJOH %.-
    <:;IBOH >
    த෦େֶϩΰ
    த෦େֶϩΰ
    w ޿͍୩ʹམ͍ͪͯΔͱԿނྑ͍ͷ͔ʁ
    ύϥϝʔλ ʢ΋͘͠͸ೖྗ ʣ͕มಈͯ͠΋ଛࣦ͸มԽ͠ͳ͍ˠ൚Խೳྗ͕ߴ͍
    *OEFQFOEFOUɿଛࣦ͕খ͘͞ͳΔ୩ʹऩଋʢ޿͍୩ͱ͸ݶΒͳ͍ʣ
    %.-ɿೋͭͷωοτϫʔΫͷग़ྗ͕ࣅΔΑ͏ʹֶशˠ޿͍୩Λ୳͢
    ˠϩεͷϥϯυεέʔϓ͕ޯ഑ͷ؇΍͔ͳತؔ਺ʹ͖ۙͮਫ਼౓΋޲্
    w x

    -PTT
    y = f(x) y = wTx
    w
    ࣸ૾ؔ਺ɿχϡʔϥϧωοτϫʔΫ
    "
    #
    %.- %.- *OEFQFOEFOU

    View full-size slide

  11. ωοτϫʔΫؒͷ஌ࣝసҠ
    த෦େֶϩΰ
    த෦େֶϩΰ
    w ޿͍୩ʹམ͍ͪͯΔͱԿނྑ͍ͷ͔ʁ஌ࣝΛ఻͑Δ͜ͱͰೝࣝੑೳ͕޲্
    ஌ࣝৠཹɿ,OPXMFEHF%JTUJMMBUJPO<()JOUPO >
    ૬ޓֶशɿ%FFQ.VUVBM-FBSOJOH<:;IBOH >

    5FBDIFS
    4UVEFOU
    ,OPXMFEHF%JTUJMMBUJPO ,%

    4UVEFOU
    4UVEFOU
    %FFQ.VUVBM-FBSOJOH %.-

    p1
    p2
    p1
    p2

    View full-size slide

  12. ஌ࣝৠཹɾ૬ޓֶशͷ೿ੜख๏
    த෦େֶϩΰ
    த෦େֶϩΰ

    4UVEFOU
    4UVEFOU
    4UVEFOU
    4UVEFOU
    4UVEFOU
    4UVEFOU
    4UVEFOU
    5FBDIFS
    4UVEFOU
    5FBDIFS
    4UVEFOU
    5FBDIFS
    5"
    4UVEFOU
    ,OPXMFEHF%JTUJMMBUJPO
    %FFQ.VUVBM-FBSOJOH
    ,OPXMFEHF%JTUJMMBUJPO
    <()JOUPO >
    #PSO"HBJO
    <'VSMBOFMMP >
    5FBDIFS"TTJTUBOU
    <.JS[BEFI >
    %FFQ.VUVBM-FBSOJOH<:;IBOH >
    -BSHFϞσϧ4NBMMϞσϧ ಉҰαΠζ தؒϞσϧ͋Γ
    -BSHFϞσϧ4NBMMϞσϧ ಉҰαΠζ Ϟσϧ਺
    p1
    p2
    p1
    p2
    p1
    p2
    p1
    p2
    p1
    p2
    p3
    p1
    p2
    p3

    View full-size slide

  13. ஌ࣝৠཹɾ૬ޓֶशͷ೿ੜख๏
    த෦େֶϩΰ
    த෦େֶϩΰ

    ,OPXMFEHF%JTUJMMBUJPO
    <()JOUPO >
    #PSO"HBJO
    <'VSMBOFMMP >
    5FBDIFS"TTJTUBOU
    <.JS[BEFI >
    %FFQ.VUVBM-FBSOJOH<:;IBOH >
    ,OPXMFEHF%JTUJMMBUJPO
    %FFQ.VUVBM-FBSOJOH
    -BSHFϞσϧ4NBMMϞσϧ ಉҰαΠζ தؒϞσϧ͋Γ
    -BSHFϞσϧ4NBMMϞσϧ ಉҰαΠζ Ϟσϧ਺
    ਓ͕ઃܭͨ͠ݶఆతͳֶशํ๏

    View full-size slide

  14. ຊݚڀͷ໨ඪ
    த෦େֶϩΰ
    த෦େֶϩΰ

    w ڞಉֶशΛΫϥεϧʔϜεέʔϧ΁֦ு
    ଟ༷ੑͷߴ͍ڞಉֶशΛ࣮ݱ
    ʮૉਓൃ૝ʯ ˠڭࣨͰͷֶशͷΑ͏ʹઌੜ͔ΒͰͳ͘ଟ͘ͷੜె͕ෳࡶʹڭ͑͋͏ֶश

    View full-size slide

  15. w ڞಉֶशΛΫϥεϧʔϜεέʔϧ΁֦ு
    ଟ༷ੑͷߴ͍ڞಉֶशΛ࣮ݱ
    w ஌ࣝసҠάϥϑͷఏҊ<.JOBNJ "$$7>
    ैདྷख๏ ,%ͱ%.-
    Λ಺แͭͭ͠ɺ৽ֶ͍͠शํ๏ΛؚΉදݱํ๏
    ̐छྨͷ(BUFؔ਺ʹΑΓ஌ࣝసҠΛ੍ޚ͢Δ͜ͱͰଟ༷ͳڞಉֶश
    ,% %.-
    ຊݚڀͷ໨ඪ
    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    ஌ࣝసҠάϥϑ

    View full-size slide

  16. w άϥϑΛ༻͍ͯ,%ͱ%.-Λදݱ
    ϊʔυɿਂ૚ֶशϞσϧ
    Τοδɿ஌ࣝৠཹͷଛࣦ
    άϥϑදݱ΁ͷม׵
    த෦େֶϩΰ
    த෦େֶϩΰ

    ,OPXMFEHF%JTUJMMBUJPO
    ,%

    5FBDIFS
    4UVEFOU
    4UVEFOU
    4UVEFOU
    %FFQ.VUVBM-FBSOJOH
    %.-

    p1
    p2
    p1
    p2
    -BSHF 4NBMM
    m1
    m2
    ํ޲ͷΤοδ

    -BSHF 4NBMM
    m1
    m2
    ૒ํ޲ͷΤοδ

    άϥϑදݱ

    View full-size slide

  17. w ิॿϊʔυ͕ධՁର৅ϊʔυͷֶशΛαϙʔτ͢Δ
    ϊʔυɿਂ૚ֶशϞσϧ
    Τοδɿ஌ࣝৠཹͷଛࣦ
    ஌ࣝసҠάϥϑ ϊʔυ਺͕ͷ৔߹

    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    ਖ਼ղϥϕϧ
    ධՁର৅ϊʔυ
    3FT/FU
    ิॿϊʔυ
    3FT/FU
    8JEF3FT/FU
    %FOTF/FU

    View full-size slide

  18. w ֤ΤοδʹҟͳΔଛࣦؔ਺Λఆٛ
    ଛࣦؔ਺ͷ૊Έ߹ΘͤΛ୳ࡧ͢Δ͜ͱͰ৽ͨͳֶशํ๏Λ࣮ݱ
    ஌ࣝసҠάϥϑ ϊʔυ਺͕ͷ৔߹

    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    ଛࣦؔ਺
    𝐿
    =
    𝐻
    (
    𝑝
    ^
    𝑦
    ,
    𝑝
    𝑛
    )
    𝐿
    =
    𝐾
    𝐿
    (
    𝑝
    𝑛
    ||
    𝑝
    𝑚
    )
    𝐿
    = 0

    ˠଟ༷ͳ஌ࣝసҠΛදݱ͢ΔϑϨʔϜϫʔΫΛઃܭ
    ʮݰਓ࣮ߦᶃʯ

    View full-size slide

  19. w ϊʔυ̎ TPVSDF
    ͔Βϊʔυ̍ EFTUJOBUJPO
    ΁ͷ஌ࣝసҠ
    Τοδͷ஌ࣝసҠͷଛࣦܭࢉ
    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝑚
    1
    𝑚
    2
    𝐿
    2,1
    -PTT
    GVOD
    4PVSDF %FTUJOBUJPO
    𝑚
    2
    𝑚
    1
    p2
    (c|x) p1
    (c|x)
    L2,1
    (p2
    , p1
    )
    'PSXBSE

    View full-size slide

  20. w ϊʔυ̎ TPVSDF
    ͔Βϊʔυ̍ EFTUJOBUJPO
    ΁ͷ஌ࣝసҠ
    Τοδͷ஌ࣝసҠͷଛࣦܭࢉ
    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝑚
    1
    𝑚
    2
    𝐿
    2,1
    -PTT
    GVOD
    4PVSDF %FTUJOBUJPO
    𝑚
    2
    𝑚
    1
    L2,1
    (p2
    , p1
    )
    #BDLQSPQ
    %FUBDI
    #BDLXBSE

    View full-size slide

  21. w ϊʔυ̎ TPVSDF
    ͔Βϊʔυ̍ EFTUJOBUJPO
    ΁ͷ஌ࣝసҠ
    Τοδͷ஌ࣝసҠͷଛࣦܭࢉ
    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝑚
    1
    𝑚
    2
    𝐿
    2,1
    -PTT
    GVOD
    4PVSDF %FTUJOBUJPO
    𝑚
    2
    𝑚
    1
    L2,1
    (p2
    , p1
    )
    Gate
    KL div
    'PSXBSE
    p2
    (c|x) p1
    (c|x)

    View full-size slide

  22. w ͲͷΑ͏ʹ஌ࣝసҠ͢Δ͔Λ੍ޚ
    ήʔτؔ਺
    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝑚
    1
    𝑚
    2
    𝐿
    2,1
    4PVSDF %FTUJOBUJPO
    𝑚
    2
    𝑚
    1
    L2,1
    (p2
    , p1
    )
    'PSXBSE
    p2
    (c|x) p1
    (c|x)
    Gate
    KL div
    $VUPGG(BUF
    -JOFBS(BUF
    5ISPVHI(BUF
    $PSSFDU(BUF

    View full-size slide

  23. w ೖྗ͞Εͨ஋Λͦͷ··ग़ྗ͢Δ
    ήʔτؔ਺ɿ5ISPVHI(BUF
    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝑚
    1
    𝑚
    2
    𝐿
    2,1
    4PVSDF %FTUJOBUJPO
    𝑚
    2
    𝑚
    1
    L2,1
    (p2
    , p1
    )
    $VUPGG(BUF
    -JOFBS(BUF
    $PSSFDU(BUF
    5ISPVHI(BUF
    𝐺
    (
    𝐷
    𝐾 𝐿
    ) =
    𝐷
    𝐾 𝐿
    มߋΛՃ͑ͣɺ
    ͦͷ··఻ୡ
    'PSXBSE
    p2
    (c|x) p1
    (c|x)
    Gate
    KL div

    View full-size slide

  24. w ೖྗʹରͯ͠ৗʹΛग़ྗ
    ήʔτؔ਺ɿ$VUP
    ff
    (BUF
    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝑚
    1
    𝑚
    2
    𝐿
    2,1
    4PVSDF %FTUJOBUJPO
    𝑚
    2
    𝑚
    1
    L2,1
    (p2
    , p1
    )
    $VUPGG(BUF
    -JOFBS(BUF
    $PSSFDU(BUF
    5ISPVHI(BUF
    ৗʹΛग़ྗ
    Τοδͷ੾அ

    𝐺
    (
    𝐷
    𝐾 𝐿
    ) = 0
    'PSXBSE
    p2
    (c|x) p1
    (c|x)
    Gate
    KL div

    View full-size slide

  25. w ֶश͕࣌ؒܦաͱͱ΋ʹग़ྗ஋͕ঃʑʹେ͖͘ͳΔ
    ήʔτؔ਺ɿ-JOFBS(BUF
    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝑚
    1
    𝑚
    2
    𝐿
    2,1
    4PVSDF %FTUJOBUJPO
    𝑚
    2
    𝑚
    1
    L2,1
    (p2
    , p1
    )
    $VUPGG(BUF
    -JOFBS(BUF
    $PSSFDU(BUF
    5ISPVHI(BUF
    Gate
    KL div
    ࣌ؒͱڞʹग़ྗ͕
    େ͖͘ͳΔ
    𝐺
    (
    𝐷
    𝐾 𝐿
    ) =
    𝑡
    𝑡
    𝑚 𝑎
    𝑥

    𝐷
    𝐾
    𝐿
    'PSXBSE
    p2
    (c|x) p1
    (c|x)

    View full-size slide

  26. w ιʔεϊʔυ͕ਖ਼ղͨ͠৔߹ͷΈग़ྗ
    ήʔτؔ਺ɿ$PSSFDUHBUF
    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝑚
    1
    𝑚
    2
    𝐿
    2,1
    4PVSDF %FTUJOBUJPO
    𝑚
    2
    𝑚
    1
    L2,1
    (p2
    , p1
    )
    $VUPGG(BUF
    -JOFBS(BUF
    $PSSFDU(BUF
    5ISPVHI(BUF
    Gate
    KL div
    ਖ਼ղͨ͠αϯϓϧͷ
    ৘ใͷΈ఻ୡ
    𝐺
    (
    𝐷
    𝐾
    𝐿
    ) =
    𝛿
    ^
    𝑦
    ,
    𝑦
    𝑚
    2

    𝐷
    𝐾 𝐿
    'PSXBSE
    p2
    (c|x) p1
    (c|x)

    View full-size slide

  27. w ͲͷΑ͏ʹ஌ࣝసҠ͢Δ͔Λ੍ޚ
    ήʔτؔ਺
    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝑚
    1
    𝑚
    2
    𝐿
    2,1
    4PVSDF %FTUJOBUJPO
    𝑚
    2
    𝑚
    1
    L2,1
    (p2
    , p1
    )
    'PSXBSE
    p2
    (c|x) p1
    (c|x)
    Gate
    KL div
    $VUPGG(BUF
    -JOFBS(BUF
    5ISPVHI(BUF
    $PSSFDU(BUF

    View full-size slide

  28. w ϋΠύʔύϥϝʔλαʔνͰ஌ࣝసҠάϥϑΛ࠷దԽ
    ࠷దԽख๏"TZODISPOPVT4VDDFTTJWF)BMWJOH"MHPSJUIN "4)"

    ύϥϝʔλήʔτؔ਺ ิॿϊʔυ
    ஌ࣝసҠάϥϑͷ࠷దԽ
    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    ήʔτؔ਺
    • 5ISPVHI(BUF
    • $VUPGG(BUF
    • -JOFBS(BUF
    • $PSSFDU(BUF
    • 3FT/FU
    ධՁର৅ϊʔυ
    • 3FT/FU
    • 3FT/FU
    • 8JEF3FT/FU
    ิॿϊʔυ
    શ૊Έ߹Θͤ਺ɿ ௨Γʢϊʔυ਺ͷ৔߹ʣ

    View full-size slide

  29. ஌ࣝసҠάϥϑͷ࠷దԽ
    த෦େֶϩΰ
    த෦େֶϩΰ

    αʔό਺ɹɹɹɹɿ
    ୳ࡧճ਺ɹɹɹɹɿ ճ
    ϑϨʔϜϫʔΫɹɿ0QUVOB

    View full-size slide

  30. w ධՁର৅ϊʔυɿ 3FT/FU

    w 7BOJMMBϞσϧɿ
    ࠷దԽʹΑͬͯ֫ಘͨ͠஌ࣝసҠάϥϑʢୈҐʣ
    த෦େֶϩΰ
    த෦େֶϩΰ

    ڭࢣϥϕϧ
    ධՁର৅ϊʔυ
    ิॿϊʔυ
    QSFUSBJOFE

    ิॿϊʔυ
    ڭࢣϥϕϧ

    View full-size slide

  31. w ධՁର৅ϊʔυɿ 3FT/FU

    w 7BOJMMBϞσϧɿ
    ࠷దԽʹΑͬͯ֫ಘͨ͠஌ࣝసҠάϥϑʢୈҐʣ
    த෦େֶϩΰ
    த෦େֶϩΰ

    ิॿϊʔυ
    QSFUSBJOFE

    ิॿϊʔυ
    ධՁର৅ϊʔυ
    ஌ࣝৠཹ

    View full-size slide

  32. w ධՁର৅ϊʔυɿ 3FT/FU

    w 7BOJMMBϞσϧɿ
    ࠷దԽʹΑͬͯ֫ಘͨ͠஌ࣝసҠάϥϑʢୈҐʣ
    த෦େֶϩΰ
    த෦େֶϩΰ

    ิॿϊʔυ
    QSFUSBJOFE

    ิॿϊʔυ
    ධՁର৅ϊʔυ
    ॳΊ͸,%ϥΠΫͳֶशɼ࣍ୈʹ,%ʴ%.-ͳֶश͕ߦΘΕΔ
    ɹ૬ޓֶशɹ

    View full-size slide

  33. w ࣮ݧ֓ཁ
    σʔληοτɿ$*'"3
    ֶशϊʔυ਺ɿ
    ࠷దԽର৅ϊʔυɿ3FT/FU
    ैདྷख๏ ,% %.-
    ͱͷൺֱ
    த෦େֶϩΰ
    த෦େֶϩΰ

    ख๏ ೝࣝ཰<> ิॿϊʔυͷϞσϧ
    *OEFQFOEFOU r
    ,% 3FT/FU QSFUSBJOFE

    %.- 3FT/FU 3FT/FU
    0VST 3FT/FU QSFUSBJOFE
    3FT/FU

    View full-size slide

  34. ࠷దԽʹΑͬͯ֫ಘͨ͠஌ࣝసҠάϥϑ $*'"3

    த෦େֶϩΰ
    த෦େֶϩΰ

    ϊʔυ਺ɿ


    ϊʔυ਺ɿ


    ϊʔυ਺ɿ


    ϊʔυ਺ɿ


    ϊʔυ਺ɿ


    ϊʔυ਺ɿ


    *OEFQFOEFOU3FT/FU

    View full-size slide

  35. ࠷దԽʹΑͬͯ֫ಘͨ͠஌ࣝసҠάϥϑ $*'"3

    த෦େֶϩΰ
    த෦େֶϩΰ

    *OEFQFOEFOU3FT/FU

    ϊʔυ਺ɿ

    View full-size slide

  36. w ஌ࣝసҠάϥϑʹΞϯαϯϒϧϊʔυͱΞςϯγϣϯϩεΛಋೖ
    ΞςϯγϣϯϩεɿΤοδͷϩεʹΞςϯγϣϯϩεΛ௥Ճ
    Ξϯαϯϒϧϊʔυɿ֤ϊʔυͷग़ྗΛΞϯαϯϒϧ͢Δػߏ
    ஌ࣝసҠάϥϑΛ༻͍ͨΞϯαϯϒϧֶश<0LBNPUP &$$7>
    த෦େֶϩΰ
    த෦େֶϩΰ

    Ξϯαϯϒϧϊʔυ
    ^
    𝑦
    𝐿
    ^
    𝑦
    ,
    𝑒
    𝑒𝑛𝑠
    𝑚
    1
    𝑚
    2
    𝑚
    3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝐿
    ^
    𝑦
    ,3
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    2,1
    𝐿
    1,2
    𝐿
    3,1
    𝐿
    1,3
    𝐿
    2,3
    𝐿
    3,2
    𝐿
    i,j
    =
    𝐾 𝐿
    (
    𝒑
    i
    ,
    𝒑
    j) ±
    𝐿
    𝐴
    𝑇
    (
    𝑸
    i
    ,
    𝑸
    j
    )
    Ξςϯγϣϯϩε

    View full-size slide

  37. w ϊʔυ͔̎Βϊʔυ̍΁ͷΞςϯγϣϯϩε
    ೋͭͷϞσϧؒͷΞςϯγϣϯΛ͚ۙͮͨΓ཭͢ޮՌ
    Ξςϯγϣϯϩε
    த෦େֶϩΰ
    த෦େֶϩΰ

    4PVSDF %FTUJOBUJPO
    𝑚
    2
    𝑚
    1
    p2
    (c|x) p1
    (c|x)
    'PSXBSE
    𝑚
    1
    𝑚
    2
    𝑚
    3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝐿
    ^
    𝑦
    ,3
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    2,1
    𝐿
    1,2
    𝐿
    3,1
    𝐿
    1,3
    𝐿
    2,3
    𝐿
    3,2
    ,-EJW
    x x
    "UUFOUJPO "UUFOUJPO
    𝑸
    =
    𝐶

    𝑖
    =1
    𝑨
    𝑖
    𝑝
    ɿಛ௃Ϛοϓ
    ɿνϟωϧ਺
    ɿϊϧϜ਺
    𝑨
    𝑖
    𝐶
    𝑝
    ैདྷͷଛࣦ
    𝐿
    𝐴𝑇
    𝐿
    𝐴𝑇
    (
    𝑸
    2
    ,
    𝑸
    1
    ) =
    1
    𝐽
    𝐽

    𝑗
    𝑸
    𝑗
    2
    𝑸
    𝑗
    2
    2

    𝑸
    𝑗
    1
    𝑸
    𝑗
    1
    2
    2
    "UUFOUJPOଛࣦ
    (BUF
    𝐿
    2,1
    = G(
    𝐾𝐿
    (
    𝒑
    2
    ,
    𝒑
    1) ±
    𝐿
    𝐴 𝑇
    (
    𝑸
    2
    ,
    𝑸
    1
    ))
    ͚ۙͮΔ
    ཭͢
    +

    View full-size slide

  38. w ,-EJWFSHFODFͱΞςϯγϣϯϩεΛ#BDLQSPQͯ͠ Λߋ৽
    𝑚
    1
    ϊʔυ̎ TPVSDF
    ͔Βϊʔυ̍ EFTUJOBUJPO
    ΁ͷ஌ࣝసҠ
    த෦େֶϩΰ
    த෦େֶϩΰ

    𝑚
    2
    𝑚
    1
    𝑚
    1
    𝑚
    2
    𝑚
    3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝐿
    ^
    𝑦
    ,3
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    1,2
    𝐿
    3,1
    𝐿
    1,3
    𝐿
    2,3
    𝐿
    3,2
    x x
    4PVSDF %FTUJOBUJPO
    ͚ۙͮΔ
    ཭͢
    +

    𝐿
    2,1
    #BDLXBSE
    𝐿
    2,1
    = G(
    𝐾𝐿
    (
    𝒑
    2
    ,
    𝒑
    1) ±
    𝐿
    𝐴 𝑇
    (
    𝑸
    2
    ,
    𝑸
    1
    ))
    ,-EJW
    𝐿
    𝐴𝑇
    (BUF
    "UUFOUJPO "UUFOUJPO
    Detach
    Back-prop

    View full-size slide

  39. w ΞϯαϯϒϧϊʔυΛλʔήοτϊʔυͱͯ͠࠷େԽ͢ΔΑ͏ʹ࠷దԽ
    ֤ϊʔυͷग़ྗΛฏۉʹΑΓΞϯαϯϒϧ͢Δػߏ
    Ξϯαϯϒϧϊʔυ
    த෦େֶϩΰ
    த෦େֶϩΰ

    ^
    𝑦
    𝐿
    ^
    𝑦
    ,
    𝑒
    𝑒𝑛𝑠
    𝑚
    1
    𝑚
    2
    𝑚
    3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦
    𝐿
    ^
    𝑦
    ,3
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    2,1
    𝐿
    1,2
    𝐿
    3,1
    𝐿
    1,3
    𝐿
    2,3
    𝐿
    3,2
    𝑚
    1
    𝑚
    2
    𝑒𝑛𝑠
    Ξϯαϯϒϧϊʔυ
    𝑚
    3
    Ξϯαϯϒϧػߏ
    𝑝
    (
    𝑐 𝑥
    ) =
    𝑝
    1
    (
    𝑐 𝑥
    ) +
    𝑝
    2
    (
    𝑐 𝑥
    ) +
    𝑝
    3
    (
    𝑐
    |
    𝑥
    )
    𝑝
    1
    (
    𝑐 𝑥
    )
    𝑝
    2
    (
    𝑐 𝑥
    )
    𝑝
    3
    (
    𝑐 𝑥
    )
    𝑝
    (
    𝑐 𝑥
    )

    View full-size slide

  40. ஌ࣝసҠάϥϑͷΞϯαϯϒϧޮՌ
    த෦େֶϩΰ
    த෦େֶϩΰ

    ˠ஌ࣝసҠάϥϑʹ͓͍ͯΞϯαϯϒϧਫ਼౓͕޲্

    View full-size slide

  41. ࠷దԽͨ͠Ξϯαϯϒϧ஌ࣝసҠάϥϑ ϊʔυ਺ɿʣ
    த෦େֶϩΰ
    த෦େֶϩΰ

    ˠҟͳΔΞςϯγϣϯϚοϓʢΞϯαϯϒϧʹదͨ͠ଟ༷ੑʣΛ֫ಘ
    w Ξϯαϯϒϧϊʔυɿ
    w 7BOJMMBϞσϧɿ
    ೖྗը૾

    View full-size slide

  42. ࠷దԽͨ͠Ξϯαϯϒϧ஌ࣝసҠάϥϑ ϊʔυ਺ɿʣ
    த෦େֶϩΰ
    த෦େֶϩΰ

    ʹ͚ۙͮΔ
    ʹ͚ۙͮΔ
    ͔Β཭͢
    ˠҟͳΔΞςϯγϣϯϚοϓʢΞϯαϯϒϧʹదͨ͠ଟ༷ੑʣΛ֫ಘ
    ʹ͚ۙͮΔ
    ͔Β཭͢
    ͓ޓ͍ʹ͚ۙͮΔ
    ʹ͚ۙͮΔ
    ʹ͚ۙͮΔ
    ʹ͚ۙͮΔ
    ೖྗը૾
    w Ξϯαϯϒϧϊʔυɿ
    w 7BOJMMBϞσϧɿ

    View full-size slide

  43. ଟ༷ͳΞϯαϯϒϧϞσϧ͔Βͷ஌ࣝৠཹ
    த෦େֶϩΰ
    த෦େֶϩΰ

    w ஌ࣝసҠάϥϑͷΞϯαϯϒϧΛڭࢣͱͯ͠஌ࣝৠཹ
    ڭࢣωοτϫʔΫɿ஌ࣝసҠάϥϑͰֶशͨ͠ෳ਺ͷ"#/ʢ3FT/FUʣ
    ੜెωοτϫʔΫɿ"#/ʢ3FT/FUʣ
    ೖྗը૾
    𝒙
    4UVEFOU/FUXPSL
    ωοτϫʔΫ
    𝑚
    1
    𝒍
    1
    (
    𝒙
    )
    𝒍
    3
    (
    𝒙
    )
    ωοτϫʔΫ
    𝑚
    3
    𝒍
    𝑒
    𝑛
    𝑠
    (
    𝒙
    )
    𝒑
    𝑠
    (
    𝒙
    ) ڭࢣϥϕϧ ^
    𝑦
    ஌ࣝసҠ
    𝒑
    𝑒
    𝑛
    𝑠
    (
    𝒙
    )
    Թ౓෇͖4PGUNBYؔ਺
    5FBDIFS/FUXPSL
    𝑒 𝑛 𝑠
    𝑚
    1
    𝑚
    2
    𝑚
    3
    ஌ࣝసҠάϥϑ

    View full-size slide

  44. ଟ༷ͳΞϯαϯϒϧϞσϧ͔Βͷ஌ࣝৠཹ
    த෦େֶϩΰ
    த෦େֶϩΰ

    ஌ࣝసҠάϥϑʹΑΔΞϯαϯϒϧΛৠཹ͢Δ͜ͱͰ
    ಉ͡ύϥϝʔλ਺Ͱߴ͍ೝࣝੑೳΛൃش

    View full-size slide

  45. ஌ࣝసҠάϥϑʹΑΔڞಉֶश
    த෦େֶϩΰ
    த෦େֶϩΰ

    w ஌ࣝసҠάϥϑΛఏҊ
    ̐छྨͷ(BUFؔ਺ʹΑΓɺ஌ࣝసҠΛ੍ޚ͢Δ͜ͱͰଟ༷ͳڞಉֶश
    ϋΠύʔύϥϝʔλαʔνʹΑΔ࠷దͳ஌ࣝసҠάϥϑΛ୳ࡧ
    ஌ࣝసҠάϥϑʹΑΔΞϯαϯϒϧֶश
    w ൃݟͨ͜͠ͱʢϊʔυ਺ͷ৔߹ʣ
    ,%ͱ%.-͕༥߹ͨ͠஌ࣝసҠάϥϑ͸ैདྷ๏Λ௒͑Δਫ਼౓Λୡ੒
    ,%ͱ%.-ͷ༥߹ͨ͠஌ࣝసҠάϥϑ
    𝑚
    3
    𝑚
    1
    𝑚
    2
    𝐿
    ^
    𝑦
    ,1
    𝐿
    ^
    𝑦
    ,2
    𝐿
    ^
    𝑦
    ,3
    𝐿
    1,2
    𝐿
    1,3
    𝐿
    2,1
    𝐿
    3,1
    𝐿
    3,2
    𝐿
    2,3
    ^
    𝑦
    ^
    𝑦
    ^
    𝑦

    View full-size slide

  46. ૉਓൃ૝ݰਓ࣮ߦ΁
    த෦େֶϩΰ
    த෦େֶϩΰ

    w ஌ࣝసҠάϥϑΛఏҊ஌ࣝసҠ
    グ
    ϥϑʹΑΔڞಉֶश<.JOBNJ .*36 "$$7>
    ݚڀऀ͸໰୊ઃఆͱղ͖ํΛݶఆͤ
    ず
    ɼ஌ࣝసҠΛදݱ͢ΔϑϨʔϜϫʔΫΛݚڀऀ
    が
    ઐ໳ੑΛൃ
    شͯ͠ઃܭ
    ࠷దԽ୳ࡧΛߦ͏͜ͱ
    で
    ৽ͨͳ஌ݟΛൃݟ
    4PGUXBSF࣌୅ͷݚڀํ๏
    ܭࢉػ
    が
    ๲େͳ
    パ
    ϥϝʔλۭ͔ؒΒ࢓༷Λຬͨ͢
    プ
    ϩ
    グ
    ϥϜΛ୳ࡧ
    ʮૉਓൃ૝ݰਓ࣮ߦʯ
    ୳ࡧʹΑΓ֫ಘͨ͠৽ͨͳ஌ݟ
    が
    ࣍ͷ৽ͨͳݚڀͷ୺ॹͱͳΔ͜ͱΛظ଴
    ݚڀۭؒ
    ͜Ε·Ͱͷݚڀ
    ݚ
    ڀ
    ͷ


    ͞
    ૉਓൃ૝ݰਓ࣮ߦ

    View full-size slide

  47. 47
    த෦େֶϩΰ
    த෦େֶϩΰ
    ஌ࣝసҠάϥϑʹΑΔ࠷దͳ൒ڭࢣ͋Γਂ૚ڞಉֶशͷ୳ࡧ<ଜຊ +4"*>

    ʮૉਓൃ૝ݰਓ࣮ߦʯ
    ୳ࡧʹΑΓ֫ಘͨ͠৽ͨͳ஌ݟ
    が
    ࣍ͷ৽ͨͳݚڀͷ୺ॹͱͳΔ͜ͱΛظ଴

    View full-size slide

  48. w ϥϕϧ͋Γσʔλͱϥϕϧͳ͠σʔλΛֶशʹར༻
    Ξϊςʔγϣϯʢϥϕϧ෇͚ʣʹ͔͔Δίετ࡟ݮ
    ֶश༻σʔλͷ֬อ͕༰қ
    ൒ڭࢣ͋Γֶशʢ4FNJTVQFSWJTFEMFBSOJOH 44-

    ֶश༻σʔλʹର͢Δσʔλͷׂ߹

    View full-size slide

  49. w Ұகੑਖ਼ଇԽʢ$POTJTUFODZSFHVMBSJ[BUJPOʣ
    ϥϕϧͳ͠σʔλʹઁಈΛ෇༩͠ɼͦͷը૾ʹର͢ΔҰகੑΛֶश
    ैདྷ๏ɿ NPEFM<-BJOF *$-3>ɼ.FBO5FBDIFS<5BSWBJOFO /FVS*14>ͳͲ
    w ٖࣅϥϕϦϯάʢ1TFVEPMBCFMJOHʣ
    ༧ଌ݁ՌΛPOFIPUԽٖͯ͠ࣅϥϕϧΛϥϕϧͳ͠σʔλʹ෇༩
    ϥϕϧ͋Γσʔλͱٖࣅϥϕϧ͋Γσʔλͷࠞ߹ηοτΛ༻͍ͯڭࢣ͋Γֶश
    ैདྷ๏ɿ1TFVEP-BCFM<-FF *$.->ͳͲ
    Π
    ൒ڭࢣ͋Γֶशͷ୅දతͳํ๏
    ϥϕϧͳ͠σʔλ
    ༧ଌ
    /FUXPSL
    ٖࣅϥϕϧ͋Γσʔλ
    ʢϥϕϧͳ͠σʔλʣ
    )BSEUBSHFU
    ラベルありデータ
    Network
    正解情報
    予測
    誤差
    ラベルなしデータ
    予測1
    摂動を付与
    ٖࣅϥϕϦϯά
    ラベルありデータ
    Network
    正解情報
    予測
    誤差
    ラベルなしデータ
    予測1
    摂動を付与
    Ұ؏ੑਖ਼ଇԽ
    ϥϕϧͳ͠σʔλ
    ラベルありデータ
    Network
    正解情報
    予測
    誤差
    ラベルなしデータ
    予測1
    摂動を付与
    ʴઁಈ
    ʴઁಈ
    ༧ଌ
    ༧ଌ
    ޡࠩ
    /FUXPSL

    View full-size slide

  50. w ٖࣅϥϕϦϯάɿऑม׵࣌ͷ༧ଌ͕ᮢ஋Λ௒͑ͨ৔߹ͷΈٖࣅϥϕϧΛੜ੒
    ऑม׵ɿࠨӈ൓స ฏߦҠಈ
    w Ұகੑଛࣦɿੜ੒ٖͨ͠ࣅϥϕϧͱڧม׵࣌ͷ༧ଌͷޡࠩ
    ڧม׵ɿෳ਺छͷը૾ม׵ʹΑΔڧ͍ઁಈʢ3BOE"VHNFOU<$VCVLF $713>ʣ
    Ұகੑਖ਼ଇԽͱٖࣅϥϕϦϯάͷϋΠϒϦουɿ'JY.BUDI<4PIO /FVS*14>
    Ұ؏ੑଛࣦ
    ༧ଌ
    /FUXPSL
    ڧม׵
    ऑม׵
    ڭࢣ͋Γଛࣦ
    ϥϕϧ
    ༧ଌ
    ٖࣅϥϕϦϯά
    ϥϕϧ͋Γσʔλ
    ラベルありデータ
    Network
    正解情報
    予測
    誤差
    ラベルなしデータ
    Network
    予測1
    予測2
    摂動を付与
    ϥϕϧͳ͠σʔλ
    )BSEUBSHFU
    ˠਓ͕ઃܭͨ͠૊Έ߹ΘͤͰ͋ΔͨΊ࠷దͳֶश๏ͱ͸ݶΒͳ͍

    View full-size slide

  51. w ਓखʹΑΒͳ͍৽͍͠൒ڭࢣ͋Γڞಉֶश๏ͷ֫ಘ
    w Ξϓϩʔν
    ֤ैདྷ๏ΛͦΕͧΕάϥϑͰ౷Ұతʹදݱ
    άϥϑදݱͷߏ੒ཁૉΛϥϯμϜʹ૊Έ߹Θͤͯߴਫ਼౓ͳֶश๏Λ୳ࡧ
    ຊݚڀͷ໨త

    ɾɾɾ
    άϥϑදݱ
    NPEFM
    Π
    .FBO5FBDIFS
    !!
    !"
    Parameter
    for Exponential
    Moving Average
    KL-div
    ɾɾɾ
    NPEFM
    Π
    .FBO5FBDIFS
    ैདྷ๏
    Network
    !!
    "
    +$! Network
    !!
    +$!
    ′ &(!!
    , " + $!
    ′)
    BackProp
    Loss
    Graphical
    representation
    !!
    !!
    !!
    &(!!
    , " + $!
    )
    KL-div
    KL-div
    Network
    EMA(%!
    )
    '
    +)!
    Exponential
    Moving Average
    Network
    %!
    +)!
    ′ +(EMA(%!
    ), ' + )!
    )
    +(%!
    , ' + )!
    ′)
    Loss
    BackProp
    !!
    !"
    Parameter
    for Exponential
    Moving Average
    Graphical
    representation
    KL-div
    ྫɿ
    NPEFM.FBO5FBDIFS
    Π
    άϥϑߏ଄ͷ୳ࡧ
    !!
    KL-div
    !"
    Parameter
    for Exponential
    Moving Average
    KL-div
    ɾɾɾ

    View full-size slide

  52. ɾɾɾ
    άϥϑදݱ
    NPEFM
    Π
    .FBO5FBDIFS
    !!
    !"
    Parameter
    for Exponential
    Moving Average
    KL-div
    ɾɾɾ
    NPEFM
    Π
    .FBO5FBDIFS
    ैདྷ๏
    Network
    !!
    "
    +$! Network
    !!
    +$!
    ′ &(!!
    , " + $!
    ′)
    BackProp
    Loss
    Graphical
    representation
    !!
    !!
    !!
    &(!!
    , " + $!
    )
    KL-div
    KL-div
    Network
    EMA(%!
    )
    '
    +)!
    Exponential
    Moving Average
    Network
    %!
    +)!
    ′ +(EMA(%!
    ), ' + )!
    )
    +(%!
    , ' + )!
    ′)
    Loss
    BackProp
    !!
    !"
    Parameter
    for Exponential
    Moving Average
    Graphical
    representation
    KL-div
    ྫɿ
    NPEFM.FBO5FBDIFS
    Π
    άϥϑߏ଄ͷ୳ࡧ
    !!
    KL-div
    !"
    Parameter
    for Exponential
    Moving Average
    KL-div
    ɾɾɾ
    w ֤ैདྷ๏ΛͦΕͧΕάϥϑͰ౷Ұతʹදݱ
    ૊Έ߹Θ͕ͤ༰қʹͳΓϋΠύʔύϥϝʔλͷΑ͏ʹௐ੔Մೳ
    ϊʔυɿωοτϫʔΫ
    Τοδɿଛࣦܭࢉ
    ैདྷͷ൒ڭࢣ͋Γֶश๏ΛάϥϑͰදݱ

    View full-size slide

  53. w ڭࢣ͋ΓଛࣦͱҰகੑଛࣦ͕খ͘͞ͳΔΑ͏ʹֶश
    ڭࢣ͋Γଛࣦɿϥϕϧ͋Γσʔλͷ༧ଌͱϥϕϧͷޡࠩ
    Ұ؏ੑଛࣦɹɿϥϕϧͳ͠σʔλʹҟͳΔઁಈΛ෇༩ͨ࣌͠ͷ༧ଌؒͷޡࠩ
    ઁಈɿ%SPQPVUɼը૾ม׵
    Ұகਖ਼ଇԽͷ୅දతͳख๏ɿ NPEFM<-BJOF *$-3>
    Π
    Ұகੑଛࣦ
    ༧ଌ
    /FUXPSL
    ڭࢣ͋Γଛࣦ
    ϥϕϧ
    ༧ଌ
    ϥϕϧ͋Γσʔλ
    ラベルありデータ
    Network
    正解情報
    予測
    誤差
    ラベルなしデータ
    Network
    予測1
    予測2
    摂動を付与
    ϥϕϧͳ͠σʔλ
    ʴઁಈ
    ʴઁಈ
    )BSEUBSHFU
    ラベルありデータ
    Network
    正解情報
    予測
    誤差
    ラベルなしデータ
    予測1
    摂動を付与

    View full-size slide

  54. w Ұகੑଛࣦɿ࢝఺ͷϊʔυͱऴ఺ͷϊʔυ͕ಉ͡ΤοδͰදݱ
    ,-EJWFSHFODFʢ,-EJWʣͰ༧ଌؒͷޡࠩΛܭࢉ
    NPEFMΛάϥϑͰදݱ
    Π
    KL(f(x), f(x′

    )) =
    C

    i
    fi
    (x)log
    fi
    (x)
    fi
    (x′

    )
    NPEFM
    Π άϥϑදݱ
    ɿ֬཰෼෍ʢ༧ଌ֬཰ʣ
    ɿΫϥε਺
    f(x), f(x′

    )
    C
    Network
    !!
    "
    +$! Network
    !!
    +$!
    ′ &(!!
    , " + $!
    ′)
    BackProp
    Loss
    Graphical
    representation
    !!
    !!
    !!
    &(!!
    , " + $!
    )
    KL-div
    KL-div

    View full-size slide

  55. w NPEFMʹࢦ਺Ҡಈฏۉʢ&."ʣωοτϫʔΫΛಋೖֶͯ͠श
    ҰகੑଛࣦɿωοτϫʔΫͱ&."ωοτϫʔΫʹ͓͚Δ༧ଌؒͷޡࠩ
    &."ωοτϫʔΫͷॏΈ͸ωοτϫʔΫͷॏΈΛՃࢉ͢Δ͜ͱͰߋ৽
    ॏΈύϥϝʔλͷΞϯαϯϒϧʹΑΓߴ͍ੑೳΛൃش͠ɼֶशΛิॿ
    Π
    Ұகੑਖ਼ଇԽͷ୅දతͳख๏ɿ.FBO5FBDIFS<5BSWBJOFO /FVS*14>
    ڭࢣ͋Γଛࣦ
    ϥϕϧ
    ϥϕϧ͋Γσʔλ
    ラベルありデータ
    Network
    正解情報
    予測
    誤差
    ラベルなしデータ
    Network
    予測1
    予測2
    摂動を付与
    )BSEUBSHFU
    Ұகੑଛࣦ
    ༧ଌ
    /FUXPSL
    ༧ଌ
    ϥϕϧͳ͠σʔλ
    ラベルありデータ
    Network
    正解情報
    予測
    誤差
    ラベルなしデータ
    Network
    予測1
    摂動を付与
    ڭࢣ͋Γଛࣦ
    ϥϕϧ
    ! " #$ % & ' (
    ) *+, -./
    ㎯1㌪刷
    卡䇦
    6〱
    ! " #8 9 & ' (
    ) *+, -./
    卡䇦:
    卡䇦;
    ㏗俍> 冊卥
    ϥϕϧ͋Γ
    σʔλ
    POFIPU
    ! " #$ % & ' (
    ) *+, -./
    ㎯1㌪刷
    卡䇦
    6〱
    ㏗俍> 冊卥
    ϥϕϧͳ͠
    σʔλ
    /FUXPSL
    ! " #$ % & ' (
    ) *+, -./
    ㎯1㌪刷
    卡䇦
    6〱
    ! " #8 9 & ' (
    ) *+, -./
    卡䇦:
    ㏗俍> 冊卥
    ! " #$ % & ' (
    ) *+, -./
    ㎯1㌪刷
    卡䇦
    6〱
    ! " #$ % & ' (
    ) *+, -./
    ㎯1㌪刷
    卡䇦
    6〱
    ༧ଌ
    ༧ଌ
    Ұ؏ੑଛࣦ
    ڭࢣ͋Γଛࣦ
    ϥϕϧ
    ! " #$ % & ' (
    ) *+, -./
    ㎯1㌪刷
    卡䇦
    6〱
    ! " #8 9 & ' (
    ) *+, -./
    卡䇦:
    卡䇦;
    ㏗俍> 冊卥
    ϥϕϧ͋Γ
    σʔλ
    POFIPU
    ! " #$ % & ' (
    ) *+, -./
    ㎯1㌪刷
    卡䇦
    6〱
    ! " #8 9 & ' (
    ) *+, -./
    卡䇦:
    卡䇦;
    ㏗俍> 冊卥
    ϥϕϧͳ͠
    σʔλ
    /FUXPSL
    ! " #$ % & ' (
    ) *+, -./
    ㎯1㌪刷
    卡䇦
    6〱
    ! " #8 9 & ' (
    ) *+, -./
    卡䇦:
    卡䇦;
    ㏗俍> 冊卥
    ! " #$ % & ' (
    ) *+, -./
    ㎯1㌪刷
    卡䇦
    6〱
    ! " #8 9 & ' (
    卡䇦:
    ㏗俍> 冊卥
    ! " #$ % & ' (
    ) *+, -./
    ㎯1㌪刷
    卡䇦
    6〱
    ! " #8 9 & ' (
    卡䇦:
    ㏗俍> 冊卥
    ༧ଌ
    ༧ଌ
    Ұ؏ੑଛࣦ
    ʴઁಈ
    ʴઁಈ
    /FUXPSL

    View full-size slide

  56. w &."ωοτϫʔΫʹՃࢉ͢Δύϥϝʔλͷํ޲ΛΤοδͰදݱ
    &."ωοτϫʔΫɿύϥϝʔλ ͷࢦ਺Ҡಈฏۉ஋ Ͱߋ৽
    θ1
    EMA(θ1
    )
    .FBO5FBDIFSΛάϥϑͰදݱ
    EMA(θ1,t
    ) = αEMA(θ1,t−1
    ) + (1 − α)θ1,t
    ɿϋΠύʔύϥϝʔλ
    ɿֶशεςοϓ
    α
    t
    .FBO5FBDIFS άϥϑදݱ
    Network
    EMA(%!
    )
    '
    +)!
    Exponential
    Moving Average
    Network
    %!
    +)!
    ′ +(EMA(%!
    ), ' + )!
    ′)
    +(%!
    , ' + )!
    )
    Loss
    BackProp
    !!
    !"
    Parameter
    for Exponential
    Moving Average
    Graphical
    representation
    KL-div

    View full-size slide

  57. w ϥϕϧͳ͠σʔλʹٖࣅϥϕϧΛ෇༩ֶͯ͠श
    ֶशং൫͸ϥϕϧ͋ΓσʔλͷΈΛ༻͍ͯڭࢣ͋Γֶश
    ༧ଌΛ΋ͱʹϥϕϧͳ͠σʔλʹٖࣅϥϕϧΛ෇༩ʢ·ͨ͸ߋ৽ʣ
    ϥϕϧ͋Γσʔλͱٖࣅϥϕϧ͋Γσʔλͷࠞ߹ηοτΛ༻͍ͯڭࢣ͋Γֶश
    ̎ͱ̏Λ܁Γฦ͢
    ϥϕϧ
    ༧ଌ
    ޡࠩ
    /FUXPSL


    ٖࣅϥϕϧ
    ͋Γσʔλ
    ࠞ߹ηοτ
    ラベルありデータ
    Network
    正解情報
    予測
    誤差
    ラベルなしデータ
    Network
    予測1
    摂動を付与
    ϥϕϧ͋Γ
    σʔλ
    ラベルありデータ
    Network
    正解情報
    予測
    誤差
    ラベルなしデータ
    Network
    予測1
    予測2
    摂動を付与

    ٖࣅϥϕϧ͋Γσʔλ
    )BSEUBSHFU
    ラベルありデータ
    Network
    正解情報
    予測
    誤差
    ラベルなしデータ
    Network
    予測1
    予測2
    摂動を付与
    ϥϕϧͳ͠σʔλ
    ༧ଌ
    /FUXPSL
    ラベルありデータ
    Network
    正解情報
    予測
    誤差
    ラベルなしデータ
    Network
    予測1
    予測2
    摂動を付与
    ٖࣅϥϕϦϯάͷ୅දతͳख๏ɿ1TFVEP-BCFM<-FF *$.->

    View full-size slide

  58. w ٖࣅϥϕϧʹର͢Δ༧ଌͷޡࠩΛٻΊΔ1TFVEP-PTTʹΑΓΤοδͰදݱ
    1TFVEP-PTTɿٖࣅϥϕϧͱ༧ଌ֬཰ͷޡࠩ
    1TFVEP-BCFMΛάϥϑͰදݱ
    AAACqXichVFNTxNBGH5Yv6B+UORi4qWxQUqAZtYQNCQmRDTx4IGChUaWbGaHabth9iO706Zls3+AP+DBEybGqD/Di0cuHOo/MB4x8eLBt9tNCBL1nczMM8+8zzvPzDihcmPN2GDMuHT5ytVr4xOF6zdu3posTt3eioNOJGRdBCqIGg6PpXJ9WdeuVrIRRpJ7jpLbzv7acH+7K6PYDfyXuh/KXY+3fLfpCq6JsotPLY/rtuAqeZHaSRjLtNKbe7yYsY6TPEvtXn82sSKvpIJW2qxYui01t82FUm/eOsjg7JxdLLMqy6J0EZg5KCOP9aD4Hhb2EECgAw8SPjRhBY6Y2g5MMITE7SIhLiLkZvsSKQqk7VCWpAxO7D6NLVrt5KxP62HNOFMLOkVRj0hZwgw7YR/YKfvCPrFv7NdfayVZjaGXPs3OSCtDe/LwzubP/6o8mjXaZ6p/etZo4lHm1SXvYcYMbyFG+u7B69PNlY2Z5D57y76T/yM2YJ/pBn73h3hXkxtvUKAPMP987otg60HVXK4u1ZbKq0/yrxjHXdxDhd77IVbxHOuo07kfcYwBvhrzRs1oGK9GqcZYrpnGuTDEb06spCc=
    Lpse(x) = E
    xy0logf(✓1, x + ⇣0
    1
    )
    AAACqXichVFNTxNBGH5Yv6B+UORi4qWxQUqAZtYQNCQmRDTx4IGChUaWbGaHabth9iO706Zls3+AP+DBEybGqD/Di0cuHOo/MB4x8eLBt9tNCBL1nczMM8+8zzvPzDihcmPN2GDMuHT5ytVr4xOF6zdu3posTt3eioNOJGRdBCqIGg6PpXJ9WdeuVrIRRpJ7jpLbzv7acH+7K6PYDfyXuh/KXY+3fLfpCq6JsotPLY/rtuAqeZHaSRjLtNKbe7yYsY6TPEvtXn82sSKvpIJW2qxYui01t82FUm/eOsjg7JxdLLMqy6J0EZg5KCOP9aD4Hhb2EECgAw8SPjRhBY6Y2g5MMITE7SIhLiLkZvsSKQqk7VCWpAxO7D6NLVrt5KxP62HNOFMLOkVRj0hZwgw7YR/YKfvCPrFv7NdfayVZjaGXPs3OSCtDe/LwzubP/6o8mjXaZ6p/etZo4lHm1SXvYcYMbyFG+u7B69PNlY2Z5D57y76T/yM2YJ/pBn73h3hXkxtvUKAPMP987otg60HVXK4u1ZbKq0/yrxjHXdxDhd77IVbxHOuo07kfcYwBvhrzRs1oGK9GqcZYrpnGuTDEb06spCc=
    Lpse(x) = E
    xy0logf(✓1, x + ⇣0
    1
    )
    AAACqXichVFNTxNBGH5Yv6B+UORi4qWxQUqAZtYQNCQmRDTx4IGChUaWbGaHabth9iO706Zls3+AP+DBEybGqD/Di0cuHOo/MB4x8eLBt9tNCBL1nczMM8+8zzvPzDihcmPN2GDMuHT5ytVr4xOF6zdu3posTt3eioNOJGRdBCqIGg6PpXJ9WdeuVrIRRpJ7jpLbzv7acH+7K6PYDfyXuh/KXY+3fLfpCq6JsotPLY/rtuAqeZHaSRjLtNKbe7yYsY6TPEvtXn82sSKvpIJW2qxYui01t82FUm/eOsjg7JxdLLMqy6J0EZg5KCOP9aD4Hhb2EECgAw8SPjRhBY6Y2g5MMITE7SIhLiLkZvsSKQqk7VCWpAxO7D6NLVrt5KxP62HNOFMLOkVRj0hZwgw7YR/YKfvCPrFv7NdfayVZjaGXPs3OSCtDe/LwzubP/6o8mjXaZ6p/etZo4lHm1SXvYcYMbyFG+u7B69PNlY2Z5D57y76T/yM2YJ/pBn73h3hXkxtvUKAPMP987otg60HVXK4u1ZbKq0/yrxjHXdxDhd77IVbxHOuo07kfcYwBvhrzRs1oGK9GqcZYrpnGuTDEb06spCc=
    Lpse(x) = E
    xy0logf(✓1, x + ⇣0
    1
    )
    1TFVEP-BCFM άϥϑදݱ
    ωοτϫʔΫ͕̎ͭͷ৔߹
    Network
    !! or !" Pseudo-Labeling
    one-hot
    Network
    !!
    "
    +$!
    +$!

    &(!!
    , " + $!
    )
    Loss
    PseudoLoss
    !!
    !!
    !"
    PseudoLoss
    Graphical
    representation or
    BackProp

    View full-size slide

  59. w ධՁର৅ϊʔυͷਫ਼౓͕࠷େԽ͢ΔΑ͏ʹάϥϑߏ଄Λ࠷దԽ
    ิॿϊʔυɿධՁର৅ϊʔυͷֶशΛαϙʔτ
    Losses:
    ・KL-divergence
    ・PseudoLoss
    Gate functions:
    ・Through Gate
    ・Cutoff Gate
    ・Linear Gate
    ・Threshold Gate
    Explore space:
    Models:
    ・ResNet32
    ・WideResNet28-2
    ・WideResNet28-6
    ・EMA model
    Edge
    Node
    Backprop
    Detach
    Gate
    Loss

    ධՁର৅ϊʔυ
    ิॿϊʔυ
    άϥϑ࠷దԽʹΑΔ൒ڭࢣ͋Γਂ૚ڞಉֶश๏ͷ୳ࡧ

    View full-size slide

  60. ධՁ݁Ռ
    ϥϕϧ͋Γσʔλ਺ BMM
    4VQFSWJTFE
    1TFVEP-BCFM

    .FBO5FBDIFS
    'JY.BUDI
    0VST ϊʔυ

    0VST ϊʔυ

    NPEFM
    Π
    ਖ਼ղ཰<>
    ˠϥϕϧ͋Γσʔλ਺ʹΑͬͯ࠷దͳ൒ڭࢣ͋Γֶश๏͸ҟͳΔ
    ଟ͍৔߹ʢ ʣɿ1TFVEP-BCFM͕ߴਫ਼౓
    গͳ͍৔߹ʢ ʙ ʣɿ NPEFM .FBO5FBDIFS͕ߴਫ਼౓
    Π

    View full-size slide

  61. ධՁ݁Ռ
    ϥϕϧ͋Γσʔλ਺ BMM
    4VQFSWJTFE
    1TFVEP-BCFM

    .FBO5FBDIFS
    'JY.BUDI
    0VST ϊʔυ

    0VST ϊʔυ

    NPEFM
    Π
    ਖ਼ղ཰<>
    ˠैདྷ๏ͱൺ΂୳ࡧͨ͠ख๏ʢ0VSTʣͷਫ਼౓͕࠷΋ߴ͍
    ʴ̒
    ʴ

    View full-size slide

  62. w ࠷దԽͨ͠άϥϑߏ଄Λௐࠪ
    ൒ڭࢣ͋Γֶशͷ܏޲
    w ܏޲ௐࠪ
    ֶशܦաʹ͓͚Δ൒ڭࢣ͋Γڞಉֶशͷ܏޲
    ϥϕϧ͋Γσʔλ਺ʹ͓͚Δ൒ڭࢣ͋Γڞಉֶशͷ܏޲
    ϊʔυ਺ͷมԽʹ͓͚Δ൒ڭࢣ͋Γڞಉֶशͷ܏޲
    ࠷దԽͨ͠άϥϑߏ଄

    View full-size slide

  63. w ϊʔυ਺ɿ̎ɼϥϕϧ͋Γσʔλ਺ɿ ʢগͳ͍ʣ
    ਖ਼ղ཰ɿ<>
    ࠷దԽͨ͠άϥϑߏ଄ɿֶशܦաʹ͓͚Δ܏޲<>
    (BUFؔ਺ ֶशํ๏
    1. ResNet32 (55.94%) Linear
    2. WRN28_6 (57.81%)
    Linear
    Linear Label
    Through
    Label
    Through
    1. ResNet32 (55.94%) Consistency
    2. WRN28_6 (57.81%)
    KD
    Consistency Label
    Supervised
    Label
    Supervised
    ڭࢣ͋Γֶश
    ڭࢣ͋Γֶश
    ֶशং൫ɿݸʑͷϊʔυͰಠཱʹڭࢣ͋Γֶश

    View full-size slide

  64. w ϊʔυ਺ɿ̎ɼϥϕϧ͋Γσʔλ਺ɿ ʢগͳ͍ʣ
    ਖ਼ղ཰ɿ<>
    ࠷దԽͨ͠άϥϑߏ଄ɿֶशܦաʹ͓͚Δ܏޲<>
    (BUFؔ਺ ֶशํ๏
    1. ResNet32 (55.94%) Linear
    2. WRN28_6 (57.81%)
    Linear
    Linear Label
    Through
    Label
    Through
    ֶशং൫ɿݸʑͷϊʔυͰಠཱʹڭࢣ͋Γֶश
    1. ResNet32 (55.94%) Consistency
    2. WRN28_6 (57.81%)
    KD
    Consistency Label
    Supervised
    Label
    Supervised
    ˠֶशલ൒ͱޙ൒ͰҟͳΔֶशઓུ͕ޮՌత
    ֶशऴ൫ɿݸʑͷϞσϧ͕ NPEFMʹΑΔֶशʹมԽͭͭ͠ɼ̎൪ϊʔυ͔Βৠཹ
    Π
    NPEFM
    Π
    NPEFM
    Π
    ৠཹ

    View full-size slide

  65. w ϊʔυ਺ɿ̎ɼϥϕϧ͋Γσʔλ਺ɿ ʢଟ͍ʣ
    ਖ਼ղ཰ɿ<>
    ࠷దԽͨ͠άϥϑߏ଄ɿֶशܦաʹ͓͚Δ܏޲<>
    (BUFؔ਺ ֶशํ๏
    1. ResNet32 (62.76%) Linear
    2. WRN28_6 (60.4%)
    Through
    1. ResNet32 (62.76%) Consistency
    2. WRN28_6 (60.4%)
    PseudoLabeling
    ֶशࡁΈϞσϧ
    ٖࣅϥϕϦϯά
    ֶशং൫ɿࣄલֶशϞσϧͷٖࣅϥϕϦϯάͰֶश
    NPEFM
    Π
    ֶशऴ൫ɿٖࣅϥϕϦϯάͱ NPEFMͰֶश
    Π
    ˠֶशऴ൫ʹҰகੑਖ਼ଇԽʢ NPEFMʣͰֶश͢Δ͜ͱ͕ޮՌత
    Π

    View full-size slide

  66. ֶशܦաʹ͓͚Δ܏޲ɿ6."1ʹΑΔՄࢹԽ
    FQPDI
    FQPDI
    1. ResNet32 (62.76%) Consistency
    2. WRN28_6 (60.4%)
    PseudoLabeling
    ֶशࡁΈϞσϧ
    ٖࣅϥϕϦϯά
    NPEFM
    Π
    ˠֶशޙ൒ͷҰகੑਖ਼ଇԽʢ NPEFMʣʹΑΓΫϥελԽ͕ਐΉ
    Π
    1. ResNet32 (62.76%) Consistency
    2. WRN28_6 (60.4%)
    PseudoLabeling
    ֶशࡁΈϞσϧ
    ٖࣅϥϕϦϯά

    View full-size slide

  67. w ϥϕϧ͋Γσʔλ͕গͳ͍ʢ ຕʣ৔߹
    ࠷దԽͨ͠άϥϑߏ଄ɿϥϕϧ͋Γσʔλ਺ʹ͓͚Δ܏޲<>
    4,000 label
    2,000 label 6,000 label
    1. ResNet32 (55.94%) Consistency
    2. WRN28_6 (57.81%)
    KD
    Consistency Label
    Supervised
    Label
    Supervised
    <> <>
    ֶशํ๏ʢϊʔυ਺̎ʣ ֶशํ๏ʢϊʔυ਺̏ʣ
    ϥϕϧ͋Γσʔλ͕গͳ͍৔߹ɿ
    Ұகੑਖ਼ଇԽʢ NPEFMʣͱৠཹʢ૬ޓֶशʣ͕ޮՌత
    Π
    ৠཹ
    NPEFM
    Π
    NPEFM
    Π
    NPEFM
    Π
    .FBO5FBDIFS
    ૬ޓֶश

    View full-size slide

  68. w ϥϕϧ͋Γσʔλ͕ଟ͍ʢ ຕʣ৔߹
    ࠷దԽͨ͠άϥϑߏ଄ɿϥϕϧ͋Γσʔλ਺ʹ͓͚Δ܏޲<>
    ֶशํ๏ʢϊʔυ਺̎ʣ ֶशํ๏ʢϊʔυ਺̏ʣ
    <> <>
    Gate関数 学習方法
    Gate function Learning method
    1. ResNet32 (62.76%) Consistency
    2. WRN28_6 (60.4%)
    PseudoLabeling
    ֶशࡁΈϞσϧ
    ٖࣅϥϕϦϯά
    ٖࣅϥϕϦϯά
    ϥϕϧ͋Γσʔλ͕ଟ͍৔߹ɿ
    ˠධՁର৅ϊʔυ͸ٖࣅϥϕϦϯάΛ༻ֶ͍ͯश

    View full-size slide

  69. w ϊʔυ਺ͷ৔߹
    ࠷దԽͨ͠άϥϑߏ଄ɿϊʔυ਺͕ଟ͍࣌ͷ܏޲
    ֶशํ๏
    ʢϥϕϧ͋Γσʔλ ຕʣ
    ֶशํ๏
    ʢϥϕϧ͋Γσʔλ ຕʣ
    <> <>
    Gate関数 学習方法
    Gate function Learning method
    label 6,000 label 10,000 label
    8,000 label
    .FBO5FBDIFS
    .FBO5FBDIFS
    ૬ޓֶश
    ิॿϊʔυ͸.FBO5FBDIFSΛ಺แˠࢦ਺ҠಈฏۉϞσϧͰΑΓྑ͍5FBDIFSΛֶश
    ϊʔυ਺͕ଟ͍৔߹͸.FBO5FBDIFSͰิॿϊʔυΛվળ͢Δ͜ͱ͕ޮՌత
    ɹˠ൒ڭࢣ͋Γڞಉֶश

    View full-size slide

  70. w ධՁର৅ϊʔυʹରͯ͠खಈͰΤοδΛ௥Ճ
    ϥϕϧ͔ΒͷΤοδɹɹɹɹɹɹɿڭࢣ͋Γֶश
    ධՁର৅ϊʔυ͔Β̎΁ͷΤοδɿ.FBO5FBDIFSͱͷ૬ޓֶश
    ୳ࡧͰಘͨ஌ݟΛ׆͔ͨ͠खಈઃܭʹΑΔߋͳΔվળ
    ख࡞ۀͰઃܭͨ͠άϥϑʢʣ
    ୳ࡧͰ֫ಘͨ͠άϥϑʢʣ
    QU޲্
    ୳ࡧͰಘͨάϥϑͱ஌ݟΛ׆͔͠खಈઃܭͰ͞Βʹվળ
    .FBO5FBDIFSͰิॿϊʔυΛվળͭͭ͠
    ٖࣅϥϕϦϯάͰֶश
    .FBO5FBDIFSͰิॿϊʔυΛվળͭͭ͠
    ٖࣅϥϕϦϯάͰ NPEFMͱ૬ޓֶश
    Π
    ֶशํ๏ ֶशํ๏
    学習方法
    n Learning method
    .FBO5FBDIFS
    ٖࣅϥϕϦϯά
    Gate関数 学習方法
    Gate function Learning method
    .FBO5FBDIFS
    ٖࣅϥϕϦϯά
    Ұ؏ੑਖ਼ଇԽ
    ૬ޓֶश 'FFECBDL

    ڭࢣ͋Γֶश
    -BCFM

    View full-size slide

  71. w άϥϑ୳ࡧʹΑΓ৽͍͠൒ڭࢣ͋Γڞಉֶश๏Λ୳ࡧ
    άϥϑ୳ࡧʹΑΓಘΒΕͨޮՌతͳ൒ڭࢣ͋Γڞಉֶशʹ͓͚Δ஌ݟ
    ֶशͷܦաͱͱ΋ʹֶशઓུΛมԽͤ͞Δ͜ͱͰߴਫ਼౓Խ
    ֶशޙ൒ʹҰ؏ੑਖ਼ଇԽʢ NPEFMʣΛߦ͏͜ͱ͕༗ޮ
    ϥϕϧ͋Γσʔλ਺͝ͱʹ࠷దͳֶशઓུ͸ҟͳΔ
    ϥϕϧ͋Γσʔλ͕ଟ͍৔߹͸ٖࣅϥϕϦϯάͰͷֶश͕༗ޮɹ
    ෳ਺ϞσϧΛ༻͍ͨڞಉֶश͸൒ڭࢣ͋Γֶशʹ΋༗ޮ
    ϊʔυ਺͕ଟ͍৔߹͸.FBO5FBDIFSͰิॿϊʔυΛվળ͢Δ͜ͱͰޮՌతͳ൒ڭࢣ͋ΓڞಉֶशΛ࣮ݱ
    ୳ࡧͰಘͨάϥϑͱ஌ݟΛ׆͔͠खಈઃܭͰ͞Βʹվળ
    ϊʔυ਺ɼϥϕϧ͋Γσʔλ਺ ͷ୳ࡧͰಘͨάϥϑͷਫ਼౓ΛखಈઃܭͰQUվળ
    Π
    ·ͱΊɿ஌ࣝసҠάϥϑʹΑΔ࠷దͳ൒ڭࢣ͋Γਂ૚ڞಉֶशͷ୳ࡧ

    View full-size slide

  72. ૉਓൃ૝ݰਓ࣮ߦ
    த෦େֶϩΰ
    த෦େֶϩΰ

    ʲϑΣϩʔ͔Βͷϝοηʔδʳ ৘ใɾγεςϜιαΠΤςΟࢽ ୈ 26 רୈ 4 ߸ʢ௨ר 105 ߸ʣ
    ૉਓൃ૝ݰਓ࣮ߦ 2.0
    ϑΣϩʔ ౻٢ ߂࿱
    த෦େֶ
    ʮண؟େہணखখہɼૉਓൃ૝ݰਓ࣮ߦʯ͸ɼ
    2006 ೥ 8 ݄ʹࡏ֎ݚڀͰถࠃΧʔωΪʔϝϩϯ
    େֶϩϘοτ޻ֶݚڀॴʹ 1 ೥ؒ଺ࡏ͠ɼؼࠃ
    ͷࡍʹۚग़෢༤ઌੜ͔Β௖͍ͨݴ༿Ͱ͋Δɽ
    ʮૉ
    ਓൃ૝ݰਓ࣮ߦʯͱ͸ɼ
    ۚग़ઌੜͷஶॻ [1] ʹΑ
    Δͱɼ
    ʮൃ૝͸୯७ɼૉ௚ɼࣗ༝ɼ؆୯Ͱͳ͚Ε
    ͹ͳΒͳ͍ɽ͔͠͠ɼൃ૝Λ࣮ߦʹҠ͢ʹ͸஌
    ͕͍ࣝΔɼख़࿅͞Εٕ͕͍ͨΔʯͱ͍͏͜ͱͰ
    ͋Δɽචऀ͸ͦΕҎདྷɼ͜ͷݴ༿ΛϞοτʔʹ
    ͯ͠ݚڀʹऔΓ૊ΜͰ͍Δɽ
    ͔͠͠ɼ
    ʮݴ͏͸қ
    ͘ɼߦ͏͸೉͠ʯͷయܕͰ͋Γɼ࣮ફ͢Δͷ͸
    ͳ͔ͳ͔೉͍͠ɽଟ͘ͷ࿦จΛಡΜͰ͍͘ͱ஌
    ͕ࣝਂ·Γઐ໳ੑ͸ߴ͘ͳΔ͕ɼͦΕ͕োนͱ
    ͳͬͯɼຊ࣭Ͱ͸ͳ͘খ͞ͳ͜ͱʹண໨ͨ͠໰
    ୊ઃఆΛߦ͍͕ͪͰ͋Δɽ·ͨɼຊ࣭Λଊ͑ͯ
    ΋޻෉Λ۪ͤͣ௚ʹ࣮૷͢Δͱ͏·͘ಈ͔ͳ͍
    ͜ͱ͕͋Δɽ
    ຊߘͰ͸ɼ
    ໿ 10 ೥Λܦͯʮૉਓൃ
    ૝ݰਓ࣮ߦʯʹগ͚͚ͩۙͮͨ͠ͷͰ͸ͱࢥ͏
    චऀΒͷݚڀ 2 ྫʹ͍ͭͯ঺հ͠ɼ࠷ۙɼࣗ෼
    ͳΓʹࢥ͍ඳ͘ʮૉਓൃ૝ݰਓ࣮ߦʯͷΞοϓ
    σʔτΛڞ༗͍ͨ͠ɽ
    2010 ೥ࠒɼը૾ؒͷରԠ఺ϚονϯάͷͨΊ
    ͷಛ௃఺ݕग़ɾهड़ͷݚڀ͕ଟ͘औΓ૊·Εͯ
    ͍ͨɽதͰ΋ɼࣹӨมԽΛ൐͏ը૾ؒͷରԠ఺
    Ϛονϯά͸ɼΩʔϙΠϯτͷಛ௃Λදݱ͢Δ
    ΞϑΟϯྖҬΛٻΊΔඞཁ͕͋Γɼ೉͍͠໰୊
    Ͱ͋ͬͨɽैདྷख๏Ͱ͸ɼΩʔϙΠϯτʹର͠
    ͯҰͭͷΞϑΟϯྖҬ͔͠ਪఆ͠ͳ͍ͨΊɼը
    ૾ͷมܗ΍ΩʔϙΠϯτͷҐஔͣΕͷӨڹʹΑ
    ΓҟͳΔΞϑΟϯྖҬΛਪఆͯ͠͠·͏ͱ͍͏
    ໰୊͕͋ͬͨɽ͜Ε͸ɼہॴత୳ࡧΛߦ͏͜ͱ
    ͕ݪҼͰ͋Γɼ
    ʮண؟খہணखখہʯͱݴ͑Δɽ
    2015 ೥ʹචऀΒ͕ࠃࡍձٞ ICCV ʹͯൃදͨ͠
    ʮඇ౳ํੑ LoG ϑΟϧλʹΑΔෳ਺ͷΞϑΟϯ
    ྖҬͷਪఆʯ
    [2] Ͱ͸ɼ༷ʑͳପԁܗঢ়ͷඇ౳ํ
    ੑ LoG ϑΟϧλΛ༻͍ͯෳ਺ͷΞϑΟϯྖҬΛ
    ਪఆ͢Δ͜ͱΛఏҊͨ͠ɽγϯϓϧʹɼҰͭͰ
    ͸ͳ͘ෳ਺ͷྖҬ͕͋ͬͯ΋Α͍ͷͰ͸ɼͱ͍
    ͏ʮૉਓൃ૝ʯͰ͋Δɽ
    ͔͠͠ɼ
    ͍࣮͟૷ͱͳΔ
    ͱɼ
    ඇ౳ํੑ LoG ϑΟϧλʹ͸ x ํ޲ͷεέʔ
    ϧɼy ํ޲ͷεέʔϧɼճస֯ͷ 3 ύϥϝʔλ
    ͕͋Γɼͦͷ૊Έ߹Θͤ͸਺ઍछྨͱͳΔɽෳ
    ਺ͷΞϑΟϯྖҬΛਪఆ͢ΔͨΊɼযΔ͕༨Γ
    ͜ͷ਺ઍछྨͷϑΟϧλશͯΛ৞ΈࠐΉॲཧΛ
    ͜ͷ··ߦ͏ͱɼ๲େͳܭࢉίετ͕ඞཁͱͳ
    Δɽ
    ͦ͜Ͱɼ
    ʮݰਓ࣮ߦʯͱͯ͠ɼ਺ઍछྨͷඇ
    ౳ํੑ LoG ϑΟϧλ܈Λಛҟ஋෼ղʹΑΓٻ
    Ίͨ 14 छྨͷݻ༗ϑΟϧλͰۙࣅ͠ɼ৞Έࠐ
    ΈॲཧΛޮ཰తʹܭࢉ͢Δ͜ͱʹͨ͠ɽ͜Εʹ
    ΑΓɼෳ਺ͷΞϑΟϯྖҬΛޮ཰తʹٻΊΔ͜
    ͱ͕Ͱ͖ɼࣹӨมԽΛ൐͏ը૾ؒͷରԠ఺Ϛο
    νϯάͷߴਫ਼౓ԽΛ࣮ݱͨ͠ɽ͜ͷݚڀʹ͓͍
    ͯɼ
    ʮૉਓൃ૝ݰਓ࣮ߦʯͷݴ༿͕ݚڀͷํ޲ੑ
    ΍ਐΊํΛܾΊΔखॿ͚Λͯ͘͠ΕͨΑ͏ʹࢥ
    ͑ɼ2006 ೥͔Β໿ 10 ೥ΛܦͯɼΑ͏΍͘ʮૉ
    ਓൃ૝ݰਓ࣮ߦʯʹҰา͚ۙͮͨͱࢥ͑Δݚڀ
    Ͱ͋ͬͨɽ͜ͷݚڀҎޙ΋ɼৗʹɼૉਓൃ૝Ͱ
    ݰਓ࣮ߦʹͳ͍ͬͯΔ͔Λࣗ໰ࣗ౴͠ͳ͕Βݚ
    ڀʹऔΓ૊ΜͰདྷͨɽ
    2012 ೥Ҏ߱ɼਂ૚ֶश͕ओମͱͳͬͨίϯϐ
    16
    IUUQTXXXKTUBHFKTUHPKQBSUJDMFJFJDFJTTKPVSOBM@@BSUJDMFDIBSKB

    View full-size slide

  73. ίϯϐϡʔλϏδϣϯ࠷લઢ
    த෦େֶϩΰ
    த෦େֶϩΰ

    IUUQTXXXLZPSJUTVQVCDPKQCPPLEFUBJM
    ίϯϐϡʔλϏδϣϯ࠷લઢ Spring 2022 ʗר಄ݴ
    5
    ר಄ݴ Spring 2022
    Visual HullతνϡʔτϦΞϧͷεεϝ
    ˙౻٢߂࿱
    2000 ೥લޙʹऔΓ૊·Ε͍ͯͨίϯϐϡʔλϏδϣϯͷΞϧΰϦζϜͰ͋Δ
    ࢹମੵަࠩ๏ʢvisual hullʣΛ͝ଘ஌ͩΖ͏͔ɻࢹମੵަࠩ๏͸ɼLaurentini
    ͕ఏҊͨ͠ Shape-from-silhouette ʹΑΔ 3D ࠶ߏ੒ख๏Ͱ͋Δɻ·ͣɼΧϝ
    ϥࢹ఺͔ΒγϧΤοτը૾1) Λ༻͍ͯ෮ݩର৅ͷΦϒδΣΫτΛ౤Өͨ͠γϧ 1) γϧΤοτը૾ͷલܠϚεΫ
    ͸ɼ෮ݩରԠͰ͋ΔΦϒδΣ
    Ϋτͷ 2 ࣍ݩ౤ӨͰ͋Δɻ
    Τοτԁਲ਼Λ࡞੒͢ΔɻҟͳΔࢹ఺ͰࡱӨͨ͠γϧΤοτը૾͔Βੜ੒͞Εͨ
    ԁਲ਼ͷަ఺͸ Visual Hull ͱݺ͹Εɼ͜ͷަ఺ΛٻΊΔ͜ͱͰΦϒδΣΫτͷ 3
    ࣍ݩܗঢ়ͷ෮ݩ͕ՄೳͱͳΔɻਤ 1 ͸ɼCV ͳΒͼʹ CG քͰ༗໊ͳ “Stanford
    Bunny”
    ʢhttp://graphics.stanford.edu/data/3Dscanrep/ʣ
    ͱݺ͹ΕΔ 3D Φ
    ϒδΣΫτΛ෮ݩͨ͠ྫͰ͋Δɻগͳ͍ࢹ఺ͷγϧΤοτը૾͔Β෮ݩͨ͠ 3
    ࣍ݩܗঢ়͸ɼຊདྷͷόχʔͷ 3 ࣍ݩܗঢ়ʹͳ͍ͬͯͳ͍ɻҰํͰɼଟ਺ͷҟͳΔ
    ࢹ఺ͷγϧΤοτը૾Λ༻͍Δͱɼਖ਼֬ͳ 3 ࣍ݩܗঢ়Λ෮ݩ͢Δ͜ͱ͕Ͱ͖Δɻ
    ͜Ε͸ɼݪஶ࿦จΛಡΉ͜ͱʹ͓͍ͯ΋ಉ༷Ͱ͋Δͱࢲ͸ࢥ͏ɻ࿦จͷຊ࣭
    ͕Ͳ͜ʹ͋Δ͔Λਂ͘ཧղ͢Δʹ͸ɼҰࢹ఺͔ΒಡΈࠐΉͷͰ͸ͳ͘ɼҟͳΔ
    (a) 3 ࢹ఺
    (b) 80 ࢹ఺
    ʜ
    A
    B
    C
    A
    B
    C
    γϧΤοτը૾ ෮ݩ݁Ռ
    ਤ 1 ࢹମੵަࠩ๏ʢvisual hullʣ
    ɻhttp://www.sanko-shoko.net/note.php?
    id=tjly ͷίʔυΛར༻ͯ͠࡞੒ɻ
    ί
    ϯϐϡʔλ
    Ϗδϣ
    ϯ࠷લઢɹ
    4QSJOH
    Ҫ৲ળٱɾ
    ڇٱ঵޹ɾ
    ยԬ༟༤ɾ
    ౻٢߂࿱ฤ
    IUUQTXXXLZPSJUTVQVCDPKQCPPLEFUBJM

    View full-size slide

  74. .13(›5PVS
    74
    த෦େֶϩΰ
    த෦େֶϩΰ
    IUUQTXXXZPVUVCFDPNXBUDI W(LV,'5&

    View full-size slide

  75. ػց஌֮ϩϘςΟΫεݚڀάϧʔϓ
    த෦େֶϩΰ
    த෦େֶϩΰ
    ڭत
    ౻٢߂࿱ Hironobu Fujiyoshi E-mail: [email protected]
    1997೥ த෦େֶେֶӃത࢜ޙظ՝ఔमྃ, 1997೥ ถΧʔωΪʔϝϩϯେֶϩϘοτ޻ֶݚڀॴPostdoctoral Fellow, 2000೥ த෦େֶ޻ֶ෦৘ใ޻ֶՊߨࢣ, 2004೥ த෦େֶ।ڭत,
    2005೥ ถΧʔωΪʔϝϩϯେֶϩϘοτ޻ֶݚڀॴ٬һݚڀһ(ʙ2006೥), 2010೥ த෦େֶڭत, 2014೥໊ݹ԰େֶ٬һڭत.

    ܭࢉػࢹ֮ɼಈը૾ॲཧɼύλʔϯೝࣝɾཧղͷݚڀʹैࣄɽ

    ϩϘΧοϓݚڀ৆(2005೥)ɼ৘ใॲཧֶձ࿦จࢽCVIM༏ल࿦จ৆(2009೥)ɼ৘ใॲཧֶձࢁԼه೦ݚڀ৆(2009೥)ɼը૾ηϯγϯάγϯϙδ΢Ϝ༏लֶज़৆(2010, 2013, 2014೥) ɼ
    ిࢠ৘ใ௨৴ֶձ ৘ใɾγεςϜιαΠΤςΟ࿦จ৆(2013೥)ଞ
    ڭत
    ࢁԼོٛ Takayoshi Yamashita E-mail:[email protected]
    2002೥ ಸྑઌ୺Պֶٕज़େֶӃେֶത࢜લظ՝ఔमྃ, 2002೥ ΦϜϩϯגࣜձࣾೖࣾ, 2009೥ த෦େֶେֶӃത࢜ޙظ՝ఔमྃ(ࣾձਓυΫλʔ), 2014೥ த෦େֶߨࢣɼ2017
    ೥ த෦େֶ।ڭतɼ2021೥ த෦େֶڭतɽ

    ਓͷཧղʹ޲͚ͨಈը૾ॲཧɼύλʔϯೝࣝɾػցֶशͷݚڀʹैࣄɽ

    ը૾ηϯγϯάγϯϙδ΢Ϝߴ໦৆(2009೥)ɼిࢠ৘ใ௨৴ֶձ ৘ใɾγεςϜιαΠΤςΟ࿦จ৆(2013೥)ɼిࢠ৘ใ௨৴ֶձPRMUݚڀձݚڀ঑ྭ৆(2013೥)ड৆ɽ
    ߨࢣ
    ฏ઒ཌྷ Tsubasa Hirakawa E-mail:[email protected]
    2013೥ ޿ౡେֶେֶӃത࢜՝ఔલظऴྃɼ2014೥ ޿ౡେֶେֶӃത࢜՝ఔޙظೖֶɼ2017೥ த෦େֶݚڀһ (ʙ2019೥)ɼ2017೥ ޿ౡେֶେֶӃത࢜ޙظ՝ఔमྃɽ2019
    ೥ த෦େֶಛ೚ॿڭɼ2021೥ த෦େֶߨࢣɽ2014೥ ಠཱߦ੓๏ਓ೔ຊֶज़ৼڵձಛผݚڀһDC1ɽ2014೥ ESIEE Paris٬һݚڀһ (ʙ2015೥)ɽ
    ίϯϐϡʔλϏδϣϯɼύλʔϯೝࣝɼҩ༻ը૾ॲཧͷݚڀʹैࣄ

    View full-size slide