MLOpsの「壁」を乗り越える、LINEヤフーの Data Quality as Code

データの品質はMLモデルの信頼性に関わる重要な指標ですが、サイエンティストと運用者の連携は容易ではなく、MLOpsをスケールアウトする上で運用面のボトルネックとなりやすいポイントです。このセッションでは、信頼性の高いMLモデル運用を効果的に実現する、LINEヤフーのData Quality as Codeの仕組みを紹介します。

Transcript

1. ˜
   -:
   $PSQPSBUJPO .-0QTͷʮนʯΛ৐Γӽ͑Δɺ -*/&Ϡϑʔͷ %BUB
   2VBMJUZ
   BT
   $PEF -*/&Ϡϑʔגࣜձࣾ
   "*ϓϥοτϑΥʔϜ෦ .-0QTΤϯδχΞ ݹ઒
   ৽ &.BJM BGVSVLBX!MZDPSQDPKQ ݸਓ9 !@PSOFX ݸਓ'BDFCPPL !PMESOFX
2. ˜
   -:
   $PSQPSBUJPO എܠ .-0QTʹ͓͚Δӡ༻ͷ৴པੑ 
3. ˜
   -:
   $PSQPSBUJPO .-0QT  .- 0QT ࣄۀୡ੒ͱ͍͏ڞ௨໨ඪͷͨΊʹ .-Ϟσϧͷ։ൃऀͱӡ༻ऀ͕ڠྗͯ͠ ʮ.-Ϟσϧ։ൃͷੜ࢈ੑʯͱʮ.-Ϟσϧӡ༻ͷ৴པੑʯΛ ಉ࣌ʹ࠷దԽ͢Δख๏ͷ͜ͱ
4. ˜
   -:
   $PSQPSBUJPO ։ൃͷੜ࢈ੑ  .- 0QT "*1'͕.-Ϟσϧ։ൃͷੜ࢈ੑΛ࠷େԽ ଟ͘ͷࣄۀͰࢪࡦΛߴ଎ʹ։ൃͰ͖Δମ੍Λ੔උ "*1' ˞"*1'ɿ"*ϓϥοτϑΥʔϜɻ ࣾ಺޲͚ʹ։ൃɾఏڙ͍ͯ͠Δ"*ج൫ɻ "*ϓϥοτϑΥʔϜʹΑΔ࠷େԽ
5. ˜
   -:
   $PSQPSBUJPO ӡ༻ͷ৴པੑ  .- 0QT .-0QTશମͱͯ͠ ։ൃͷੜ࢈ੑΑΓ΋ӡ༻ͷ৴པੑ͕ εέʔϧͷϘτϧωοΫʹͳ͖ͬͯͨ ࣄނͷ૿Ճ 55%ɾ55.ͷ௕ظԽ ˞55%ɿ5JNF
   UP
   %FUFDU
   ʢࣄނݕग़࣌ؒʣ ˞55.ɿ5JNF
   UP
   .JUJHBUF
   ʢࣄނܰݮ࣌ؒʣ
6. ˜
   -:
   $PSQPSBUJPO .-0QTͷߏ଄త՝୊  .- 0QT ӡ༻ͷ৴པੑΛ௿Լ͍ͤͯ͞Δσʔλࣄނͷ55%ɾ55.͕௕ظԽ͢Δͷ͸ ։ൃऀ΍ӡ༻ऀͷೳྗ͕ݪҼͰ͸ͳ͘ ։ൃऀͱӡ༻ऀͰσʔλ඼࣭͕͏·͘ڞ༗Ͱ͖͍ͯͳ͍͜ͱ͕ݪҼ σʔλ඼࣭͕Θ͔Βͳ͍ 55%ɾ55.͕ ѱԽ͠΍͍͢ σʔλ඼࣭ͷཁ݅ఆ͕ٛ ෆे෼ͳ··ϦϦʔε ͞Ε΍͍͢ σʔλ඼࣭վળͷ ίϛϡχέʔγϣϯϩε͕ ى͖΍͍͢ อकతͳ඼࣭؅ཧʹ େ͖ͳ޻਺Λׂ͚ͳ͍
7. ˜
   -:
   $PSQPSBUJPO ՝୊എܠ΍໰୊ҙࣝɺऔΓ૊Έͷৄࡉ ୈճ
   %BUB$FOUSJD
   "*ษڧձͷൃදΛࢀর  σʔλ඼࣭ΛίʔυԽʂ
   -*/&Ϡϑʔͷ.-0QTΛ࠷దԽ͢Δ
   "$1
   %BUB
   2VBMJUZ
   ͷ঺հ IUUQTTQFBLFSEFDLDPNMZDPSQUFDI@KQEBUBDFOUSJDBJBDQEBUBRVBMJUZ https://www.youtube.com/watch?v=x_jwctxjy_U&t=2032s :PV5VCFͰಈը΋ແྉެ։தʂ
8. ˜
   -:
   $PSQPSBUJPO Ξϓϩʔν "$1
   %BUB
   2VBMJUZ 
9. ˜
   -:
   $PSQPSBUJPO "$1
   %BUB
   2VBMJUZ  "$1
   %BUB
   2VBMJUZ͸*40ʹجͮ͘σʔλ඼࣭؅ཧϓϩηεߏஙΛࢧԉ͢ΔαʔϏε ։ൃऀͱӡ༻ऀ͕ίϥϘϨʔγϣϯͯ͠αεςφϒϧͳσʔλ඼࣭؅ཧΛߏஙͰ͖Δ σʔλ඼࣭ܭը %BUB
   2VBMJUZ
   1MBOOJOH σʔλ඼੍࣭ޚ %BUB
   2VBMJUZ
   $POUSPM σʔλ඼࣭อূ %BUB
   2VBMJUZ
   "TTVSBODF σʔλ඼࣭վળ %BUB
   2VBMJUZ
   *NQSPWFNFOU σʔλ඼࣭؅ཧϓϩηε Ϧιʔεنఆ σʔλؔ࿈αϙʔτ ˞"$1ɿࣾ಺޲͚ʹఏڙ͍ͯ͠Δ"*ಛԽͷ,VCFSOFUFT؀ڥɻ
10. ˜
    -:
    $PSQPSBUJPO %2.- %BUB
    2VBMJUZ
    .PEFM
    -BOHVBHF  σʔλ඼࣭Ϟσϧ Λ ίʔυͰఆٛ͢ΔͨΊͷݴޠΛ։ൃ ඼ ࣭ ཁ ٻ ର৅ σʔλ9 ໨త Ϟσϧ.ͷֶश༻σʔλ ཁٻ ͋ΔϞσϧ.ͷֶशʹ࢖͏σʔλ9͸ɺ༗ޮͳֶश݁ՌΛಘΔͨΊʹ  ݅Ҏ্ͷσʔλ͕ͳ͚Ε͹ͳΒͳ͍ ඼ ࣭ ཁ ݅ ଌఆؔ਺ " ଌఆྔཁૉ" σʔλ9ͷ݅਺ ଌఆج४  ݅ΑΓଟ͘ͳ͚Ε͹ͳΒͳ͍ $PVOU NVTU
    CF
     
    ͜ͷʮσʔλ඼࣭Ϟσϧʯͱ͸ l*40*&$
    
    4PGUXBSF
    FOHJOFFSJOH 4PGUXBSF
    QSPEVDU
    2VBMJUZ
    3FRVJSFNFOUT
    BOE
    &WBMVBUJPO
    42VB3& %BUB
    RVBMJUZ
    NPEFMz Ͱఆٛ͞ΕΔ΋ͷΛࢦ͢ɻ
11. ˜
    -:
    $PSQPSBUJPO %2.-ʹΑΔ؅ཧϓϩηε  .- 0QT ੍ޚ อূ ܭը վળ ίʔυʹΑΔཁ݅ఆٛ ίʔυϨϏϡʔ ίʔυ͔ΒࣗಈධՁ ίʔυʹΑΔҰ؏ੑ
12. ˜
    -:
    $PSQPSBUJPO %2.-ͷޮՌ  .- 0QT σʔλ඼࣭ͷఆ͔ٛΒ ࣗಈͰ੍ޚ 55%ɾ55.ͷվળ σʔλ඼࣭ͷཁ݅Λ ίʔυͰఆٛ͢Δ͚ͩͰ ੒Ռ෺ͷ඼࣭Λ޲্ ίʔυϕʔεͰ ඼࣭ͷ؅ཧͱվળΛ ܧଓతʹ࣮ࢪ ίʔυϕʔεͰ ඼࣭ཁ݅ͷվળ͚ͩʹूத ඞཁ࠷খݶͷ޻਺Ͱӡ༻ʹߩݙ %BUB$FOUSJDͳϓϩηεʹΑͬͯ.-0QTΛՃ଎
13. ˜
    -:
    $PSQPSBUJPO ಺෦ߏ଄ "$1
    %BUB
    2VBMJUZ 
14. ˜
    -:
    $PSQPSBUJPO ඼࣭ܭը 
    ཁ݅ఆٛ ,VCFSOFUFTΧελϜϦιʔεͱͯ͠σʔλ඼࣭ϞσϧΛఆٛ  σʔλ඼࣭ཁ݅Λ%2.-ʹΑΔίʔυͰఆٛ͠ɺ:".-΍$6&ͳͲͷܗࣜͰ؅ཧ BQJ7FSTJPO
    EBUBRVBMJUZZBIPPDPKQWBMQIB
    LJOE %BUB2VBMJUZ.PEFM NFUBEBUB
    OBNF FYBNQMF TQFD
    
    EBUB4QFD USJOP UBCMF USBJO@EBUB@Y
    SFRVJSFNFOUT
    
    
    
     DPEF
    $PVOU
    NVTU
    CF
      
    ϦιʔεͷϑΟʔϧυ͸εϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ
15. ˜
    -:
    $PSQPSBUJPO ඼࣭ܭը 
    ཁ݅؅ཧ (JU)VCͰόʔδϣϯ؅ཧɾ,VCFSOFUFTʹಉظ  σʔλ඼࣭Ϟσϧ
    
    %BUB2VBMJUZ.PEFM
    %2. BQJ7FSTJPO
    EBUBRVBMJUZZBIPPDPKQWBMQIB
    LJOE %BUB2VBMJUZ.PEFM NFUBEBUB
    OBNF FYBNQMF TQFD
    
    EBUB4QFD USJOP UBCMF USBJO@EBUB@Y
    SFRVJSFNFOUT
    
    
    
     DPEF
    $PVOU
    NVTU
    CF
       %2. FYBNQMF (JU)VC BQQMZ ,VCFSOFUFT $*$%Ͱݕূɾಉظ
    ϦιʔεͷϑΟʔϧυ͸εϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ
16. ˜
    -:
    $PSQPSBUJPO ඼࣭ܭը 
    ੩తղੳ "ENJTTJPO
    8FCIPPLͱ੩తղੳ  "ENJTTJPO
    8FCIPPLͰཁ݅Λ੩తղੳ ߏจղੳɾܕνΣοΫͳͲɺཁ݅ͷݕূ͕Մೳ BQJ7FSTJPO
    EBUBRVBMJUZZBIPPDPKQWBMQIB
    LJOE %BUB2VBMJUZ.PEFM NFUBEBUB
    OBNF FYBNQMF TQFD
    
    EBUB4QFD USJOP UBCMF USBJO@EBUB@Y
    SFRVJSFNFOUT
    
    
    
     DPEF
     $PVOU
    XIFSF
    BHF
    JT
    
    
    lBCDz NVTU
    CF
       %2. FYBNQMF BHFͷܕ͕OVNCFSͱTUSJOHͰিಥ͍ͯ͠Δ  "ENJTTJPO
    8FCIPPLʹΑͬͯ $SFBUF6QEBUF࣌ʹղੳ ΋͠Τϥʔ͕͋Ε͹ૢ࡞ΛϒϩοΫ͢Δ WBMJEBUF 
    %2.-͸$6&ݴޠʹΠϯεύΠΞ͞ΕͨܕγεςϜΛ࣮૷͓ͯ͠Γɺ ܕਪ࿦Λߦ͍σʔλ඼࣭ཁ݅ͱͯ͠ໃ६͕͋Δ৔߹ʹΤϥʔΛใࠂ͢Δɻ ͜ͷྫͰ͸ɺϑΟʔϧυ͕਺஋͔ͭจࣈྻͱ͍͏ཁ݅͸͋Γಘͳ͍ͷͰɺ ܕγεςϜ͸OVNCFSͱTUSJOHͷিಥΛใࠂ͢Δɻ ,VCFSOFUFT (JU)VC
    ϦιʔεͷϑΟʔϧυ͸εϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ
17. ˜
    -:
    $PSQPSBUJPO ඼੍࣭ޚ 
    ධՁ ,VCFSOFUFTϦιʔεͰσʔλ඼࣭ධՁΛ࣮ߦ  σʔλ඼࣭ධՁ
    
    %BUB2VBMJUZ&WBMVBUJPO
    %2& BQJ7FSTJPO
    EBUBRVBMJUZZBIPPDPKQWBMQIB
    LJOE %BUB2VBMJUZ&WBMVBUJPO TQFD
    
    NPEFM3FG OBNF FYBNQMF %2& FYBNQMFYYYY  %2. FYBNQMF ,VCFSOFUFT %2&ϦιʔεΛ࡞੒ ධՁର৅ͷϞσϧΛࢀর SFG DSFBUF
    ϦιʔεͷϑΟʔϧυ͸εϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ
18. ˜
    -:
    $PSQPSBUJPO ඼੍࣭ޚ 
    ࣗಈଌఆ ඼࣭ධՁʹΑΔཁ݅ͷࣗಈଌఆ  BQJ7FSTJPO
    EBUBRVBMJUZZBIPPDPKQWBMQIB
    LJOE %BUB2VBMJUZ&WBMVBUJPO TQFD
    
    NPEFM3FG OBNF FYBNQMF 4&-&$5
    DPVOU '30.
    USBJO@EBUB@Y %2. FYBNQMF %2& FYBNQMFYYYY 8' FYBNQMFYYYYYYYY TUBUVT SFTVMUT  NFBTVSF
    $PVOU DSJUFSJB  WBMVF
     DPNQMJBOU GBMTF %2&3 FYBNQMFYYYY    %BUBTPVSDF  ,VCFSOFUFT ࣗಈͰϫʔΫϑϩʔΛ࡞੒ σʔλιʔεʹ߹ΘͤͯΫΤϦͳͲΛੜ੒ ཁ݅ͷධՁʹඞཁͳଌఆྔΛશͯࣗಈͰଌఆ ධՁ݁Ռͷ൓ө DSFBUF NFBTVSF TBWF
    ࣮ࡍ͸΋ͬͱෳࡶɺେྔͷଌఆྔΛ ॲཧͰ͖ΔΑ͏ʹ࠷దԽ͞ΕΔɻ ෳ਺ͷσʔλιʔεΛαϙʔτ͠ ࣾ಺ͷଟ͘ͷσʔλʹରԠ͍ͯ͠Δɻ
    ϦιʔεͷϑΟʔϧυ͸εϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ
19. ˜
    -:
    $PSQPSBUJPO ඼੍࣭ޚ 
    ΞϥʔτɾϨϙʔτ ධՁ݁Ռ͔ΒΞϥʔτɾϨϙʔτͷ࡞੒  BQJ7FSTJPO
    EBUBRVBMJUZZBIPPDPKQWBMQIB
    LJOE %BUB2VBMJUZ&WBMVBUJPO TQFD
    
    NPEFM3FG OBNF FYBNQMF TUBUVT SFTVMUT  NFBTVSF
    $PVOU DSJUFSJB  WBMVF
     DPNQMJBOU GBMTF %2& FYBNQMFYYYY 4MBDL %BUBCBTF 8FC
    "QQ   ඼࣭ʹҟৗ͕͋Γ·͢ʂ $PVOU
    NVTU
    CF
    
    HPU
     ,VCFSOFUFT 4MBDL౳ʹΞϥʔτ %#ʹଌఆྔ΍ධՁΛه࿥ աڈͷଌఆྔͷਪҠͷՄࢹԽ΍ ධՁͷ෼ੳϨϙʔτΛӾཡͰ͖Δ BMFSU QFSTJTU
    ϦιʔεͷϑΟʔϧυ͸εϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ
20. ˜
    -:
    $PSQPSBUJPO ඼੍࣭ޚ 
    ؂ࢹɾ੍ޚ ,VCFSOFUFTΧελϜϦιʔεͰσʔλ඼࣭ϞσϧΛఆٛ  BQJ7FSTJPO
    BSHPQSPKJPWBMQIB
    LJOE $SPO8PSLGMPX TQFD TDIFEVMF
    l!EBJMZz
    
    XPSLGMPX4QFD UFNQMBUFT  OBNF
    EBUBRVBMJUZFWBM SFTPVSDF BDUJPO
    DSFBUF NBOJGFTU c BQJ7FSTJPO
    EBUBRVBMJUZ LJOE %BUB2VBMJUZ&WBMVBUJPO TQFD
    
    NPEFM3FG OBNF FYBNQMF %2& FWBMFYBNQMFYYYY 8' FWBMFYBNQMFYYYY TUBUVT SFTVMUT  NFBTVSF
    $PVOU DSJUFSJB  WBMVF
     DPNQMJBOU GBMTF BQJ7FSTJPO
    EBUBRVBMJUZ
    LJOE %BUB2VBMJUZ&WBMVBUJPO TQFD
    
    NPEFM3FG OBNF FYBNQMF   ,VCFSOFUFT ϫʔΫϑϩʔ͔Β%2&Λ࡞੒ ධՁ݁Ռʹج͍ͮͯ෼ذɺ੍ޚ ྫɿϩʔϧόοΫɺதࢭͳͲ DSFBUF DPOUSPM
    ϦιʔεͷϑΟʔϧυ͸εϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ
21. ˜
    -:
    $PSQPSBUJPO ඼੍࣭ޚ 
    طଘγεςϜͱͷ౷߹ ඪ४తͳ,VCFSOFUFT
    "1*Λ࢖ͬͨ౷߹  %2& FWBMFYBNQMF 5BTL FWBMFYBNQMFYYYY TUBUVT SFTVMUT  NFBTVSF
    $PVOU DSJUFSJB  WBMVF
     DPNQMJBOU GBMTF BQJ7FSTJPO
    EBUBRVBMJUZ
    LJOE %BUB2VBMJUZ&WBMVBUJPO TQFD
    
    NPEFM3FG OBNF FYBNQMF   ,VCFSOFUFT "JSGMPX GSPN BJSGMPX
    JNQPSU %"( EBH
    
    %"( EBH@JElBJSGMPXFWBMz 
     FWBMVBUJPO 
    %BUB2VBMJUZ&WBMVBUJPO0QFSBUPS OBNFlFWBMFYBNQMFz UBTL@JElFWBMFYBNQMFz NPEFM@OBNFlFYBNQMFz 
     EBHEBH FWBMVBUJPO ,VCFSOFUFT
    "1*Λ࢖͑͹ ೚ҙͷ֎෦γεςϜʹ؆୯ʹ౷߹Ͱ͖Δ DSFBUF DPOUSPM
    ϦιʔεͷϑΟʔϧυ͸εϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ
22. ˜
    -:
    $PSQPSBUJPO ඼࣭վળ 
    ϨϏϡʔ ,VCFSOFUFTΧελϜϦιʔεͰσʔλ඼࣭ϞσϧΛఆٛ  BQJ7FSTJPO
    EBUBRVBMJUZZBIPPDPKQWBMQIB
    LJOE %BUB2VBMJUZ.PEFM NFUBEBUB
    OBNF FYBNQMF TQFD
    
    EBUB4QFD USJOP UBCMF USBJO@EBUB@Y
    SFRVJSFNFOUT
    
    
    
     DPEF
    $PVOU
    NVTU
    CF
     (JU)VC BQJ7FSTJPO
    EBUBRVBMJUZZBIPPDPKQWBMQIB
    LJOE %BUB2VBMJUZ.PEFM NFUBEBUB
    OBNF FYBNQMF TQFD
    
    EBUB4QFD USJOP UBCMF USBJO@EBUB@Y
    SFRVJSFNFOUT
    
    
    
     DPEF
    $PVOU
    NVTU
    CF
      DPEF
    .JO
    PG
    BHF
    NVTU
    CF
       ίʔυϕʔεͰίϛοτɾϨϏϡʔ αΠΤϯςΟετͱӡ༻ऀ͕ ඇಉظͰίϛϡχέʔγϣϯ͠΍͍͢ $*$% DPNNJU  SFWJFX $*$%ͰͷಉظʹΑΓ ཁ͔݅Β੍ޚ·Ͱ඼࣭ͷҰ؏ੑΛอূ
    ϦιʔεͷϑΟʔϧυ͸εϥΠυ༻ʹলུɾมܗ͍ͯ͠ΔՕॴ͕͋Γ·͢ɻ
23. ˜
    -:
    $PSQPSBUJPO ·ͱΊ 
24. ˜
    -:
    $PSQPSBUJPO %BUB
    2VBMJUZ
    BT
    $PEF %2.-  • ,VCFSOFUFT
    $VTUPN
    3FTPVSDF • େྔͷϦιʔεΛॲཧͰ͖ΔΑ͏ʹίϯτϩʔϥΛ࠷దԽɺ ਺ઍΛΏ͏ʹ௒͑Δσʔλ඼࣭ϞσϧͷධՁΛ҆ఆͯ͠ॲཧ • %2.-ʢ%BUB
    2VBMJUZ
    .PEFM
    -BOHVBHFʣ • ܕγεςϜΛ׬උɺఆٛஈ֊Ͱ੩తղੳʹΑΔݕূ͕Մೳ • ༷ʑͳσʔλιʔεΛαϙʔτ͢ΔίϯύΠϥΛఏڙ ͋ΒΏΔσʔλʹରͯ͠ଌఆͱධՁ͕Մೳ
25. ˜
    -:
    $PSQPSBUJPO %BUB
    2VBMJUZ
    BT
    B
    4FSWJDF "$1
    %BUB
    2VBMJUZ  • 6OJWFSTBM • υϝΠϯಛԽ
    ಛఆͷ؀ڥʹґଘ͠ͳ͍ • อकੑ
    ΧελϜϦιʔεͱ%2.-ʹΑΔએݴతͳఆٛ • ൚༻ੑ
    ,VCFSOFUFT
    "1*ʹ४ڌ͠೚ҙͷγεςϜʹ౷߹Մೳ • Ҡ২ੑ
    ҟͳΔ؀ڥͰ΋ಉ͡඼࣭ཁ݅ΛཁٻͰ͖Δ • ࣮૷ෆཁ
    σʔλ඼࣭؅ཧͷ࣮ߦͷͨΊʹཁ݅Ҏ֎ͷ࣮૷͕͍Βͳ͍ • 4FSWFSMFTT • ଈ࣌ಋೖ
    σʔλ඼࣭؅ཧΛਝ଎ʹಋೖ • ӡ༻ෆཁ
    ௥Ճͷӡ༻࡞ۀͳ͠Ͱ඼࣭؅ཧʹूத • εέʔϥϏϦςΟͱߴՄ༻ੑ
    ,VCFSOFUFTʹΑΔେن໛ల։͕Մೳ
26. ˜
    -:
    $PSQPSBUJPO ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠