LINE FACT Checker QA at Test Corner

LINE FACT Checker QA at Test Corner

LINE 訊息查證 by Hack Chen and Johnson Wu @ Test Corner #26 https://testcorner.kktix.cc/events/testcorner26

2102a6b8760bd6f57f672805723dd83a?s=128

line_developers_tw

July 07, 2020
Tweet

Transcript

  1. /,1(懱௳礚挨  %\/,1(+DQN&KHQ -RKQVRQ:X

  2. $JHQGD 碻绚胙ว /,1(碍֖吚揣懯向 $JLOH介手窕纷 介手獉਻ &,&'碉ݳ㬙粚窕纷 獮ᒒ䛑አ Machine Learning 

         
  3. - 9    1 9 碻绚胙ว

  4. 6DZ5HFHLYHGVXVSLFLRXV ,QIRUPDWLRQRQOLQH $FWLYHO\'RWKH)DFW &KHFNLQJ $IWHU)DFW&KHFNLQJZLOO IXUWKHUVKDUHWKHFRUUHFWLQIR  %  %

    % 碻绚胙ว
  5. GOV /,1(碍֖吚揣懯向 肥ፑफ़֎

  6. $JLOH介手窕纷

  7. 介手獉਻ 介殻ୌ缏ᓕቘ  ๐率玟璸  玟璸蟂֖ *8,)XQFWLRQ  玟璸蟂֖ )XQFWLRQ

     狕ྋ㮆礯介殻  介手褩ྦྷ  ஠ᥝ௔ /HYHO  %76橕肥  咳㬙粚蒈  䋿ֺ瑽෈
  8. 介手獉਻ $3,緳介  &OLHQW6LGH6FHQDULR  *RRJOH,QVSHFWRU  玟璸蟂֖  $3,6SHFGRFXPHQW

     ਞ蕕1HZPDQ物  QSP LQVWDOOJQHZPDQ  ๜瑿ᒒ珸㵕脻๜物  QHZPDQ UXQKWWSVZZZSRVWPDQFRPFROOHFWLRQVFE HHIHHGHYBHQYLURQPHQWMVRQ
  9. 介手獉਻ ,VVXH7UDFNLQJ  7LFNHW7\SH  6WDWXV  )L[YHUVLRQ  (QYL26%URZVHU

     6WHS$VLV7REH  $VVLJQWRPH  6SULQW  7LFNHWTXDQWLW\  0\FUHDWH  6WRU\ %XJ  'HYHORSLQJ  4$  &ORVH
  10. 介手獉਻ 介手奾䯤 Ticket Management API Monitoring Test Manual Test Functional

    Test Regression Test Exploratory Test  %76PDQDJHDOOWLFNHWV  7HVW5DLOPDQDJHPDQXDO WHVWFDVH  3RVWPDQPDQDJH$3, PRQLWRULQJFDVH
  11. &,&'碉ݳ㬙粚窕纷

  12. 獮ᒒ䛑አ ਥො:HE

  13. 獮ᒒ䛑አ ਥො2$

  14. 獮ᒒ䛑አ ਥො2$ 礚挨奾ຎࢧ薟 礚挨奾ຎ蝢Ꭳ

  15. 獮ᒒ䛑አ ਥො2$ • 1 5 • 6 & & •

    &
  16. 獮ᒒ䛑አ ਥො2$ 蜍獈ጱ礚挨෈ਁ 磪奾ຎ ૪礚໐ 篷奾ຎ 篷ፘ橕 篷奾ຎ 磪ፘ橕 ᤩ礚໐蝢Ꭳ

    ᤩ礚໐蝢Ꭳ
  17. Machine Learning in LINE  - ? - , -

    , , , ,
  18. suspicious? True ”Fake News”

  19. How ML helps LINE Fact Checker? • Coverage of Verified/Unverified

    Verified Messages: 300+ Similar Messages : 35k+ Total Messages : 40k/d Near- Duplication Classification
  20. Near-Duplication - Use Cases Verified Fake Message: • "footage on

    Captain's Instagram Stories showed them wearing wedding rings on their both hands, which prove s Captain America and Captain Marvel get married in Las Vegas" Query Checked Result Type Captain America and Captain Marvel get married in Las Vegas True Partial The wedding in Las Vegas is hosted by Captain America and Captain Marvel couple True Semantically Ironman and Black Widow get married in Los Angeles False Syntactically
  21. ML models • An Approximate Nearest Neighbor model: find similar

    articles using pretrained embeddings • Pre-trained BERT classifier: verify topics • fine-tuned with TODAY articles !"" #, %, & = ()*"+!, #%- ./ & +01"2ℎ4!. !"" = 5)65!"(!""8~: )
  22. ML system under hood Predicted Label Training Data Ingestion Scheduling

    Orchestration Async Execution Model Management Model Deployment Index-based Vector Search (ANN) Serving
  23. Query Article Near-duplicate Articles Human evaluated results

  24. Classification of Articles

  25. How users are influenced? • 46% users recognize the suspicious

    news • 33% users actively do the fact check • 25% would share the checked results • 10x times of facts checked • 4M+ impression of the checked fact  %  %  %
  26.  

  27. 7KDQN<RX