LINE FACT Checker QA at Test Corner

LINE 訊息查證 by Hack Chen and Johnson Wu @ Test Corner #26 https://testcorner.kktix.cc/events/testcorner26

Transcript

1. /,1(懱௳礚挨  %\/,1(+DQN&KHQ -RKQVRQ:X
2. $JHQGD 碻绚胙ว /,1(碍֖吚揣懯向 $JLOH介手窕纷 介手獉਻ &,&'碉ݳ㬙粚窕纷 獮ᒒ䛑አ Machine Learning       
3. - 9 
    
    
   
   1 9 碻绚胙ว
4. 6DZ5HFHLYHGVXVSLFLRXV ,QIRUPDWLRQRQOLQH $FWLYHO\'RWKH)DFW &KHFNLQJ $IWHU)DFW&KHFNLQJZLOO IXUWKHUVKDUHWKHFRUUHFWLQIR  %  % % 碻绚胙ว
5. GOV /,1(碍֖吚揣懯向 肥ፑफ़֎
6. $JLOH介手窕纷
7. 介手獉਻ 介殻ୌ缏ᓕቘ  ๐率玟璸  玟璸蟂֖ *8,)XQFWLRQ  玟璸蟂֖ )XQFWLRQ  狕ྋ㮆礯介殻  介手褩ྦྷ  ஠ᥝ௔ /HYHO  %76橕肥  咳㬙粚蒈  䋿ֺ瑽෈
8. 介手獉਻ $3,緳介  &OLHQW6LGH6FHQDULR  *RRJOH,QVSHFWRU  玟璸蟂֖  $3,6SHFGRFXPHQW  ਞ蕕1HZPDQ物  QSP LQVWDOOJQHZPDQ  ๜瑿ᒒ珸㵕脻๜物  QHZPDQ UXQKWWSVZZZSRVWPDQFRPFROOHFWLRQVFE HHIHHGHYBHQYLURQPHQWMVRQ
9. 介手獉਻ ,VVXH7UDFNLQJ  7LFNHW7\SH  6WDWXV  )L[YHUVLRQ  (QYL26%URZVHU  6WHS$VLV7REH  $VVLJQWRPH  6SULQW  7LFNHWTXDQWLW\  0\FUHDWH  6WRU\ %XJ  'HYHORSLQJ  4$  &ORVH
10. 介手獉਻ 介手奾䯤 Ticket Management API Monitoring Test Manual Test Functional Test Regression Test Exploratory Test  %76PDQDJHDOOWLFNHWV  7HVW5DLOPDQDJHPDQXDO WHVWFDVH  3RVWPDQPDQDJH$3, PRQLWRULQJFDVH
11. &,&'碉ݳ㬙粚窕纷
12. 獮ᒒ䛑አ ਥො:HE
13. 獮ᒒ䛑አ ਥො2$
14. 獮ᒒ䛑አ ਥො2$ 礚挨奾ຎࢧ薟 礚挨奾ຎ蝢Ꭳ
15. 獮ᒒ䛑አ ਥො2$ • 1 5 • 6 & & • &
16. 獮ᒒ䛑አ ਥො2$ 蜍獈ጱ礚挨෈ਁ 磪奾ຎ ૪礚໐ 篷奾ຎ 篷ፘ橕 篷奾ຎ 磪ፘ橕 ᤩ礚໐蝢Ꭳ ᤩ礚໐蝢Ꭳ
17. Machine Learning in LINE 
     - ? - , - , , , ,
18. suspicious? True ”Fake News”
19. How ML helps LINE Fact Checker? • Coverage of Verified/Unverified Verified Messages: 300+ Similar Messages : 35k+ Total Messages : 40k/d Near- Duplication Classification
20. Near-Duplication - Use Cases Verified Fake Message: • "footage on Captain's Instagram Stories showed them wearing wedding rings on their both hands, which prove s Captain America and Captain Marvel get married in Las Vegas" Query Checked Result Type Captain America and Captain Marvel get married in Las Vegas True Partial The wedding in Las Vegas is hosted by Captain America and Captain Marvel couple True Semantically Ironman and Black Widow get married in Los Angeles False Syntactically
21. ML models • An Approximate Nearest Neighbor model: find similar articles using pretrained embeddings • Pre-trained BERT classifier: verify topics • fine-tuned with TODAY articles !"" #, %, & = ()*"+!, #%- ./ & +01"2ℎ4!. !"" = 5)65!"(!""8~: )
22. ML system under hood Predicted Label Training Data Ingestion Scheduling Orchestration Async Execution Model Management Model Deployment Index-based Vector Search (ANN) Serving
23. Query Article Near-duplicate Articles Human evaluated results
24. Classification of Articles
25. How users are influenced? • 46% users recognize the suspicious news • 33% users actively do the fact check • 25% would share the checked results • 10x times of facts checked • 4M+ impression of the checked fact  %  %  %
26. 
     
    
27. 7KDQN<RX