Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Analysis and Estimation of News Article Reading Time with Multimodal Machine Learning

Analysis and Estimation of News Article Reading Time with Multimodal Machine Learning

Analysis and Estimation of News Article Reading Time with Multimodal Machine Learning
Shotaro Ishihara, Yasufumi Nakama (IEEE BigData 2022, Industrial & Government Track)

Shotaro Ishihara

December 20, 2022
Tweet

More Decks by Shotaro Ishihara

Other Decks in Research

Transcript

  1. Shotaro Ishihara (Nikkei Inc.), and Yasufumi Nakama [email protected] IEEE BigData

    2022, Industry and Government Program Does Text Length matter? Analysis and Estimation of News Article Reading Time with Multimodal Machine Learning
  2. Research Overview 2 • text length • headline / body

    text • thumbnail image • others like genre • past reading history reading time
  3. Summary 1: Dataset 3 • text length • headline /

    body text • thumbnail image • others like genre • past reading history reading time ✅ Real-world content and access log of Nikkei
  4. Summary 2: Text length 4 • text length • headline

    / body text • thumbnail image • others like genre • past reading history reading time ✅ Doesn’t strongly correlate with reading time
  5. Summary 3: Multimodal 5 • text length • headline /

    body text • thumbnail image • others like genre • past reading history reading time ✅ Boosted performance
  6. Outline 6 • Introduction • Problem Formulation • Proposed Method

    • Experiments • Conclusion and Future Work
  7. Reading time estimation helps: 7 • Push notifications [1] •

    Recommendation [2, 4-6] • User decision support [3, 7] • Clickbait analysis [22-23]
  8. How can we estimate reading time? 8 • text length

    • headline / body text • thumbnail image • others like genre • past reading history reading time
  9. Research questions 9 1. How much does text length correlate

    with reading time? 2. How much do features other than text length improve the performance of reading time estimation?
  10. Reading time dataset 10 • A large dataset that includes

    reading time from Japanese financial news from the Nikkei. ◦ About 1,000 articles a day, 800,000 paid subscribers (and data infrastructure) ◦ Larger and more scalable than some existing data on recording eye movements [8] [9] and brain activity [10]
  11. Dataset details 11 100,000 sessions * 3 • train: 21-12-01

    • val: 21-12-08 • test: 21-12-15
  12. RQ1: text length (x) & reading time (y) 12 Correlation

    coefficient is 0.04 (and 0.31)
  13. 13 • Architecture corresponding to the specific data • E2E

    fine-tuning Proposed Method
  14. Experiments: Features & Models 14 1. The model was fixed

    to LightGBM [16] and the features were explored. 2. We fixed the features and observed differences. a. Ridge regression b. MLP c. Proposed method (w/wo E2E fine-tuning)
  15. Experiments: Features 15 Additional features improved the metric. •

  16. 1. mean reading time 2. text length 3. minimum reading

    time 4. embedding of body text (dimension 193) 5. embedding of thumbnail image (dimension 88) Important features by LightGBM 16
  17. Experiments: Models 17 • LightGBM worked better in the same

    feature. • Proposed method outperformed LightGBM by adding LSTM, and e2e fine-tuning.
  18. Multimodal training tips 18 • Different learning rate: 2e-5 for

    BERT, 1e-4 for Swin Transformer, and 1e-2 for the others • CosineAnnealingLR: For training stability
  19. Conclusion 19 • We highlighted the importance of reading time

    and evaluated the implementation. • Our analysis revealed reading time does not strongly correlate with text length. • Our experiments showed a multimodal machine learning approach led to a more accurate estimation than simply using text length.
  20. Future Work 20 • Offline evaluation => Online operation •

    Further feature & model exploration • Clickbait analysis