Analysis and Estimation of News Article Reading Time with Multimodal Machine Learning

Analysis and Estimation of News Article Reading Time with Multimodal Machine Learning
Shotaro Ishihara, Yasufumi Nakama (IEEE BigData 2022, Industrial & Government Track)

Transcript

1. Shotaro Ishihara (Nikkei Inc.), and Yasufumi Nakama
   shotaro.ishihara@nex.nikkei.com
   IEEE BigData 2022, Industry and Government Program
   Does Text Length matter?
   Analysis and Estimation of News
   Article Reading Time with Multimodal
   Machine Learning
   

   View Slide

2. Research Overview
   2
   ● text length
   ● headline / body text
   ● thumbnail image
   ● others like genre
   ● past reading history
   reading time
   

   View Slide

3. Summary 1: Dataset
   3
   ● text length
   ● headline / body text
   ● thumbnail image
   ● others like genre
   ● past reading history
   reading time
   ✅ Real-world content and access log of Nikkei
   

   View Slide

4. Summary 2: Text length
   4
   ● text length
   ● headline / body text
   ● thumbnail image
   ● others like genre
   ● past reading history
   reading time
   ✅ Doesn’t strongly correlate with reading time
   

   View Slide

5. Summary 3: Multimodal
   5
   ● text length
   ● headline / body text
   ● thumbnail image
   ● others like genre
   ● past reading history
   reading time
   ✅ Boosted performance
   

   View Slide

6. Outline
   6
   ● Introduction
   ● Problem Formulation
   ● Proposed Method
   ● Experiments
   ● Conclusion and Future Work
   

   View Slide

7. Reading time estimation helps:
   7
   ● Push notifications [1]
   ● Recommendation [2, 4-6]
   ● User decision support [3, 7]
   ● Clickbait analysis [22-23]
   

   View Slide

8. How can we estimate reading time?
   8
   ● text length
   ● headline / body text
   ● thumbnail image
   ● others like genre
   ● past reading history
   reading time
   

   View Slide

9. Research questions
   9
   1. How much does text length correlate with
   reading time?
   2. How much do features other than text length
   improve the performance of reading time
   estimation?
   

   View Slide

10. Reading time dataset
    10
    ● A large dataset that includes reading time from
    Japanese financial news from the Nikkei.
    ○ About 1,000 articles a day, 800,000 paid
    subscribers (and data infrastructure)
    ○ Larger and more scalable than some existing
    data on recording eye movements [8] [9] and
    brain activity [10]
    

    View Slide

11. Dataset details
    11
    100,000 sessions * 3
    ● train: 21-12-01
    ● val: 21-12-08
    ● test: 21-12-15
    

    View Slide

12. RQ1: text length (x) & reading time (y)
    12
    Correlation coefficient is 0.04 (and 0.31)
    

    View Slide

13. 13
    ● Architecture
    corresponding to
    the specific data
    ● E2E fine-tuning
    Proposed Method
    

    View Slide

14. Experiments: Features & Models
    14
    1. The model was fixed to LightGBM [16] and the
    features were explored.
    2. We fixed the features and observed differences.
    a. Ridge regression
    b. MLP
    c. Proposed method (w/wo E2E fine-tuning)
    

    View Slide

15. Experiments: Features
    15
    Additional features improved the metric.
    ●
    

    View Slide

16. 1. mean reading time
    2. text length
    3. minimum reading time
    4. embedding of body text (dimension 193)
    5. embedding of thumbnail image (dimension 88)
    Important features by LightGBM
    16
    

    View Slide

17. Experiments: Models
    17
    ● LightGBM worked better in the same feature.
    ● Proposed method outperformed LightGBM by
    adding LSTM, and e2e fine-tuning.
    

    View Slide

18. Multimodal training tips
    18
    ● Different learning rate: 2e-5 for BERT, 1e-4 for
    Swin Transformer, and 1e-2 for the others
    ● CosineAnnealingLR: For training stability
    

    View Slide

19. Conclusion
    19
    ● We highlighted the importance of reading time
    and evaluated the implementation.
    ● Our analysis revealed reading time does not
    strongly correlate with text length.
    ● Our experiments showed a multimodal machine
    learning approach led to a more accurate
    estimation than simply using text length.
    

    View Slide

20. Future Work
    20
    ● Offline evaluation => Online operation
    ● Further feature & model exploration
    ● Clickbait analysis
    

    View Slide