Assessing the Feasibility of Web-Request Prediction Models on Mobile Platforms

Presentation slides of our paper "Assessing the Feasibility of Web-Request Prediction Models on Mobile Platforms" at MOBILESoft 2021.
Presentation: https://youtu.be/91b3juLFbeU

Transcript

1. Assessing the Feasibility of
   Web-Request Prediction Models on
   Mobile Platforms
   Yixue Zhao1, Siwei Yin2, Adriana Sejfia3, Marcelo Schmitt Laser3,
   Haoyu Wang2, Nenad Medvidović3
   MOBILESoft 2021, Virtual Event
   1 2 3
   

   View Slide

2. PhD
   Thesis
   How to speed up
   mobile apps
   using prefetching?
   2
   Reducing User-Perceived
   Latency in Mobile Apps via
   Prefetching and Caching
   Yixue Zhao
   tinyurl.com/yixuedissertation
   

   View Slide

3. History-based
   Prefetching
   ▪ Input:
   user’s historical requests
   ▪ Method:
   prediction model
   ▪ Output:
   user’s future request(s)
   3
   

   View Slide

4. History-based
   Prefetching
   ▪ Input:
   user’s historical requests
   ▪ Method:
   prediction model
   ▪ Output:
   user’s future request(s)
   4
   Biggest
   Challenge!
   

   View Slide

5. Why
   no dataset?
   5
   Privacy!
   

   View Slide

6. Public
   Dataset: LiveLab
   6
   Ref: Shepard et al. LiveLab: measuring wireless
   networks and smartphone users in the field.
   SIGMETRICS Performance Evaluation Review. 2011
   ▪ Subject:
   25 iPhone users
   ▪ Size:
   an entire year
   ▪ Time:
   2011 (a decade ago)
   

   View Slide

7. ▪ Subject:
   25 iPhone users
   ▪ Size:
   an entire year
   ▪ Time:
   2011 (a decade ago)
   Public
   Dataset: LiveLab
   7
   

   View Slide

8. ▪ Subject:
   25 iPhone users
   ▪ Size:
   an entire year
   ▪ Time:
   2011 (a decade ago)
   Public
   Dataset
   8
   Small
   models!
   

   View Slide

9. Easier said than done…
   9
   

   View Slide

10. ICSE 2018, Gothenburg, Sweden
    10
    Co-author to-
    be: Haoyu
    PhD advisor:
    Neno
    Me after my
    ICSE talk
    

    View Slide

11. We got
    data!
    after tons of paper work, back and
    forth, ethical considerations etc…
    11
    

    View Slide

12. LiveLab
    ▪ An entire year
    LiveLab vs.
    Our dataset
    Our Dataset
    ▪ A random day (24hrs)
    12
    400X shorter time
    

    View Slide

13. LiveLab
    ▪ An entire year
    ▪ 25 iPhone-using undergraduates
    at Rice university
    LiveLab vs.
    Our dataset
    Our Dataset
    ▪ A random day (24hrs)
    ▪ 10K+ diverse mobile users
    at BUPT university
    13
    400X more users
    

    View Slide

14. LiveLab
    ▪ An entire year
    ▪ 25 iPhone-using undergraduates
    at Rice university
    ▪ Hire participants
    LiveLab vs.
    Our dataset
    Our Dataset
    ▪ A random day (24hrs)
    ▪ 10K+ diverse mobile users
    at BUPT university
    ▪ No contact with users
    14
    

    View Slide

15. 3 Research
    Questions
    Possibility?
    Do small prediction
    models work?
    à repetitive
    requests
    Existing solution?
    Can we reuse
    existing algorithms?
    à accuracy of DG,
    PPM, MP, Naïve
    (baseline)
    Even smaller?
    Can we reduce
    training size even
    more?
    à good & enough
    training data
    15
    

    View Slide

16. 3 Research
    Questions
    Possibility?
    Do small prediction
    models work?
    à repetitive
    requests
    Existing solution?
    Can we reuse
    existing algorithms?
    à accuracy of DG,
    PPM, MP, Naïve
    (baseline)
    Even smaller?
    Can we reduce
    training size even
    more?
    à good & enough
    training data
    16
    

    View Slide

17. 3 Research
    Questions
    Possibility?
    Do small prediction
    models work?
    à repetitive
    requests
    Existing solution?
    Can we reuse
    existing algorithms?
    à accuracy of DG,
    PPM, MP, Naïve
    (baseline)
    Even smaller?
    Can we reduce
    training size even
    more?
    à good & enough
    training data
    17
    

    View Slide

18. HiPHarness framework
    18
    

    View Slide

19. HiPHarness framework
    19
    Per user
    15 million requests
    à 7 million models
    Prediction accuracy
    

    View Slide

20. Results
    of 7+ million models for individual users
    20
    

    View Slide

21. Results (RQ2)
    21
    Existing solution?
    Can we reuse
    existing algorithms?
    à accuracy of DG,
    PPM, MP, Naïve
    (baseline)
    

    View Slide

22. Results (RQ2)
    22
    Existing solution?
    Can we reuse
    existing algorithms?
    à accuracy of DG,
    PPM, MP, Naïve
    (baseline)
    

    View Slide

23. Results (RQ2)
    23
    Existing solution?
    Can we reuse
    existing algorithms?
    à accuracy of DG,
    PPM, MP, Naïve
    (baseline)
    MP Static Precision: 0.16
    [Wang et al. WWW’12]
    

    View Slide

24. Results (RQ2)
    24
    Existing solution?
    Can we reuse
    existing algorithms?
    à accuracy of DG,
    PPM, MP, Naïve
    (baseline)
    MP Static Precision: 0.16
    [Wang et al. WWW’12]
    Small
    models
    are
    promising!
    

    View Slide

25. Results (RQ2)
    25
    Existing solution?
    Can we reuse
    existing algorithms?
    à accuracy of DG,
    PPM, MP, Naïve
    (baseline)
    Static Precision: 0.478
    [Zhao et al. ICSE’18]
    

    View Slide

26. Results (RQ2)
    26
    Existing solution?
    Can we reuse
    existing algorithms?
    à accuracy of DG,
    PPM, MP, Naïve
    (baseline)
    Static Precision: 0.478
    [Zhao et al. ICSE’18]
    Existing
    algorithms
    are
    promising!
    

    View Slide

27. Even smaller?
    Can we reduce
    training size even
    more?
    à good & enough
    training data
    Results (RQ3)
    27
    

    View Slide

28. Results (RQ3)
    28
    Even smaller?
    Can we reduce
    training size even
    more?
    à good & enough
    training data
    

    View Slide

29. Results (RQ3)
    29
    Even smaller?
    Can we reduce
    training size even
    more?
    à good & enough
    training data
    200 400 600 800 1000
    Static Precision trend w.r.t. #request
    

    View Slide

30. Results (RQ3)
    30
    Even smaller?
    Can we reduce
    training size even
    more?
    à good & enough
    training data
    200 400 600 800 1000
    Static Precision trend w.r.t. #request
    Cut-off point?
    

    View Slide

31. Results (RQ3)
    31
    ▪ Sliding-Window approach to explore cut-off points
    ▪ 11 window sizes (50, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1,000)
    ▪ ANOVA post-hoc test (pair-wise comparison)
    

    View Slide

32. Results (RQ3)
    32
    ▪ Sliding-Window approach to explore cut-off points
    ▪ 11 window sizes (50, 100, 200, 300, 400, 500, 600, 700, 800, 900, 10,000)
    ▪ ANOVA post-hoc test (pair-wise comparison)
    

    View Slide

33. Results (RQ3)
    33
    ▪ Sliding-Window approach to explore cut-off points
    ▪ 11 window sizes (50, 100, 200, 300, 400, 500, 600, 700, 800, 900, 10,000)
    ▪ ANOVA post-hoc test (pair-wise comparison)
    

    View Slide

34. Takeaways
    34
    ▪ Small models work!
    ▪ We can reuse existing solutions
    ▪ Less is more (reduce size AND improve accuracy)
    ▪ Challenged prior conclusion
    ▪ Re-open this area
    

    View Slide

35. Acknowledgement
    Co-authors: Siwei Yin, Adriana Sejfia, Marcelo
    Schmitt Laser, Haoyu Wang, Nenad Medvidović
    35
    

    View Slide

36. Thanks!
    36
    Any questions?
    yixuezhao@cs.umass.edu
    @yixue_zhao
    https://people.cs.umass.edu/~yixuezhao/
    

    View Slide