Data Loss Detector: Automatically Revealing Data Loss Bugs in Android Apps

In this video, I present Data Loss Detector (DLD), an automatic testing technique that can reveal data loss problems in Android apps. DLD integrates three capabilities to effectively reveal data loss problems: (i) a biased model-based exploration strategy that steers the exploration towards new app states that may be affected by data loss problems, (ii) data-loss-revealing actions that increase the likelihood to expose data loss problems, and (iii) two state-based oracles, a snapshot-based oracle and a property-based oracle, that have a high data loss detection accuracy, especially if used jointly.

Transcript

1. Data Loss Detector: Automatically Revealing Data Loss Bugs in Android Apps Oliviero Riganelli University of Milano Bicocca joint work with Simone Paolo Mottadelli, Claudio Rota, Daniela Micucci and Leonardo Mariani Distinguished Artifact
2. Recreating Android Activity https://disco.unimib.it oliviero.riganelli …………. https://disco.unimib.it oliviero.riganelli ………….
3. Recreating Android Activity Running (visible) Destroyed Created Running (visible) create activity restore activity state save activity state 1 2 3
4. https://disco.unimib.it oliviero.riganelli …………. Instance State https://disco.unimib.it oliviero.riganelli …………. Recreating an Activity
5. https://disco.unim ib.it oliviero.riganelli …………. Running (visible) Destroyed save activity state 1 https://disco.unimib.it oliviero.riganelli …………. Instance State Recreating an Activity
6. https://disco.unimib.it oliviero.riganelli …………. Created create activity 2 https://disco.unim ib.it oliviero.riganelli …………. Instance State Recreating an Activity
7. https://disco.unimib.it oliviero.riganelli …………. https://disco.unimib.it oliviero.riganelli …………. Created Running (visible) restore activity state 3 Instance State Recreating an Activity
8. https://disco.unimib.it oliviero.riganelli …………. Data Loss Instance State https://disco.unimib.it oliviero.riganelli ………….
9. https://disco.unim ib.it Running (visible) Destroyed save activity state 1 https://disco.unimib.it oliviero.riganelli …………. Data Loss Instance State
10. https://disco.unimib.it oliviero.riganelli …………. Created create activity 2 https://disco.unim ib.it Instance State Data Loss
11. https://disco.unimib.it oliviero.riganelli …………. https://disco.unimib.it Created Running (visible) restore activity state 3 Instance State Data Loss
12. https://disco.unimib.it oliviero.riganelli …………. https://disco.unimib.it Created Running (visible) restore activity state 3 Instance State Data Loss The metronome is frustrating. I use it when I practice but if I ever stop it, and turn off the screen, it resets to 40bpm and 4/4 time As soon as I opened a comic through this app I was really happy given its functionality and ease of use. But everytime I rotate the screen, the app exits saying " file could not be loaded" Randomly sets back to 100% and auto brightness after using camera
13. Pervasiveness of Data Loss Problems APP APP APP APP APP
14. Pervasiveness of Data Loss Problems 88% Apps with data loss 12% App without data loss
15. Pervasiveness of Data Loss Problems 88% Apps with data loss 12% App without data loss
16. Data Loss Detector (DLD) https://disco.unimib.it oliviero.riganelli …………. https://disco.unimib.it
17. Data Loss Detector (DLD) Touch buttons Type text Press HOME key …
18. Data Loss Detector (DLD) Touch buttons Type text Press HOME key …
19. TouchEvent(“3”) GrantPermissionsActivity <FIRST> GrantPermissionsActivity MainActivity MainActivity Launcher TouchEvent(“ALLOW”) TouchEvent(“ALLOW”) KeyEvent(HOME) GUI Model
20. TouchEvent(“3”) GrantPermissionsActivity <FIRST> GrantPermissionsActivity MainActivity MainActivity Launcher TouchEvent(“ALLOW”) TouchEvent(“ALLOW”) KeyEvent(HOME) GUI Model GUI: ACTIONS: KeyEvent(HOME) TouchEvent(“ALLOW”) … STATE
21. TouchEvent(“3”) GrantPermissionsActivity <FIRST> GrantPermissionsActivity MainActivity MainActivity Launcher TouchEvent(“ALLOW”) TouchEvent(“ALLOW”) KeyEvent(HOME) GUI Model Type: Touch View: Command: adb shell input … EVENT
22. https://disco.unimib.it oliviero.riganelli …………. Data Loss Detector (DLD) oliviero.riganelli Love is the answer Biased Model-Based Exploration Data Loss Oracles Data-Loss-Revealing Actions
23. TouchEvent(“OK”) Sn TouchEvent(“NO THANKS”) KeyEvent(MENU) Biased Model-Based Exploration
24. Sn KeyEvent(MENU) Biased Model-Based Exploration 0.9 Unexecuted event 0.1 Any event TouchEvent(“OK”) TouchEvent(“NO THANKS”)
25. Sn KeyEvent(MENU) Biased Model-Based Exploration P:0.333 P:0.333 P:0.333 TouchEvent(“OK”) TouchEvent(“NO THANKS”) 0.9 Unexecuted event 0.1 Any event
26. Sn KeyEvent(MENU) Biased Model-Based Exploration P:0.333 P:0.333 P:0.333 TouchEvent(“OK”) TouchEvent(“NO THANKS”) 0.9 Unexecuted event 0.1 Any event
27. Sn KeyEvent(MENU) Biased Model-Based Exploration P:0.033 P:0.483 P:0.483 TouchEvent(“OK”) TouchEvent(“NO THANKS”) 0.9 Unexecuted event 0.1 Any event
28. Sn KeyEvent(MENU) Biased Model-Based Exploration TouchEvent(“OK”) TouchEvent(“NO THANKS”) P:0.033 P:0.483 P:0.483 0.9 Unexecuted event 0.1 Any event
29. Sn KeyEvent(HOME) Biased Model-Based Exploration P:0.033 P:0.933 P:0.033 TouchEvent(“OK”) TouchEvent(“NO THANKS”) 0.9 Unexecuted event 0.1 Any event
30. Data-Loss-Revealing Actions NEW STATE Systematic Probabilistic ALREADY VISITED STATE
31. Systematic data-loss-revealing action https:// oliviero.riganelli ………….
32. Systematic data-loss-revealing action https:// oliviero.riganelli …………. https://disco.unimib.it oliviero.riganelli …………. https:// oliviero.riganelli ………….
33. https:// oliviero.riganelli …………. https://disco.unimib.it oliviero.riganelli …………. https:// oliviero.riganelli …………. https:// oliviero.riganelli … … … … . https://disco.unim ib.it Systematic data-loss-revealing action
34. Probabilistic data-loss-revealing action https:// oliviero.rig …………. https:// oliviero.rig … … … … . https:// oliviero.riganelli … … … … . https:// oliviero.rig … … … … . https:// oliviero.riganelli … … … … .
35. Data Loss Oracles [ … { 'content_description': None, 'resource_id': None, 'text': 'Editing "test123"', 'visible': True, 'checkable': False, 'children': [], 'size': '720*81', 'checked': False, 'temp_id': 4, 'selected': False, 'child_count': 0, ... } ... ] Snapshot-based oracle Property-based oracle
36. Data Loss Oracles Snapshot Property … ‘content_description': … … ‘content_description': ... Before double rotation After double rotation
37. Data Loss Oracles Snapshot Property … ‘content_description': 'Open navigation drawer', … … ‘content_description': ‘Close navigation drawer', ... Before double rotation After double rotation
38. Evaluation Evaluation Research Questions APPs Configuration Google Nexus 5 Android 6.0 RQ0 - What is the ε that provides the best exploration? RQ1 - How effective is DLD with data loss problems? RQ2 - Is DLD more effective than state- of-the-art techniques? RQ3 - What is the tradeoff between the snapshot- and property- based oracles? RQ4 - Are data loss faults relevant to developers? 110 real data loss 54 app releases Riganelli et al. "A Benchmark of Data Loss Bugs for Android Apps”, MSR 2019.
39. RQ1 - How effective is DLD with data loss problems? 110 Benchmark Data Loss
40. RQ1 - How effective is DLD with data loss problems? 27 83 267 Benchmark Data Loss Detected by DLD
41. RQ1 - How effective is DLD with data loss problems? 27 83 267 Benchmark Data Loss Detected by DLD 90% of faults confirmed by developers, see RQ4 in the paper
42. RQ2 - Is DLD more effective than state-of-the-art techniques? 0 50 100 150 200 Faulty Activity Benchmark Faults New Faults Spuriuos Violations Alaric DLD
43. RQ2 - Is DLD more effective than state-of-the-art techniques? 0 50 100 150 200 Faulty Activity Benchmark Faults New Faults Spuriuos Violations Alaric DLD COMPARISON TO QUANTUM IN THE PAPER
44. RQ3 - What is the tradeoff between the snapshot- and property- based oracles? 73% 18% 9% snapshot-based oracle property-based oracle both incomplete rotation 74% 21% 5% Data loss detected Spurious oracle violations
45. RQ3 - What is the tradeoff between the snapshot- and property- based oracles? 73% 18% 9% snapshot-based oracle property-based oracle both incomplete rotation 74% 21% 5% Data loss detected Spurious oracle violations 0.1%
46. https://github.com/datalossdetector/DLD Get Your Copy Now!!!