Automated layout failure detection for responsive web pages without an explicit oracle

Interested in learning more about this topic? Visit this web site to read the paper: https://www.gregorykapfhammer.com/research/papers/Walsh2017/

Transcript

1. AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH1, PHIL MCMINN1, GREGORY KAPFHAMMER2 1UNIVERSITY OF SHEFFIELD, 2ALLEGHENY COLLEGE 1
2. ISSTA 2017 A LONG TIME AGO… 2 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
3. ISSTA 2017 THE WEB TODAY 3 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE https://blog.pivotal.io/labs/labs/an-introduction-to-qa-at-xtreme-labs
4. ISSTA 2017 WHY SUPPORT ALL DEVICES? 4 AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK
5. ISSTA 2017 HOW CAN DEVELOPERS MAKE WEBSITES SUPPORT ALL DEVICES? USE RESPONSIVE WEB DESIGN. 5 AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK
6. ISSTA 2017 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE RESPONSIVE WEB DESIGN 6 http://getbootstrap.com
7. ISSTA 2017 RESPONSIVE LAYOUT FAILURES 7 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
8. ISSTA 2017 RESPONSIVE LAYOUT FAILURES 8 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
9. ISSTA 2017 RESPONSIVE LAYOUT FAILURES 9 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
10. ISSTA 2017 RESPONSIVE LAYOUT FAILURES 10 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
11. ISSTA 2017 RESPONSIVE LAYOUT FAILURES 11 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
12. ISSTA 2017 RESPONSIVE LAYOUT FAILURES 12 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
13. ISSTA 2017 HOW DO DEVELOPERS TEST RESPONSIVE WEBSITES? THEY “SPOT-CHECK”. WHAT IS THE ORACLE FOR RESPONSIVE WEBSITES? THE HUMAN TESTER. 13 AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK
14. ISSTA 2017 A SPOTCHECKING EXAMPLE 14 AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK 320 pixels 768 pixels 1024 pixels 1200 pixels 980 pixels
15. ISSTA 2017 SUMMARY OF APPROACH 15 AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK RESPONSIVE LAYOUT GRAPH (RLG) COMMON RESPONSIVE LAYOUT FAILURE DETECTION
16. ISSTA 2017 THE RESPONSIVE LAYOUT GRAPH 16 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE DIV[1] DIV[2] DIV[3] DIV[1] DIV[2] DIV[3] IMG 320PX-1400PX BODY 768PX-1400PX IMG 320PX-1400PX DIV[3] 320PX-1400PX DIV[2] 320PX-1400PX DIV[1] 320px-767px, above 768px-1400px, left-of 320px-767px , centre-justified 768px-1400px, left-justified 320 pixels 768 pixels
17. ISSTA 2017 RLF 1 - ELEMENT COLLISION 17 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
20. ISSTA 2017 RLF 1 - ELEMENT COLLISION 1. Check whether any pair of elements are overlapping at any point. 2. If two elements are overlapping, check the layout at the wider viewport width. 3. If elements no longer overlapping, report an element collision failure. 20 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
21. ISSTA 2017 RLF 1 - ELEMENT COLLISION 21 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE 320PX-1400PX FORM 320PX-1400PX DIV 320px-767px, above 768px-1000px, overlapping 1001px-1400px, left-of
22. ISSTA 2017 RLF 1 - ELEMENT COLLISION 1. Check whether any pair of elements are overlapping at any point. 2. If two elements are overlapping, check the layout at the wider viewport width. 3. If elements no longer overlapping, report an element collision failure. 22 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
23. ISSTA 2017 RLF 1 - ELEMENT COLLISION 23 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE 320PX-1400PX FORM 320PX-1400PX DIV 320px-767px, above 768px-1000px, overlapping 1001px-1400px, left-of
24. ISSTA 2017 RLF 1 - ELEMENT COLLISION 1. Check whether any pair of elements are overlapping at any point. 2. If two elements are overlapping, check the layout at the wider viewport width. 3. If elements no longer overlapping, report an element collision failure. 24 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
25. ISSTA 2017 RLF 1 - ELEMENT COLLISION 25 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE Element collision between 768px and 1000px. 320PX-1400PX FORM 320PX-1400PX DIV 320px-767px, above 768px-1000px, overlapping 1001px-1400px, left-of
26. ISSTA 2017 RLF 2 - ELEMENT PROTRUSION 26 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
29. ISSTA 2017 RLF 2 - ELEMENT PROTRUSION 1. Check whether any pair of elements are overlapping at any point. 2. If two elements are overlapping, check the layout at the wider viewport width. 3. If one element now contains the other, report an element protrusion failure. 29 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
30. ISSTA 2017 RLF 2 - ELEMENT PROTRUSION 30 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE 320PX-1400PX HEADER 320PX-1400PX FORM 320px-640px, overlapping 641px-1400px, contains
31. ISSTA 2017 RLF 2 - ELEMENT PROTRUSION 1. Check whether any pair of elements are overlapping at any point. 2. If two elements are overlapping, check the layout at the wider viewport width. 3. If one element now contains the other, report an element protrusion failure. 31 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
32. ISSTA 2017 RLF 2 - ELEMENT PROTRUSION 32 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE Element protrusion between 320px and 640px. 320px-640px, overlapping 641px-1400px, contains 320PX-1400PX HEADER 320PX-1400PX FORM
33. ISSTA 2017 RLF 3 - VIEWPORT PROTRUSION 33 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
36. ISSTA 2017 RLF 3 - VIEWPORT PROTRUSION 1. Look through each element. 2. Check that for every viewport width the element is visible, it is contained within some other element. 3. If, at any point, the element is not contained at all, report a viewport protrusion failure. 36 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
37. ISSTA 2017 RLF 3 - VIEWPORT PROTRUSION 37 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE 981PX-1400PX DIV 1050px-1400px, contains 320PX-1400PX BODY
38. ISSTA 2017 RLF 3 - VIEWPORT PROTRUSION 1. Look through each element. 2. Check that for every viewport width the element is visible, it is contained within some other element. 3. If, at any point, the element is not contained at all, report a viewport protrusion failure. 38 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
39. ISSTA 2017 RLF 3 - VIEWPORT PROTRUSION 39 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE 1050px-1400px, contains 981PX-1400PX DIV 320PX-1400PX BODY
40. ISSTA 2017 RLF 3 - VIEWPORT PROTRUSION 1. Look through each element. 2. Check that for every viewport width the element is visible, it is contained within some other element. 3. If, at any point, the element is not contained at all, report a viewport protrusion failure. 40 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
41. ISSTA 2017 RLF 3 - VIEWPORT PROTRUSION 41 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE 1050px-1400px, contains Viewport protrusion between 981px and 1049px. 981PX-1400PX DIV 320PX-1400PX BODY
42. ISSTA 2017 RLF 4 - SMALL-RANGE LAYOUTS 42 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
45. ISSTA 2017 1. Are any layouts only applied for a small number of viewport widths (e.g., 5)? 2. If so, check if similar layouts exist both before and after the small-range layout. 3. If so, report the small-range layout failure. 45 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE RLF 4 - SMALL-RANGE LAYOUTS
46. ISSTA 2017 46 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE 320PX-1400PX HEADER 320PX-1400PX BANNER 320px-979px, above 980px-980px, overlapping 981px-1400px, above RLF 4 - SMALL-RANGE LAYOUTS
47. ISSTA 2017 1. Are any layouts only applied for a small number of viewport widths (e.g., 5)? 2. If so, check if similar layouts exist both before and after the small-range layout. 3. If so, report the small-range layout failure. 47 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE RLF 4 - SMALL-RANGE LAYOUTS
48. ISSTA 2017 48 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE 320PX-1400PX HEADER 320PX-1400PX BANNER 320px-979px, above 980px-980px, overlapping 981px-1400px, above RLF 4 - SMALL-RANGE LAYOUTS
49. ISSTA 2017 1. Are any layouts only applied for a small number of viewport widths (e.g., 5)? 2. If so, check if similar layouts exist both before and after the small-range layout. 3. If so, report the small-range layout failure. 49 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE RLF 4 - SMALL-RANGE LAYOUTS
50. ISSTA 2017 50 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE Small-range layout at 980 pixels. 320PX-1400PX HEADER 320PX-1400PX BANNER 320px-979px, above 980px-980px, overlapping 981px-1400px, above RLF 4 - SMALL-RANGE LAYOUTS
51. ISSTA 2017 RLF 5 - ELEMENT WRAPPING 51 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
54. ISSTA 2017 RLF 5 - ELEMENT WRAPPING 1. Analyse layout information to determine which elements are laid out in rows. 2. Are there any rows which one element is not in? 3. If this element is in the row at a wider viewport, report an element wrapping failure. 54 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
55. ISSTA 2017 RLF 5 - ELEMENT WRAPPING 55 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE LI[3] LI[1] LI[2] 768px-1400px, left-of 768px-800px, above 801px-1400px, left-of 768px-800px, above 801px-1400px, left-of
56. ISSTA 2017 RLF 5 - ELEMENT WRAPPING 1. Analyse layout information to determine which elements are laid out in rows. 2. Are there any rows which one element is not in? 3. If this element is in the row at a wider viewport, report an element wrapping failure. 56 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
57. ISSTA 2017 RLF 5 - ELEMENT WRAPPING 57 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE LI[3] LI[1] LI[2] 768px-1400px, left- 768px-800px, above 801px-1400px, left-of 768px-800px, above 801px-1400px, left-of
58. ISSTA 2017 RLF 5 - ELEMENT WRAPPING 1. Analyse layout information to determine which elements are laid out in rows. 2. Are there any rows which one element is not in? 3. If this element is in the row at a wider viewport, report an element wrapping failure. 58 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
59. ISSTA 2017 RLF 5 - ELEMENT WRAPPING 59 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE Element wrapping between 768px and 800px. LI[3] LI[1] LI[2] 768px-1400px, left- 768px-800px, above 801px-1400px, left-of 768px-800px, above 801px-1400px, left-of
60. ISSTA 2017 EMPIRICAL EVALUATION 1. Can the approach detect responsive layout failures in real-world responsive web pages? 2. How effective are manual “spotchecking” approaches in comparison to our approach? 3. How long do our techniques take to run? ReDeCheck - Responsive Design Checker 60 AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK
61. ISSTA 2017 61 AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK
62. ISSTA 2017 RQ1 CAN THE APPROACH DETECT RESPONSIVE LAYOUT FAILURES IN REAL-WORLD RESPONSIVE WEB PAGES? 62 AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK
63. ISSTA 2017 63 AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK
64. ISSTA 2017 64 AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK
65. ISSTA 2017 RESEARCH QUESTION ONE ▸ Responsive layout failures found in 16 of the 26 web pages. ▸ 196 true positive results conflated to 33 distinct RLFs. ▸ False positives mainly caused by “coincidental” attributes being assigned to constraints. ▸ Many NOIs showed significant structural issues, which could easily manifest as TPs in the future. ▸ On average, developers need to check 4 distinct viewport ranges to observe each TP failure. 65 Failure Type TP FP NOI RLFs Element Collision 8 0 24 8 Element Protrusion 3 0 36 3 Viewport Protrusion 24 0 23 11 Small-Range 152 43 0 1 Wrapping 10 5 0 10 Total 196 48 83 33 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
66. ISSTA 2017 RQ2 HOW EFFECTIVE ARE MANUAL “SPOTCHECKING” APPROACHES IN COMPARISON TO OUR APPROACH? 66 AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK
67. ISSTA 2017 RESEARCH QUESTION TWO 67 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE 0 6 12 18 24 30 36 KERSLEY RESPONSINATOR RD CHECKER V-RESIZER FIREFOX RWD REDECHECK 33 27 26 23 22 22
68. ISSTA 2017 RQ3 HOW LONG DO OUR TECHNIQUES TAKE TO RUN? 68 AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK
69. ISSTA 2017 RESEARCH QUESTION THREE 69 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE 0 60 120 180 240 300 3-MINUTE-JOURNAL ACCOUNTKILLER AIRBNB BUGMENOT CLOUDCONVERT CONSUMER-REPORTS COVERED-CALENDAR DAYS-OLD DICTATION DUOLINGO HONEY HOTELWIFITEST MAILINATOR MIDWAYMEETUP NINITE PDF-ESCAPE PEPFEED POCKET RAINYMOOD RUNPEE STUMBLEUPON TOPDOCUMENTARY USERSEARCH WSIRN WILLMYPHONEWORK ZERODOLLARMOVIES
70. ISSTA 2017 THE REDECHECK TOOL 70 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE https://github.com/redecheck/redecheck http://redecheck.org https://github.com/redecheck/example-webpages
71. ISSTA 2017 71 THOMAS WALSH TWALSH1@SHEFFIELD.AC.UK AUTOMATED LAYOUT FAILURE DETECTION FOR RESPONSIVE WEB PAGES WITHOUT AN EXPLICIT ORACLE
72. THANK YOU TWALSH1@SHEFFIELD.AC.UK THOMASWALSH.CO.UK REDECHECK.ORG 72