Using Machine Learning to Test Web Applications

Elena Treshcheva
Researcher, Exactpro

San Diego State University, 12 June 2020

Video : https://youtu.be/hnBlsGs9LzE

In this presentation, Elena Treshcheva, Researcher, Business Development Manager at Exactpro, describes the fundamental concepts of machine learning and compares the capabilities of DOM-based testing to those of visual AI. The presentation was recorded at the open webinar held for San Diego State University students on 12 June 2020.

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

Transcript

1. 1 Using Machine Learning to Test Web Applications Elena Treshcheva Researcher, Exactpro San Diego State University, 12 June 2020
2. 2 Build Software to Test Software exactpro.com Overview • References / Useful links • A web application from the point of view of a human being • Software testing, test automation and machine learning • A human being from the point of view of a web application • Testing with Selenium • The Oracle problem in software testing • ML applied to testing of DOM • ML applied to visual testing
3. 3 Build Software to Test Software exactpro.com A specialist firm focused on functional and non-functional testing of exchanges, clearing houses, depositories, trade repositories and other financial market infrastructures. About Exactpro Headquartered in the UK with operations in the US and Eastern Europe. The clients are regulated by the FCA, Bank of England and their counterparts from other countries. Exactpro provides software testing services for mission critical technology that underpins global financial markets. The firm is experienced with trading and clearing and settlement platforms, market data systems, collateral management and risk management systems, central data warehouses, regulatory reporting, etc. Incorporated in 2009 with 10 people, Exactpro is now employing over 590 specialists. Part of the London Stock Exchange Group (LSEG) from May 2015 until January 2018, when the Exactpro management proceeded through the buyout of the company from LSEG.
4. 4 Build Software to Test Software exactpro.com EXACTPRO Non-Deterministic Probabilistic Non-Testable Oracle Problem Mission-critical Financial Markets Infrastructures
5. 5 Build Software to Test Software exactpro.com Typical requirements for an Exchange system Daily capacity - 100+ mln transactions Peak rates - 40k+ transactions per second Average round-trip latency - <100 microseconds Availability - 100% 3000 trx 2.5 cm <1 mm
6. 6 Build Software to Test Software exactpro.com EXACTPRO Useful Links https://frontendmasters.com/books/front-end-handbook/2019/
7. 7 Build Software to Test Software exactpro.com EXACTPRO https://youtu.be/nvSqTlzivNM https://www.slideshare.net/roesslerj/how-to-apply-ai-to-testing Useful Links
8. 8 Build Software to Test Software exactpro.com 25 February 2020 https://youtu.be/KGVQVBdjvCU Useful Links
9. 9 Build Software to Test Software exactpro.com EXACTPRO https://www.gasq.org/en/certification/selenium-tester.html Useful Links
10. 10 Build Software to Test Software exactpro.com EXACTPRO https://www.youtube.com/watch?v=S9iytpte6OA https://www.testim.io/ Useful Links
11. 11 Build Software to Test Software exactpro.com EXACTPRO https://testautomationu.applitools.com/ai-for-element-selection-erasing-the-pain-of-fragile-test-scripts Useful Links
12. 12 Build Software to Test Software exactpro.com EXACTPRO The structure of a Web Application Browser send requests to the server, receive responses and visualize them as a page Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. HTTP(S) Request HTTP(S) Response 1 2 CSS HTML Javascript Internet Web Servers Static Content Browser DOM model
13. 13 Build Software to Test Software exactpro.com EXACTPRO Browser send requests to the server, receive responses and visualize them as a page Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. HTTP(S) Request HTTP(S) Response 1 2 CSS HTML Javascript Internet Application Servers Data Web Servers Static Content Browser DOM model The structure of a Web Application
14. 14 Build Software to Test Software exactpro.com Browser send requests to the server, receive responses and visualize them as a page Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 2 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Internet The structure of a Web Application
15. 15 Build Software to Test Software exactpro.com EXACTPRO The structure of a Web Application Browser send requests to the server, receive responses and visualize them as a page Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 AJAX allows to create asynchronous web interactions without interfering with the display and behaviour of the existing page 4 2 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Internet
16. 16 Build Software to Test Software exactpro.com EXACTPRO Browser send requests to the server, receive responses and visualize them as a page WebSocket enables interaction between a web browser and a web server with lower overhead than half-duplex alternatives such as HTTP polling Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 AJAX allows to create asynchronous web interactions without interfering with the display and behaviour of the existing page 4 2 5 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet The structure of a Web Application
17. 17 Build Software to Test Software exactpro.com EXACTPRO Browser send requests to the server, receive responses and visualize them as a page WebSocket enables interaction between a web browser and a web server with lower overhead than half-duplex alternatives such as HTTP polling Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 AJAX allows to create asynchronous web interactions without interfering with the display and behaviour of the existing page 4 2 5 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet The structure of a Web Application
18. 18 Build Software to Test Software exactpro.com EXACTPRO Browser send requests to the server, receive responses and visualize them as a page WebSocket enables interaction between a web browser and a web server with lower overhead than half-duplex alternatives such as HTTP polling Responsive front-end design allows a site to adapt to user’s device. Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 AJAX allows to create asynchronous web interactions without interfering with the display and behaviour of the existing page 4 2 5 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet The structure of a Web Application
19. 19 Build Software to Test Software exactpro.com EXACTPRO Browser send requests to the server, receive responses and visualize them as a page WebSocket enables interaction between a web browser and a web server with lower overhead than half-duplex alternatives such as HTTP polling Responsive front-end design allows a site to adapt to user’s device. Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 AJAX allows to create asynchronous web interactions without interfering with the display and behaviour of the existing page 4 2 5 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet Web applications are always multi-user applications and many of them work under heavy load and are subject to concurrency access 6 The structure of a Web Application
20. 20 Build Software to Test Software exactpro.com EXACTPRO Quality - The degree to which a component or system satisfies the stated and implied needs of its various stakeholders. Software Testing - The process consisting of all lifecycle activities, both static and dynamic, concerned with planning, preparation and evaluation of a component or system and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects. A person can make an error, which can lead to the introduction of a defect (fault or bug) in the software code. If a defect in the code is executed, this may cause a failure Software Testing
21. 21 Build Software to Test Software exactpro.com EXACTPRO Testing Levels CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet A C D E F B
22. 22 Build Software to Test Software exactpro.com EXACTPRO Automation Automation is the technology by which a process or procedure is performed with minimal human assistance. “The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency” Bill Gates
23. 23 Build Software to Test Software exactpro.com EXACTPRO A Model of the Human from a Web App’s Perspective
24. 24 Build Software to Test Software exactpro.com EXACTPRO A Model of the Human from a Web App’s Perspective
25. 25 Build Software to Test Software exactpro.com EXACTPRO A Model of the Human from a Web App’s Perspective
26. 26 Build Software to Test Software exactpro.com EXACTPRO Document <HTML> <head> <body> <h1> Root element Element “The Header” <h1> “the link” “href” “Sub Header” <h2> “The title” <title> Element Element Element Element Text Attribute Text Text Element Text The DOM defines: All HTML elements as objects • The properties of all HTML elements • The methods that can be used to access all HTML elements • The events that affect all HTML elements "The W3C Document Object Model (DOM) is a platform and language-neutral interface that allows programs and scripts to dynamically access and update the content, structure, and style of a document." DOM (Document Object model)
27. 27 Build Software to Test Software exactpro.com EXACTPRO DOM модель
28. 28 Build Software to Test Software exactpro.com EXACTPRO DOM (Document Object model)
29. 29 Build Software to Test Software exactpro.com EXACTPRO Document <HTML> <head> <body> <h1> Root element Element “The Header” <h1> “the link” “href” “Sub Header” <h2> “The title” <title> Element Element Element Element Text Attribute Text Text Element Text The DOM defines: All HTML elements as objects • The properties of all HTML elements • The methods that can be used to access all HTML elements • The events that affect all HTML elements "The W3C Document Object Model (DOM) is a platform and language-neutral interface that allows programs and scripts to dynamically access and update the content, structure, and style of a document." DOM (Document Object model)
30. 30 Build Software to Test Software exactpro.com EXACTPRO Testing Levels CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet A C D E F B
31. 31 Build Software to Test Software exactpro.com EXACTPRO DOM (Document Object model)
32. 32 Build Software to Test Software exactpro.com DOM (Document Object model) street car car crosswalk person traffic light road building tree sidewalk
33. 33 Build Software to Test Software exactpro.com EXACTPRO street sidewalk tree crosswalk car building road pedestrian city car car branch DOM (Document Object model) street car car crosswalk person traffic light road building tree sidewalk
34. 34 Build Software to Test Software exactpro.com EXACTPRO The Generic Test Automation Architecture
35. 35 Build Software to Test Software exactpro.com EXACTPRO Advantages: • You can find elements that don't have any unique attributes (id, class, name, etc.) • You can use XPath in generic locators, using the different "By" strategies (by id, by class, etc.) as needed Disadvantages: • Absolute XPath code is "brittle" and can break with the smallest change to the HTML structure • Relative XPath code can find the wrong node if the attribute or element that you look for is not unique on the page • Because XPath may be implemented differently between browsers, extra effort may be required to run WebDriver tests in those environments The Generic Test Automation Architecture
36. 36 Build Software to Test Software exactpro.com EXACTPRO State of AI https://www.stateof.ai/ • 136 слайдов
37. 37 Build Software to Test Software exactpro.com EXACTPRO Two main factors 1) Data volume 2) Computing power
38. 38 Build Software to Test Software exactpro.com EXACTPRO Big Data: How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet
39. 39 Build Software to Test Software exactpro.com EXACTPRO Big Data: How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 1) a database ☺
40. 40 Build Software to Test Software exactpro.com EXACTPRO Big Data: How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 2) web server logs 1) a database ☺
41. 41 Build Software to Test Software exactpro.com EXACTPRO Big Data: How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 2) web server logs 1) a database ☺ 3) access logs
42. 42 Build Software to Test Software exactpro.com EXACTPRO Big Data: How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 2) web server logs 1) a database ☺ 3) access logs 4) user data
43. 43 Build Software to Test Software exactpro.com EXACTPRO Big Data: How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 2) web server logs 1) a database ☺ 3) access logs 4) user data 5) random clicking
44. 44 Build Software to Test Software exactpro.com EXACTPRO Big Data: How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 2) web server logs 1) a database ☺ 3) access logs 4) user data 5) random clicking 6) other applications
45. 45 Build Software to Test Software exactpro.com EXACTPRO Tasks AI Can Be Applied To
46. 46 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias Rößler https://retest.de https://youtu.be/nvSqTlzivNM?t=1287
47. 47 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias Rößler https://retest.de
48. 48 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias Rößler https://retest.de
49. 49 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias Rößler https://retest.de
50. 50 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias Rößler https://retest.de
51. 51 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias Rößler https://retest.de
52. 52 Build Software to Test Software exactpro.com EXACTPRO Oracle Problem In computing, software engineering, and software testing, a test oracle is a mechanism for determining whether a test has passed or failed. The problem in applying AI to testing is the specification of the expected result that the outcome should be compared to. This is called the oracle problem, and it is more complex than it appears to be at first sight. In a broader sense, the oracle problem is the hardest problem of all: telling right from wrong. If this could be achieved in a general sense, all other theoretical problems could be reformulated as instances of the oracle problem and hence be solved. Note that this is not only true for mathematical or logical problems, but also for empirical and philosophical problems, and even for theological problems. For example, write a program that answers the question, “Does God exist?” and ask the test oracle whether the program is correct.
53. 53 Build Software to Test Software exactpro.com EXACTPRO Humans vs. Machines “The classic aim of automation is to replace human manual control, planning and problem solving by automatic devices and computers. …the increased interest in human factors among engineers reflects the irony that the more advanced a control system is, so the more crucial may be the contribution of the human operator.” Lisanne Bainbridge, “Ironies of Automation” (1983) OR ? World Development Report 2019: The Changing Nature of Work
54. 54 Build Software to Test Software exactpro.com EXACTPRO Tasks AI Cannot Be Applied To
55. 55 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias Rößler https://retest.de
56. 56 Build Software to Test Software exactpro.com EXACTPRO The Generic Test Automation Architecture Advantages: • You can find elements that don't have any unique attributes (id, class, name, etc.) • You can use XPath in generic locators, using the different "By" strategies (by id, by class, etc.) as needed Disadvantages: • Absolute XPath code is "brittle" and can break with the smallest change to the HTML structure • Relative XPath code can find the wrong node if the attribute or element that you look for is not unique on the page • Because XPath may be implemented differently between browsers, extra effort may be required to run WebDriver tests in those environments
57. 57 Build Software to Test Software exactpro.com EXACTPRO DOM (Document Object model)
58. 58 Build Software to Test Software exactpro.com EXACTPRO Is AI Taking Over FrontEnd Testing https://testim.io
59. 59 Build Software to Test Software exactpro.com EXACTPRO Is AI Taking Over FrontEnd Testing https://youtu.be/S9iytpte6OA?t=1212 https://testim.io
60. 60 Build Software to Test Software exactpro.com EXACTPRO street sidewalk tree crosswalk car building road pedestrian city car car branch DOM (Document Object model) street car car crosswalk person traffic light road building tree sidewalk
61. 61 Build Software to Test Software exactpro.com EXACTPRO Visual Testing https://applitools.com/
62. 62 Build Software to Test Software exactpro.com EXACTPRO Visual Testing https://youtu.be/2CnOzf_DgP8?t=82
63. 63 Build Software to Test Software exactpro.com EXACTPRO https://testautomationu.applitools.com/ai-for-element-selection-erasing-the-pain-of-fragile-test-scripts Useful Links
64. 64 Build Software to Test Software exactpro.com EXACTPRO Visual Testing
65. 65 Build Software to Test Software exactpro.com Click to edit the title text format - ტესტირება ეჯაილში - თინათინ ცირეკიძე, GeoSTQB-ის ვიცე-პრეზიდენტი და ლიბერთი ბანკის დეპარტამენტის ხარისხის მართვის ჯგუფის უფროსი - Future of Software Testing - Kari Kakkonen, Treasurer at FiSTB, Secretary at ISTQB, Director of Training at Knowit Solutions Oy and CEO and Author at Dragons Out Oy - გადასვლა ავტომატიზაციაზე - მიხეილ მაჩაბლიშვილი, Tulaco, ავტომატიზაციის უფროსი ინჟინერი - Testing Software Resiliency - Iosif Itkin, Exactpro, CEO and Co-Founder. არ გამოტოვოთ და მოინიშნეთ კალენდარში 15 ივნისი საღამოს 6 საათი. გელოდებით! https://www.facebook.com/events/ 1640675652767046
66. 66 Build Software to Test Software exactpro.com Click to edit the title text format Register here
67. 67 Build Software to Test Software exactpro.com https://www.youtube.com/c/exactprosystems https://www.youtube.com/c/exactprovlog