Using Machine Learning to Test Web Applications

Using Machine Learning to Test Web Applications

Elena Treshcheva
Researcher, Exactpro

San Diego State University, 12 June 2020

Video : https://youtu.be/hnBlsGs9LzE

In this presentation, Elena Treshcheva, Researcher, Business Development Manager at Exactpro, describes the fundamental concepts of machine learning and compares the capabilities of DOM-based testing to those of visual AI. The presentation was recorded at the open webinar held for San Diego State University students on 12 June 2020.

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

5206c19df417b8876825b5561344c1a0?s=128

Exactpro

June 12, 2020
Tweet

Transcript

  1. 1 Using Machine Learning to Test Web Applications Elena Treshcheva

    Researcher, Exactpro San Diego State University, 12 June 2020
  2. 2 Build Software to Test Software exactpro.com Overview • References

    / Useful links • A web application from the point of view of a human being • Software testing, test automation and machine learning • A human being from the point of view of a web application • Testing with Selenium • The Oracle problem in software testing • ML applied to testing of DOM • ML applied to visual testing
  3. 3 Build Software to Test Software exactpro.com A specialist firm

    focused on functional and non-functional testing of exchanges, clearing houses, depositories, trade repositories and other financial market infrastructures. About Exactpro Headquartered in the UK with operations in the US and Eastern Europe. The clients are regulated by the FCA, Bank of England and their counterparts from other countries. Exactpro provides software testing services for mission critical technology that underpins global financial markets. The firm is experienced with trading and clearing and settlement platforms, market data systems, collateral management and risk management systems, central data warehouses, regulatory reporting, etc. Incorporated in 2009 with 10 people, Exactpro is now employing over 590 specialists. Part of the London Stock Exchange Group (LSEG) from May 2015 until January 2018, when the Exactpro management proceeded through the buyout of the company from LSEG.
  4. 4 Build Software to Test Software exactpro.com EXACTPRO Non-Deterministic Probabilistic

    Non-Testable Oracle Problem Mission-critical Financial Markets Infrastructures
  5. 5 Build Software to Test Software exactpro.com Typical requirements for

    an Exchange system Daily capacity - 100+ mln transactions Peak rates - 40k+ transactions per second Average round-trip latency - <100 microseconds Availability - 100% 3000 trx 2.5 cm <1 mm
  6. 6 Build Software to Test Software exactpro.com EXACTPRO Useful Links

    https://frontendmasters.com/books/front-end-handbook/2019/
  7. 7 Build Software to Test Software exactpro.com EXACTPRO https://youtu.be/nvSqTlzivNM https://www.slideshare.net/roesslerj/how-to-apply-ai-to-testing

    Useful Links
  8. 8 Build Software to Test Software exactpro.com 25 February 2020

    https://youtu.be/KGVQVBdjvCU Useful Links
  9. 9 Build Software to Test Software exactpro.com EXACTPRO https://www.gasq.org/en/certification/selenium-tester.html Useful

    Links
  10. 10 Build Software to Test Software exactpro.com EXACTPRO https://www.youtube.com/watch?v=S9iytpte6OA https://www.testim.io/

    Useful Links
  11. 11 Build Software to Test Software exactpro.com EXACTPRO https://testautomationu.applitools.com/ai-for-element-selection-erasing-the-pain-of-fragile-test-scripts Useful

    Links
  12. 12 Build Software to Test Software exactpro.com EXACTPRO The structure

    of a Web Application Browser send requests to the server, receive responses and visualize them as a page Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. HTTP(S) Request HTTP(S) Response 1 2 CSS HTML Javascript Internet Web Servers Static Content Browser DOM model
  13. 13 Build Software to Test Software exactpro.com EXACTPRO Browser send

    requests to the server, receive responses and visualize them as a page Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. HTTP(S) Request HTTP(S) Response 1 2 CSS HTML Javascript Internet Application Servers Data Web Servers Static Content Browser DOM model The structure of a Web Application
  14. 14 Build Software to Test Software exactpro.com Browser send requests

    to the server, receive responses and visualize them as a page Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 2 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Internet The structure of a Web Application
  15. 15 Build Software to Test Software exactpro.com EXACTPRO The structure

    of a Web Application Browser send requests to the server, receive responses and visualize them as a page Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 AJAX allows to create asynchronous web interactions without interfering with the display and behaviour of the existing page 4 2 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Internet
  16. 16 Build Software to Test Software exactpro.com EXACTPRO Browser send

    requests to the server, receive responses and visualize them as a page WebSocket enables interaction between a web browser and a web server with lower overhead than half-duplex alternatives such as HTTP polling Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 AJAX allows to create asynchronous web interactions without interfering with the display and behaviour of the existing page 4 2 5 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet The structure of a Web Application
  17. 17 Build Software to Test Software exactpro.com EXACTPRO Browser send

    requests to the server, receive responses and visualize them as a page WebSocket enables interaction between a web browser and a web server with lower overhead than half-duplex alternatives such as HTTP polling Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 AJAX allows to create asynchronous web interactions without interfering with the display and behaviour of the existing page 4 2 5 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet The structure of a Web Application
  18. 18 Build Software to Test Software exactpro.com EXACTPRO Browser send

    requests to the server, receive responses and visualize them as a page WebSocket enables interaction between a web browser and a web server with lower overhead than half-duplex alternatives such as HTTP polling Responsive front-end design allows a site to adapt to user’s device. Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 AJAX allows to create asynchronous web interactions without interfering with the display and behaviour of the existing page 4 2 5 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet The structure of a Web Application
  19. 19 Build Software to Test Software exactpro.com EXACTPRO Browser send

    requests to the server, receive responses and visualize them as a page WebSocket enables interaction between a web browser and a web server with lower overhead than half-duplex alternatives such as HTTP polling Responsive front-end design allows a site to adapt to user’s device. Everything a user sees in the browser is a mix of HTML, CSS, and JavaScript. The back-end server-side scripts process the request, pull what they need from the database and application servers and then send it back. 1 Client-side scripts Run in the browser and process information without sending requests to the server 3 AJAX allows to create asynchronous web interactions without interfering with the display and behaviour of the existing page 4 2 5 CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet Web applications are always multi-user applications and many of them work under heavy load and are subject to concurrency access 6 The structure of a Web Application
  20. 20 Build Software to Test Software exactpro.com EXACTPRO Quality -

    The degree to which a component or system satisfies the stated and implied needs of its various stakeholders. Software Testing - The process consisting of all lifecycle activities, both static and dynamic, concerned with planning, preparation and evaluation of a component or system and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects. A person can make an error, which can lead to the introduction of a defect (fault or bug) in the software code. If a defect in the code is executed, this may cause a failure Software Testing
  21. 21 Build Software to Test Software exactpro.com EXACTPRO Testing Levels

    CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet A C D E F B
  22. 22 Build Software to Test Software exactpro.com EXACTPRO Automation Automation

    is the technology by which a process or procedure is performed with minimal human assistance. “The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency” Bill Gates
  23. 23 Build Software to Test Software exactpro.com EXACTPRO A Model

    of the Human from a Web App’s Perspective
  24. 24 Build Software to Test Software exactpro.com EXACTPRO A Model

    of the Human from a Web App’s Perspective
  25. 25 Build Software to Test Software exactpro.com EXACTPRO A Model

    of the Human from a Web App’s Perspective
  26. 26 Build Software to Test Software exactpro.com EXACTPRO Document <HTML>

    <head> <body> <h1> Root element Element “The Header” <h1> “the link” “href” “Sub Header” <h2> “The title” <title> Element Element Element Element Text Attribute Text Text Element Text The DOM defines: All HTML elements as objects • The properties of all HTML elements • The methods that can be used to access all HTML elements • The events that affect all HTML elements "The W3C Document Object Model (DOM) is a platform and language-neutral interface that allows programs and scripts to dynamically access and update the content, structure, and style of a document." DOM (Document Object model)
  27. 27 Build Software to Test Software exactpro.com EXACTPRO DOM модель

  28. 28 Build Software to Test Software exactpro.com EXACTPRO DOM (Document

    Object model)
  29. 29 Build Software to Test Software exactpro.com EXACTPRO Document <HTML>

    <head> <body> <h1> Root element Element “The Header” <h1> “the link” “href” “Sub Header” <h2> “The title” <title> Element Element Element Element Text Attribute Text Text Element Text The DOM defines: All HTML elements as objects • The properties of all HTML elements • The methods that can be used to access all HTML elements • The events that affect all HTML elements "The W3C Document Object Model (DOM) is a platform and language-neutral interface that allows programs and scripts to dynamically access and update the content, structure, and style of a document." DOM (Document Object model)
  30. 30 Build Software to Test Software exactpro.com EXACTPRO Testing Levels

    CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet A C D E F B
  31. 31 Build Software to Test Software exactpro.com EXACTPRO DOM (Document

    Object model)
  32. 32 Build Software to Test Software exactpro.com DOM (Document Object

    model) street car car crosswalk person traffic light road building tree sidewalk
  33. 33 Build Software to Test Software exactpro.com EXACTPRO street sidewalk

    tree crosswalk car building road pedestrian city car car branch DOM (Document Object model) street car car crosswalk person traffic light road building tree sidewalk
  34. 34 Build Software to Test Software exactpro.com EXACTPRO The Generic

    Test Automation Architecture
  35. 35 Build Software to Test Software exactpro.com EXACTPRO Advantages: •

    You can find elements that don't have any unique attributes (id, class, name, etc.) • You can use XPath in generic locators, using the different "By" strategies (by id, by class, etc.) as needed Disadvantages: • Absolute XPath code is "brittle" and can break with the smallest change to the HTML structure • Relative XPath code can find the wrong node if the attribute or element that you look for is not unique on the page • Because XPath may be implemented differently between browsers, extra effort may be required to run WebDriver tests in those environments The Generic Test Automation Architecture
  36. 36 Build Software to Test Software exactpro.com EXACTPRO State of

    AI https://www.stateof.ai/ • 136 слайдов
  37. 37 Build Software to Test Software exactpro.com EXACTPRO Two main

    factors 1) Data volume 2) Computing power
  38. 38 Build Software to Test Software exactpro.com EXACTPRO Big Data:

    How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet
  39. 39 Build Software to Test Software exactpro.com EXACTPRO Big Data:

    How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 1) a database ☺
  40. 40 Build Software to Test Software exactpro.com EXACTPRO Big Data:

    How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 2) web server logs 1) a database ☺
  41. 41 Build Software to Test Software exactpro.com EXACTPRO Big Data:

    How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 2) web server logs 1) a database ☺ 3) access logs
  42. 42 Build Software to Test Software exactpro.com EXACTPRO Big Data:

    How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 2) web server logs 1) a database ☺ 3) access logs 4) user data
  43. 43 Build Software to Test Software exactpro.com EXACTPRO Big Data:

    How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 2) web server logs 1) a database ☺ 3) access logs 4) user data 5) random clicking
  44. 44 Build Software to Test Software exactpro.com EXACTPRO Big Data:

    How to obtain it? CSS HTML Javascript Application Servers Data Web Servers Static Content Browser DOM model HTTP(S) Request HTTP(S) Response Websocket Internet 2) web server logs 1) a database ☺ 3) access logs 4) user data 5) random clicking 6) other applications
  45. 45 Build Software to Test Software exactpro.com EXACTPRO Tasks AI

    Can Be Applied To
  46. 46 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias

    Rößler https://retest.de https://youtu.be/nvSqTlzivNM?t=1287
  47. 47 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias

    Rößler https://retest.de
  48. 48 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias

    Rößler https://retest.de
  49. 49 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias

    Rößler https://retest.de
  50. 50 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias

    Rößler https://retest.de
  51. 51 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias

    Rößler https://retest.de
  52. 52 Build Software to Test Software exactpro.com EXACTPRO Oracle Problem

    In computing, software engineering, and software testing, a test oracle is a mechanism for determining whether a test has passed or failed. The problem in applying AI to testing is the specification of the expected result that the outcome should be compared to. This is called the oracle problem, and it is more complex than it appears to be at first sight. In a broader sense, the oracle problem is the hardest problem of all: telling right from wrong. If this could be achieved in a general sense, all other theoretical problems could be reformulated as instances of the oracle problem and hence be solved. Note that this is not only true for mathematical or logical problems, but also for empirical and philosophical problems, and even for theological problems. For example, write a program that answers the question, “Does God exist?” and ask the test oracle whether the program is correct.
  53. 53 Build Software to Test Software exactpro.com EXACTPRO Humans vs.

    Machines “The classic aim of automation is to replace human manual control, planning and problem solving by automatic devices and computers. …the increased interest in human factors among engineers reflects the irony that the more advanced a control system is, so the more crucial may be the contribution of the human operator.” Lisanne Bainbridge, “Ironies of Automation” (1983) OR ? World Development Report 2019: The Changing Nature of Work
  54. 54 Build Software to Test Software exactpro.com EXACTPRO Tasks AI

    Cannot Be Applied To
  55. 55 Build Software to Test Software exactpro.com EXACTPRO Dr. Jeremias

    Rößler https://retest.de
  56. 56 Build Software to Test Software exactpro.com EXACTPRO The Generic

    Test Automation Architecture Advantages: • You can find elements that don't have any unique attributes (id, class, name, etc.) • You can use XPath in generic locators, using the different "By" strategies (by id, by class, etc.) as needed Disadvantages: • Absolute XPath code is "brittle" and can break with the smallest change to the HTML structure • Relative XPath code can find the wrong node if the attribute or element that you look for is not unique on the page • Because XPath may be implemented differently between browsers, extra effort may be required to run WebDriver tests in those environments
  57. 57 Build Software to Test Software exactpro.com EXACTPRO DOM (Document

    Object model)
  58. 58 Build Software to Test Software exactpro.com EXACTPRO Is AI

    Taking Over FrontEnd Testing https://testim.io
  59. 59 Build Software to Test Software exactpro.com EXACTPRO Is AI

    Taking Over FrontEnd Testing https://youtu.be/S9iytpte6OA?t=1212 https://testim.io
  60. 60 Build Software to Test Software exactpro.com EXACTPRO street sidewalk

    tree crosswalk car building road pedestrian city car car branch DOM (Document Object model) street car car crosswalk person traffic light road building tree sidewalk
  61. 61 Build Software to Test Software exactpro.com EXACTPRO Visual Testing

    https://applitools.com/
  62. 62 Build Software to Test Software exactpro.com EXACTPRO Visual Testing

    https://youtu.be/2CnOzf_DgP8?t=82
  63. 63 Build Software to Test Software exactpro.com EXACTPRO https://testautomationu.applitools.com/ai-for-element-selection-erasing-the-pain-of-fragile-test-scripts Useful

    Links
  64. 64 Build Software to Test Software exactpro.com EXACTPRO Visual Testing

  65. 65 Build Software to Test Software exactpro.com Click to edit

    the title text format - ტესტირება ეჯაილში - თინათინ ცირეკიძე, GeoSTQB-ის ვიცე-პრეზიდენტი და ლიბერთი ბანკის დეპარტამენტის ხარისხის მართვის ჯგუფის უფროსი - Future of Software Testing - Kari Kakkonen, Treasurer at FiSTB, Secretary at ISTQB, Director of Training at Knowit Solutions Oy and CEO and Author at Dragons Out Oy - გადასვლა ავტომატიზაციაზე - მიხეილ მაჩაბლიშვილი, Tulaco, ავტომატიზაციის უფროსი ინჟინერი - Testing Software Resiliency - Iosif Itkin, Exactpro, CEO and Co-Founder. არ გამოტოვოთ და მოინიშნეთ კალენდარში 15 ივნისი საღამოს 6 საათი. გელოდებით! https://www.facebook.com/events/ 1640675652767046
  66. 66 Build Software to Test Software exactpro.com Click to edit

    the title text format Register here
  67. 67 Build Software to Test Software exactpro.com https://www.youtube.com/c/exactprosystems https://www.youtube.com/c/exactprovlog