Slide 1

Slide 1 text

How MagicPod and its AI evolve test automation with Selenium From code to no-code Nozomi Ito

Slide 2

Slide 2 text

CEO of MagicPod, a Japanese test automation tool company The organizer of the Japan Selenium User Community Author of a Japanese Selenium book About me Nozomi Ito

Slide 3

Slide 3 text

This is my second trip to Indonesia First trip Second trip

Slide 4

Slide 4 text

No-code, cloud-based test automation SaaS For browser and mobile app testing Simple UI but with powerful features Free plan and paid plan Trusted by more than 500 companies No-code test automation tools

Slide 5

Slide 5 text

Agenda 1. My journey from Selenium to MagicPod 2. How MagicPod leverages Selenium internally 3. Values MagicPod adds to Selenium 4. Utilization of AI 5. MagicPod’s goal for the future of no-code

Slide 6

Slide 6 text

Agenda 1. My journey from Selenium to MagicPod 2. How MagicPod leverages Selenium internally 3. Values MagicPod adds to Selenium 4. Utilization of AI 5. MagicPod’s goal for the future of no-code

Slide 7

Slide 7 text

Established Selenium company (2012) Worked on consulting/support for test automation with Selenium in Japan Selenium: Open source browser automation framework

Slide 8

Slide 8 text

Promoted Selenium in Japan Launched Japan Selenium User community (from PublicKey) Published "Selenium Practical introduction" Held periodical Selenium workshops Organized SeleniumConf Tokyo

Slide 9

Slide 9 text

Challenges during the promotion Many Japanese companies started test automation with Selenium But large amount of Japanese companies still prefer to manual test

Slide 10

Slide 10 text

Why they did not automate E2E tests Lack motivation and knowedge about Selenium Cost of maintaining E2E test was too high In Japan, most QAs don't have technical background. Lack skills to use Selenium. Even commercial tools (Katalon, Ranorex, etc.) were too difficult for them QAs Engineers

Slide 11

Slide 11 text

From Selenium to no-code (2017) I decided to create MagicPod to spread E2E test automation No-code tool using Selenium/Appium internally Cloud service with easy setup Self healing AI to reduce maintenance costs Concepts

Slide 12

Slide 12 text

No-code has been widely accepted in Japan (2024) Growth of number of MagicPod users

Slide 13

Slide 13 text

Many industrial experts joined MagicPod Authors of test automation books Translators of test automation books Appium/Selenium conf presenters Selenium/Appium contributors/committers

Slide 14

Slide 14 text

Agenda 1. My journey from Selenium to MagicPod 2. How MagicPod leverages Selenium internally 3. Values MagicPod adds to Selenium 4. Utilization of AI 5. MagicPod’s goal for the future of no-code

Slide 15

Slide 15 text

How MagicPod works

Slide 16

Slide 16 text

Internal mechanism - test creation UI scan logic Analyze Server Upload Detect UI image UI tree XML human-readable caption locator

Slide 17

Slide 17 text

Internal mechanism - test creation [ { "command": "input", "element": "\"Name\" input area", "value": "Demo user" }, ... ] Test script Database Drag & Drop Store A command is added depending on the dropped item Scan result

Slide 18

Slide 18 text

Internal mechanism - test execution driver.findElementbyName ('"Name" input area') .sendKeys("Demo user"); ... driver.findElementbyName ('"Name" input area') .sendKeys("Demo user"); ... driver.findElementbyName ('"Name" input area') .sendKeys("Demo user"); ... Server JSON Node.js × Mocha × Selenium(or Appium) script Download Convert Run Test result Upload Cloud device Database [ { "command": "input", "element": "\"Name\" input area", "value": "Demo user" }, ... ]

Slide 19

Slide 19 text

System structure of MagicPod Run Run MagicPod has 2 types of execution methods. Server Local Local Server Database Database Cloud Browser device Local Browser device Cloud test Local test Script Script MagicPod Desktop

Slide 20

Slide 20 text

Agenda 1. My journey from Selenium to MagicPod 2. How MagicPod leverages Selenium internally 3. Values MagicPod adds to Selenium 4. Utilization of AI 5. MagicPod’s goal for the future of no-code

Slide 21

Slide 21 text

Sophisticated locator calculation Automatically calculates and suggests the most robust locator Locator candidates in the recommended order

Slide 22

Slide 22 text

Sophisticated locator calculation Order of locator recommendation Unique id data-testid → id → name → text() → alt → value → title → aria-label → placeholder → class: id = userName tag name + unique id xpath = //button[@id='log in'] label xpath = //input[@id=//label[text()='User name']/@for]] tag name xpath = //input[2] child xpath = //div[@id='mainArea']/button[1] parent xpath = //span[text()='log in']/parent::button[1] sibling xpath = //div[text()='Name']/following-sibling::input[1] full path xpath xpath = /body[1]/div[1]/div[2]/div[3]/a

Slide 23

Slide 23 text

Sophisticated locator calculation Random unique identifiers have low priority Use statistical approach to judge "random" xpath=//button[text()='Sign up'] xpath=//button[2] xpath=//div[@id='mainForm']/div[1]/button[1] id=577656711eee44a

Slide 24

Slide 24 text

Automatic wait and retry fdffff Selenium Various explicit wait commands Automtically wait for elements to appear with implictWait driver.manage().timeouts() .implicitlyWait( Duration.ofSeconds(2));

Slide 25

Slide 25 text

Automatic wait and retry MagicPod Explicit wait commands like selenium More sophisticated implicit wait Wait for element position to be settled like Playwright Retry for stale element error fdffff WebElement e = driver.findElement( By.id("logIn")); e.click();

Slide 26

Slide 26 text

Powerful features without code Features for maintainability Shared step Conditionals While loop Variables coming soon

Slide 27

Slide 27 text

Powerful features without code Powerful commands resembling code-based testing Web api call 2FA code calculation Regular expression match Date calculation

Slide 28

Slide 28 text

Seamless frame and webview handling driver.switchTo().frame( driver.findElement( By.id("iframe1"))); driver.findElement( By.id("OK")).click(); Automatically calculate which context to switch Frame for web test and webview for native mobile app test Selenium MagicPod “OK” Button id=OK frame: iframe1

Slide 29

Slide 29 text

Built-in visual diff check feature Easily detect broken design issues. Detected the visual diff

Slide 30

Slide 30 text

Agenda 1. My journey from Selenium to MagicPod 2. How MagicPod leverages Selenium internally 3. Values MagicPod adds to Selenium 4. Utilization of AI 5. MagicPod’s goal for the future of no-code

Slide 31

Slide 31 text

AI uses in MagicPod Human-readable UI element captions 1. Self-healing 2. Summarization of test script 3.

Slide 32

Slide 32 text

1. Human-readable UI element captions Human-readable caption ・Calculated by AI ・Not used by the test runner Selenium/Appium locator ・Calculated from UI tree ・Used by the test runner

Slide 33

Slide 33 text

1. Human-readable UI element captions Information on the UI tree: attributes, texts, tag names, etc. Parent-child relation between elements

Slide 34

Slide 34 text

1. Human-readable UI element captions OCR result (Recognize characters in the image) Object recognition by the deep learning engine

Slide 35

Slide 35 text

2. Self-healing When a test fails to find a UI element due to a change in the application’s page structure, AI modifies the script according to the page change.

Slide 36

Slide 36 text

2. Self-healing Click “register” Test script name attribute updated! What happens if it is executed? name=register name=registerButton

Slide 37

Slide 37 text

2. Self-healing The AI engine updates script and the test passes The user is notified of the update detail

Slide 38

Slide 38 text

3. Summarization of test script Summaries of test cases are automatically generated.

Slide 39

Slide 39 text

3. Summarization of test script Navigate to URL https://hotel.testplanisphere.dev/en-US/signup.html // Log out once if being logged in If UI element "Logout" button is visible Click "Logout" button Navigate to URL https://hotel.testplanisphere.dev/en-US/signup.html End branch // Sing up by using an unique email address Store unique value generated based on the current time to variable UNIQUE_KEY Input test-user${UNIQUE_KEY}@example.com to "Email" input area Input ${PASSWORD} to "Password" input area Input ${PASSWORD} to "Password(confirmation)" input area Input Test user to "Name" input area Input Tokyo, Japan to "Address" input area Input 00011122233 to "Tel" input area Select I do not answer. from "Gender" combo box Input 2000-07-04 to "Date of birth" input area Check "Receive notification" checkbox Click "Sign up" button // Assert values set on the previous page are correctly displayed Assert value of UI element Email value equals test-user${UNIQUE_KEY}@example.com Assert value of UI element Name value equals Test user Assert value of UI element Membership value equals Premium Assert value of UI element Address value equals Tokyo, Japan Assert value of UI element Tel value equals 00011122233 Assert value of UI element Gender value equals not answered Assert value of UI element Date of birth value equals July 4, 2000 Assert value of UI element Notification value equals received // Confirm that the user is successfully created and log-in by the user succeeds Click "Logout" button Click Login Input test-user${UNIQUE_KEY}@example.com to "Email" input area Input ${PASSWORD} to "Password" input area Click "Login" button Assert UI element "MyPage" header exists Human-readable text The script "Check that signup works correctly" automates end-to-end testing for user signup on a hotel website: 1. It ensures logout if the user is already logged in. 2. Fills out the signup form with unique details like email, password, name, etc. 3. Submits the form and verifies entered data's correctness on the confirmation page. 4. Confirms successful signup by logging out and logging back in, checking for the "MyPage" header's presence. Test script Summary MagicPod public API Generative AI

Slide 40

Slide 40 text

3. Summarization of test script Prompt Create a short summary (less than 300 characters) of the following browser testing user flow briefly in English: [Human-readable test script ]

Slide 41

Slide 41 text

Agenda 1. My journey from Selenium to MagicPod 2. How MagicPod leverages Selenium internally 3. Values MagicPod adds to Selenium 4. Utilization of AI 5. MagicPod’s goal for the future of no-code

Slide 42

Slide 42 text

Integration of code and no-code Encourage collaboration between engineers and non-engineers Short-term goal of MagicPod Mutually convertible LaunchApp() input(e('register', '"name" input area'), 'Yamada Taro') input(e('register', '"gender" input area'), 'Male') tap(e('register', '"Register" button'))

Slide 43

Slide 43 text

Execute tests directly from test cases designed for manual testing. Next major goal: creating test cases entirely from scratch. Log in with [email protected] / pass001 Long-term goal of MagicPod

Slide 44

Slide 44 text

No content