Testing Fedora in openQA ..or how to start testing anything in openQA in less than 24 hours Richard Brown – openSUSE Chairman [email protected]

What is openQA?

4 openQA • Open Source distribution testing framework • Started in 2009 • Tests Operating Systems and their applications • GUI & Console Testing • Used by the openSUSE ® distribution, Tumbleweed, and now SUSE ®

5 The Magic Behind Tumbleweed • openQA is an integral part of openSUSE development • Incoming changes are 'Staged' and tested in isolation atop of the last 'known good' build • In depth testing is then carried out on the whole distribution before release • openSUSE Tumbleweed snapshots released after successful testing • New Tumbleweed release every 2-3 days, sometimes faster.

6 Installation Testing

7 Graphical Application Testing

8 Console Testing

9 PROPER Console Testing << type_string(string='\'curl ­f ­v http://10.0.2.2:20063/current_script > /tmp/scriptrPOv.sh && echo "curl­0" > /dev/ttyS0 \'' , max_interval='\'250\'' ) [Fri Apr 10 14:07:01 2015] [debug] GET "/current_script". [Fri Apr 10 14:07:01 2015] [debug] Routing to a callback. [Fri Apr 10 14:07:01 2015] [debug] Request from 127.0.0.1. [Fri Apr 10 14:07:01 2015] [debug] Routing to a callback. [Fri Apr 10 14:07:01 2015] [debug] 200 OK (0.001168s, 856.164/s). Debug: /var/lib/openqa/share/tests/sle­11­SP4­ Alpha/tests/console/curl_https.pm:8 called testapi::validate_script_output <<< wait_serial(regex='curl­0' , timeout=2 ) >>> wait_serial: curl­0: ok

10 The Team • Ancor Gonzalez Sosa • Anja Stock • Dinar Valeev • Dominik Heidler • Dominique Leuenberger • Howard Guo • Jozef Pupava • Ludwig Nussel • Marita Werner • Max Lin • Ondřej Holeček • Pavel Sladek • Razvan Gherlea • Richard Brown • Stephan Kulow • Susanne Oberhauser-Hirschof • Vladimir Nadvornik • Wei Jiang

11 The Team • Ancor Gonzalez Sosa • Anja Stock • Dinar Valeev • Dominik Heidler • Dominique Leuenberger • Howard Guo • Jozef Pupava • Ludwig Nussel • Marita Werner and Red Hat! • Max Lin • Ondřej Holeček • Pavel Sladek • Razvan Gherlea • Richard Brown • Stephan Kulow • Susanne Oberhauser-Hirschof • Vladimir Nadvornik • Wei Jiang

12 Red Hat? • Using openQA to test Fedora 22 and Rawhide • Dockerised openQA • Contributing Patches, Ideas, and New Tooling • Public Instance https://openqa.happyassassin.net/

openQA in Action

14 Contributing to openQA • Main Website http://os-autoinst.github.io/openQA • Documentation http://os-autoinst.github.io/openQA/documentation/

15 Contributing to openQA • Repositories ‒ openSUSE/SLE Tests https://github.com/os-autoinst/os-autoinst-distri-opensuse ‒ Fedora Tests https://bitbucket.org/rajcze/openqa_fedora ‒ Backend https://github.com/os-autoinst/os-autoinst ‒ Frontend/WebUI https://github.com/os-autoinst/openQA

16 To Learn More Overview of openQA Ludwig Nussel Tomorrow 13:30 Second Room

Writing openQA tests for Fedora

18 You need one of these...

19 ...and one of these

20 Best way to start • Fire up a VM of Fedora • Install as usual ‒ Keyboard Shortcuts are easier to write ‒ Mark when would be a good place to 'needle' (Take a screenshot and check the screen) • Write down screens/stages, keypresses and needle points

21 Writing good test notes • One 'test' (screen, stage, function) per page • One line per keypress • One line per needle point

22 Example Notes

23 The 'driver' – main.pm • For each distribution/test suite the main.pm decides which test modules run in which order • For a simple test suite, this is straight forward ‒ One page of notes = one 'loadtest “$filename.pm”' line in main.pm

24 Making Test Files • Test modules go in a /tests folder • Subfolders help with organisation (installation, console, $application, etc) • One page of notes = one $filename.pm in the tests folder

25 From notes, to code Check Needle assert_screen (FATAL) check_screen (NON-FATAL) Keypress send_key Type String type_string

26 Example Notes

27 Example Code assert_screen "inst-welcome", 120; send_key 'right'; send_key 'ret'; assert_screen "inst-langselection", 60; send_key 'alt-c'; if (get_var("BETA")) { assert_screen "inst-betawarning", 30; send_key "tab"; assert_screen "inst-fateaccept", 5; send_key "ret"; } assert_screen "inst-summary", 5;

Rinse, Repeat… then…

Questions?

30 Have a Lot of Fun, and Join Us At: www.opensuse.org

General Disclaimer This document is not to be construed as a promise by any participating organisation to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. openSUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for openSUSE products remains at the sole discretion of openSUSE. Further, openSUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All openSUSE marks referenced in this presentation are trademarks or registered trademarks of SUSE LLC, in the United States and other countries. All third-party trademarks are the property of their respective owners. License This slide deck is licensed under the Creative Commons Attribution-ShareAlike 4.0 International license. It can be shared and adapted for any purpose (even commercially) as long as Attribution is given and any derivative work is distributed under the same license. Details can be found at https://creativecommons.org/licenses/by-sa/4.0/ Credits Template Richard Brown [email protected] Design & Inspiration openSUSE Design Team http://opensuse.github.io/branding- guidelines/