Testing Fedora in openQA

This session will be serve as an introduction to openQA and demonstrate how it can be used to write tests for more than just openSUSE and it's distributions. Using the example of Fedora Linux, this session will describe a methodology for designing an openQA test suite and give an guide on how attendees could go about building their own openQA test suites.
Attendees do not need any past experience with openQA or it's Perl-based Domain Specific Language, but it will help.

Transcript

1. Testing Fedora in openQA ..or how to start testing anything in openQA in less than 24 hours Richard Brown – openSUSE Chairman rbrown@opensuse.org
2. What is openQA?
3. 4 openQA • Open Source distribution testing framework • Started in 2009 • Tests Operating Systems and their applications • GUI & Console Testing • Used by the openSUSE ® distribution, Tumbleweed, and now SUSE ®
4. 5 The Magic Behind Tumbleweed • openQA is an integral part of openSUSE development • Incoming changes are 'Staged' and tested in isolation atop of the last 'known good' build • In depth testing is then carried out on the whole distribution before release • openSUSE Tumbleweed snapshots released after successful testing • New Tumbleweed release every 2-3 days, sometimes faster.
5. 6 Installation Testing
6. 7 Graphical Application Testing
7. 8 Console Testing
8. 9 PROPER Console Testing << type_string(string='\'curl ­f ­v http://10.0.2.2:20063/current_script > /tmp/scriptrPOv.sh && echo "curl­0" > /dev/ttyS0 \'' , max_interval='\'250\'' ) [Fri Apr 10 14:07:01 2015] [debug] GET "/current_script". [Fri Apr 10 14:07:01 2015] [debug] Routing to a callback. [Fri Apr 10 14:07:01 2015] [debug] Request from 127.0.0.1. [Fri Apr 10 14:07:01 2015] [debug] Routing to a callback. [Fri Apr 10 14:07:01 2015] [debug] 200 OK (0.001168s, 856.164/s). Debug: /var/lib/openqa/share/tests/sle­11­SP4­ Alpha/tests/console/curl_https.pm:8 called testapi::validate_script_output <<< wait_serial(regex='curl­0' , timeout=2 ) >>> wait_serial: curl­0: ok
9. 10 The Team • Ancor Gonzalez Sosa • Anja Stock • Dinar Valeev • Dominik Heidler • Dominique Leuenberger • Howard Guo • Jozef Pupava • Ludwig Nussel • Marita Werner • Max Lin • Ondřej Holeček • Pavel Sladek • Razvan Gherlea • Richard Brown • Stephan Kulow • Susanne Oberhauser-Hirschof • Vladimir Nadvornik • Wei Jiang
10. 11 The Team • Ancor Gonzalez Sosa • Anja Stock • Dinar Valeev • Dominik Heidler • Dominique Leuenberger • Howard Guo • Jozef Pupava • Ludwig Nussel • Marita Werner and Red Hat! • Max Lin • Ondřej Holeček • Pavel Sladek • Razvan Gherlea • Richard Brown • Stephan Kulow • Susanne Oberhauser-Hirschof • Vladimir Nadvornik • Wei Jiang
11. 12 Red Hat? • Using openQA to test Fedora 22 and Rawhide • Dockerised openQA • Contributing Patches, Ideas, and New Tooling • Public Instance https://openqa.happyassassin.net/
12. openQA in Action
13. 14 Contributing to openQA • Main Website http://os-autoinst.github.io/openQA • Documentation http://os-autoinst.github.io/openQA/documentation/
14. 15 Contributing to openQA • Repositories ‒ openSUSE/SLE Tests https://github.com/os-autoinst/os-autoinst-distri-opensuse ‒ Fedora Tests https://bitbucket.org/rajcze/openqa_fedora ‒ Backend https://github.com/os-autoinst/os-autoinst ‒ Frontend/WebUI https://github.com/os-autoinst/openQA
15. 16 To Learn More Overview of openQA Ludwig Nussel Tomorrow 13:30 Second Room
16. Writing openQA tests for Fedora
17. 18 You need one of these...
18. 19 ...and one of these
19. 20 Best way to start • Fire up a VM of Fedora • Install as usual ‒ Keyboard Shortcuts are easier to write ‒ Mark when would be a good place to 'needle' (Take a screenshot and check the screen) • Write down screens/stages, keypresses and needle points
20. 21 Writing good test notes • One 'test' (screen, stage, function) per page • One line per keypress • One line per needle point
21. 22 Example Notes
22. 23 The 'driver' – main.pm • For each distribution/test suite the main.pm decides which test modules run in which order • For a simple test suite, this is straight forward ‒ One page of notes = one 'loadtest “$filename.pm”' line in main.pm
23. 24 Making Test Files • Test modules go in a /tests folder • Subfolders help with organisation (installation, console, $application, etc) • One page of notes = one $filename.pm in the tests folder
24. 25 From notes, to code Check Needle assert_screen (FATAL) check_screen (NON-FATAL) Keypress send_key Type String type_string
25. 26 Example Notes
26. 27 Example Code assert_screen "inst-welcome", 120; send_key 'right'; send_key 'ret'; assert_screen "inst-langselection", 60; send_key 'alt-c'; if (get_var("BETA")) { assert_screen "inst-betawarning", 30; send_key "tab"; assert_screen "inst-fateaccept", 5; send_key "ret"; } assert_screen "inst-summary", 5;
27. Rinse, Repeat… then…
28. Questions?
29. 30 Have a Lot of Fun, and Join Us At: www.opensuse.org
30. General Disclaimer This document is not to be construed as a promise by any participating organisation to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. openSUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for openSUSE products remains at the sole discretion of openSUSE. Further, openSUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All openSUSE marks referenced in this presentation are trademarks or registered trademarks of SUSE LLC, in the United States and other countries. All third-party trademarks are the property of their respective owners. License This slide deck is licensed under the Creative Commons Attribution-ShareAlike 4.0 International license. It can be shared and adapted for any purpose (even commercially) as long as Attribution is given and any derivative work is distributed under the same license. Details can be found at https://creativecommons.org/licenses/by-sa/4.0/ Credits Template Richard Brown rbrown@opensuse.org Design & Inspiration openSUSE Design Team http://opensuse.github.io/branding- guidelines/