How openQA works

Technical Background and implementation details

Transcript

1. Stephan Kulow Coolo coolo@suse.de How openQA works Technical background
2. About me • Working for SUSE • openSUSE Release Manager until Leap 42.1 • Acting as openQA Architect • Father of 2
3. What is this talk about? • Details you might not know • Architecture of openQA • Implementation details that can bite you • Q & A
4. What is this talk not? • Introduction into the tests API • Beginner’s Guide to openQA
5. (brief) Introduction to openQA • openQA started as ‘Create videos of installations’ project of Bernhard M. Wiedemann early 2010 (isotovideo) • End of 2011 first release of openQA – still mainly a tool for Bernhard to create bug reports • A set of CGI scripts to display results
6. openQA 1.0
7. openQA 4.4
9. Backends • Common between all backends: VNC and power management • Main use case of openQA is still qemu/kvm. • Qemu covers several ‘hardware’ setups – Laptop – Uefi/legacy bios – IDE/SCSI/NVME – RAID/Multipath
10. Needle Matching
11. Needle matching • Needle is region of interest in a reference screen
12. Xterm started!
13. Xterm started!
14. Xterm started!
15. Xterm started!
16. Xterm started!
17. Error of dif • Most likely spots are evaluated • Number of difference counted and weighted • Match level decides if the area is good
18. Error of dif vs => => 100% Match
19. Why font changes hurt • Even small font changes can have a big impact on the spacing • To users the change is minimal, but many pixels change • Example...
20. Xterm started!
21. Xterm started!
22. Error of dif vs => => 34% Match
23. assert_screen in action • It’s important to understand the blocking nature of tests • Tests are blocking perl code assert_screen ‘xterm’; type_string ‘hallo’; assert_screen ‘hallo-typed’;
24. Isotovideo is command based • Test process sends ‘check_screen’ to isotovideo and then blocks waiting for reply • Isotovideo is event loop awaiting events from various parts, but is only the messenger • Heavy lifting is done by the backend process • Regulary (like every 500ms) taking screenshot from VNC • Checking screen against needles
25. Consoles
26. Consoles • Consoles are an important concept to support other platforms • Even for qemu we have them though: select_console ‘user-console’; select_console ‘root-console’; • TtyConsoles are second most used, most prominent is ‘sut’ - which is VNC protocol • Consoles can be stacked
27. Xterm consoles • To test remote machines we sometimes need local consoles connected • We start local Xvnc with an xterm and a ssh connection • select_console ‘root-console’ as in qemu • type_string + assert_screen just as in qemu • But actually connected through ssh
28. Bare Metal Testing • You can do more with local xterms • Test drive remote machines completely • Xterm + IPMITOOL + PXE Server → The world is ours
29. Answers!
30. Join Us at www.opensuse.org
31. License This slide deck is licensed under the Creative Commons Attribution-ShareAlike 4.0 International license. It can be shared and adapted for any purpose (even commercially) as long as Attribution is given and any derivative work is distributed under the same license. Details can be found at https://creativecommons.org/licenses/by-sa/4.0/ General Disclaimer This document is not to be construed as a promise by any participating organisation to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. openSUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for openSUSE products remains at the sole discretion of openSUSE. Further, openSUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All openSUSE marks referenced in this presentation are trademarks or registered trademarks of SUSE LLC, in the United States and other countries. All third-party trademarks are the property of their respective owners. Credits Template Richard Brown rbrown@opensuse.org Design & Inspiration openSUSE Design Team http://opensuse.github.io/branding- guidelines/