Automating installation Tests for ownCloud’s Desktop Client

Talk given in the https://occon17.owncloud.org about Desktop Automation

Transcript

1. Automating installation Tests for ownCloud’s Desktop Client NUE, 22/09/2017
2. $ whoami ! @alfageme " @SamuAlfageme
3. Background
4. 4 Why Does This Matter at All • OS Support Policy: github.com/owncloud/client/wiki/OS-Support-Policy#linux • Recurrent issues when something changed upstream in the distribution mirrors / there were dep.issues • 2017: the year of Linux in the Desktop !!! tl;dr: 1 Different Distributions 8 # apt-get install owncloud-client The following packages have unmet dependencies: owncloud-client:i386 : Depends: libowncloudsync0:i386 (= 2.2.4-1.1) but it is not going to be installed Depends: libqt5keychain1:i386 but it is not going to be installed Depends: owncloud-client-l10n:i386 but it is not installable
5. Outline
6. 6 1.Automating Installations 101 2.Extensibility: GUI testing 3.Future of Desktop App Distribution
7. How2: Automation
8. 8 Not Science • Issue: fragmented panorama. Need to abstract & adapt dynamically • Req. Offer a good interface for package distribution: ‘repository descriptor’ 1:1 rel. - platform (distribution) : repository • Discover the platform (@runtime) and apply the right config
9. 9 Open Source & FOSS traction Simple & Integrated w/ VCS Pretty Decent Free Runners Lots of Docs and Examples I Know & Contribute to GitLab Why GitLab CI? gitlab.com/alfageme/client-linux-tests/pipelines
10. 10 after_script: - after_install/test01.sh test:Centos_6: image: centos:6 before_script: - ./setup_installation.py -u $REPOSITORY_ENDPOINT -d CentOS -v 6 script: - yum install -y $APPLICATION_EXECUTABLE-client - $APPLICATION_EXECUTABLE'cmd' –version 1. (image) – Spawn a vanilla container 2. (before_script) – Run the Python helper script (w/ distro, version): 1. Parse and map the repo descriptor 2. Set the right environment variables 3. Call the repository setup script 3. (script) – Install & run 4. (after_script) – Test(s)
11. 11 […] "openSUSE_42.3":{ ...}, "CentOS_6":{ "repo":"https://.../CentOS_6", ...}, […] # CentOS.sh #!/usr/bin/env bash cd /etc/yum.repos.d/ wget $REPOSITORY $REPOSITORY $ $APPLICATION_EXECUTABLE'cmd' --version ownCloud version 2.3.3 (build 8294) ... $ after_install/exclude_list_test.sh ❌ - Exclusion list not placed in the right dir. ◦ ◦ ◦ ubuntu:16_04 centos:6 centos:7 Recap: centos:6 &
12. 12 • Decoupled – (not just for the ownCloud-client) • Lightweight: ' 5-10’ successful test runs • Scalable – Parallel runs isolated – Include new distros in the pool in a breeze (literally) • Desktop managers ≠ lightweight • Scenarios should be described as matrix – More Abstraction - e.g. .travis.yml Pros / Cons images: - ubuntu:16.10 - ubuntu:17.04 - centos:6 - centos:7 [...] env: - XFCE - UNITY - KDE_PLASMA
13. 13 • Multistage pipeline: – Install & Update over frozen states (images) – Distribution Upgrade • Extend the test suite: – More post-install assertions – Bootstrapping the client config. – Translations – Branded Client – behaviors/configurations Moving Forward Debian 9 Debian 8 Debian 7 openSUSE 42.3 2.3.0 2.3.1 2.3.2 2.3.3 2.4.0 Update Path
14. GUI Testing the ownCloud client
15. 15 Why? • /me not a fan of UI testing • Also /me needed some kind of fixtures for fast bug reproduction – Gherkin language / Easily business-translatable • Coupled interfaces + owncloudcmd ≠ the full picture Meet the Socket API • Basically I needed an interface to the running ownCloud process • Luckily, such tool is already built-in the client • Overlays: pipe info from the sync protocol state to the OS Shell Integration ( ( ( ownCloud client
16. 16 Experimental Implementation • github.com/owncloud/client/pull/5682 – Build a client w/ -DGUI_TESTING=ON • gitlab.com/alfageme/client-GUI-tests – xvfb + imagemagick to record test results - ppl. ❤ gifs var s = require('net').Socket(); s.connect('/tmp/runtime-root/ownCloud/socket'); s.write(message + '\n'); NodeJS test-harness:
17. 17 Future of UI testing • Reasonable next steps: – Good Interface Design: representative and explanatory names for UI items – Behavior Driven Approach • Headless browsers – People is coming up with all sorts of crazy stuff: e.g. pupeteer • Run QML (Qt) apps in-browser for e2e testing
18. Future of App Distribution
19. 19 APPIMAGE appimage.org SNAP PACKAGES snapcraft.io Universal App Distribution https://central.owncloud.org/t/appimage-for-desktop-client-test-it-now
20. 20 Containers in the Desktop • What?!? Sandboxed apps: blog.jessfraz.com/post/docker-containers-on-the-desktop/ • Get some native feel: • What ‘bout the container-limitations we just talked about? – Remember we also spoke about the coolness of the socket API? – Standalone shell integration process – communicate with it from the container $ docker run –it -d \ -v /tmp/.X11-unix:/tmp/.X11-unix \ # X11 socket -e DISPLAY=unix$DISPLAY \ # display -v $HOME/ownCloud:/root/ownCloud \ -v $HOME/.local/share/data/ownCloud:/root/.local/share/data/ownCloud \ --name owncloud-client \ alfageme/owncloud-client Connect to your X11 server in ssh-forward fashion Use your OS’s xopen() script Share your KDE Wallet Mount your local sync folder
21. Farewell David A. Velasco
22. Thank You owncloud.com