Robot framework

Pycon Otto talk on robot framework and configuring with multiple clouds like Google Cloud Platform, Openstack, Vcenter, Azure

Transcript

1. Robot framework Test code coverage for cloud services Increasing velocity without decreasing the quality. By: Khushbu Parakh @khushbuparakh
2. Who am I? Worked with AVI Networks. GSOc mentor Anita Borg Organization <AUT> 2017 Coach for Django girls pycon UK
3. What is Robot framework? Robot Framework is implemented with Python and other tools To install: pip install robotframework
4. What can you do with Robot Framework? • Web GUI test • Database test • API test • File system test • Mobile Web & App tests • ...and more!
5. 4 Important tables in Robot *** Settings *** *** Variables *** *** Test Cases *** *** Keywords ***
6. Tabular test data syntax keyword-driven testing approach *** Settings *** Suite Setup setup ${deploy} ${config_file} ... Suite Teardown cleanup Library ../../lib/api/avi_lib.py Library Collections Message
7. Tabular test data syntax *** Variables *** *** Variables *** ${config_file} VSconfig.conf ${personalize} 1 ${deploy} 1 ${HTTP_SERVICE1_PORT} 9000 ${HTTP_SERVICE2_PORT} 10000 ${HTTP_SERVICE1_PORT_UPDATE} 9100
8. The hidden power of robot framework! developers and testers working side-by-side at the same time in an agile environment.
9. But first... A field trip to GOOGLE CLOUD
10. gCloud compute instance create $ gcloud compute instances create example-instance-1 \ example-instance-2 example-instance-3 --zone us-central1-a $ gcloud compute instances create pycon-test-controller --boot-disk-size 80 --can-ip-forward --image-family ubuntu-1404-lts --image-project ubuntu-os-cloud --machine-type n1-standard-4 --subnet subnet-name —scopes compute-rw --tags http-server,https-server https://cloud.google.com/sdk/gcloud/reference/compute/instances/create
11. Files needed for tests • Cloud Configuration File - gcloud.conf • Test cases / Test suite - VStest.txt • Test Configuration File - VSconfig.conf • Backend definitions file - vm.py
12. Objects • Configuration Objects ◦ Tenant, ◦ VirtualServer, ◦ Pool, Server, ◦ Network Profile, Application Profile etc.
13. Objects and LibrAries • Use the python protobuf objects where possible. Extend the protobuf object where more specialized functionality needs to be added that is exposed as keywords for test cases. Eventually, this will translate to a REST API. • Capability to provide tenant scoped objects and organized easily to switch between tenants to write test cases.
14. pycon-client us-central1-a g1-small 10.146.1.13 pycon-se1 us-central1-a n1-standard-1 10.146.1.12 pycon-se2 us-central1-a n1-standard-1 10.146.1.11 pycon-server us-central1-a g1-small 10.146.1.14 pycon-test-controller us-central1-a n1-standard-4 10.146.1.10 gcloud.conf Cloud Configuration File
15. Files needed for tests • Cloud Configuration File - gcloud.conf • Test Case / Test Suite File - VStest.txt • Test Configuration File - VSconfig.conf • Backend definitions file - vm.py • Teardown file - z_cleanup.txt
16. Test suite structure • Tests in a suite should be related to each other. ◦ Common setup and/or teardown is often a good indicator. • Should not have too many tests (max 10) in one file unless they are data-driven tests. • Tests should be independent. Initialization using setup/teardown. • Sometimes dependencies between tests cannot be avoided. ◦ For example, it can take too much time to initialize all tests separately. ◦ Never have long chains of dependent tests. ◦ Consider verifying the status of the previous test using the built-in ${PREV TEST STATUS} variable.
17. Test suite architecture TestRoot |- libs |- mylib.py(vm.py) |- resource |- testdata (Raw data used by test files) |- testinputfiles (Files(XML/JSON) that tell actual SUT what to do and where to find test data) |- Features |-F1 |- myfeature1file_1.XML |-F2 |-- myfeature2file_1.XML |- QA |- mytestfile_1.XML |- mytestfile_2.JSON |- testoutput (Output of test) |- tests |- Features |- features.conf(vsconfig.conf) |- QA |- test1.txt(vsconfig.txt) |- test2.txt
18. VStest.txt Test case file
19. VStest.txt Keywords *** keyword *** verifications all_vs_should_be_assigned check_log_controllers_sanity controller and state db cache checks
20. Files needed for tests • Cloud Configuration File - gcloud.conf • Test cases / Test suite - VStest.txt • Test Configuration File - VSconfig.conf • Backend definitions file - vm.py • Teardown file - z_cleanup.txt
21. VSconfig.conf Test Configuration file
22. Files needed for tests • Cloud Configuration File - gcloud.conf • Test cases / Test suite - VStest.txt • Test Configuration File - VSconfig.conf • Backend definitions file - vm.py • Teardown file - z_cleanup.txt
23. Example of the Tests
24. Files needed for tests • Cloud Configuration File - gcloud.conf • Test cases / Test suite - VStest.txt • Test Configuration File - VSconfig.conf • Backend definitions file - vm.py • Teardown file - z_cleanup.txt
25. Z_cleanup.txt Test setup and teardown
26. Test Logs
27. Examples of test cases Running & Scheduling from Jenkins • How many time each job ran (include daily and nightly in the past week • How many testsuites were executed and how many times • how many time each test case ran over this period, how many times it passed, how many times it failed.. • what’s the pass % and fail % of tests in each run (exclude any run that had all tests failed) • in case of failed cases which ones are failing (is it the same test case that is failing all the time) • Support for Test Negative testing as well.
28. Robot test result in Jenkins run
29. Why testing ?what does it do?. How it helps ? • The entire cloud testing process becomes error-prone as the human factor diminishes.
30. Google cloud Compute engine Instances Instances Automated build Test application Test complete Test application Test complete Your VM/ dev machine
31. Thank you! @khushbuparakh