Thomi Richards: Using Autopilot for Functional UI Testing

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Thomi Richards:
Using Autopilot for Functional UI Testing
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
@ Kiwi PyCon 2013 - Saturday, 07 Sep 2013 - Track 1
http://nz.pycon.org/

**Audience level**

Novice

**Description**

Autopilot is a tool designed for writing high-level functional tests for GUI applications. It is written by a team of engineers at Canonical, and is being used to test everything from the Unity desktop shell to desktop applications, to core phone applications for the Ubuntu Touch project.

**Abstract**

This talk will cover:

* What autopilot is, how it's made, and how it works.
* How we use it at Canonical, and what benefits we've seen from using it.
* How you can use it to test your own applications.

**YouTube**

http://www.youtube.com/watch?v=1NrykxuCcM0

Transcript

1. Autopilot Functional Testing Graphical Applications Thomi Richards
2. Who I am Thomi Richards t h o m i r on Twitter, launchpad, IRC, steam, etc...
3. Where I work Quality Engineer at:
4. What I do
5. A Poor Test Strategy
6. A Better Testing Strategy
7. Functional Testing Given: Design Specifications Performance Requirements We need to be able to: Verify that those specifications have been implemented correctly. Automatically When things break, find out why
8. Enter Autopilot Autopilot is: Implemented in Python & C++ Built on top of testtools Currently available for python2 Almost ported to python3 Runs on Ubuntu Desktop & Ubuntu Touch platforms* Supports Qt4,5, Gtk, and nux-based applications*
9. How it works
10. Input Events Supported Events: Keyboard Mouse Touch, Multitouch, Gestures Soon: accelerometer, other phone sensors
11. State Information Retrieving state is a little bit more complicated: UI Widgets are stored in a tree. We convince the UI toolkit to load an autopilot "plugin". This plugin register on DBus, and understands a very simple communication protocol. We retrieve widget state via DBus. We reconstitute that state into a "proxy object". Proxy objects transparently update to match their counterpart in the application under test!
12. A Simple Example: The simplest possible application:
13. And some python code: First, the boring imports: f r o m a u t o p i l o t . t e s t c a s e i m p o r t A u t o p i l o t T e s t C a s e f r o m a u t o p i l o t . m a t c h e r s i m p o r t E v e n t u a l l y f r o m o s . p a t h i m p o r t j o i n , d i r n a m e f r o m t e s t t o o l s . m a t c h e r s i m p o r t E q u a l s
14. Test Class & Launching The App c l a s s S i m p l e T e s t s ( A u t o p i l o t T e s t C a s e ) : d e f s e t U p ( s e l f ) : s u p e r ( S i m p l e T e s t s , s e l f ) . s e t U p ( ) a p p _ p a t h = j o i n ( d i r n a m e ( _ _ f i l e _ _ ) , ' h e l l o w o r l d ' ) s e l f . a p p _ r o o t = s e l f . l a u n c h _ t e s t _ a p p l i c a t i o n ( a p p _ p a t h )
15. The High-level test d e f t e s t _ c l i c k _ b u t t o n _ w o r k s ( s e l f ) : s e l f . f o c u s _ i n p u t _ a r e a ( ) s e l f . k e y b o a r d . t y p e ( " H e l l o K i w i P y C o n 2 0 1 3 ! " ) s e l f . c l i c k _ m a g i c _ b u t t o n ( ) o u t p u t _ a r e a = s e l f . g e t _ o u t p u t _ l a b e l ( ) s e l f . a s s e r t T h a t ( o u t p u t _ a r e a . t e x t , E v e n t u a l l y ( E q u a l s ( " H e l l o K i w i P y C o n 2 0 1 3 ! " ) ) ) s e l f . f o c u s _ i n p u t _ a r e a , s e l f . c l i c k _ m a g i c _ b u t t o n and s e l f . g e t _ o u t p u t _ l a b e l are yet to be defined!
16. The fiddly bits d e f f o c u s _ i n p u t _ a r e a ( s e l f ) : i n p u t _ a r e a = s e l f . a p p _ r o o t . s e l e c t _ s i n g l e ( ' Q L i n e E d i t ' , o b j e c t N a m e = ' n a m e I n p u t ' ) s e l f . m o u s e . c l i c k _ o b j e c t ( i n p u t _ a r e a )
17. Fiddly Bits 2 d e f c l i c k _ m a g i c _ b u t t o n ( s e l f ) : b u t t o n = s e l f . a p p _ r o o t . s e l e c t _ s i n g l e ( ' Q P u s h B u t t o n ' ) s e l f . m o u s e . c l i c k _ o b j e c t ( b u t t o n )
18. Fiddly Bits 3 d e f g e t _ o u t p u t _ l a b e l ( s e l f ) : r e t u r n s e l f . a p p _ r o o t . s e l e c t _ s i n g l e ( o b j e c t N a m e = ' l a b e l ' )
19. All 30 SLOC: f r o m a u t o p i l o t . t e s t c a s e i m p o r t A u t o p i l o t T e s t C a s e f r o m a u t o p i l o t . m a t c h e r s i m p o r t E v e n t u a l l y f r o m o s . p a t h i m p o r t j o i n , d i r n a m e f r o m t e s t t o o l s . m a t c h e r s i m p o r t E q u a l s c l a s s S i m p l e T e s t s ( A u t o p i l o t T e s t C a s e ) : d e f s e t U p ( s e l f ) : s u p e r ( S i m p l e T e s t s , s e l f ) . s e t U p ( ) a p p _ p a t h = j o i n ( d i r n a m e ( _ _ f i l e _ _ ) , ' h e l l o w o r l d ' ) s e l f . a p p _ r o o t = s e l f . l a u n c h _ t e s t _ a p p l i c a t i o n ( a p p _ p a t h ) d e f f o c u s _ i n p u t _ a r e a ( s e l f , ) : i n p u t _ a r e a = s e l f . a p p _ r o o t . s e l e c t _ s i n g l e ( ' Q L i n e E d i t ' , o b j e c t N a m e = ' n a m e I n p u t ' ) s e l f . m o u s e . c l i c k _ o b j e c t ( i n p u t _ a r e a ) d e f c l i c k _ m a g i c _ b u t t o n ( s e l f ) : b u t t o n = s e l f . a p p _ r o o t . s e l e c t _ s i n g l e ( ' Q P u s h B u t t o n ' ) s e l f . m o u s e . c l i c k _ o b j e c t ( b u t t o n ) d e f g e t _ o u t p u t _ l a b e l ( s e l f ) : r e t u r n s e l f . a p p _ r o o t . s e l e c t _ s i n g l e ( o b j e c t N a m e = ' l a b e l ' ) d e f t e s t _ c l i c k _ b u t t o n _ w o r k s ( s e l f ) : s e l f . f o c u s _ i n p u t _ a r e a ( ) s e l f . k e y b o a r d . t y p e ( " H e l l o K i w i P y C o n 2 0 1 3 ! " ) s e l f . c l i c k _ m a g i c _ b u t t o n ( ) o u t p u t _ a r e a = s e l f . g e t _ o u t p u t _ l a b e l ( ) s e l f . a s s e r t T h a t ( o u t p u t _ a r e a . t e x t , E v e n t u a l l y ( E q u a l s ( " H e l l o K i w i P y C o n 2 0 1 3 ! " ) ) )
20. Live Demo! List & run the test: a u t o p i l o t l i s t e x a m p l e s a u t o p i l o t r u n e x a m p l e s . e x 1
21. How We use Autopilot
22. Autopilot is used to gate Ubuntu Touch image releases:
23. Autopilot is used to do non-functional testing too! Power Consumption Memory Usage FPS & Video Memory counters Lots more!
24. Autopilot is used to gate Ubuntu Desktop Components Unity 7 Desktop shell Indicators WebApps Integration Important upstream applications (gedit, nautilus, etc).
25. Integrated into the Ubuntu SDK.
26. More interesting things... All Ubuntu Touch applications ship an autopilot test suite. These can be run like so: a u t o p i l o t r u n < a p p n a m e > a u t o p i l o t r u n u n i t y 8
27. State of Autopilot Today Heavy Development Has proven to be a useful tool for both Mobile and Desktop development. Not going anywhere!
28. Additional Information http://unity.ubuntu.com/autopilot #ubuntu-autopilot on irc.freenode.net autopilot-users@lists.launchpad.net mailing list Questions?