Django Testing @ PyLadies Helsinki 2014-12-16

Unit testing and Django. Talk for the PyLadies Helsinki 16th Dec 2014 meetup at the Anders Innovations office in Helsinki.

Talk and slides by Jason Robinson and Charlie Hornsby, original slides with reveal.js at http://jaywink.github.io/djangogirls/slides/

Transcript

1. Django Testing Jason Robinson & Charlie Hornsby Anders Innovations Oy PyLadies Helsinki 16th Dec 2014
2. We do e-commerce. Join us! Open source e-commerce platform - coming in 2015!
3. Why write tests? "Isn't this the job of the QA engineer?"
4. Tests are not for QA people only Different types of testing Unit tests Integration tests Acceptance tests Disclaimer: Testing is a large subject - this is only one view into it.
5. Unit tests Test individual units of source code, like single methods, views or components. Benefits 1. Find problems early 2. Facilitates change 3. Documentation 4. Design (source: ) http://en.wikipedia.org/wiki/Unit_testing#Benefits Written by developers.
6. Integration tests Groups together several project components according to a test plan. Benefits 1. Find problems with different system components working together. 2. Find breakage with business logic across system Written by developers or QA team.
7. Acceptance tests Ensure system meets requirements via set of test cases. Crosses components and to validate whole user stories from a purely user centric view, without manipulating the system below. Benefits 1. Ensure business logic is filled via user stories 2. Ensure requirements are fulfilled Written by QA team. "Developers should not test their own code" -applies here.
8. So which one are we talking about here? Testing in relating to what a developer does when writing code, we are talking primarily about unit testing. The Django and unittest framework can be used for integration testing too. For acceptance testing, other tools are available, for example browser automation via Selenium IDE, which works purely in the user level.
9. Let's start with two simple functions d e f p a r s e _ i n t e g e r ( s t r i n g _ i n ) : " " " R e t u r n a n i n t e g e r f r o m a s t r i n g o r e l s e N o n e . " " " t r y : r e t u r n i n t ( s t r i n g _ i n ) e x c e p t V a l u e E r r o r : r e t u r n N o n e d e f w i l l _ t h e _ s n o w _ m e l t _ t o d a y ( t e m p e r a t u r e _ s t r i n g ) : " " " T e l l t h e u s e r w h e t h e r t h e s n o w w i l l m e l t t o d a y . " " " t e m p e r a t u r e _ i n t e g e r = p a r s e _ i n t e g e r ( t e m p e r a t u r e _ s t r i n g ) i f t e m p e r a t u r e _ i n t e g e r i s N o n e : r e t u r n " I d o n ' t k n o w w h a t t h a t m e a n s ! " i f t e m p e r a t u r e _ i n t e g e r > 3 : r e t u r n " I t p r o b a b l y w i l l . " r e t u r n " N o p e ! I t ' s h e r e t o s t a y . "
10. Does it still work? d e f p a r s e _ i n t e g e r ( s t r i n g _ i n ) : " " " R e t u r n a n i n t e g e r f r o m a s t r i n g o r e l s e N o n e . " " " t r y : r e t u r n i n t ( s t r i n g _ i n ) e x c e p t V a l u e E r r o r : r e t u r n F a l s e # W e d e c i d e t h a t w e ' d p r e f e r t o r e t u r n F a l s e h e r e . d e f w i l l _ t h e _ s n o w _ m e l t _ t o d a y ( t e m p e r a t u r e _ s t r i n g ) : " " " T e l l t h e u s e r w h e t h e r t h e s n o w w i l l m e l t t o d a y . " " " t e m p e r a t u r e _ i n t e g e r = p a r s e _ i n t e g e r ( t e m p e r a t u r e _ s t r i n g ) i f n o t t e m p e r a t u r e _ i n t e g e r : # W e " f i x " t h i s l i n e t o c h e c k f o r F a l s e . r e t u r n " I d o n ' t k n o w w h a t t h a t m e a n s ! " i f t e m p e r a t u r e _ i n t e g e r > 3 : r e t u r n " I t p r o b a b l y w i l l . " r e t u r n " N o p e ! I t ' s h e r e t o s t a y . "
11. It looks good at the start > > > w i l l _ t h e _ s n o w _ m e l t _ t o d a y ( ' - 5 ' ) > > > " N o p e ! I t ' s h e r e t o s t a y . " > > > w i l l _ t h e _ s n o w _ m e l t _ t o d a y ( ' 2 0 ' ) > > > " I t p r o b a b l y w i l l . " > > > w i l l _ t h e _ s n o w _ m e l t _ t o d a y ( ' 0 ' ) > > > " I d o n ' t k n o w w h a t t h a t m e a n s ! " ... but what's this?
12. Meet the unittest library i m p o r t u n i t t e s t f r o m w i l l _ t h e _ s n o w _ m e l t i m p o r t w i l l _ i t _ s n o w _ t o d a y c l a s s T e s t W i l l T h e S n o w M e l t ( u n i t t e s t . T e s t C a s e ) : d e f t e s t _ s n o w _ m e l t s _ a b o v e _ t h r e e _ d e g r e e s ( s e l f ) : a n s w e r = w i l l _ i t _ s n o w _ t o d a y ( ' 5 ' ) s e l f . a s s e r t E q u a l ( " I t p r o b a b l y w i l l . " , a n s w e r ) d e f t e s t _ s n o w _ d o e s _ n o t _ m e l t _ b e l o w _ t h r e e _ d e g r e e s ( s e l f ) : a n s w e r = w i l l _ i t _ s n o w _ t o d a y ( ' 0 ' ) s e l f . a s s e r t E q u a l ( " N o p e ! I t ' s h e r e t o s t a y . " , a n s w e r ) d e f t e s t _ n o n _ n u m b e r _ c o n f u s e s _ t h e _ f u n c t i o n ( s e l f ) : a n s w e r = w i l l _ i t _ s n o w _ t o d a y ( ' l u m i ' ) s e l f . a s s e r t E q u a l ( " I d o n ' t k n o w w h a t t h a t m e a n s ! " , a n s w e r )
13. Sample output T r a c e b a c k ( m o s t r e c e n t c a l l l a s t ) : F i l e " f o o . p y " , l i n e 4 4 , i n t e s t _ s n o w _ d o e s _ n o t _ m e l t _ b e l o w _ t h r e e _ d e g r e e s s e l f . a s s e r t E q u a l ( " N o p e ! I t ' s h e r e t o s t a y . " , a n s w e r ) A s s e r t i o n E r r o r : " N o p e ! I t ' s h e r e t o s t a y . " ! = " I d o n ' t k n o w w h a t t h a t m e a n s ! " - N o p e ! I t ' s h e r e t o s t a y . + I d o n ' t k n o w w h a t t h a t m e a n s !
14. Django testing framework Based on Python . Unittest framework Comprehensive documentation . online
15. Django testing tutorial Explains not only why to test, but also walks trough testing in Django using the Django tutorial poll app. https://docs.djangoproject.com/en/1.7/intro/tutorial05/ ... could have just used that tutorial here, but oh well, this is for PyLadies... :)
16. Placing test files Place application specific tests in application folders. Remember, unit tests should be as independant as possible. Try to keep applications reusable by not depending tests on other project applications. Once you have more than a few tests, consider creating a t e s t s folder and clearly separating different types of tests into separate files. For example: b l o g / b l o g / t e s t s / b l o g / t e s t s / _ _ i n i t _ _ . p y b l o g / t e s t s / t e s t _ m o d e l s . p y b l o g / t e s t s / t e s t _ v i e w s . p y b l o g / t e s t s / t e s t _ t e m p l a t e s . p y
17. Running tests p y t h o n m a n a g e . p y t e s t Running a single app p y t h o n m a n a g e . p y t e s t b l o g Running a single test case p y t h o n m a n a g e . p y t e s t b l o g . t e s t s . B l o g P o s t V i e w T e s t C a s e Running a single test in a test case p y t h o n m a n a g e . p y t e s t b l o g . t e s t s . B l o g P o s t V i e w T e s t C a s e . t e s t _ v i e w _ r e n d e r s
18. Django test Client Since we're likely testing a web application, we're likely a lot of the time going to need to make requests to the app. Easy! f r o m d j a n g o . t e s t i m p o r t C l i e n t c l i e n t = C l i e n t ( ) r e s p o n s e = c l i e n t . g e t ( ' / ' ) r e s p o n s e = c l i e n t . p o s t ( ' / p o s t s / e d i t / 1 / ' , { " t e x t " : " T e x t c o n t e n t " } ) The response from client provides everything you need to check what happened, including status code, rendered HTML, context, request, templates used, etc. Note! Client is not browser emulation. You cannot do UI tests with it.
19. Django test classes s t a n d a r d l i b r a r y u n i t t e s t d j a n g o . t e s t L i v e S e r v e r T e s t C a s e T e s t C a s e T r a n s a c t i o n T e s t C a s e S i m p l e T e s t C a s e T e s t C a s e source: Django testing documentation Consider subclassing one of these (probably basic TestCase) and extend it to your own project needs.
20. Django test classes For database functionality in your tests (ie the ORM), you need T r a n s a c t i o n T e s t C a s e or T e s t C a s e . If not, you can stick to S i m p l e T e s t C a s e . T r a n s a c t i o n T e s t C a s e enables testing database rollback and commit functionality. T e s t C a s e wraps each test in a transaction, speeding up resetting database. Additionally T e s t C a s e automatically creates a C l i e n t , automatically loads fixtures and offers extra assertions useful for testing web sites. If in doubt, always use T e s t C a s e .
21. Django test classes For real (or emulated) browser testing - you need the L i v e S e r v e r T e s t C a s e . It spawns a r u n s e r v e r instance for the tests to use, allowing real UI testing using for example , which is a high level framework for browser testing. Splinter
22. Simple but effective f r o m d j a n g o . c o r e . u r l r e s o l v e r s i m p o r t r e v e r s e f r o m d j a n g o . t e s t i m p o r t T e s t C a s e c l a s s S u p e r U s e r T e s t C a s e ( T e s t C a s e ) : d e f t e s t _ s u p e r u s e r _ c a n _ v i e w _ h o m e p a g e ( s e l f ) : r e s p o n s e = s e l f . c l i e n t . g e t ( r e v e r s e ( ' p o s t _ l i s t ' ) ) s e l f . a s s e r t C o n t a i n s ( r e s p o n s e , ' D j a n g o G i r l s B l o g ' )
23. Testing POST requests d e f t e s t _ s u p e r u s e r _ c a n _ a d d _ n e w _ b l o g _ p o s t ( s e l f ) : t i t l e = ' T e s t T i t l e 2 ' t e x t = ' H e r e i s l o t s o f p o s t c o n t e n t . . . b l a h b l a h b l a h . . . ' r e s p o n s e = s e l f . c l i e n t . p o s t ( r e v e r s e ( ' p o s t _ n e w ' ) , { ' t i t l e ' : t i t l e , ' t e x t ' : t e x t , } , f o l l o w = T r u e ) p o s t = P o s t . o b j e c t s . g e t ( t i t l e = t i t l e ) s e l f . a s s e r t R e d i r e c t s ( r e s p o n s e , r e v e r s e ( ' p o s t _ d e t a i l ' , k w a r g s = { ' p k ' : p o s t . p k } ) ) s e l f . a s s e r t C o n t a i n s ( r e s p o n s e , t i t l e )
24. The setUp method helps keep things DRY d e f s e t U p ( s e l f ) : " " " A d d s o m e o b j e c t s t o t h e d a t a b a s e t h a t a l l t e s t s w i l l n e e d . " " " s e l f . u s e r = U s e r . o b j e c t s . c r e a t e _ s u p e r u s e r ( u s e r n a m e = ' a d m i n ' , e m a i l = ' ' , p a s s w o r d = ' a d m i n ' ) s e l f . p o s t = s e l f . _ c r e a t e _ p o s t ( a u t h o r = U s e r . o b j e c t s . g e t ( u s e r n a m e = ' a d m i n ' ) , t i t l e = ' T e s t T i t l e 1 ' , t e x t = ' H e r e i s l o t s o f p o s t c o n t e n t . . . b l a h b l a h b l a h . . . ' ) s e l f . c l i e n t . l o g i n ( u s e r n a m e = ' a d m i n ' , p a s s w o r d = ' a d m i n ' )
25. Testing with a Real Browser is also possible f r o m d j a n g o . c o n t r i b . s t a t i c f i l e s . t e s t i n g i m p o r t S t a t i c L i v e S e r v e r T e s t C a s e c l a s s S e l e n i u m T e s t C a s e ( S t a t i c L i v e S e r v e r T e s t C a s e ) : d e f t e s t _ b r o w s e _ t o _ p a g e ( s e l f ) : # B r o w s e t o t h e l o g i n p a g e s e l f . s e l e n i u m . g e t ( s e l f . b u i l d _ a b s o l u t e _ u r l ( ' / a d m i n / l o g i n / ? n e x t = / ' ) ) # F i l l o u t i n p u t u s e r n a m e _ i n p u t = s e l f . s e l e n i u m . f i n d _ e l e m e n t _ b y _ n a m e ( " u s e r n a m e " ) u s e r n a m e _ i n p u t . s e n d _ k e y s ( ' a d m i n ' ) p a s s w o r d _ i n p u t = s e l f . s e l e n i u m . f i n d _ e l e m e n t _ b y _ n a m e ( " p a s s w o r d " ) p a s s w o r d _ i n p u t . s e n d _ k e y s ( ' s e c r e t p a s s w o r d ' ) # L o g i n s e l f . s e l e n i u m . f i n d _ e l e m e n t _ b y _ x p a t h ( ' / / i n p u t [ @ v a l u e = " L o g i n " ] ' ) . c l i c k ( )
26. TDD example - publish functionality So we're missing blog publishing functionality in our app, which is bad, so we will add it now. We could just write the code for it, but instead we will write a test for it. d e f t e s t _ s u p e r u s e r _ c a n _ p u b l i s h _ b l o g _ p o s t ( s e l f ) : # N o t e : i f t h i s w a s a l i v e s e r v e r t e s t c a s e , w e c o u l d b r o w s e t o t h e # p o s t a n d c l i c k t h e b u t t o n . W h i l e t h a t i s f i n e , t h e p u b l i s h v i e w # c a n b e s e p a r a t e l y a n d i n d i v i d u a l l y t e s t e d b y d i r e c t l y p o s t i n g t o i t . r e s p o n s e = s e l f . c l i e n t . p o s t ( r e v e r s e ( " p o s t _ p u b l i s h " , k w a r g s = { " p k " : s e l f . p o s t # W e c o u l d c h e c k f o r s t a t u s c o d e , b u t ` a s s e r t R e d i r e c t s ` i s b e t t e r s e l f . a s s e r t R e d i r e c t s ( r e s p o n s e , r e v e r s e ( " p o s t _ d e t a i l " , k w a r g s = { " p k " : s e l f . p o s t # N o t e : W e c h e c k h e r e i f t h e p u b l i s h w a s s u c c e s s f u l , e v e n t h o u g h t h a t i s a l r e a d y # h a n d l e d b y t h e m o d e l t e s t . # W h e n t e s t i n g , m o r e c a n b e b e t t e r . p o s t = P o s t . o b j e c t s . g e t ( i d = s e l f . p o s t . p k ) s e l f . a s s e r t I s N o t N o n e ( p o s t . p u b l i s h e d _ d a t e )
27. TDD example - continued Running this test will obviously fail, since the functionality does not exist. $ p y t h o n m a n a g e . p y t e s t b l o g . t e s t s . S u p e r U s e r T e s t C a s e C r e a t i n g t e s t d a t a b a s e f o r a l i a s ' d e f a u l t ' . . . . . E . . = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = E R R O R : t e s t _ s u p e r u s e r _ c a n _ p u b l i s h _ b l o g _ p o s t ( b l o g . t e s t s . S u p e r U s e r T e s t C a s e ) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - T r a c e b a c k ( m o s t r e c e n t c a l l l a s t ) : F i l e " / h o m e / j a s o n / w o r k s p a c e / d j a n g o g i r l s / b l o g / t e s t s . p y " , l i n e 4 9 , i n t e s t _ s u p e r u s e r _ c a n _ r e s p o n s e = s e l f . c l i e n t . p o s t ( r e v e r s e ( " p o s t _ p u b l i s h " , k w a r g s = { " p k " : s e l f . p o s t F i l e " / h o m e / j a s o n / . v i r t u a l e n v s / d j a n g o g i r l s / l i b / p y t h o n 3 . 4 / s i t e - p a c k a g e s / d j a n g o / c o r e / u r l r r e t u r n i r i _ t o _ u r i ( r e s o l v e r . _ r e v e r s e _ w i t h _ p r e f i x ( v i e w , p r e f i x , * a r g s , * * k w a r g s ) ) F i l e " / h o m e / j a s o n / . v i r t u a l e n v s / d j a n g o g i r l s / l i b / p y t h o n 3 . 4 / s i t e - p a c k a g e s / d j a n g o / c o r e / u r l r ( l o o k u p _ v i e w _ s , a r g s , k w a r g s , l e n ( p a t t e r n s ) , p a t t e r n s ) ) d j a n g o . c o r e . u r l r e s o l v e r s . N o R e v e r s e M a t c h : R e v e r s e f o r ' p o s t _ p u b l i s h ' w i t h a r g u m e n t s - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - R a n 5 t e s t s i n 0 . 4 1 6 s
28. TDD example - template test Since we're likely going to add a link in the template for this feature, we should write a test for it too. We know our requirements says the button/link should have the text "Publish" - so we can write a test for it even though we don't know yet how we will implement it - a button, A- tag, etc. d e f t e s t _ s u p e r u s e r _ h a s _ p u b l i s h _ b u t t o n _ i n _ u n p u b l i s h e d _ b l o g _ p o s t ( s e l f ) : r e s p o n s e = s e l f . c l i e n t . g e t ( r e v e r s e ( " p o s t _ d e t a i l " , k w a r g s = { " p k " : s e l f . p o s t . p k } ) ) # # W e c a n s a f e l y c h e c k f o r j u s t t h e w o r d ' P u b l i s h ' , s i n c e w e k n o w t h e # # t e s t p o s t w i l l n o t i n c l u d e i t s e l f . a s s e r t C o n t a i n s ( r e s p o n s e , " P u b l i s h " )
29. TDD example - continued As expected, this test, will also fail. F A I L : t e s t _ s u p e r u s e r _ h a s _ p u b l i s h _ b u t t o n _ i n _ u n p u b l i s h e d _ b l o g _ p o s t ( b l o g . t e s t s . S u p e r U s e r T e s - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - T r a c e b a c k ( m o s t r e c e n t c a l l l a s t ) : F i l e " / h o m e / j a s o n / w o r k s p a c e / d j a n g o g i r l s / b l o g / t e s t s . p y " , l i n e 6 1 , i n t e s t _ s u p e r u s e r _ h a s _ s e l f . a s s e r t C o n t a i n s ( r e s p o n s e , " P u b l i s h " ) F i l e " / h o m e / j a s o n / . v i r t u a l e n v s / d j a n g o g i r l s / l i b / p y t h o n 3 . 4 / s i t e - p a c k a g e s / d j a n g o / t e s t / t e s t m s g _ p r e f i x + " C o u l d n ' t f i n d % s i n r e s p o n s e " % t e x t _ r e p r ) A s s e r t i o n E r r o r : F a l s e i s n o t t r u e : C o u l d n ' t f i n d ' P u b l i s h ' i n r e s p o n s e What about the model . p u b l i s h ( ) functionality? We don't need a test for that, we're already covered there.
30. TDD example - implement! Let's implement our feature, as per the . DjangoGirls extensions book Note, we're skipping the draft page, we will just add the button. We've already placed drafts in the main list..
31. TDD example - run tests again $ p y t h o n m a n a g e . p y t e s t b l o g . t e s t s . S u p e r U s e r T e s t C a s e C r e a t i n g t e s t d a t a b a s e f o r a l i a s ' d e f a u l t ' . . . . . . . . . - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - R a n 6 t e s t s i n 0 . 5 1 0 s O K D e s t r o y i n g t e s t d a t a b a s e f o r a l i a s ' d e f a u l t ' . . . No more failures!
32. TDD example - why? It's tempting to just write the code and add a test later. It is however also a bad thing to do. Once the code is finished, you will be tempted, or forced by deadlines, to just commit the code and move on. Result: no test coverage You will be writing the test to pass which is always worse than writing a test to fail. When writing a test to pass, it is possible the test doesn't even fail if the feature breaks. When writing a test to fail, you will always see it start passing once the feature is implemented. This means it will also break if the feature stops working.
33. TDD example - why? Writing tests before writing code improves design. Creating the test is like writing out the specifications. You will more likely spot problems in for example business logic before wasting time writing code.
34. Use descriptive names for tests Bad: d e f t e s t _ v i e w _ p u b l i s h i n g _ s t u f f ( s e l f ) : Good: d e f t e s t _ s t a f f _ m e m b e r _ c a n _ p u b l i s h _ b l o g _ p o s t ( s e l f ) : Keep tests small - above example should only test if staff member can publish a post, as the name says.
35. Follow TDD when possible TDD (Test Driven Development) basically means write failing test, then code to pass the test. Not always possible (or sensible) when creating features, but always sensible when fixing bugs.
36. Increase test coverage over time Lots of code and no tests getting you down? Don't worry. Instead, when you fix or change something, or write a new feature - write a test for it. Your code coverage will increasy organically. Don't try to reserve time to implement code coverage - that time will never become available!
37. More is better
38. The end ... of the slides. Let's hack on some tests!
39. Some things to hack on Add view count to blog post add to model a count add method to increment count add call to method from view view tests test counter increments when viewing view Add commenting to posts comment model with two fields - name, text add view save comment action add model 'add comment' action tests test template has input fields test view for comment saving functionality test comment saving on model side
40. Some things to hack on pt.2 Add ordering of posts on different criteria criterias: title, view count, published date (default) add to blog list template: ordering, triggers querystring change add to view ordering saving to session and queryset change tests test new session by default orders by published date test calling view with different querystring produces different ordering test ordering sticks via session
41. - - - Follow us or give feedback Jason Robinson GitHub Home page diaspora* Twitter Charlie Hornsby GitHub