Decorators Demystified Anand Chitipothu 

Decorators 

Decorators @ l o g i n _ r e q u i r e d d e f a c c o u n t _ s e t t i n g s ( ) : . . . 

Decorators @ r o u t e ( " / h e l l o " ) d e f h e l l o ( ) : r e t u r n " h e l l o , w o r l d ! " 

Decorators @ d i s k c a c h e ( " { 0 } / d i s t r i c t s . t s v " ) d e f d o w n l o a d _ d i s t r i c t s ( s t a t e ) : . . . 

What is a decorator? 

Syntactic Sugar! @ m y _ d e c o r a t o r d e f f ( ) : p a s s is equivalent to: d e f f ( ) : p a s s f = m y _ d e c o r a t o r ( f ) 

How does a decorator look like? d e f m y _ d e c o r a t o r ( f u n c ) : . . . r e t u r n n e w _ f u n c It is a function It takes a function as argument And returns a new function back 

Applying multiple decorators @ d e c o r a t o r _ 2 @ d e c o r a t o r _ 1 d e f f ( ) : p a s s is equivalent to: d e f f ( ) : p a s s f = d e c o r a t o r _ 1 ( f ) f = d e c o r a t o r _ 2 ( f ) 

Decorator with arguments @ r o u t e ( " / h e l l o " ) d e f h e l l o ( ) : p a s s is equivalent to: d e c o r = r o u t e ( " / h e l l o " ) d e f h e l l o ( ) : p a s s h e l l o = d e c o r ( h e l l o ) route is not the decorator. It is a function that returns the decorator. 

Functions - Review! 

Functions x = 1 0 d e f s q u a r e ( x ) : r e t u r n x * x > > > s q u a r e ( 4 ) 1 6 > > > s q u a r e < f u n c t i o n s q u a r e a t 0 x 1 0 9 e c 5 1 b 8 > 

Functions Functions can be assigned to variable like any other data types. x = 1 0 d e f s q u a r e ( x ) : r e t u r n x * x y = x f = s q u a r e > > > f ( 4 ) 1 6 > > > f < f u n c t i o n s q u a r e a t 0 x 1 0 9 e c 5 1 b 8 > 

Functions as arguments Functions can be passed as arguments to other functions. > > > d e f f s u m ( f , x , y ) : . . . r e t u r n f ( x ) + f ( y ) . . . > > > f s u m ( s q u a r e , 3 , 4 ) 2 5 > > > f s u m ( l e n , " h e l l o " , " s i n g a p o r e " ) 1 4 In fact, there are many standard library functions that take function arguments. > > > m a x ( [ " a l i c e " , " c h a r l i e " , " e v e " , d a v e " , " b o b " ] ) ' e v e ' > > > m a x ( [ " a l i c e " , " c h a r l i e " , " e v e " , d a v e " , " b o b " ] , k e y = l e n ) ' c h a r l i e ' 

Functions as return values Functions can even return new functions. d e f m a k e _ a d d e r ( x ) : d e f a d d ( y ) : r e t u r n x + y r e t u r n a d d a d d 5 = m a k e _ a d d e r ( 5 ) p r i n t a d d 5 ( 2 ) Output: 7 

Functions taking variable number of arguments In Python, function can take variable number of arguments. > > > m a x ( 3 , 9 , 4 ) 9 > > > m a x ( 3 , 9 , 4 , 2 3 ) 2 3 And here is how they are written. d e f l o g ( l a b e l , * a r g s ) : f o r a i n a r g s : p r i n t l a b e l , a l o g ( " I N F O " , 1 , 2 , 3 ) Output: I N F O 1 I N F O 2 I N F O 3 

Functions taking variable number of arguments How about the reverse? > > > a r g s = [ 1 , 2 ] > > > l o g ( " I N F O " , * a r g s ) I N F O 1 I N F O 2 > > > a r g s = [ ' D E B U G ' , 1 , 2 ] > > > l o g ( * a r g s ) D E B U G 1 D E B U G 2 This can be used for any function. > > > a r g s = [ ' f f ' , 1 6 ] > > > i n t ( * a r g s ) 2 5 5 > > > i n t ( ' f f ' , 1 6 ) 2 5 5 

Function Metadata > > > f r o m t i m e i m p o r t a s c t i m e > > > > > > a s c t i m e . _ _ n a m e _ _ ' a s c t i m e ' > > > > > > a s c t i m e . _ _ m o d u l e _ _ ' t i m e ' > > > > > > p r i n t a s c t i m e . _ _ d o c _ _ a s c t i m e ( [ t u p l e ] ) - > s t r i n g C o n v e r t a t i m e t u p l e t o a s t r i n g , e . g . ' S a t J u n 0 6 1 6 : 2 6 : 1 1 1 9 9 8 ' . W h e n t h e t i m e t u p l e i s n o t p r e s e n t , c u r r e n t t i m e a s r e t u r n e d b y l o c a l t i m e ( ) i s u s e d . 

Example: Tracing Function Calls 

Tracing Function Calls (1) Lets say we want to trace execution of this code. d e f s q u a r e ( x ) : r e t u r n x * x d e f s u m _ o f _ s q u a r e s ( x , y ) : r e t u r n s q u a r e ( x ) + s q u a r e ( y ) p r i n t s u m _ o f _ s q u a r e ( 3 , 4 ) 

Tracing Function Calls (2) We can do that with print statements. d e f s q u a r e ( x ) : p r i n t " s q u a r e " , x r e t u r n x * x d e f s u m _ o f _ s q u a r e s ( x , y ) : p r i n t " s u m _ o f _ s q u a r e s " , x , y r e t u r n s q u a r e ( x ) + s q u a r e ( y ) p r i n t s u m _ o f _ s q u a r e ( 3 , 4 ) Output: s u m _ o f _ s q u a r e s 3 4 s q u a r e 3 s q u a r e 4 2 5 

Tracing Function Calls (3) But can do that without modifying those functions using a decorator. d e f t r a c e ( f ) : " " " D e c o r a t o r t o t r a c e c a l l s t o a f u n c t i o n . " " " d e f g ( * a r g s ) : p r i n t f . _ _ n a m e _ _ , a r g s v a l u e = f ( * a r g s ) p r i n t " r e t u r n " , v a l u e r e t u r n v a l u e r e t u r n g This also prints the return value along with function name. 

Tracing Function Calls (4) Lets try using it: @ t r a c e d e f s q u a r e ( x ) : r e t u r n x * x @ t r a c e d e f s u m _ o f _ s q u a r e s ( x , y ) : r e t u r n s q u a r e ( x ) + s q u a r e ( y ) p r i n t s u m _ o f _ s q u a r e ( 3 , 4 ) Output: s u m _ o f _ s q u a r e s 3 4 s q u a r e 3 r e t u r n 9 s q u a r e 4 r e t u r n 1 6 r e t u r n 2 5 2 5 

Tracing Function Calls (6) @ t r a c e d e f s q u a r e ( x ) : " " " C o m p u t e s s q u a r e o f a n u m b e r . " " " r e t u r n x * x Lets look at function metadata: > > > s q u a r e < f u n c t i o n g a t 0 x 1 0 4 f d d 2 a 8 > > > > s q u a r e . _ _ n a m e _ _ ' g ' > > > h e l p ( s q u a r e ) H e l p o n f u n c t i o n g i n m o d u l e t r a c e : g ( * a r g s ) What happened to the function metadata? 

Tracing Function Calls (7) The f u n c t o o l s module provides a helpers to fix that. d e f t r a c e ( f ) : " " " D e c o r a t o r t o t r a c e c a l l s t o a f u n c t i o n . " " " @ f u n c t o o l s . w r a p s ( f ) d e f g ( * a r g s ) : p r i n t f . _ _ n a m e _ _ , a r g s v a l u e = f ( * a r g s ) p r i n t " r e t u r n " , v a l u e r e t u r n v a l u e r e t u r n g 

Tracing Function Calls (8) Lets try to pretty print the call log. l e v e l = 0 d e f l o g ( t e x t ) : i n d e n t = ' | ' * l e v e l + ' | - - ' p r i n t i n d e n t + t e x t d e f t r a c e ( f ) : d e f g ( * a r g s ) : g l o b a l l e v e l a r g s t r = ' ( ' + " , " . j o i n ( [ r e p r ( a ) f o r a i n a r g s ] ) + ' ) ' l o g ( f . _ _ n a m e _ _ + a r g s t r ) l e v e l + = 1 v a l u e = f ( * a r g s ) l o g ( " r e t u r n " + r e p r ( v a l u e ) ) l e v e l - = 1 r e t u r n v a l u e r e t u r n g 

Tracing Function Calls (9) Output: | - - s u m _ o f _ s q u a r e s ( 3 , 4 ) | | - - s q u a r e ( 3 ) | | | - - r e t u r n 9 | | - - s q u a r e ( 4 ) | | | - - r e t u r n 1 6 | | - - r e t u r n 2 5 2 5 

Tracing Function Calls (10) Lets try tracing fibonacci function. @ t r a c e d e f f i b ( n ) : i f n = = 0 o r n = = 1 : r e t u r n 1 e l s e : r e t u r n f i b ( n - 1 ) + f i b ( n - 2 ) 

Tracing Function Calls (11) > > > p r i n t f i b ( 4 ) | - - f i b ( 4 ) | | - - f i b ( 3 ) | | | - - f i b ( 2 ) | | | | - - f i b ( 1 ) | | | | | - - r e t u r n 1 | | | | - - f i b ( 0 ) | | | | | - - r e t u r n 1 | | | | - - r e t u r n 2 | | | - - f i b ( 1 ) | | | | - - r e t u r n 1 | | | - - r e t u r n 3 | | - - f i b ( 2 ) | | | - - f i b ( 1 ) | | | | - - r e t u r n 1 | | | - - f i b ( 0 ) | | | | - - r e t u r n 1 | | | - - r e t u r n 2 | | - - r e t u r n 5 5 

Example: Memoize 

Memoize (1) How can we get rid of the redundant computation in our fib function? How about caching the return values? Doing this is very popular in functional programming world and it is called memoize. d e f m e m o i z e ( f ) : # c a c h e t o s t o r e t h e r e t u r n v a l u e s o f t h e f u n c t i o n c a c h e = { } d e f g ( * a r g s ) : i f a r g s n o t i n c a c h e : c a c h e [ a r g s ] = f ( * a r g s ) r e t u r n c a c h e [ a r g s ] r e t u r n g 

Memoize (2) Lets apply memoize to the fib function. @ m e m o i z e @ t r a c e d e f f i b ( n ) : i f n = = 0 o r n = = 1 : r e t u r n 1 e l s e : r e t u r n f i b ( n - 1 ) + f i b ( n - 2 ) The t r a c e is still kept to see the new execution trace after decorating with memoize. 

Memoize (3) > > > f i b ( 4 ) | - - f i b ( 4 ) | | - - f i b ( 3 ) | | | - - f i b ( 2 ) | | | | - - f i b ( 1 ) | | | | | - - r e t u r n 1 | | | | - - f i b ( 0 ) | | | | | - - r e t u r n 1 | | | | - - r e t u r n 2 | | | - - r e t u r n 3 | | - - r e t u r n 5 5 

Example: with_retries 

with_retries (1) Problem: Write a decorator function w i t h _ r e t r i e s that continue to retry for 5 times if there is any exception raised in the function. @ w i t h _ r e t r i e s d e f w g e t ( u r l ) : r e t u r n u r l l i b 2 . u r l o p e n ( u r l ) . r e a d ( ) w g e t ( " h t t p : / / g o o g l e . c o m / n o - s u c h - p a g e " ) Should print: w g e t f a i l e d , r e t r y i n g . . . w g e t f a i l e d , r e t r y i n g . . . w g e t f a i l e d , r e t r y i n g . . . w g e t f a i l e d , r e t r y i n g . . . w g e t f a i l e d , r e t r y i n g . . . T r a c e b a c k ( m o s t r e c e n t c a l l l a s t ) : . . . u r l l i b 2 . H T T P E r r o r : H T T P E r r o r 4 0 4 : N o t F o u n d 

with_retries (2) i m p o r t f u n c t o o l s d e f w i t h _ r e t r i e s ( f ) : @ f u n c t o o l s . w r a p s ( f ) d e f g ( * a r g s ) : f o r i i n r a n g e ( 5 ) : # c a l l t h e o r i g i n a l f u n c t i o n a n d i g n o r e a n y e x c e p t i o n t r y : r e t u r n f ( * a r g s ) e x c e p t : p r i n t f . _ _ n a m e _ _ , " f a i l e d , r e t r y i n g . . . " # c a l l t h e o r i g i n a l f u n c t i o n . # t h i s t i m e , l e t t h e e x c e p t i o n g o o u t . r e t u r n f ( * a r g s ) r e t u r n g 

with_retries (3) How to make the number of retries as an argument to the decorator? @ w i t h _ r e t r i e s ( n u m _ r e t r i e s = 3 ) d e f w g e t ( u r l ) : r e t u r n u r l l i b 2 . u r l o p e n ( u r l ) . r e a d ( ) even better: @ w i t h _ r e t r i e s ( n u m _ r e t r i e s = 3 , d e l a y = 0 . 1 ) d e f w g e t ( u r l ) : r e t u r n u r l l i b 2 . u r l o p e n ( u r l ) . r e a d ( ) 

with_retries (4) Remember: w i t h _ r e t r i e s is not the decorator now. The return value of w i t h _ r e t r i e s is a decorator. i m p o r t f u n c t o o l s i m p o r t t i m e d e f w i t h _ r e t r i e s ( n u m _ r e t r i e s = 5 , d e l a y = 0 ) : d e f d e c o r a t o r ( f ) : @ f u n c t o o l s . w r a p s ( f ) d e f g ( * a r g s ) : f o r i i n r a n g e ( n u m _ r e t r i e s ) : t r y : r e t u r n f ( * a r g s ) e x c e p t : p r i n t " F a i l e d t o d o w n l o a d , r e t r y i n g . . . " t i m e . d e l a y ( d e l a y ) # C a l l t h e o r i g i n a l f u n c t i o n . T h i s t i m e , l e t t h e e x c e p t i o n g o o u t . r e t u r n f ( * a r g s ) r e t u r n g r e t u r n d e c o r a t o r 

Example: A web framework 

fakeweb - the web framework The skeleton of the web framework. " " " F a k e w e b f r a m e w o r k . " " " d e f r o u t e ( p a t h ) : " " " R e t u r n s a d e c o r a t o r t o r e g i s t e r m a p p i n g f r o m a p a t h t o a f u n c t i o n . " " " d e f d e c o r ( f ) : # T O D O : F I X M E p a s s r e t u r n d e c o r d e f r e q u e s t ( p a t h ) : " " " R e t u r n s t h e r e s p o n s e r e t u r n e d b y t h e w e b a p p f o r g i v e n p a t h . " " " r e t u r n " 4 0 4 - N o t F o u n d " d e f r u n ( p o r t = 8 0 8 0 ) : " " " R u n s a w e b s e r v e r t o s e r v e t h e w e b a p p . " " " p r i n t " N o t y e t i m p l e m e n t e d " 

The web application webapp.py: f r o m f a k e w e b i m p o r t r o u t e , r u n @ r o u t e ( " / h e l l o " ) d e f h e l l o ( ) : r e t u r n " H e l l o , w o r l d ! " @ r o u t e ( " / b y e " ) d e f b y e ( ) : r e t u r n " G o o d b y e ! " i f _ _ n a m e _ _ = = " _ _ m a i n _ _ " : r u n ( ) 

The web client client.py: # i m p o r t w e b a p p t o l o a d t h e m a p p i n g i m p o r t w e b a p p f r o m f a k e w e b i m p o r t r e q u e s t i f _ _ n a m e _ _ = = " _ _ m a i n _ _ " : p r i n t r e q u e s t ( " / h e l l o " ) p r i n t r e q u e s t ( " / b y e " ) p r i n t r e q u e s t ( " / n o - s u c h - p a g e " ) Output: 4 0 4 - N o t F o u n d 4 0 4 - N o t F o u n d 4 0 4 - N o t F o u n d 

fakeweb - v0.1 _ r o u t e s = [ ] d e f r o u t e ( p a t h ) : d e f d e c o r ( f ) : _ r o u t e s . a p p e n d ( ( p a t h , f ) ) r e t u r n f r e t u r n d e c o r d e f r e q u e s t ( p a t h ) : f o r _ p a t h , f u n c i n _ r o u t e s : i f _ p a t h = = p a t h : r e t u r n f u n c ( ) r e t u r n " 4 0 4 - N o t F o u n d " 

The Output $ p y t h o n c l i e n t . p y H e l l o , w o r l d ! G o o d b y e ! 4 0 4 - N o t F o u n d 

Making it real d e f w s g i f u n c ( e n v , s t a r t _ r e s p o n s e ) : p a t h = e n v [ ' P A T H _ I N F O ' ] s t a r t _ r e s p o n s e ( ' 2 0 0 O K ' , [ ( " C o n t e n t - t y p e " , " t e x t / p l a i n " ) ] ) r e t u r n r e q u e s t ( p a t h ) d e f r u n ( p o r t = 8 0 8 0 ) : f r o m w s g i r e f . s i m p l e _ s e r v e r i m p o r t m a k e _ s e r v e r s e r v e r = m a k e _ s e r v e r ( " l o c a l h o s t " , p o r t , w s g i f u n c ) p r i n t " h t t p : / / l o c a l h o s t : { } / " . f o r m a t ( p o r t ) s e r v e r . s e r v e _ f o r e v e r ( ) 

Final Steps Lets run the webapp. $ p y t h o n w e b a p p . p y h t t p : / / l o c a l h o s t : 8 0 8 0 / Lets try visiting the URLs. $ c u r l h t t p : / / l o c a l h o s t : 8 0 8 0 / h e l l o H e l l o , w o r l d ! $ c u r l h t t p : / / l o c a l h o s t : 8 0 8 0 / b y e G o o d b y e ! $ c u r l h t t p : / / l o c a l h o s t : 8 0 8 0 / n o - s u c h - p a g e 4 0 4 - N o t F o u n d 

Questions? 

Thank you Anand Chitipothu @anandology (http://twitter.com/anandology) http://anandology.com/ (http://anandology.com/) 

No content