Demystifying how imports work in Python

It just so happens python abstracts away the inner workings of import statement beautifully. But are we aware of how python behaves when it encounters an import statement for a module? Better yet what is a module actually? Packages anybody? How would different import styles compare with each other. Any Best practices when importing modules. What about that __init__.py you saw the last time you read someone else's code.

Code Snippets: https://github.com/prodicus/talks/tree/master/accepted/chennaipy/october?ref=speakerdeck

Presented at ChennaiPy, October 2016 meetup

http://tasdikrahman.me/?ref=speakerdeck

Transcript

1. Materials @ h ps://github.com/prodicus/talks Demys fying how imports work in Python Tasdik Rahman (@tasdikrahman) Presented @ ChennaiPy, October'16 meetup
2. Requirements Python 3.4 or newer. No extra 3rd party extensions needed. Coming over for this meetup! 2
3. Modules Any p y t h o n source file would be counted as a module. You import a module to execute and access its classes, func on defini ons, a ributes. > > > i m p o r t o s > > > o s . p a t h . a b s p a t h ( ' . ' ) ' / h o m e / t a s d i k / D r o p b o x / t a l k s / c h e n n a i p y / o c t o b e r / s a m p l e c o d e ' > > > p o s i x p a t h would be the module name where the method a b s p a t h ( ) resides. p o s i x p a t h being the alias name for o s . p a t h in linux systems. 3
4. What happens when you import a module? It being a python script, the statements start ge ng executed from top to bo om of the source file. If there are any tasks in the statements ( eg: a p r i n t ( ) statement ), then they get executed when the module is being imported. # ' s a m p l e c o d e / b a s i c p a c k a g e / ' > > > i m p o r t b a s i c p a c k a g e . b a r i n s i d e b a s i c p a c k a g e / _ _ i n i t _ _ . p y i n s i d e ' b a s i c p a c k a g e / b a r ' > > > 4
5. Packages Used to orgranize larger number of modules in a systema c manner. One way of accessing individual modules is using the i m p o r t f o o . b a r import style. # t y p i c a l p a c k a g e s t r u c t u r i n g $ t r e e b a s i c p a c k a g e / b a s i c p a c k a g e ├ ─ ─ b a r . p y ├ ─ ─ f o o . p y └ ─ ─ _ _ i n i t _ _ . p y 5
6. Why packages? b u m b l e b e e ├ ─ ─ c o n s t a n t s . p y ├ ─ ─ c o r e . p y ├ ─ ─ e x c e p t i o n s . p y ├ ─ ─ h e l p e r s │ ├ ─ ─ _ _ i n i t _ _ . p y │ └ ─ ─ v w o _ h e l p e r s . p y ├ ─ ─ _ _ i n i t _ _ . p y └ ─ ─ v w o ├ ─ ─ _ _ i n i t _ _ . p y └ ─ ─ s m a r t _ c o d e . p y Looks good? 6
7. Different styles for impor ng modules 7
8. from module import foo This essen ally imports the module first then picks up specific parts from the module to be available locally. > > > f r o m b a s i c p a c k a g e i m p o r t f o o i n s i d e b a s i c p a c k a g e / _ _ i n i t _ _ . p y i n s i d e ' b a s i c p a c k a g e / f o o . p y ' w i t h a v a r i a b l e i n i t > > > allows using the parts of the module without giving the full prefix before it. 8
9. from module import * Brings out all the symbols from the module and makes them available in the namespace. > > > f r o m b a s i c p a c k a g e _ a l l i m p o r t * i n s i d e b a s i c p a c k a g e _ a l l / _ _ i n i t _ _ . p y i n s i d e ' b a s i c p a c k a g e _ a l l / f o o . p y ' w i t h a v a r i a b l e i n i t i n s i d e ' b a s i c p a c k a g e _ a l l / b a r . p y ' > > > You can use _ _ a l l _ _ inside your _ _ i n i t _ _ . p y module to import the modules which you need. Generally not a good idea! Namespace collisions can occur. 9
10. Takeaways so far The way you import a module doesn't actually change the working of the module. Difference between i m p o r t f o o . b a r and f r o m f o o i m p o r t b a r ? the difference is subjec ve. Pick one style and be consistent with it. doing a f r o m f o o i m p o r t b a r is more efficient. p y t h o n imports the whole file! period. 10
11. Module names naming modules follow the general variable naming conven on. # B a d c h o i c e s $ t o u c h 2 f o o . p y M y A w e s o m e F o o . p y o s . p y # G o o d c h o i c e s $ t o u c h f o o . p y a _ l a r g e _ m o d u l e _ n a m e . p y Don't use Non‐ASCII characters while doing so. Avoid crea ng module names which conflict with the standard library modules. 11
12. Module lookup If it's not in the python path, it just won't import. > > > p p r i n t ( s y s . p a t h ) [ ' ' , ' / u s r / l i b / p y t h o n 3 5 . z i p ' , . . . ' / u s r / l i b / p y t h o n 3 / d i s t ­ p a c k a g e s ' ] Explicitly bring a module inside your path > > > i m p o r t s y s > > > s y s . p a t h . a p p e n d ( ' / a b s o u l e / p a t h / t o / m o d u l e ' ) 12
13. Modules get imported Only once! 13
14. But I really want to import it again! > > > f r o m i m p o r t l i b i m p o r t r e l o a d > > > r e l o a d ( f o o ) This is generally not recommended! If you do so, zombies will spawn. No really! 14
15. Implicit Rela ve imports $ r o d / f o o . p y b a r . p y _ _ i n i t _ _ . p y So want to have some things from f o o . p y inside b a r . p y ? Nothing uncommon. # p y t h o n 2 # i n s i d e " b a r . p y " i m p o r t f o o Don't do it! Works in p y t h o n 2 but doesn't work in p y t h o n 3 15
16. How do I fix it? 16
17. Absolute rela ve imports One way to fix it would be using the name of it's top level package name r e l a t i v e i m p o r t s. # r e l a t i v e i m p o r t s / f o o . p y f r o m r e l a t i v e i m p o r t s i m p o r t b a r This works, but is bri le! What if you wanted to change the name of the top level package? Errors!!!! 17
18. Explicit rela ve imports A be er way would be to # e x p l i c i t i m p o r t s / b a r . p y f r o m . i m p o r t f o o Works even when you rename the root level package for whatever the reason may be (eg: you renamed it to e x p l i c i t i m p o r t s _ v 1 _ 0 ) $ m v e x p l i c i t i m p o r t s / n e w i m p o r t s / 18
19. The leading (.) would be used to move up a directory. # l o o k f o r f o o . p y i n t h e s a m e l e v e l f r o m . i m p o r t f o o # g o a d i r u p a n d i m p o r t f o o . p y f r o m . . i m p o r t f o o # g o a d i r u p a n d e n t e r p l i n o / a n d l o o k f o r b a r . p y f r o m . . p l i n o i m p o r t b a r 19
20. _ _ i n i t _ _ . p y 20
21. What should you put into it? Most of the me, it's empty! S ching together submodules: # m i n i o n s / f o o . p y c l a s s F o o ( o b j e c t ) : p a s s # m i n i o n s / b a r . p y c l a s s B a r ( o b j e c t ) : p a s s # m i n i o n s / _ _ i n i t _ _ . p y f r o m . f o o i m p o r t F o o f r o m . b a r i m p o r t B a r 21
22. Advantage? Headache free imports for small modules > > > i m p o r t m i n i o n s i n s i d e m i n i o n s / _ _ i n i t _ _ . p y i n s i d e ' m i n i o n s / f o o . p y ' w i t h a v a r i a b l e i n i t i n s i d e ' m i n i o n s / b a r . p y ' > > > a = m i n i o n s . F o o ( ) > > > b = m i n i o n s . B a r ( ) controlling import behaviour of f r o m f o o i m p o r t * using the _ _ a l l _ _ variable inside _ _ i n i t _ _ . p y 22
23. Performance anybody? Should I put the whole python package inside the _ _ i n i t _ _ . p y? Yes. If it's small! Not a good idea if you have a very large project! 23
24. References h ps://docs.python.org/3/tutorial/modules.html h ps://docs.python.org/3/reference/import.html h ps://docs.python.org/3/reference/execu onm odel.html h ps://docs.python.org/3/library/distribu on.ht ml 24
25. Ques ons? Would be happy to answer them! h p:/ /tasdikrahman.me/ Twi er (@tasdikrahman) Github (@prodicus) 25