What happens behind execution of an import statement?

Transcript

1. What happens behind execution of an import statement? - Shivashis Padhi
2. Who am I? - data/software at Grofers - tinker with earth observation data - past GSoC with the PSF - love FOSS. worked on felicette, MSS
4. xkcd/353
5. Packages in Python
6. Packages ‣ directories? not necessarily ‣ special types of modules ‣ the __path__ attribute ‣ What are subpackages?
8. But how can we import a package from a different directory?
9. $PATH, and $PYTHONPATH
11. Regular and Namespace packages
12. Regular packages
13. Namespace packages
14. Namespace packages - example
15. Types of packages supported by default ‣ built-in modules ‣ frozen modules ‣ path entries in $PATH, and $PYTHONPATH
16. The import system
17. sys.modules ‣ A dictionary to map module names to modules which are ‘already loaded’. ‣ Lives in the global namespace of sys
18. sys.modules
19. importlib ‣ provide the implementation of the import statement ‣ expose parts of the import system for programmatic usage
20. Common use-cases of importlib ‣ Programmatically loading a module ‣ Reloading a module in run-time ‣ Base classes for finders and loaders (to be used with import hooks)
21. Finders and loaders
22. Finders ‣ find the module ‣ return a spec object ‣ Finder object must have a find_spec method defined.
23. Finders - examples ‣ locate built-in modules - BuiltinImporter ‣ locate frozen modules - FrozenImporter ‣ import path - imagine $PATH and $PYTHONPATH - PathFinder
24. Finders - function ‣ If a finder is capable of handling import, it returns a module spec. ‣ module spec contains module’s metadata i.e path, name, etc. The import machinery uses this spec to load modules later.
25. find_spec
26. importing spam.ham
27. Import hooks and PEP 302
28. sys.path_hooks sys.meta_path
29. an example of sys.meta_path and sys.path_hooks
30. What are import hooks? ‣ Inserting custom finders and loaders into the import mechanism ‣ Dependency injection?
31. Loaders After spec object is returned from a finder, loaders are used to execute the module and store its reference in sys.modules
32. Tasks of loaders ‣ load from source/byte-code ‣ compile source ‣ execute the module and store reference in sys.modules[spec.name]
33. example - loading
34. example - loading
35. example - loading
36. ImportError ModuleNotFoundError When no finder can find a module When loading fails for a package/module
37. chronological order
38. Code demonstration of an import hooks’ implementation.
39. What else can you do with import hooks? ‣ full control after what happens after import x ‣ importing from ftp, https, authorization checks. ‣ importing from a variety of storage, not just .py/.pyc
40. How have people used import hooks? ‣ https://github.com/kragniz/json-sempai ‣ https://github.com/nvbn/import_from_github_com ‣ https://github.com/runsascoded/ur
41. sys.meta_path with virtual environment disabled
42. sys.meta_path with virtual environment enabled Virtual Environment’s finder
43. Thank you! https://shivashis.xyz