What happens behind execution of an import statement?

Transcript

1. What happens behind execution of an import statement? - Shivashis

   Padhi

2. Who am I? - data/software at Grofers - tinker with

   earth observation data - past GSoC with the PSF - love FOSS. worked on felicette, MSS
3. None

4. xkcd/353

5. Packages in Python

6. Packages ‣ directories? not necessarily ‣ special types of modules

   ‣ the __path__ attribute ‣ What are subpackages?
7. None

8. But how can we import a package from a different

   directory?

9. $PATH, and $PYTHONPATH

10. None

11. Regular and Namespace packages

12. Regular packages

13. Namespace packages

14. Namespace packages - example

15. Types of packages supported by default ‣ built-in modules ‣

    frozen modules ‣ path entries in $PATH, and $PYTHONPATH

16. The import system

17. sys.modules ‣ A dictionary to map module names to modules

    which are ‘already loaded’. ‣ Lives in the global namespace of sys

18. sys.modules

19. importlib ‣ provide the implementation of the import statement ‣

    expose parts of the import system for programmatic usage

20. Common use-cases of importlib ‣ Programmatically loading a module ‣

    Reloading a module in run-time ‣ Base classes for finders and loaders (to be used with import hooks)

21. Finders and loaders

22. Finders ‣ find the module ‣ return a spec object

    ‣ Finder object must have a find_spec method defined.

23. Finders - examples ‣ locate built-in modules - BuiltinImporter ‣

    locate frozen modules - FrozenImporter ‣ import path - imagine $PATH and $PYTHONPATH - PathFinder

24. Finders - function ‣ If a finder is capable of

    handling import, it returns a module spec. ‣ module spec contains module’s metadata i.e path, name, etc. The import machinery uses this spec to load modules later.

25. find_spec

26. importing spam.ham

27. Import hooks and PEP 302

28. sys.path_hooks sys.meta_path

29. an example of sys.meta_path and sys.path_hooks

30. What are import hooks? ‣ Inserting custom finders and loaders

    into the import mechanism ‣ Dependency injection?

31. Loaders After spec object is returned from a finder, loaders

    are used to execute the module and store its reference in sys.modules

32. Tasks of loaders ‣ load from source/byte-code ‣ compile source

    ‣ execute the module and store reference in sys.modules[spec.name]

33. example - loading

34. example - loading

35. example - loading

36. ImportError ModuleNotFoundError When no finder can find a module When

    loading fails for a package/module

37. chronological order

38. Code demonstration of an import hooks’ implementation.

39. What else can you do with import hooks? ‣ full

    control after what happens after import x ‣ importing from ftp, https, authorization checks. ‣ importing from a variety of storage, not just .py/.pyc

40. How have people used import hooks? ‣ https://github.com/kragniz/json-sempai ‣ https://github.com/nvbn/import_from_github_com

    ‣ https://github.com/runsascoded/ur

41. sys.meta_path with virtual environment disabled

42. sys.meta_path with virtual environment enabled Virtual Environment’s finder

43. Thank you! https://shivashis.xyz