Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What happens behind execution of an import statement?

What happens behind execution of an import statement?

Shivashis Padhi

August 16, 2020

More Decks by Shivashis Padhi

Other Decks in Education


  1. Who am I? - data/software at Grofers - tinker with

    earth observation data - past GSoC with the PSF - love FOSS. worked on felicette, MSS
  2. Packages ‣ directories? not necessarily ‣ special types of modules

    ‣ the __path__ attribute ‣ What are subpackages?
  3. Types of packages supported by default ‣ built-in modules ‣

    frozen modules ‣ path entries in $PATH, and $PYTHONPATH
  4. sys.modules ‣ A dictionary to map module names to modules

    which are ‘already loaded’. ‣ Lives in the global namespace of sys
  5. importlib ‣ provide the implementation of the import statement ‣

    expose parts of the import system for programmatic usage
  6. Common use-cases of importlib ‣ Programmatically loading a module ‣

    Reloading a module in run-time ‣ Base classes for finders and loaders (to be used with import hooks)
  7. Finders ‣ find the module ‣ return a spec object

    ‣ Finder object must have a find_spec method defined.
  8. Finders - examples ‣ locate built-in modules - BuiltinImporter ‣

    locate frozen modules - FrozenImporter ‣ import path - imagine $PATH and $PYTHONPATH - PathFinder
  9. Finders - function ‣ If a finder is capable of

    handling import, it returns a module spec. ‣ module spec contains module’s metadata i.e path, name, etc. The import machinery uses this spec to load modules later.
  10. What are import hooks? ‣ Inserting custom finders and loaders

    into the import mechanism ‣ Dependency injection?
  11. Loaders After spec object is returned from a finder, loaders

    are used to execute the module and store its reference in sys.modules
  12. Tasks of loaders ‣ load from source/byte-code ‣ compile source

    ‣ execute the module and store reference in sys.modules[spec.name]
  13. What else can you do with import hooks? ‣ full

    control after what happens after import x ‣ importing from ftp, https, authorization checks. ‣ importing from a variety of storage, not just .py/.pyc