Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Demystifying how imports work in Python

Demystifying how imports work in Python

It just so happens python abstracts away the inner workings of import statement beautifully. But are we aware of how python behaves when it encounters an import statement for a module? Better yet what is a module actually? Packages anybody? How would different import styles compare with each other. Any Best practices when importing modules. What about that __init__.py you saw the last time you read someone else's code.

Code Snippets: https://github.com/prodicus/talks/tree/master/accepted/chennaipy/october?ref=speakerdeck

Presented at ChennaiPy, October 2016 meetup

http://tasdikrahman.me/?ref=speakerdeck

Tasdik Rahman

October 22, 2016
Tweet

More Decks by Tasdik Rahman

Other Decks in Programming

Transcript

  1. Materials @ h ps://github.com/prodicus/talks
    Demys fying how imports
    work in Python
    Tasdik Rahman (@tasdikrahman)
    Presented @ ChennaiPy, October'16 meetup

    View Slide

  2. Requirements
    Python 3.4 or newer.
    No extra 3rd party extensions needed.
    Coming over for this meetup!
    2

    View Slide

  3. Modules
    Any p
    y
    t
    h
    o
    n source file would be counted as a
    module.
    You import a module to execute and access its
    classes, func on defini ons, a ributes.
    >
    >
    > i
    m
    p
    o
    r
    t o
    s
    >
    >
    > o
    s
    .
    p
    a
    t
    h
    .
    a
    b
    s
    p
    a
    t
    h
    (
    '
    .
    '
    )
    '
    /
    h
    o
    m
    e
    /
    t
    a
    s
    d
    i
    k
    /
    D
    r
    o
    p
    b
    o
    x
    /
    t
    a
    l
    k
    s
    /
    c
    h
    e
    n
    n
    a
    i
    p
    y
    /
    o
    c
    t
    o
    b
    e
    r
    /
    s
    a
    m
    p
    l
    e
    c
    o
    d
    e
    '
    >
    >
    >
    p
    o
    s
    i
    x
    p
    a
    t
    h would be the module name where the
    method a
    b
    s
    p
    a
    t
    h
    (
    ) resides. p
    o
    s
    i
    x
    p
    a
    t
    h being the
    alias name for o
    s
    .
    p
    a
    t
    h in linux systems. 3

    View Slide

  4. What happens when you import a module?
    It being a python script, the statements start
    ge ng executed from top to bo om of the
    source file.
    If there are any tasks in the statements ( eg: a
    p
    r
    i
    n
    t
    (
    ) statement ), then they get executed
    when the module is being imported.
    # '
    s
    a
    m
    p
    l
    e
    c
    o
    d
    e
    /
    b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e
    /
    '
    >
    >
    > i
    m
    p
    o
    r
    t b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e
    .
    b
    a
    r
    i
    n
    s
    i
    d
    e b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e
    /
    _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    i
    n
    s
    i
    d
    e '
    b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e
    /
    b
    a
    r
    '
    >
    >
    >
    4

    View Slide

  5. Packages
    Used to orgranize larger number of modules in a
    systema c manner.
    One way of accessing individual modules is using
    the i
    m
    p
    o
    r
    t f
    o
    o
    .
    b
    a
    r import style.
    # t
    y
    p
    i
    c
    a
    l p
    a
    c
    k
    a
    g
    e s
    t
    r
    u
    c
    t
    u
    r
    i
    n
    g
    $ t
    r
    e
    e b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e
    /
    b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e


    ─ b
    a
    r
    .
    p
    y


    ─ f
    o
    o
    .
    p
    y


    ─ _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    5

    View Slide

  6. Why packages?
    b
    u
    m
    b
    l
    e
    b
    e
    e


    ─ c
    o
    n
    s
    t
    a
    n
    t
    s
    .
    p
    y


    ─ c
    o
    r
    e
    .
    p
    y


    ─ e
    x
    c
    e
    p
    t
    i
    o
    n
    s
    .
    p
    y


    ─ h
    e
    l
    p
    e
    r
    s
    │ ├

    ─ _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    │ └

    ─ v
    w
    o
    _
    h
    e
    l
    p
    e
    r
    s
    .
    p
    y


    ─ _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y


    ─ v
    w
    o


    ─ _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y


    ─ s
    m
    a
    r
    t
    _
    c
    o
    d
    e
    .
    p
    y
    Looks good?
    6

    View Slide

  7. Different styles for impor ng
    modules
    7

    View Slide

  8. from module import foo
    This essen ally imports the module first then
    picks up specific parts from the module to be
    available locally.
    >
    >
    > f
    r
    o
    m b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e i
    m
    p
    o
    r
    t f
    o
    o
    i
    n
    s
    i
    d
    e b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e
    /
    _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    i
    n
    s
    i
    d
    e '
    b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e
    /
    f
    o
    o
    .
    p
    y
    ' w
    i
    t
    h a v
    a
    r
    i
    a
    b
    l
    e i
    n i
    t
    >
    >
    >
    allows using the parts of the module without
    giving the full prefix before it.
    8

    View Slide

  9. from module import *
    Brings out all the symbols from the module and
    makes them available in the namespace.
    >
    >
    > f
    r
    o
    m b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e
    _
    a
    l
    l i
    m
    p
    o
    r
    t *
    i
    n
    s
    i
    d
    e b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e
    _
    a
    l
    l
    /
    _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    i
    n
    s
    i
    d
    e '
    b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e
    _
    a
    l
    l
    /
    f
    o
    o
    .
    p
    y
    ' w
    i
    t
    h a v
    a
    r
    i
    a
    b
    l
    e i
    n i
    t
    i
    n
    s
    i
    d
    e '
    b
    a
    s
    i
    c
    p
    a
    c
    k
    a
    g
    e
    _
    a
    l
    l
    /
    b
    a
    r
    .
    p
    y
    '
    >
    >
    >
    You can use _
    _
    a
    l
    l
    _
    _ inside your _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    module to import the modules which you need.
    Generally not a good idea! Namespace collisions
    can occur.
    9

    View Slide

  10. Takeaways so far
    The way you import a module doesn't actually
    change the working of the module.
    Difference between
    i
    m
    p
    o
    r
    t f
    o
    o
    .
    b
    a
    r and f
    r
    o
    m f
    o
    o i
    m
    p
    o
    r
    t b
    a
    r ?
    the difference is subjec ve. Pick one style and
    be consistent with it.
    doing a f
    r
    o
    m f
    o
    o i
    m
    p
    o
    r
    t b
    a
    r is more efficient.
    p
    y
    t
    h
    o
    n imports the whole file! period.
    10

    View Slide

  11. Module names
    naming modules follow the general variable
    naming conven on.
    # B
    a
    d c
    h
    o
    i
    c
    e
    s
    $ t
    o
    u
    c
    h 2
    f
    o
    o
    .
    p
    y M
    y
    A
    w
    e
    s
    o
    m
    e
    F
    o
    o
    .
    p
    y o
    s
    .
    p
    y
    # G
    o
    o
    d c
    h
    o
    i
    c
    e
    s
    $ t
    o
    u
    c
    h f
    o
    o
    .
    p
    y a
    _
    l
    a
    r
    g
    e
    _
    m
    o
    d
    u
    l
    e
    _
    n
    a
    m
    e
    .
    p
    y
    Don't use Non‐ASCII characters while doing so.
    Avoid crea ng module names which conflict with
    the standard library modules.
    11

    View Slide

  12. Module lookup
    If it's not in the python path, it just won't import.
    >
    >
    > p
    p
    r
    i
    n
    t
    (
    s
    y
    s
    .
    p
    a
    t
    h
    )
    [
    '
    '
    ,
    '
    /
    u
    s
    r
    /
    l
    i
    b
    /
    p
    y
    t
    h
    o
    n
    3
    5
    .
    z
    i
    p
    '
    ,
    .
    .
    .
    '
    /
    u
    s
    r
    /
    l
    i
    b
    /
    p
    y
    t
    h
    o
    n
    3
    /
    d
    i
    s
    t
    ­
    p
    a
    c
    k
    a
    g
    e
    s
    '
    ]
    Explicitly bring a module inside your path
    >
    >
    > i
    m
    p
    o
    r
    t s
    y
    s
    >
    >
    > s
    y
    s
    .
    p
    a
    t
    h
    .
    a
    p
    p
    e
    n
    d
    (
    '
    /
    a
    b
    s
    o
    u
    l
    e
    /
    p
    a
    t
    h
    /
    t
    o
    /
    m
    o
    d
    u
    l
    e
    '
    )
    12

    View Slide

  13. Modules get imported Only once!
    13

    View Slide

  14. But I really want to import it again!
    >
    >
    > f
    r
    o
    m i
    m
    p
    o
    r
    t
    l
    i
    b i
    m
    p
    o
    r
    t r
    e
    l
    o
    a
    d
    >
    >
    > r
    e
    l
    o
    a
    d
    (
    f
    o
    o
    )
    This is generally not recommended!
    If you do so, zombies will spawn.
    No really!
    14

    View Slide

  15. Implicit Rela ve imports
    $ r
    o
    d
    /
    f
    o
    o
    .
    p
    y
    b
    a
    r
    .
    p
    y
    _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    So want to have some things from f
    o
    o
    .
    p
    y inside
    b
    a
    r
    .
    p
    y ? Nothing uncommon.
    # p
    y
    t
    h
    o
    n 2
    # i
    n
    s
    i
    d
    e "
    b
    a
    r
    .
    p
    y
    "
    i
    m
    p
    o
    r
    t f
    o
    o
    Don't do it! Works in p
    y
    t
    h
    o
    n
    2 but doesn't work in
    p
    y
    t
    h
    o
    n
    3 15

    View Slide

  16. How do I fix it?
    16

    View Slide

  17. Absolute rela ve imports
    One way to fix it would be using the name of it's
    top level package name r
    e
    l
    a
    t
    i
    v
    e
    i
    m
    p
    o
    r
    t
    s.
    # r
    e
    l
    a
    t
    i
    v
    e
    i
    m
    p
    o
    r
    t
    s
    /
    f
    o
    o
    .
    p
    y
    f
    r
    o
    m r
    e
    l
    a
    t
    i
    v
    e
    i
    m
    p
    o
    r
    t
    s i
    m
    p
    o
    r
    t b
    a
    r
    This works, but is bri le!
    What if you wanted to change the name of the
    top level package?
    Errors!!!!
    17

    View Slide

  18. Explicit rela ve imports
    A be er way would be to
    # e
    x
    p
    l
    i
    c
    i
    t
    i
    m
    p
    o
    r
    t
    s
    /
    b
    a
    r
    .
    p
    y
    f
    r
    o
    m . i
    m
    p
    o
    r
    t f
    o
    o
    Works even when you rename the root level
    package for whatever the reason may be
    (eg: you renamed it to e
    x
    p
    l
    i
    c
    i
    t
    i
    m
    p
    o
    r
    t
    s
    _
    v
    1
    _
    0 )
    $ m
    v e
    x
    p
    l
    i
    c
    i
    t
    i
    m
    p
    o
    r
    t
    s
    / n
    e
    w
    i
    m
    p
    o
    r
    t
    s
    /
    18

    View Slide

  19. The leading (.) would be used to move up a
    directory.
    # l
    o
    o
    k f
    o
    r f
    o
    o
    .
    p
    y i
    n t
    h
    e s
    a
    m
    e l
    e
    v
    e
    l
    f
    r
    o
    m . i
    m
    p
    o
    r
    t f
    o
    o
    # g
    o a d
    i
    r u
    p a
    n
    d i
    m
    p
    o
    r
    t f
    o
    o
    .
    p
    y
    f
    r
    o
    m .
    . i
    m
    p
    o
    r
    t f
    o
    o
    # g
    o a d
    i
    r u
    p a
    n
    d e
    n
    t
    e
    r p
    l
    i
    n
    o
    / a
    n
    d l
    o
    o
    k f
    o
    r b
    a
    r
    .
    p
    y
    f
    r
    o
    m .
    .
    p
    l
    i
    n
    o i
    m
    p
    o
    r
    t b
    a
    r
    19

    View Slide

  20. _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    20

    View Slide

  21. What should you put into it?
    Most of the me, it's empty!
    S ching together submodules:
    # m
    i
    n
    i
    o
    n
    s
    /
    f
    o
    o
    .
    p
    y
    c
    l
    a
    s
    s F
    o
    o
    (
    o
    b
    j
    e
    c
    t
    )
    :
    p
    a
    s
    s
    # m
    i
    n
    i
    o
    n
    s
    /
    b
    a
    r
    .
    p
    y
    c
    l
    a
    s
    s B
    a
    r
    (
    o
    b
    j
    e
    c
    t
    )
    :
    p
    a
    s
    s
    # m
    i
    n
    i
    o
    n
    s
    /
    _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    f
    r
    o
    m .
    f
    o
    o i
    m
    p
    o
    r
    t F
    o
    o
    f
    r
    o
    m .
    b
    a
    r i
    m
    p
    o
    r
    t B
    a
    r
    21

    View Slide

  22. Advantage?
    Headache free imports for small modules
    >
    >
    > i
    m
    p
    o
    r
    t m
    i
    n
    i
    o
    n
    s
    i
    n
    s
    i
    d
    e m
    i
    n
    i
    o
    n
    s
    /
    _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    i
    n
    s
    i
    d
    e '
    m
    i
    n
    i
    o
    n
    s
    /
    f
    o
    o
    .
    p
    y
    ' w
    i
    t
    h a v
    a
    r
    i
    a
    b
    l
    e i
    n i
    t
    i
    n
    s
    i
    d
    e '
    m
    i
    n
    i
    o
    n
    s
    /
    b
    a
    r
    .
    p
    y
    '
    >
    >
    > a = m
    i
    n
    i
    o
    n
    s
    .
    F
    o
    o
    (
    )
    >
    >
    > b = m
    i
    n
    i
    o
    n
    s
    .
    B
    a
    r
    (
    )
    controlling import behaviour of f
    r
    o
    m f
    o
    o i
    m
    p
    o
    r
    t *
    using the _
    _
    a
    l
    l
    _
    _ variable inside _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y
    22

    View Slide

  23. Performance anybody?
    Should I put the whole python package inside the
    _
    _
    i
    n
    i
    t
    _
    _
    .
    p
    y?
    Yes. If it's small!
    Not a good idea if you have a very large project!
    23

    View Slide

  24. References
    h ps://docs.python.org/3/tutorial/modules.html
    h ps://docs.python.org/3/reference/import.html
    h ps://docs.python.org/3/reference/execu onm
    odel.html
    h ps://docs.python.org/3/library/distribu on.ht
    ml
    24

    View Slide

  25. Ques ons?
    Would be happy to answer them!
    h p:/
    /tasdikrahman.me/
    Twi er (@tasdikrahman)
    Github (@prodicus)
    25

    View Slide