Namespaces in Python by Eric Snow

Afcfefa1f067d10bd021de0cc2e5e806?s=47 PyCon 2013
March 18, 2013

Namespaces in Python by Eric Snow

Afcfefa1f067d10bd021de0cc2e5e806?s=128

PyCon 2013

March 18, 2013
Tweet

Transcript

  1. 1.
  2. 4.

    "Namespaces are one honking great idea -- let's do more

    of those!" - Tim Peters, The Zen of Python
  3. 5.
  4. 7.

    A concept: a set of independently identifiable expectations + a

    unique (within set) label for each member
  5. 8.

    A concept: a set of independently identifiable expectations + a

    unique (within set) label for each member context + name => value
  6. 12.

    Namespaces in C? PyObject Py_TYPE PyDict_SetItem PyErr_SetString Py_None Py .

    Object Py . TYPE Py . Dict . SetItem Py . Err . SetString Py . None
  7. 18.

    A Type for sys.implementation • Python implementation specifics: PEP 421

    • the type must communicate the nature of the namespace
  8. 19.

    A Type for sys.implementation • Python implementation specifics: PEP 421

    • the type must communicate the nature of the namespace • dict too soft
  9. 20.

    A Type for sys.implementation • Python implementation specifics: PEP 421

    • the type must communicate the nature of the namespace • dict too soft • namedtuple too hard
  10. 21.

    A Type for sys.implementation • Python implementation specifics: PEP 421

    • the type must communicate the nature of the namespace • dict too soft • namedtuple too hard • object just right...
  11. 22.

    A Type for sys.implementation • Python implementation specifics: PEP 421

    • the type must communicate the nature of the namespace • dict too soft • namedtuple too hard • object just right... • types.SimpleNamespace
  12. 28.

    Some available types: • dict • collections.defaultdict • collections.OrderedDict •

    configparser.ConfigParser • shelve.Shelf • types.MappingProxyType • weakref.WeakKeyDictionary • weakref.WeakValueDictionary
  13. 29.

    How they work: obj[key] obj[key] = value del obj[key] key

    in obj obj.__getitem__(key) obj.__setitem__(key, val) obj.__delitem__(key) obj.__contains__(key)
  14. 30.

    How they work: obj[key] obj[key] = value del obj[key] key

    in obj collections.abc.(Mutable)Mapping obj.__getitem__(key) obj.__setitem__(key, val) obj.__delitem__(key) obj.__contains__(key)
  15. 31.

    class ReadOnlyProxy(Mapping): def __init__(self, mapping): self._mapping = mapping def __len__(self):

    return len(self._mapping) def __iter__(self): return iter(self._mapping) def __getitem__(self, key): return self._mapping[key]
  16. 32.

    Gems: • dict(mapping, **other_mapping) • d.update(mapping, **other_mapping) • dict.from_keys() •

    dict.setdefault() • str.formatmap() • operator.itemgetter() • threading.local
  17. 34.

    property-based containers (static namespaces) • a proper name aliases an

    underlying key • the key is associated with a value • the names and values are fixed in place (i.e. read-only)
  18. 35.

    Example: >>> sys.version_info sys.version_info(major=3, minor=4, micro=0, releaselevel='alpha', serial=0) >>> os.times()

    posix.times_result(user=0.09, system=0.04, children_user=0.0, children_system=0.0, elapsed=17248877.17)
  19. 37.

    namedtuple: context + names Spam = namedtuple('Spam', "a b c")

    class Spam(tuple): def __new__(cls, a, b, c): return super().__new__(cls, a, b, c) @property def a(self): return self[0] ...
  20. 38.

    class ReadOnlyProxy: def __init__(self, obj): self._obj = obj @property def

    spam(self): return self._obj.spam @property def eggs(self): return self._obj.eggs
  21. 41.
  22. 42.
  23. 43.

    How __getattribute__() works: 1. use a "data" descriptor, if there

    is one 2. pull from the instance, if it has __dict__ 3. use a "non-data" descriptor, if there is one 4. use __getattr__(), if there is one 5. raise AttributeError
  24. 48.

    How __getattribute__() works: 1. use a "data" descriptor, if there

    is one 2. pull from the instance, if it has __dict__ 3. use a "non-data" descriptor, if there is one 4. use __getattr__(), if there is one 5. raise AttributeError
  25. 49.

    How __getattribute__() works: 1. use a "data" descriptor, if there

    is one 2. pull from the instance, if it has __dict__ 3. use a "non-data" descriptor, if there is one 4. use __getattr__(), if there is one 5. raise AttributeError MRO!
  26. 50.

    How __getattribute__() works: 1. use a "data" descriptor, if there

    is one 2. pull from the instance, if it has __dict__ 3. use a "non-data" descriptor, if there is one 4. use __getattr__(), if there is one 5. raise AttributeError Method Resolution Order!
  27. 52.

    Some available types: • object • types.SimpleNamespace • argparse.Namespace •

    multiprocessing.Manager.Namespace • xmlrpc.client.ServerProxy • unittest.Mock
  28. 55.
  29. 56.
  30. 57.

    Objects • are namespace containers • have namespaces (mostly) •

    ergo, as container, proxies a namespace >>> ns = SimpleNamespace() >>> ns_ns = vars(ns) >>> ns.__dict__ is ns_ns True
  31. 58.

    Objects • are namespace containers • have namespaces (mostly) •

    ergo, as container, proxies a namespace • base object type has slots • assignment/definition: "bind" object to namespace
  32. 59.

    Object access helpers: • getattr() • setattr() • delattr() •

    hasattr() wrapper around getattr() • dir() calls obj.__dir__() • vars() exposes namespace* • inspect.getattr_static() • inspect.getmembers()
  33. 61.

    __dict__ • most builtins don't have one • key exception:

    function objects • module namespaces exposed as __dict__
  34. 62.

    __dict__ • most builtins don't have one • key exception:

    function objects • module namespaces exposed as __dict__ • read-only proxy of class namespaces exposed as __dict__ • use metaclass __prepare__() to customize the namespace used during class definition
  35. 64.
  36. 65.

    Name lookup: • explicit lookup happens relative to containers •

    Method Resolution Order (MRO) • lookup on class vs. on objects • special method lookup
  37. 66.

    Name lookup: • explicit lookup happens relative to containers •

    Method Resolution Order (MRO) • lookup on class vs. on objects • special method lookup • implicit lookup happens relative to execution namespaces...
  38. 68.

    Scope (a.k.a. implicit lookup): 1. locals • function execution namespace

    • use vars() or locals() 2. non-locals (closures) 3. globals • module execution namespace • use vars() or globals() 4. builtins • interpreter execution namespace • use builtins or __builtins__
  39. 72.

    sys.modules >>> import mordor Traceback (most recent call last): File

    ".../mordor.py", line 14, in <module> raise ImportError(boromir_says) ImportError: One does not simply import Mordor.
  40. 75.
  41. 76.

    APIs: mapping-based • see collections.abc.Mapping property-based • use properties +

    "private" attributes attribute-based • __getattribute__() and __getattr__() • __setattr__() • __delattr__()
  42. 78.

    Helpers: • globals() • locals() • vars() • dir() •

    getattr() • setattr() • delattr() • hasattr() • inspect.getattr_static() • id()
  43. 79.

    • namespaces: context + name => value • containers: ◦

    mapping-based (volatile) ◦ property-based (static) ◦ attribute-based (stable) • pervasive in Python • rich APIs • rich tools for objects