Underscores in Python

Underscores don't have meanings (for the most part) in python. But we often give it meaning by convention. In this talk I explore the uses of underscores in Python.

This was presented at Sydney Python for August 2017. It was mostly presented on the terminal, and the code has been copied into a slide for clarity

Transcript

1. _
   @chewxy
   #SyPy August 2017
   Follow @chewxy on TwiIt's mandatory
   

   View Slide

2. _
   def foo():
   return 1, 2.0, "hello", [3,4,5], (6,7)
   _, x, _, y_, _, _ = foo()
   # x = 2.0
   # y = [3,4,5]
   Follow @chewxy on TwiIt's mandatory
   

   View Slide

3. _
   Follow @chewxy on TwiIt's mandatory
   

   View Slide

4. _name
   _x = "internal use only"
   Follow @chewxy on TwiIt's mandatory
   

   View Slide

5. _name
   class Foo:
   def __init__(self, x):
   self._x = x
   >>> f = Foo(1)
   >>> f._x # still accessible!
   1
   Follow @chewxy on TwiIt's mandatory
   

   View Slide

6. _name
   A Gotcha:
   In mypkg.py:
   _x = "internal use only"
   x = "externally accessible variable"
   Follow @chewxy on TwiIt's mandatory
   

   View Slide

7. _name
   A Gotcha:
   In mypkg.py:
   _x = "internal use only"
   x = "externally accessible variable"
   >>> from mypkg import *
   >>> x
   'externally accessible variable'
   Follow @chewxy on TwiIt's mandatory
   

   View Slide

8. _name
   A Gotcha:
   In mypkg.py:
   _x = "internal use only"
   x = "externally accessible variable"
   >>> from mypkg import *
   >>> x
   'externally accessible variable'
   >>> _x
   Traceback (most recent call last):
   File "", line 1, in
   NameError: name '_x' is not defined
   Follow @chewxy on TwiIt's mandatory
   

   View Slide

9. _name
   A Gotcha:
   In mypkg.py:
   _x = "internal use only"
   x = "externally accessible variable"
   >>> from mypkg import *
   >>> x
   'externally accessible variable'
   >>> _x
   Traceback (most recent call last):
   File "", line 1, in
   NameError: name '_x' is not defined
   Avoid this!
   Follow @chewxy on TwiIt's mandatory
   

   View Slide

10. __name
    DANGER ZONE
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

11. __name
    class Foo:
    def __init__(self, x):
    self.__x = x
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

12. __name
    class Foo:
    def __init__(self, x):
    self.__x = x
    >>> f = Foo(1)
    >>> f.__x
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

13. __name
    class Foo:
    def __init__(self, x):
    self.__x = x
    >>> f = Foo(1)
    >>> f.__x
    Traceback (most recent call last):
    File "", line 1, in
    AttributeError: 'Foo' object has no attribute '__x'
    >>>
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

14. __name
    class Foo:
    def __init__(self, x):
    self.__x = x
    >>> f = Foo(1)
    >>> f.__x
    Traceback (most recent call last):
    File "", line 1, in
    AttributeError: 'Foo' object has no attribute '__x'
    >>>
    WTF IS GOING ON??
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

15. __name
    class Foo:
    def __init__(self, x):
    self.__x = x
    >>> f = Foo(1)
    >>> dir(f)
    ['_Foo__x', '__class__', '__dela'__eq__', '__format__', '__ge__', '__geta'__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__',
    '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__seta'__sizeof__', '__str__', '__subclasshook__', '__weakref__']
    >>>
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

16. __name
    class Foo:
    def __init__(self, x):
    self.__x = x
    >>> f = Foo(1)
    >>> dir(f)
    ['_Foo__x', '__class__', '__dela'__eq__', '__format__', '__ge__', '__geta'__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__',
    '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__seta'__sizeof__', '__str__', '__subclasshook__', '__weakref__']
    >>>
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

17. __name
    class Foo:
    def __init__(self, x):
    self.__x = x
    >>> f = Foo(1)
    >>> f._Foo__x
    1
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

18. __name
    class Foo:
    def __init__(self, x):
    self.__x = x
    def get_x(self):
    return self.__x
    >>> f = Foo("SyPy")
    >>> f.get_x()
    'SyPy'
    >>> f.__x
    Traceback (most recent call last):
    File "", line 1, in
    AttributeError: 'Foo' object has no attribute '__x'
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

19. __name
    class Foo:
    def __init__(self, x):
    self.__x = x
    >>> f = Foo(1)
    >>> dir(f)
    ['_Foo__x', '__class__', '__dela'__eq__', '__format__', '__ge__', '__geta'__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__',
    '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__seta'__sizeof__', '__str__', '__subclasshook__', '__weakref__']
    >>>
    Name Mangling
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

20. Why Mangle Names?
    class Foo:
    def __init__(self, x):
    self.__x = x
    class Bar(Foo):
    def __init__(self, x):
    super().__init__(x+"1337")
    self.__x = x
    >>> b = Bar("SyPy")
    >>> dir(b)
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

21. Why Mangle Names?
    class Foo:
    def __init__(self, x):
    self.__x = x
    class Bar(Foo):
    def __init__(self, x):
    super().__init__(x+"1337")
    self.__x = x
    >>> b = Bar("SyPy")
    >>> dir(b)
    ['_Bar__x', '_Foo__x', '__class__', '__delattr__',
    '__dict__', '__dir__', '__doc__', '__eq__',
    '__format__', '__ge__', '__getattribute__',
    '__gt__', '__hash__', '__init__', '__le__',
    '__lt__', '__module__', '__ne__', '__new__',
    '__reduce__', '__reduce_ex__', '__repr__',
    '__setattr__', '__sizeof__', '__str__',
    '__subclasshook__', '__weakref__']
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

22. Why Mangle Names?
    class Foo:
    def __init__(self, x):
    self.__x = x
    class Bar(Foo):
    def __init__(self, x):
    super().__init__(x+"1337")
    self.__x = x
    >>> b = Bar("SyPy")
    >>> dir(b)
    ['_Bar__x', '_Foo__x', '__class__', '__delattr__',
    '__dict__', '__dir__', '__doc__', '__eq__',
    '__format__', '__ge__', '__getattribute__',
    '__gt__', '__hash__', '__init__', '__le__',
    '__lt__', '__module__', '__ne__', '__new__',
    '__reduce__', '__reduce_ex__', '__repr__',
    '__setattr__', '__sizeof__', '__str__',
    '__subclasshook__', '__weakref__']
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

23. Why Mangle Names?
    class Foo:
    def __init__(self, x):
    self.__x = x
    class Bar(Foo):
    def __init__(self, x):
    super().__init__(x+"1337")
    self.__x = x
    >>> b = Bar("SyPy")
    >>> b._Bar__x
    'SyPy'
    >>> b._Foo__x
    'SyPy1337'
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

24. Name Mangling NaughZness
    _Foo__x = 'SyPy still rocks'
    class Foo:
    def get_x(self):
    return __x
    >>> f = Foo()
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

25. Name Mangling NaughZness
    _Foo__x = 'SyPy still rocks'
    class Foo:
    def get_x(self):
    return __x
    >>> f = Foo()
    >>> f.get_x()
    'SyPy still rocks'
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

26. Name Mangling NaughZness
    _Foo__x = 'SyPy still rocks'
    class Foo:
    def get_x(self):
    return __x
    >>> f = Foo()
    >>> f.get_x()
    'SyPy still rocks'
    Python will mangle
    any name that
    starts with __
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

27. __name__
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

28. __name__
    class Foo:
    def __init__(self, x):
    self.x = x
    def __add__(self, other):
    return self.x + other
    >>> f = Foo(1)
    >>> f + 1
    2
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

29. AutomaZc DifferenZaZon
    class Value(object):
    def __init__(self, v, d):
    self.v = v
    self.d = d
    def __mul__(self, other):
    other = promote(other)
    return Value(self.v * other.v, self.d*other.v +
    self.v*other.d)
    def __rmul__(self, other): return Constant(other) * self
    def __repr__(self):
    return repr(self.v)
    def promote(value):
    return value if isinstance(value, Value) else Value(value,1)
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

30. AutomaZc DifferenZaZon
    For context:
    d(x2)/dx = 2x
    >>> x = promote(3)
    >>> y = x * x
    >>> y
    9
    >>> y.d
    6
    With a bit more jiggering you get full proper automaZc
    parZal derivaZves
    Follow @chewxy on TwiIt's mandatory
    

    View Slide

31. THE END
    Follow @chewxy on TwiIt's mandatory
    

    View Slide