How to Except When You’re Excepting by Esther Nam

How to Except When You’re Excepting by Esther Nam

Afcfefa1f067d10bd021de0cc2e5e806?s=128

PyCon 2013

March 17, 2013
Tweet

Transcript

  1. from The New Book of Etiquette (1925) by Lillian Eichler

  2. How to Except When You’re Excepting Esther Nam PyCon 2013

    Santa Clara, CA Text
  3. •Python developer About Me

  4. About Me •Python developer •Pro since 2011

  5. •Python developer •Pro since 2011 •No CS degree About Me

  6. You’ve Met Me At... •LA Hackathons

  7. •LA Hackathons •PyLadies You’ve Met Me At...

  8. •LA Hackathons •PyLadies •LA Girl Geek Dinners You’ve Met Me

    At...
  9. In this talk

  10. In this talk •What are exceptions?

  11. In this talk •What are exceptions? •Handling exceptions

  12. In this talk •What are exceptions? •Handling exceptions •Exceptions in

    Python
  13. In this talk •What are exceptions? •Handling exceptions •Exceptions in

    Python •Defensive coding
  14. What is an exception?

  15. def  square_this(number):        """  Return  the  number  squared

     """        return  number  *  number  >>> square_this(3) 9
  16. def  square_this(number):        """  Return  the  number  squared

     """        return  number  *  number  >>> square_this(3) 9 >>> square_this(“number”) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in square_this TypeError: can't multiply sequence by non-int of type 'str'
  17. "the function's assumptions about its inputs are violated"

  18. def  square_this(number):        return  number  *  number  

    >>> square_this(3) 9 >>> square_this(“number”) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in square_this TypeError: can't multiply sequence by non-int of type 'str'
  19. def  square_this(number):        return  number  *  number  

    >>> square_this(3) 9 >>> square_this(“number”) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in square_this TypeError: can't multiply sequence by non-int of type 'str'
  20. def  square_this(number):        return  number  *  number  

    >>> square_this(3) 9 >>> square_this(“number”) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in square_this TypeError: can't multiply sequence by non-int of type 'str'
  21. def  make_sandwich(self):        """  Make  a  sandwich  """

           sandwich.add_meat()        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich
  22. def  make_sandwich(self):        """  Make  a  sandwich  """

           sandwich.add_meat()        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  raise  ValueError
  23. def  make_sandwich(self):        """  Make  a  sandwich  """

           sandwich.add_meat()        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  raise  ValueError
  24. >>> sandwich = Sandwich("ham") >>> sandwich.make_sandwich() def  make_sandwich(self):    

       """  Make  a  sandwich  """        sandwich.add_meat()        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  raise  ValueError
  25. >>> sandwich = Sandwich("ham") >>> sandwich.make_sandwich() Traceback (most recent call

    last): […] ValueError def  make_sandwich(self):        """  Make  a  sandwich  """        sandwich.add_meat()        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  raise  ValueError
  26. What is exception handling?

  27. def  make_sandwich(self):        """  Make  a  sandwich  """

           try:                sandwich.add_meat()        except  ValueError:                print  "Meat  is  murder!  Cheese  only"        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich  
  28. def  make_sandwich(self):        """  Make  a  sandwich  """

           try:                sandwich.add_meat()        except  ValueError:                print  "Meat  is  murder!  Cheese  only"        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  sandwich.meat  =  None                raise  ValueError
  29. >>> sandwich.make_sandwich() def  make_sandwich(self):        """  Make  a

     sandwich  """        try:                sandwich.add_meat()        except  ValueError:                print  "Meat  is  murder!  Cheese  only"        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  sandwich.meat  =  None                  raise  ValueError
  30. >>> sandwich.make_sandwich() Meat is murder! Cheese only >>> sandwich.meat >>>

    def  make_sandwich(self):        """  Make  a  sandwich  """        try:                sandwich.add_meat()        except  ValueError:                print  "Meat  is  murder!  Cheese  only"        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  sandwich.meat  =  None                  raise  ValueError
  31. What causes exceptions?

  32. •Programming errors

  33. •Programming errors •Client code errors

  34. •Programming errors •Client code errors •Resource failures

  35. Exiting may be preferable to compounding errors

  36. Why not use error codes?

  37. ERROR  =  -­‐1 def  a():        value  =

     b()                if  value  !=  ERROR:                do_the_next_thing()        else:                barf()  
  38. ERROR  =  -­‐1 def  a():        value  =

     b()                if  value  !=  ERROR:                do_the_next_thing()        else:                barf()   def  b():        bvalue  =  c()                if  bvalue  !=  ERROR:                return  bvalue  +  1        else:                barf()
  39. ERROR  =  -­‐1 def  a():        value  =

     b()                if  value  !=  ERROR:                do_the_next_thing()        else:                barf()   def  b():        bvalue  =  c()                if  bvalue  !=  ERROR:                return  bvalue  +  1        else:                barf() def  c():        return  ERROR
  40. def  a():        try:        

             value  =  b()                do_the_next_thing()        except  ValueError:                barf()   def  b():        bvalue  =  c()              return  bvalue  +  1 def  c():        something_that_causes_ValueError()        #  Oh  no,  a  ValueError  was  raised!        return  cvalue
  41. enum  {        CMDERR_OPTION_UNKNOWN  =  -­‐3,  /*  unknown

     -­‐option  */        CMDERR_OPTION_AMBIGUOUS  =  -­‐2,  /*  ambiguous  -­‐option  */        CMDERR_OPTION_ARG_MISSING  =  -­‐1,  /*  argument  missing  for  -­‐option  */        CMDERR_UNKNOWN,  /*  unknown  command  */        CMDERR_AMBIGUOUS,  /*  ambiguous  command  */        CMDERR_ERRNO,  /*  get  the  error  from  errno  */        CMDERR_NOT_ENOUGH_PARAMS,  /*  not  enough  parameters  given  */        CMDERR_NOT_CONNECTED,  /*  not  connected  to  server  */        CMDERR_NOT_JOINED,  /*  not  joined  to  any  channels  in  this  window  */        CMDERR_CHAN_NOT_FOUND,  /*  channel  not  found  */        CMDERR_CHAN_NOT_SYNCED,  /*  channel  not  fully  synchronized  yet  */        CMDERR_ILLEGAL_PROTO,  /*  requires  different  chat  protocol  than  the  active  server  */        CMDERR_NOT_GOOD_IDEA,  /*  not  good  idea  to  do,  -­‐yes  overrides  this  */        CMDERR_INVALID_TIME,  /*  invalid  time  specification  */        CMDERR_INVALID_CHARSET,  /*  invalid  charset  specification  */        CMDERR_EVAL_MAX_RECURSE,  /*  eval  hit  recursion  limit  */        CMDERR_PROGRAM_NOT_FOUND  /*  program  not  found  */ };
  42. Why handle exceptions?

  43. Error may not be fatal

  44. What kinds of exceptions can I catch?

  45. BaseException +-- SystemExit +-- KeyboardInterrupt +-- GeneratorExit +-- Exception

  46. BaseException +-- SystemExit +-- KeyboardInterrupt +-- GeneratorExit +-- Exception

  47. +-- Exception +-- StopIteration +-- StandardError | +-- BufferError |

    +-- ArithmeticError | | +-- FloatingPointError | | +-- OverflowError | | +-- ZeroDivisionError | +-- AssertionError | +-- AttributeError | +-- EnvironmentError | | +-- IOError | | +-- OSError | | +-- WindowsError (Windows) | | +-- VMSError (VMS) | +-- EOFError | +-- ImportError | +-- LookupError | | +-- IndexError | | +-- KeyError | +-- MemoryError | +-- NameError | | +-- UnboundLocalError | +-- ReferenceError | +-- RuntimeError | | +-- NotImplementedError | +-- SyntaxError | | +-- IndentationError | | +-- TabError | +-- SystemError | +-- TypeError | +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- PendingDeprecationWarning +-- RuntimeWarning +-- SyntaxWarning +-- UserWarning +-- FutureWarning +-- ImportWarning +-- UnicodeWarning +-- BytesWarning http://docs.python.org/2/library/exceptions.html#exception-hierarchy
  48. Who should catch the exception?

  49. Where things break •Application •Framework •System-level

  50. Can I just catch all of them?

  51. Catch all the exceptions? try:        results  =

     run_query() except:        handle_error()  
  52. BaseException +-- SystemExit +-- KeyboardInterrupt +-- GeneratorExit +-- Exception

  53. Catch all the exceptions? try:        results  =

     run_query() except:        handle_error()
  54. Catch all the exceptions? try:        results  =

     run_query() except  Exception:        handle_error()  
  55. try:        results  =  run_query() except  Exception:  

         return_empty_rows()   def  run_query():        connect_to_db()        sort_rows()         def  connect_to_db():        if  connection_is_dead:                raise  DatabaseNotConnectedError
  56. try:        results  =  run_query() except  Exception:  

         handle_error() def  run_query():        connect_to_db()        sort_rows()         def  connect_to_db():        if  connection_is_dead:                raise  DatabaseNotConnectedError
  57. try:        results  =  run_query() except  Exception:  

         handle_error() def  run_query():        connect_to_db()        sort_rows()         def  connect_to_db():        if  connection_is_dead:                raise  DatabaseNotConnectedError
  58. What if there’s more than one possible exception?

  59. try: do_something() do_the_next_thing() except AnException: handle_exception() Catching multiple exceptions

  60. try: do_something() do_the_next_thing() except (ExceptionA, ExceptionB): handle_exception() Catching multiple exceptions

  61. try: do_something() do_the_next_thing() except AnException: handle_exception() except FatalException: just_die() Catching

    multiple exceptions
  62. try: do_some_math() # ZeroDivisionError is a subclass of # ArithmeticError

    except ZeroDivisionError: just_die() except ArithmeticError: try_to_handle_this() Catching granular exceptions
  63. Catching lots of exceptions? try: do_some_math() except OneException: just_die() except

    AnotherException: try_to_handle_this() except YetAnotherException: just_log_this_one() except YetAnotherException: i_dunno_what_to_do_anymore()
  64. try: do_some_math() except OneException: just_die() except AnotherException: try_to_handle_this() except YetAnotherException:

    just_log_this_one() except YetAnotherException: i_dunno_what_to_do_anymore() Catching lots of exceptions? Re-think your inputs
  65. So you’ve caught an exception

  66. try: do_something() except ValueError: pass Handling exceptions

  67. Continue as though nothing went wrong?

  68. Exceptions are always handled

  69. try: do_something() except ValueError: logger.error()

  70. Delay the exception try: open_file() read_the_file() except AnException as exception:

    clean_up()
  71. try: open_file() read_the_file() except AnException as exception: clean_up() logger(“Log: %s””

    % exception) Delay the exception
  72. try: open_file() read_the_file() except AnException as exception: clean_up() logger(“Log: %s””

    % exception) raise Delay the exception
  73. Re-try for attempt in number_of_attempts: connected = False try: connect_to_server()

    connected = True except ConnectionFail: wait_one_second() return connected
  74. try: do_something() do_the_next_thing() except: # Something bad happened, not sure

    what Re-raising an exception
  75. try: do_something() do_the_next_thing() except: # Something bad happened, not sure

    what raise Re-raising an exception
  76. try: do_something() do_the_next_thing() except ValueError: # Something bad happened, not

    sure what raise ValueError Re-raising an exception
  77. try: do_something() do_the_next_thing() except IOError: raise Raising another exception

  78. try: do_something() do_the_next_thing() except MyCustomError: raise ValueError Raising another exception

  79. Why would I create my own exceptions?

  80. Isolate exceptions caused by your application from exceptions caused by

    other code
  81. Custom exceptions class SandwichException(Exception): pass

  82. +-- Exception +-- StopIteration +-- StandardError | +-- BufferError |

    +-- ArithmeticError | | +-- FloatingPointError | | +-- OverflowError | | +-- ZeroDivisionError | +-- AssertionError | +-- AttributeError | +-- EnvironmentError | | +-- IOError | | +-- OSError | | +-- WindowsError (Windows) | | +-- VMSError (VMS) | +-- EOFError | +-- ImportError | +-- LookupError | | +-- IndexError | | +-- KeyError | +-- MemoryError | +-- NameError | | +-- UnboundLocalError | +-- ReferenceError | +-- RuntimeError | | +-- NotImplementedError | +-- SyntaxError | | +-- IndentationError | | +-- TabError | +-- SystemError | +-- TypeError | +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- PendingDeprecationWarning +-- RuntimeWarning +-- SyntaxWarning +-- UserWarning +-- FutureWarning +-- ImportWarning +-- UnicodeWarning +-- BytesWarning http://docs.python.org/2/library/exceptions.html#exception-hierarchy
  83. +-- Exception +-- StopIteration +-- StandardError | +-- BufferError |

    +-- ArithmeticError | | +-- FloatingPointError | | +-- OverflowError | | +-- ZeroDivisionError | +-- AssertionError | +-- AttributeError | +-- EnvironmentError | | +-- IOError | | +-- OSError | | +-- WindowsError (Windows) | | +-- VMSError (VMS) | +-- EOFError | +-- ImportError | +-- LookupError | | +-- IndexError | | +-- KeyError | +-- MemoryError | +-- NameError | | +-- UnboundLocalError | +-- ReferenceError | +-- RuntimeError | | +-- NotImplementedError | +-- SyntaxError | | +-- IndentationError | | +-- TabError | +-- SystemError | +-- TypeError | +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- PendingDeprecationWarning +-- RuntimeWarning +-- SyntaxWarning +-- UserWarning +-- FutureWarning +-- ImportWarning +-- UnicodeWarning +-- BytesWarning http://docs.python.org/2/library/exceptions.html#exception-hierarchy
  84. +-- Exception +-- StopIteration +-- StandardError | +-- BufferError |

    +-- ArithmeticError | | +-- FloatingPointError | | +-- OverflowError | | +-- ZeroDivisionError | +-- AssertionError | +-- AttributeError | +-- EnvironmentError | | +-- IOError | | +-- OSError | | +-- WindowsError (Windows) | | +-- VMSError (VMS) | +-- EOFError | +-- ImportError | +-- LookupError | | +-- IndexError | | +-- KeyError | +-- MemoryError | +-- NameError | | +-- UnboundLocalError | +-- ReferenceError | +-- RuntimeError | | +-- NotImplementedError | +-- SyntaxError | | +-- IndentationError | | +-- TabError | +-- SystemError | +-- TypeError | +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- PendingDeprecationWarning +-- RuntimeWarning +-- SyntaxWarning +-- UserWarning +-- FutureWarning +-- ImportWarning +-- UnicodeWarning +-- BytesWarning http://docs.python.org/2/library/exceptions.html#exception-hierarchy | +-- RuntimeError | | +-- NotImplementedError | +-- SyntaxError | | +-- IndentationError | | +-- TabError | +-- SystemError | +-- TypeError | +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning
  85. exception ValueError •Raised when a built-in operation or function receives

    an argument that has the right type but an inappropriate value, and the situation is not described by a more precise exception such as IndexError.
  86. Custom exceptions class SandwichException(Exception): pass

  87. Custom exceptions class SandwichException(ValueError): pass

  88. Granular custom exceptions class SandwichException(ValueError): pass class MeatException(SandwichException): pass

  89. def make_sandwich(self): try: sandwich.get_bread() sandwich.get_meat() except SandwichException: return_failed_sandwich() def get_bread(self):

    bread = this_raisesValueError() # oh no! that raised a ValueError! return bread
  90. def make_sandwich(self): try: sandwich.get_bread() sandwich.get_meat() except SandwichException: return_failed_sandwich() def get_bread(self):

    bread = this_raisesValueError() # oh no! that raised a ValueError! return bread
  91. Don't expose implementation details

  92. from Bakeries import NewYorkBakery from Bakeries import NewYorkBakeryException def make_sandwich(self):

    try: sandwich.get_bread() sandwich.get_meat() except MeatException: return_cheese_sandwich() except SandwichException: return_failed_sandwich() def get_bread(self): try: bread = NewYorkBakery() except NewYorkBakeryException: raise SandwichException
  93. from Bakeries import CaliforniaBakery from Bakeries import CaliforniaBakeryException def make_sandwich(self):

    try: sandwich.get_bread() sandwich.get_meat() except MeatException: return_cheese_sandwich() except SandwichException: return_failed_sandwich() def get_bread(self): try: bread = CaliforniaBakery() except CaliforniaBakeryException: raise SandwichException
  94. try: file = open(‘file.txt’, ‘r’) except IOError as error: print

    “Can’t open file, screw it” else: # We can do this only if the exception # was not raised. If read() raises an # IOError, we can handle it differently contents = file.read() else
  95. finally try: file = open(‘file.txt’, ‘r’) except IOError as error:

    print “Can’t open file, screw it” finally: file.close()
  96. The context manager: Cleaner code with with

  97. def my_function(): try: do_something() do_the_next_thing() except AnException as exception: handle_that_exception(exception)

    finally: do_this_no_matter_what()
  98. from contextlib import contextmanager @contextmanager def try_this(): try: yield #

    calls functions in try_this() except (ValueError, TypeError): handle_that_exception() finally: clean_up_step() def my_function(): with try_this(): # this is the context do_something() do_something_else()
  99. Look Before You Leap v. Easier to Ask for Forgiveness

    than Permission
  100. None
  101. if foo: do_foo() elif bar: do_bar() elif baz: do_baz() else:

    barf()
  102. •__iter__() raises StopIteration •__getitem__() raises IndexError

  103. Defensive Programming

  104. Expecting the unexpected

  105. known knowns Expecting the unexpected

  106. known knowns known unknowns Expecting the unexpected

  107. known knowns unknown unknowns known unknowns Expecting the unexpected

  108. unknown knowns known knowns unknown unknowns known unknowns Expecting the

    unexpected
  109. Think through your assumptions

  110. Every bug is 2 bugs

  111. Every bug is 2 bugs 1. the bug in the

    code
  112. Every bug is 2 bugs 1. the bug in the

    code 2. the test you didn’t write
  113. How do you know you’ve anticipated necessary exceptions?

  114. Test your error-handling

  115. All exceptions are handled

  116. http://newcars.com/about Thanks

  117. NewCars Tech Team SoCal Python Audrey Roy Danny Greenfeld Chris

    McDonough Doug Napoleone Rachel Sanders Thanks
  118. estherbester on: •Twitter •GitHub •IRC (freenode)

  119. Related topics •Logging •Debugging exceptions •sys.exc_info() •Exception chaining (Python 3)

    •Testing
  120. References http://blog.ianbicking.org/2007/09/12/re-raising-exceptions/ http://codemonkeyism.com/7-good-rules-to-log-exceptions/ http://docs.python.org/2/tutorial/errors.html http://docs.python.org/3.0/whatsnew/3.0.html http://doughellmann.com/2008/05/pymotw-contextlib.html http://en.wikipedia.org/wiki/Defensive_programming http://en.wikipedia.org/wiki/Exception_safety http://jeffknupp.com/blog/2013/02/06/write-cleaner-python-use-exceptions/ http://jessenoller.com/blog/2009/02/03/get-with-the-program-as-contextmanager-completely-

    different http://politechnosis.kataire.com/2008/02/all-exceptions-are-handled.html http://stackoverflow.com/questions/696047/re-raising-exceptions-with-a-different-type-and- message-preserving-existing-inf http://stackoverflow.com/questions/7108193/frequently-repeated-try-except-in-python?rq=1 http://stackoverflow.com/questions/77127/when-to-throw-an-exception http://today.java.net/article/2006/04/04/exception-handling-antipatterns http://www.blog.pythonlibrary.org/2012/08/02/python-101-an-intro-to-logging/ http://www.itmaybeahack.com/homepage/books/nonprog/html/p09_exc_iter/ p09_c01_exception.html#designing-exceptionshttps://en.wikipedia.org/wiki/ There_are_known_knowns#Quoted_from_Persian_literature