How to Except When You’re Excepting by Esther Nam

Transcript

1. from The New Book of Etiquette (1925) by Lillian Eichler
2. How to Except When You’re Excepting Esther Nam PyCon 2013 Santa Clara, CA Text
3. •Python developer About Me
4. About Me •Python developer •Pro since 2011
5. •Python developer •Pro since 2011 •No CS degree About Me
6. You’ve Met Me At... •LA Hackathons
7. •LA Hackathons •PyLadies You’ve Met Me At...
8. •LA Hackathons •PyLadies •LA Girl Geek Dinners You’ve Met Me At...
9. In this talk
10. In this talk •What are exceptions?
11. In this talk •What are exceptions? •Handling exceptions
12. In this talk •What are exceptions? •Handling exceptions •Exceptions in Python
13. In this talk •What are exceptions? •Handling exceptions •Exceptions in Python •Defensive coding
14. What is an exception?
15. def  square_this(number):        """  Return  the  number  squared  """        return  number  *  number  >>> square_this(3) 9
16. def  square_this(number):        """  Return  the  number  squared  """        return  number  *  number  >>> square_this(3) 9 >>> square_this(“number”) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in square_this TypeError: can't multiply sequence by non-int of type 'str'
17. "the function's assumptions about its inputs are violated"
18. def  square_this(number):        return  number  *  number   >>> square_this(3) 9 >>> square_this(“number”) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in square_this TypeError: can't multiply sequence by non-int of type 'str'
19. def  square_this(number):        return  number  *  number   >>> square_this(3) 9 >>> square_this(“number”) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in square_this TypeError: can't multiply sequence by non-int of type 'str'
20. def  square_this(number):        return  number  *  number   >>> square_this(3) 9 >>> square_this(“number”) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in square_this TypeError: can't multiply sequence by non-int of type 'str'
21. def  make_sandwich(self):        """  Make  a  sandwich  """        sandwich.add_meat()        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich
22. def  make_sandwich(self):        """  Make  a  sandwich  """        sandwich.add_meat()        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  raise  ValueError
23. def  make_sandwich(self):        """  Make  a  sandwich  """        sandwich.add_meat()        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  raise  ValueError
24. >>> sandwich = Sandwich("ham") >>> sandwich.make_sandwich() def  make_sandwich(self):        """  Make  a  sandwich  """        sandwich.add_meat()        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  raise  ValueError
25. >>> sandwich = Sandwich("ham") >>> sandwich.make_sandwich() Traceback (most recent call last): […] ValueError def  make_sandwich(self):        """  Make  a  sandwich  """        sandwich.add_meat()        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  raise  ValueError
26. What is exception handling?
27. def  make_sandwich(self):        """  Make  a  sandwich  """        try:                sandwich.add_meat()        except  ValueError:                print  "Meat  is  murder!  Cheese  only"        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich  
28. def  make_sandwich(self):        """  Make  a  sandwich  """        try:                sandwich.add_meat()        except  ValueError:                print  "Meat  is  murder!  Cheese  only"        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  sandwich.meat  =  None                raise  ValueError
29. >>> sandwich.make_sandwich() def  make_sandwich(self):        """  Make  a  sandwich  """        try:                sandwich.add_meat()        except  ValueError:                print  "Meat  is  murder!  Cheese  only"        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  sandwich.meat  =  None                  raise  ValueError
30. >>> sandwich.make_sandwich() Meat is murder! Cheese only >>> sandwich.meat >>> def  make_sandwich(self):        """  Make  a  sandwich  """        try:                sandwich.add_meat()        except  ValueError:                print  "Meat  is  murder!  Cheese  only"        sandwich.add_cheese()        sandwich.add_pickle()        return  sandwich   def  add_meat(self):        """  Slice  meat  """        if  sandwich.meat  in  ['ham',  'beef']:                  sandwich.meat  =  None                  raise  ValueError
31. What causes exceptions?
32. •Programming errors
33. •Programming errors •Client code errors
34. •Programming errors •Client code errors •Resource failures
35. Exiting may be preferable to compounding errors
36. Why not use error codes?
37. ERROR  =  -­‐1 def  a():        value  =  b()                if  value  !=  ERROR:                do_the_next_thing()        else:                barf()  
38. ERROR  =  -­‐1 def  a():        value  =  b()                if  value  !=  ERROR:                do_the_next_thing()        else:                barf()   def  b():        bvalue  =  c()                if  bvalue  !=  ERROR:                return  bvalue  +  1        else:                barf()
39. ERROR  =  -­‐1 def  a():        value  =  b()                if  value  !=  ERROR:                do_the_next_thing()        else:                barf()   def  b():        bvalue  =  c()                if  bvalue  !=  ERROR:                return  bvalue  +  1        else:                barf() def  c():        return  ERROR
40. def  a():        try:                  value  =  b()                do_the_next_thing()        except  ValueError:                barf()   def  b():        bvalue  =  c()              return  bvalue  +  1 def  c():        something_that_causes_ValueError()        #  Oh  no,  a  ValueError  was  raised!        return  cvalue
41. enum  {        CMDERR_OPTION_UNKNOWN  =  -­‐3,  /*  unknown  -­‐option  */        CMDERR_OPTION_AMBIGUOUS  =  -­‐2,  /*  ambiguous  -­‐option  */        CMDERR_OPTION_ARG_MISSING  =  -­‐1,  /*  argument  missing  for  -­‐option  */        CMDERR_UNKNOWN,  /*  unknown  command  */        CMDERR_AMBIGUOUS,  /*  ambiguous  command  */        CMDERR_ERRNO,  /*  get  the  error  from  errno  */        CMDERR_NOT_ENOUGH_PARAMS,  /*  not  enough  parameters  given  */        CMDERR_NOT_CONNECTED,  /*  not  connected  to  server  */        CMDERR_NOT_JOINED,  /*  not  joined  to  any  channels  in  this  window  */        CMDERR_CHAN_NOT_FOUND,  /*  channel  not  found  */        CMDERR_CHAN_NOT_SYNCED,  /*  channel  not  fully  synchronized  yet  */        CMDERR_ILLEGAL_PROTO,  /*  requires  different  chat  protocol  than  the  active  server  */        CMDERR_NOT_GOOD_IDEA,  /*  not  good  idea  to  do,  -­‐yes  overrides  this  */        CMDERR_INVALID_TIME,  /*  invalid  time  specification  */        CMDERR_INVALID_CHARSET,  /*  invalid  charset  specification  */        CMDERR_EVAL_MAX_RECURSE,  /*  eval  hit  recursion  limit  */        CMDERR_PROGRAM_NOT_FOUND  /*  program  not  found  */ };
42. Why handle exceptions?
43. Error may not be fatal
44. What kinds of exceptions can I catch?
45. BaseException +-- SystemExit +-- KeyboardInterrupt +-- GeneratorExit +-- Exception
46. BaseException +-- SystemExit +-- KeyboardInterrupt +-- GeneratorExit +-- Exception
47. +-- Exception +-- StopIteration +-- StandardError | +-- BufferError | +-- ArithmeticError | | +-- FloatingPointError | | +-- OverflowError | | +-- ZeroDivisionError | +-- AssertionError | +-- AttributeError | +-- EnvironmentError | | +-- IOError | | +-- OSError | | +-- WindowsError (Windows) | | +-- VMSError (VMS) | +-- EOFError | +-- ImportError | +-- LookupError | | +-- IndexError | | +-- KeyError | +-- MemoryError | +-- NameError | | +-- UnboundLocalError | +-- ReferenceError | +-- RuntimeError | | +-- NotImplementedError | +-- SyntaxError | | +-- IndentationError | | +-- TabError | +-- SystemError | +-- TypeError | +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- PendingDeprecationWarning +-- RuntimeWarning +-- SyntaxWarning +-- UserWarning +-- FutureWarning +-- ImportWarning +-- UnicodeWarning +-- BytesWarning http://docs.python.org/2/library/exceptions.html#exception-hierarchy
48. Who should catch the exception?
49. Where things break •Application •Framework •System-level
50. Can I just catch all of them?
51. Catch all the exceptions? try:        results  =  run_query() except:        handle_error()  
52. BaseException +-- SystemExit +-- KeyboardInterrupt +-- GeneratorExit +-- Exception
53. Catch all the exceptions? try:        results  =  run_query() except:        handle_error()
54. Catch all the exceptions? try:        results  =  run_query() except  Exception:        handle_error()  
55. try:        results  =  run_query() except  Exception:        return_empty_rows()   def  run_query():        connect_to_db()        sort_rows()         def  connect_to_db():        if  connection_is_dead:                raise  DatabaseNotConnectedError
56. try:        results  =  run_query() except  Exception:        handle_error() def  run_query():        connect_to_db()        sort_rows()         def  connect_to_db():        if  connection_is_dead:                raise  DatabaseNotConnectedError
57. try:        results  =  run_query() except  Exception:        handle_error() def  run_query():        connect_to_db()        sort_rows()         def  connect_to_db():        if  connection_is_dead:                raise  DatabaseNotConnectedError
58. What if there’s more than one possible exception?
59. try: do_something() do_the_next_thing() except AnException: handle_exception() Catching multiple exceptions
60. try: do_something() do_the_next_thing() except (ExceptionA, ExceptionB): handle_exception() Catching multiple exceptions
61. try: do_something() do_the_next_thing() except AnException: handle_exception() except FatalException: just_die() Catching multiple exceptions
62. try: do_some_math() # ZeroDivisionError is a subclass of # ArithmeticError except ZeroDivisionError: just_die() except ArithmeticError: try_to_handle_this() Catching granular exceptions
63. Catching lots of exceptions? try: do_some_math() except OneException: just_die() except AnotherException: try_to_handle_this() except YetAnotherException: just_log_this_one() except YetAnotherException: i_dunno_what_to_do_anymore()
64. try: do_some_math() except OneException: just_die() except AnotherException: try_to_handle_this() except YetAnotherException: just_log_this_one() except YetAnotherException: i_dunno_what_to_do_anymore() Catching lots of exceptions? Re-think your inputs
65. So you’ve caught an exception
66. try: do_something() except ValueError: pass Handling exceptions
67. Continue as though nothing went wrong?
68. Exceptions are always handled
69. try: do_something() except ValueError: logger.error()
70. Delay the exception try: open_file() read_the_file() except AnException as exception: clean_up()
71. try: open_file() read_the_file() except AnException as exception: clean_up() logger(“Log: %s”” % exception) Delay the exception
72. try: open_file() read_the_file() except AnException as exception: clean_up() logger(“Log: %s”” % exception) raise Delay the exception
73. Re-try for attempt in number_of_attempts: connected = False try: connect_to_server() connected = True except ConnectionFail: wait_one_second() return connected
74. try: do_something() do_the_next_thing() except: # Something bad happened, not sure what Re-raising an exception
75. try: do_something() do_the_next_thing() except: # Something bad happened, not sure what raise Re-raising an exception
76. try: do_something() do_the_next_thing() except ValueError: # Something bad happened, not sure what raise ValueError Re-raising an exception
77. try: do_something() do_the_next_thing() except IOError: raise Raising another exception
78. try: do_something() do_the_next_thing() except MyCustomError: raise ValueError Raising another exception
79. Why would I create my own exceptions?
80. Isolate exceptions caused by your application from exceptions caused by other code
81. Custom exceptions class SandwichException(Exception): pass
82. +-- Exception +-- StopIteration +-- StandardError | +-- BufferError | +-- ArithmeticError | | +-- FloatingPointError | | +-- OverflowError | | +-- ZeroDivisionError | +-- AssertionError | +-- AttributeError | +-- EnvironmentError | | +-- IOError | | +-- OSError | | +-- WindowsError (Windows) | | +-- VMSError (VMS) | +-- EOFError | +-- ImportError | +-- LookupError | | +-- IndexError | | +-- KeyError | +-- MemoryError | +-- NameError | | +-- UnboundLocalError | +-- ReferenceError | +-- RuntimeError | | +-- NotImplementedError | +-- SyntaxError | | +-- IndentationError | | +-- TabError | +-- SystemError | +-- TypeError | +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- PendingDeprecationWarning +-- RuntimeWarning +-- SyntaxWarning +-- UserWarning +-- FutureWarning +-- ImportWarning +-- UnicodeWarning +-- BytesWarning http://docs.python.org/2/library/exceptions.html#exception-hierarchy
83. +-- Exception +-- StopIteration +-- StandardError | +-- BufferError | +-- ArithmeticError | | +-- FloatingPointError | | +-- OverflowError | | +-- ZeroDivisionError | +-- AssertionError | +-- AttributeError | +-- EnvironmentError | | +-- IOError | | +-- OSError | | +-- WindowsError (Windows) | | +-- VMSError (VMS) | +-- EOFError | +-- ImportError | +-- LookupError | | +-- IndexError | | +-- KeyError | +-- MemoryError | +-- NameError | | +-- UnboundLocalError | +-- ReferenceError | +-- RuntimeError | | +-- NotImplementedError | +-- SyntaxError | | +-- IndentationError | | +-- TabError | +-- SystemError | +-- TypeError | +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- PendingDeprecationWarning +-- RuntimeWarning +-- SyntaxWarning +-- UserWarning +-- FutureWarning +-- ImportWarning +-- UnicodeWarning +-- BytesWarning http://docs.python.org/2/library/exceptions.html#exception-hierarchy
84. +-- Exception +-- StopIteration +-- StandardError | +-- BufferError | +-- ArithmeticError | | +-- FloatingPointError | | +-- OverflowError | | +-- ZeroDivisionError | +-- AssertionError | +-- AttributeError | +-- EnvironmentError | | +-- IOError | | +-- OSError | | +-- WindowsError (Windows) | | +-- VMSError (VMS) | +-- EOFError | +-- ImportError | +-- LookupError | | +-- IndexError | | +-- KeyError | +-- MemoryError | +-- NameError | | +-- UnboundLocalError | +-- ReferenceError | +-- RuntimeError | | +-- NotImplementedError | +-- SyntaxError | | +-- IndentationError | | +-- TabError | +-- SystemError | +-- TypeError | +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning +-- PendingDeprecationWarning +-- RuntimeWarning +-- SyntaxWarning +-- UserWarning +-- FutureWarning +-- ImportWarning +-- UnicodeWarning +-- BytesWarning http://docs.python.org/2/library/exceptions.html#exception-hierarchy | +-- RuntimeError | | +-- NotImplementedError | +-- SyntaxError | | +-- IndentationError | | +-- TabError | +-- SystemError | +-- TypeError | +-- ValueError | +-- UnicodeError | +-- UnicodeDecodeError | +-- UnicodeEncodeError | +-- UnicodeTranslateError +-- Warning +-- DeprecationWarning
85. exception ValueError •Raised when a built-in operation or function receives an argument that has the right type but an inappropriate value, and the situation is not described by a more precise exception such as IndexError.
86. Custom exceptions class SandwichException(Exception): pass
87. Custom exceptions class SandwichException(ValueError): pass
88. Granular custom exceptions class SandwichException(ValueError): pass class MeatException(SandwichException): pass
89. def make_sandwich(self): try: sandwich.get_bread() sandwich.get_meat() except SandwichException: return_failed_sandwich() def get_bread(self): bread = this_raisesValueError() # oh no! that raised a ValueError! return bread
90. def make_sandwich(self): try: sandwich.get_bread() sandwich.get_meat() except SandwichException: return_failed_sandwich() def get_bread(self): bread = this_raisesValueError() # oh no! that raised a ValueError! return bread
91. Don't expose implementation details
92. from Bakeries import NewYorkBakery from Bakeries import NewYorkBakeryException def make_sandwich(self): try: sandwich.get_bread() sandwich.get_meat() except MeatException: return_cheese_sandwich() except SandwichException: return_failed_sandwich() def get_bread(self): try: bread = NewYorkBakery() except NewYorkBakeryException: raise SandwichException
93. from Bakeries import CaliforniaBakery from Bakeries import CaliforniaBakeryException def make_sandwich(self): try: sandwich.get_bread() sandwich.get_meat() except MeatException: return_cheese_sandwich() except SandwichException: return_failed_sandwich() def get_bread(self): try: bread = CaliforniaBakery() except CaliforniaBakeryException: raise SandwichException
94. try: file = open(‘file.txt’, ‘r’) except IOError as error: print “Can’t open file, screw it” else: # We can do this only if the exception # was not raised. If read() raises an # IOError, we can handle it differently contents = file.read() else
95. finally try: file = open(‘file.txt’, ‘r’) except IOError as error: print “Can’t open file, screw it” finally: file.close()
96. The context manager: Cleaner code with with
97. def my_function(): try: do_something() do_the_next_thing() except AnException as exception: handle_that_exception(exception) finally: do_this_no_matter_what()
98. from contextlib import contextmanager @contextmanager def try_this(): try: yield # calls functions in try_this() except (ValueError, TypeError): handle_that_exception() finally: clean_up_step() def my_function(): with try_this(): # this is the context do_something() do_something_else()
99. Look Before You Leap v. Easier to Ask for Forgiveness than Permission
101. if foo: do_foo() elif bar: do_bar() elif baz: do_baz() else: barf()
102. •__iter__() raises StopIteration •__getitem__() raises IndexError
103. Defensive Programming
104. Expecting the unexpected
105. known knowns Expecting the unexpected
106. known knowns known unknowns Expecting the unexpected
107. known knowns unknown unknowns known unknowns Expecting the unexpected
108. unknown knowns known knowns unknown unknowns known unknowns Expecting the unexpected
109. Think through your assumptions
110. Every bug is 2 bugs
111. Every bug is 2 bugs 1. the bug in the code
112. Every bug is 2 bugs 1. the bug in the code 2. the test you didn’t write
113. How do you know you’ve anticipated necessary exceptions?
114. Test your error-handling
115. All exceptions are handled
116. http://newcars.com/about Thanks
117. NewCars Tech Team SoCal Python Audrey Roy Danny Greenfeld Chris McDonough Doug Napoleone Rachel Sanders Thanks
118. estherbester on: •Twitter •GitHub •IRC (freenode)
119. Related topics •Logging •Debugging exceptions •sys.exc_info() •Exception chaining (Python 3) •Testing
120. References http://blog.ianbicking.org/2007/09/12/re-raising-exceptions/ http://codemonkeyism.com/7-good-rules-to-log-exceptions/ http://docs.python.org/2/tutorial/errors.html http://docs.python.org/3.0/whatsnew/3.0.html http://doughellmann.com/2008/05/pymotw-contextlib.html http://en.wikipedia.org/wiki/Defensive_programming http://en.wikipedia.org/wiki/Exception_safety http://jeffknupp.com/blog/2013/02/06/write-cleaner-python-use-exceptions/ http://jessenoller.com/blog/2009/02/03/get-with-the-program-as-contextmanager-completely- different http://politechnosis.kataire.com/2008/02/all-exceptions-are-handled.html http://stackoverflow.com/questions/696047/re-raising-exceptions-with-a-different-type-and- message-preserving-existing-inf http://stackoverflow.com/questions/7108193/frequently-repeated-try-except-in-python?rq=1 http://stackoverflow.com/questions/77127/when-to-throw-an-exception http://today.java.net/article/2006/04/04/exception-handling-antipatterns http://www.blog.pythonlibrary.org/2012/08/02/python-101-an-intro-to-logging/ http://www.itmaybeahack.com/homepage/books/nonprog/html/p09_exc_iter/ p09_c01_exception.html#designing-exceptionshttps://en.wikipedia.org/wiki/ There_are_known_knowns#Quoted_from_Persian_literature