PyCon 2013
March 17, 2013
970

# How to Except When You’re Excepting by Esther Nam

March 17, 2013

## Transcript

1. from The New Book of Etiquette (1925) by Lillian Eichler

2. How to Except When
You’re Excepting
Esther Nam PyCon 2013
Santa Clara, CA
Text

3. •Python developer

•Python developer
•Pro since 2011

5. •Python developer
•Pro since 2011
•No CS degree

6. You’ve Met Me At...
•LA Hackathons

7. •LA Hackathons
You’ve Met Me At...

8. •LA Hackathons
•LA Girl Geek
Dinners
You’ve Met Me At...

9. In this talk

10. In this talk
•What are exceptions?

11. In this talk
•What are exceptions?
•Handling exceptions

12. In this talk
•What are exceptions?
•Handling exceptions
•Exceptions in Python

13. In this talk
•What are exceptions?
•Handling exceptions
•Exceptions in Python
•Defensive coding

14. What is an exception?

15. def  square_this(number):
"""  Return  the  number  squared  """
return  number  *  number
>>> square_this(3)
9

16. def  square_this(number):
"""  Return  the  number  squared  """
return  number  *  number
>>> square_this(3)
9
>>> square_this(“number”)
Traceback (most recent call last):
File "", line 1, in
File "", line 2, in square_this
TypeError: can't multiply sequence by non-int of type
'str'

17. "the function's assumptions

18. def  square_this(number):
return  number  *  number

>>> square_this(3)
9
>>> square_this(“number”)
Traceback (most recent call last):
File "", line 1, in
File "", line 2, in square_this
TypeError: can't multiply sequence by non-int of type
'str'

19. def  square_this(number):
return  number  *  number

>>> square_this(3)
9
>>> square_this(“number”)
Traceback (most recent call last):
File "", line 1, in
File "", line 2, in square_this
TypeError: can't multiply sequence by non-int of type
'str'

20. def  square_this(number):
return  number  *  number

>>> square_this(3)
9
>>> square_this(“number”)
Traceback (most recent call last):
File "", line 1, in
File "", line 2, in square_this
TypeError: can't multiply sequence by non-int of type
'str'

21. def  make_sandwich(self):
"""  Make  a  sandwich  """
return  sandwich

22. def  make_sandwich(self):
"""  Make  a  sandwich  """
return  sandwich

"""  Slice  meat  """
if  sandwich.meat  in  ['ham',  'beef']:
raise  ValueError

23. def  make_sandwich(self):
"""  Make  a  sandwich  """
return  sandwich

"""  Slice  meat  """
if  sandwich.meat  in  ['ham',  'beef']:
raise  ValueError

24. >>> sandwich = Sandwich("ham")
>>> sandwich.make_sandwich()
def  make_sandwich(self):
"""  Make  a  sandwich  """
return  sandwich

"""  Slice  meat  """
if  sandwich.meat  in  ['ham',  'beef']:
raise  ValueError

25. >>> sandwich = Sandwich("ham")
>>> sandwich.make_sandwich()
Traceback (most recent call last):
[…]
ValueError
def  make_sandwich(self):
"""  Make  a  sandwich  """
return  sandwich

"""  Slice  meat  """
if  sandwich.meat  in  ['ham',  'beef']:
raise  ValueError

26. What is exception
handling?

27. def  make_sandwich(self):
"""  Make  a  sandwich  """
try:
except  ValueError:
print  "Meat  is  murder!  Cheese  only"
return  sandwich

28. def  make_sandwich(self):
"""  Make  a  sandwich  """
try:
except  ValueError:
print  "Meat  is  murder!  Cheese  only"
return  sandwich

"""  Slice  meat  """
if  sandwich.meat  in  ['ham',  'beef']:
sandwich.meat  =  None
raise  ValueError

29. >>> sandwich.make_sandwich()
def  make_sandwich(self):
"""  Make  a  sandwich  """
try:
except  ValueError:
print  "Meat  is  murder!  Cheese  only"
return  sandwich

"""  Slice  meat  """
if  sandwich.meat  in  ['ham',  'beef']:
sandwich.meat  =  None
raise  ValueError

30. >>> sandwich.make_sandwich()
Meat is murder! Cheese only
>>> sandwich.meat
>>>
def  make_sandwich(self):
"""  Make  a  sandwich  """
try:
except  ValueError:
print  "Meat  is  murder!  Cheese  only"
return  sandwich

"""  Slice  meat  """
if  sandwich.meat  in  ['ham',  'beef']:
sandwich.meat  =  None
raise  ValueError

31. What causes exceptions?

32. •Programming errors

33. •Programming errors
•Client code errors

34. •Programming errors
•Client code errors
•Resource failures

35. Exiting may be preferable to
compounding errors

36. Why not use error codes?

37. ERROR  =  -­‐1
def  a():
value  =  b()

if  value  !=  ERROR:
do_the_next_thing()
else:
barf()

38. ERROR  =  -­‐1
def  a():
value  =  b()

if  value  !=  ERROR:
do_the_next_thing()
else:
barf()

def  b():
bvalue  =  c()

if  bvalue  !=  ERROR:
return  bvalue  +  1
else:
barf()

39. ERROR  =  -­‐1
def  a():
value  =  b()

if  value  !=  ERROR:
do_the_next_thing()
else:
barf()

def  b():
bvalue  =  c()

if  bvalue  !=  ERROR:
return  bvalue  +  1
else:
barf()
def  c():
return  ERROR

40. def  a():
try:
value  =  b()
do_the_next_thing()
except  ValueError:
barf()

def  b():
bvalue  =  c()
return  bvalue  +  1
def  c():
something_that_causes_ValueError()
#  Oh  no,  a  ValueError  was  raised!
return  cvalue

41. enum  {
CMDERR_OPTION_UNKNOWN  =  -­‐3,  /*  unknown  -­‐option  */
CMDERR_OPTION_AMBIGUOUS  =  -­‐2,  /*  ambiguous  -­‐option  */
CMDERR_OPTION_ARG_MISSING  =  -­‐1,  /*  argument  missing  for  -­‐option  */
CMDERR_UNKNOWN,  /*  unknown  command  */
CMDERR_AMBIGUOUS,  /*  ambiguous  command  */
CMDERR_ERRNO,  /*  get  the  error  from  errno  */
CMDERR_NOT_ENOUGH_PARAMS,  /*  not  enough  parameters  given  */
CMDERR_NOT_CONNECTED,  /*  not  connected  to  server  */
CMDERR_NOT_JOINED,  /*  not  joined  to  any  channels  in  this  window  */
CMDERR_CHAN_NOT_FOUND,  /*  channel  not  found  */
CMDERR_CHAN_NOT_SYNCED,  /*  channel  not  fully  synchronized  yet  */
CMDERR_ILLEGAL_PROTO,  /*  requires  different  chat  protocol  than  the  active  server  */
CMDERR_NOT_GOOD_IDEA,  /*  not  good  idea  to  do,  -­‐yes  overrides  this  */
CMDERR_INVALID_TIME,  /*  invalid  time  specification  */
CMDERR_INVALID_CHARSET,  /*  invalid  charset  specification  */
CMDERR_EVAL_MAX_RECURSE,  /*  eval  hit  recursion  limit  */
CMDERR_PROGRAM_NOT_FOUND  /*  program  not  found  */
};

42. Why handle exceptions?

43. Error may not be fatal

44. What kinds of exceptions
can I catch?

45. BaseException
+-- SystemExit
+-- KeyboardInterrupt
+-- GeneratorExit
+-- Exception

46. BaseException
+-- SystemExit
+-- KeyboardInterrupt
+-- GeneratorExit
+-- Exception

47. +-- Exception
+-- StopIteration
+-- StandardError
| +-- BufferError
| +-- ArithmeticError
| | +-- FloatingPointError
| | +-- OverflowError
| | +-- ZeroDivisionError
| +-- AssertionError
| +-- AttributeError
| +-- EnvironmentError
| | +-- IOError
| | +-- OSError
| | +-- WindowsError (Windows)
| | +-- VMSError (VMS)
| +-- EOFError
| +-- ImportError
| +-- LookupError
| | +-- IndexError
| | +-- KeyError
| +-- MemoryError
| +-- NameError
| | +-- UnboundLocalError
| +-- ReferenceError
| +-- RuntimeError
| | +-- NotImplementedError
| +-- SyntaxError
| | +-- IndentationError
| | +-- TabError
| +-- SystemError
| +-- TypeError
| +-- ValueError
| +-- UnicodeError
| +-- UnicodeDecodeError
| +-- UnicodeEncodeError
| +-- UnicodeTranslateError
+-- Warning
+-- DeprecationWarning
+-- PendingDeprecationWarning
+-- RuntimeWarning
+-- SyntaxWarning
+-- UserWarning
+-- FutureWarning
+-- ImportWarning
+-- UnicodeWarning
+-- BytesWarning
http://docs.python.org/2/library/exceptions.html#exception-hierarchy

48. Who should catch the
exception?

49. Where things break
•Application
•Framework
•System-level

50. Can I just catch all of them?

51. Catch all the exceptions?
try:
results  =  run_query()
except:
handle_error()

52. BaseException
+-- SystemExit
+-- KeyboardInterrupt
+-- GeneratorExit
+-- Exception

53. Catch all the exceptions?
try:
results  =  run_query()
except:
handle_error()

54. Catch all the exceptions?
try:
results  =  run_query()
except  Exception:
handle_error()

55. try:
results  =  run_query()
except  Exception:
return_empty_rows()

def  run_query():
connect_to_db()
sort_rows()

def  connect_to_db():
raise  DatabaseNotConnectedError

56. try:
results  =  run_query()
except  Exception:
handle_error()
def  run_query():
connect_to_db()
sort_rows()

def  connect_to_db():
raise  DatabaseNotConnectedError

57. try:
results  =  run_query()
except  Exception:
handle_error()
def  run_query():
connect_to_db()
sort_rows()

def  connect_to_db():
raise  DatabaseNotConnectedError

58. What if there’s more than one
possible exception?

59. try:
do_something()
do_the_next_thing()
except AnException:
handle_exception()
Catching multiple exceptions

60. try:
do_something()
do_the_next_thing()
except (ExceptionA, ExceptionB):
handle_exception()
Catching multiple exceptions

61. try:
do_something()
do_the_next_thing()
except AnException:
handle_exception()
except FatalException:
just_die()
Catching multiple exceptions

62. try:
do_some_math()
# ZeroDivisionError is a subclass of
# ArithmeticError
except ZeroDivisionError:
just_die()
except ArithmeticError:
try_to_handle_this()
Catching granular exceptions

63. Catching lots of exceptions?
try:
do_some_math()
except OneException:
just_die()
except AnotherException:
try_to_handle_this()
except YetAnotherException:
just_log_this_one()
except YetAnotherException:
i_dunno_what_to_do_anymore()

64. try:
do_some_math()
except OneException:
just_die()
except AnotherException:
try_to_handle_this()
except YetAnotherException:
just_log_this_one()
except YetAnotherException:
i_dunno_what_to_do_anymore()
Catching lots of exceptions?

65. So you’ve caught an exception

66. try:
do_something()
except ValueError:
pass
Handling exceptions

67. Continue as though nothing
went wrong?

68. Exceptions are always handled

69. try:
do_something()
except ValueError:
logger.error()

70. Delay the exception
try:
open_file()
except AnException as exception:
clean_up()

71. try:
open_file()
except AnException as exception:
clean_up()
logger(“Log: %s”” % exception)
Delay the exception

72. try:
open_file()
except AnException as exception:
clean_up()
logger(“Log: %s”” % exception)
raise
Delay the exception

73. Re-try
for attempt in number_of_attempts:
connected = False
try:
connect_to_server()
connected = True
except ConnectionFail:
wait_one_second()
return connected

74. try:
do_something()
do_the_next_thing()
except:
# Something bad happened, not sure what
Re-raising an exception

75. try:
do_something()
do_the_next_thing()
except:
# Something bad happened, not sure what
raise
Re-raising an exception

76. try:
do_something()
do_the_next_thing()
except ValueError:
# Something bad happened, not sure what
raise ValueError
Re-raising an exception

77. try:
do_something()
do_the_next_thing()
except IOError:
raise
Raising another exception

78. try:
do_something()
do_the_next_thing()
except MyCustomError:
raise ValueError
Raising another exception

79. Why would I create
my own exceptions?

80. Isolate exceptions caused by
exceptions caused by other
code

81. Custom exceptions
class SandwichException(Exception):
pass

82. +-- Exception
+-- StopIteration
+-- StandardError
| +-- BufferError
| +-- ArithmeticError
| | +-- FloatingPointError
| | +-- OverflowError
| | +-- ZeroDivisionError
| +-- AssertionError
| +-- AttributeError
| +-- EnvironmentError
| | +-- IOError
| | +-- OSError
| | +-- WindowsError (Windows)
| | +-- VMSError (VMS)
| +-- EOFError
| +-- ImportError
| +-- LookupError
| | +-- IndexError
| | +-- KeyError
| +-- MemoryError
| +-- NameError
| | +-- UnboundLocalError
| +-- ReferenceError
| +-- RuntimeError
| | +-- NotImplementedError
| +-- SyntaxError
| | +-- IndentationError
| | +-- TabError
| +-- SystemError
| +-- TypeError
| +-- ValueError
| +-- UnicodeError
| +-- UnicodeDecodeError
| +-- UnicodeEncodeError
| +-- UnicodeTranslateError
+-- Warning
+-- DeprecationWarning
+-- PendingDeprecationWarning
+-- RuntimeWarning
+-- SyntaxWarning
+-- UserWarning
+-- FutureWarning
+-- ImportWarning
+-- UnicodeWarning
+-- BytesWarning
http://docs.python.org/2/library/exceptions.html#exception-hierarchy

83. +-- Exception
+-- StopIteration
+-- StandardError
| +-- BufferError
| +-- ArithmeticError
| | +-- FloatingPointError
| | +-- OverflowError
| | +-- ZeroDivisionError
| +-- AssertionError
| +-- AttributeError
| +-- EnvironmentError
| | +-- IOError
| | +-- OSError
| | +-- WindowsError (Windows)
| | +-- VMSError (VMS)
| +-- EOFError
| +-- ImportError
| +-- LookupError
| | +-- IndexError
| | +-- KeyError
| +-- MemoryError
| +-- NameError
| | +-- UnboundLocalError
| +-- ReferenceError
| +-- RuntimeError
| | +-- NotImplementedError
| +-- SyntaxError
| | +-- IndentationError
| | +-- TabError
| +-- SystemError
| +-- TypeError
| +-- ValueError
| +-- UnicodeError
| +-- UnicodeDecodeError
| +-- UnicodeEncodeError
| +-- UnicodeTranslateError
+-- Warning
+-- DeprecationWarning
+-- PendingDeprecationWarning
+-- RuntimeWarning
+-- SyntaxWarning
+-- UserWarning
+-- FutureWarning
+-- ImportWarning
+-- UnicodeWarning
+-- BytesWarning
http://docs.python.org/2/library/exceptions.html#exception-hierarchy

84. +-- Exception
+-- StopIteration
+-- StandardError
| +-- BufferError
| +-- ArithmeticError
| | +-- FloatingPointError
| | +-- OverflowError
| | +-- ZeroDivisionError
| +-- AssertionError
| +-- AttributeError
| +-- EnvironmentError
| | +-- IOError
| | +-- OSError
| | +-- WindowsError (Windows)
| | +-- VMSError (VMS)
| +-- EOFError
| +-- ImportError
| +-- LookupError
| | +-- IndexError
| | +-- KeyError
| +-- MemoryError
| +-- NameError
| | +-- UnboundLocalError
| +-- ReferenceError
| +-- RuntimeError
| | +-- NotImplementedError
| +-- SyntaxError
| | +-- IndentationError
| | +-- TabError
| +-- SystemError
| +-- TypeError
| +-- ValueError
| +-- UnicodeError
| +-- UnicodeDecodeError
| +-- UnicodeEncodeError
| +-- UnicodeTranslateError
+-- Warning
+-- DeprecationWarning
+-- PendingDeprecationWarning
+-- RuntimeWarning
+-- SyntaxWarning
+-- UserWarning
+-- FutureWarning
+-- ImportWarning
+-- UnicodeWarning
+-- BytesWarning
http://docs.python.org/2/library/exceptions.html#exception-hierarchy
| +-- RuntimeError
| | +-- NotImplementedError
| +-- SyntaxError
| | +-- IndentationError
| | +-- TabError
| +-- SystemError
| +-- TypeError
| +-- ValueError
| +-- UnicodeError
| +-- UnicodeDecodeError
| +-- UnicodeEncodeError
| +-- UnicodeTranslateError
+-- Warning
+-- DeprecationWarning

85. exception ValueError
•Raised when a built-in operation
that has the right type but an
inappropriate value, and the
situation is not described by a
more precise exception such as
IndexError.

86. Custom exceptions
class SandwichException(Exception):
pass

87. Custom exceptions
class SandwichException(ValueError):
pass

88. Granular custom exceptions
class SandwichException(ValueError):
pass
class MeatException(SandwichException):
pass

89. def make_sandwich(self):
try:
sandwich.get_meat()
except SandwichException:
return_failed_sandwich()
# oh no! that raised a ValueError!

90. def make_sandwich(self):
try:
sandwich.get_meat()
except SandwichException:
return_failed_sandwich()
# oh no! that raised a ValueError!

91. Don't expose
implementation details

92. from Bakeries import NewYorkBakery
from Bakeries import NewYorkBakeryException
def make_sandwich(self):
try:
sandwich.get_meat()
except MeatException:
return_cheese_sandwich()
except SandwichException:
return_failed_sandwich()
try:
except NewYorkBakeryException:
raise SandwichException

93. from Bakeries import CaliforniaBakery
from Bakeries import CaliforniaBakeryException
def make_sandwich(self):
try:
sandwich.get_meat()
except MeatException:
return_cheese_sandwich()
except SandwichException:
return_failed_sandwich()
try:
except CaliforniaBakeryException:
raise SandwichException

94. try:
file = open(‘file.txt’, ‘r’)
except IOError as error:
print “Can’t open file, screw it”
else:
# We can do this only if the exception
# was not raised. If read() raises an
# IOError, we can handle it differently
else

95. finally
try:
file = open(‘file.txt’, ‘r’)
except IOError as error:
print “Can’t open file, screw it”
finally:
file.close()

96. The context manager:
Cleaner code with with

97. def my_function():
try:
do_something()
do_the_next_thing()
except AnException as exception:
handle_that_exception(exception)
finally:
do_this_no_matter_what()

98. from contextlib import contextmanager
@contextmanager
def try_this():
try:
yield # calls functions in try_this()
except (ValueError, TypeError):
handle_that_exception()
finally:
clean_up_step()
def my_function():
with try_this(): # this is the context
do_something()
do_something_else()

99. Look Before You Leap
v.
Forgiveness than Permission

100. if foo:
do_foo()
elif bar:
do_bar()
elif baz:
do_baz()
else:
barf()

101. •__iter__() raises
StopIteration
•__getitem__() raises
IndexError

102. Defensive Programming

103. Expecting the unexpected

104. known
knowns
Expecting the unexpected

105. known
knowns
known
unknowns
Expecting the unexpected

106. known
knowns
unknown
unknowns
known
unknowns
Expecting the unexpected

107. unknown
knowns
known
knowns
unknown
unknowns
known
unknowns
Expecting the unexpected

109. Every bug is 2 bugs

110. Every bug is 2 bugs
1. the bug in the code

111. Every bug is 2 bugs
1. the bug in the code
2. the test you didn’t write

112. How do you know you’ve
anticipated necessary
exceptions?

114. All exceptions are handled

Thanks

116. NewCars Tech Team
SoCal Python
Audrey Roy
Danny Greenfeld
Chris McDonough
Doug Napoleone
Rachel Sanders
Thanks

117. estherbester on:
•GitHub
•IRC (freenode)

118. Related topics
•Logging
•Debugging exceptions
•sys.exc_info()
•Exception chaining (Python 3)
•Testing

119. References
http://blog.ianbicking.org/2007/09/12/re-raising-exceptions/
http://codemonkeyism.com/7-good-rules-to-log-exceptions/
http://docs.python.org/2/tutorial/errors.html
http://docs.python.org/3.0/whatsnew/3.0.html
http://doughellmann.com/2008/05/pymotw-contextlib.html
http://en.wikipedia.org/wiki/Defensive_programming
http://en.wikipedia.org/wiki/Exception_safety
http://jeffknupp.com/blog/2013/02/06/write-cleaner-python-use-exceptions/
http://jessenoller.com/blog/2009/02/03/get-with-the-program-as-contextmanager-completely-
different
http://politechnosis.kataire.com/2008/02/all-exceptions-are-handled.html
http://stackoverﬂow.com/questions/696047/re-raising-exceptions-with-a-different-type-and-
message-preserving-existing-inf
http://stackoverﬂow.com/questions/7108193/frequently-repeated-try-except-in-python?rq=1
http://stackoverﬂow.com/questions/77127/when-to-throw-an-exception
http://today.java.net/article/2006/04/04/exception-handling-antipatterns
http://www.blog.pythonlibrary.org/2012/08/02/python-101-an-intro-to-logging/
http://www.itmaybeahack.com/homepage/books/nonprog/html/p09_exc_iter/
p09_c01_exception.html#designing-exceptionshttps://en.wikipedia.org/wiki/
There_are_known_knowns#Quoted_from_Persian_literature