Shit Happens... (v2)

Errors are the nightmare of every programmer. Talking about them leads to passionate discussions (and maybe some trolls) and although programming languages date back to more than 60 years, nobody can tell you what is the best way to deal with those @!& errors. Return codes, exceptions, custom data types, warnings, function incompleteness status codes, ..., this talk present an aerial view of how various languages deal with these errors (C/Ruby/Python/Haskell and maybe Go). Most of the concepts I will be presenting will be applicable to many programming languages (as a lot of them share identical concepts) and code examples will be mainly written in python.

However, let's be realistic: this presentation will not fix your errors, but it will give you some insight on how to train them.

This talk is almost identical to the previous version (also available on my speakerdeck account) with the following (subtle) differences:
- a more python centric overview with annotations and enumerations exemples
- some rewording

This talk was given at PyconFR 2013

Transcript

1. SHIT HAPPENS...
2. but don’t let it hit the fan ... shit is so common that a unicode character exists for it!
3. otherwise... Ariane 5 inaugural flight, 1996/06/04 500 million dollars lost in flames because of .... ! one uncaught floating point exception SHIT!
4. why do I care? • (freely) inspired by some talks • (deeply) inspired by real world examples • errare humanum est, perseverare diabolicum (to err is human, persist in error is diabolical)
5. diabolical? • as humans we make mistakes. software we build is error prone (every single one!) • but everybody wants error-proof software! • bad error handling produces bad programs • hopefully, languages implement systems to manage these errors. how helpful are they?
6. where? delivers delivers US
7. me lead engineer at @_olivier_
8. 3 of a kind • there is (mainly) three ways to manage errors: • return code • exception • monadic error control • every language has (at least) 2 of them
9. once upon a C-time • no exceptions • errors are managed either by: • return codes • the errno special register • (... and also non-local goto)
10. return codes • the good: easy to implement • the bad: • can be easily ignored • not universal - for instance operators and constructors (in C++) can’t return a return code • need to be translated
11. return codes • developers rely on conventions (some of us say documentation ...) • safely managing possible errors returned by each function is painful
13. saved by python enum • will be introduced in python 3.4 • easy way to implement meaningful return codes (easily translatable error codes) from enum import Enum! class Errno(Enum):! ARG_LIST_TOO_LONG = 0! PERM_DENIED = 1! ADDR_IN_USE = 2! ADDR_NOT_AVAIL = 3! ADDR_FAM_NOT_SUPPORT = 4 !
14. teaser • In addition some methods could return a status which is not truly an error • We have all used one of these at least once (more to come soon ...)
15. errno, a solution? • Extract of the C99 reference: <errno.h> is a header invented to encapsulate the error handling mechanism used by many of the library routines in <math.h> and <stdlib.h>.
 The error reporting machinery centered about the setting of errno is generally regarded with tolerance at best. It requires a “pathological coupling” between library functions and makes use of a static writable memory cell, which interferes with the construction of shareable libraries. Nevertheless, the C89 Committee preferred to standardize this existing, however deficient, machinery rather than invent something more ambitious.
16. safely using errno is hard #include <errno.h> #include <limits.h> #include <stdlib.h> void func(const char *c_str) { unsigned long number; char *endptr; number = strtoul(c_str, &endptr, 0); if (endptr == c_str || (number == ULONG_MAX && errno == ERANGE)) { /* Handle error */ } else { /* Computation succeeded */ } } #include <errno.h> #include <limits.h> #include <stdlib.h> void func(const char *c_str) { unsigned long number; char *endptr; errno = 0; number = strtoul(c_str, &endptr, 0); if (endptr == c_str || (number == ULONG_MAX && errno == ERANGE)) { /* Handle error */ } else { /* Computation succeeded */ } } errno should be reset manually
17. safely using errno is hard #include <errno.h> #include <stdio.h> void func(const char *filename) { FILE *fileptr; errno = 0; fileptr = fopen(filename, "rb"); if (errno != 0) { /* Handle error */ } } fopen may set errno even if no error has occured #include <errno.h> #include <stdio.h> void func(const char *filename) { FILE *fileptr; errno = 0; fileptr = fopen(filename, "rb"); if (fileptr == NULL) { /* An error occurred in fopen(), now it's valid to examine errno. */ if (errno != 0) { /* Handle error */ } } } And so much more: • setlocale • signal
19. wait, what? • python has a errno module! • contains only a dict of the available error code • documentation is sooooo laconic that it clearly indicates that nobody cares!
20. return codes in go • initially, there was no exception in go • return codes are a part of the method signature - example with os.Open func Open(name string) (file *File, err error) ! f, err := os.Open("filename.ext") if err != nil { log.Fatal(err)} go does not define tuple, thus you need to declare a variable to handle the error
21. saved by annotations? • unused python feature • useful to define explicitly what are input/ output of a method • usable only if documentation tools support them … • syntax is ugly (subjective opinion) def open(file_path: "str") -> ("file", "err"):! ...!
22. tale of exceptions • Jim Mitchell about exceptions: “termination is preferred over resumption; this is not a matter of opinion but a matter of years of experience. Resumption is seductive, but not valid” • It is admitted that exceptions should be used in exceptional cases (is this a truism?)
23. exceptions’ summary • exceptions are great when everything goes wrong • sadly, they do not cross the process boundaries • in some languages you should take care of your memory • exceptions are typed, catching all of them leeds to many catch/except blocks
24. go exception wording • Go implements exceptions, via keywords • panic • resume • I think these are carefully chosen terms
25. are exceptions really exceptional? • dynamic languages love exceptions... even in non exceptional cases • in python, end of iteration is indicated by an exception • (but not end of file...) • novice (and non-novice) programmers abuse this language feature
27. the pokemon exception! Gotta catch’em all!
28. ignored(OSError) @contextmanager! def ignored(*exceptions):! """! >>> with ignored(IndexError): #doctest +ELLIPSIS! ... ()[0]! Ignored exception of type ...! """! try:! yield! except exceptions as e:! print("Ignored exception of type `{}` has been caught - msg: `{}`".format(e.__class__, e))!
29. exc vs rc: a summary • = : knowing what kind of exception will be raised is difficult (most of the time)... • - : ...even if you know what this method could raise (sub-called method can also raise exceptions) • + : exceptions allow function composition
30. checked exceptions • once again, it’s hard to know exceptions that will be raised in each method • Java introduced “checked exceptions” to solve this issue • I’m not a Java expert, but checked exceptions aren’t widely used, or so it seems!
31. back to the real world what is the behavior of socket.send in python?
32. socket.send socket.send(bytes[, flags])¶ Send data to the socket. The socket must be connected to a remote socket. The optional flags argument has the same meaning as for recv() above. Returns the number of bytes sent. Applications are responsible for checking that all data has been sent; if only some of the data was transmitted, the application needs to attempt delivery of the remaining data. For further information on this topic, consult the Socket Programming HOWTO.
33. socket.send while total_bytes_sent < len(data):! total_bytes_sent += sock.send(data[total_bytes_sent:]) • a not-so-correct pattern could be: ! • but what happen if socket is closed? • what happen if total_bytes_send stalled?
34. digging ohloh • easy to find code that does not check the return value of socket.send • thankfully, it’s also easy to find code that checks this value • lesson learned: use socket.sendall ;-)
35. fantasy monads! A monad is a monoid in the category of endofunctors, what’s the problem? – James Iry
36. some definitions • high-order function: a function that does at least one of the following: • take one or more functions as an input • output a function • functor: a type T that obeys to: • composition: T(a∘b) == T(a)∘T(b) • identity: T(id(x)) == id(T(x))
37. monads! • A monad is essentially just a functor T with two extra methods: • join, of type: T (T a) -> T a • unit (sometimes called return, fork, or pure) of type a -> T a
39. monadic error control • in Haskell: monads Maybe or Either • in Scala: Option, Either (and Try and ...)
40. use case
    (the classical div by 0) def divide(x:Int, y:Int):Option[Double] =! if (y == 0) None else Some(x/y)! ! println(divide(4, 2))! println(divide(4, 0)) Some(2.0) None outputs:
41. use case
    map! def divide(x:Int, y:Int):Option[Double] =! if (y == 0) None else Some(x/y)! ! val l = List(1, 0, 2, 4, 0, 2)! l.map(x => divide(42, x)) List[Option[Double]] = List(Some(42.0), None, Some(21.0), Some(10.0), None, Some(21.0)) outputs:
42. use case
    flatMap! def divide(x:Int, y:Int):Option[Double] =! if (y == 0) None else Some(x/y)! ! val l = List(1, 0, 2, 4, 0, 2)! l.flatMap(x => divide(42, x))! List[Option[Double]] = List(42.0, 21.0, 10.0, 21.0) outputs:
43. the either case def divide(x:Double, y:Double):Either[String, Double] =! if (y == 0) Left("Can't divide by 0") else Right(x/y)! ! def test(n:Int) =! divide(42, n) match {! case Left(why) => why ! case Right(result) => result! }! ! println(test(0))! println(test(1)) outputs: Can't divide by 0 42.0
44. either in composition def add(x:Either[String, Double], y:Either[String, Double]) =! if (x.isRight && y.isRight) {Right(x.right.get + y.right.get)} else if (x.isLeft) {x} else {y}! ! def add2(eX:Either[String, Double], eY:Either[String, Double]) =! eX.right.flatMap { x => eY.right.map { y => x + y } }! ! def add3(eX:Either[String, Double], eY:Either[String, Double]) = for {! x <- eX.right! y <- eY.right! } yield (x+y)! ! println(add3(divide(4, 2), divide(3, 4)))! println(add3(divide(4, 2), divide(3, 0))) outputs: Right(2.75) Left(Can't divide by 0)
45. advantages • “easy” composition allowed by monads • filter or preserve your errors without raising exceptions
46. implementable in python? • some of us may argue that python is not a functional language and does not implement type constructor as in haskell • but it’s all about plumbing, and monad are nothing else than another design pattern
47. either in python class Result(object):! def __init__(self, result, fail=False):! self.result = result! self.fail = fail! ! def __repr__(self):! return "Result:`{}`".format(self.result) if self.result else "Fail:`{}`".format(self.fail)! ! class Fail(Result):! def __init__(self, reason=True):! Result.__init__(self, None, reason)! ! def f(n):! return Result(n) if n > 10 else Fail("too small")! ! l = [10, 20, 4, 0, 60, 8]! print map(f, l)
48. (my) rule of thumb • exceptions should signal unexpected failures. Thus, you should consider “normal execution flow” code in an except block as a code smell • in other words, if a method potentially returns an error, use return codes • if a method potentially returns an error but composition is needed, use monadic error control
49. (my) observations • linters (such as pylint) are more helpful when using return codes than exceptions • it’s fine to disagree with my rules but please be consistent in your own way of managing errors
50. end happy error handling!