Failure: Or the Unexpected Virtue of Functional Programming

Failure: Or the Unexpected Virtue of Functional Programming

Even correct software fails.

So what happens if we shift the focus of functional programming to reliable systems? Let’s attack the hard and ugly set of programming problems, the solutions that don’t naturally fall out from building a neat library. Let’s apply our functional programming toolkit to delivering systems end-to-end.

This is a talk on reliable systems. What it takes to build them, how functional programming can and is being leveraged, and perhaps of more interest, where current approaches are letting us down.

*This talk was presented as the keynote at YOW Lambdajam 2015 in Brisbane*

42d9867a0fee0fa6de6534e9df0f1e9b?s=128

Mark Hibberd

May 22, 2015
Tweet

Transcript

  1. 2.
  2. 4.
  3. 5.
  4. 13.
  5. 14.
  6. 15.
  7. 16.
  8. 17.
  9. 18.

    “Several of them would have protested if they could have

    found the right arguments.” - George Orwell, Animal Farm
  10. 20.
  11. 21.
  12. 22.
  13. 23.
  14. 24.
  15. 26.
  16. 28.
  17. 47.
  18. 50.

    Correctness Reliability (produce the decisions by X o’clock using the

    last vetted dataset) (the best set of measurable decisions for today) Correctness
  19. 51.
  20. 57.

    If you have untangled your computation from your data, someone

    has probably solved your data storage requirements
  21. 58.

    But… Failure is never clean. One of the most difficult

    challenges is ensuring that we only have known good states, failure must not corrupt.
  22. 60.

    If you detected a failure, would you be able to

    identify the downstream effects?
  23. 61.

    Are there multiple paths to build a dataset? Could we

    rebuild from an alternate source if we needed to?
  24. 63.

    “fault isolation advocates that the process software be fail-fast, it

    should either function correctly or it should detect the fault, signal failure and stop operating” - Jim Gray, Why Do Computers Stop and What Can Be Done About It?
  25. 67.
  26. 77.
  27. 78.
  28. 79.

    “All animals are equal, but some animals are more equal

    than others.” - George Orwell, Animal Farm
  29. 86.
  30. 87.
  31. 88.
  32. 89.
  33. 90.
  34. 91.
  35. 94.
  36. 98.

    “ZFS has been subjected to over a million forced, violent

    crashes without losing data integrity or leaking a single block.” - Bonwick & Moore, ZFS The Last Word in File Systems
  37. 100.
  38. 102.
  39. 103.
  40. 104.
  41. 106.
  42. 107.

    If I can’t run multiple versions of the same code

    in parallel, one programming error can bring everything down
  43. 108.
  44. 110.
  45. 111.
  46. 112.
  47. 113.
  48. 114.
  49. 115.

    If I can run multiple versions of my code, but

    only one version of my infrastructure…
  50. 116.
  51. 117.
  52. 118.
  53. 119.
  54. 120.
  55. 121.
  56. 123.

    “construct reliable systems from unreliable parts … from the knowledge

    that any component in the system might fail” - Holzman & Joshi, Reliable Software Systems Design
  57. 125.
  58. 132.
  59. 141.

    “A beach house isn’t just real estate. It’s a state

    of mind.” - Douglas Adams, Mostly Harmless
  60. 142.