Hanging on in Quiet Desperation: Time & Programming

Hanging on in Quiet Desperation: Time & Programming

Time has a profound impact on the complexity of the systems we build.

A significant amount of this software complexity comes from either
an inability to recall previous states or the inability to
understand how a state was arrived at.

From the foundations of AI, LISP and functional programming [1],
to causality in distributed systems [2], to the more grungy practices
of immutable infrastructure, or the unreasonable effectiveness
of fact-based approaches to large scale data systems; the ability
to adequately cope with time, and the change and conflict it
inevitable creates, is a common thread to being able to build and
reason about these systems.

This talk looks at the impact of time on system design. We will
walk through examples of large-scale systems and their battles
with complexity. At the end of the talk, the audience should start to
see the common spectre of time and have an appreciation of
how understanding time is fundamental to maintaining clarity,
correctness and reliability in systems.

[1] Situations, Actions, and Causal Laws - John
McCarthy - http://www.dtic.mil/dtic/tr/fulltext/u2/785031.pdf

[2] Times, Clocks and the Ordering of Events in a Distributed
System - Leslie Lamport - https://amturing.acm.org/p558-lamport.pdf

42d9867a0fee0fa6de6534e9df0f1e9b?s=128

Mark Hibberd

May 23, 2018
Tweet

Transcript

  1. Hanging on in Quiet Desperation: @markhibberd Time & Programming

  2. Ticking away the moments that make up a dull day.

    You fritter and waste the hours in an offhand way. … And then one day you find ten years have got behind you. No one told you where to run, you missed the starting gun.” - Roger Waters “
  3. Complexity.

  4. 1986.

  5. None
  6. Changeability. Conformity. Complexity. Invisibility.

  7. 2006.

  8. None
  9. 2011.

  10. None
  11. Present Day.

  12. 1948.

  13. None
  14. 1958-1963.

  15. None
  16. The big idea.

  17. If we can reason about all the states of a

    system.
  18. And we can understand how each state is arrived at.

  19. We can make better decisions, and start to unwind the

    interdependencies and complexity caused by time and ordering.
  20. None
  21. None
  22. Present Day.

  23. 2004-2006.

  24. Disconnected networks.

  25. None
  26. None
  27. None
  28. None
  29. None
  30. None
  31. {dock:0 ,red:0 ,purple:0} {dock:0 ,red:0 ,purple:0} {dock:0 ,red:0 ,purple:0} owner|id|item|count

    red |0 |x |99 red |1 |y |1 dock |2 |z |33
  32. {dock:3 ,red:0 ,purple:0} {dock:0 ,red:0 ,purple:1} {dock:0 ,red:7 ,purple:0}

  33. {dock:5 ,red:0 ,purple:0} {dock:0 ,red:7 ,purple:1} {dock:0 ,red:7 ,purple:1}

  34. {dock:5 ,red:0 ,purple:0} {dock:3 ,red:7 ,purple:1} {dock:0 ,red:7 ,purple:1}

  35. {dock:7 ,red:7 ,purple:3} {dock:7 ,red:7 ,purple:3} {dock:0 ,red:8 ,purple:1}

  36. {dock:7 ,red:8 ,purple:3} {dock:7 ,red:8 ,purple:3} {dock:7 ,red:8 ,purple:3}

  37. None
  38. 2009-2010.

  39. The most complex system I ever worked on.

  40. A web API with 2 resources.

  41. POST /activate POST /check Call to activate a product, validate

    and register license details Check license and validate installation details.
  42. What could possibly go wrong?

  43. None
  44. POST /activate POST /check

  45. POST /activate POST /check POST /check2007

  46. POST /activate?magik=true POST /check POST /check2007

  47. POST /activate?magik=true POST /check POST /check2007 POST /activatev3

  48. POST /activate?magik=true POST /check POST /check2007 POST /activatev3 POST /check9

  49. POST /activate?magik=true POST /check POST /check2007 POST /activatev3 POST /activate-new

    POST /check9
  50. POST /activate?magik=true POST /check POST /check2007 POST /activatev3 POST /activate-new?legacy=1

    POST /check9
  51. POST /activate?magik=true POST /check POST /check2007 POST /activatev3 POST /activate-new?legacy=1

    POST /check9
  52. POST /activate?magik=true POST /check POST /check2007 POST /activatev3 POST /activate-new?legacy=1

    POST /check9
  53. POST /activate?magik=true POST /check POST /check2007 POST /activatev3 POST /activate-new?legacy=1

    POST /check9
  54. POST /activate?magik=true POST /check POST /check2007 POST /activatev3 POST /activate-new?legacy=1

    POST /check9
  55. From: https://stripe.com/blog/api-versioning

  56. 2013-2017.

  57. Full circle. A machine learning problem.

  58. We need to describe the world, now and in the

    past.
  59. cause(π)(α)

  60. at(I)(desk)

  61. at(I)(desk) at(desk)(home)

  62. at(I)(desk) at(desk)(home) ∴ at(I)(home)

  63. TX CUSTOMER ITEM AMOUNT TIMESTAMP 1 X0001 Pen $5 2016-08-01T12:24:13

    2 X0001 Notepad $20 2017-01-01T02:19:01 3 X0002 Calculator $55 2015-01-01T09:55:21 4 X0003 Book $30 2017-07-01T15:34:53
  64. CUSTOMER LOCATION X0001 NY X0002 SINGAPORE X0003 HONG KONG X0004

    SYDNEY X0005 SYDNEY
  65. CUSTOMER LOCATION TIMESTAMP X0001 NY 2016-08-01T00:00:00 X0002 SINGAPORE 2017-08-04T00:00:00 X0003

    HONG KONG 2016-01-21T00:00:00 X0004 SYDNEY 2016-08-01T00:00:00 X0005 SYDNEY 2014-12-24T00:00:00
  66. CUSTOMER LOCATION TIMESTAMP X0001 NY 2016-08-01T00:00:00 X0002 SINGAPORE 2017-08-04T00:00:00 X0003

    HONG KONG 2016-01-21T00:00:00 X0004 SYDNEY 2016-08-01T00:00:00 X0005 SYDNEY 2014-12-24T00:00:00 X0001 LONDON 2017-09-07T00:00:00
  67. What is in a time?

  68. World Time System Time

  69. When is this fact valid? When did we learn this

    fact?
  70. Intervals. X0001 NY 2016-08-01T00:00:00 X0001 LONDON 2017-09-07T00:00:00

  71. Instants. 1 X0001 Pen $5 2016-08-01T12:24:13

  72. Horizons. 1 X0001 Pen $5 2016-08-01T12:24:13

  73. Horizons. 1 X0001 Pen $5 1234

  74. 2013-2017.

  75. Operations as a system.

  76. This is complexity.

  77. Techniques!

  78. How does time impact the system?

  79. None
  80. None
  81. 2014.

  82. towards annɛx

  83. Semantic versioning is not the whole story.

  84. Open world.

  85. Time matters.

  86. commit

  87. ci

  88. publish

  89. platform test

  90. production

  91. performance

  92. cve

  93. eminence boxer napoleon snowball

  94. boxer 1.2.1 commit: bd2f074…02 api-signature: […] feature: it-works artifact: tag:

    bin/boxer flags: […] address: e2f1…bc74
  95. :boxer :has :feature “multi-part-put” :has :commit “abcd-1345” :has :branch “master”

    :snowball :is :compatable-with atom/… :napoleon :semver >= 1.3 < 1.4 :no-cve +annex.example.com@HEAD
  96. None
  97. None
  98. None
  99. None
  100. None
  101. None
  102. None
  103. Present Day.

  104. Artifacts over constructs.

  105. Time and ordering are out to get us.

  106. Techniques abound.

  107. If we can reason about all the states of a

    system.
  108. And we can understand how each state is arrived at.

  109. We can make better decisions, and start to unwind the

    interdependencies and complexity caused by time and ordering.
  110. Hanging on in Quiet Desperation: @markhibberd Time & Programming