Save 37% off PRO during our Black Friday Sale! »

Contracts for building reliable systems

Contracts for building reliable systems

06f8b41980eb4c577fa40c41d5030c19?s=128

Chris Keathley

August 30, 2019
Tweet

Transcript

  1. Contracts for building robust systems Chris Keathley / @ChrisKeathley /

    c@keathley.io
  2. Confession

  3. I like dynamic languages

  4. Contracts for building robust systems Chris Keathley / @ChrisKeathley /

    c@keathley.io
  5. Chris Keathley / @ChrisKeathley / c@keathley.io Contracts for building robust

    systems Correct
  6. Correctness: the quality or state of being free from error;

    accuracy.
  7. Robust: able to withstand or overcome adverse conditions.

  8. My life Service Service Service Service Service

  9. My life Service Service Service Service Service Teams

  10. My life Service Service Service Service Service Teams Me

  11. Session Types

  12. My life Service Service Service Service Service Teams Me

  13. Correctness

  14. Correctness(t) Correctness over time

  15. Change

  16. Change Growth Breakage

  17. Changes Service Service

  18. Changes Service Service Expects users

  19. Changes Service Service

  20. Changes Service Service User{}

  21. Changes Service Service User{}

  22. Changes Service Service

  23. Changes Service Service Now you can send me Users or

    Cats
  24. Changes Service Service Now you can send me Users or

    Cats User{}
  25. Changes Service Service Now you can send me Users or

    Cats User{}
  26. Changes Service Service

  27. Changes Service Service Now you can only send me Cats

  28. Changes Service Service Now you can only send me Cats

    User{}
  29. Changes Service Service Now you can only send me Cats

    User{}
  30. Breaking changes require coordination

  31. Why does software change?

  32. Why does software change? Aesthetics

  33. Why does software change? Aesthetics Refactors

  34. Refactor?

  35. Why does software change? Aesthetics Refactors

  36. Why does software change? Aesthetics Technology Refactors

  37. Why does software change? Aesthetics Technology Requirements Refactors

  38. Why does software change? Aesthetics Technology Requirements Tacitly expects no

    breaking changes Refactors
  39. We want to ensure that we didn’t break the api

    while allowing for growth
  40. Lets talk about… Contracts Data specification Testing strategies

  41. Lets talk about… Contracts Data specification Testing strategies

  42. function

  43. function Input

  44. function Input Output

  45. function Input Output Side-effects

  46. function Input Output Side-effects Enforce the callers are correct

  47. function Input Output Side-effects Enforce the callers are correct Ensure

    the function is correct
  48. function Input Output Side-effects

  49. function Input Output Side-effects Pre-conditions Post-conditions

  50. rgb_to_hex

  51. rgb_to_hex(0, 10, 32) => “#000A20”

  52. rgb_to_hex ::

  53. rgb_to_hex :: Integer -> Integer -> Integer

  54. rgb_to_hex :: Integer -> Integer -> Integer -> String

  55. rgb_to_hex :: Integer -> Integer -> Integer -> String 0

    <= x <= 255
  56. rgb_to_hex :: Integer -> Integer -> Integer -> String 0

    <= x <= 255 Starts with a ‘#’
  57. rgb_to_hex :: Integer -> Integer -> Integer -> String 0

    <= x <= 255 Starts with a ‘#’ How do we type this?
  58. Dependent Types

  59. rgb_to_hex :: Integer -> Integer -> Integer -> String 0

    <= x <= 255 Starts with a ‘#’ How do we type this?
  60. rgb_to_hex :: Integer -> Integer -> Integer -> String 0

    <= x <= 255 Starts with a ‘#’ How do we type this? (today)
  61. Design by Contract github.com/JDUnity/ex_contract

  62. defmodule Contractual do use ExContract def rgb_to_hex(r, g, b) do

    #… end end
  63. defmodule Contractual do use ExContract requires is_integer(r) and 0 <=

    r and r <= 255 requires is_integer(g) and 0 <= g and g <= 255 requires is_integer(b) and 0 <= b and b <= 255 def rgb_to_hex(r, g, b) do #… end end
  64. defmodule Contractual do use ExContract requires is_integer(r) and 0 <=

    r and r <= 255 requires is_integer(g) and 0 <= g and g <= 255 requires is_integer(b) and 0 <= b and b <= 255 ensures is_binary(result) ensures String.starts_with?(result, "#") def rgb_to_hex(r, g, b) do #… end end
  65. defmodule Contractual do use Vow requires is_integer(r) and 0 <=

    r and r <= 255 requires is_integer(g) and 0 <= g and g <= 255 requires is_integer(b) and 0 <= b and b <= 255 ensures is_binary(result) ensures String.starts_with?(result, "#") def rgb_to_hex(r, g, b) do #… end end This is your invariant
  66. What happens when we get an error? rgb_to_hex(-10, 300, 0.5)

  67. What happens when we get an error? rgb_to_hex(-10, 300, 0.5)

    ** (ExContract.RequiresException) Pre-condition ["is_integer(r) and 0 <= r and r <= 255"] violated. Invalid implementation of caller to function [rgb_to_hex/3]
  68. Don’t we already have guard clauses?

  69. We can also check side-effects requires !check_in_db?(user), "user should not

    be stored" ensures check_in_db?(user), "user should be stored" def store_user(user) do # ... end
  70. We can also check side-effects requires !check_in_db?(user), "user should not

    be stored" ensures check_in_db?(user), "user should be stored" def store_user(user) do # ... end Shouldn’t be in the db afterwords it should
  71. Contracts are powerful

  72. Isn’t this really slow?

  73. Yes!

  74. Lets talk about… Contracts Data specification Testing strategies

  75. Lets talk about… Contracts Data specification Testing strategies

  76. defmodule Contractual do use ExContract requires is_integer(r) and 0 <=

    r and r <= 255 requires is_integer(g) and 0 <= g and g <= 255 requires is_integer(b) and 0 <= b and b <= 255 ensures is_binary(result) ensures String.starts_with?(result, "#") def rgb_to_hex(r, g, b) do #… end end
  77. requires is_integer(r) and 0 <= r and r <= 255

    requires is_integer(g) and 0 <= g and g <= 255 requires is_integer(b) and 0 <= b and b <= 255 ensures is_binary(result) ensures String.starts_with?(result, "#")
  78. is_integer(r) and 0 <= r and r <= 255 is_integer(g)

    and 0 <= g and g <= 255 is_integer(b) and 0 <= b and b <= 255 is_binary(result) String.starts_with?(result, "#")
  79. is_integer(r) and 0 <= r and r <= 255 is_integer(g)

    and 0 <= g and g <= 255 is_integer(b) and 0 <= b and b <= 255 is_binary(result) String.starts_with?(result, "#") Is the data correct?
  80. Norm

  81. is_integer(r) && 0 <= r and r <= 255 is_integer(g)

    && 0 <= g and g <= 255 is_integer(b) && 0 <= b and b <= 255 is_binary(result) && String.starts_with?(result, "#") Norm
  82. def rgb, do: spec(is_integer() and (& 0 <= &1 and

    &1 <= 255)) is_binary(result) && String.starts_with?(result, "#") Norm
  83. def rgb, do: spec(is_integer() and (& 0 <= &1 and

    &1 <= 255)) def hex, do: spec(is_binary() and (&String.starts_with?(&1, "#"))) Norm
  84. Norm conform!(123, rgb()) 123 def rgb, do: spec(is_integer() and (&

    0 <= &1 and &1 <= 255)) def hex, do: spec(is_binary() and (&String.starts_with?(&1, "#")))
  85. Norm conform!(123, rgb()) 123 conform!(-10, rgb()) ** (Norm.MismatchError) Could not

    conform input: val: -10 fails: &(0 <= &1 and &1 <= 255) (norm) lib/norm.ex:385: Norm.conform!/2 def rgb, do: spec(is_integer() and (& 0 <= &1 and &1 <= 255)) def hex, do: spec(is_binary() and (&String.starts_with?(&1, "#")))
  86. defmodule Contractual do use ExContract requires is_integer(r) and 0 <=

    r and r <= 255 requires is_integer(g) and 0 <= g and g <= 255 requires is_integer(b) and 0 <= b and b <= 255 ensures is_binary(result) ensures String.starts_with?(result, "#") def rgb_to_hex(r, g, b) do #… end end
  87. defmodule Contractual do use ExContract def rgb, do: spec(is_integer() and

    &(0 <= &1 and &1 <= 255)) def hex, do: spec(is_binary() and (&String.starts_with?(&1, "#"))) requires is_integer(r) and 0 <= r and r <= 255 requires is_integer(g) and 0 <= g and g <= 255 requires is_integer(b) and 0 <= b and b <= 255 ensures is_binary(result) ensures String.starts_with?(result, "#") def rgb_to_hex(r, g, b) do #… end end
  88. defmodule Contractual do use ExContract def rgb, do: spec(is_integer() and

    &(0 <= &1 and &1 <= 255)) def hex, do: spec(is_binary() and (&String.starts_with?(&1, "#"))) requires valid?(r, rgb()) and valid?(g, rgb()) and valid?(b, rgb()) ensures valid?(result, hex()) def rgb_to_hex(r, g, b) do #… end end
  89. Norm can match on atoms and tuples

  90. Conforming atoms and tuples :atom = conform!(:atom, :atom) {1, "hello"}

    = conform!( {1, "hello"}, {spec(is_integer()), spec(is_binary())}) conform!({1, 2}, {:one, :two}) ** (Norm.MismatchError) val: 1 in: 0 fails: is not an atom. val: 2 in: 1 fails: is not an atom.
  91. Conforming atoms and tuples {:ok, spec(is_binary())}

  92. Conforming atoms and tuples result_spec = one_of([ {:ok, spec(is_binary())}, {:error,

    spec(fn _ -> true end)}, ])
  93. Conforming atoms and tuples result_spec = one_of([ {:ok, spec(is_binary())}, {:error,

    spec(fn _ -> true end)}, ]) {:ok, "alice"} = conform!(User.get_name(123), result_spec) {:error, "user does not exist"} = conform!(User.get_name(-42), result_spec)
  94. We can compose specs into schema’s

  95. Norm supports Schema’s user_schema = schema(%{ user: schema(%{ name: spec(is_binary()),

    age: spec(is_integer() and &(&1 > 0)) }) })
  96. Norm supports Schema’s user_schema = schema(%{ user: schema(%{ name: spec(is_binary()),

    age: spec(is_integer() and &(&1 > 0)) }) }) input = %{user: %{name: "chris", age: 30, email: “c@keathley.io"}
  97. Norm supports Schema’s user_schema = schema(%{ user: schema(%{ name: spec(is_binary()),

    age: spec(is_integer() and &(&1 > 0)) }) }) input = %{user: %{name: "chris", age: 30, email: “c@keathley.io"} conform!(input, user_schema) => %{user: %{name: "chris", age: 30}}
  98. Norm supports Schema’s user_schema = schema(%{ user: schema(%{ name: spec(is_binary()),

    age: spec(is_integer() and &(&1 > 0)) }) }) input = %{user: %{name: "chris", age: 30, email: “c@keathley.io"} conform!(input, user_schema) => %{user: %{name: "chris", age: 30}} Can start sending us new data whenever they need to
  99. Norm supports optionality user_schema = schema(%{ user: schema(%{ name: spec(is_binary()),

    age: spec(is_integer() and &(&1 > 0)) }) })
  100. Norm supports optionality user_schema = schema(%{ user: schema(%{ name: spec(is_binary()),

    age: spec(is_integer() and &(&1 > 0)) }) }) How do you mark a key as optional?
  101. Optionality is defined by the callsite. Not the data structure.

  102. Optionality plug plug plug assigns: %{}

  103. Optionality assigns: %{} plug plug plug

  104. Optionality assigns: %{} plug plug plug Assigns a user id

  105. Optionality assigns: %{user_id: 42} plug plug plug Assigns a user

    id
  106. Optionality assigns: %{user_id: 42} plug plug plug

  107. Optionality assigns: %{user_id: 42} plug plug plug Requires the user

    id
  108. Norm supports optionality user_schema = schema(%{ user: schema(%{ name: spec(is_binary()),

    age: spec(is_integer() and &(&1 > 0)) }) })
  109. Norm supports optionality user_schema = schema(%{ user: schema(%{ name: spec(is_binary()),

    age: spec(is_integer() and &(&1 > 0)) }) }) just_age = selection(user_schema, [user: [:age]])
  110. Norm supports optionality user_schema = schema(%{ user: schema(%{ name: spec(is_binary()),

    age: spec(is_integer() and &(&1 > 0)) }) }) just_age = selection(user_schema, [user: [:age]]) conform!(%{user: %{name: "chris", age: 31}}, just_age) => %{user: %{age: 31}}
  111. Lets talk about… Contracts Data specification Testing strategies

  112. Lets talk about… Contracts Data specification Testing strategies

  113. test "rgb works correctly" do end

  114. test "rgb works correctly" do assert rgb_to_hex(0, 10, 32) end

  115. test "rgb works correctly" do assert rgb_to_hex(0, 10, 32) assert

    rgb_to_hex(nil, [:cat], "hey!") end
  116. test "rgb works correctly" do assert rgb_to_hex(0, 10, 32) assert

    rgb_to_hex(nil, [:cat], "hey!") assert rgb_to_hex(-10, 300, 0.5) end
  117. test "rgb works correctly" do assert rgb_to_hex(0, 10, 32) assert

    rgb_to_hex(nil, [:cat], "hey!") assert rgb_to_hex(-10, 300, 0.5) end Known Knowns
  118. “Writing unit tests is a waste of time” - John

    Hughes
  119. “Writing unit tests is a waste of time” - John

    Hughes - Chris Keathley
  120. We already know what kind of data we want def

    rgb, do: spec(is_integer() and (& 0 <= &1 and &1 <= 255))
  121. def rgb, do: spec(is_integer() and (& 0 <= &1 and

    &1 <= 255))
  122. def rgb, do: spec(is_integer() and (& 0 <= &1 and

    &1 <= 255)) rgb_gen = Norm.gen(rgb()) Lets generate it instead
  123. def rgb, do: spec(is_integer() and (& 0 <= &1 and

    &1 <= 255)) rgb_gen = Norm.gen(rgb()) rgb_gen |> Enum.take(5) [124, 4, 172, 217, 162]
  124. test "but good this time" do check all end end

    Generates values
  125. test "but good this time" do check all end end

  126. test "but good this time" do check all r <-

    gen(rgb()), end end
  127. test "but good this time" do check all r <-

    gen(rgb()), g <- gen(rgb()), end end
  128. test "but good this time" do check all r <-

    gen(rgb()), g <- gen(rgb()), b <- gen(rgb()) do end end
  129. test "but good this time" do check all r <-

    gen(rgb()), g <- gen(rgb()), b <- gen(rgb()) do end end What test do we write?
  130. defmodule Contractual do use ExContract requires is_integer(r) and 0 <=

    r and r <= 255 requires is_integer(g) and 0 <= g and g <= 255 requires is_integer(b) and 0 <= b and b <= 255 ensures is_binary(result) ensures String.starts_with?(result, "#") def rgb_to_hex(r, g, b) do #… end end
  131. test "but good this time" do check all r <-

    gen(rgb()), g <- gen(rgb()), b <- gen(rgb()) do end end
  132. test "but good this time" do check all r <-

    gen(rgb()), g <- gen(rgb()), b <- gen(rgb()) do assert rgb_to_hex(r, g, b) end end
  133. So now what?

  134. “Great artists steal” Ada Eiffel Haskell Python Clojure

  135. Links and references https://en.wikipedia.org/wiki/Design_by_contract https://www.clojure.org/about/spec https://www.cs.tufts.edu/~nr/cs257/archive/john-hughes/quick.pdf https://people.seas.harvard.edu/~chong/pubs/icfp17-whip.pdf https://www.youtube.com/watch?v=ROor6_NGIWU https://arxiv.org/abs/1607.05443

  136. Special Thanks! * Jeff Utter * Jason Stewart * Lance

    Halvorsen * Greg Mefford * Jeff Weiss * The Outlaws
  137. Thanks! Chris Keathley / @ChrisKeathley / c@keathley.io