Configuration With Dhall

Transcript

1. Dhall: An Introduction Markus Hauck @markus1189
2. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Motivation 1 library 2 build-depends: base >=4.7 && <5 3 , aeson 4 , text 5 , lens <-----------+ 6 , rio | 7 , more... | 8 executable my-app | 9 build-depends: base >=4.7 && <5 | 10 , aeson -- <-+ | 11 , text -- \ | 12 , lens -- +-- SAME! --+ 13 , rio -- / 14 , more... -- <-+ Markus Hauck @markus1189 Dhall: An Introduction 1
3. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Motivation 1 # docker-compose.yml 2 services: 3 service1: 4 environment: # \ 5 LOGSTASH_HOST: logstash # |--+ 6 LOGSTASH_PORT: 5607 # / | 7 # | 8 service2: # | 9 environment: # \ | 10 LOGSTASH_HOST: logstash # |--+ SAME! 11 LOGSTASH_PORT: 5607 # / | 12 # | 13 service42: # | 14 environment: # \ | 15 LOGSTASH_HOST: logstash # |--+ 16 LOGSTASH_PORT: 5607 # / Markus Hauck @markus1189 Dhall: An Introduction 2
4. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Motivation 1 # liquibase migration file 2 - changeSet: 3 changes: 4 - createTable: 5 tableName: my-table 6 columns: 7 - column: 8 name: firstname 9 type: varchar 10 constraints: 11 nullable: false 12 - column: 13 name: lastname 14 type: varchar 15 constraints: 16 nullable: false 17 - column: 18 name: address 19 type: varchar 20 constraints: 21 nullable: false Markus Hauck @markus1189 Dhall: An Introduction 3
5. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion What Is Dhall A configuration language guaranteed to terminate dhall is a total programming language specialized to configuration files • from Gabriel Gonzalez (Pipes, Turtle, …) • language: github.com/dhall-lang/dhall-lang • haskell: github.com/dhall-lang/dhall-haskell • This presentation: dhall-1.15.0 Markus Hauck @markus1189 Dhall: An Introduction 4
6. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Features • Haskell integration — Dhall expressions can be marshalled into Haskell • Total — Programs always terminate and will never hang • Safe — Programs never crash or throw exceptions • Distributed — Expressions can reference other expressions by URL or path • Strong normalization — Every expression can be reduced to a normal form • Statically typed — Configuration files can be validated ahead-of-time • Strongly typed — No coercions, casts or subtyping • Built-in data types — Includes lists, anonymous records and anonymous unions Markus Hauck @markus1189 Dhall: An Introduction 5
7. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion First Example 1 let twentyOne = 21 in 2 let answer = twentyOne + twentyOne in 3 { x = 1 4 , y = 2 : Natural -- with type 5 , foo = "foo" -- text 6 , bar = "bar" 7 , foobar = "foo" ++ "bar" -- append 8 } Markus Hauck @markus1189 Dhall: An Introduction 6
8. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion First Example — Evaluation 1 $ dhall < example.dhall 2 { x : Natural 3 , y : Natural 4 , foo : Text 5 , bar : Text 6 , foobar : Text 7 } 8 9 { x = 1 10 , y = 2 11 , foo = "foo" 12 , bar = "bar" 13 , foobar = "foobar" 14 } Markus Hauck @markus1189 Dhall: An Introduction 7
9. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion The Dhall Tool 1 Usage: dhall ([version] | [resolve] | [type] | [normalize] | [repl] | [diff] | 2 [hash] | [lint] | [format] | [freeze]) [--explain] [--plain] 3 Interpreter for the Dhall language 4 5 Available options: 6 -h,--help Show this help text 7 --explain Explain error messages in more detail 8 --plain Disable syntax highlighting 9 10 Available commands: 11 version Display version 12 resolve Resolve an expression's imports 13 type Infer an expression's type 14 normalize Normalize an expression 15 repl Interpret expressions in a REPL 16 diff Render the difference between the normal form of two 17 expressions 18 hash Compute semantic hashes for Dhall expressions 19 lint Improve Dhall code 20 format Formatter for the Dhall language 21 freeze Add hashes to all import statements of an expression Markus Hauck @markus1189 Dhall: An Introduction 8
10. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Dhall Language Markus Hauck @markus1189 Dhall: An Introduction 9
11. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Language Features • Integers • Naturals • Doubles • Booleans • Strings • Lists • Records • Optionals • Unions • Functions • Imports • Let expressions Markus Hauck @markus1189 Dhall: An Introduction 10
12. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Dhall Is Not A General Purpose Language • before we go in deeper, important disclaimer • dhall is not a general purpose language • you might get frustrated if you want to program in it • that’s not the purpose Markus Hauck @markus1189 Dhall: An Introduction 11
13. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Integers 1 +1 2 -1 3 [+1, -1, +0] 4 +1 : Integer 5 +1 + 5 Markus Hauck @markus1189 Dhall: An Introduction 12
14. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Doubles 1 1.0 2 1.337 3 :type -4.2 4 [-1.0, 2.0, 3.0] Markus Hauck @markus1189 Dhall: An Introduction 13
15. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Booleans 1 True 2 False 3 True || False 4 True && False 5 True == True 6 True != True 7 if True then 42 else 1337 8 if False then 42 else 1337 Markus Hauck @markus1189 Dhall: An Introduction 14
16. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Text 1 "this is a text" 2 "Won't work: ${1}" 3 "Will work: ${Natural/show (1 + 1)}" 4 "Hello, " ++ "World!" Markus Hauck @markus1189 Dhall: An Introduction 15
17. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Reading Types 1 -- :type Natural/fold 2 Natural 3 -> forall (natural : Type) 4 -> forall (succ : natural -> natural) 5 -> forall (zero : natural) 6 -> natural Markus Hauck @markus1189 Dhall: An Introduction 16
18. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Naturals 1 :type 1 2 1 3 2 : Natural 4 2 : Integer 5 [1, 2, 3] 6 1 + 1 7 1 - 1 8 1 * 1 9 Natural/even 4 10 Natural/fold 11 Natural/fold 3 Text (\(t : Text) -> t ++ "?") "" Markus Hauck @markus1189 Dhall: An Introduction 17
19. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Let Bindings 1 let x = 100 in 2 let y = (200 : Natural) in 3 let z : Natural = 300 in 4 [ x, y, z] 5 -- [100, 200, 300] Markus Hauck @markus1189 Dhall: An Introduction 18
20. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Lists 1 [] : List Natural 2 [ 1, 2, 3 ] 3 [ 1, 2 ] # [ 3, 4 ] 4 List/length Text ["foo", "bar"] 5 List/head Natural [1, 2, 3] 6 List/last Natural [1, 2, 3] 7 List/reverse Natural [1, 2, 3] 8 :type List/fold 9 :type List/build 1 let plus = (\(x : Natural) -> \(y : Natural) -> x + y) in 2 List/fold Natural [ 1, 2, 3] Natural plus 0 3 4 -- => 6 Markus Hauck @markus1189 Dhall: An Introduction 19
21. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Optionals 1 [] : Optional Integer 2 [] : Optional Natural 3 ["foo"] : Optional Text 4 :type Optional/fold 1 let optSome = [ 1 ] : Optional Natural in 2 let optNone = [ ] : Optional Natural in 3 let onJust = (\(n : Natural) -> "It's a " ++ Natural/show n ++ "!") in 4 [ Optional/fold Natural optSome Text onJust "Nothing :(" 5 , Optional/fold Natural optNone Text onJust "Nothing :(" 6 ] 7 8 -- => [ "It's a 1!", "Nothing :(" ] Markus Hauck @markus1189 Dhall: An Introduction 20
22. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Records 1 { 1 = 2 } -- invalid 2 { x = 1, y = 2.0, z = +3 } 3 { foo = "foo", y = "bar"} 4 {=} : {} 5 { x = 1, y = 2, z = 3}.x 6 { x = 1, y = 2, z = 3}.y 7 { x = 1, y = 2, z = 3}.{x, y} Markus Hauck @markus1189 Dhall: An Introduction 21
23. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Combining Records 1 • not recursive • changes type if necessary, right side has priority • useful to specify defaults 1 { x = 1 } // { y = 2 } 2 { x = 1 } // { x = 2 } 3 { x = 1 } // { x = +2 } 4 { x = 1 } // { x = "foo" } 5 { x = 1 } // {=} 6 { = } // { x = 1 } 7 { x = { y = 1 }} // { x = "nope" } Markus Hauck @markus1189 Dhall: An Introduction 22
24. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Combining Records 2 • recursive • error on field collision (not a record) 1 { x = 1 } /\ { x = 2 } -- nope 2 { x = { y = 1 } } /\ { x = { z = 2 } } Markus Hauck @markus1189 Dhall: An Introduction 23
25. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Unions 1 < Left = 1 | Right : Text > 2 < Left : Natural | Right = "text" > 3 \(a : Type) -> < None = {=} | Some : a > 4 \(a : Type) -> \(x : a) -> < None : {} | Some = x > 5 < Dog : Text | Cat : Text | Bird : Text | Fish = "nemo" > Markus Hauck @markus1189 Dhall: An Introduction 24
26. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Unions • thankfully there is constructors 1 let Animal = constructors < Dog : Text | 2 Cat : Text | 3 Bird : Text | 4 Fish : Text > in 5 [ Animal.Dog "barney" 6 , Animal.Cat "garfield" 7 , Animal.Bird "tweety" 8 ] Markus Hauck @markus1189 Dhall: An Introduction 25
27. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Environment Variables 1 let Schema = { home : Text 2 , user : Text 3 , id : Optional Natural 4 } in 5 let Some = https://raw.githubusercontent.com/dhall-lang/Prelud 6 let None = https://raw.githubusercontent.com/dhall-lang/Prelud 7 { home = env:HOME as Text 8 , user = env:USER as Text 9 , id = Some Natural env:IDENTITY ? None Natural 10 } : Schema Markus Hauck @markus1189 Dhall: An Introduction 26
28. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Imports • imports are a central feature • import via path or url • terms and types Markus Hauck @markus1189 Dhall: An Introduction 27
29. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Imports 1 -- people 2 [ { first = "Markus", last = "Hauck" } 3 , { first = "Lotte", last = "Rie" } 4 , { first = "Rainer", last = "Zufall" } 5 ] 1 -- Person 2 { first : Text 3 , last : Text 4 } 1 ./people : List ./Person Markus Hauck @markus1189 Dhall: An Introduction 28
30. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Imports • import via url 1 let Person = https://bit.ly/dhall-person-type in 2 let people = https://bit.ly/dhall-people in 3 people : List Person Markus Hauck @markus1189 Dhall: An Introduction 29
31. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Imports • but is that safe? • you can add checksums to imports • dhall freeze to the rescue 1 let Person = https://bit.ly/dhall-person-type 2 sha256:19b699c1ec8b2dc12b75c835db0d2b2faed1a911226146c545c98bb0f1890fca in 3 let people = https://bit.ly/dhall-people 4 sha256:d754b3d6074b056d7ff244c4625e3d55cfcaf0a493eceeed915474946c2ec9b3 in 5 people : List Person Markus Hauck @markus1189 Dhall: An Introduction 30
32. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Imports: Alternatives • scenario: what if an import fails • we would like to have a fallback / mirror 1 let Person = https://bit.ly/dhall-person-type ? 2 https://cutt.ly/dhall-person-type in 3 let people = https://bit.ly/dhall-people in 4 people : List Person Markus Hauck @markus1189 Dhall: An Introduction 31
33. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion The Prelude • Dhall also has a Prelude with lots of functions • import a-la-carte or get everything • https://github.com/dhall-lang/Prelude Markus Hauck @markus1189 Dhall: An Introduction 32
34. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion The Prelude 1 let Prelude = 2 https://raw.githubusercontent.com/dhall-lang/Prelude/master/package.dhall in 3 4 Prelude.`List`.iterate 5 Natural (\(x : Natural) -> x + 1) 0 Markus Hauck @markus1189 Dhall: An Introduction 33
35. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Using Types As A Schema • need a schema? • use types! 1 { remote : Text 2 , port : Natural 3 , timeout : Optional Natural 4 , consumerKey : Text 5 , accessToken : Text 6 } 1 ./myConfig : ./Schema Markus Hauck @markus1189 Dhall: An Introduction 34
36. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Dhall Is Total • total: every program terminates (eventually) • seems like a heavy restriction • some pretty cool optimizations • reduce/optimize programs with partial input possible Markus Hauck @markus1189 Dhall: An Introduction 35
37. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Dhall Is Total 1 let Prelude = 2 https://raw.githubusercontent.com/dhall-lang/Prelude/master/package.dhall in 3 4 \(a : Type) -> 5 \(xs : List a) -> 6 \(f : a -> a) -> 7 Prelude.`List`.map a a f xs 1 ./partial.dhall Text ["a", "b", "c"] 2 -- \ (f : Text -> Text) -> [ f "a", f "b", f "c" ] Markus Hauck @markus1189 Dhall: An Introduction 36
38. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Dhall And Haskell Markus Hauck @markus1189 Dhall: An Introduction 37
39. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Mapping To Haskell • have: type checked configuration • need: make values accessible in Haskell • dhall-haskell is the Haskell implementation of Dhall • map Dhall types to Haskell types • the best: also unions and functions! Markus Hauck @markus1189 Dhall: An Introduction 38
40. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Mapping To Haskell Dhall Haskell Text Text Natural Natural Integer Int Double Double List a Vector a Optional a Maybe a records ADT unions ADT functions functions Markus Hauck @markus1189 Dhall: An Introduction 39
41. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Reading Persons 1 -- people 2 [ { first = "Markus", last = "Hauck" } 3 , { first = "Lotte", last = "Rie" } 4 , { first = "Rainer", last = "Zufall" } 5 ] Markus Hauck @markus1189 Dhall: An Introduction 40
42. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Reading Persons 1 data Person = Person 2 { personFirst :: Text 3 , personLast :: Text 4 } deriving (Show, Generic) 5 6 instance Interpret Person 7 8 personInput :: Text -> IO [Person] 9 personInput = input (autoWith opts) 10 where 11 opts = 12 defaultInterpretOptions 13 {fieldModifier = Text.toLower . Text.drop 6} Markus Hauck @markus1189 Dhall: An Introduction 41
43. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Reading Persons 1 main :: IO () 2 main = do 3 p <- personInput "../dhall/persons/people" 4 print p 5 -- Results in the following output 6 {- 7 [ Person {personFirst = "Markus", personLast = "Hauck"} 8 , Person {personFirst = "Lotte", personLast = "Rie"} 9 , Person {personFirst = "Rainer", personLast = "Zufall"} 10 ] 11 -} Markus Hauck @markus1189 Dhall: An Introduction 42
44. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Reading Unions 1 let FooBarBaz = constructors < Foo : { foo : Text } 2 | Bar : { bar : Natural } 3 | Baz : {} 4 > in 5 [ FooBarBaz.Foo { foo = "a foo" } 6 , FooBarBaz.Bar { bar = 42 } 7 , FooBarBaz.Baz {=} 8 ] Markus Hauck @markus1189 Dhall: An Introduction 43
45. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Reading Unions 1 data FooBarBaz 2 = Foo { foo :: Text } 3 | Bar { bar :: Natural } 4 | Baz 5 deriving (Show, Generic) 6 7 instance Interpret FooBarBaz 8 9 fooBarBazInput :: Text -> IO [FooBarBaz] 10 fooBarBazInput = input auto • reading the previous dhall expression: 1 [Foo {foo = "a foo"},Bar {bar = 42},Baz] Markus Hauck @markus1189 Dhall: An Introduction 44
46. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Reading Functions • think a moment how cool that is 1 interpolate :: Text -> IO (Text -> Text) 2 interpolate = input auto 1 main2 = do 2 f <- interpolate "\\(t : Text) -> t ++ \"!!!\"" 3 print (f "Hello, World") 4 -- Hello, World!!! Markus Hauck @markus1189 Dhall: An Introduction 45
47. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Other Formats Markus Hauck @markus1189 Dhall: An Introduction 46
48. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Other Formats • unfortunately not everything is in Haskell • lucky: Dhall can be converted into other formats • JSON, YAML, Bash, Cabal, … Markus Hauck @markus1189 Dhall: An Introduction 47
49. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion But Why Not X??? • JSON • no comments • repetition • annoying to write • no types! • YAML • it has comments! • but: even more awful • think you know YAML? Let’s see Markus Hauck @markus1189 Dhall: An Introduction 48
50. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion YAML — Make A Guess 1 DE: Germany 2 EN: England 3 NO: Norway 4 ON: Ontario, Canada Markus Hauck @markus1189 Dhall: An Introduction 49
51. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion YAML — Boolean 1 - yes 2 - Yes 3 - YES 4 - true 5 - True 6 - TRUE 7 - on 8 - On 9 - ON Markus Hauck @markus1189 Dhall: An Introduction 50
52. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion YAML 1 python: 3.5.3 2 postgres: 9.3 Markus Hauck @markus1189 Dhall: An Introduction 51
53. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion YAML • very hard to edit, easy to mess up • indentation error does still parse, but differently • 9 different ways to write multiline strings http://bit.ly/2lT0NEd • non-intuitive quoting rules • … Markus Hauck @markus1189 Dhall: An Introduction 52
54. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion From Dhall • the good news: you can convert Dhall • JSON • YAML • Bash • Nix Markus Hauck @markus1189 Dhall: An Introduction 53
55. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Demo Markus Hauck @markus1189 Dhall: An Introduction 54
56. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Other Formats • JSON, YAML: https://github.com/dhall-lang/dhall-json • Bash: https://github.com/dhall-lang/dhall-bash • Text Templates: https://github.com/dhall-lang/dhall-text • Nix: https://github.com/dhall-lang/dhall-nix • Cabal: https://github.com/dhall-lang/dhall-to-cabal • K8s: https://github.com/dhall-lang/dhall-kubernetes Markus Hauck @markus1189 Dhall: An Introduction 55
57. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Dhall Gotchas • hard if the target JSON/YAML is unprincipled • most of the translation support is very new • Dhall takes some time getting used to • some documentation missing (you can help!) Markus Hauck @markus1189 Dhall: An Introduction 56
58. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Dhall Advantages • get rid of YAML madness • type of your configuration as documentation • know if your config is correct • keep the config DRY and build abstraction • best case: map directly to your Haskell types • use functions, unions and advanced constructs Markus Hauck @markus1189 Dhall: An Introduction 57
59. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion References • language: github.com/dhall-lang/dhall-lang • haskell: github.com/dhall-lang/dhall-haskell • tutorial: hackage.haskell.org/package/dhall/docs/ Dhall-Tutorial.html • this talk: https://github.com/markus1189/dhall-intro Markus Hauck @markus1189 Dhall: An Introduction 58
60. Introduction Dhall Language Dhall And Haskell Other Formats Conclusion Thank you for your attention Markus Hauck @markus1189 Dhall: An Introduction 59