Designing and Building the Oden Programming Language

Designing and Building the Oden Programming Language

This first part of this talk will introduce Oden, an experimental, statically typed, functional programming language being built for the Go ecosystem. We will look at how Oden aims to leverage the great features of Go — static linking, cross-compilation, goroutines, channels and the great set of libraries and tools — and enable higher-level abstractions, generics and a safer yet more flexible type system.

The second part will delve more deeply into the implementation of the Oden compiler. Why was it first written in Racket and then rewritten in Haskell? What pros and cons are there in writing compilers in Haskell? We will look at how the type system can help us build safe and robust intermediate representations and transformations between them.

Be279c8e94a0825d7c982e745a3b3ba9?s=128

Oskar Wickström

April 13, 2016
Tweet

Transcript

  1. 2.

    Part One: Designing Oden • Introduction • Why Oden? •

    Project Goals • Current State • What’s next?
  2. 3.

    Part Two: Building Oden • Racket & miniKanren • Rewriting

    in Haskell • Explicit Modelling • AST, IR and Transformations • Libraries and Tools
  3. 5.

    Me • Musical education • Started with PHP about 5

    years ago • .NET, Java, some FP and a lot of Web • Haskell, languages and compilers on spare time
  4. 6.

    Project Background • Started off writing LISP interpreters • Interested

    in static type systems • Began exploring statically typed LISP • Implementing Hindley-Milner inference • Reusing an existing ecosystem
  5. 8.

    The Go Programming Language • Simple language • Concurrency, channels

    • Garbage Collection • No class hierarchies • Static linking • Cross-compilation • Fast compiler • Gets shit done
  6. 9.

    The Ecosystem • Great standard library • Tools • fmt

    • Coverage • Linting • Profiling • Race detection • Lots of libraries • Lots of projects
  7. 10.

    What I’m Missing • Expressions rather than statements • Abstractions

    • Generics • Custom generic data structures • Higher-order functions • Error handling (example coming up) • nil checking • Better type inference
  8. 11.

    func (c *dropboxClient) GetFile(accessToken, dropboxSrc, localTarget string) error { url

    := "https://content.dropboxapi.com/1/files/auto" + dropboxSrc req, err := http.NewRequest("GET", url, nil) req.Header.Add("Authorization", "Bearer "+accessToken) if err != nil { return err } client := &http.Client{} res, err := client.Do(req) if err != nil { return err } if res.StatusCode == 404 { return ErrNotFound } if res.StatusCode < 200 || res.StatusCode >= 300 { return errors.New("Dropbox request failed: " + res.Status) } if err != nil { return err } f, err := os.Create(localTarget) if err != nil { return err } io.Copy(f, res.Body) if err != nil { return err } return nil }
  9. 12.

    func (c *dropboxClient) GetFile(accessToken, dropboxSrc, localTarget string) error { url

    := "https://content.dropboxapi.com/1/files/auto" + dropboxSrc req, err := http.NewRequest("GET", url, nil) req.Header.Add("Authorization", "Bearer "+accessToken) if err != nil { return err } client := &http.Client{} res, err := client.Do(req) if err != nil { return err } if res.StatusCode == 404 { return ErrNotFound } if res.StatusCode < 200 || res.StatusCode >= 300 { return errors.New("Dropbox request failed: " + res.Status) } if err != nil { return err } f, err := os.Create(localTarget) if err != nil { return err } io.Copy(f, res.Body) if err != nil { return err } return nil }
  10. 13.

    func (c *dropboxClient) GetFile(accessToken, dropboxSrc, localTarget string) error { url

    := "https://content.dropboxapi.com/1/files/auto" + dropboxSrc req, err := http.NewRequest("GET", url, nil) req.Header.Add("Authorization", "Bearer "+accessToken) if err != nil { return err } client := &http.Client{} res, err := client.Do(req) if err != nil { return err } if res.StatusCode == 404 { return ErrNotFound } if res.StatusCode < 200 || res.StatusCode >= 300 { return errors.New("Dropbox request failed: " + res.Status) } if err != nil { return err } f, err := os.Create(localTarget) if err != nil { return err } io.Copy(f, res.Body) if err != nil { return err } return nil }
  11. 22.

    Erlang Pattern Matching dolphin() -> receive {From, do_a_flip} -> From

    ! "How about no?"; {From, fish} -> From ! "So long and thanks for all the fish!"; _ -> io:format("Heh, we're smarter than you humans.~n") end. http://learnyousomeerlang.com/the-hitchhikers-guide-to-concurrency
  12. 24.

    Haskell Exhaustiveness Checking data Message = DoAFlip | Fish |

    Ohai dolphin :: Message -> String dolphin msg = case msg of DoAFlip -> "How about no?" Fish -> "So long and thanks for all the fish!"
  13. 25.

    $ ghc -Wall Dolphin.hs ... Dolphin.hs:6:15: Warning: Pattern match(es) are

    non-exhaustive In a case alternative: Patterns not matched: Ohai
  14. 26.

    Covering all cases data Message = DoAFlip | Fish |

    Ohai dolphin :: Message -> String dolphin msg = case msg of DoAFlip -> "How about no?" Fish -> "So long and thanks for all the fish!" Ohai -> "Heh, we're smarter than you humans."
  15. 28.
  16. 32.

    Operators 1 + 2 +y -(1 - 2) 4 /

    2 (100 - 50) / 2 1 == 2 1 != 2 !x && (true || (1 == 2)) "Foo" ++ "Bar"
  17. 34.
  18. 38.
  19. 39.

    factorial(n) = { if n < 2 then { 1

    } else { n * factorial(n - 1) } } Control Flow with Blocks
  20. 40.

    Curried Functions plus(x, y) = x + y plusTen =

    plus(10) thirty = plusTen(20)
  21. 43.

    Curried Higher-Order Functions plus(x, y) = x + y twice(f,

    x) = f(f(x)) plusTwenty = twice(plus(10)) forty = plusTwenty(20)
  22. 46.

    Tuple Literals jessica = ("Jessica", 31) frank = ("Frank", 26)

    people = []{jessica, frank, ("Maxime", 25)}
  23. 47.

    Type Signatures n : int n = 1 f :

    forall a. a -> a f(x) = x
  24. 48.

    Imports from Go package main import html main() = {

    println(html.EscapeString("Simon & Garfunkel")) // Simon &amp; Garfunkel }
  25. 49.

    Curried Foreign Functions // multi-parameter func in Go func PlusInGo(x,

    y int) int { return x + y } // curried in Oden incr = pkg.PlusInGo(1)
  26. 50.

    Constructing Records sweden = { official = "Kingdom Of Sweden",

    population = 9858794, capital = "Stockholm" } usa = { official = "United States of America", population = 322369319, capital = "Washington D.C." }
  27. 53.

    package main bonnie = { firstName = "Bonnie", lastName =

    "Swanson", age = 37 } peter = { firstName = "Peter", lastName = "Griffin" } fullName(person) = person.firstName ++ " " ++ person.lastName main() = { println(fullName(bonnie)) println(fullName(peter)) }
  28. 54.

    Adding Types for Persons bonnie : { firstName : string,

    lastName : string, age : int } bonnie = { firstName = "Bonnie", lastName = "Swanson", age = 37 } peter : { firstName : string, lastName : string } peter = { firstName = "Peter", lastName = "Griffin", age = 41 }
  29. 56.

    The Inferred Type of fullName fullName : forall r. {

    firstName : string, lastName : string | r } -> string fullName(person) = person.firstName ++ " " ++ person.lastName
  30. 61.

    What’s Next? • Record extension, better interoperability with Go structs

    • Protocols (like Haskell type classes, but integrating with Go interfaces) • Support all primitive types, channels, maps (requires protocols for overloaded operators) • Explicit type conversions (also based on protocols) • Support for requiring Oden packages • Better Go codegen (right now a lot of anonymous functions) • Pattern matching • Better validation and error messages
  31. 62.
  32. 63.
  33. 65.

    The Starting Point • Started digging in to Hindley-Milner type

    inference • Learned about miniKanren (http://minikanren.org/) • The Reasoned Schemer (Daniel P. Friedman, William E. Byrd and Oleg Kiselyov) • Hindley-Milner using Constraint Solving • Decided to use Racket (PLT Scheme) • First step: statically typed LISP
  34. 66.

    Super Fast Intro to miniKanren • relations are functions that

    returns goals • goals either fail or succeed • fresh creates new unbound logic variables • run gets answers for a relation • conde does logical disjunction (OR) • == unifies terms
  35. 68.
  36. 71.

    conde > (run 2 (q) (== q 2) (conde [(==

    q 1)] [(== q 2)])) '(2)
  37. 72.

    Lists > (run 1 (q) (fresh (x) (== '(1 2

    3) x) (== x (list 1 q 3)))) '(2)
  38. 73.

    Scheme Quasiquotes > `(0 1 2) '(0 1 2) >

    `(1 ,(+ 1 2) 4) '(1 3 4)
  39. 74.

    Quasiquoting and Logic Variables > (run 1 (q) (fresh (x)

    (== '(1 2 3) x) (== x `(1 ,q 3)))) '(2)
  40. 77.

    pairo > (define (pairo f s o) (== `(,f ,s)

    o)) > (run 1 (q) (fresh (x y) (pairo x y q) (== x 1) (== y 2))) '((1 2))
  41. 79.

    https://en.wikipedia.org/wiki/Abstract_syntax_tree “In computer science, an abstract syntax tree (AST) […]

    is a tree representation of the abstract syntactic structure of source code written in a programming language. ”
  42. 83.

    Input and Output > (infer '(fn (x) x)) '((fn ((x

    : _.0)) (x : _.0)) : (_.0 -> _.0))
  43. 84.

    infero (define (infero expr env t) (conde [(fresh (x) (symbolo

    expr) (lookupo expr env x) (== `(,expr : ,x) t))] ... ))
  44. 85.

    infero (cont.) (define (infero expr env t) (conde ... [(fresh

    (f ft f-ignore a at a-ignore x et) (== `(,f ,a) expr) (infero f env ft) (infero a env at) (== `(,f-ignore : (,x -> ,et)) ft) (== `(,a-ignore : ,x) at) (== `((,ft ,at) : ,et) t))] ... ))
  45. 86.

    My miniKanren Experience • No, or very primitive, error handling

    • Type safety became a concern as it grew • No success with Typed Racket • The state of miniKanren • No canonical implementation (?) • Documentation • Missing and/or broken functions
  46. 87.

    I miss my Monads • miniKanren is basically a constraint

    solver monad • Cannot combine with other monads (error handling, state etc) • https://github.com/webyrd/error-handling-relational- interpreter/blob/master/error-interp-specific-all.scm
  47. 88.

    Racket Itches • Tooling • Racket REPL in Emacs (Geiser)

    slow • Crashed every now and then • Compiled program startup time: ~400ms
  48. 89.
  49. 91.

    Write You A Haskell • Inferencer based on Write You

    A Haskell, by Stephen Diehl • Rewritten to return typed expressions http://dev.stephendiehl.com/fun/006_hindley_milner.html
  50. 93.

    https://en.wikipedia.org/wiki/Intermediate_representation “An Intermediate representation (IR) is the data structure or

    code used internally by a compiler or virtual machine to represent source code.”
  51. 94.

    Explicit Modelling • Represent valid cases explicitly in data types

    • Prohibit invalid cases in data types • Distinct types for transformations inputs and outputs • Transformations will basically write themselves & • The type system will help you
  52. 95.

    Syntax Excerpt data Expr = Symbol SourceInfo Identifier | Application

    SourceInfo Expr [Expr] | Fn SourceInfo [NameBinding] Expr | Block SourceInfo [Expr] ...
  53. 96.

    Syntax Excerpt data Expr = Symbol SourceInfo Identifier | Application

    SourceInfo Expr [Expr] | Fn SourceInfo [NameBinding] Expr | Block SourceInfo [Expr] ...
  54. 97.

    Untyped IR data Expr = Symbol (Metadata SourceInfo) Identifier |

    Application (Metadata SourceInfo) Expr [Expr] | Fn (Metadata SourceInfo) NameBinding Expr | Block (Metadata SourceInfo) [Expr] ...
  55. 98.

    Untyped IR data Expr = Symbol (Metadata SourceInfo) Identifier |

    Application (Metadata SourceInfo) Expr [Expr] | Fn (Metadata SourceInfo) NameBinding Expr | Block (Metadata SourceInfo) [Expr] ...
  56. 100.

    Curried Functions // In Oden we write: (x, y, z)

    -> x // ...but we get: (x) -> (y) -> (z) -> x
  57. 102.

    Exploding Functions explodeExpr' (Fn sourceInfo (param:params) body) = Untyped.Fn (Metadata

    sourceInfo) (explodeNameBinding param) <$> explodeExpr' (Fn sourceInfo params body) ...
  58. 104.

    Example Code package identity // A polymorphic function. identity(x) =

    x // Use it with different types. three = identity(3) hello = identity("hello")
  59. 105.

    Resolve Imports resolveImports :: Importer -> Untyped.Package [Untyped.ImportReference] -> IO

    (Either PackageImportError (Untyped.Package [Core.ImportedPackage], [UnsupportedTypesWarning]))
  60. 106.

    Resolve Imports resolveImports :: Importer -> Untyped.Package [Untyped.ImportReference] -> IO

    (Either PackageImportError (Untyped.Package [Core.ImportedPackage], [UnsupportedTypesWarning]))
  61. 108.

    After Type Inference ~$ oden print-inferred identity.oden package identity identity

    : forall a. a -> a identity = ((x) -> x) three : int three = identity(3) hello : string hello = identity("hello")
  62. 112.

    After Monomorphization ~$ oden print-compiled identity.oden package identity identity_inst_int_to_int :

    int -> int identity_inst_int_to_int = ((x) -> x) identity_inst_string_to_string : string -> string identity_inst_string_to_string = ((x) -> x) hello : string hello = identity_inst_string_to_string("hello") three : int three = identity_inst_int_to_int(3)
  63. 113.
  64. 114.

    After Codegen package identity import fmt "fmt" func print(x interface{})

    { fmt.Print(x) } func println(x interface{}) { fmt.Println(x) } func identity_inst_int_to_int(x int) int { return x } func identity_inst_string_to_string(x string) string { return x } var hello string = identity_inst_string_to_string("hello") var three int = identity_inst_int_to_int(3)
  65. 115.

    After Codegen package identity import fmt "fmt" func print(x interface{})

    { fmt.Print(x) } func println(x interface{}) { fmt.Println(x) } func identity_inst_int_to_int(x int) int { return x } func identity_inst_string_to_string(x string) string { return x } var hello string = identity_inst_string_to_string("hello") var three int = identity_inst_int_to_int(3)
  66. 117.

    Haskell Libraries • parsec • Good to begin with, expressive

    • Might rewrite with alex and happy • mtl (monad transformers library) • Used throughout the compiler • wl-pprint • Better control over whitespace than standard Text.PrettyPrint
  67. 118.

    Haskell Tooling • hdevtools or ghc-mod with <insert editor here>

    • Warnings • HLint • -Wall • HSpec with nodemon/guard
  68. 119.
  69. 120.

    Summary • Oden: a functional programming language built for the

    Go ecosystem • The current state of Oden, what’s next • Racket, miniKanren and the Haskell rewrite • Explicit Modelling and Transformations in Haskell • Haskell libraries and tooling
  70. 121.

    Monads as Design Pattern The Door to Real-World Functional Programming

    Wednesday, April 27 http://www.foocafe.org/event/monads-as-design-pattern
  71. 122.

    Contribute • Try it out • Discuss • https://groups.google.com/forum/#!forum/oden-lang •

    #oden at Freenode • Report issues on GitHub • Send pull requests
  72. 123.