Evolving Existing Languages: The Typed Racket Experience

A talk given in November 2010 at Indiana University

Transcript

1. Evolving Existing Languages The Typed Racket Experience Sam Tobin-Hochstadt PLT @ Northeastern University Nov 29, 2010 Indiana University
2. Programs Evolve
3. Automated Testing (for ([f (in-directory (find-collects-dir))] #:when (file-exists? f)) (system* "racket" "-t" (path->string f))) 3 lines
4. DrDr
5. BuildBot
6. Languages Must Evolve
7. What is Language Evolution? Smooth Interoperation Easy Migration Comprehensive Reuse
8. Typed Racket Smooth Interoperation - Typed/Untyped Interaction Easy Migration - Types for Existing Programs Comprehensive Reuse - Building on Existing Languages
9. Typed Racket in 3 Slides
10. Hello World hello #lang racket (printf "Hello World\n")
11. Hello World hello #lang typed/racket (printf "Hello World\n")
12. Functions ack #lang racket ; ack : Integer Integer -> Integer (define (ack m n) (cond [(<= m 0) (+ n 1)] [(<= n 0) (ack (- m 1) 1)] [else (ack (- m 1) (ack m (- n 1)))])) (ack 2 3)
13. Functions ack #lang typed/racket (: ack : Integer Integer -> Integer) (define (ack m n) (cond [(<= m 0) (+ n 1)] [(<= n 0) (ack (- m 1) 1)] [else (ack (- m 1) (ack m (- n 1)))])) (ack 2 3)
14. Modules ack #lang racket ; ack : Integer Integer -> Integer (define (ack m n) (cond [(<= m 0) (+ n 1)] [(<= n 0) (ack (- m 1) 1)] [else (ack (- m 1) (ack m (- n 1)))])) compute #lang racket (require ack) (ack 2 3)
15. Modules ack #lang typed/racket (: ack : Integer Integer -> Integer) (define (ack m n) (cond [(<= m 0) (+ n 1)] [(<= n 0) (ack (- m 1) 1)] [else (ack (- m 1) (ack m (- n 1)))])) compute #lang racket (require ack) (ack 2 3)
16. Modules ack #lang racket ; ack : Integer Integer -> Integer (define (ack m n) (cond [(<= m 0) (+ n 1)] [(<= n 0) (ack (- m 1) 1)] [else (ack (- m 1) (ack m (- n 1)))])) compute #lang typed/racket (require [ack (Integer Integer -> Integer)]) (ack 2 3)
17. Modules ack #lang typed/racket (: ack : Integer Integer -> Integer) (define (ack m n) (cond [(<= m 0) (+ n 1)] [(<= n 0) (ack (- m 1) 1)] [else (ack (- m 1) (ack m (- n 1)))])) compute #lang typed/racket (require ack) (ack 2 3)
18. Interoperating with Existing Languages
19. Typed & Untyped server #lang typed/racket (: add5 (Number -> Number)) (define (add5 x) (+ x 5)) client #lang racket (require server) (add5 7)
20. Typed & Untyped Untyped code can make mistakes server #lang typed/racket (: add5 (Number -> Number)) (define (add5 x) (+ x 5)) client #lang racket (require server) (add5 "seven")
21. Typed & Untyped Untyped code can make mistakes server #lang typed/racket (: add5 (Number -> Number)) (define (add5 x) ) client #lang racket (require server) (add5 "seven") +: expects type <number> as 1st argument
22. Typed & Untyped Catch errors dynamically at the boundary server #lang typed/racket (: add5 (Number -> Number)) (define (add5 x) (+ x 5)) client #lang racket (require server) (add5 "seven") client broke the contract on add5
23. Typed & Untyped Catch errors dynamically at the boundary server #lang racket (define (add5 x) "x plus 5") client #lang typed/racket (require server [add5 (Number -> Number)]) (add5 7) server interface broke the contract on add5
24. Typed & Untyped Catch errors dynamically at the boundary server #lang typed/racket (: addx (Number -> (Number -> Number))) (define (addx x) (lambda (y) (+ x y))) client #lang racket (require server) ((addx 7) 'bad) client broke the contract on add5
25. The Blame Theorem If the program raises a contract error, the blame is not assigned to a typed module.
26. The Blame Theorem Well-typed modules can’t get blamed. [Wadler & Findler]
27. The Blame Theorem Allows local reasoning about typed modules, without changing untyped modules. Choose how much static checking you want.
28. Checking Existing Languages Occurrence Typing Variable-Arity Refinement Types Ad-Hoc Data
29. Types for Existing Programs occur #lang typed/racket (define-type Peano (U 'Z (List 'S Peano))) (: convert : Peano -> Number) (define (convert n) (cond [(symbol? n) 0] [else (add1 (convert (rest n)))]))
30. Types for Existing Programs refine #lang typed/racket (: check : String -> (Refinement sql-safe?)) (define (check s) (if (sql-safe? s) s (error "unsafe string!")))
31. Types for Existing Programs union #lang typed/racket (define-type BT (U Number (Pair BT BT))) (: sizeof : BT -> Number) (define (sizeof b) (if (number? b) 1 (+ 1 (sizeof (car b)) (sizeof (cdr b)))))
32. Types for Existing Programs varar #lang typed/racket (: wrap (∀ (B A ...) (A ... -> B) -> (A ... -> B))) (define (wrap f) (λ args (printf "args are: ~a\n" args) (apply f args)))
33. Occurrence Typing (define-type Peano (U 'Z (List 'S Peano))) (: convert : Peano -> Number) (define (convert n) (cond [(symbol? n) 0] [else (add1 (convert (rest n)))]))
34. Occurrence Typing (define-type Peano (U 'Z (List 'S Peano))) (: convert : Peano -> Number) (define (convert n) (cond [(symbol? n) 0] [else (add1 (convert (rest n)))])) n : Peano n n : Peano
35. Occurrence Typing (define-type Peano (U 'Z (List 'S Peano))) (: convert : Peano -> Number) (define (convert n) (cond [(symbol? n) 0] [else (add1 (convert (rest n)))])) n : Peano n n : Peano
36. Occurrence Typing (define-type Peano (U 'Z (List 'S Peano))) (: convert : Peano -> Number) (define (convert n) (cond [(symbol? n) 0] [else (add1 (convert (rest n)))])) n : 'Z 0 n : 'Z
37. Occurrence Typing (define-type Peano (U 'Z (List 'S Peano))) (: convert : Peano -> Number) (define (convert n) (cond [(symbol? n) 0] [else (add1 (convert (rest n)))])) n : (List 'S Peano) n n : (List 'S Peano)
38. Occurrence Typing (: combine : (U String Symbol) (U String Symbol) -> String) (define (combine s s*) (cond [(and (string? s) (string? s*)) (string-append s s*)] [(string? s) (string-append s (symbol->string s*))] [(string? s*) (string-append (symbol->string s) s*)] [else (string-append (symbol->string s) (symbol->string s*))]))
39. Occurrence Typing (: combine : (U String Symbol) (U String Symbol) -> String) (define (combine s s*) (cond [(and (string? s) (string? s*)) (string-append s s*)] [(string? s) (string-append s (symbol->string s*))] [(string? s*) (string-append (symbol->string s) s*)] [else (string-append (symbol->string s) (symbol->string s*))])) s : (U String Symbol) s* : (U String Symbol) s s : (U String Symbol) s* s* : (U String Symbol)
40. Occurrence Typing (: combine : (U String Symbol) (U String Symbol) -> String) (define (combine s s*) (cond [(and (string? s) (string? s*)) (string-append s s*)] [(string? s) (string-append s (symbol->string s*))] [(string? s*) (string-append (symbol->string s) s*)] [else (string-append (symbol->string s) (symbol->string s*))])) s : String s* : String s s : String s* s* : String
41. Occurrence Typing (: combine : (U String Symbol) (U String Symbol) -> String) (define (combine s s*) (cond [(and (string? s) (string? s*)) (string-append s s*)] [(string? s) (string-append s (symbol->string s*))] [(string? s*) (string-append (symbol->string s) s*)] [else (string-append (symbol->string s) (symbol->string s*))])) s : String s* : Symbol s s : String s* s* : Symbol
42. Occurrence Typing (: combine : (U String Symbol) (U String Symbol) -> String) (define (combine s s*) (cond [(and (string? s) (string? s*)) (string-append s s*)] [(string? s) (string-append s (symbol->string s*))] [(string? s*) (string-append (symbol->string s) s*)] [else (string-append (symbol->string s) (symbol->string s*))])) s : Symbol s* : String s s : Symbol s* s* : String
43. Occurrence Typing (: combine : (U String Symbol) (U String Symbol) -> String) (define (combine s s*) (cond [(and (string? s) (string? s*)) (string-append s s*)] [(string? s) (string-append s (symbol->string s*))] [(string? s*) (string-append (symbol->string s) s*)] [else (string-append (symbol->string s) (symbol->string s*))])) s : Symbol s* : Symbol s s : Symbol s* s* : Symbol
44. Occurrence Typing (define-type Peano (U 'Z (List 'S Peano))) (: convert : Peano -> Number) (define (convert n) (cond [(symbol? n) 0] [else (add1 (convert (rest n)))]))
45. Occurrence Typing (define-type Peano (U 'Z (List 'S Peano))) (: convert : Peano -> Number) (define (convert n) (cond [(symbol? n) 0] [else (add1 (convert (rest n)))])) n : (List 'S Peano) (add1 (convert (rest n))) n : (List 'S Peano)
46. Occurrence Typing (define-type Peano (U 'Z (List 'S Peano))) (: convert : Peano -> Number) (define (convert n) (cond [(symbol? n) 0] [else (add1 (convert (rest n)))])) ⊢ (List 'S Peano) @ n (add1 (convert (rest n))) ⊢ (List 'S Peano) @ n
47. Occurrence Typing (define-type Peano (U 'Z (List 'S Peano))) (: convert : Peano -> Number) (define (convert n) (cond [(symbol? n) 0] [else (add1 (convert (rest n)))])) ⊢ (List 'S Peano) @ n (symbol? n) ⊢ Symbol @ n
48. Occurrence Typing (define-type Peano (U 'Z (List 'S Peano))) (: convert : Peano -> Number) (define (convert n) (cond [(symbol? n) 0] [else (add1 (convert (rest n)))])) ⊢ Peano @ n ⊢ Symbol @ n ⊢ (List 'S Peano) @ n (symbol? n) ⊢ Symbol @ n
49. Occurrence Typing (define-type Peano (U 'Z (List 'S Peano))) (: convert : Peano -> Number) (define (convert n) (cond [(symbol? n) 0] [else (add1 (convert (rest n)))])) ⊢ (U 'Z (List 'S Peano)) @ n ⊢ Symbol @ n ⊢ (List 'S Peano) @ n (symbol? n) ⊢ Symbol @ n
50. Occurrence Typing (: combine : (U String Symbol) (U String Symbol) -> String) (define (combine s s*) (cond [(and (string? s) (string? s*)) (string-append s s*)] [(string? s) (string-append s (symbol->string s*))] [(string? s*) (string-append (symbol->string s) s*)] [else (string-append (symbol->string s) (symbol->string s*))]))
51. Occurrence Typing (: combine : (U String Symbol) (U String Symbol) -> String) (define (combine s s*) (cond [(and (string? s) (string? s*)) (string-append s s*)] [(string? s) (string-append s (symbol->string s*))] [(string? s*) (string-append (symbol->string s) s*)] [else (string-append (symbol->string s) (symbol->string s*))])) ⊢ String @ s ⊢ String @ s* (string-append s s*) ⊢ String @ s ⊢ String @ s*
52. Occurrence Typing (: combine : (U String Symbol) (U String Symbol) -> String) (define (combine s s*) (cond [(and (string? s) (string? s*)) (string-append s s*)] [(string? s) (string-append s (symbol->string s*))] [(string? s*) (string-append (symbol->string s) s*)] [else (string-append (symbol->string s) (symbol->string s*))])) ⊢ String @ s ∧ String @ s* ⊢ String @ s ⊢ String @ s* (string? s) String @ s (string? s*) String @ s*
53. Occurrence Typing (: combine : (U String Symbol) (U String Symbol) -> String) (define (combine s s*) (cond [(and (string? s) (string? s*)) (string-append s s*)] [(string? s) (string-append s (symbol->string s*))] [(string? s*) (string-append (symbol->string s) s*)] [else (string-append (symbol->string s) (symbol->string s*))])) ⊢ Symbol @ s* (string-append s (symbol->string s*)) Symbol @ s*
54. Occurrence Typing (: combine : (U String Symbol) (U String Symbol) -> String) (define (combine s s*) (cond [(and (string? s) (string? s*)) (string-append s s*)] [(string? s) (string-append s (symbol->string s*))] [(string? s*) (string-append (symbol->string s) s*)] [else (string-append (symbol->string s) (symbol->string s*))])) ⊢ String @ s ⊢ String @ s ⊃ String @ s* ⊢ Symbol @ s* (and (string? s) (string? s*)) String @ s ⊃ String @ s* (string? s) String @ s
55. Building on Existing Languages
56. Languages as Libraries name #lang typed/racket ....
57. Languages as Libraries (module name typed/racket (#%module-begin ....) )
58. Languages as Libraries typed/racket #lang racket (define-syntax (#%module-begin stx) ....) (module name typed/racket (#%module-begin ....) ) #%module-begin #%module-begin
59. Defining A Language (define-syntax (#%module-begin stx) (syntax-parse stx [(_ forms ...) (let () (for ([f (syntax->list #'(forms ...))]) (typecheck f)) #'(#%plain-module-begin forms ...))]))
60. Defining A Language (define-syntax (#%module-begin stx) (syntax-parse stx [(_ forms ...) (let () (for ([f (syntax->list #'(forms ...))]) (typecheck f)) #'(#%plain-module-begin forms ...))])) f
61. Defining A Language (define-syntax (#%module-begin stx) (syntax-parse stx [(_ forms ...) (let ([forms* (local-expand #'(forms ...))]) (for ([f (rest (syntax->list forms*))]) (typecheck f)) #'(#%plain-module-begin #,forms*))]))
62. Defining A Language (define-syntax (#%module-begin stx) (syntax-parse stx [(_ forms ...) (let ([forms* (local-expand #'(forms ...))]) (for ([f (rest (syntax->list forms*))]) (typecheck f)) (let ([forms** (optimize forms*)]) #'(#%plain-module-begin #,forms**)))]))
63. Typechecking (define (typecheck f) (syntax-parse f ; variables [v:identifier (lookup-type #'v)] ; abstractions [(lambda (x) e) (define t (syntax-property #'x 'type-label)) (set-type! #'x t) (typecheck #'e)] ; about 10 more cases ....))
64. Typechecking (define (typecheck f) (syntax-parse f ; variables [v:identifier (lookup-type #'v)] ; abstractions [(lambda (x) e) (define t (syntax-property #'x 'type-label)) (set-type! #'x t) (typecheck #'e)] ; about 10 more cases ....))
65. Typechecking (define (typecheck f) (syntax-parse f ; variables [v:identifier (lookup-type #'v)] ; abstractions [(lambda (x) e) (define t (syntax-property #'x 'type-label)) (set-type! #'x t) (typecheck #'e)] ; about 10 more cases ....))
66. Optimization #lang typed/racket (: norm : Float Float -> Float) (define (norm x y) (flsqrt (+ (* x x) (* y y))))
67. Optimization #lang typed/racket (: norm : Float Float -> Float) (define (norm x y) (unsafe-flsqrt (unsafe-fl+ (unsafe-fl* x x) (unsafe-fl* y y))))
68. Typed Racket as Research Agenda
69. Contracts (Vectorof (Integer -> Integer)) (class/c ....) (unit/c ....) Analysis using contracts With Stevie Strickland, Robby Findler, Matthew Flatt, David Van Horn
70. Types First-class Classes Generic Operations Variable-Arity Functions .... and Inference With Stevie Strickland, Asumu Takikawa, Matthias Felleisen
71. Macros syntax-parse Macro Debugging Defensible Abstractions With Ryan Culpepper
72. Education Beginner-Level Error Messages #lang typed/racket (first 3) Type Checker: Polymorphic function first could not be applied to arguments: Domains: (Pairof a (Listof b)) (Listof a) Arguments: Positive-Fixnum With Eli Barzilay, Matthias Felleisen
73. Optimization Low-Level Operations for • Structure Representation • Memory Allocation • ... With Vincent St-Amour, Matthew Flatt
74. Jesse Tov Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
75. Jesse Tov Felix Klock Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
76. Jesse Tov Felix Klock Eli Barzilay Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
77. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
78. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
79. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
80. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
81. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
82. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Jay McCarthy Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
83. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Jay McCarthy Matthew Flatt Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
84. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Jay McCarthy Matthew Flatt Hari Prashanth Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
85. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Jay McCarthy Matthew Flatt Hari Prashanth Ryan Culpepper Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
86. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Jay McCarthy Matthew Flatt Hari Prashanth Ryan Culpepper David Van Horn Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
87. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Jay McCarthy Matthew Flatt Hari Prashanth Ryan Culpepper David Van Horn Stevie Strickland Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
88. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Jay McCarthy Matthew Flatt Hari Prashanth Ryan Culpepper David Van Horn Stevie Strickland Matthias Felleisen Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
89. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Jay McCarthy Matthew Flatt Hari Prashanth Ryan Culpepper David Van Horn Stevie Strickland Matthias Felleisen Vincent St-Amour Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
90. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Jay McCarthy Matthew Flatt Hari Prashanth Ryan Culpepper David Van Horn Stevie Strickland Matthias Felleisen Vincent St-Amour Shriram Krishnamurthi Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
91. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Jay McCarthy Matthew Flatt Hari Prashanth Ryan Culpepper David Van Horn Stevie Strickland Matthias Felleisen Vincent St-Amour Shriram Krishnamurthi Students at Northeastern and Brown Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
92. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Jay McCarthy Matthew Flatt Hari Prashanth Ryan Culpepper David Van Horn Stevie Strickland Matthias Felleisen Vincent St-Amour Shriram Krishnamurthi Students at Northeastern and Brown And everyone who has tried Typed Racket! Thanks! Available from racket-lang.org Supported by the Mozilla Foundation
93. Jesse Tov Felix Klock Eli Barzilay Ivan Gazeau Neil Toronto Aaron Turon Carl Eastlund Robby Findler Jay McCarthy Matthew Flatt Hari Prashanth Ryan Culpepper David Van Horn Stevie Strickland Matthias Felleisen Vincent St-Amour Shriram Krishnamurthi Students at Northeastern and Brown And everyone who has tried Typed Racket! Thanks! Available from racket-lang.org Supported by the Mozilla Foundation