Tweet
Tweet

Slide 1

Slide 1 text

A Type Inferencer for ML In 200 Lines of Scala Ionuț G. Stan — igstan.ro — [email protected]

Slide 2

Slide 2 text

• Software Developer at • Worked with Scala for the past 5 years • FP, programming languages, compilers • Mostly-tech blog at igstan.ro About Me

Slide 3

Slide 3 text

Plan

Slide 4

Slide 4 text

Plan • Compilers Overview

Slide 5

Slide 5 text

Plan • Compilers Overview • Vehicle Language: µML

Slide 6

Slide 6 text

Plan • Compilers Overview • Vehicle Language: µML • Wand's Type Inference Algorithm

Slide 7

Slide 7 text

Plan • Compilers Overview • Vehicle Language: µML • Wand's Type Inference Algorithm • Intuition

Slide 8

Slide 8 text

Plan • Compilers Overview • Vehicle Language: µML • Wand's Type Inference Algorithm • Intuition • Code

Slide 9

Slide 9 text

Compilers Overview

Slide 10

Slide 10 text

Compiler Compilers Overview

Slide 11

Slide 11 text

Compiler Compilers Overview Source Language

Slide 12

Slide 12 text

Target Language Compiler Compilers Overview Source Language

Slide 13

Slide 13 text

Target Language Compiler (fn a => a) 2 Compilers Overview Source Language

Slide 14

Slide 14 text

Target Language Compiler (fn a => a) 2 (function(a){return a})(2) Compilers Overview Source Language

Slide 15

Slide 15 text

Compilers Overview uage T Compiler ) 2 (funct

Slide 16

Slide 16 text

Parsing T Compiler ) 2 Parser uage (funct

Slide 17

Slide 17 text

Abstract Syntax Tree T Compiler ) 2 Parser APP FUN a VAR a INT 2 Abstract Syntax Tree (AST) uage (funct

Slide 18

Slide 18 text

Code Generation T Compiler ) 2 Parser CodeGen APP FUN a VAR a INT 2 Abstract Syntax Tree (AST) uage (funct

Slide 19

Slide 19 text

Many Intermediate Phases T Compiler ) 2 Parser CodeGen ... AST (funct uage

Slide 20

Slide 20 text

Type Checking T Compiler ) 2 Parser CodeGen Type Checker AST ... uage (funct

Slide 21

Slide 21 text

Today's Talk T Compiler ) 2 Parser CodeGen Type Checker AST Today's T alk ... uage (funct

Slide 22

Slide 22 text

Type Checking vs Type Inference

Slide 23

Slide 23 text

• Type Checking • Ensures declared types are used consistently • All types must be declared • Traverse AST and compare def site with use site • Type Inference • Ensures consistency as well • Types need not be declared, though; are deduced • Two main classes of algorithms • We'll see one instance today Type Checking vs Inference

Slide 24

Slide 24 text

Vehicle Language: µML

Slide 25

Slide 25 text

• Surface Syntax • What does the language look like • Type System • What types the language supports Vehicle Language: µML

Slide 26

Slide 26 text

1. Integers: 1, 23, 456, etc. µML — Surface Syntax

Slide 27

Slide 27 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. µML — Surface Syntax

Slide 28

Slide 28 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. 3. Booleans: true and false µML — Surface Syntax

Slide 29

Slide 29 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. 3. Booleans: true and false 4. Single-argument anonymous functions: fn a => a µML — Surface Syntax

Slide 30

Slide 30 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. 3. Booleans: true and false 4. Single-argument anonymous functions: fn a => a 5. Function application: inc 42 µML — Surface Syntax

Slide 31

Slide 31 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. 3. Booleans: true and false 4. Single-argument anonymous functions: fn a => a 5. Function application: inc 42 6. If expressions: if cond then t else f µML — Surface Syntax

Slide 32

Slide 32 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. 3. Booleans: true and false 4. Single-argument anonymous functions: fn a => a 5. Function application: inc 42 6. If expressions: if cond then t else f 7. Addition and subtraction: a + b, a - b µML — Surface Syntax

Slide 33

Slide 33 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. 3. Booleans: true and false 4. Single-argument anonymous functions: fn a => a 5. Function application: inc 42 6. If expressions: if cond then t else f 7. Addition and subtraction: a + b, a - b 8. Parenthesized expressions: (a + b) µML — Surface Syntax

Slide 34

Slide 34 text

9. Let blocks/expressions:
 
 let
 val name = ...
 in
 name
 end µML — Surface Syntax

Slide 35

Slide 35 text

Small Example let val inc = fn a => a + 1 in inc 42 end

Slide 36

Slide 36 text

µML — Language Types

Slide 37

Slide 37 text

1. Integer type: int µML — Language Types

Slide 38

Slide 38 text

1. Integer type: int 2. Boolean type: bool µML — Language Types

Slide 39

Slide 39 text

1. Integer type: int 2. Boolean type: bool 3. Function type: int -> bool µML — Language Types

Slide 40

Slide 40 text

Today's Algorithm Overview

Slide 41

Slide 41 text

Wand's Algorithm Overview

Slide 42

Slide 42 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST

Slide 43

Slide 43 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST fn isZero => if isZero 1 then 2 else 3

Slide 44

Slide 44 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 45

Slide 45 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 46

Slide 46 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 47

Slide 47 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 48

Slide 48 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 49

Slide 49 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 50

Slide 50 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 51

Slide 51 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 52

Slide 52 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 53

Slide 53 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 54

Slide 54 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN IF APP AR ero INT 1 INT 2 INT 3 FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3

Slide 55

Slide 55 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN IF APP AR ero INT 1 INT 2 INT 3 FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3

Slide 56

Slide 56 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 Constraint Set

Slide 57

Slide 57 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 t1 Constraint Set

Slide 58

Slide 58 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int t1 t5 Constraint Set

Slide 59

Slide 59 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool t1 t5 t4 Constraint Set

Slide 60

Slide 60 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool ≡ int t1 t5 t4 t6 Constraint Set

Slide 61

Slide 61 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool ≡ int ≡ int t1 t5 t4 t6 t7 Constraint Set

Slide 62

Slide 62 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool ≡ int ≡ int ≡ t3 t1 t5 t4 t6 t7 t6 Constraint Set

Slide 63

Slide 63 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 t1 t5 t4 t6 t7 t6 t7 Constraint Set

Slide 64

Slide 64 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ t1 → t3 t1 t5 t4 t6 t7 t6 t7 t2 Constraint Set

Slide 65

Slide 65 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ t1 → t3 t1 t5 t4 t6 t7 t6 t7 t2 Constraint Set

Slide 66

Slide 66 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST ≡ t5 → t4 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ t1 → t3 t1 t5 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 67

Slide 67 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : t5 → t4 t1 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ t1 → t3 t5 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 68

Slide 68 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : t5 → t4 t1 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ t1 → t3 t5 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 69

Slide 69 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : t5 → t4 t1 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ (t5 → t4) → t3 t5 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 70

Slide 70 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : t5 → t4 : int t1 t5 ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ (t5 → t4) → t3 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 71

Slide 71 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : t5 → t4 : int t1 t5 ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ (t5 → t4) → t3 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 72

Slide 72 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → t4 : int t1 t5 ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ (int → t4) → t3 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 73

Slide 73 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → t4 : int : bool t1 t5 t4 ≡ int ≡ int ≡ t3 ≡ t3 ≡ (int → t4) → t3 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 74

Slide 74 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → t4 : int : bool t1 t5 t4 ≡ int ≡ int ≡ t3 ≡ t3 ≡ (int → t4) → t3 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 75

Slide 75 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool t1 t5 t4 ≡ int ≡ int ≡ t3 ≡ t3 ≡ (int → bool) → t3 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 76

Slide 76 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int t1 t5 t4 t6 ≡ int ≡ t3 ≡ t3 ≡ (int → bool) → t3 t7 t6 t7 t2 Constraint Set Solution Map

Slide 77

Slide 77 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int t1 t5 t4 t6 ≡ int ≡ t3 ≡ t3 ≡ (int → bool) → t3 t7 t6 t7 t2 Constraint Set Solution Map

Slide 78

Slide 78 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int t1 t5 t4 t6 ≡ int ≡ t3 ≡ t3 ≡ (int → bool) → t3 t7 int t7 t2 Constraint Set Solution Map

Slide 79

Slide 79 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int t1 t5 t4 t6 t7 ≡ t3 ≡ t3 ≡ (int → bool) → t3 int t7 t2 Constraint Set Solution Map

Slide 80

Slide 80 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int t1 t5 t4 t6 t7 ≡ t3 ≡ t3 ≡ (int → bool) → t3 int t7 t2 Constraint Set Solution Map

Slide 81

Slide 81 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int t1 t5 t4 t6 t7 ≡ t3 ≡ t3 ≡ (int → bool) → t3 int int t2 Constraint Set Solution Map

Slide 82

Slide 82 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int : int t1 t5 t4 t6 t7 t3 ≡ t3 ≡ (int → bool) → t3 int t2 Constraint Set Solution Map

Slide 83

Slide 83 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int : int t1 t5 t4 t6 t7 t3 ≡ t3 ≡ (int → bool) → t3 int t2 Constraint Set Solution Map

Slide 84

Slide 84 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int : int t1 t5 t4 t6 t7 t3 ≡ int ≡ (int → bool) → int int t2 Constraint Set Solution Map

Slide 85

Slide 85 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int : int t1 t5 t4 t6 t7 t3 ≡ int ≡ (int → bool) → int int t2 Constraint Set Solution Map

Slide 86

Slide 86 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int : int t1 t5 t4 t6 t7 t3 ≡ (int → bool) → int t2 Constraint Set Solution Map

Slide 87

Slide 87 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int : int : (int → bool) → int t1 t5 t4 t6 t7 t3 t2 Constraint Set Solution Map

Slide 88

Slide 88 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 : int → bool : int : bool : int : int : int : (int → bool) → int t1 t5 t4 t6 t7 t3 t2 Solution Map

Slide 89

Slide 89 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST fn isZero => if isZero 1 then 2 else 3 (int -> bool) -> int

Slide 90

Slide 90 text

• As I said, there are two main classes. • Constraint-based: we've just seen one example — Wand's algorithm. • Substitution-based: all phases are interleaved, e.g., Algorithm W. Type Inference Algorithms

Slide 91

Slide 91 text

• Sunil Kothari and James L. Caldwell. Type Reconstruction Algorithms - A Survey • Mitchell Wand. A simple algorithm and proof for type inference • Bastiaan Heeren, Jurriaan Hage and Doaitse Swierstra. Generalizing Hindley-Milner Type Inference Algorithms • Oleg Kiselyov and Chung-chieh Shan. Interpreting Types as Abstract Values • Shriram Krishnamurthi. Programming Languages: Application and Interpretation, chapter 15 • Shriram Krishnamurthi. Programming Languages: Application and Interpretation, lecture 24 • Shriram Krishnamurthi. Programming Languages: Application and Interpretation, lecture 25 • Bastiaan Heeren. Top Quality Type Error Messages • Stephen Diehl. Write You a Haskell, chapter 6 • Andrew Appel. Modern Compiler Implementation in ML, chapter 16 • Benjamin Pierce. Types and Programming Languages, chapter 22 • Martin Odersky. Scala by Example, chapter 16 • Danny Gratzer. https://github.com/jozefg/hm • Arlen Cox. ML Type Inference and Unification • Radu Rugină. CS 312, Type Inference Resources

Slide 92

Slide 92 text

github.com / igstan / typelevel-nyc-2017

Slide 93

Slide 93 text

Thank You!

Slide 94

Slide 94 text

Questions!