Tweet
Tweet

Slide 1

Slide 1 text

A Type Inferencer for ML In 200 Lines of Scala Ionuț G. Stan — CODΞCΔMP Iași — October 2016

Slide 2

Slide 2 text

About Me

Slide 3

Slide 3 text

• Software Developer at Eloquentix About Me

Slide 4

Slide 4 text

• Software Developer at Eloquentix • I work mostly with Scala About Me

Slide 5

Slide 5 text

• Software Developer at Eloquentix • I work mostly with Scala • I like FP, programming languages, compilers About Me

Slide 6

Slide 6 text

• Software Developer at Eloquentix • I work mostly with Scala • I like FP, programming languages, compilers • I started the Bucharest FP meetup group About Me

Slide 7

Slide 7 text

• Software Developer at Eloquentix • I work mostly with Scala • I like FP, programming languages, compilers • I started the Bucharest FP meetup group • I occasionally blog at igstan.ro About Me

Slide 8

Slide 8 text

• Software Developer at Eloquentix • I work mostly with Scala • I like FP, programming languages, compilers • I started the Bucharest FP meetup group • I occasionally blog at igstan.ro • Happy to discuss tech stuff at [email protected] About Me

Slide 9

Slide 9 text

Plan

Slide 10

Slide 10 text

Plan • Compilers Overview

Slide 11

Slide 11 text

Plan • Compilers Overview • Vehicle Language: µML

Slide 12

Slide 12 text

Plan • Compilers Overview • Vehicle Language: µML • Wand's Type Inference Algorithm

Slide 13

Slide 13 text

Plan • Compilers Overview • Vehicle Language: µML • Wand's Type Inference Algorithm • Intuition

Slide 14

Slide 14 text

Plan • Compilers Overview • Vehicle Language: µML • Wand's Type Inference Algorithm • Intuition • Code

Slide 15

Slide 15 text

Compilers Overview

Slide 16

Slide 16 text

Compiler Compilers Overview

Slide 17

Slide 17 text

Compiler Compilers Overview Source Language

Slide 18

Slide 18 text

Target Language Compiler Compilers Overview Source Language

Slide 19

Slide 19 text

Target Language Compiler (fn a => a) 2 Compilers Overview Source Language

Slide 20

Slide 20 text

Target Language Compiler (fn a => a) 2 (function(a){return a})(2) Compilers Overview Source Language

Slide 21

Slide 21 text

Compilers Overview uage T Compiler ) 2 (funct

Slide 22

Slide 22 text

Parsing T Compiler ) 2 Parser uage (funct

Slide 23

Slide 23 text

Abstract Syntax Tree T Compiler ) 2 Parser APP FUN a VAR a INT 2 Abstract Syntax Tree (AST) uage (funct

Slide 24

Slide 24 text

Code Generation T Compiler ) 2 Parser CodeGen APP FUN a VAR a INT 2 Abstract Syntax Tree (AST) uage (funct

Slide 25

Slide 25 text

Many Intermediate Phases T Compiler ) 2 Parser CodeGen ... AST (funct uage

Slide 26

Slide 26 text

Type Checking T Compiler ) 2 Parser CodeGen Type Checker AST ... uage (funct

Slide 27

Slide 27 text

Today's Talk T Compiler ) 2 Parser CodeGen Type Checker AST Today's T alk ... uage (funct

Slide 28

Slide 28 text

Type Checking vs Type Inference

Slide 29

Slide 29 text

• Type Checking • Ensures declared types are used consistently • All types must be declared • Traverse AST and compare def site with use site • Type Inference • Ensures consistency as well • Types need not be declared, though; are deduced! • Two main classes of algorithms • We'll see one instance today Type Checking vs Inference

Slide 30

Slide 30 text

Vehicle Language: µML

Slide 31

Slide 31 text

• Surface Syntax • What does the language look like • Type System • What types the language supports Vehicle Language: µML

Slide 32

Slide 32 text

1. Integers: 1, 23, 456, etc. µML — Surface Syntax

Slide 33

Slide 33 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. µML — Surface Syntax

Slide 34

Slide 34 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. 3. Booleans: true and false µML — Surface Syntax

Slide 35

Slide 35 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. 3. Booleans: true and false 4. Single-argument anonymous functions: fn a => a µML — Surface Syntax

Slide 36

Slide 36 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. 3. Booleans: true and false 4. Single-argument anonymous functions: fn a => a 5. Function application: inc 42 µML — Surface Syntax

Slide 37

Slide 37 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. 3. Booleans: true and false 4. Single-argument anonymous functions: fn a => a 5. Function application: inc 42 6. If expressions: if cond then t else f µML — Surface Syntax

Slide 38

Slide 38 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. 3. Booleans: true and false 4. Single-argument anonymous functions: fn a => a 5. Function application: inc 42 6. If expressions: if cond then t else f 7. Addition and subtraction: a + b, a - b µML — Surface Syntax

Slide 39

Slide 39 text

1. Integers: 1, 23, 456, etc. 2. Identifiers (only letters): inc, cond, a, etc. 3. Booleans: true and false 4. Single-argument anonymous functions: fn a => a 5. Function application: inc 42 6. If expressions: if cond then t else f 7. Addition and subtraction: a + b, a - b 8. Parenthesized expressions: (a + b) µML — Surface Syntax

Slide 40

Slide 40 text

9. Let blocks/expressions:
 
 let
 val name = ...
 in
 name
 end µML — Surface Syntax

Slide 41

Slide 41 text

Small Example let val inc = fn a => a + 1 in inc 42 end

Slide 42

Slide 42 text

µML — Language Types

Slide 43

Slide 43 text

1. Integer type: int µML — Language Types

Slide 44

Slide 44 text

1. Integer type: int 2. Boolean type: bool µML — Language Types

Slide 45

Slide 45 text

1. Integer type: int 2. Boolean type: bool 3. Function type: int -> bool µML — Language Types

Slide 46

Slide 46 text

Today's Algorithm Overview

Slide 47

Slide 47 text

Wand's Algorithm Overview

Slide 48

Slide 48 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST

Slide 49

Slide 49 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST fn isZero => if isZero 1 then 2 else 3

Slide 50

Slide 50 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 51

Slide 51 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 52

Slide 52 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 53

Slide 53 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 54

Slide 54 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 55

Slide 55 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 56

Slide 56 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 57

Slide 57 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 58

Slide 58 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 59

Slide 59 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN isZero IF APP VAR isZero INT 1 INT 2 INT 3 fn isZero => if isZero 1 then 2 else 3

Slide 60

Slide 60 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN IF APP AR ero INT 1 INT 2 INT 3 FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3

Slide 61

Slide 61 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUN IF APP AR ero INT 1 INT 2 INT 3 FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3

Slide 62

Slide 62 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 Constraint Set

Slide 63

Slide 63 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 t1 Constraint Set

Slide 64

Slide 64 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int t1 t5 Constraint Set

Slide 65

Slide 65 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool t1 t5 t4 Constraint Set

Slide 66

Slide 66 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool ≡ int t1 t5 t4 t6 Constraint Set

Slide 67

Slide 67 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool ≡ int ≡ int t1 t5 t4 t6 t7 Constraint Set

Slide 68

Slide 68 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool ≡ int ≡ int ≡ t3 t1 t5 t4 t6 t7 t6 Constraint Set

Slide 69

Slide 69 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 t1 t5 t4 t6 t7 t6 t7 Constraint Set

Slide 70

Slide 70 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ t1 → t3 t1 t5 t4 t6 t7 t6 t7 t2 Constraint Set

Slide 71

Slide 71 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 ≡ t5 → t4 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ t1 → t3 t1 t5 t4 t6 t7 t6 t7 t2 Constraint Set

Slide 72

Slide 72 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST ≡ t5 → t4 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ t1 → t3 t1 t5 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 73

Slide 73 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : t5 → t4 t1 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ t1 → t3 t5 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 74

Slide 74 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : t5 → t4 t1 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ t1 → t3 t5 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 75

Slide 75 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : t5 → t4 t1 ≡ int ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ (t5 → t4) → t3 t5 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 76

Slide 76 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : t5 → t4 : int t1 t5 ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ (t5 → t4) → t3 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 77

Slide 77 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : t5 → t4 : int t1 t5 ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ (t5 → t4) → t3 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 78

Slide 78 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → t4 : int t1 t5 ≡ bool ≡ int ≡ int ≡ t3 ≡ t3 ≡ (int → t4) → t3 t4 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 79

Slide 79 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → t4 : int : bool t1 t5 t4 ≡ int ≡ int ≡ t3 ≡ t3 ≡ (int → t4) → t3 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 80

Slide 80 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → t4 : int : bool t1 t5 t4 ≡ int ≡ int ≡ t3 ≡ t3 ≡ (int → t4) → t3 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 81

Slide 81 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool t1 t5 t4 ≡ int ≡ int ≡ t3 ≡ t3 ≡ (int → bool) → t3 t6 t7 t6 t7 t2 Constraint Set Solution Map

Slide 82

Slide 82 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int t1 t5 t4 t6 ≡ int ≡ t3 ≡ t3 ≡ (int → bool) → t3 t7 t6 t7 t2 Constraint Set Solution Map

Slide 83

Slide 83 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int t1 t5 t4 t6 ≡ int ≡ t3 ≡ t3 ≡ (int → bool) → t3 t7 t6 t7 t2 Constraint Set Solution Map

Slide 84

Slide 84 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int t1 t5 t4 t6 ≡ int ≡ t3 ≡ t3 ≡ (int → bool) → t3 t7 int t7 t2 Constraint Set Solution Map

Slide 85

Slide 85 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int t1 t5 t4 t6 t7 ≡ t3 ≡ t3 ≡ (int → bool) → t3 int t7 t2 Constraint Set Solution Map

Slide 86

Slide 86 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int t1 t5 t4 t6 t7 ≡ t3 ≡ t3 ≡ (int → bool) → t3 int t7 t2 Constraint Set Solution Map

Slide 87

Slide 87 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int t1 t5 t4 t6 t7 ≡ t3 ≡ t3 ≡ (int → bool) → t3 int int t2 Constraint Set Solution Map

Slide 88

Slide 88 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int : int t1 t5 t4 t6 t7 t3 ≡ t3 ≡ (int → bool) → t3 int t2 Constraint Set Solution Map

Slide 89

Slide 89 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int : int t1 t5 t4 t6 t7 t3 ≡ t3 ≡ (int → bool) → t3 int t2 Constraint Set Solution Map

Slide 90

Slide 90 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int : int t1 t5 t4 t6 t7 t3 ≡ int ≡ (int → bool) → int int t2 Constraint Set Solution Map

Slide 91

Slide 91 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int : int t1 t5 t4 t6 t7 t3 ≡ int ≡ (int → bool) → int int t2 Constraint Set Solution Map

Slide 92

Slide 92 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int : int t1 t5 t4 t6 t7 t3 ≡ (int → bool) → int t2 Constraint Set Solution Map

Slide 93

Slide 93 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST : int → bool : int : bool : int : int : int : (int → bool) → int t1 t5 t4 t6 t7 t3 t2 Constraint Set Solution Map

Slide 94

Slide 94 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST FUNt2 isZerot1 IFt3 APPt4 VARt1 isZero INTt5 1 INTt6 2 INTt7 3 : int → bool : int : bool : int : int : int : (int → bool) → int t1 t5 t4 t6 t7 t3 t2 Solution Map

Slide 95

Slide 95 text

Wand's Algorithm Overview Type Annotation Constraint Generation Constraint Solving Parsing AST fn isZero => if isZero 1 then 2 else 3 (int -> bool) -> int

Slide 96

Slide 96 text

• As I said, there are two main classes. • Constraint-based: we've just seen one example — Wand's algorithm. • Substitution-based: all phases are interleaved, e.g., Algorithm W. Type Inference Algorithms

Slide 97

Slide 97 text

• Sunil Kothari and James L. Caldwell. Type Reconstruction Algorithms - A Survey • Mitchell Wand. A simple algorithm and proof for type inference • Bastiaan Heeren, Jurriaan Hage and Doaitse Swierstra. Generalizing Hindley-Milner Type Inference Algorithms • Oleg Kiselyov and Chung-chieh Shan. Interpreting Types as Abstract Values • Shriram Krishnamurthi. Programming Languages: Application and Interpretation, chapter 15 • Shriram Krishnamurthi. Programming Languages: Application and Interpretation, lecture 24 • Shriram Krishnamurthi. Programming Languages: Application and Interpretation, lecture 25 • Bastiaan Heeren. Top Quality Type Error Messages • Stephen Diehl. Write You a Haskell, chapter 6 • Andrew Appel. Modern Compiler Implementation in ML, chapter 16 • Benjamin Pierce. Types and Programming Languages, chapter 22 • Martin Odersky. Scala by Example, chapter 16 • Danny Gratzer. https://github.com/jozefg/hm • Arlen Cox. ML Type Inference and Unification! • Radu Rugină. CS 312, Type Inference Resources

Slide 98

Slide 98 text

github.com / igstan / codecamp-2016

Slide 99

Slide 99 text

Thank You!