Building a Programming Language 101

Slide 1

Slide 1 text

Building a Programming Language 101 @giosakti

Slide 2

Slide 2 text

Hi, My name is Gio @giosakti giosakti

Slide 3

Slide 3 text

My Company

Slide 4

Slide 4 text

Communities (1) ps: we’re looking for speakers for the next meetup on july 25th ! mention me

Slide 5

Slide 5 text

Communities (2) http://ruby.id

Slide 6

Slide 6 text

Communities (2) October 6th-7th CFP Open: ruby.id/cfp Support our cause: ruby.id/perkumpulan

Slide 7

Slide 7 text

Building a Programming Language 101 @giosakti

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

How many of you know & understand 5 or more of these terms - Turing Completeness - Lexer - Parser - AST - IR - DSL - LLVM - Covfefe - Compiled vs Interpreted Raise your hands!

Slide 10

Slide 10 text

The Truth Is…

Slide 11

Slide 11 text

So many things to discuss, So little time.. Intro Compiled vs Interpreted Lexer Lex, Yacc Parser AST IR LLVM GPL vs DSL 25 mins …

Slide 12

Slide 12 text

I’ll give pointers on where & what to begin instead We can discuss later or you can come to the id-ruby meetup :D

Slide 13

Slide 13 text

Why is all of this important? The $1,000,000 question

Slide 14

Slide 14 text

Immersion

Slide 15

Slide 15 text

Immersion Deeper understanding + learning experience

Slide 16

Slide 16 text

Reduce the ‘magical’ feeling A.K.A unexpected behavior

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

Build your own OK for learning purpose, but don’t reinvent the wheel

Slide 19

Slide 19 text

Most popular programming languages are very old and/or has considerable backing 34 Years 21 Years 22 Years 3 Years, but Backed by Apple

Slide 20

Slide 20 text

Contribute to Open Source

Slide 21

Slide 21 text

DSL This is why I learned this topic I’ll explain in detail later

Slide 22

Slide 22 text

So.. Languages..

Slide 23

Slide 23 text

“Languages can shape the way we think” Sapir-whorf hypotheses

Slide 24

Slide 24 text

In case of programming language

Slide 25

Slide 25 text

GAP What you want to do What computer will actually do

Slide 26

Slide 26 text

Natural Language Processing Hi computer, I want you to say “ABC” “ABC”

Slide 27

Slide 27 text

In natural language processing, Computer understands you! Hi computer, I want you to say “ABC” “ABC”

Slide 28

Slide 28 text

Structured Language Processing def start puts “ABC” end “ABC”

Slide 29

Slide 29 text

Now you’re the one that try more to understand computer def start puts “ABC” end “ABC”

Slide 30

Slide 30 text

Building a structured (programming) language

Slide 31

Slide 31 text

There’s actually still a considerable gap even if you talk to computer using structured language

Slide 32

Slide 32 text

At the very low level (which is the CPU) Actual computer hardware only understand very basic instruction

Slide 33

Slide 33 text

mov eax, ebx — copy the value in ebx into eax mov byte ptr [var], 5 — store the value 5 into the byte at location var push eax — push eax on the stack push [var] — push the 4 bytes at address var onto the stack

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

What’s even more perplexing, different hardware understand different instruction sets

Slide 36

Slide 36 text

Structured Language Instruction Sets As a good engineer, now we will try to breakdown this complex problems

Slide 37

Slide 37 text

Steps

Slide 38

Slide 38 text

Generally, there are two approaches to tackle this problem: Compiling or Interpreting

Slide 39

Slide 39 text

Compiling You yourself somehow use dictionary to convert a manuscript into other language that you understand better (for example from English to Bahasa)

Slide 40

Slide 40 text

def start puts “ABC” end [:trace, 1], [:putnil], [:putstring, "hello world"], [:send, :puts, 1, nil, 8, 0], [:leave]]] “ABC”

Slide 41

Slide 41 text

Interpreting You hire a translator to read and understand the manuscript, then you ask him/her to explain it to you

Slide 42

Slide 42 text

def start puts “ABC” end “ABC” Hey bro say “abc”

Slide 43

Slide 43 text

But both approaches start the same

Slide 44

Slide 44 text

Lexing Lexical analysis

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

Lexing by hand

Slide 47

Slide 47 text

Lexing by tools Lex.

Slide 48

Slide 48 text

Parsing Syntactic analysis

Slide 49

Slide 49 text

7 number + operator 3 number * operator 5 number -‐ operator 2 number

Slide 50

Slide 50 text

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Slide 56

Slide 56 text

Slide 57

Slide 57 text

Parsing with tools Yacc or Bison.

Slide 58

Slide 58 text

No content

Slide 59

Slide 59 text

… if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite] while_stmt: 'while' test ':' suite ['else' ':' suite] for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite] …

Slide 60

Slide 60 text

Optimization optional

Slide 61

Slide 61 text

JVM Warmup time

Slide 62

Slide 62 text

No content

Slide 63

Slide 63 text

Lexing Parsing Optimizing Executing Compiling

Slide 64

Slide 64 text

Generaly you use “faster” language or more low-level environment to do everything above

Slide 65

Slide 65 text

Basically after AST is created, you can directly execute the nodes and the process is finished

Slide 66

Slide 66 text

Operator (*)

Slide 67

Slide 67 text

Eval left-hand * eval right-hand

Slide 68

Slide 68 text

Operator (+)

Slide 69

Slide 69 text

7 + 3

Slide 70

Slide 70 text

And so on…

Slide 71

Slide 71 text

Lexing Parsing Optimizing Executing Compiling

Slide 72

Slide 72 text

public JavapTest(); Code: 0: aload_0 1: invokespecial #1 // Method java/lang/Object."":()V 4: return public static void main(java.lang.String[]); Code: 0: getstatic #2 // Field java/lang/System.out:Ljava/io/PrintStream; 3: bipush 20 5: invokevirtual #3 // Method java/io/PrintStream.println:(I)V 8: return

Slide 73

Slide 73 text

DSL

Slide 74

Slide 74 text

DSL Domain Specific Language

Slide 75

Slide 75 text

DSL vs GPL GPL is General Purpose Language

Slide 76

Slide 76 text

No content

Slide 77

Slide 77 text

SELECT * FROM users WHERE status = “ACTIVE”

Slide 78

Slide 78 text

Turing Completeness Prog. Language is called turing complete when it can simulate the turing machine

Slide 79

Slide 79 text

The ability to read and write "variables" (or arbitrary data) The ability to simulate moving the read/write head The ability to simulate a finite state machine A "halt" state

Slide 80

Slide 80 text

DSL is most likely not turing complete

Slide 81

Slide 81 text

Xtext

Slide 82

Slide 82 text

Summary

Slide 83

Slide 83 text

If you think this is your cup of tea…

Slide 84

Slide 84 text

If you want to understand clearly the concept first.. Build from scratch http://kanaka.github.io/lambdaconf/#/ https://github.com/kanaka/mal

Slide 85

Slide 85 text

If you can create an interpreter for LISP using x language, then you will understand (almost) all of x language functionalities

Slide 86

Slide 86 text

No content

Slide 87

Slide 87 text

No content

Slide 88

Slide 88 text

If you want to have a taste in creating programming language using modern tools Learn lex + yacc/bison or ANTLR

Slide 89

Slide 89 text

If you somehow think that DSL suits your use case Learn to create DSL Learn Xtext

Slide 90

Slide 90 text

Thanks! Enjoy your lunch.. Let’s discuss @giosakti