Implementing Languages (FluentConf)

Programming is hard. Making compilers is harder. Or so people think. The truth is that making a compiler is just a series of small steps, using regular language constructs. It's so easy, we'll make one in this talk.

Transcript

1. Implementing
   Languages
   

   View Slide

2. Thanks FluentConf
   

   View Slide

3. Thanks SilverStripe
   

   View Slide

4. I Am Not A
   Special Snowflake
   

   View Slide

5. Compilers Are
   Not Important
   

   View Slide

6. Great Books
   On The Subject
   

   View Slide

7. View Slide

8. View Slide

9. Why Is This Useful?
   

   View Slide

10. Why Is This Useful?
    ▸ Improving productivity
    ▸ Communicating in a common domain language
    ▸ Alternative computational models
    

    View Slide

11. Tools You Can Use
    

    View Slide

12. Tools You Can Use
    ▸ Compiler generators
    ▸ String manipulation and state machines
    

    View Slide

13. int minutes = 90;
    

    View Slide

14. Grammar
    

    View Slide

15. Grammar
    ▸ List of rules which define structure and syntax
    ▸ Each rule is called a lexeme
    

    View Slide

16. int minutes = 90;
    e ! type id = value | type id
    

    View Slide

17. Lexer
    

    View Slide

18. Lexer
    ▸ Breaks code string into tokens
    ▸ Lexer ignores nesting
    

    View Slide

19. e ! type id = value | type id
    ["type", "int"],
    ["id", "minutes"],
    ["assign", "="],
    ["value", "90"],
    

    View Slide

20. Parser
    

    View Slide

21. Parser
    ▸ Arranges tokens in a hierarchy
    ▸ Checks tokens for syntax errors
    

    View Slide

22. ["type", "int"],
    ["id", "minutes"],
    ["assign", "="],
    ["value", "90"],
    ["definition", [
    "int",
    "minutes",
    "90",
    ]]
    

    View Slide

23. Abstract Syntax Tree
    

    View Slide

24. Abstract Syntax Tree
    ▸ Represents constructs in the compiler's language
    

    View Slide

25. ["definition", [
    "int",
    "minutes",
    "90",
    ]]
    new Definition(
    "int",
    "minutes",
    "90",
    )
    

    View Slide

26. Interpreter
    

    View Slide

27. Interpreter
    ▸ Calculates immediate effect to context
    ▸ Always accepts and returns a context
    

    View Slide

28. new Definition(
    "int",
    "minutes",
    "90",
    )
    let context = {};
    for (let i = 0; i < objects.length; i++) {
    context = objects[i].applyTo(context);
    }
    

    View Slide

29. Compiler
    

    View Slide

30. Compiler
    ▸ Writes the AST to a new language
    ▸ Checks constructs for semantic errors in target language
    

    View Slide

31. Recap
    

    View Slide

32. int minutes = 90;
    e ! type id = value | type id
    

    View Slide

33. e ! type id = value | type id
    ["type", "int"],
    ["id", "minutes"],
    ["assign", "="],
    ["value", "90"],
    

    View Slide

34. ["type", "int"],
    ["id", "minutes"],
    ["assign", "="],
    ["value", "90"],
    ["definition", [
    "int",
    "minutes",
    "90",
    ]]
    

    View Slide

35. ["definition", [
    "int",
    "minutes",
    "90",
    ]]
    new Definition(
    "int",
    "minutes",
    "90",
    )
    

    View Slide

36. new Definition(
    "int",
    "minutes",
    "90",
    )
    let context = {};
    for (let i = 0; i < objects.length; i++) {
    context = objects[i].applyTo(context);
    }
    

    View Slide

37. Yay!
    

    View Slide

38. Parser Expression
    Grammar
    

    View Slide

39. Parser Expression Grammar
    ▸ Uses pattern matching to tokenise and organise lexemes
    ▸ Generates a reusable interpreter and/or compiler
    

    View Slide

40. {
    var context = {};
    function __set(key, value) {
    context[key] = value;
    }
    function __get(key) {
    return context[key];
    }
    }
    

    View Slide

41. start =
    expression*
    expression =
    _ definition _
    /
    _ inspection _
    _ "whitespace" =
    [ \t\r\n]*
    

    View Slide

42. definition "definition" =
    t:type _ i:id _ assign _ v:value _ terminal {
    __set(i, v);
    }
    /
    t:type _ i:id _ terminal {
    __set(i, null);
    }
    

    View Slide

43. type "type" =
    "int"
    id "identity" =
    letters:[a-z]+ {
    return letters.join("");
    }
    assign "assign" =
    "="
    value "value" =
    numbers:[0-9]+ {
    return parseInt(numbers.join(""), 10);
    }
    terminal "terminal" =
    ";"
    

    View Slide

44. inspection "inspection" =
    "inspect" _ i:id _ terminal {
    console.log(__get(i));
    }
    

    View Slide

45. Thanks
    CONFERENCES.OREILLY.COM/FLUENT/JAVASCRIPT-HTML-US/PUBLIC/SCHEDULE/DETAIL/46342
    @ASSERTCHRIS
    

    View Slide