Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gore: A Tale of Go REPL

motemen
June 21, 2015

Gore: A Tale of Go REPL

Go Conference Summer 2015

motemen

June 21, 2015
Tweet

More Decks by motemen

Other Decks in Programming

Transcript

  1. A Tale of Go REPL
    motemen
    Go Conference 2015 Summer @ Tokyo

    View full-size slide

  2. About Me
    - @motemen
    - Web engineer
    - Chief engineer at Hatena, Kyoto

    View full-size slide

  3. Mackerel mackerel.io

    View full-size slide

  4. Agenda
    - Introduction of ! motemen/gore
    - Gore’s architecture
    - Implementation details

    View full-size slide

  5. Gore /gɔː/

    View full-size slide

  6. Case: Learning New Languages
    - Playing with language features
    - Trying libraries

    View full-size slide

  7. My Case — Perl
    - perl ~/sketch.pl
    - Edit and Run

    View full-size slide

  8. My Case — Ruby
    - irb
    - pry
    - A “REPL”

    (Read-Eval-Print-Loop)

    View full-size slide

  9. My Case — Go?
    - tmp.go
    - “packge main” “func main()”
    - Removing unused variables by hand
    - Question: Can gophers haz a REPL?

    View full-size slide

  10. Go "REPL" impls (AFAIK)
    - ! sbinet/igo — built upon ! sbinet/go-eval, with liner
    - ! emicklei/rango — importing, statements, listing variables
    - ! vito/go-repl — importing, statements, useful commands
    - ! d4l3k/go-pry — attaches to running program, uses reflection
    - (play.golang.org)

    View full-size slide

  11. Needed a REPL that:
    - Evaluates statements
    - Imports any module
    - Pretty prints values automatically
    - Completes code

    View full-size slide

  12. The Difficulties
    - How can we achieve Go semantics?
    - No APIs provided
    - Re-implement Go compiler?

    View full-size slide

  13. Solution: “go run”

    View full-size slide

  14. Solution: “go run”
    - An “eval” to Go
    - Perfect implementation & Always up-to-date
    - Dead simple
    - In: a program Out: the result

    View full-size slide

  15. Gore: A REPL using “go run”
    go get github.com/motemen/gore

    View full-size slide

  16. Caveats
    - Gore runs all code input for each run
    - Code with side effects will be repeated
    - eg. Printing output / Sleeping

    View full-size slide

  17. Stories Inside Gore

    View full-size slide

  18. Overview: not a “REPL” exactly
    - REPL: Read-Eval-Print-Loop
    - No “Eval” and “Print” phase
    - As we do them by “go run”
    - Much like: Read-Generate-Run-Loop

    View full-size slide

  19. Read-Gen-Run-Loop
    1. Read user input to evaluate
    2. Generate a .go file that prints result
    3. “go run” it
    4. Back to 1.

    View full-size slide

  20. Step: Read
    - Codes go under AST (abstract syntax tree) form
    - Syntax checking
    - Easier to manipulate
    - Input String → AST → .go → “go run”

    View full-size slide

  21. Read: Input
    - ! peterh/liner
    - Line editing
    - History
    - Supports Windows

    View full-size slide

  22. Read: Input to AST
    - package go/parser
    - Go has rich support for parsing/typing its code
    - Easy to generate & manipulate

    View full-size slide

  23. Read: Input to AST
    - Parse input as an expression
    - If failed: a statement
    - Otherwise: continue input (multi-line)

    View full-size slide

  24. Read: Expression
    - Easy
    - Dedicated API
    - parser.ParseExpr(in)

    View full-size slide

  25. Read: Statements
    - $
    func (s *Session) evalStmt(in string) error {
    src := fmt.Sprintf("package P; func F() { %s }", in)
    f, err := parser.ParseFile(s.Fset, "stmt.go", src, 0)
    if err != nil {
    return err
    }
    enclosingFunc := f.Scope.Lookup("F").Decl.
    (*ast.FuncDecl)
    stmts := enclosingFunc.Body.List
    }

    View full-size slide

  26. Generate: The Delicious Part
    - Append the input AST nodes to our main()
    - Add the code to print the value

    View full-size slide

  27. Printing Values
    - The generated program prints values (not gore)
    - “New Values” should be printed
    - Expression: Its resulting value
    - Statement: Values assigned

    View full-size slide

  28. Printing Values: Expression
    - User: foo.Bar(a+1)
    - Gore: __gore_p(foo.Bar(a+1))

    View full-size slide

  29. Printing Values: Statements
    - ast.AssignStmt stands for assign/define
    - User: a, b := foo()
    - Gore: a, b := foo(); __gore_p(a, b)

    View full-size slide

  30. Printing Values: __gore_p()
    - “Pretty prints” values
    - Defined along with the main()
    - Depends on installed packages
    - ! k0kubun/pp, ! davecgh/go-spew or plain old %#v

    View full-size slide

  31. The Initial .go File (pp)
    package main
    import "github.com/k0kubun/pp"
    func __gore_p(xx ...interface{}) {
    for _, x := range xx {
    pp.Println(x)
    }
    }
    func main() {
    // User input goes here
    }

    View full-size slide

  32. The Initial .go File (go-spew)
    package main
    import "github.com/davecgh/go-spew/spew"
    func __gore_p(xx ...interface{}) {
    for _, x := range xx {
    spew.Printf("%#v\n", x)
    }
    }
    func main() {
    // User input goes here
    }

    View full-size slide

  33. The Initial .go File (%#v)
    package main
    import "fmt"
    func __gore_p(xx ...interface{}) {
    for _, x := range xx {
    fmt.Printf("%#v\n", x)
    }
    }
    func main() {
    // User input goes here
    }

    View full-size slide

  34. Generate: Append to main()
    - Just append those generated statements

    s.mainBody.List = append(s.mainBody.List, stmts…)
    - Now we’ve got the working code!
    - Really? No! %

    View full-size slide

  35. Go compiler complaints (you know)
    - “x declared but not used”
    - “p imported but not used”
    - “no new variables of left side of :=“
    - & Should remove these to a successful go run

    View full-size slide

  36. Quick Fixing Erroneous Program
    - “x declared but not used”
    - “p imported but not used”
    - “no new variables on left side of :=“
    Use it!!
    Anonymize it!!
    Make it `=‘ !!

    View full-size slide

  37. “declared but not used”
    package P
    func main() {
    s := "hello"
    }
    package P
    func main() {
    s := "hello"
    _ = s
    }

    View full-size slide

  38. ”imported but not used”
    package P
    import "fmt"
    func main() {
    }
    package P
    import _ "fmt"
    func main() {
    }

    View full-size slide

  39. “no new variables on left side of :=“
    package P
    func main() {
    var a int
    a := 1
    }
    package P
    func main() {
    var a int
    a = 1
    }

    View full-size slide

  40. ! motemen/go-quickfix to do the work
    - Type check source code using go/types.Check()
    - Catch errors and modify AST
    - Packed with a bin
    - goquickfix -w main.go

    View full-size slide

  41. Gore specific quickfix
    - __gore_p() for non-value expressions

    View full-size slide

  42. Running
    - Output the AST to file — go/printer
    - Then go run
    - If failed to run, revert the last input

    View full-size slide

  43. Recap: Read-Gen-Quickfix-Run-Loop
    1. Read and parse the input to obtain AST
    2. Add printing function call __gore_p()
    3. Append the code to main()
    4. Quickfix it so that it compiles well
    5. go run

    View full-size slide

  44. PRs welcome!
    - ! motemen/gore
    - ! motemen/go-quickfix

    View full-size slide

  45. htn.to/intern2015

    View full-size slide

  46. A Tale Gore
    motemen
    Go Conference 2015 Summer @ Tokyo

    View full-size slide

  47. Appendix:
    More Features

    View full-size slide

  48. Code Completion
    - liner has support for completion
    - Great tool for editors: ! nsf/gocode
    - Server/client model
    - And an interface to it: motemen/gore/gocode
    - If gocode binary can be located, use it

    View full-size slide

  49. Commands
    - :print
    - :write <file>
    - :import
    - :doc

    View full-size slide

  50. :import
    - Imports arbitrary packages
    - Just append import statement to the file
    - Unused imports are handled by quickfix

    View full-size slide

  51. :doc
    - Invokes go doc for given expression
    - Package: :doc json
    - Function: :doc json.Marshal
    - Method: :doc json.NewEncoder().Encode

    View full-size slide