Slide 1

Slide 1 text

A Tale of Go REPL motemen Go Conference 2015 Summer @ Tokyo

Slide 2

Slide 2 text

About Me - @motemen - Web engineer - Chief engineer at Hatena, Kyoto

Slide 3

Slide 3 text

Mackerel mackerel.io

Slide 4

Slide 4 text

Agenda - Introduction of ! motemen/gore - Gore’s architecture - Implementation details

Slide 5

Slide 5 text

Gore /gɔː/

Slide 6

Slide 6 text

Case: Learning New Languages - Playing with language features - Trying libraries

Slide 7

Slide 7 text

My Case — Perl - perl ~/sketch.pl - Edit and Run

Slide 8

Slide 8 text

My Case — Ruby - irb - pry - A “REPL”
 (Read-Eval-Print-Loop)

Slide 9

Slide 9 text

My Case — Go? - tmp.go - “packge main” “func main()” - Removing unused variables by hand - Question: Can gophers haz a REPL?

Slide 10

Slide 10 text

Go "REPL" impls (AFAIK) - ! sbinet/igo — built upon ! sbinet/go-eval, with liner - ! emicklei/rango — importing, statements, listing variables - ! vito/go-repl — importing, statements, useful commands - ! d4l3k/go-pry — attaches to running program, uses reflection - (play.golang.org)

Slide 11

Slide 11 text

Needed a REPL that: - Evaluates statements - Imports any module - Pretty prints values automatically - Completes code

Slide 12

Slide 12 text

The Difficulties - How can we achieve Go semantics? - No APIs provided - Re-implement Go compiler?

Slide 13

Slide 13 text

Solution: “go run”

Slide 14

Slide 14 text

Solution: “go run” - An “eval” to Go - Perfect implementation & Always up-to-date - Dead simple - In: a program Out: the result

Slide 15

Slide 15 text

Gore: A REPL using “go run” go get github.com/motemen/gore

Slide 16

Slide 16 text

Caveats - Gore runs all code input for each run - Code with side effects will be repeated - eg. Printing output / Sleeping

Slide 17

Slide 17 text

Stories Inside Gore

Slide 18

Slide 18 text

Overview: not a “REPL” exactly - REPL: Read-Eval-Print-Loop - No “Eval” and “Print” phase - As we do them by “go run” - Much like: Read-Generate-Run-Loop

Slide 19

Slide 19 text

Read-Gen-Run-Loop 1. Read user input to evaluate 2. Generate a .go file that prints result 3. “go run” it 4. Back to 1.

Slide 20

Slide 20 text

Step: Read - Codes go under AST (abstract syntax tree) form - Syntax checking - Easier to manipulate - Input String → AST → .go → “go run”

Slide 21

Slide 21 text

Read: Input - ! peterh/liner - Line editing - History - Supports Windows

Slide 22

Slide 22 text

Read: Input to AST - package go/parser - Go has rich support for parsing/typing its code - Easy to generate & manipulate

Slide 23

Slide 23 text

Read: Input to AST - Parse input as an expression - If failed: a statement - Otherwise: continue input (multi-line)

Slide 24

Slide 24 text

Read: Expression - Easy - Dedicated API - parser.ParseExpr(in)

Slide 25

Slide 25 text

Read: Statements - $ func (s *Session) evalStmt(in string) error { src := fmt.Sprintf("package P; func F() { %s }", in) f, err := parser.ParseFile(s.Fset, "stmt.go", src, 0) if err != nil { return err } enclosingFunc := f.Scope.Lookup("F").Decl. (*ast.FuncDecl) stmts := enclosingFunc.Body.List }

Slide 26

Slide 26 text

Generate: The Delicious Part - Append the input AST nodes to our main() - Add the code to print the value

Slide 27

Slide 27 text

Printing Values - The generated program prints values (not gore) - “New Values” should be printed - Expression: Its resulting value - Statement: Values assigned

Slide 28

Slide 28 text

Printing Values: Expression - User: foo.Bar(a+1) - Gore: __gore_p(foo.Bar(a+1))

Slide 29

Slide 29 text

Printing Values: Statements - ast.AssignStmt stands for assign/define - User: a, b := foo() - Gore: a, b := foo(); __gore_p(a, b)

Slide 30

Slide 30 text

Printing Values: __gore_p() - “Pretty prints” values - Defined along with the main() - Depends on installed packages - ! k0kubun/pp, ! davecgh/go-spew or plain old %#v

Slide 31

Slide 31 text

The Initial .go File (pp) package main import "github.com/k0kubun/pp" func __gore_p(xx ...interface{}) { for _, x := range xx { pp.Println(x) } } func main() { // User input goes here }

Slide 32

Slide 32 text

The Initial .go File (go-spew) package main import "github.com/davecgh/go-spew/spew" func __gore_p(xx ...interface{}) { for _, x := range xx { spew.Printf("%#v\n", x) } } func main() { // User input goes here }

Slide 33

Slide 33 text

The Initial .go File (%#v) package main import "fmt" func __gore_p(xx ...interface{}) { for _, x := range xx { fmt.Printf("%#v\n", x) } } func main() { // User input goes here }

Slide 34

Slide 34 text

Generate: Append to main() - Just append those generated statements
 s.mainBody.List = append(s.mainBody.List, stmts…) - Now we’ve got the working code! - Really? No! %

Slide 35

Slide 35 text

Go compiler complaints (you know) - “x declared but not used” - “p imported but not used” - “no new variables of left side of :=“ - & Should remove these to a successful go run

Slide 36

Slide 36 text

Quick Fixing Erroneous Program - “x declared but not used” - “p imported but not used” - “no new variables on left side of :=“ Use it!! Anonymize it!! Make it `=‘ !!

Slide 37

Slide 37 text

“declared but not used” package P func main() { s := "hello" } package P func main() { s := "hello" _ = s }

Slide 38

Slide 38 text

”imported but not used” package P import "fmt" func main() { } package P import _ "fmt" func main() { }

Slide 39

Slide 39 text

“no new variables on left side of :=“ package P func main() { var a int a := 1 } package P func main() { var a int a = 1 }

Slide 40

Slide 40 text

! motemen/go-quickfix to do the work - Type check source code using go/types.Check() - Catch errors and modify AST - Packed with a bin - goquickfix -w main.go

Slide 41

Slide 41 text

Gore specific quickfix - __gore_p() for non-value expressions

Slide 42

Slide 42 text

Running - Output the AST to file — go/printer - Then go run - If failed to run, revert the last input

Slide 43

Slide 43 text

Recap: Read-Gen-Quickfix-Run-Loop 1. Read and parse the input to obtain AST 2. Add printing function call __gore_p() 3. Append the code to main() 4. Quickfix it so that it compiles well 5. go run

Slide 44

Slide 44 text

PRs welcome! - ! motemen/gore - ! motemen/go-quickfix

Slide 45

Slide 45 text

htn.to/intern2015

Slide 46

Slide 46 text

A Tale Gore motemen Go Conference 2015 Summer @ Tokyo

Slide 47

Slide 47 text

Appendix: More Features

Slide 48

Slide 48 text

Code Completion - liner has support for completion - Great tool for editors: ! nsf/gocode - Server/client model - And an interface to it: motemen/gore/gocode - If gocode binary can be located, use it

Slide 49

Slide 49 text

Commands - :print - :write <file> - :import - :doc

Slide 50

Slide 50 text

:import - Imports arbitrary packages - Just append import statement to the file - Unused imports are handled by quickfix

Slide 51

Slide 51 text

:doc - Invokes go doc for given expression - Package: :doc json - Function: :doc json.Marshal - Method: :doc json.NewEncoder().Encode