Slide 1
Slide 1 text
A Tale of Go REPL
motemen
Go Conference 2015 Summer @ Tokyo
Slide 2
Slide 2 text
About Me
- @motemen
- Web engineer
- Chief engineer at Hatena, Kyoto
Slide 3
Slide 3 text
Mackerel mackerel.io
Slide 4
Slide 4 text
Agenda
- Introduction of ! motemen/gore
- Gore’s architecture
- Implementation details
Slide 5
Slide 5 text
Gore /gɔː/
Slide 6
Slide 6 text
Case: Learning New Languages
- Playing with language features
- Trying libraries
Slide 7
Slide 7 text
My Case — Perl
- perl ~/sketch.pl
- Edit and Run
Slide 8
Slide 8 text
My Case — Ruby
- irb
- pry
- A “REPL”
(Read-Eval-Print-Loop)
Slide 9
Slide 9 text
My Case — Go?
- tmp.go
- “packge main” “func main()”
- Removing unused variables by hand
- Question: Can gophers haz a REPL?
Slide 10
Slide 10 text
Go "REPL" impls (AFAIK)
- ! sbinet/igo — built upon ! sbinet/go-eval, with liner
- ! emicklei/rango — importing, statements, listing variables
- ! vito/go-repl — importing, statements, useful commands
- ! d4l3k/go-pry — attaches to running program, uses reflection
- (play.golang.org)
Slide 11
Slide 11 text
Needed a REPL that:
- Evaluates statements
- Imports any module
- Pretty prints values automatically
- Completes code
Slide 12
Slide 12 text
The Difficulties
- How can we achieve Go semantics?
- No APIs provided
- Re-implement Go compiler?
Slide 13
Slide 13 text
Solution: “go run”
Slide 14
Slide 14 text
Solution: “go run”
- An “eval” to Go
- Perfect implementation & Always up-to-date
- Dead simple
- In: a program Out: the result
Slide 15
Slide 15 text
Gore: A REPL using “go run”
go get github.com/motemen/gore
Slide 16
Slide 16 text
Caveats
- Gore runs all code input for each run
- Code with side effects will be repeated
- eg. Printing output / Sleeping
Slide 17
Slide 17 text
Stories Inside Gore
Slide 18
Slide 18 text
Overview: not a “REPL” exactly
- REPL: Read-Eval-Print-Loop
- No “Eval” and “Print” phase
- As we do them by “go run”
- Much like: Read-Generate-Run-Loop
Slide 19
Slide 19 text
Read-Gen-Run-Loop
1. Read user input to evaluate
2. Generate a .go file that prints result
3. “go run” it
4. Back to 1.
Slide 20
Slide 20 text
Step: Read
- Codes go under AST (abstract syntax tree) form
- Syntax checking
- Easier to manipulate
- Input String → AST → .go → “go run”
Slide 21
Slide 21 text
Read: Input
- ! peterh/liner
- Line editing
- History
- Supports Windows
Slide 22
Slide 22 text
Read: Input to AST
- package go/parser
- Go has rich support for parsing/typing its code
- Easy to generate & manipulate
Slide 23
Slide 23 text
Read: Input to AST
- Parse input as an expression
- If failed: a statement
- Otherwise: continue input (multi-line)
Slide 24
Slide 24 text
Read: Expression
- Easy
- Dedicated API
- parser.ParseExpr(in)
Slide 25
Slide 25 text
Read: Statements
- $
func (s *Session) evalStmt(in string) error {
src := fmt.Sprintf("package P; func F() { %s }", in)
f, err := parser.ParseFile(s.Fset, "stmt.go", src, 0)
if err != nil {
return err
}
enclosingFunc := f.Scope.Lookup("F").Decl.
(*ast.FuncDecl)
stmts := enclosingFunc.Body.List
}
Slide 26
Slide 26 text
Generate: The Delicious Part
- Append the input AST nodes to our main()
- Add the code to print the value
Slide 27
Slide 27 text
Printing Values
- The generated program prints values (not gore)
- “New Values” should be printed
- Expression: Its resulting value
- Statement: Values assigned
Slide 28
Slide 28 text
Printing Values: Expression
- User: foo.Bar(a+1)
- Gore: __gore_p(foo.Bar(a+1))
Slide 29
Slide 29 text
Printing Values: Statements
- ast.AssignStmt stands for assign/define
- User: a, b := foo()
- Gore: a, b := foo(); __gore_p(a, b)
Slide 30
Slide 30 text
Printing Values: __gore_p()
- “Pretty prints” values
- Defined along with the main()
- Depends on installed packages
- ! k0kubun/pp, ! davecgh/go-spew or plain old %#v
Slide 31
Slide 31 text
The Initial .go File (pp)
package main
import "github.com/k0kubun/pp"
func __gore_p(xx ...interface{}) {
for _, x := range xx {
pp.Println(x)
}
}
func main() {
// User input goes here
}
Slide 32
Slide 32 text
The Initial .go File (go-spew)
package main
import "github.com/davecgh/go-spew/spew"
func __gore_p(xx ...interface{}) {
for _, x := range xx {
spew.Printf("%#v\n", x)
}
}
func main() {
// User input goes here
}
Slide 33
Slide 33 text
The Initial .go File (%#v)
package main
import "fmt"
func __gore_p(xx ...interface{}) {
for _, x := range xx {
fmt.Printf("%#v\n", x)
}
}
func main() {
// User input goes here
}
Slide 34
Slide 34 text
Generate: Append to main()
- Just append those generated statements
s.mainBody.List = append(s.mainBody.List, stmts…)
- Now we’ve got the working code!
- Really? No! %
Slide 35
Slide 35 text
Go compiler complaints (you know)
- “x declared but not used”
- “p imported but not used”
- “no new variables of left side of :=“
- & Should remove these to a successful go run
Slide 36
Slide 36 text
Quick Fixing Erroneous Program
- “x declared but not used”
- “p imported but not used”
- “no new variables on left side of :=“
Use it!!
Anonymize it!!
Make it `=‘ !!
Slide 37
Slide 37 text
“declared but not used”
package P
func main() {
s := "hello"
}
package P
func main() {
s := "hello"
_ = s
}
Slide 38
Slide 38 text
”imported but not used”
package P
import "fmt"
func main() {
}
package P
import _ "fmt"
func main() {
}
Slide 39
Slide 39 text
“no new variables on left side of :=“
package P
func main() {
var a int
a := 1
}
package P
func main() {
var a int
a = 1
}
Slide 40
Slide 40 text
! motemen/go-quickfix to do the work
- Type check source code using go/types.Check()
- Catch errors and modify AST
- Packed with a bin
- goquickfix -w main.go
Slide 41
Slide 41 text
Gore specific quickfix
- __gore_p() for non-value expressions
Slide 42
Slide 42 text
Running
- Output the AST to file — go/printer
- Then go run
- If failed to run, revert the last input
Slide 43
Slide 43 text
Recap: Read-Gen-Quickfix-Run-Loop
1. Read and parse the input to obtain AST
2. Add printing function call __gore_p()
3. Append the code to main()
4. Quickfix it so that it compiles well
5. go run
Slide 44
Slide 44 text
PRs welcome!
- ! motemen/gore
- ! motemen/go-quickfix
Slide 45
Slide 45 text
htn.to/intern2015
Slide 46
Slide 46 text
A Tale Gore
motemen
Go Conference 2015 Summer @ Tokyo
Slide 47
Slide 47 text
Appendix:
More Features
Slide 48
Slide 48 text
Code Completion
- liner has support for completion
- Great tool for editors: ! nsf/gocode
- Server/client model
- And an interface to it: motemen/gore/gocode
- If gocode binary can be located, use it
Slide 49
Slide 49 text
Commands
- :print
- :write <file>
- :import
- :doc
Slide 50
Slide 50 text
:import
- Imports arbitrary packages
- Just append import statement to the file
- Unused imports are handled by quickfix
Slide 51
Slide 51 text
:doc
- Invokes go doc for given expression
- Package: :doc json
- Function: :doc json.Marshal
- Method: :doc json.NewEncoder().Encode