Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Gore: A Tale of Go REPL
motemen
June 21, 2015
Programming
8
5.2k
Gore: A Tale of Go REPL
Go Conference Summer 2015
motemen
June 21, 2015
Tweet
Share
More Decks by motemen
See All by motemen
入門 Bubble Tea
motemen
1
230
ソフトウェア開発と私 / Software Development and I
motemen
13
5.9k
はてなサービスプラットフォームチームにおける Mackerel / Mackerel in Hatena Platform Team
motemen
1
780
はてなシステムの考古学 / History of development at Hatena
motemen
3
300k
Reading Go Tools - GoCon 2016 Spring
motemen
6
2.4k
Diff'ing Go library API changes
motemen
4
2.3k
はてな技術部紹介 2015(公開版) / Hatena Technical Division Orientation 2015 (Public Edition)
motemen
4
18k
Workflow at Hatena Mackerel Team
motemen
9
6.3k
YAPC::Asia 2014: Writing tools with Git
motemen
4
6.4k
Other Decks in Programming
See All in Programming
trocco® の品質を守る、とても普通な取り組み
kekekenta
0
350
ISUCON12 事前講習
rosylilly
3
3.6k
Java アプリとAWS の良い関係 - AWS でJava アプリを実行する一番簡単な方法教えます / AWS for Javarista
kanamasa
2
1.1k
engineer
spacemarket
0
460
RFC 9111: HTTP Caching
jxck
0
150
短納期でローンチした新サービスをJavaで開発した話/launched new service using Java
eichisanden
5
1.8k
io22 extended What's new in app performance
veronikapj
0
320
Oracle REST Data Service: APEX Office Hours
thatjeffsmith
0
650
Web API連携でCSRF対策がどう実装されてるか調べた / how to implements csrf-detection on Web API
yasuakiomokawa
2
210
Cross Deviceチームにおけるスマートテレビアプリ開発ってどんな感じ?
cokaholic
0
120
EFFICIENT CREATION OF AN EMPTY COLLECTION IN .NET
abt
0
150
Keeping your team in top shape with the Gradle Enterprise API
runningcode
3
120
Featured
See All Featured
Dealing with People You Can't Stand - Big Design 2015
cassininazir
351
21k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
37
3.2k
Building an army of robots
kneath
299
40k
Put a Button on it: Removing Barriers to Going Fast.
kastner
56
2.3k
Docker and Python
trallard
27
1.6k
Statistics for Hackers
jakevdp
781
210k
Building Your Own Lightsaber
phodgson
94
4.6k
YesSQL, Process and Tooling at Scale
rocio
157
12k
What's new in Ruby 2.0
geeforr
336
30k
Building Flexible Design Systems
yeseniaperezcruz
310
34k
Side Projects
sachag
450
37k
It's Worth the Effort
3n
172
25k
Transcript
A Tale of Go REPL motemen Go Conference 2015 Summer
@ Tokyo
About Me - @motemen - Web engineer - Chief engineer
at Hatena, Kyoto
Mackerel mackerel.io
Agenda - Introduction of ! motemen/gore - Gore’s architecture -
Implementation details
Gore /gɔː/
Case: Learning New Languages - Playing with language features -
Trying libraries
My Case — Perl - perl ~/sketch.pl - Edit and
Run
My Case — Ruby - irb - pry - A
“REPL” (Read-Eval-Print-Loop)
My Case — Go? - tmp.go - “packge main” “func
main()” - Removing unused variables by hand - Question: Can gophers haz a REPL?
Go "REPL" impls (AFAIK) - ! sbinet/igo — built upon
! sbinet/go-eval, with liner - ! emicklei/rango — importing, statements, listing variables - ! vito/go-repl — importing, statements, useful commands - ! d4l3k/go-pry — attaches to running program, uses reflection - (play.golang.org)
Needed a REPL that: - Evaluates statements - Imports any
module - Pretty prints values automatically - Completes code
The Difficulties - How can we achieve Go semantics? -
No APIs provided - Re-implement Go compiler?
Solution: “go run”
Solution: “go run” - An “eval” to Go - Perfect
implementation & Always up-to-date - Dead simple - In: a program Out: the result
Gore: A REPL using “go run” go get github.com/motemen/gore
Caveats - Gore runs all code input for each run
- Code with side effects will be repeated - eg. Printing output / Sleeping
Stories Inside Gore
Overview: not a “REPL” exactly - REPL: Read-Eval-Print-Loop - No
“Eval” and “Print” phase - As we do them by “go run” - Much like: Read-Generate-Run-Loop
Read-Gen-Run-Loop 1. Read user input to evaluate 2. Generate a
.go file that prints result 3. “go run” it 4. Back to 1.
Step: Read - Codes go under AST (abstract syntax tree)
form - Syntax checking - Easier to manipulate - Input String → AST → .go → “go run”
Read: Input - ! peterh/liner - Line editing - History
- Supports Windows
Read: Input to AST - package go/parser - Go has
rich support for parsing/typing its code - Easy to generate & manipulate
Read: Input to AST - Parse input as an expression
- If failed: a statement - Otherwise: continue input (multi-line)
Read: Expression - Easy - Dedicated API - parser.ParseExpr(in)
Read: Statements - $ func (s *Session) evalStmt(in string) error
{ src := fmt.Sprintf("package P; func F() { %s }", in) f, err := parser.ParseFile(s.Fset, "stmt.go", src, 0) if err != nil { return err } enclosingFunc := f.Scope.Lookup("F").Decl. (*ast.FuncDecl) stmts := enclosingFunc.Body.List }
Generate: The Delicious Part - Append the input AST nodes
to our main() - Add the code to print the value
Printing Values - The generated program prints values (not gore)
- “New Values” should be printed - Expression: Its resulting value - Statement: Values assigned
Printing Values: Expression - User: foo.Bar(a+1) - Gore: __gore_p(foo.Bar(a+1))
Printing Values: Statements - ast.AssignStmt stands for assign/define - User:
a, b := foo() - Gore: a, b := foo(); __gore_p(a, b)
Printing Values: __gore_p() - “Pretty prints” values - Defined along
with the main() - Depends on installed packages - ! k0kubun/pp, ! davecgh/go-spew or plain old %#v
The Initial .go File (pp) package main import "github.com/k0kubun/pp" func
__gore_p(xx ...interface{}) { for _, x := range xx { pp.Println(x) } } func main() { // User input goes here }
The Initial .go File (go-spew) package main import "github.com/davecgh/go-spew/spew" func
__gore_p(xx ...interface{}) { for _, x := range xx { spew.Printf("%#v\n", x) } } func main() { // User input goes here }
The Initial .go File (%#v) package main import "fmt" func
__gore_p(xx ...interface{}) { for _, x := range xx { fmt.Printf("%#v\n", x) } } func main() { // User input goes here }
Generate: Append to main() - Just append those generated statements
s.mainBody.List = append(s.mainBody.List, stmts…) - Now we’ve got the working code! - Really? No! %
Go compiler complaints (you know) - “x declared but not
used” - “p imported but not used” - “no new variables of left side of :=“ - & Should remove these to a successful go run
Quick Fixing Erroneous Program - “x declared but not used”
- “p imported but not used” - “no new variables on left side of :=“ Use it!! Anonymize it!! Make it `=‘ !!
“declared but not used” package P func main() { s
:= "hello" } package P func main() { s := "hello" _ = s }
”imported but not used” package P import "fmt" func main()
{ } package P import _ "fmt" func main() { }
“no new variables on left side of :=“ package P
func main() { var a int a := 1 } package P func main() { var a int a = 1 }
! motemen/go-quickfix to do the work - Type check source
code using go/types.Check() - Catch errors and modify AST - Packed with a bin - goquickfix -w main.go
Gore specific quickfix - __gore_p() for non-value expressions
Running - Output the AST to file — go/printer -
Then go run - If failed to run, revert the last input
Recap: Read-Gen-Quickfix-Run-Loop 1. Read and parse the input to obtain
AST 2. Add printing function call __gore_p() 3. Append the code to main() 4. Quickfix it so that it compiles well 5. go run
PRs welcome! - ! motemen/gore - ! motemen/go-quickfix
htn.to/intern2015
A Tale Gore motemen Go Conference 2015 Summer @ Tokyo
Appendix: More Features
Code Completion - liner has support for completion - Great
tool for editors: ! nsf/gocode - Server/client model - And an interface to it: motemen/gore/gocode - If gocode binary can be located, use it
Commands - :print - :write <file> - :import <pkg> -
:doc <expr>
:import <pkg> - Imports arbitrary packages - Just append import
statement to the file - Unused imports are handled by quickfix
:doc <expr> - Invokes go doc for given expression -
Package: :doc json - Function: :doc json.Marshal - Method: :doc json.NewEncoder().Encode