Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Gore: A Tale of Go REPL
Search
motemen
June 21, 2015
Programming
8
5.9k
Gore: A Tale of Go REPL
Go Conference Summer 2015
motemen
June 21, 2015
Tweet
Share
More Decks by motemen
See All by motemen
はてなのエンジニアリングマネジメント、これまでとこれから / Engineering Management in Hatena, Hatena Engineer Seminar 26
motemen
2
1.8k
入門 Bubble Tea
motemen
1
710
ソフトウェア開発と私 / Software Development and I
motemen
13
6.5k
はてなサービスプラットフォームチームにおける Mackerel / Mackerel in Hatena Platform Team
motemen
1
1.1k
はてなシステムの考古学 / History of development at Hatena
motemen
3
300k
Reading Go Tools - GoCon 2016 Spring
motemen
5
2.7k
Diff'ing Go library API changes
motemen
4
2.6k
はてな技術部紹介 2015(公開版) / Hatena Technical Division Orientation 2015 (Public Edition)
motemen
5
20k
Workflow at Hatena Mackerel Team
motemen
9
6.9k
Other Decks in Programming
See All in Programming
Iteratorでページネーションを実現する
sonatard
3
710
Cohesion in Modeling and Design
mploed
3
190
ACES Meet におけるリリース作業改善の取り組み
fukucheee
0
130
DjangoNinjaで高速なAPI開発を実現する
masaya00
0
500
2024-10-01 dev2next - Observability for Modern JVM Applications
jonatan_ivanov
0
100
ECS向けのドリフト検知機構を実装してみた
tkikuc
0
280
Vue :: Better Testing 2024
up1
1
390
全方位強化 Python 服務可觀測性:以 FastAPI 和 Grafana Stack 為例
blueswen
1
380
Cloud Adoption Frameworkにみる組織とクラウド導入戦略(縮小版)
tomokusaba
1
180
Pythonによるイベントソーシングへの挑戦と現状に対する考察 / Challenging Event Sourcing with Python and Reflections on the Current State
nrslib
3
1k
4年間変わらなかった YOUTRUSTのアーキテクチャ
daiki1003
1
460
クラウドサービスの 利用コストを削減する技術 - 円安の真南風を感じて -
pyama86
3
370
Featured
See All Featured
Creatively Recalculating Your Daily Design Routine
revolveconf
217
12k
StorybookのUI Testing Handbookを読んだ
zakiyama
26
5.1k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
7
280
Happy Clients
brianwarren
97
6.7k
How to train your dragon (web standard)
notwaldorf
87
5.6k
Rebuilding a faster, lazier Slack
samanthasiow
79
8.6k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
364
22k
YesSQL, Process and Tooling at Scale
rocio
167
14k
Designing with Data
zakiwarfel
98
5.1k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
131
32k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
27
1.9k
Transcript
A Tale of Go REPL motemen Go Conference 2015 Summer
@ Tokyo
About Me - @motemen - Web engineer - Chief engineer
at Hatena, Kyoto
Mackerel mackerel.io
Agenda - Introduction of ! motemen/gore - Gore’s architecture -
Implementation details
Gore /gɔː/
Case: Learning New Languages - Playing with language features -
Trying libraries
My Case — Perl - perl ~/sketch.pl - Edit and
Run
My Case — Ruby - irb - pry - A
“REPL” (Read-Eval-Print-Loop)
My Case — Go? - tmp.go - “packge main” “func
main()” - Removing unused variables by hand - Question: Can gophers haz a REPL?
Go "REPL" impls (AFAIK) - ! sbinet/igo — built upon
! sbinet/go-eval, with liner - ! emicklei/rango — importing, statements, listing variables - ! vito/go-repl — importing, statements, useful commands - ! d4l3k/go-pry — attaches to running program, uses reflection - (play.golang.org)
Needed a REPL that: - Evaluates statements - Imports any
module - Pretty prints values automatically - Completes code
The Difficulties - How can we achieve Go semantics? -
No APIs provided - Re-implement Go compiler?
Solution: “go run”
Solution: “go run” - An “eval” to Go - Perfect
implementation & Always up-to-date - Dead simple - In: a program Out: the result
Gore: A REPL using “go run” go get github.com/motemen/gore
Caveats - Gore runs all code input for each run
- Code with side effects will be repeated - eg. Printing output / Sleeping
Stories Inside Gore
Overview: not a “REPL” exactly - REPL: Read-Eval-Print-Loop - No
“Eval” and “Print” phase - As we do them by “go run” - Much like: Read-Generate-Run-Loop
Read-Gen-Run-Loop 1. Read user input to evaluate 2. Generate a
.go file that prints result 3. “go run” it 4. Back to 1.
Step: Read - Codes go under AST (abstract syntax tree)
form - Syntax checking - Easier to manipulate - Input String → AST → .go → “go run”
Read: Input - ! peterh/liner - Line editing - History
- Supports Windows
Read: Input to AST - package go/parser - Go has
rich support for parsing/typing its code - Easy to generate & manipulate
Read: Input to AST - Parse input as an expression
- If failed: a statement - Otherwise: continue input (multi-line)
Read: Expression - Easy - Dedicated API - parser.ParseExpr(in)
Read: Statements - $ func (s *Session) evalStmt(in string) error
{ src := fmt.Sprintf("package P; func F() { %s }", in) f, err := parser.ParseFile(s.Fset, "stmt.go", src, 0) if err != nil { return err } enclosingFunc := f.Scope.Lookup("F").Decl. (*ast.FuncDecl) stmts := enclosingFunc.Body.List }
Generate: The Delicious Part - Append the input AST nodes
to our main() - Add the code to print the value
Printing Values - The generated program prints values (not gore)
- “New Values” should be printed - Expression: Its resulting value - Statement: Values assigned
Printing Values: Expression - User: foo.Bar(a+1) - Gore: __gore_p(foo.Bar(a+1))
Printing Values: Statements - ast.AssignStmt stands for assign/define - User:
a, b := foo() - Gore: a, b := foo(); __gore_p(a, b)
Printing Values: __gore_p() - “Pretty prints” values - Defined along
with the main() - Depends on installed packages - ! k0kubun/pp, ! davecgh/go-spew or plain old %#v
The Initial .go File (pp) package main import "github.com/k0kubun/pp" func
__gore_p(xx ...interface{}) { for _, x := range xx { pp.Println(x) } } func main() { // User input goes here }
The Initial .go File (go-spew) package main import "github.com/davecgh/go-spew/spew" func
__gore_p(xx ...interface{}) { for _, x := range xx { spew.Printf("%#v\n", x) } } func main() { // User input goes here }
The Initial .go File (%#v) package main import "fmt" func
__gore_p(xx ...interface{}) { for _, x := range xx { fmt.Printf("%#v\n", x) } } func main() { // User input goes here }
Generate: Append to main() - Just append those generated statements
s.mainBody.List = append(s.mainBody.List, stmts…) - Now we’ve got the working code! - Really? No! %
Go compiler complaints (you know) - “x declared but not
used” - “p imported but not used” - “no new variables of left side of :=“ - & Should remove these to a successful go run
Quick Fixing Erroneous Program - “x declared but not used”
- “p imported but not used” - “no new variables on left side of :=“ Use it!! Anonymize it!! Make it `=‘ !!
“declared but not used” package P func main() { s
:= "hello" } package P func main() { s := "hello" _ = s }
”imported but not used” package P import "fmt" func main()
{ } package P import _ "fmt" func main() { }
“no new variables on left side of :=“ package P
func main() { var a int a := 1 } package P func main() { var a int a = 1 }
! motemen/go-quickfix to do the work - Type check source
code using go/types.Check() - Catch errors and modify AST - Packed with a bin - goquickfix -w main.go
Gore specific quickfix - __gore_p() for non-value expressions
Running - Output the AST to file — go/printer -
Then go run - If failed to run, revert the last input
Recap: Read-Gen-Quickfix-Run-Loop 1. Read and parse the input to obtain
AST 2. Add printing function call __gore_p() 3. Append the code to main() 4. Quickfix it so that it compiles well 5. go run
PRs welcome! - ! motemen/gore - ! motemen/go-quickfix
htn.to/intern2015
A Tale Gore motemen Go Conference 2015 Summer @ Tokyo
Appendix: More Features
Code Completion - liner has support for completion - Great
tool for editors: ! nsf/gocode - Server/client model - And an interface to it: motemen/gore/gocode - If gocode binary can be located, use it
Commands - :print - :write <file> - :import <pkg> -
:doc <expr>
:import <pkg> - Imports arbitrary packages - Just append import
statement to the file - Unused imports are handled by quickfix
:doc <expr> - Invokes go doc for given expression -
Package: :doc json - Function: :doc json.Marshal - Method: :doc json.NewEncoder().Encode