Ways To Do Things

A few productive patterns for designing and orchestrating components (actors) in Go programs. Presented at the Go SF Meetup on 25 August 2017.

Transcript

1. Ways To Do Things By your friend,
 @peterbourgon
2. Outline Actor stuff Running stuff Orchestrating stuff
3. Outline Actor stuff Running stuff Orchestrating stuff ➫
4. ctrl-C handler Storage HTTP API Cron State machine
5. ctrl-C handler Storage HTTP API Cron State machine
6. Actors (basic) type stateMachine struct {
 state string
 fooc chan fooReq
 barc chan barReq
 quitc chan struct{}
 } func (sm *stateMachine) loop() {
 for {
 select {
 case r := <-sm.fooc:
 sm.handleFoo(r)
 case r := <-sm.barc:
 sm.handleBar(r)
 case <-sm.quitc:
 return
 }
 }
 } func (sm *stateMachine) foo() int {
 c := make(chan int)
 sm.fooc <- fooReq{c}
 return <-c
 }
7. Actors (advanced?) type stateMachine struct {
 state string
 actionc chan func()
 quitc chan struct{}
 } func (sm *stateMachine) loop() {
 for {
 select {
 case f := <-sm.actionc:
 f()
 case <-sm.quitc:
 return
 }
 }
 } func (sm *stateMachine) foo() int {
 c := make(chan int)
 sm.actionc <- func() {
 if sm.state == "A" {
 sm.state = "B"
 }
 c <- 123
 }
 return <-c
 }
9. N chans 1 chan + Clean separation of concerns - Logic separate from methods - More types and fields; verbose
 - Mixing mechanics and logic
 
 · Relatively few API methods
 · Request/response pattern - Maybe not totally obvious 
 + Colocate method and logic + Fewer types and fields
 + Mechanics and logic separate
 
 · More API methods
 · More complex input/output
10. Outline Actor stuff Running stuff Orchestrating stuff ➫
11. Outline Actor stuff Running stuff Orchestrating stuff ➫
12. Hidden goroutine func newStateMachine() *stateMachine {
 sm := &stateMachine{
 state: "initial",
 actionc: make(chan func()),
 quitc: make(chan struct{}),
 }
 go sm.loop()
 return sm
 } func (sm *stateMachine) loop() {
 for {
 select {
 // ...
 case <-sm.quitc:
 return
 }
 }
 } func (sm *stateMachine) stop() {
 close(sm.quitc)
 }
13. Hidden goroutine++ func newStateMachine() *stateMachine {
 sm := &stateMachine{
 state: "initial",
 actionc: make(chan func()),
 quitc: make(chan chan struct{}),
 }
 go sm.loop()
 return sm
 } func (sm *stateMachine) loop() {
 for {
 select {
 // ...
 case q := <-sm.quitc:
 close(q)
 return
 }
 }
 } func (sm *stateMachine) stop() {
 q := make(chan struct{})
 sm.quitc <- q
 <-q
 }
14. Run method func newStateMachine() *stateMachine {
 return &stateMachine{
 state: "initial",
 actionc: make(chan func()),
 }
 } func (sm *stateMachine) Run(cancel <-chan struct{}) {
 for {
 select {
 // ...
 case <-cancel:
 return
 }
 }
 }
15. Run method func newStateMachine() *stateMachine {
 return &stateMachine{
 state: "initial",
 actionc: make(chan func()),
 }
 } func (sm *stateMachine) Run(ctx context.Context) error {
 for {
 select {
 // ...
 case <-ctx.Done():
 return ctx.Err()
 }
 }
 }
16. go loop Run + Obvious (?)
 + Self-contained component + Lifecycle encapsulation - Tricky to get determinism - Tricky to make re-entrant
 
 · Simpler components
 · When easy > correct - More work for caller
 - More state to track
 + Less scaffolding in component
 + Straightforward determinism + Much easier to test! 
 · More complex components
 · When correct > easy
17. Outline Actor stuff Running stuff Orchestrating stuff ➫
18. Outline Actor stuff Running stuff Orchestrating stuff ➫
19. ctrl-C handler Storage HTTP API Cron State machine
20. // Setup
 ctx, cancel := context.WithCancel(context.Background()) 
 
 go sm.Run(ctx) 
 go http.ListenAndServe(":8080", api)
 go cronJobs(sm)
 go signalCatcher(???)
 
 // wait for shutdown somehow
21. // Setup
 ctx, cancel := context.WithCancel(context.Background())
 
 go sm.Run(ctx) // cancel() 
 go http.ListenAndServe(":8080", api) // ?
 go cronJobs(sm) // ?
 go signalCatcher(???) // ?
 
 // wait for shutdown somehow
22. // Setup
 ctx, cancel := context.WithCancel(context.Background()) 
 ln, _ := net.Listen("tcp", ":8080")
 
 go sm.Run(ctx) // cancel()
 go http.Serve(ln, api) // ln.Close()
 go cronJobs(sm) // ?
 go signalCatcher(???) // ?
 
 // wait for shutdown somehow
23. // Setup
 ctx, cancel := context.WithCancel(context.Background()) 
 ln, _ := net.Listen("tcp", ":8080")
 cancelCron := make(chan struct{})
 
 go sm.Run(ctx) // cancel()
 go http.Serve(ln, api) // ln.Close()
 go cronJobs(cancelCron, sm) // close(cancelCron)
 go signalCatcher(???) // ?
 
 // wait for shutdown somehow
24. // Setup
 ctx, cancel := context.WithCancel(context.Background()) 
 ln, _ := net.Listen("tcp", ":8080")
 cancelCron := make(chan struct{})
 cancelSig := make(chan struct{})
 
 go sm.Run(ctx) // cancel()
 go http.Serve(ln, api) // ln.Close()
 go cronJobs(cancelCron, sm) // close(cancelCron)
 go signalCatcher(cancelSig) // close(cancelSig)
 
 // wait for shutdown somehow
25. gopkg.in/tomb.v2
26. golang.org/x/sync/errgroup
27. // Setup
 ctx, cancel := context.WithCancel(context.Background()) 
 ln, _ := net.Listen("tcp", ":8080")
 cancelCron := make(chan struct{})
 cancelSig := make(chan struct{})
 
 go sm.Run(ctx) // cancel()
 go http.Serve(ln, api) // ln.Close()
 go cronJobs(cancelCron, sm) // close(cancelCron)
 go signalCatcher(cancelSig) // close(cancelSig)
 
 // wait for shutdown somehow
28. github.com/oklog/oklog/pkg/group
29. github.com/oklog/oklog/pkg/group
30. github.com/oklog/oklog/pkg/group
31. Use a group var g group.Group
 {
 ctx, cancel := context.WithCancel(context.Background())
 g.Add(func() error {
 return sm.Run(ctx)
 }, func(error) {
 cancel()
 })
 }

32. Use a group 
 {
 ln, _ := net.Listen("tcp", ":8080")
 g.Add(func() error {
 return http.Serve(ln, api)
 }, func(error) {
 ln.Close()
 })
 }

33. Use a group 
 {
 cancel := make(chan struct{})
 g.Add(func() error {
 return cronJobs(cancel, sm)
 }, func(error) {
 close(cancel)
 })
 }

34. Use a group 
 {
 cancel := make(chan struct{})
 g.Add(func() error {
 return signalCatcher(cancel)
 }, func(error) {
 close(cancel)
 })
 }
 g.Run()
35. tomb errgroup + Complex lifecycles
 - Complex API - net.Listener? group + Small API
 - Context-only - net.Listener? + Super flexible
 - Esoteric API?
 + net.Listener!
36. Maybe the moral here is
 Give more control to the caller
 ~idk~
37. Ways To Do Things Thanks y'all