Ways To Do Things

A few productive patterns for designing and orchestrating components (actors) in Go programs. Presented at the Go SF Meetup on 25 August 2017.

1. Ways To Do Things By your friend,
 @peterbourgon
2. Outline Actor stuff Running stuff Orchestrating stuff
3. Outline Actor stuff Running stuff Orchestrating stuff ➫
4. ctrl-C handler Storage HTTP API Cron State machine
5. ctrl-C handler Storage HTTP API Cron State machine
6. Actors (basic) type stateMachine struct {
 state string
 fooc chan fooReq
 barc chan barReq
 quitc chan struct{}
 } func (sm *stateMachine) loop() {
 for {
 select {
 case r := <-sm.fooc:
 sm.handleFoo(r)
 case r := <-sm.barc:
 sm.handleBar(r)
 case <-sm.quitc:
 return
 }
 }
 } func (sm *stateMachine) foo() int {
 c := make(chan int)
 sm.fooc <- fooReq{c}
 return <-c
 }
7. Actors (advanced?) type stateMachine struct {
 state string
 actionc chan func()
 quitc chan struct{}
 } func (sm *stateMachine) loop() {
 for {
 select {
 case f := <-sm.actionc:
 f()
 case <-sm.quitc:
 return
 }
 }
 } func (sm *stateMachine) foo() int {
 c := make(chan int)
 sm.actionc <- func() {
 if sm.state == "A" {
 sm.state = "B"
 }
 c <- 123
 }
 return <-c
 }
9. N chans 1 chan + Clean separation of concerns - Logic separate from methods - More types and fields; verbose
 - Mixing mechanics and logic
 
 · Relatively few API methods
 · Request/response pattern - Maybe not totally obvious 
 + Colocate method and logic + Fewer types and fields
 + Mechanics and logic separate
 
 · More API methods
 · More complex input/output
10. Outline Actor stuff Running stuff Orchestrating stuff ➫
11. Outline Actor stuff Running stuff Orchestrating stuff ➫
12. Hidden goroutine func newStateMachine() *stateMachine {
 sm := &stateMachine{
 state: "initial",
 actionc: make(chan func()),
 quitc: make(chan struct{}),
 }
 go sm.loop()
 return sm
 } func (sm *stateMachine) loop() {
 for {
 select {
 // ...
 case <-sm.quitc:
 return
 }
 }
 } func (sm *stateMachine) stop() {
 close(sm.quitc)
 }
13. Hidden goroutine++ func newStateMachine() *stateMachine {
 sm := &stateMachine{
 state: "initial",
 actionc: make(chan func()),
 quitc: make(chan chan struct{}),
 }
 go sm.loop()
 return sm
 } func (sm *stateMachine) loop() {
 for {
 select {
 // ...
 case q := <-sm.quitc:
 close(q)
 return
 }
 }
 } func (sm *stateMachine) stop() {
 q := make(chan struct{})
 sm.quitc <- q
 <-q
 }
14. Run method func newStateMachine() *stateMachine {
 return &stateMachine{
 state: "initial",
 actionc: make(chan func()),
 }
 } func (sm *stateMachine) Run(cancel <-chan struct{}) {
 for {
 select {
 // ...
 case <-cancel:
 return
 }
 }
 }
15. Run method func newStateMachine() *stateMachine {
 return &stateMachine{
 state: "initial",
 actionc: make(chan func()),
 }
 } func (sm *stateMachine) Run(ctx context.Context) error {
 for {
 select {
 // ...
 case <-ctx.Done():
 return ctx.Err()
 }
 }
 }
16. go loop Run + Obvious (?)
 + Self-contained component + Lifecycle encapsulation - Tricky to get determinism - Tricky to make re-entrant
 
 · Simpler components
 · When easy > correct - More work for caller
 - More state to track
 + Less scaffolding in component
 + Straightforward determinism + Much easier to test! 
 · More complex components
 · When correct > easy
17. Outline Actor stuff Running stuff Orchestrating stuff ➫
18. Outline Actor stuff Running stuff Orchestrating stuff ➫
19. ctrl-C handler Storage HTTP API Cron State machine
20. // Setup
 ctx, cancel := context.WithCancel(context.Background()) 
 
 go sm.Run(ctx) 
 go http.ListenAndServe(":8080", api)
 go cronJobs(sm)
 go signalCatcher(???)
 
 // wait for shutdown somehow
21. // Setup
 ctx, cancel := context.WithCancel(context.Background())
 
 go sm.Run(ctx) // cancel() 
 go http.ListenAndServe(":8080", api) // ?
 go cronJobs(sm) // ?
 go signalCatcher(???) // ?
 
 // wait for shutdown somehow
22. // Setup
 ctx, cancel := context.WithCancel(context.Background()) 
 ln, _ := net.Listen("tcp", ":8080")
 
 go sm.Run(ctx) // cancel()
 go http.Serve(ln, api) // ln.Close()
 go cronJobs(sm) // ?
 go signalCatcher(???) // ?
 
 // wait for shutdown somehow
23. // Setup
 ctx, cancel := context.WithCancel(context.Background()) 
 ln, _ := net.Listen("tcp", ":8080")
 cancelCron := make(chan struct{})
 
 go sm.Run(ctx) // cancel()
 go http.Serve(ln, api) // ln.Close()
 go cronJobs(cancelCron, sm) // close(cancelCron)
 go signalCatcher(???) // ?
 
 // wait for shutdown somehow
24. // Setup
 ctx, cancel := context.WithCancel(context.Background()) 
 ln, _ := net.Listen("tcp", ":8080")
 cancelCron := make(chan struct{})
 cancelSig := make(chan struct{})
 
 go sm.Run(ctx) // cancel()
 go http.Serve(ln, api) // ln.Close()
 go cronJobs(cancelCron, sm) // close(cancelCron)
 go signalCatcher(cancelSig) // close(cancelSig)
 
 // wait for shutdown somehow
25. gopkg.in/tomb.v2
26. golang.org/x/sync/errgroup
27. // Setup
 ctx, cancel := context.WithCancel(context.Background()) 
 ln, _ := net.Listen("tcp", ":8080")
 cancelCron := make(chan struct{})
 cancelSig := make(chan struct{})
 
 go sm.Run(ctx) // cancel()
 go http.Serve(ln, api) // ln.Close()
 go cronJobs(cancelCron, sm) // close(cancelCron)
 go signalCatcher(cancelSig) // close(cancelSig)
 
 // wait for shutdown somehow
28. github.com/oklog/oklog/pkg/group
29. github.com/oklog/oklog/pkg/group
30. github.com/oklog/oklog/pkg/group
31. Use a group var g group.Group
 {
 ctx, cancel := context.WithCancel(context.Background())
 g.Add(func() error {
 return sm.Run(ctx)
 }, func(error) {
 cancel()
 })
 }

32. Use a group 
 {
 ln, _ := net.Listen("tcp", ":8080")
 g.Add(func() error {
 return http.Serve(ln, api)
 }, func(error) {
 ln.Close()
 })
 }

33. Use a group 
 {
 cancel := make(chan struct{})
 g.Add(func() error {
 return cronJobs(cancel, sm)
 }, func(error) {
 close(cancel)
 })
 }

34. Use a group 
 {
 cancel := make(chan struct{})
 g.Add(func() error {
 return signalCatcher(cancel)
 }, func(error) {
 close(cancel)
 })
 }
 g.Run()
35. tomb errgroup + Complex lifecycles
 - Complex API - net.Listener? group + Small API
 - Context-only - net.Listener? + Super flexible
 - Esoteric API?
 + net.Listener!
36. Maybe the moral here is
 Give more control to the caller
 ~idk~
37. Ways To Do Things Thanks y'all