Share
Embed
Slide 1
Slide 1 text
(Sync)testing Concurrent Code with Confidence
Alexander Baygeldin @ Evil Martians
Slide 2
Slide 2 text
fi
my rst quick &
easy Go project
Slide 3
Slide 3 text
my rst quick &
easy Go project
fi
new stack
Slide 4
Slide 4 text
my rst quick &
easy Go projec
fi
deadlines
new stack
Slide 5
Slide 5 text
my rst q
easy Go
fi
SLOs deadlines
new stack
Slide 6
Slide 6 text
Next steps:
1. Pray.
2. Do lots of testing.
Slide 7
Slide 7 text
Next steps:
1. Pray.
2. Do lots of testing.
concurrency
Slide 8
Slide 8 text
I know how to test
concurrent code!
Slide 9
Slide 9 text
I know how to test
concurrent code...
Slide 10
Slide 10 text
✨ testing/synctest ✨
Slide 11
Slide 11 text
“Non-deterministic tests have two problems, rstly
they are useless, secondly they are a virulent infection
that can completely ruin your entire test suite.”
fi
— Martin Fowler
Slide 12
Slide 12 text
func Greet(name string) string {
greetings := []string{"Hello", "Hi"}
idx := rand.Intn(len(greetings)) // Nondeterministic!
return fmt.Sprintf("%s, %s!", greetings[idx], name)
}
func TestGreet(t *testing.T) { // Flaky!
want := "Hi, Bob!"
if got := Greet("Bob"); got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
Why is my test
nondeterministic?
1.
Random number generation.
Slide 13
Slide 13 text
func Greet(name string) string {
if time.Now().Hour() < 12 { // Nondeterministic!
return "Good morning, " + name + "!"
}
return "Good afternoon, " + name + "!"
}
func TestGreet(t *testing.T) { // Flaky!
want := "Good morning, Alice!"
if got := Greet("Alice"); got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
Why is my test
nondeterministic?
1.
Random number generation.
2.
Unpredictable outside world.
Slide 14
Slide 14 text
func Greet(people map[string]struct{}) string {
keys := []string{}
for key := range people { // Nondeterministic!
keys = append(keys, key)
}
return "Hi, " + strings.Join(keys, " & ") + "!"
}
func TestGreet(t *testing.T) { // Flaky!
people := map[string]struct{}{
"Alice": {},
"Bob":
{},
}
want := "Hi, Alice & Bob!"
fi
if got := Greet(people); got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
Why is my test
nondeterministic?
1.
Random number generation.
2.
Unpredictable outside world.
3.
Unspeci ed language behavior.
Slide 15
Slide 15 text
“A programmer had a problem. He thought to
himself, "I know, I'll solve it with threads!". has
Now problems. two he”
— Lao Tzu, probably
Slide 16
Slide 16 text
func BatchGreeter(batchSize int) (
< chan string,
func(int),
) {
out := make(chan string)
names := make(chan string)
go func() {
batch := make([]string, 0, batchSize)
for name := range names {
batch = append(batch, name)
if len(batch) == batchSize {
out < "Hi, " + strings.Join(batch, " & ") + "!"
batch = batch[:0]
}
}
}()
-
-
fi
-
}
return out, func(userID int) {
go func() {
names < fetchUserName(userID)
}()
}
Why is my test
nondeterministic?
1.
Random number generation.
2.
Unpredictable outside world.
3.
Unspeci ed language behavior.
4.
Unreliable order of execution.
Slide 17
Slide 17 text
func BatchGreeter(batchSize int) (
< chan string,
func(int),
) {
out := make(chan string)
names := make(chan string)
go func() {
batch := make([]string, 0, batchSize)
for name := range names {
batch = append(batch, name)
if len(batch) == batchSize {
out < "Hi, " + strings.Join(batch, " & ") + "!"
batch = batch[:0]
}
}
}()
-
-
fi
-
}
return out, func(userID int) {
go func() {
names < fetchUserName(userID)
}()
}
Why is my test
nondeterministic?
1.
Random number generation.
2.
Unpredictable outside world.
3.
Unspeci ed language behavior.
4.
Unreliable order of execution.
Slide 18
Slide 18 text
func BatchGreeter(batchSize int) (
< chan string,
func(int),
) {
out := make(chan string)
names := make(chan string)
go func() {
batch := make([]string, 0, batchSize)
for name := range names {
batch = append(batch, name)
if len(batch) == batchSize {
out < "Hi, " + strings.Join(batch, " & ") + "!"
batch = batch[:0]
}
}
}()
return out, func(userID int) {
go func() {
names < fetchUserName(userID)
}()
-
-
fi
-
}
}
Why is my test
nondeterministic?
1.
Random number generation.
2.
Unpredictable outside world.
3.
Unspeci ed language behavior.
4.
Unreliable order of execution.
Slide 19
Slide 19 text
func BatchGreeter(batchSize int) (
< chan string,
func(int),
) {
out := make(chan string)
names := make(chan string)
go func() {
batch := make([]string, 0, batchSize)
for name := range names {
batch = append(batch, name)
if len(batch) == batchSize {
out < "Hi, " + strings.Join(batch, " & ") + "!"
batch = batch[:0]
}
}
}()
return out, func(userID int) {
go func() {
names < fetchUserName(userID)
}()
-
-
-
-
}
}
func TestBatchGreeter(t *testing.T) { // Flaky!
greetings, greet := BatchGreeter(2)
greet(1) // ID=1 is Alice
greet(2) // ID=2 is Bob
want := "Hi, Alice & Bob!"
}
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
Slide 20
Slide 20 text
flaky tests
func TestBatchGreeter(t *testing.T) {
greetings, greet := BatchGreeter(2)
greet(1) // ID=1 is Alice
time.Sleep(100 * time.Millisecond)
greet(2) // ID=2 is Bob
want := "Hi, Alice & Bob!"
-
time.Sleep
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
Slide 21
Slide 21 text
flaky tests
func TestBatchGreeter(t *testing.T) {
greetings, greet := BatchGreeter(2)
greet(1) // ID=1 is Alice
time.Sleep(100 * time.Millisecond)
greet(2) // ID=2 is Bob
want := "Hi, Alice & Bob!"
time.Sleep
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
-
eww! an ugly hack!
Slide 22
Slide 22 text
func BatchGreeter(batchSize int) (
< chan string,
func(int) < chan struct{},
) {
out := make(chan string)
names := make(chan string)
// ...
return out, func(userID int) < chan struct{} {
done := make(chan struct{})
go func() {
defer close(done)
names <
}()
-
-
-
return done
-
-
-
-
}
}
fetchUserName(userID)
func TestBatchGreeter(t *testing.T) {
greetings, greet := BatchGreeter(2)
go func() {
< greet(1) // ID=1 is Alice
< greet(2) // ID=2 is Bob
}()
want := "Hi, Alice & Bob!"
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
Slide 23
Slide 23 text
func BatchGreeter(batchSize int) (
< chan string,
func(int) < chan struct{},
) {
out := make(chan string)
names := make(chan string)
// ...
return out, func(userID int) < chan struct{} {
done := make(chan struct{})
go func() {
defer close(done)
names < fetchUserName(userID)
}()
-
-
-
-
-
-
}
}
return done
func TestBatchGreeterDoesNotGreet(
t *testing.T,
) {
greetings, greet := BatchGreeter(2)
< greet(1) // ID=1 is Alice
select {
case got := < greetings:
t.Fatalf("unexpected greeting: %q", got)
default:
}
}
Slide 24
Slide 24 text
func BatchGreeter(batchSize int) (
< chan string,
func(int) < chan struct{},
) {
out := make(chan string)
names := make(chan string)
// ...
return out, func(userID int) < chan struct{} {
done := make(chan struct{})
go func() {
defer close(done)
names < fetchUserName(userID)
}()
< greet(1) // ID=1 is Alice
select {
case got := < greetings:
t.Fatalf("unexpected greeting: %q", got)
default:
}
}
how do you know that
something did not happen?
-
-
-
-
-
-
}
}
return done
func TestBatchGreeterDoesNotGreet(
t *testing.T,
) {
greetings, greet := BatchGreeter(2)
Slide 25
Slide 25 text
func BatchGreeter(batchSize int) (
< chan string,
func(int) < chan struct{},
) {
out := make(chan string)
names := make(chan string)
// ...
go func() {
defer close(done)
names < fetchUserName(userID)
}()
-
-
-
return done
-
-
-
< greet(1) // ID=1 is Alice
time.Sleep(100 * time.Millisecond)
return out, func(userID int) < chan struct{} {
done := make(chan struct{})
}
}
func TestBatchGreeterNotGreet(t *testing.T) {
greetings, greet := BatchGreeter(2)
select {
case got := < greetings:
t.Fatalf("unexpected greeting: %q", got)
default:
}
}
Slide 26
Slide 26 text
func BatchGreeter(batchSize int) (
< chan string,
func(int) < chan struct{},
) {
out := make(chan string)
names := make(chan string)
// ...
return out, func(userID int) < chan struct{} {
done := make(chan struct{})
go func() {
defer close(done)
names < fetchUserName(userID)
}()
-
-
-
-
-
-
-
}
}
return done
func TestBatchGreeterNotGreet(t *testing.T) {
greetings, greet := BatchGreeter(2)
< greet(1) // ID=1 is Alice
select {
case got := < greetings:
t.Fatalf("unexpected greeting: %q", got)
case < time.After(100 * time.Millisecond):
}
}
Slide 27
Slide 27 text
type request struct {
name string
done chan struct{}
}
func BatchGreeter(batchSize int) (
< chan string,
func(int) < chan struct{},
) {
out := make(chan string)
names := make(chan request)
go func() {
batch := make([]string, 0, batchSize)
for request := range names {
batch = append(batch, request.name)
if len(batch) == batchSize {
out < "Hi, " + strings.Join(batch, " & ") + "!"
batch = batch[:0]
}
close(request.done)
}
}()
return out, func(userID int) < chan struct{} {
done := make(chan struct{})
go func() {
names < request{
name: fetchUserName(userID),
done: done,
}
}()
-
-
-
}
-
-
-
-
}
return done
func TestBatchGreeterNotGreet(t *testing.T) {
greetings, greet := BatchGreeter(2)
< greet(1) // ID=1 is Alice
select {
case got := < greetings:
t.Fatalf("unexpected greeting: %q", got)
default:
}
}
Slide 28
Slide 28 text
type request struct {
name string
done chan struct{}
}
func BatchGreeter(batchSize int) (
< chan string,
func(int) < chan struct{},
) {
out := make(chan string)
names := make(chan request)
go func() {
batch := make([]string, 0, batchSize)
for request := range names {
batch = append(batch, request.name)
if len(batch) == batchSize {
out < "Hi, " + strings.Join(batch, " & ") + "!"
batch = batch[:0]
}
close(request.done)
}
}()
return out, func(userID int) < chan struct{} {
done := make(chan struct{})
go func() {
names < request{
name: fetchUserName(userID),
done: done,
}
}()
-
-
-
}
-
-
-
-
}
return done
func TestBatchGreeterNotGreet(t *testing.T) {
greetings, greet := BatchGreeter(2)
< greet(1) // ID=1 is Alice
select {
case got := < greetings:
t.Fatalf("unexpected greeting: %q", got)
default:
}
}
how do you know
it actually works?
Slide 29
Slide 29 text
type request struct {
name string
done chan struct{}
}
func BatchGreeter(batchSize int) (
< chan string,
func(int) < chan struct{},
) {
out := make(chan string)
names := make(chan request)
go func() {
batch := make([]string, 0, batchSize)
for request := range names {
batch = append(batch, request.name)
if len(batch) == batchSize {
out < "Hi, " + strings.Join(batch, " & ") + "!"
batch = batch[:0]
}
close(request.done)
}
}()
return out, func(userID int) < chan struct{} {
done := make(chan struct{})
go func() {
names < request{
name: fetchUserName(userID),
done: done,
}
}()
-
-
-
-
-
-
-
}
}
-
return done
func TestBatchGreeterNotGreet(t *testing.T) {
greetings, greet := BatchGreeter(2)
select {
case < greet(1): // ID=1 is Alice
case < time.After(time.Second):
t.Fatal("timed out waiting for Alice")
}
select {
case got := < greetings:
t.Fatalf("unexpected greeting: %q", got)
default:
}
}
Slide 30
Slide 30 text
type request struct {
name string
done chan struct{}
}
func BatchGreeter(batchSize int) (
< chan string,
func(int),
) {
out := make(chan string)
names := make(chan string)
func BatchGreeter(batchSize int) (
< chan string,
func(int) < chan struct{},
) {
out := make(chan string)
names := make(chan request)
go func() {
batch := make([]string, 0, batchSize)
go func() {
batch := make([]string, 0, batchSize)
}()
for request := range names {
batch = append(batch, request.name)
if len(batch) == batchSize {
out < "Hi, " + strings.Join(batch, " & ") + "!"
batch = batch[:0]
}
for name := range names {
batch = append(batch, name)
if len(batch) == batchSize {
out < "Hi, " + strings.Join(batch, " & ") + "!"
batch = batch[:0]
}
}
}()
return out, func(userID int) {
go func() {
names < fetchUserName(userID)
}()
}
}
return out, func(userID int) < chan struct{} {
done := make(chan struct{})
go func() {
names < request{
name: fetchUserName(userID),
done: done,
}
}()
-
-
-
-
-
-
-
}
-
}
close(request.done)
}
return done
Slide 31
Slide 31 text
type request struct {
name string
done chan struct{}
}
func BatchGreeter(batchSize int) (
< chan string,
func(int) < chan struct{},
) {
out := make(chan string)
names := make(chan request)
func BatchGreeter(batchSize int) (
< chan string,
func(int),
) {
out := make(chan string)
names := make(chan string)
go func() {
batch := make([]string, 0, batchSize)
}()
for name := range names {
batch = append(batch, name)
if len(batch) == batchSize {
out < "Hi, " + strings.Join(batch, " & ") + "!"
batch = batch[:0]
}
}
go func() {
batch := make([]string, 0, batchSize)
was it
worth it?
for request := range names {
batch = append(batch, request.name)
if len(batch) == batchSize {
out < "Hi, " + strings.Join(batch, " & ") + "!"
batch = batch[:0]
}
🤔🤔🤔
}()
return out, func(userID int) {
go func() {
names < fetchUserName(userID)
}()
}
}
return out, func(userID int) < chan struct{} {
done := make(chan struct{})
go func() {
names < request{
name: fetchUserName(userID),
done: done,
}
}()
-
-
-
-
-
-
-
}
-
}
close(request.done)
}
return done
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
No content
Slide 34
Slide 34 text
Instrumentation:
1.
Complicated.
2.
Fragile.
Slide 35
Slide 35 text
Instrumentation:
time.Sleep:
1.
Complicated.
1.
2.
Fragile.
Slow.
Slide 36
Slide 36 text
Quiesce nce
[ k w i ˈ ɛ s ə n s ]
n o u n
The state of being at rest, quiet,
still, inactive or motionless.
Slide 37
Slide 37 text
Q
time
Slide 38
Slide 38 text
greet("Alice")
Q
time
Slide 39
Slide 39 text
greet("Alice")
some computation happening
Q
Q
time
Slide 40
Slide 40 text
greet("Alice")
greet("Bob")
some computation happening
Q
Q
time
Slide 41
Slide 41 text
greet("Alice")
greet("Bob")
some computation happening
Q
some computation happening
Q
time
Q
Slide 42
Slide 42 text
greet("Alice")
greet("Bob")
some computation happening
-
Q
< greetings
some computation happening
Q
time
Q
Slide 43
Slide 43 text
time.Sleep(100 * time.Millisecond)
greet("Alice")
greet("Bob")
some computation happening
Q
Q
time
Slide 44
Slide 44 text
fl
aky 👎 👎
less
slow 👎
time.Sleep
more
Slide 45
Slide 45 text
func TestBatchGreeter(t *testing.T) {
greetings, greet := BatchGreeter(2)
func TestBatchGreeter(t *testing.T) {
greetings, greet := BatchGreeter(2)
greet(1) // ID=1 is Alice
greet(1) // ID=1 is Alice
time.Sleep(100 * time.Millisecond)
synctest.Wait()
want := "Hi, Alice & Bob!"
want := "Hi, Alice & Bob!"
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
-
-
greet(2) // ID=2 is Bob
greet(2) // ID=2 is Bob
Slide 46
Slide 46 text
func TestBatchGreeter(t *testing.T) {
greetings, greet := BatchGreeter(2)
greet(1) // ID=1 is Alice
synctest.Wait()
greet(2) // ID=2 is Bob
want := "Hi, Alice & Bob!"
-
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
Slide 47
Slide 47 text
func TestBatchGreeter(t *testing.T) {
greetings, greet := BatchGreeter(2)
greet(1) // ID=1 is Alice
synctest.Wait()
greet(2) // ID=2 is Bob
want := "Hi, Alice & Bob!"
-
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
}
Slide 48
Slide 48 text
func TestBatchGreeter(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
greetings, greet := BatchGreeter(2)
greet(1) // ID=1 is Alice
synctest.Wait()
greet(2) // ID=2 is Bob
want := "Hi, Alice & Bob!"
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
})
-
}
Slide 49
Slide 49 text
func TestBatchGreeter(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
greetings, greet := BatchGreeter(2)
greet(1) // ID=1 is Alice
synctest.Wait()
greet(2) // ID=2 is Bob
want := "Hi, Alice & Bob!"
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
})
-
}
Slide 50
Slide 50 text
Quiesce nce*
* s y n c t e s t
s t y l e
The state when all goroutines in
the bubble are durably blocked.
Slide 51
Slide 51 text
Durably blocking operations
• time.Sleep
• sync.WaitGroup.Wait
• sync.Cond.Wait
• send/receive to/from a bubbled channel
• send/receive to/from a nil channel
• select{}
• select where all cases are durably blocked
synctest.Wait() returns only when all
goroutines are blocked in one of these ways
Slide 52
Slide 52 text
Durably blocking operations
• time.Sleep
• sync.WaitGroup.Wait
• sync.Cond.Wait
• send/receive to/from a bubbled channel
• send/receive to/from a nil channel
• select{}
• select where all cases are durably blocked
synctest.Wait() returns only when all
goroutines are blocked in one of these ways
Slide 53
Slide 53 text
Durably blocking operations
• time.Sleep
• sync.WaitGroup.Wait
• sync.Cond.Wait
• send/receive to/from a bubbled channel
• send/receive to/from a nil channel
• select{}
• select where all cases are durably blocked
synctest.Wait() returns only when all
goroutines are blocked in one of these ways
Slide 54
Slide 54 text
Durably blocking operations
• time.Sleep
• sync.WaitGroup.Wait
• sync.Cond.Wait
• send/receive to/from a bubbled channel
• send/receive to/from a nil channel
• select{}
• select where all cases are durably blocked
synctest.Wait() returns only when all
goroutines are blocked in one of these ways
Slide 55
Slide 55 text
Durably blocking operations
• time.Sleep
• sync.WaitGroup.Wait
• sync.Cond.Wait
• send/receive to/from a bubbled channel
• send/receive to/from a nil channel
• select{}
• select where all cases are durably blocked
synctest.Wait() returns only when all
goroutines are blocked in one of these ways
Slide 56
Slide 56 text
func Test(t *testing.T, f func(*testing.T)) {
bubble := &synctestBubble{
done:
false, // Whether the main goroutine has exited
waiting: false, // Whether a synctest.Wait call is pending
running: 1,
// Number of goroutines not durably blocked
}
currentGoroutine.bubble = bubble
defer func() {
currentGoroutine.bubble = nil
}()
go f(t) // Start the main goroutine!
// ...
}
Slide 57
Slide 57 text
func Test(t *testing.T, f func(*testing.T)) {
bubble := &synctestBubble{
done:
false, // Whether the main goroutine has exited
waiting: false, // Whether a synctest.Wait call is pending
running: 1,
// Number of goroutines not durably blocked
}
currentGoroutine.bubble = bubble
defer func() {
currentGoroutine.bubble = nil
}()
go f(t) // Start the main goroutine!
// ...
}
Slide 58
Slide 58 text
func Test(t *testing.T, f func(*testing.T)) {
bubble := &synctestBubble{
done:
false, // Whether the main goroutine has exited
waiting: false, // Whether a synctest.Wait call is pending
running: 1,
// Number of goroutines not durably blocked
}
currentGoroutine.bubble = bubble
defer func() {
currentGoroutine.bubble = nil
}()
go f(t) // Start the main goroutine!
// ...
}
Slide 59
Slide 59 text
When a durable operation happens,
Go runtime updates the bubble!
func Test(t *testing.T, f func(*testing.T)) {
bubble := &synctestBubble{
done:
false, // Whether the main goroutine has exited
waiting: false, // Whether a synctest.Wait call is pending
running: 1,
// Number of goroutines not durably blocked
}
currentGoroutine.bubble = bubble
defer func() {
currentGoroutine.bubble = nil
}()
go f(t) // Start the main goroutine!
// ...
}
Slide 60
Slide 60 text
Non-durably blocking operations
• syscalls, cgo calls, anything that isn’t Go
fi
• I/O ( les, pipes, network connections, etc.)
Slide 61
Slide 61 text
Proceed with caution
⚠ Using the network.
⚠ Interacting with external processes.
Slide 62
Slide 62 text
Proceed with caution
⚠ Using the network.
⚠ Interacting with external processes.
func BatchGreeter(batchSize int) (
< chan string,
func(int),
) {
// ...
return out, func(userID int) {
go func() {
-
-
}
}
names <
}()
fetchUserName(userID)
Slide 63
Slide 63 text
Proceed with caution
⚠ Using the network.
⚠ Interacting with external processes.
func BatchGreeter(batchSize int) (
< chan string,
func(int),
) {
// ...
return out, func(userID int) {
go func() {
}
}
names <
}()
fetchUserName(userID)
-
-
just make sure this returns
Slide 64
Slide 64 text
func GreetServer(conn net.Conn) error {
name, err := bufio.NewReader(conn).ReadString('!')
if err != nil {
return err
}
}
_, err = fmt.Fprintf(conn, "Hi, %s!", name[:len(name)-1])
return err
func GreetClient(conn net.Conn, name string) (string, error) {
if _, err := fmt.Fprintf(conn, "%s!", name); err != nil {
return "", err
}
}
return bufio.NewReader(conn).ReadString('!')
Slide 65
Slide 65 text
func GreetServer(conn net.Conn) error {
name, err := bufio.NewReader(conn).ReadString('!')
if err != nil {
return err
}
}
_, err = fmt.Fprintf(conn, "Hi, %s!", name[:len(name)-1])
return err
func GreetClient(conn net.Conn, name string) (string, error) {
if _, err := fmt.Fprintf(conn, "%s!", name); err != nil {
return "", err
}
}
return bufio.NewReader(conn).ReadString('!')
func TestGreetServerClient(t *testing.T) {
synctest.Test(t, func(*testing.T) {
serverConn, clientConn := net.Pipe()
defer clientConn.Close()
defer serverConn.Close()
})
}
// ...
use a fake network connection!
Slide 66
Slide 66 text
Non-durably blocking operations
• syscalls, cgo calls, anything that isn’t Go
• I/O ( les, pipes, network connections, etc.)
• sync.Mutex.Lock and sync.RWMutex.Lock ???
• Runtime instrumentation is not free.
fi
• Mutexes are not usually held for long periods of time.
Slide 67
Slide 67 text
type ThreadSafeGreeter struct {
mu sync.Mutex
}
type ThreadSafeGreeter struct {
mu fakemutex.Mutex
}
Slide 68
Slide 68 text
//go:build !fakemutex
package fakemutex
import "sync"
type ThreadSafeGreeter struct {
mu sync.Mutex
}
type Mutex = sync.Mutex
type ThreadSafeGreeter struct {
mu fakemutex.Mutex
}
Slide 69
Slide 69 text
//go:build !fakemutex
package fakemutex
import "sync"
type ThreadSafeGreeter struct {
mu sync.Mutex
}
type Mutex = sync.Mutex
//go:build fakemutex
package fakemutex
type Mutex struct { /* ... */ }
func (m *Mutex) Lock() { /* ... */ }
func (m *Mutex) Unlock() { /* ... */ }
func (m *Mutex) TryLock() bool { /* ... */ }
type ThreadSafeGreeter struct {
mu fakemutex.Mutex
}
Slide 70
Slide 70 text
type Mutex struct {
once sync.Once
ch
chan struct{}
}
func (m *Mutex) init() {
m.once.Do(func() {
m.ch := make(chan struct{}, 1)
m.ch < struct{}{}
})
}
YO DAWG I HEARD YOU LIKE MUTEXES
func (m *Mutex) Lock() {
m.init()
< m.ch
}
func (m *Mutex) Unlock() {
m.init()
select {
case m.ch < struct{}{}:
default:
panic("sync: unlock of unlocked mutex")
}
}
-
-
-
// ...
SO I PUT CHANNELS IN YOUR MUTEXES
SO YOU CAN USE MUTEXES WHILE YOU USE MUTEXES
Slide 71
Slide 71 text
No-no's
🚫 Operating on a bubbled channel, timer,
or ticker from outside the bubble.
🚫 Calling Add or Go on a bubbled
sync.WaitGroup from outside the bubble.
🚫 Waking a bubbled goroutine blocked on
Cond.Wait from outside the bubble.
Slide 72
Slide 72 text
No-no's
🚫 Operating on a bubbled channel, timer,
or ticker from outside the bubble.
func TestBatchGreeter(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
greetings, greet := BatchGreeter(2)
🚫 Calling Add or Go on a bubbled
greet(1) // ID=1 is Alice
synctest.Wait()
sync.WaitGroup from outside the bubble.
greet(2) // ID=2 is Bob
🚫 Waking a bubbled goroutine blocked on
want := "Hi, Alice & Bob!"
Cond.Wait from outside the bubble.
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
})
-
}
Slide 73
Slide 73 text
No-no's
🚫 Operating on a bubbled channel, timer,
or ticker from outside the bubble.
func TestBatchGreeter(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
greetings, greet := BatchGreeter(2)
🚫 Calling Add or Go on a bubbled
greet(1) // ID=1 is Alice
synctest.Wait()
sync.WaitGroup from outside the bubble.
greet(2) // ID=2 is Bob
🚫 Waking a bubbled goroutine blocked on
want := "Hi, Alice & Bob!"
Cond.Wait from outside the bubble.
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
})
-
}
Slide 74
Slide 74 text
No-no's
🚫 Operating on a bubbled channel, timer,
or ticker from outside the bubble.
func TestBatchGreeter(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
greetings, greet := BatchGreeter(2)
🚫 Calling Add or Go on a bubbled
greet(1) // ID=1 is Alice
synctest.Wait()
sync.WaitGroup from outside the bubble.
greet(2) // ID=2 is Bob
🚫 Waking a bubbled goroutine blocked on
want := "Hi, Alice & Bob!"
Cond.Wait from outside the bubble.
-
🚫 Leaking goroutines inside the bubble.
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
})
}
Slide 75
Slide 75 text
func BatchGreeter(batchSize int) (
< chan string,
func(int),
) {
out := make(chan string)
names := make(chan string)
synctest.Test(t, func(t *testing.T) {
greetings, greet := BatchGreeter(2)
greet(1) // ID=1 is Alice
go func() {
batch := make([]string, 0, batchSize)
synctest.Wait()
for name := range names {
// ...
}
}()
want := "Hi, Alice & Bob!"
-
-
return out, func(userID int) {
go func() {
names < fetchUserName(userID)
}()
}
}
-
func TestBatchGreeter(t *testing.T) {
greet(2) // ID=2 is Bob
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
})
}
panic: main bubble goroutine has exited
but blocked goroutines remain! 😱 😱 😱
Slide 76
Slide 76 text
func BatchGreeter(batchSize int) (
< chan string,
func(int),
) {
out := make(chan string)
names := make(chan string)
synctest.Test(t, func(t *testing.T) {
greetings, greet := BatchGreeter(2)
greet(1) // ID=1 is Alice
go func() {
batch := make([]string, 0, batchSize)
synctest.Wait()
for name := range names {
// ...
}
}()
want := "Hi, Alice & Bob!"
-
-
return out, func(userID int) {
go func() {
names < fetchUserName(userID)
}()
}
}
-
func TestBatchGreeter(t *testing.T) {
greet(2) // ID=2 is Bob
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
})
}
panic: main bubble goroutine has exited
but blocked goroutines remain! 😱 😱 😱
Slide 77
Slide 77 text
func BatchGreeter(batchSize int) (
< chan string,
func(int),
func(),
) {
out := make(chan string)
names := make(chan string)
func TestBatchGreeter(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
greetings, greet, stop := BatchGreeter(2)
greet(1) // ID=1 is Alice
synctest.Wait()
greet(2) // ID=2 is Bob
go func() {
batch := make([]string, 0, batchSize)
stop() // All good now!
for name := range names {
// ...
}
want := "Hi, Alice & Bob!"
}()
-
-
-
return out, func(userID int) {
go func() {
names < fetchUserName(userID)
}()
}, func() { close(names) }
}
})
}
if got := < greetings; got != want {
t.Fatalf("got %q, want %q", got, want)
}
Slide 78
Slide 78 text
The Catch?
👍 Test isolation.
👍 Healthy test pyramid.
👍 Using channels for communication.
⏱ TIME!
>synctest's limitations
>look inside
>good practices
Slide 79
Slide 79 text
THAT MOMENT WHEN
synctest.Wait() RETURNS
Slide 80
Slide 80 text
func Test(t *testing.T, f func(*testing.T)) {
// ...
go f(t)
for {
bubble.waitUntilQuiescent()
if bubble.waiting {
bubble.resume() // synctest.Wait() returns
continue
}
if !bubble.done && bubble.hasSomethingScheduled() {
bubble.advanceTimeUntilSomethingHappens()
continue
}
}
break
// ...
}
Slide 81
Slide 81 text
func Test(t *testing.T, f func(*testing.T)) {
// ...
go f(t)
for {
bubble.waitUntilQuiescent()
if bubble.waiting {
bubble.resume() // synctest.Wait() returns
continue
}
if !bubble.done && bubble.hasSomethingScheduled() {
bubble.advanceTimeUntilSomethingHappens()
continue
}
}
break
// ...
}
Slide 82
Slide 82 text
func Test(t *testing.T, f func(*testing.T)) {
// ...
go f(t)
for {
bubble.waitUntilQuiescent()
if bubble.waiting {
bubble.resume() // synctest.Wait() returns
continue
}
if !bubble.done && bubble.hasSomethingScheduled() {
bubble.advanceTimeUntilSomethingHappens()
continue
}
}
break
// ...
}
Slide 83
Slide 83 text
func Test(t *testing.T, f func(*testing.T)) {
// ...
go f(t)
for {
bubble.waitUntilQuiescent()
if bubble.waiting {
bubble.resume() // synctest.Wait() returns
continue
}
if !bubble.done && bubble.hasSomethingScheduled() {
bubble.advanceTimeUntilSomethingHappens()
continue
}
}
break
// ...
}
Slide 84
Slide 84 text
func Test(t *testing.T, f func(*testing.T)) {
// ...
go f(t)
for {
bubble.waitUntilQuiescent()
if bubble.waiting {
bubble.resume() // synctest.Wait() returns
continue
}
if !bubble.done && bubble.hasSomethingScheduled() {
bubble.advanceTimeUntilSomethingHappens()
continue
}
}
break
// ...
}
Slide 85
Slide 85 text
func Test(t *testing.T, f func(*testing.T)) {
// ...
go f(t)
for {
bubble.waitUntilQuiescent()
if bubble.waiting {
bubble.resume() // synctest.Wait() returns
continue
}
if !bubble.done && bubble.hasSomethingScheduled() {
bubble.advanceTimeUntilSomethingHappens()
continue
}
}
break
// ...
}
Slide 86
Slide 86 text
“You can think of the bubble as simulating an
in nitely fast computer: Any amount of
computation takes no time.”
fi
— Damien Neil, author of synctest
Slide 87
Slide 87 text
func TestSynctestUnderstanding(t *testing.T) {
var mu sync.Mutex
go func() {
mu.Lock()
defer mu.Unlock()
time.Sleep(10 * time.Millisecond)
}()
mu.Lock() // OK!
})
}
Slide 88
Slide 88 text
func TestSynctestUnderstanding(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
var mu sync.Mutex
go func() {
mu.Lock()
defer mu.Unlock()
time.Sleep(10 * time.Millisecond)
}()
synctest.Wait()
mu.Lock() // Deadlock!
})
}
Slide 89
Slide 89 text
func TestSynctestUnderstanding(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
var mu sync.Mutex
go func() {
mu.Lock()
defer mu.Unlock()
time.Sleep(10 * time.Millisecond)
}()
synctest.Wait()
mu.Lock() // Deadlock!
})
}
Slide 90
Slide 90 text
func TestSynctestUnderstanding(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
var mu sync.Mutex
go func() {
mu.Lock()
defer mu.Unlock()
time.Sleep(10 * time.Millisecond)
}()
synctest.Wait()
mu.Lock() // Deadlock!
})
}
Slide 91
Slide 91 text
func TestSynctestUnderstanding(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
var mu sync.Mutex
go func() {
mu.Lock()
defer mu.Unlock()
time.Sleep(10 * time.Millisecond)
}()
synctest.Wait()
mu.Lock() // Deadlock!
})
}
Slide 92
Slide 92 text
func TestSynctestUnderstanding(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
var mu sync.Mutex
go func() {
mu.Lock()
defer mu.Unlock()
time.Sleep(10 * time.Millisecond)
}()
synctest.Wait()
mu.Lock() // Deadlock!
})
}
Slide 93
Slide 93 text
func TestSynctestUnderstanding(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
var mu sync.Mutex
go func() {
mu.Lock()
defer mu.Unlock()
time.Sleep(10 * time.Millisecond)
}()
synctest.Wait()
mu.Lock() // Deadlock!
})
}
Slide 94
Slide 94 text
func TestSynctestUnderstanding(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
var mu sync.Mutex
go func() {
mu.Lock()
defer mu.Unlock()
time.Sleep(10 * time.Millisecond)
}()
synctest.Wait()
mu.Lock() // Deadlock!
})
}
Slide 95
Slide 95 text
func TestSynctestUnderstanding(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
var mu sync.Mutex
go func() {
mu.Lock()
defer mu.Unlock()
time.Sleep(10 * time.Millisecond)
}()
synctest.Wait()
mu.Lock() // Deadlock!
})
}
Slide 96
Slide 96 text
func TestSynctestUnderstanding(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
var mu sync.Mutex
go func() {
mu.Lock()
defer mu.Unlock()
time.Sleep(10 * time.Millisecond)
}()
synctest.Wait()
mu.Lock() // Timelock 😱
})
}
Slide 97
Slide 97 text
Do not wait for a future event in a way
that prevents it from ever happening!
func TestSynctestUnderstanding(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
var mu sync.Mutex
go func() {
mu.Lock()
defer mu.Unlock()
time.Sleep(10 * time.Millisecond)
}()
synctest.Wait()
mu.Lock() // Timelock 😱
})
}
Slide 98
Slide 98 text
My life before synctest
1. I still needed fast feedback...
2. So, I did some refactoring!
• How can I maximize coverage...
• ...while minimizing time.Sleep?
Slide 99
Slide 99 text
T
T
B
= Transport layer
E
= Execution policy
B
= Business logic
E
E
E
B
B
T
B
B
B
E
B
E
T
Slide 100
Slide 100 text
E
T
E
E
T
= Transport layer
E
= Execution policy
B
= Business logic
T
E
T
E
B
B
B
B
B
B
B
Slide 101
Slide 101 text
E
time.Sleep(...)
T
= Transport layer
E
= Execution policy
B
= Business logic
T
+
E
E
T
E
T
E
B
B
B
=
B
B
B
B
still meh
Slide 102
Slide 102 text
T
T
E
T
= Transport layer
E
= Execution policy
B
= Business logic
E
E
E
B
B
B
T
B
B
E
B
B
Slide 103
Slide 103 text
T
T
E
T
= Transport layer
E
= Execution policy
B
= Business logic
E
E
E
B
B
B
T
B
B
E
B
B
Slide 104
Slide 104 text
T
T
E
T
= Transport layer
E
= Execution policy
B
= Business logic
time.Sleep(...)
+
E
E
E
B
B
B
T
B
B
=
E
B
B
acceptable
Slide 105
Slide 105 text
T
T
E
T
= Transport layer
E
= Execution policy
B
= Business logic
synctest.Wait()
+
E
E
E
B
B
B
T
B
B
=
E
B
B
0.010s
Slide 106
Slide 106 text
T
T
E
T
= Transport layer
E
= Execution policy
B
= Business logic
E
E
E
B
B
B
T
B
B
E
B
B
Slide 107
Slide 107 text
T
T
E
T
= Transport layer
E
= Execution policy
B
= Business logic
high cohesion
E
E
E
B
B
B
T
B
B
low coupling
E
B
B
Slide 108
Slide 108 text
Lessons!
• Testing with synctest is awesome: not complicated, not fragile, not slow.
• A clean design tends to also be synctest-friendly!
• If testing concurrent code feels painful, perhaps it's not you. It's the tooling.
Slide 109
Slide 109 text
Lessons!
• Testing with synctest is awesome: not complicated, not fragile, not slow.
• A clean design tends to also be synctest-friendly!
• If testing concurrent code feels painful, perhaps it's not you. It's the tooling.
I started building it for Ruby!
github.com/baygeldin/synctest-rb
Slide 110
Slide 110 text
THANK
YOU!
evilmartians.com/chronicles
github.com/baygeldin/synctest-rb