The end of_execution in Go code (DevoxxFR 2023)

There are many ways a function, including a unit test, can end its execution in Go code. Some of these ways include return, os.Exit, runtime.Goexit.

They all interact in different ways with the defer instruction, the testing.Cleanup function, and the actual goroutine execution.

Let's see how to get our code to end its life properly, ensuring defer and Cleanup work as expected even in the face of panics, fatals, and exits.

This session was presented at the DevoxxFR 2023 conference.

Transcript

1. Frédéric G. MARAND - 23 avril 2023
   The end of execution
   
   
   (In a Go program)
   Devoxx FR 2023
   1
   

   View Slide

2. © 2023 OSInet
   The problem to solve
   2
   

   View Slide

3. © 2023 OSInet 3
   Why we can’t have nice things
   What we want:
   
   
   ● defer / recover is a real nice Go
   pair of features
   
   
   ● Let’s use them both !
   How Go breaks them:
   
   
   ● func main() doesn’t return a
   process exit code to the OS
   
   
   ● os.Exit() emits a process exit
   code, but skips defer.
   
   
   ● os.Exit(0) in SUT panics
   
   
   ● os.Exit(nonZero) in SUT kills
   the test
   

   View Slide

4. © 2023 OSInet 4
   How execution ends
   Functions / goroutines:
   
   
   ● return
   ○ Runs defer-red code
   
   
   ○ Terminates goroutine if called using go fn()
   
   
   ○ Does not set exit code in main
   
   
   ● panic
   
   
   ○ Prepare stack trace
   
   
   ○ Return (see above)
   
   
   ○ If not recovered, dump trace
   
   
   ● os.Exit, syscall.Exit, log.Fatal*
   
   
   ○ Exit without returning
   Goroutines:
   
   
   ● All userland goroutines
   
   
   ○ End of return
   ● os.Exit, syscall.Exit, log.Fatal*
   ○ No defer-red code
   

   Exit without return-ing
   
   
   ● runtime.Goexit
   ○ Runs defer-red code
   
   
   ○ Ends goroutine
   
   
   ○ Does not handle #1 finishing program: deadlock
   Two separate notions: the call flow, and the scheduling (goroutine)
   

   View Slide

5. © 2023 OSInet 5
   os.Exit() vs defer-red code
   

   View Slide

6. © 2023 OSInet 6
   runtime.Goexit vs main()
   package main
   
   
   import (
   
   
   "log"
   
   
   "runtime"
   
   
   )
   
   
   func main() {
   
   
   defer func() {
   
   
   log.Println("Deferred code")
   
   
   }()
   
   
   log.Println("In main")
   
   
   runtime.Goexit()
   
   
   // DEADLOCK PANIC !
   
   
   log.Println("Nothing after os.Exit")
   
   
   }
   

   View Slide

7. © 2023 OSInet 7
   os.Exit() in tests
   package problem3
   
   
   import (
   
   
   "log"
   
   
   "os"
   
   
   )
   
   
   func Exit0() {
   
   
   log.Println("In Exit0")
   
   
   os.Exit(0)
   
   
   }
   
   
   func Exit1() {
   
   
   log.Println("In Exit1")
   
   
   os.Exit(1)
   
   
   }
   func TestExit(t *testing.T) {
   
   
   for _, test := range [...]struct {
   
   
   name string
   
   
   fn func()
   
   
   }{
   
   
   {"Exit0", Exit0},
   
   
   {"Exit1", Exit1},
   
   
   } {
   
   
   t.Run(fmt.Sprintf("%#v", test.name),
   

   func(t *testing.T) {
   
   
   defer func() {
   
   
   rec := recover()
   
   
   switch x := rec.(type) {
   
   
   case nil:
   
   
   t.Fatal("nothing recovered")
   
   
   case string:
   
   
   if !strings.Contains(x,
   

   "unexpected call to os.Exit(0)") {
   
   
   t.Fatalf("unexpected string: %q", x)
   
   
   }
   
   
   t.Log("successful recovery")
   
   
   default:
   
   
   t.Fatalf("unexpected recovery: %#v", x)
   
   
   }
   
   
   }()
   
   
   test.fn()
   
   
   })
   
   
   }
   
   
   }
   

   View Slide

8. © 2023 OSInet 8
   os.Exit() in tests: 0 vs 1
   % go test -v -run TestExit/Exit0
   
   
   === RUN TestExit
   
   
   === RUN TestExit/"Exit0"
   
   
   2023/01/09 14:40:02 In Exit0
   
   
   problem3_test.go:27: successful recovery
   
   
   --- PASS: TestExit (0.00s)
   
   
   --- PASS: TestExit/"Exit0" (0.00s)
   
   
   PASS
   
   
   ok code.osinet.fr/fgm/go__deferplus/step01-problem/problem3 0.107s
   
   
   

   % go test -v -run TestExit/Exit1
   
   
   === RUN TestExit
   
   
   === RUN TestExit/"Exit1"
   
   
   2023/01/09 14:40:05 In Exit1
   
   
   exit status 1
   
   
   FAIL code.osinet.fr/fgm/go__deferplus/step01-problem/problem3 0.100s
   

   View Slide

9. © 2023 OSInet 9
   os.Exit() vs os.Exit(1) vs syscall.Exit(n)
   // Exit causes the current program to exit with the given status code.
   
   
   // Conventionally, code zero indicates success, non-zero an error.
   
   
   // The program terminates immediately; deferred functions are not run.
   
   
   //
   
   
   // For portability, the status code should be in the range [0, 125].
   
   
   func Exit(code int) {
   
   
   if code == 0 {
   
   
   if testlog.PanicOnExit0() {
   
   
   // We were told to panic on calls to os.Exit(0).
   
   
   // This is used to fail tests that make an early
   
   
   // unexpected call to os.Exit(0).
   
   
   panic("unexpected call to os.Exit(0) during test")
   
   
   }
   
   
   // Give race detector a chance to fail the program.
   
   
   // Racy programs do not have the right to finish successfully.
   
   
   runtime_beforeExit()
   
   
   }
   
   
   syscall.Exit(code)
   
   
   }
   

   View Slide

10. © 2023 OSInet
    Practical solutions
    10
    

    View Slide

11. © 2023 OSInet 11
    Combining os.Exit() and defer
    package main
    
    
    import (
    
    
    "log"
    
    
    "os"
    
    
    )
    
    
    var someCondition = true
    
    
    func main() {
    
    
    var exitCode = 0
    
    
    defer func() {
    
    
    log.Println("Deferred code")
    
    
    os.Exit(exitCode)
    
    
    }()
    
    
    log.Println("Dans main")
    
    
    if someCondition {
    
    
    // Instead of os.Exit(1)
    
    
    exitCode = 1
    
    
    return
    
    
    }
    
    
    }
    

    View Slide

12. © 2023 OSInet 12
    A better way to exit from func main()
    package main
    
    
    import "os"
    
    
    func main() {
    
    
    os.Exit(int(realMain()))
    
    
    }
    
    
    // realMain is the actual main. Exit codes must be in [0, 125].
    
    
    func realMain() byte {
    
    
    // Do whatever you want. Use defer freely.
    
    
    var someExitCode byte
    
    
    // Return will set the exit code, just like in C and many others.
    
    
    return someExitCode
    
    
    }
    

    View Slide

13. © 2023 OSInet
    ● When running tests, log.Fatal* allows finishing the test
    early with a message
    
    
    ● The downside is that this uses os.Exit(), hence skipping
    defer execution
    
    
    ● A better equivalent is t.FailNow which uses the testing
    package’s own test termination mechanism, based on
    runtime.Goexit(), which allows defer to work after a
    failure, unlike log.Fatal*.
    13
    Keep defers: avoiding exit in tests
    

    View Slide

14. © 2023 OSInet 14
    Setup and cleanup in tests
    Using defer:
    
    
    ● setup functions
    
    
    ○ return their actual result
    
    
    ○ return a defer-rable func
    
    
    ● tests
    
    
    ○ call setup functions
    
    
    ○ need to defer each such defer-rable in order
    
    
    ○ defer-red run :
    
    
    ■ in the test scope
    
    
    ■ before the end of the test
    Using testing.Cleanup():
    
    
    ● setup functions
    
    
    ○ return their actual result
    
    
    ○ call t.Cleanup with their teardown callback
    
    
    ● tests
    
    
    ○ call setup functions
    
    
    ○ no cleanup work needed
    
    
    ○ cleanups run :
    
    
    ■ in the setup scope
    
    
    ■ after the end of the test
    

    View Slide

15. © 2023 OSInet 15
    Testing an os.Exit-ing(0) function
    func TestExit(t *testing.T) {
    
    
    for _, test := range [...]struct {
    
    
    name string
    
    
    fn func()
    
    
    }{{"Exit0", Exit0},{"Exit1", Exit1}} {
    
    
    t.Run(fmt.Sprintf("%#v", test.name),
    

    func(t *testing.T) {
    
    
    defer func() {
    
    
    rec := recover()
    
    
    switch x := rec.(type) {
    
    
    case nil:
    
    
    t.Fatal("nothing recovered")
    
    
    case string:
    
    
    if !strings.Contains(x,
    

    "unexpected call to os.Exit(0)") {
    
    
    t.Fatalf("unexpected: %q", x)
    
    
    }
    
    
    t.Log("successful recovery")
    
    
    default:
    
    
    t.Fatalf("unexpected: %#v", x)
    
    
    }
    
    
    }()
    
    
    test.fn()
    
    
    })
    
    
    }
    
    
    }
    As seen previously, this only works if the
    exit code is zero: otherwise in os.Exit, the
    value of PanicOnExit0 will not be
    considered.
    
    
    It might even be false with alternative test
    runners using
    internal.testlog.SetPanicOnExit
    0
    

    View Slide

16. © 2023 OSInet 16
    Testing an os.Exit-ing(nonZero) func
    Method:
    
    
    ● Test creates a new command
    using the same test binary running
    itself
    
    
    ● Runs itself recursively, passing a
    flag to identify the recursive call
    
    
    ● Outer code receives the exit code
    of the recursive call
    Examples:
    
    
    ● https://blog.antoine-augusti.fr/
    2015/12/testing-an-os-exit-
    scenario-in-golang/
    
    
    ● https://github.com/kohirens/
    tmpltoapp/blob/
    46e536e50eaf55ea60d66bf815c
    9a4d6f51d27b9/
    main_test.go#L31
    

    View Slide

17. © OSInet
    Thanks for
    watching
    Questions ?
    fgm@osinet.fr
    

    View Slide