Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Troubleshooting features of golang

Troubleshooting features of golang

An instruction to some debug features of golang

Satoru Takeuchi
PRO

October 12, 2017
Tweet

More Decks by Satoru Takeuchi

Other Decks in Programming

Transcript

  1. 1 The troubleshooting features for golang v1.1 Oct 10 2015

    Satoru Takeuchi <satoru.takeuchi@gmail.com>
  2. 2 Introduction • Purpose – Introduce the troubleshooting features for

    golang • Motivation – There are few documents about the troubleshooting features (e.g. debugger) for golang – The available documents are scattered on the many web pages – I consider it's nice to gather these information in the one place • Software version • Golang: 1.5.1 • OS: fedora22/x86_64
  3. 3 Table of Contents • gdb • Race Detector •

    godebug • Heapdump • pprof • Conclusion
  4. 4 gdb: Capture core dump • Settings – Environment variable

    • GOTRACEBACK=crash # Capture core dump when user program aborts • Tips: Prevent variables from disappearing by optimization – Append `-gcflags “-N -l“` option to “go build” • How to capture – Terminate program abnormally, for example by segmentation fault – Send SIGABRT
  5. 5 gdb: Analyze • Load the following script to use

    extended gdb commands for golang • (gdb) source $GOROOT/src/runtime/runtime-gdb.py • Note: On golang 1.4.2, this script failed to load and doesn't work • Analyze program with the extended command written in the following URL • https://golang.org/doc/gdb – Note: Currently it can only analyze running process • Analyzing core dump doesn't work – Process is stopped at unsure point – “bt” doesn't work properly – “goroutine” fails with error
  6. 6 gdb: Future prospects • It won't be better in

    the near future • Please refer to the official document – https://golang.org/doc/gdb … As a consequence, although GDB can be useful in some situations, it is not a reliable debugger for Go programs, particularly heavily concurrent ones. Moreover, it is not a priority for the Go project to address these issues, which are difficult. In short, the instructions below should be taken only as a guide to how to use GDB when it works, not as a guarantee of success. In time, a more Go-centric debugging architecture may be required. …
  7. 7 Race Detector: Summary • A tool for detecting problems

    coming from race condition – https://golang.org/doc/articles/race_detector.html • Show warning messages if your program has a race condition • Usage – Build your program with “-race” option and run it • $ go run –race foo.go • $ go build –race && ./foo
  8. 8 Race Detector: Example $ go run -race race.go ==================

    WARNING: DATA RACE Write by goroutine 4: main.func·001() /home/sat/gopath/src/test/race/race.go:7 +0x43 Previous write by main goroutine: main.main() /home/sat/gopath/src/test/race/race.go:10 +0x176 Goroutine 4 (running) created at: main.main() /home/sat/gopath/src/test/race/race.go:9 +0x166 ================== Found 1 data race(s) exit status 66 1 package main 2 3 func main() { 4 c := make(chan bool) 5 i := 0 6 go func() { 7 i = 1 8 c <- true 9 }() 10 i = 2 11 <- c 12} At this time, whether “i” is 1 or 2 is not sure. In other word, there is a race between line 7 and line 10. Detected the race
  9. 9 godebug: Summary • A debugger for golang – https://github.com/mailgun/godebug

    • Usage is written in “README.md” at the above mentioned URL – Although there is only “go get” in “Installation” section, the following commands should be run to use “godebug” program 1. go get github.com/mailgun/godebug 2. cd $GOPATH/src/github.com/mailgun/godebug 3. go install # Install godebug under $GOPATH/bin • Currently there few (less than gdb) features • It would be better since this project is brand new and under development
  10. 10 godebug: Compare features with gdb godebug gdb Setting breakpoint

    Should be embedded in source code x Analyze core dump Commands for goroutine x Step/next x x Continue x x Show source code x x Show variables x x
  11. 11 Heapdump • WriteHeapDump() function in “runtime/debug” package – https://golang.org/pkg/runtime/debug

    • Dump the contents of heap • Although it will be a nice feature in the future, it's difficult to use for now – Output is binary format and there is no tool to read it as human readable text format – Output format is not documented yet (Only version 1.3 and 1.4 has such document) • https://github.com/golang/go/wiki/heapdump13 • https://github.com/golang/go/wiki/heapdump14
  12. 12 pprof: Summary • A profiling tool for golang program

    • Capture and analyze CPU/memory profiling information • The package contains the profiling APIs – https://golang.org/pkg/runtime/pprof/ • Description – http://blog.golang.org/profiling-go-programs • Usage 1. Call profiling APIs from your program and capture profiling information • There is no build option such as gcc's “-pg”. It's a bit inconvenient ;-( 2. Visualize and analyze the profiling information • $ go tool pprof <program name> <the file contains the profiling information>
  13. 13 pprof: Sample code package main import ( "os" "log"

    "runtime/pprof" ) func bar() { for i := 0; i < 1000000; i++ { } } func foo() { for i := 0; i < 10000000; i++ { if i % 1000 == 0 { bar() } } } func main() { f, err := os.Create("cpu.pprof") if err != nil { log.Fatal(err) } pprof.StartCPUProfile(f) defer pprof.StopCPUProfile() foo() } Start to capture CPU profiling information Stop to capture CPU profiling information at the end of main() prof.go: Open the file to contain CPU profiling information
  14. 14 pprof: Run & profiling $ ls prof.go $ go

    build $ ./pprof $ ls cpu.pprof prof prof.go # cpu.pprof contains the CPU profiling information $ go tool pprof prof cpu.pprof Entering interactive mode (type "help" for commands) (pprof) top 9.79s of 9.80s total (99.90%) Dropped 2 nodes (cum <= 0.05s) flat flat% sum% cum cum% 9.74s 99.39% 99.39% 9.74s 99.39% main.bar 0.05s 0.51% 99.90% 9.79s 99.90% main.foo 0 0% 99.90% 9.79s 99.90% main.main 0 0% 99.90% 9.79s 99.90% runtime.goexit 0 0% 99.90% 9.79s 99.90% runtime.main Spend most of the time (99.39%) in main.bar() Spend a little time(0.51%) In main.foo() Spend almost 0% In other functions
  15. 15 Conclusion • There are many nice troubleshooting features for

    golang • However, there is no way to analyze core dump – I consider it's one of the biggest weak point of golang – How to find root cause if your program crashes in a important system? Stack trace is insufficient in many case