You, latency and profiling @ GolangUK 2017

You, latency and profiling @ GolangUK 2017

9fdab9d005b82612cadbfe699b541f83?s=128

Filippo Valsorda

August 17, 2017
Tweet

Transcript

  1. 6.

    A fast database… … can store many GB per second

    … or replies to queries in a few ms
  2. 8.

    A fast API… … can handle many clients … or

    answers in a few milliseconds
  3. 13.
  4. 14.

    Running on CPU I/O I/O I/O I/O h>p.Handler 1 h>p.Handler

    2 h>p.Handler 3 h>p.Handler 4 CPU profiling
  5. 16.

    func Write(data []byte) { for i := 0; i <

    50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } } func Hash(data []byte) { for i := 0; i < 50; i++ { sha256.Sum256(data) } }
  6. 18.
  7. 19.

    $ time curl http://127.0.0.1:12345/hash-and-write
 9.831 total
 
 
 $ time

    curl http://127.0.0.1:12345/write-no-hash
 7.692 total
  8. 22.
  9. 23.
  10. 24.

    The tracer Detailed nanosecond-level log of execuOon events: • gorouOne

    scheduling, blocking • syscalls, network, I/O • garbage collecOon
  11. 30.
  12. 31.

    go tool trace -pprof=TYPE trace.out Supported profile types are: -

    net: network blocking profile - sync: synchronization blocking profile - syscall: syscall blocking profile - sched: scheduler latency profile
  13. 34.
  14. 35.
  15. 36.

    -pprof=syscall func Write(data []byte) { for i := 0; i

    < 50; i++ { tmpfile, _ := ioutil.TempFile("", "ex") defer os.Remove(tmpfile.Name()) _, err = tmpfile.Write(data) tmpfile.Close() } }
  16. 37.
  17. 40.
  18. 44.
  19. 45.

    WriIng our own profile prof := make(map[uint64]pprof.Record) for _, ev

    := range events { if ev.Type != trace.EvGoBlockNet { continue } rec := prof[ev.StkID] rec.Stk = ev.Stk rec.N++ rec.Time += ev.Link.Ts - ev.Ts prof[ev.StkID] = rec } pprof.BuildProfile(prof).Write(os.Stdout)
  20. 46.

    var childG = make(map[uint64]struct{}) var lastGLen int for { for

    _, ev := range events { if ev.Type != trace.EvGoCreate { continue } if _, ok := childG[ev.G]; !ok && !filterStack(ev.Stk, re) { continue } childG[ev.Args[0]] = struct{}{} } if len(childG) == lastGLen { break } lastGLen = len(childG) }
  21. 48.
  22. 49.
  23. 50.

    Build more tools! • Focus on gorouOne number • Aggregate

    all blocking types • VisualizaOons • …
  24. 52.