Performance Analysis 101

Palestra sobre analise de performance de aplicações python em ambiente linux.

Transcript

1. PERFORMANCE ANALYSIS 101 Python Brasil 2016 @tarsisazevedo
2. Tarsis Azevedo
3. First things first Thanks @romulomachado_ for this amazing slide design.
4. Performance
5. O que aprendemos na faculdade?
6. Big O notation list comprehension busca binária heapsort bogosort quicksort merge sort array to string com join
7. E fora da faculdade?
8. Client-Side Alto Workload Algoritmos Aplicação resiliente Scale up/down Efficient *language
9. Alto Workload Aplicação resiliente Scale up/down
10. Performance Good Cheap Fast
11. Performance Performance On Time Inexpensive Good Cheap Fast
15. Ferramentas
16. Stress wrk (https://github.com/wg/wrk)
17. Stress vaurien (http://vaurien.readthedocs.io/)
18. Stress locust (http://locust.io)
19. Locust locust -f load.py -H http://192.168.0.101:8000 \ -c 100 --no-web -r 10 -n 1000
20. Profiling timeit; cProfile; line_profiler;
 
 memory_profiler; objgraph.
21. Demo https://github.com/tarsisazevedo/talk-performance-analysis
22. Perf + FlameGraph FlameGraph (github.com/brendangregg/FlameGraph) Perf (www.brendangregg.com/perf.html)
23. FlameGraph
24. pyflame https://github.com/uber/pyflame
25. Problemas Comuns
26. Problemas Comuns Arquitetura Workload
27. Sem cache;
 
 Hardware mal dimensionado;
 
 Aplicação single-thread; Latência de rede não prevista. Arquitetura
28. Mais usuários que o esperado;
 
 Cache mal planejado. Workload
29. Dica cache hit-ratio = cache hit ÷ total request (quanto maior, melhor)
30. E por último…
31. Não faça otimizações precocemente.
32. NÃO FAÇA OTIMIZAÇÕES PRECOCEMENTE.
33. MESMO!
34. func CheckUserAccess(teamNames []string, u *User) bool { q := bson.M{"_id": bson.M{"$in": teamNames}} var teams []Team conn, err := db.Conn() if err != nil { log.Errorf("Failed to connect to the database: %s", err) return false } defer conn.Close() conn.Teams().Find(q).All(&teams) var wg sync.WaitGroup found := make(chan bool, len(teams)+1) for _, team := range teams { wg.Add(1) go func(t Team) { if t.ContainsUser(u) { found <- true } wg.Done() }(team) } go func() { wg.Wait() found <- false }() return <-found } https://github.com/tsuru/tsuru/blob/c54c2c073aa9bf1db888a0bed45ffafbf62cbf18/auth/team.go
35. var wg sync.WaitGroup found := make(chan bool, len(teams)+1) for _, team := range teams { wg.Add(1) go func(t Team) { if t.ContainsUser(u) { found <- true } wg.Done() }(team) } go func() { wg.Wait() found <- false }() return <-found
36. func CheckUserAccess(teamNames []string, u *User) bool { q := bson.M{"_id": bson.M{"$in": teamNames}} var teams []Team conn, err := db.Conn() if err != nil { log.Errorf("Failed to connect to the database: %s", err) return false } defer conn.Close() conn.Teams().Find(q).All(&teams) for _, team := range teams { if team.ContainsUser(u) { return true } } return false } https://github.com/tsuru/tsuru/blob/fafa683c5cc1e736df03116b11fc5868c54e30d5/auth/team.go#L138
37. conn.Teams().Find(q).All(&teams) for _, team := range teams { if team.ContainsUser(u) { return true } } return false
38. Mas às vezes você precisa…
39. //go:generate bash -c "ragel -Z -G2 -o parser.go parser.rl && gofmt -s -w parser.go && gofmt -s -w parser.go" %%{ machine lineparser; write data; }%% func parseLogLine(data []byte) ([][]byte, bool, bool) { parts := make([][]byte, 8) start := 0 pIdx := -1 cs, p, pe := 0, 0, len(data) eof := pe withMsg := false withPID := false %%{ action di { pIdx++; start = p } action dd { parts[pIdx] = data[start:p] } action msgok { withMsg = true } action withpid { withPID = true } main := '<' ( digit )+ >di %dd '>' space* ( any - space )+ >di %dd space+ ( ( digit+ space digit+ ':' digit+ ':' digit+ ) >di %dd space+ )? ( any - space )+ >di %dd space+ ( any - space - '[' - ']' )+ >di %dd ('[' ( digit )+ >di %dd >withpid ']')? ':' space+ ( any )+ >di %dd >msgok; write init; write exec; }%% return parts, withMsg, withPID } https://github.com/tsuru/bs/blob/master/log/parser.rl
40. Em Produção
41. Métricas Importam
42. Sentry https://getsentry.com/welcome/
43. New Relic https://newrelic.com
44. Gray Log https://www.graylog.org
45. Logstash https://www.elastic.co/products/logstash
46. On trouble
47. On trouble
48. On trouble Streetlight Anti-Method
49. On trouble Blame-Someone-Else Anti-Method
50. On trouble Problem Statement
51. On trouble
52. On trouble
53. On trouble Utilization
 Saturation Error
54. Esteja pronto
58. Resumindo
59. Resumindo Não faça otimizações precocemente;
 
 Estresse sua aplicação localmente;
 
 Defina o problema primeiro; Busque métricas melhores sempre.
60. • System Performance book - Brendon Gregg • Talk: Linux Performance Tools • Julia Evans’ blog: tcpdump, tcpdump 2 • Unix/Linux Handbook • Diving into the Wreck: a postmortem look at real-works performance • Ragel • Profiling Go program • perf • FlameGraph • CPU Flame Graph Referências
61. OBRIGADO