Slide 1

Slide 1 text

PERFORMANCE ANALYSIS 101 Python Brasil 2016 @tarsisazevedo

Slide 2

Slide 2 text

Tarsis Azevedo

Slide 3

Slide 3 text

First things first Thanks @romulomachado_ for this amazing slide design.

Slide 4

Slide 4 text

Performance

Slide 5

Slide 5 text

O que aprendemos na faculdade?

Slide 6

Slide 6 text

Big O notation list comprehension busca binária heapsort bogosort quicksort merge sort array to string com join

Slide 7

Slide 7 text

E fora da faculdade?

Slide 8

Slide 8 text

Client-Side Alto Workload Algoritmos Aplicação resiliente Scale up/down Efficient *language

Slide 9

Slide 9 text

Alto Workload Aplicação resiliente Scale up/down

Slide 10

Slide 10 text

Performance Good Cheap Fast

Slide 11

Slide 11 text

Performance Performance On Time Inexpensive Good Cheap Fast

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

Ferramentas

Slide 16

Slide 16 text

Stress wrk (https://github.com/wg/wrk)

Slide 17

Slide 17 text

Stress vaurien (http://vaurien.readthedocs.io/)

Slide 18

Slide 18 text

Stress locust (http://locust.io)

Slide 19

Slide 19 text

Locust locust -f load.py -H http://192.168.0.101:8000 \ -c 100 --no-web -r 10 -n 1000

Slide 20

Slide 20 text

Profiling timeit; cProfile; line_profiler;
 
 memory_profiler; objgraph.

Slide 21

Slide 21 text

Demo https://github.com/tarsisazevedo/talk-performance-analysis

Slide 22

Slide 22 text

Perf + FlameGraph FlameGraph (github.com/brendangregg/FlameGraph) Perf (www.brendangregg.com/perf.html)

Slide 23

Slide 23 text

FlameGraph

Slide 24

Slide 24 text

pyflame https://github.com/uber/pyflame

Slide 25

Slide 25 text

Problemas Comuns

Slide 26

Slide 26 text

Problemas Comuns Arquitetura Workload

Slide 27

Slide 27 text

Sem cache;
 
 Hardware mal dimensionado;
 
 Aplicação single-thread; Latência de rede não prevista. Arquitetura

Slide 28

Slide 28 text

Mais usuários que o esperado;
 
 Cache mal planejado. Workload

Slide 29

Slide 29 text

Dica cache hit-ratio = cache hit ÷ total request (quanto maior, melhor)

Slide 30

Slide 30 text

E por último…

Slide 31

Slide 31 text

Não faça otimizações precocemente.

Slide 32

Slide 32 text

NÃO FAÇA OTIMIZAÇÕES PRECOCEMENTE.

Slide 33

Slide 33 text

MESMO!

Slide 34

Slide 34 text

func CheckUserAccess(teamNames []string, u *User) bool { q := bson.M{"_id": bson.M{"$in": teamNames}} var teams []Team conn, err := db.Conn() if err != nil { log.Errorf("Failed to connect to the database: %s", err) return false } defer conn.Close() conn.Teams().Find(q).All(&teams) var wg sync.WaitGroup found := make(chan bool, len(teams)+1) for _, team := range teams { wg.Add(1) go func(t Team) { if t.ContainsUser(u) { found <- true } wg.Done() }(team) } go func() { wg.Wait() found <- false }() return <-found } https://github.com/tsuru/tsuru/blob/c54c2c073aa9bf1db888a0bed45ffafbf62cbf18/auth/team.go

Slide 35

Slide 35 text

var wg sync.WaitGroup found := make(chan bool, len(teams)+1) for _, team := range teams { wg.Add(1) go func(t Team) { if t.ContainsUser(u) { found <- true } wg.Done() }(team) } go func() { wg.Wait() found <- false }() return <-found

Slide 36

Slide 36 text

func CheckUserAccess(teamNames []string, u *User) bool { q := bson.M{"_id": bson.M{"$in": teamNames}} var teams []Team conn, err := db.Conn() if err != nil { log.Errorf("Failed to connect to the database: %s", err) return false } defer conn.Close() conn.Teams().Find(q).All(&teams) for _, team := range teams { if team.ContainsUser(u) { return true } } return false } https://github.com/tsuru/tsuru/blob/fafa683c5cc1e736df03116b11fc5868c54e30d5/auth/team.go#L138

Slide 37

Slide 37 text

conn.Teams().Find(q).All(&teams) for _, team := range teams { if team.ContainsUser(u) { return true } } return false

Slide 38

Slide 38 text

Mas às vezes você precisa…

Slide 39

Slide 39 text

//go:generate bash -c "ragel -Z -G2 -o parser.go parser.rl && gofmt -s -w parser.go && gofmt -s -w parser.go" %%{ machine lineparser; write data; }%% func parseLogLine(data []byte) ([][]byte, bool, bool) { parts := make([][]byte, 8) start := 0 pIdx := -1 cs, p, pe := 0, 0, len(data) eof := pe withMsg := false withPID := false %%{ action di { pIdx++; start = p } action dd { parts[pIdx] = data[start:p] } action msgok { withMsg = true } action withpid { withPID = true } main := '<' ( digit )+ >di %dd '>' space* ( any - space )+ >di %dd space+ ( ( digit+ space digit+ ':' digit+ ':' digit+ ) >di %dd space+ )? ( any - space )+ >di %dd space+ ( any - space - '[' - ']' )+ >di %dd ('[' ( digit )+ >di %dd >withpid ']')? ':' space+ ( any )+ >di %dd >msgok; write init; write exec; }%% return parts, withMsg, withPID } https://github.com/tsuru/bs/blob/master/log/parser.rl

Slide 40

Slide 40 text

Em Produção

Slide 41

Slide 41 text

Métricas Importam

Slide 42

Slide 42 text

Sentry https://getsentry.com/welcome/

Slide 43

Slide 43 text

New Relic https://newrelic.com

Slide 44

Slide 44 text

Gray Log https://www.graylog.org

Slide 45

Slide 45 text

Logstash https://www.elastic.co/products/logstash

Slide 46

Slide 46 text

On trouble

Slide 47

Slide 47 text

On trouble

Slide 48

Slide 48 text

On trouble Streetlight Anti-Method

Slide 49

Slide 49 text

On trouble Blame-Someone-Else Anti-Method

Slide 50

Slide 50 text

On trouble Problem Statement

Slide 51

Slide 51 text

On trouble

Slide 52

Slide 52 text

On trouble

Slide 53

Slide 53 text

On trouble Utilization
 Saturation Error

Slide 54

Slide 54 text

Esteja pronto

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

No content

Slide 58

Slide 58 text

Resumindo

Slide 59

Slide 59 text

Resumindo Não faça otimizações precocemente;
 
 Estresse sua aplicação localmente;
 
 Defina o problema primeiro; Busque métricas melhores sempre.

Slide 60

Slide 60 text

• System Performance book - Brendon Gregg • Talk: Linux Performance Tools • Julia Evans’ blog: tcpdump, tcpdump 2 • Unix/Linux Handbook • Diving into the Wreck: a postmortem look at real-works performance • Ragel • Profiling Go program • perf • FlameGraph • CPU Flame Graph Referências

Slide 61

Slide 61 text

OBRIGADO