http.Client in go

By Raf

Transcript

1. http.Client in go 1.15.6
2. http.Client가 쓰이는 곳 - RESTful API - elastic search - AWS endpoint http.Client 안씀 - RDBMS connection (postgresql, mysql) - gRPC - redis
3. http.Client, http.Transport http.Client: 껍데기 http.Transport: 실제 요청 처리
4. http.Transport 실제 http요청이 처리되는 구현체 connection pool을 들고 있어서 goroutine-safe 함
5. client.Do(req) client := &http.Client{} // init 시점에 한번만 생성 req := http.Request { … } res, err := client.Do(req) if err != nil { … } defer res.Body.Close() …
6. Transport shared variables
7. getConn()
8. tryPutIdleConn()
9. default values
10. http.ClientTrace trace := &httptrace.ClientTrace{ GetConn: func(hostPort string) { fmt.Println(“%s”, hostPort) }, } req.WithContext(httptrace.WithClientTrace(req.Context(), trace)) res, err := client.Do(req)
11. httptrace.ClientTrace Request 1. http.Client.Do(req) 2. RoundTrip() 3. GetConn() 4. idleConn 가져오기 실패 5. idleConnWait에 req 등록 6. go dialConnFor()
12. 1. DNS Lookup 2. DNSStart() 3. DNSDone() 4. Connect 5. ConnectStart() 6. ConnectDone() 7. TLSHandshake 8. TLSHandshakeStart() 9. TLSHandshakeDone() 10. go readLoop() 11. go writeLoop() httptrace.ClientTrace Request dialConnFor()
13. httptrace.ClientTrace Request dialConnFor() pc.readLoop() pc.writeLoop() 1. <- pc.reqch 12. tryDeliver(pc) 7. GotConn() 8. pc.roundTrip() 9. pc.writech <- WriteRequest 10. pc.reqch <- requestAndChan 11. <- WriteErrch, resc 1. <- pc.writech
14. httptrace.ClientTrace Request pc.readLoop() pc.writeLoop() 2. br.peek(1) 3. GotFirstResponseByte() 4. ReadResponse(br) 5. Got100Continue() 6. continueCh <- {} 2. WroteHeaderField() 3. WroteHeaders() 4. Wait100Continue() 5. <- continueCh 6. bw.write(req) 7. WroteRequest() <- pc.writech
15. httptrace.ClientTrace Request pc.readLoop() 7. ReadResponse(br) 8. requestAndChan.resc <- body 9. <- waitForBodyRead 10. tryPutIdleConn() 11. PutIdleConn() <- pc.reqch 12. ended roundTrip() 13. ended http.Client.Do() 14. call res.body.Close()
16. request가 완료되었을때 trace는?? - 커넥션을 받고 난 뒤 실제 요청을 던지는 시점: GotConn() - request가 완료되었을때 trace는 RoundTripper interface를 활용해야함 - http.Client에 transport가 interface로 들어가므로, wrapper를 씌워서 transport.RoundTrip이 끝나는 시점에 trace를 남김
17. Datadog에 timespan을 보여주기 적절한 trace 방식 1. Transport Wrapper에서 RoundTrip 시작 전에 Trace 2. Transport.RoundTrip() 호출 3. Transport에서 GetConn() 호출 3. Transport에서 GotConn() 호출 4. Transport Wrapper에서 RoundTrip이 완료되었을때 Trace connection을 받는데 걸리는 시간 요청을 보내고, 응답을 받는데 걸리는 시간
18. 요약 1. http.Transport는 한개만 쓸것 2. http.Client.Do()가 끝나고 err가 없으면 res.Body.Close()를 호출할것 3. httptrace.GetConn/GotConn을 활용하여 connect 의 timespan 분리 4. MaxIdleConnsPerHost를 잘 조정하자 (=MaxIdleConns / #hosts)
19. 끝 컨플루언스 문서 • [Golang] http connection management • [Golang] http.persistConn, httptrace.ClientTrace