CA-Go#14 Queue, ErrGroup, Semaphore Story

Step-by-step make the efficiency of using concurrency in the Golang application.

Transcript

1. Queue, ErrGroup, Semaphore で並行処理を実現した話 Nguyen Duc Hoang - S23208
2. •Joined in shsh •Hanoi DevCenter •Backend Eng •X: @ndhoang96 Nguyen Duc Hoang
3. 1.Background & Prerequisites 2.First design 3.v2: Worker-pool pattern with static number of workers 4.v3: Dynamic worker number with semaphore 5.v4: Simplify with ErrGroup 6.v5: Graceful shutdown
4. Background & Prerequisites
5. Background Background • FIFO Queue has many events, each events has many messages • Go consumer receives events from Queue • Events frequently is unpredictable Prerequisites • Calling OpenAI model takes time: 2- 5 minutes • There are maybe many messages →Handle the messages efficiently
6. First design
7. First design Components 1. Fetcher: periodically get events from Queue 2. Channel: store the events 3. Handler: handle the events
8. First design - Fetcher Fetcher • use ticker to fetch new event every second.
9. First design - Handler Handler • get event from channel and process
10. First design In last: ➔ create a channel ➔ run goroutine to handle fetcher and handler Conclusion: ➔ Go consumer can process the message ➔ Can handle concurrently by multiple go consumer (multiple instances - scale by infra)
11. First design Problem: ➔ handler handle event sequentially, not concurrently
12. v2: Worker-pool pattern with static number of workers
13. Static worker-pool pattern Static worker-pool pattern: ➔ handle events from channel by multiple workers ➔ define the fixed number of workers and run workers concurrently
14. Static worker-pool pattern Conclusion: ➔ Go consumer can process the message concurrently internal ➔ Can manage resources as desired
15. Static worker-pool pattern Problem: ➔ Sometimes there isn’t any messages ➔ sometimes there are so many messages. ➔ The static goroutine number can’t scale as the number of coming messages.
16. v3: Dynamic worker number with semaphore
17. Dynamic worker-pool pattern Dynamic worker-pool pattern: ➔ inside handler, spawn many goroutines to handle message ➔ the number of goroutine is controlled by Semaphore
18. Dynamic worker-pool pattern Dynamic worker-pool pattern: ➔ NewWeighted(10): set the upper bound of goroutine number with Semaphore ➔ the number of goroutine follow by the number of messages in channel.
19. v4: Simplify with ErrGroup
20. Errgroup Errgroup: ➔ same package sync with Semaphore ➔ inside Errgroup, there is a combination of waitGroup and Semaphore ➔ can simplify v3 implementation ➔ source
21. Errgroup Errgroup: ➔ simplified version of v3
22. v5: Graceful shutdown
23. Graceful shutdown - Future Work ➔ Before app exits: ◆ Complete ongoing tasks ◆ Release resources properly ◆ Clean up any necessary state ◆ … ➔ This case’s problems: ◆ Notify the exit signal. ◆ Handle the being processed messages.
24. Graceful shutdown - SIGTERM,SIGINT Exit Signal: ➔ make a channel to catch exit signal like SIGTERM, SIGINT ➔ when exit signal comes, properly notify to the goroutine (context)
25. Graceful shutdown - Queue Message ➔ Problems ◆ The queue invisibility is set to 5 minutes. ◆ If the app received message, process it 10 seconds, and restarted; the message will be kept for 4 minutes 50 seconds.
26. Graceful shutdown - Queue Message ➔ Solution ◆ when received exit signal, set the invisibility to 0, make the message available again
27. Thank you very much