Introduction to singleflight

This slide is the presentation material I presented at mercari.go #11.
https://mercari.connpass.com/event/148913/

Transcript

1. mercari.go #11 @nsega Introduction to singleflight
2. About Me • Naoki Sega (Twitter/GitHub: @nsega ) • Backend Team in Mercari JP / Category Growth, Catalogs • Joined Mercari JP in November 2018. • Go : Experienced about 1 year in coding as the Production Code. • Community : Kubernetes Meetup, Cloud Native Meetup, etc • My current interests: Go, GCP, Kubernetes, Microservices,etc.
3. Agenda
4. Agenda • What is singleflight? • Introduction to use case 1 • Introduction to use case 2 • Conclusion
5. What is singleflight?
6. What is singleflight? • “Package singleflight provides a duplicate function call suppression mechanism.” • If your app receive multiple requests for the same resource, sync/singleflight is helpful. ◦ RDB( master data ) ◦ Image file ◦ Lookup IP address ◦ Client Certs • “x/sync/singleflight” is imported by 81 packages (As of Oct 7 2019). ◦ github.com/hashicorp/consul/agent/consul ◦ gopkg.in/ebay/fabio.v1/cert ▪ https://github.com/fabiolb/fabio/blob/master/cert/source.go https://godoc.org/golang.org/x/sync/singleflight?importers
7. What is singleflight? • “Package singleflight provides a duplicate function call suppression mechanism.” • • • • • https://godoc.org/golang.org/x/sync/singleflight
8. • Originally, singleflight had been using as standard library. (e.g. “net/lookup.go”) The spec is a little bit different between internal/singleflight and x/sync/singleflight. • GoDoc - x/sync/singleflight, GoDoc - internal/singleflight Go Sync ( This repository provides Go concurrency primitives in addition to the ones provided by the language and "sync" and "sync/atomic" packages. ) FYI: a trend of singleflight https://github.com/golang/sync
9. FYI: a trend of singleflight • According to https://dev.golang.org/release, internal/singleflight is going to delete and recommend x/sync/singleflight to use from 1.14. https://go-review.googlesource.com/c/go/+/174080/ https://github.com/golang/go/issues/31697
10. Introduction to Use Case 1
11. • This Use Case in net/lookup ◦ LookupIPAddr calls together for lookups for the same host. ◦ LookupIPAddr looks up host using the local resolver. Introduction to Use Case 1 https://github.com/golang/go/blob/release-branch.go1.13/src/net/lookup.go#L204
12. net/lookup.go https://github.com/golang/go/blob/release-branch.go1.13/src/net/lookup.go#L152 https://github.com/golang/go/blob/release-branch.go1.13/src/net/lookup.go#L237
13. Introduction to Use Case 2
14. Introduction to Use Case 2 • This Use Case of the technical spec in Microservices ◦ Client ▪ Request the API of GetCatalogComponent to Microservice A ▪ To get the recommended Catalog( ID:1 ) ▪ To get Catalog inserted within 1 week. ◦ Microservice A - BFF layer ▪ GetCatalogComponent provides the recommended Catalog(ID:1) and the Catalog inserted within 1 week as a aggregator. • Request to API of GetCatalog to Microservice B • Request to API of ListCatalogs to Microservice B ◦ Microservice B - Database layer ▪ GetCatalog provides Catalog specified by using ID ▪ ListCatalogs provides Catalog specified by using 1 weeks.
15. Microservice A - BFF Layer GetCatalogComponent Microservice B - Catalog Service HTTP/gRPC Use Case 2 - Overview (before) GetCatalog - id:1 ListCatalog GetCatalog - id:1 ListCatalog - between 7 days catalog client
16. Microservice A - BFF Layer GetCatalogComponent Microservice B - Catalog Service HTTP/gRPC Use Case 2 - Overview (after) GetCatalog - id:1 ListCatalog GetCatalog - id:1 ListCatalog - between 7 days catalog singleflight client
17. Conclusion
18. • Package singleflight provides a duplicate function call suppression mechanism. • If you need to tune the API that requests the same resource, singleflight will be helpful. • It seems to me that such as BFF in Microservices is easy to apply the use case by using singleflight. • If you have the interests, please check the other use case of other products imported singleflight, https://godoc.org/golang.org/x/sync/singleflight?importers Conclusion
19. Thank you. Any Questions??