Location-aware Marketing Carousel

How Go, DynamoDB and geohashing allow us to do location checks millions of times per second (with 0.8 cpu core and 600mb memory).

Transcript

1. Location-aware Marketing Carousel Ben Adrian Sarmiento
2. v Outline 1. Introduction 2. Problem 3. Solution 4. Q&A
3. Ben Adrian Sarmiento Delivery Hero engineering manager / full stack developer self-learned programmer Carmudi (acquired), Cloud Sherpas (acquired), Bright (acquired)
5. PRESENT DAY! Presence in 11 Countries! May 2012 foodpanda was founded! Acquired by Delivery Hero One of Europe’s biggest start-ups! Dec 2016 Dec 2016 Introduced 24/7 delivery islandwide in Singapore! Nov 2017 foodpanda rebranded from orange to Pink Our History Singapore Bangladesh Hong Kong Taiwan Thailand Myanmar Philippines Pakistan Malaysia Cambodia Laos Jan 2020 We launched pandamart!
6. PRESENT DAY! Presence in 11 Countries! May 2012 foodpanda was founded! Acquired by Delivery Hero One of Europe’s biggest start-ups! Dec 2016 Dec 2016 Introduced 24/7 delivery islandwide in Singapore! Nov 2017 foodpanda rebranded from orange to Pink Our History Singapore Bangladesh Hong Kong Taiwan Thailand Myanmar Philippines Pakistan Malaysia Cambodia Laos Jan 2020 We launched pandamart! Launched in PH 6 years ago...
7. The Problem
8. Campaign Carousel
9. foodpanda clients (Web, Android, iOS) Campaigns API open the app request the list of campaigns send the list of campaigns back • Kubernetes • Go • DynamoDB
10. What is a Campaign?
11. Campaign Setup Marketing team creates a campaign and assigns restaurants to it using our internal tool
12. Restaurant Delivery Areas sample data
13. How to Show Campaigns Based on User Location?
14. UX Problem We have to make sure there is at least one restaurant delivering to the user before we show the campaign tile
15. Check One by One if User Location is Inside Restaurant Delivery Areas campaign1 restaurant1 restaurant2 restaurant3 delivery_area1a x delivery_area1b x delivery_area1c x delivery_area2a x delivery_area2b x delivery_area2c x delivery_area3a x delivery_area3b x delivery_area3c ✓ Is user location inside delivery area? found
16. • ~10 active campaigns per country • ~10,000 restaurants per campaign • ~10 delivery areas per restaurant We need to do ~1,000,000 location checks per location
17. Parallelize all location checks WaitGroup and channels and goroutines var wg sync.WaitGroup visible := make(chan string, 1000000) for i := 0; i <= len(campaigns); i++ { ... go checkIfVendorsDeliver( campaigns[i].Vendors, customerLocation, campaigns[i].ID, visible, &wg, ) } wg.Wait() close(status) ...
18. Parallelize all location checks WaitGroup and channels and goroutines func checkIfVendorsDeliver(...) { for i := 0; i <= len(vendors); i++ { ... // skip some vendors? go checkIfInsideDeliveryAreas( vendor[i].DeliveryAreas, customerLocation, campaignID, visible, &wg, ) } }
19. Parallelize all location checks WaitGroup and channels and goroutines func checkIfInsideDeliveryAreas(...) { for i := 0; i <=len(deliveryAreas); i++ { ... // merge delivery areas? wg.Add(1) defer wg.Done() visible <- checkIfInsideDeliveryArea( deliveryArea, customerLocation, campaignID, visible, ) }
20. Parallelize all location checks WaitGroup and channels and goroutines import "github.com/paulsmith/gogeos/geos" func checkIfInsideDeliveryArea(...) { polygon := geos.NewPolygon(deliveryArea.coordinates) if polygon.Contains(customerLocation) { return campaignID } }
21. Max number of goroutines Memory consumption Response time Limits: 800 millicore cpu 600M memory 10 pods
22. The Solution
23. Geohash
24. Geohash Dimensions Length width x height 1 5,009.4km x 4,992.6km 2 1,252.3km x 624.1km 3 156.5km x 156km 4 39.1km x 19.5km 5 4.9km x 4.9km 6 1.2km x 609.4m 7 152.9m x 152.4m 8 38.2m x 19m http://geohash.gofreerange.com/
25. Example Use Case campaign1 restaurant1 restaurant2 restaurant3 delivery_area1a delivery_area1b delivery_area1c delivery_area2a delivery_area2b delivery_area2c delivery_area3a delivery_area3b delivery_area3c campaign1 campaign2 campaign3 campaign4
26. Generate Geohashes (Precision 5) Inside Delivery Area { Type: "Polygon", Coordinates: [][][]float64{{ { 103.87813568115234, 1.331457449380893, }, { 103.88328552246094, 1.306744762128643, }, { 103.90997886657715, 1.3105203274201689, }, { 103.90379905700684, 1.335833628726838, }, { 103.8812255859375, 1.3370349314986247, }, { 103.87813568115234, 1.331457449380893, }, }}, } Geohashes: w21z7, w21ze, w21zk, w21zs Compromise in accuracy
27. Map Campaign ID with Geohashes campaign1:w21z7 campaign1:w21ze campaign1:w21zk campaign1:w21zs campaign1:... campaign1:... {campaignID}:{restaurantGeohash}
28. Location Check 1. Customer opens the foodpanda client 2. Client sends a request (with customer’s geo coordinates) to get the list of campaigns from Campaign API. 3. In Campaign API: a. Convert the customer’s geo coordinates to geohash with precision=5 to match the encoded geohashes of the delivery areas b. Get the list of campaigns c. For each campaign, query the database to check if the campaignID:customerGeohash exists. If yes, we can show the campaign to the user. d. Send back the list of visible campaigns to the frontend. 4. Customer sees the carousel Only 1 query to database per campaign campaignID:customerGeohash == campaignID:restaurantGeohash customerGeohash and restaurantGeohash should have the same precision
29. Convert User Lat Long to Geohash (Precision = 5) Latitude: 1.3203024473401612 Longitude: 103.89461517333984 Geohash: w21zs
30. Get all campaigns in Singapore campaign1 campaign2 campaign3 campaign4
31. Check if campaignID:customerGeohash exists campaign1:w21z7 campaign1:w21ze campaign1:w21zk campaign1:w21zs campaign2:abcde campaign3:bcdef campaign4:cdefg campaign1:w21zs ✓ campaign1 campaign2 campaign3 campaign4 + :w21zs = campaign2:w21zs X campaign3:w21zs X campaign4:w21zs X Result: Show campaign1 to the user Key begins_with Query DynamoDB Sort Key index Parallelize Cache the list Do not unmarshal results.Items result.Count >= 1 https://github.com/dgraph-io/ristretto
32. DynamoDB SDK in Go https://aws.amazon.com/sdk-for-go/ Pitfall: Be careful if you are fetching List of Maps, unmarshaling consumers significant amount of memory Campaign.Vendors with type []Vendor Vendor.DeliveryAreas with type []DeliveryArea DeliveryArea.Coordinates with type []Coordinate Solution: exclude the field by projection if you don’t need it If you need, consider optimising the encoding by using a comma-separated string type instead of a list type Final pointers
33. Single Table Design https://www.youtube.com/watch?v=HaEPXoXVf2k Provisioning is simpler, only one table and you’re good for the rest of the application. Planning your access patterns is essential Final pointers
34. 470k peak requests per minute 1.5 ms avg response time
35. Thank You! https://www.foodpanda.com/careers/