Application of Online Resource Allocation Based on Ray (Fang Fengbin, Ant Group)

We build online resource allocation system based on Ray, which is widely used in search recommendation advertising system of ant group. I will introduce the application scenario of online resource allocation in ant group and our work based on Ray.

Transcript

1. Application of online resource allocation based on ray Fengbin Fang Software Engineer, Ant Group 2021
2. Problem Description: How to maximize the total sales volume of the platform when the stock of some coupons is limited? Sequence CVR Uplift user1 0.1 user2 0.2 user3 0.3 user4 0.5 Stock = 2 The increase of purchase rate brought by coupons 2021
3. Recommender system Personalization Group control Personalization Group control ROI optimization Personalization Operation strategy Group control Online Resource Allocation: • Make immediate decisions on resource allocation requests. • Maximizing the overall ROI under the constraint of limited resources. 2021
4. Abstract Model Of Online Resource Allocation: Optimization Objective Constraint Condition ! 𝑚#$ % 𝑥#$ ≤ 𝑏% 𝑀𝑎𝑥 ! 𝑐#$ 𝑥#$ 𝑐#$ : The profit value obtained by distributing the j-th resource to user 𝑖. 𝑖 : The user i. 𝑗 : The j-th resource. 𝑥#$ : The probability of assigning the j-th resource to the i-th user. 𝑏% : There are K constraints on resources. 𝑚#$ % : The consumption on the k-th resource constraint of assigning the j-th resource to the i-th user. Difficulties: • Large scale problem solving. • Future information is unpredictable and the global optimal solution cannot be determined in advance. 2021
5. Solution: Two stage modeling scheme Model Prediction • Click through rate • Write off rate • Conversion rate • … Step 1 Step 2 ü Integer programming modeling ü Online Solver ü Online Re-rank Service Operational Research Ø Theoretical guarantee of effect Implementation Effect Ø Adaptive control ability • Flow change • Model update • Crowd change • Real time intervention Ø Low cost access 2021
6. Operational Research Solution: Assume that the data of adjacent time periods are of the same distribution, so we can decompose the global problem into real-time subproblems for learning. Use the data of the previous stage to guide the online decision of the current stage. Dual Variable Resource Manager Real Time Planning Online Decision Service Model Prediction Log Store DB Constraint Scoring Snapshot 2021
7. Real Time Planning: Capture Traffic Changes In Real Time Model Score Real Target Score Constraint Deviation 0 Sample Collection Model Training Correction Training Flow Estimation Constraint&Target Disassembly Allocation Training Correction Model Constraint Progress Statistics Constraint Correction Constraint & Target Model Sample Collection LP Solver Score Correction Constraint& Target Assembly Online LP Optimization 2021
8. Kubernetes Scenarios Algorithms Ray Deployment Marketing Advertisement Recommendation Search … Streaming Serving GPU Acceleration Distributed Programming API (Java/Python) Iterative Calculation Common Libs for Distributed Programming Calibration Traffic Prediction Constraints Reorganization … Optimization With Constraints Logical architecture of the online resource allocation system based on Ray Collective Communication Services Online Re-rank Config Manager Data Dashboard … 2021
9. Physical architecture of the online resource allocation system based on Ray Constraint Correction Real Time Data Source Streaming Offline Data Source Correction Training Iterative Calculation Worker Master Allocation Training Iterative Calculation Worker Master Offline Data Source Cache Sample Collection Real Time Data Source Streaming Serving Virtual IP GPU Solver Storage LP Solver Serving Virtual IP Online Decision Service Ray Actor Batch Sample Lagrange Multiplier Correction Model Constraint & Target Model Real time constraints Online LP Optimization 2021
10. Why online resource allocation system choose Ray: • Ray provides a simple and easy-to-use API • Ray supports convenient resource scheduling • Ray supports • … Future planning: • Using Ray collective communication to realize multi card data parallelization • Using Ray ML to accelerate LP solution • … 2021
11. Thanks ! 2021