Technical Challenges of Real-World Agent-Based Modelling

Thomas French, Sr. Software Engineer at Sandtable. Talk at Data Science London @ds_ldn 29/05/13

Transcript

1. Thomas French 29th May 2013 www.sandtable.com Technical Challenges of Real- World Agent-Based Modelling Thursday, 30 May 13
2. Outline • What is ABM? • Why use ABM? • Classic ABM example • Real World ABM • Three Key Technical Challenges Thursday, 30 May 13
3. “Essentially, all models are wrong, but some are useful” G.E. Box (1987) Thursday, 30 May 13
4. ABM in a nutshell AGENT ENVIRONMENT SENSORS MESSAGES ACTIONS PERCEPTS OBJECT ACTUATORS Based&on&Bordini&et&al&&(2007) Thursday, 30 May 13
5. Why are we talking about ABM? • It shows promise for understanding complex systems: – heterogeneous and adaptive actors – complex interactions: interdependencies; feedback loops – dynamic environment • It provides an accessible metaphor for modelling – modelling individuals • More and more data is available for our models - Finer levels of granularity • Computing power is available on-demand - Costs continue to reduce Thursday, 30 May 13
6. Classic ABM: Schelling Segregation Model • Developed by Thomas Schelling in 1970s. • Study racial segregation of populations emerging from individual discriminatory behaviours. Thursday, 30 May 13
7. Source: Eric Fisher Thursday, 30 May 13
8. Schelling Segregation Model Thursday, 30 May 13
9. Schelling Segregation Model Thursday, 30 May 13
10. Schelling Behaviour Tree Thursday, 30 May 13
11. Real World ABM Thursday, 30 May 13
12. QuitSIM Behaviour Tree - QUIT SIM 2 QS Tree in Colour Censor Thu May 30 2013 Thursday, 30 May 13
13. QuitSIM Behaviour Tree Take up smoking? Never smoker = 1 Become smoker Age, gender Do nothing Cut down attempt length, route, age, dependency Consume media / ingest experience Smoker Smoker = 1 Never Smoker Never Smoker = 1 Consume media / ingest experience Get support? Set support flag Planned or Unplanned? Do something about smoking? motivation, events, price, GP, social, pregnant, media, random 8#2013 Thursday, 30 May 13
14. Technical Challenges BUILD VALIDATE EXPERIMENT Designing*and building*models Building Confidence* in*Models Conducting Large?Scale Experiments HARD VERY,*VERY*HARD VERY*HARD Thursday, 30 May 13
15. Building Models BUILD VALIDATE EXPERIMENT Behavioural+ Data Survey Data Assumptions Intuition Analyse Build Individual+Agent+ Attributes Behaviour+Tree Environment (e.g.+Media) Representative+ Population Data+Sources Simulation Components Thursday, 30 May 13
16. Validation - Building Confidence VALIDATE EXPERIMENT Does the implemented model reflect the real-world system? Thursday, 30 May 13
17. Validation – Establishing Criteria A framework for evaluating state of validity of models for on-going monitoring. VALIDATE EXPERIMENT VALIDATION INTERNAL VALIDATION EXTERNAL VALIDATION Model& implemented& correctly Behaviours& predicted&make& sense&/&are&logical Model&stands&up& to&comparison& with&external&data Thursday, 30 May 13
18. Validation - Examples Represented in a formal logic • linear-time temporal logic with extensions Internal: (s_Att.gender = f) => (G (s_Att.gender = f) ) G (!((s_Att.smoker = 1) && (s_Att.takeUp = 1))) G (!((s_Att.smoker = 1) && (s_Att.age < 11))) External: n_MSE (s_Val1.prevalence, r_Val1.prevalence) n_MSE (s_Val2.quit_atts, r_Val2.quit_atts) VALIDATE EXPERIMENT Thursday, 30 May 13
19. Validation – Solving Multi-Criteria Problems VALIDATE EXPERIMENT Thursday, 30 May 13
20. Validation - Workflow VALIDATE EXPERIMENT Select&Model Select&Tests Select& Reference&Data Configure&Test& Suite Execute& Replications Summarise& Individual&Tests Summarise&Test& Suite Thursday, 30 May 13
21. Experimentation - Approaches • Empirical Calibration • Sensitivity Analysis • Scenario Exploration • Goal-Directed Search EXPERIMENT Thursday, 30 May 13
22. Experimentation – Exploring Parameter Spaces EXPERIMENT Small Large Explore Exhaustive+Search Simple+Random+Sampling,+ Latin+Hypercube+Sampling e.g.+7+vars,+10/100+values+=+ 1+Trillion+parameter+sets Seek Exhaustive+Search Noisy,+MultiEObjective+ Evolutionary+Algorithms Parameter+Space Search+Type Thursday, 30 May 13
23. Experimentation - Handling Noise EXPERIMENT Thursday, 30 May 13
24. Experimentation – Handling Output Data EXPERIMENT Thursday, 30 May 13
25. Experimentation – Platform Architecture EXPERIMENT CATALOG REST API WORKFLOW SCENARIOS ANALYSIS VALIDATION OPTIMISATION SERVICES mongoDB MANAGER WORKER 1 PLATFORM RabbitMQ MESSAGING http:// sandtable.com Sandtable Simulation Platform CLIENT simulation analysis validation 1 2 k 2 3 N S3 Sandtable)Simulation)Platform Thursday, 30 May 13
26. Experimentation - Managing Workflow EXPERIMENT Thursday, 30 May 13
27. Thursday, 30 May 13
28. “Nothing is built on stone; all is built on sand. But we must build as if sand were stone.” J.L. Borges Thursday, 30 May 13
29. Thanks for listening! thomas@sandtable.com www.sandtable.com Thursday, 30 May 13
30. Further study Book: • John Miller and Scott Page: 'Complex Adaptive Systems: An Introduction to Computational Models of Social Life' (2007) Coursera: • Scott Page: 'Model Thinking' • https://www.coursera.org/course/modelthinking Thursday, 30 May 13