Technical Challenges of Real-World Agent-Based Modelling

Slide 1

Slide 1 text

Thomas French 29th May 2013 www.sandtable.com Technical Challenges of Real- World Agent-Based Modelling Thursday, 30 May 13

Slide 2

Slide 2 text

Outline • What is ABM? • Why use ABM? • Classic ABM example • Real World ABM • Three Key Technical Challenges Thursday, 30 May 13

Slide 3

Slide 3 text

“Essentially, all models are wrong, but some are useful” G.E. Box (1987) Thursday, 30 May 13

Slide 4

Slide 4 text

ABM in a nutshell AGENT ENVIRONMENT SENSORS MESSAGES ACTIONS PERCEPTS OBJECT ACTUATORS Based&on&Bordini&et&al&&(2007) Thursday, 30 May 13

Slide 5

Slide 5 text

Why are we talking about ABM? • It shows promise for understanding complex systems: – heterogeneous and adaptive actors – complex interactions: interdependencies; feedback loops – dynamic environment • It provides an accessible metaphor for modelling – modelling individuals • More and more data is available for our models - Finer levels of granularity • Computing power is available on-demand - Costs continue to reduce Thursday, 30 May 13

Slide 6

Slide 6 text

Classic ABM: Schelling Segregation Model • Developed by Thomas Schelling in 1970s. • Study racial segregation of populations emerging from individual discriminatory behaviours. Thursday, 30 May 13

Slide 7

Slide 7 text

Source: Eric Fisher Thursday, 30 May 13

Slide 8

Slide 8 text

Schelling Segregation Model Thursday, 30 May 13

Slide 9

Slide 9 text

Schelling Segregation Model Thursday, 30 May 13

Slide 10

Slide 10 text

Schelling Behaviour Tree Thursday, 30 May 13

Slide 11

Slide 11 text

Real World ABM Thursday, 30 May 13

Slide 12

Slide 12 text

QuitSIM Behaviour Tree - QUIT SIM 2 QS Tree in Colour Censor Thu May 30 2013 Thursday, 30 May 13

Slide 13

Slide 13 text

QuitSIM Behaviour Tree Take up smoking? Never smoker = 1 Become smoker Age, gender Do nothing Cut down attempt length, route, age, dependency Consume media / ingest experience Smoker Smoker = 1 Never Smoker Never Smoker = 1 Consume media / ingest experience Get support? Set support ﬂag Planned or Unplanned? Do something about smoking? motivation, events, price, GP, social, pregnant, media, random 8#2013 Thursday, 30 May 13

Slide 14

Slide 14 text

Technical Challenges BUILD VALIDATE EXPERIMENT Designing*and building*models Building Conﬁdence* in*Models Conducting Large?Scale Experiments HARD VERY,*VERY*HARD VERY*HARD Thursday, 30 May 13

Slide 15

Slide 15 text

Building Models BUILD VALIDATE EXPERIMENT Behavioural+ Data Survey Data Assumptions Intuition Analyse Build Individual+Agent+ Attributes Behaviour+Tree Environment (e.g.+Media) Representative+ Population Data+Sources Simulation Components Thursday, 30 May 13

Slide 16

Slide 16 text

Validation - Building Conﬁdence VALIDATE EXPERIMENT Does the implemented model reﬂect the real-world system? Thursday, 30 May 13

Slide 17

Slide 17 text

Validation – Establishing Criteria A framework for evaluating state of validity of models for on-going monitoring. VALIDATE EXPERIMENT VALIDATION INTERNAL VALIDATION EXTERNAL VALIDATION Model& implemented& correctly Behaviours& predicted&make& sense&/&are&logical Model&stands&up& to&comparison& with&external&data Thursday, 30 May 13

Slide 18

Slide 18 text

Validation - Examples Represented in a formal logic • linear-time temporal logic with extensions Internal: (s_Att.gender = f) => (G (s_Att.gender = f) ) G (!((s_Att.smoker = 1) && (s_Att.takeUp = 1))) G (!((s_Att.smoker = 1) && (s_Att.age < 11))) External: n_MSE (s_Val1.prevalence, r_Val1.prevalence) n_MSE (s_Val2.quit_atts, r_Val2.quit_atts) VALIDATE EXPERIMENT Thursday, 30 May 13

Slide 19

Slide 19 text

Validation – Solving Multi-Criteria Problems VALIDATE EXPERIMENT Thursday, 30 May 13

Slide 20

Slide 20 text

Validation - Workﬂow VALIDATE EXPERIMENT Select&Model Select&Tests Select& Reference&Data Conﬁgure&Test& Suite Execute& Replications Summarise& Individual&Tests Summarise&Test& Suite Thursday, 30 May 13

Slide 21

Slide 21 text

Experimentation - Approaches • Empirical Calibration • Sensitivity Analysis • Scenario Exploration • Goal-Directed Search EXPERIMENT Thursday, 30 May 13

Slide 22

Slide 22 text

Experimentation – Exploring Parameter Spaces EXPERIMENT Small Large Explore Exhaustive+Search Simple+Random+Sampling,+ Latin+Hypercube+Sampling e.g.+7+vars,+10/100+values+=+ 1+Trillion+parameter+sets Seek Exhaustive+Search Noisy,+MultiEObjective+ Evolutionary+Algorithms Parameter+Space Search+Type Thursday, 30 May 13

Slide 23

Slide 23 text

Experimentation - Handling Noise EXPERIMENT Thursday, 30 May 13

Slide 24

Slide 24 text

Experimentation – Handling Output Data EXPERIMENT Thursday, 30 May 13

Slide 25

Slide 25 text

Experimentation – Platform Architecture EXPERIMENT CATALOG REST API WORKFLOW SCENARIOS ANALYSIS VALIDATION OPTIMISATION SERVICES mongoDB MANAGER WORKER 1 PLATFORM RabbitMQ MESSAGING http:// sandtable.com Sandtable Simulation Platform CLIENT simulation analysis validation 1 2 k 2 3 N S3 Sandtable)Simulation)Platform Thursday, 30 May 13

Slide 26

Slide 26 text

Experimentation - Managing Workﬂow EXPERIMENT Thursday, 30 May 13

Slide 27

Slide 27 text

Thursday, 30 May 13

Slide 28

Slide 28 text

“Nothing is built on stone; all is built on sand. But we must build as if sand were stone.” J.L. Borges Thursday, 30 May 13

Slide 29

Slide 29 text

Thanks for listening! [email protected] www.sandtable.com Thursday, 30 May 13

Slide 30

Slide 30 text

Further study Book: • John Miller and Scott Page: 'Complex Adaptive Systems: An Introduction to Computational Models of Social Life' (2007) Coursera: • Scott Page: 'Model Thinking' • https://www.coursera.org/course/modelthinking Thursday, 30 May 13