OACIS introduction

Transcript

1. An open-source job management framework for parameter-space exploration: OACIS Yohsuke Murase RIKEN Advanced Institute of Computational Science Collaborators: Takeshi Uchitane, Nobuyasu Ito
2. challenges in social simulations I. Noda et al. (2015) to understand the global landscape of the phase space, which requires us many trial and errors to explore broad parameter space.
3. Typical research workflow tend to be a bottleneck • comparison of many models, parameters, jobs… • prone to human-errors • iteration can be longer as the amount of computation grows consider a model() write a source code() write a script for analysis() while ( necessary ) { select a suitable parameter set() execute simulation run() take a note to remember what we are doing() wait for completion of the simulation run() transfer the simulation results to suitable folders() keep a note to remember what is done() analyze results() create a graph() } write a paper() present in a meeting() essential in research activities
4. 
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE 
      
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O  
   GJOJTIFE     
   O 
5. SSH login is required many times to submit jobs vast amount of directories we must repeat what we have done when we find a bug in the simulation code After a few months, the simulation results are no longer traceable
6. OACIS Organizing Assistant for Comprehensive and Interactive Simulations Available as an Open-Source software under the MIT license https://github.com/crest-cassia/oacis developed by Discrete-Event Simulation Research Team, AICS
7. Worker Host C SSH Connection Application Server HTTP Web browser XSUB Host B XSUB Host A XSUB File Storage APIs Job Scheudler Job Scheudler       or System Overview 8PSLFST
   DSFBUFT
   TIFMM
   TDSJQUT
   BOE
   TVCNJU
   KPCT
   WJB
   44) IBOEMFT
   SFRVFTUT
   GSPN
   VTFST 6TFST
   DBO
   DSFBUF
   KPCT
   WJB
   8FCCSPXTFS 3FTVMUT
   BSF
   TUPSFE
   JO
   pMF
   TUPSBHF
   BOE
   %# 3FNPUF
   IPTUT
   FYFDVUFT
   TJNVMBUJPO
   KPCT 8PSLFST
   EPXOMPBET
   SFTVMUT :PV
   DBO
   BDDFTT
   TJNVMBUJPO
   SFTVMUT Ruby on Rails + MongoDB, Unix-based OS
8. Data-structure overview AnalyzerA Sample Simulator { p1: 0, p2: 0.0 } { p1: 1, p2: 0.0 } { p1: 2, p2: 0.0 } { p1: 0, p2: 1.0 } seed: xxxx seed: aaaa seed: bbbb seed: yyyy seed: zzzz seed: cccc seed: dddd seed: eeee Simulator ParameterSet Run Analysis Analyzer AnalyzerB Analysis 1 Analysis 2 Analysis 3 Analysis 4 Analysis
9. DEMO
12. Command-line based execution A command is embedded into a shell script. We can implement simulators in any language. bash script
13. Sharing data • To share the data, we provide “read-only” mode. • To share data, you may prepare another read-only instance. Each simulation result and plot has an URL. Simulation Server OACIS Synchronize job submission results See Results can’t submit a job OACIS on Cloud (Read-Only mode)
14. APIs Python and Ruby APIs are available to automate the workflow. loop on parameter “p1” loop on parameter “p2” creation of PS and Runs
15. Identifying the input factors that affect simulation outputs Agent simulation of evacuations from Tsunami Sensitivity Analysis
16. Auto phase-boundary search Order parameter for ferromagnetic Ising model Ultimately, we would like to obtain the phase diagram automatically. An efficient algorithm for the sampling in parameter-space is an uncultivated issue.
17. Use cases • modeling social networks • Y. Murase et al. (2014,2015), Torok et al.(2016), Jo et al. (2016) • agent-based simulation of stock markets • Kusada et al. (2014), Torii et al. (2015) • agent-based simulation of traffic and pedestrians • Matsushima(2016), Tsuji(2015), Uchitane et al.(2016), Yoshioka et al. • studies on open evolving systems • Shimada et al. (2014,2015), Murase et al.(2015) • molecular dynamics simulation of granular material • Kuwabara et al. • first-principle calculation of condensed matter physics • Pham et al.(2017) • simulation of rescue robots • Takayanagi et al. (2016), Takami et al. (2017) スプレッドが市場 草田 裕紀†, 水田 孝 背景と目的 研究の目的 マーケットメイカーが市場間出来高シェアに与える影響 そのメカニズムの解明 人工市場モデル モデル概要 近年高頻度取引や代替市場の増加により、金融市場は す複雑化している。安定した市場設計にはこれらが市場 る影響の分析が必要。 高頻度取引を行うマーケットメイカーが２市場間の出来高 アに与える影響を人工市場シミュレーションを用いて分析 †東京大学大学院工学系研究科,  ‡スパークス・アセット・マネジ 本研究では水田ら(2013)「人工市場シミュレーションを用いた取引市場間 けるティックサイズと取引量の関係性分析」（ＪＰＸワーキングペーパーno 照）をベースにモデルを構築した。以下にモデルの概要とマーケットメイカ 詳細を図にまとめた。
18. Development History initial commit major version up released as an OSS 2013/4 2014/4 2015/4 2016/4 v1.15.2 v1.14.0 v1.13.0 v1.12.0 v1.11.0 v1.10.1 v1.9.0 v1.8.0 v1.7.0 v2.0.0 v2.1.0 v2.2.0 v2.3.0 v2.4.0 v2.5.0 v2.6.0 v2.7.0 v2.8.0
19. Conclusion • A software framework for parameter-space exploration is presented. https://github.com/crest-cassia/oacis