Slide 1

Slide 1 text

OACIS hands-on tutorial Yohsuke Murase, RIKEN R-CCS 2020 Jan. 23, NetSciX post-conference school

Slide 2

Slide 2 text

Agenda 2 l Introduction on OACIS l Hands-on session 1 l how to submit your jobs using OACIS l Hands-on session 2 l how to integrate your program (such as jupyter notebook) into OACIS l Hands-on session 3 l how to automate your workflow We are not telling you how to install OACIS today. The official document is available at http://crest-cassia.github.io/oacis/

Slide 3

Slide 3 text

3 Virtual Machine used in this hands-on l We launch instances on AWS, where OACIS is pre- installed. l We are going to distribute IP addresses to each of you. Access the following URLs with a browser. l http://[your IP]:3000 l http://[your IP]:8888 https://console.aws.amazon.com/ec2/v2/home?region=ap-northeast- 1#LaunchInstanceWizard:ami=ami-00aab2fcf2a9b6d87 The image is available at this URL.

Slide 4

Slide 4 text

About me 4 l Yohsuke Murase l http://yohm.github.io/ l RIKEN Center for Computational Science l Research interest l complex systems l model development for social networks l (evolutionary) games

Slide 5

Slide 5 text

Introduction What is OACIS?

Slide 6

Slide 6 text

Typical research workflow 6 consider a model() write a source code() write a script for analysis() while ( necessary ) { select a suitable parameter set() execute simulation run() take a note to remember what we are doing() wait for completion of the simulation run() transfer the simulation results to suitable folders() keep a note to remember what is done() analyze results() create a graph() } write a paper() present in a meeting() tend to be a bottleneck l comparison of many models, parameters, jobs… l prone to human-errors l iteration can be longer as the amount of computation grows essential in research activities

Slide 7

Slide 7 text

7

Slide 8

Slide 8 text

8 vast amount of directories SSH login is required many ti mes to submit jobs we must repeat what we ha ve done when we find a bug in our code After a few months, the sim ulation results are no longer traceable.

Slide 9

Slide 9 text

9 Organizing Assistant for Comprehensive and Interactive Simulations Available as an open-source software under the MIT license https://github.com/crest-cassia/oacis developed by Discrete-Event Simulation Research Team, R-CCS

Slide 10

Slide 10 text

System Overview 10 Ruby on Rails, MongoDB, Unix-based OS

Slide 11

Slide 11 text

Data-structure overview 11 AnalyzerA Sample Simulator { p1: 0, p2: 0.0 } { p1: 1, p2: 0.0 } { p1: 2, p2: 0.0 } { p1: 0, p2: 1.0 } seed: xxxx seed: aaaa seed: bbbb seed: yyyy seed: zzzz seed: cccc seed: dddd seed: eeee Simulator ParameterSet Run Analysis Analyzer AnalyzerB Analysis 1 Analysis 2 Analysis 3 Analysis 4 Analysis

Slide 12

Slide 12 text

12 l DEMO DEMO

Slide 13

Slide 13 text

Command-line based execution 13 A command is embedded into a shell script. We can implement simulators in any language.

Slide 14

Slide 14 text

Sharing Data 14 Simulation Server OACIS Synchronize job submission results See Results can’t submit a job OACIS on Cloud (Read-Only mode) Each simulation result and plot has an URL. To share the data, we provide “read-only” mode. To share data, you may prepare another read-only instance.

Slide 15

Slide 15 text

APIs 15 Python and Ruby APIs are available to automate your workflows.

Slide 16

Slide 16 text

Use cases of APIs 16 Optimization Auto detection of phase boundary sensitivity analysis, data assimilation…

Slide 17

Slide 17 text

Software stack 17 Worker Host C SSH Connection Application Server HTTP Web browser XSUB Host B XSUB Host A XSUB File Storage CLI Job Scheudler Job Scheudler server side • Ruby • Ruby on Rails • MongoDB front-end • D3.js • jQuery • twitter-bootstrap ~ 12,000 lines for app ~ 12,000 lines for unit tests

Slide 18

Slide 18 text

Development history 18 2013/4 2014/4 2015/4 2016/4 v1.15.2 v1.14.0 v1.13.0 v1.12.0 v1.11.0 v1.10.1 v1.9.0 v1.8.0 v1.7.0 v2.0.0 v2.1.0 v2.2.0 v2.3.0 v2.4.0 v2.5.0 v2.6.0 v2.7.0 v2.8.0 2017/4 v2.9.0 v2.10.0 v2.11.0 v2.12.0 v2.13.2 2018/4 v3.0.0 v3.1.0 v3.2.1 2019/4 v3.3.0 v3.4.0 3390 commits, 660 issues & PRs

Slide 19

Slide 19 text

Use cases 19 が市場出来 草田 裕紀†, 水田 孝信† 背景と目的 研究の目的 ケ イカ が市場間出来高 ェアに与える影響と その カ の解明 人工市場 概要 近年高頻度取引や代替市場の増加により、金融市場はますま す複雑化している。安定した市場設計にはこれらが市場に与え る影響の分析が必要。 高頻度取引を行う ケ イカ が 市場間の出来高 ェ アに与える影響を人工市場 を用いて分析。 †東京大学大学院工学系研究科,  ‡ ク ア 株 本研究では水田ら(2013)「人工市場 を用いた取引市場間にお ける ィ ク イ と取引量の関係性分析」 キ グ no2参 照 を に を構築した。以下に の概要と ケ イカ の 詳細を図にまとめた。 modeling social networks • Y. Murase et al. (2014,2015), Torok et al.(2016), Jo et al. (2016) agent-based simulation of stock markets • Kusada et al. (2014), Torii et al. (2015) agent-based simulation of traffic and pedestrians • Matsushima(2016), Tsuji(2015), Uchitane et al.(2016), Yoshioka et al. studies on open evolving systems • Shimada et al. (2014,2015), Murase et al.(2015,2017) molecular dynamics simulation of granular material • Kuwabara et al.(2016) first-principle calculation of condensed matter physics • Pham et al.(2017) simulation of rescue robots • Takayanagi et al. (2016), Takami et al. (2017)

Slide 20

Slide 20 text

Our article 21 l We would greatly appreciate if you cite the following article when you publish your research using OACIS. l Y. Murase, T. Uchitane, and N. Ito, "An open-source job management framework for parameter-space exploration: OACIS", Journal of Physics: Conference Series, 921, 012001 (2017)

Slide 21

Slide 21 text

OACIS hands-on session 1 How to submit jobs on OACIS l Reference l http://crest-cassia.github.io/oacis/en/tutorial.html

Slide 22

Slide 22 text

Nagel-Schreckenberg model 26 l Nagel-Schreckenberg is a cellular-automaton model for traffic congestion, proposed in 1990s. l See [Wikipedia](https://en.wikipedia.org/wiki/Nagel%E2%80%93Schreckenberg_model) Key Description l Road length v Maximum velocity rho Car density p deceleration probability t_init thermalization steps t_measure measurement steps http://www.civil.iitb.ac.in/tvm/1111_nptel/544_TrCA/plain/ plain.html

Slide 23

Slide 23 text

27 l It reproduces the phase transition between the free- flow phase and the congestion phase. l Source code of this simulator l https://github.com/yohm/si m_ns_model l Output files of this simulator l a JSON file containing average velocity and flow l a snapshot PNG file. position time

Slide 24

Slide 24 text

Selecting a Simulator 28 l Select a Simulator l Check Simulator settings Click Click Definition of input parameters & configurations of the simulator are registered.

Slide 25

Slide 25 text

Creating a PS and Run 29 l Select ʼParameter Setsʼ l List of ParameterSets are shown. l Click “New ParameterSet” button to create a new PS. Click Click

Slide 26

Slide 26 text

30 l Creating a ParameterSet and Runs l Fill in the values of parameters l v = 5 l rho = 0.2 l Click “Create” button

Slide 27

Slide 27 text

31 l A new ParameterSet and a Run are created. l The status of the Run will change in a few seconds. List of Runs under this ParameterSet. Click

Slide 28

Slide 28 text

32 A button to download the archive of these results. Figures (bmp,jpg,png…) are displayed inline. List of output files. Click it to access. Contents of “_output.json” file is saved in OACIS DB. File path to the results.

Slide 29

Slide 29 text

Making multiple jobs 33 l In the form to create ParameterSets l fill in the values of parameters as comma-separated values l v = “1,3,5” l rho = “0.05,0.1,0.15,0.2,0.25,0.3,0.35,0.4,0.45,0.5” fill in values in a CSV form Click

Slide 30

Slide 30 text

34 45 ParameterSets are created in total. List of ParameterSets Progress bars for job executions.

Slide 31

Slide 31 text

Making a plot 35 Click a PS Click “Plot”

Slide 32

Slide 32 text

36 Select “line plot” X-Axis: “rho” Y-Axis: “flow” Series: "v" Click Results are displayed. Each plot has a unique URL

Slide 33

Slide 33 text

Making a heat map 37 Select “scatter plot” X-Axis: “rho” Y-Axis: “v” Result: “.flow” Jump to a PS page by double clicking a point

Slide 34

Slide 34 text

Tile view of figures 38 Select “figure viewer” X-Axis: “rho” Y-Axis: “v” Result: “.traffic.png” A magnified image is displayed by mouse-over.

Slide 35

Slide 35 text

Model 2: Barabasi-Albert model 40 http://networksciencebook.com/chapter/5

Slide 36

Slide 36 text

41 l Simulation code: https://github.com/yohm/sim_BA_model/blob/master/BA.ipynb https://pypi.org/project/powerlaw/ implemented as a jupyter notebook

Slide 37

Slide 37 text

Model 3: BA model with attractiveness 43 l An extended BA model. l Node has an initial attractiveness A. The probability of making a link is proportional to (A+ki ). http://networksciencebook.com/chapter/6#evolving-networks

Slide 38

Slide 38 text

44 l Try m=[3,5], A=[0,1,2,3,4,8,16] l Plot alpha (degree exponent) as a function of A (initial attractiveness) for various m. l See how P(k) changes as A or m changes.

Slide 39

Slide 39 text

Summary of session 1 49 l hands-on: executing simualtions by OACIS l selecting the Simulator l creating ParameterSets and Runs l accessing the results l output files l plots ⇒ In the next hands-on we will instruct how to implement YOUR simulator

Slide 40

Slide 40 text

OACIS hands-on session 2 how to set up your simulator l Reference l http://crest-cassia.github.io/oacis/en/configuring_simulator.html l http://crest-cassia.github.io/oacis/ja/configuring_simulator.html

Slide 41

Slide 41 text

51 How jobs are executed database job scheduler (ex. Torque) OACIS Computational Host qsub SSH login sh execute in the work directory the work directory is compressed into a single file. tar download (scp) & expand parse job status & update the record 1. Work directory and shell script are created. Jobs are submitted to the scheduler. 2. Jobs are executed by the job scheduler. Current directory is set to the work directory. 3. Results are downloaded to OACIS. The log files are parsed and the records are updated.

Slide 42

Slide 42 text

requirements for Simulator 52 l Input parameters must be given as command line arguments or by a JSON file (Details are shown in the next slide.) l Output files must be created under the current directory. l These files are created by OACIS. Simulator must not conflict with these files. l _status.json, _time.txt, _version.txt, _log.txt, _stdout.txt, _stderr.txt l Returns 0 when it finished successfully. Returns a non zero code when it failed.

Slide 43

Slide 43 text

how to set input parameters 53 l By command-line arguments l Input parameters and the random number seed are appended to the command. l ~/simulator.out ... l By a JSON file l When running the command, a file “_input.json” is created, in which input parameters are written. l {"p1": 30,"p2": 10,"_seed":12345}

Slide 44

Slide 44 text

in NS model simulator 54 https://github.com/yohm/sim_ns_model/blob/master/ns.rb

Slide 45

Slide 45 text

Output file format 56 l As long as output files are created in the current directory, you have nothing to do. l If you would like to plot scalar values in OACIS, the values should be written in “_output.json” file. l If you would like to use "figure viewer", write files in bmp, png, jpg and other figure format. (eps files are not supported.) {"flow": 0.235, "velocity": 1.245 }

Slide 46

Slide 46 text

in NS model simulator 57

Slide 47

Slide 47 text

Making a Simulator 58 Simulator name Definition of Parameters name, type, default value preprocess command (optional) command of Simulator JSON or Argument input A note of the Simulator List of executable hosts

Slide 48

Slide 48 text

Hands-on 1 59 l Register “echo” command as a simulator. l name: echo l definition of parameters: l (p1, Integer, 0), (p2, Float, 1.0) l Command: echo l Input type: Argument l Executable_on: localhost l Run a job, and see “_stdout.txt"

Slide 49

Slide 49 text

60 Simulator name Definition of Parameters name, type, default value “echo” Select “Argument” Select “localhost”

Slide 50

Slide 50 text

Tips for debugging 61 l When the set up is inappropriate, runs will fail. In such cases, see “_stderr.txt” for debugging. Go to the page for a failed Run Click “_stderr.txt”

Slide 51

Slide 51 text

Duplicating a Simulator 62 l You may want to try a simulation model which is slightly different from the current one. In such case, you can create a new simulator based on an existing one. Duplicating Simulator

Slide 52

Slide 52 text

“Oacisizing” your jupyter notebook 65 l Suppose that you have a prototype in jupyter notebook. l We use `papermill` to make jupyter batch executable. https://github.com/nteract/papermill $ papermill mynote.ipynb out.ipynb -f parameters.yml execute a notebook from Terminal specifying parameters

Slide 53

Slide 53 text

How to use papermill 66 l `pip install papermill` l To parameterize your notebook designate a cell with the tag parameters. l After you executed `papermill in.ipynb out.ipynb -f parameters.yml`, youʼll find “injected-parameters” in out.ipynb. parameters.yml

Slide 54

Slide 54 text

integrating with OACIS 67 1. Modify your notebook such that it accepts parameters. https://github.com/yohm/sim_BA_model/blob/master/BA.ipynb Use ”_seed” to specify th e random number seed. Set ”parameters” tag. 2. (optional) `plt.savefig` to save figure to a file. Save a png file. 3. (optional) write “_output.json” file. Write a JSON file.

Slide 55

Slide 55 text

68 4. Prepare a shell script to run `papermill`. This part is an idiom to use pipenv. Specify your note as an input. (optional) you might want to convert ipynb to html. https://github.com/yohm/sim_BA_model/blob/master/run_BA.sh

Slide 56

Slide 56 text

69 5. Register “run.sh” as a Simulator on OACIS. Use the same parameter nam es as the ones in your note. Input type should be JSON. Specify the shell script.

Slide 57

Slide 57 text

70 6. Submit a test job on OACIS. If it failed, inspect “_stderr.txt”.

Slide 58

Slide 58 text

“Oacisize” more complicated Simulators 71 l In order to register your simulator on OACIS, create a script which makes your simulator conform to the input format of OACIS. l Register the script as a Simulator of OACIS. wrap script simulation program run.py _input.json a.out 1. Input parameters given by OACIS 2. parse command line args and make an XML file 3. launch the simulation program as a subprocess

Slide 59

Slide 59 text

OACIS hands-on session 3 how to automate your workflow l Reference l http://crest-cassia.github.io/oacis/en/api_python.html l http://crest-cassia.github.io/oacis/en/api_samples.html

Slide 60

Slide 60 text

What can we do with APIs? 82 Basically we can conduct any operations on OACIS. OACIS web-UI is implemented based on these methods. • creating • finding • getting info • deleting • Simulator • ParameterSet • Run • Host • Analyzer • Analysis x

Slide 61

Slide 61 text

Basic usage of APIs 83 Set "OACIS_ROOT" environment variable to the path where OACIS is installed. $ export OACIS_ROOT="$HOME/oacis" $ python3 my_script.py import os,sys sys.path.append( os.environ['OACIS_ROOT'] ) import oacis Load "oacis" module. (In the docker container, PYTHON_PATH is already set. You can skip the first two lines.) •Our sample is available at https://github.com/yohm/sim_ns_model/blob/master/oacis_find_optimum_r ho.ipynb

Slide 62

Slide 62 text

OACIS + Jupyter 84

Slide 63

Slide 63 text

Demo: Searching optimum rho 85 l Let us search optimum density for NS model l Amount of traffic flow shows non-monotonic behavior against rho.

Slide 64

Slide 64 text

Algorithm 86 1. Take the PS having the largest "flow". 2. Create two ParameterSets at the centers between its neighboring PSs. 3. Go back to 1 if we do not have an enough resolution. rho flow

Slide 65

Slide 65 text

Async & Await 87 l We want to iteratively determine parameters based on the results of finished jobs. (2) is executed when “ps1” have finished Use “OacisWatcher” class for asynchronous calls start an event-loop to monitor the completion of jobs “f1” and “f2” are asynchronously executed (4) is executed when all PS in “ps_list” have finished http://crest-cassia.github.io/oacis/en/api_watcher.html

Slide 66

Slide 66 text

Result 88

Slide 67

Slide 67 text

Conclusion 89 l We have shown how to register your Simulators on OACIS. l This might be a bit complicated, but once you have done the registration, the remaining workflow becomes much more productive and reproducible. l What we skipped includes l APIs l installation l Host and HostGroup l Analyzers l backup http://crest-cassia.github.io/oacis/

Slide 68

Slide 68 text

90 l If you have any questions or feedbacks l [email protected] l or @yohm13 l OACIS is still an on-going project. l Try using it in your research. Your feedbacks are really appreciated. l We are looking for users, collaborators and contributors.