OACIS hands-on tutorial

Slide for the post-conference school of NetSciX2020.

Transcript

1. OACIS hands-on tutorial Yohsuke Murase, RIKEN R-CCS 2020 Jan. 23, NetSciX post-conference school
2. Agenda 2 l Introduction on OACIS l Hands-on session 1 l how to submit your jobs using OACIS l Hands-on session 2 l how to integrate your program (such as jupyter notebook) into OACIS l Hands-on session 3 l how to automate your workflow We are not telling you how to install OACIS today. The official document is available at http://crest-cassia.github.io/oacis/
3. 3 Virtual Machine used in this hands-on l We launch instances on AWS, where OACIS is pre- installed. l We are going to distribute IP addresses to each of you. Access the following URLs with a browser. l http://[your IP]:3000 l http://[your IP]:8888 https://console.aws.amazon.com/ec2/v2/home?region=ap-northeast- 1#LaunchInstanceWizard:ami=ami-00aab2fcf2a9b6d87 The image is available at this URL.
4. About me 4 l Yohsuke Murase l http://yohm.github.io/ l RIKEN Center for Computational Science l Research interest l complex systems l model development for social networks l (evolutionary) games
5. Introduction What is OACIS?
6. Typical research workflow 6 consider a model() write a source code() write a script for analysis() while ( necessary ) { select a suitable parameter set() execute simulation run() take a note to remember what we are doing() wait for completion of the simulation run() transfer the simulation results to suitable folders() keep a note to remember what is done() analyze results() create a graph() } write a paper() present in a meeting() tend to be a bottleneck l comparison of many models, parameters, jobs… l prone to human-errors l iteration can be longer as the amount of computation grows essential in research activities
8. 8 vast amount of directories SSH login is required many ti mes to submit jobs we must repeat what we ha ve done when we find a bug in our code After a few months, the sim ulation results are no longer traceable.
9. 9 Organizing Assistant for Comprehensive and Interactive Simulations Available as an open-source software under the MIT license https://github.com/crest-cassia/oacis developed by Discrete-Event Simulation Research Team, R-CCS
10. System Overview 10 Ruby on Rails, MongoDB, Unix-based OS
11. Data-structure overview 11 AnalyzerA Sample Simulator { p1: 0, p2: 0.0 } { p1: 1, p2: 0.0 } { p1: 2, p2: 0.0 } { p1: 0, p2: 1.0 } seed: xxxx seed: aaaa seed: bbbb seed: yyyy seed: zzzz seed: cccc seed: dddd seed: eeee Simulator ParameterSet Run Analysis Analyzer AnalyzerB Analysis 1 Analysis 2 Analysis 3 Analysis 4 Analysis
12. 12 l DEMO DEMO
13. Command-line based execution 13 A command is embedded into a shell script. We can implement simulators in any language.
14. Sharing Data 14 Simulation Server OACIS Synchronize job submission results See Results can’t submit a job OACIS on Cloud (Read-Only mode) Each simulation result and plot has an URL. To share the data, we provide “read-only” mode. To share data, you may prepare another read-only instance.
15. APIs 15 Python and Ruby APIs are available to automate your workflows.
16. Use cases of APIs 16 Optimization Auto detection of phase boundary sensitivity analysis, data assimilation…
17. Software stack 17 Worker Host C SSH Connection Application Server HTTP Web browser XSUB Host B XSUB Host A XSUB File Storage CLI Job Scheudler Job Scheudler server side • Ruby • Ruby on Rails • MongoDB front-end • D3.js • jQuery • twitter-bootstrap ~ 12,000 lines for app ~ 12,000 lines for unit tests
18. Development history 18 2013/4 2014/4 2015/4 2016/4 v1.15.2 v1.14.0 v1.13.0 v1.12.0 v1.11.0 v1.10.1 v1.9.0 v1.8.0 v1.7.0 v2.0.0 v2.1.0 v2.2.0 v2.3.0 v2.4.0 v2.5.0 v2.6.0 v2.7.0 v2.8.0 2017/4 v2.9.0 v2.10.0 v2.11.0 v2.12.0 v2.13.2 2018/4 v3.0.0 v3.1.0 v3.2.1 2019/4 v3.3.0 v3.4.0 3390 commits, 660 issues & PRs
19. Use cases 19 が市場出来 草田 裕紀†, 水田 孝信† 背景と目的 研究の目的 ケ イカ が市場間出来高 ェアに与える影響と その カ の解明 人工市場 概要 近年高頻度取引や代替市場の増加により、金融市場はますま す複雑化している。安定した市場設計にはこれらが市場に与え る影響の分析が必要。 高頻度取引を行う ケ イカ が 市場間の出来高 ェ アに与える影響を人工市場 を用いて分析。 †東京大学大学院工学系研究科,  ‡ ク ア 株 本研究では水田ら(2013)「人工市場 を用いた取引市場間にお ける ィ ク イ と取引量の関係性分析」 キ グ no2参 照 を に を構築した。以下に の概要と ケ イカ の 詳細を図にまとめた。 modeling social networks • Y. Murase et al. (2014,2015), Torok et al.(2016), Jo et al. (2016) agent-based simulation of stock markets • Kusada et al. (2014), Torii et al. (2015) agent-based simulation of traffic and pedestrians • Matsushima(2016), Tsuji(2015), Uchitane et al.(2016), Yoshioka et al. studies on open evolving systems • Shimada et al. (2014,2015), Murase et al.(2015,2017) molecular dynamics simulation of granular material • Kuwabara et al.(2016) first-principle calculation of condensed matter physics • Pham et al.(2017) simulation of rescue robots • Takayanagi et al. (2016), Takami et al. (2017)
20. Our article 21 l We would greatly appreciate if you cite the following article when you publish your research using OACIS. l Y. Murase, T. Uchitane, and N. Ito, "An open-source job management framework for parameter-space exploration: OACIS", Journal of Physics: Conference Series, 921, 012001 (2017)
21. OACIS hands-on session 1 How to submit jobs on OACIS l Reference l http://crest-cassia.github.io/oacis/en/tutorial.html
22. Nagel-Schreckenberg model 26 l Nagel-Schreckenberg is a cellular-automaton model for traffic congestion, proposed in 1990s. l See [Wikipedia](https://en.wikipedia.org/wiki/Nagel%E2%80%93Schreckenberg_model) Key Description l Road length v Maximum velocity rho Car density p deceleration probability t_init thermalization steps t_measure measurement steps http://www.civil.iitb.ac.in/tvm/1111_nptel/544_TrCA/plain/ plain.html
23. 27 l It reproduces the phase transition between the free- flow phase and the congestion phase. l Source code of this simulator l https://github.com/yohm/si m_ns_model l Output files of this simulator l a JSON file containing average velocity and flow l a snapshot PNG file. position time
24. Selecting a Simulator 28 l Select a Simulator l Check Simulator settings Click Click Definition of input parameters & configurations of the simulator are registered.
25. Creating a PS and Run 29 l Select ʼParameter Setsʼ l List of ParameterSets are shown. l Click “New ParameterSet” button to create a new PS. Click Click
26. 30 l Creating a ParameterSet and Runs l Fill in the values of parameters l v = 5 l rho = 0.2 l Click “Create” button
27. 31 l A new ParameterSet and a Run are created. l The status of the Run will change in a few seconds. List of Runs under this ParameterSet. Click
28. 32 A button to download the archive of these results. Figures (bmp,jpg,png…) are displayed inline. List of output files. Click it to access. Contents of “_output.json” file is saved in OACIS DB. File path to the results.
29. Making multiple jobs 33 l In the form to create ParameterSets l fill in the values of parameters as comma-separated values l v = “1,3,5” l rho = “0.05,0.1,0.15,0.2,0.25,0.3,0.35,0.4,0.45,0.5” fill in values in a CSV form Click
30. 34 45 ParameterSets are created in total. List of ParameterSets Progress bars for job executions.
31. Making a plot 35 Click a PS Click “Plot”
32. 36 Select “line plot” X-Axis: “rho” Y-Axis: “flow” Series: "v" Click Results are displayed. Each plot has a unique URL
33. Making a heat map 37 Select “scatter plot” X-Axis: “rho” Y-Axis: “v” Result: “.flow” Jump to a PS page by double clicking a point
34. Tile view of figures 38 Select “figure viewer” X-Axis: “rho” Y-Axis: “v” Result: “.traffic.png” A magnified image is displayed by mouse-over.
35. Model 2: Barabasi-Albert model 40 http://networksciencebook.com/chapter/5
36. 41 l Simulation code: https://github.com/yohm/sim_BA_model/blob/master/BA.ipynb https://pypi.org/project/powerlaw/ implemented as a jupyter notebook
37. Model 3: BA model with attractiveness 43 l An extended BA model. l Node has an initial attractiveness A. The probability of making a link is proportional to (A+ki ). http://networksciencebook.com/chapter/6#evolving-networks
38. 44 l Try m=[3,5], A=[0,1,2,3,4,8,16] l Plot alpha (degree exponent) as a function of A (initial attractiveness) for various m. l See how P(k) changes as A or m changes.
39. Summary of session 1 49 l hands-on: executing simualtions by OACIS l selecting the Simulator l creating ParameterSets and Runs l accessing the results l output files l plots ⇒ In the next hands-on we will instruct how to implement YOUR simulator
40. OACIS hands-on session 2 how to set up your simulator l Reference l http://crest-cassia.github.io/oacis/en/configuring_simulator.html l http://crest-cassia.github.io/oacis/ja/configuring_simulator.html
41. 51 How jobs are executed database job scheduler (ex. Torque) OACIS Computational Host qsub SSH login sh execute in the work directory the work directory is compressed into a single file. tar download (scp) & expand parse job status & update the record 1. Work directory and shell script are created. Jobs are submitted to the scheduler. 2. Jobs are executed by the job scheduler. Current directory is set to the work directory. 3. Results are downloaded to OACIS. The log files are parsed and the records are updated.
42. requirements for Simulator 52 l Input parameters must be given as command line arguments or by a JSON file (Details are shown in the next slide.) l Output files must be created under the current directory. l These files are created by OACIS. Simulator must not conflict with these files. l _status.json, _time.txt, _version.txt, _log.txt, _stdout.txt, _stderr.txt l Returns 0 when it finished successfully. Returns a non zero code when it failed.
43. how to set input parameters 53 l By command-line arguments l Input parameters and the random number seed are appended to the command. l ~/simulator.out <p1> <p2> <p3> ... <seed> l By a JSON file l When running the command, a file “_input.json” is created, in which input parameters are written. l {"p1": 30,"p2": 10,"_seed":12345}
44. in NS model simulator 54 https://github.com/yohm/sim_ns_model/blob/master/ns.rb
45. Output file format 56 l As long as output files are created in the current directory, you have nothing to do. l If you would like to plot scalar values in OACIS, the values should be written in “_output.json” file. l If you would like to use "figure viewer", write files in bmp, png, jpg and other figure format. (eps files are not supported.) {"flow": 0.235, "velocity": 1.245 }
46. in NS model simulator 57
47. Making a Simulator 58 Simulator name Definition of Parameters name, type, default value preprocess command （optional） command of Simulator JSON or Argument input A note of the Simulator List of executable hosts
48. Hands-on 1 59 l Register “echo” command as a simulator. l name: echo l definition of parameters: l (p1, Integer, 0), (p2, Float, 1.0) l Command: echo l Input type: Argument l Executable_on: localhost l Run a job, and see “_stdout.txt"
49. 60 Simulator name Definition of Parameters name, type, default value “echo” Select “Argument” Select “localhost”
50. Tips for debugging 61 l When the set up is inappropriate, runs will fail. In such cases, see “_stderr.txt” for debugging. Go to the page for a failed Run Click “_stderr.txt”
51. Duplicating a Simulator 62 l You may want to try a simulation model which is slightly different from the current one. In such case, you can create a new simulator based on an existing one. Duplicating Simulator
52. “Oacisizing” your jupyter notebook 65 l Suppose that you have a prototype in jupyter notebook. l We use `papermill` to make jupyter batch executable. https://github.com/nteract/papermill $ papermill mynote.ipynb out.ipynb -f parameters.yml execute a notebook from Terminal specifying parameters
53. How to use papermill 66 l `pip install papermill` l To parameterize your notebook designate a cell with the tag parameters. l After you executed `papermill in.ipynb out.ipynb -f parameters.yml`, youʼll find “injected-parameters” in out.ipynb. parameters.yml
54. integrating with OACIS 67 1. Modify your notebook such that it accepts parameters. https://github.com/yohm/sim_BA_model/blob/master/BA.ipynb Use ”_seed” to specify th e random number seed. Set ”parameters” tag. 2. (optional) `plt.savefig` to save figure to a file. Save a png file. 3. (optional) write “_output.json” file. Write a JSON file.
55. 68 4. Prepare a shell script to run `papermill`. This part is an idiom to use pipenv. Specify your note as an input. (optional) you might want to convert ipynb to html. https://github.com/yohm/sim_BA_model/blob/master/run_BA.sh
56. 69 5. Register “run.sh” as a Simulator on OACIS. Use the same parameter nam es as the ones in your note. Input type should be JSON. Specify the shell script.
57. 70 6. Submit a test job on OACIS. If it failed, inspect “_stderr.txt”.
58. “Oacisize” more complicated Simulators 71 l In order to register your simulator on OACIS, create a script which makes your simulator conform to the input format of OACIS. l Register the script as a Simulator of OACIS. wrap script simulation program run.py _input.json a.out 1. Input parameters given by OACIS 2. parse command line args and make an XML file 3. launch the simulation program as a subprocess
59. OACIS hands-on session 3 how to automate your workflow l Reference l http://crest-cassia.github.io/oacis/en/api_python.html l http://crest-cassia.github.io/oacis/en/api_samples.html
60. What can we do with APIs? 82 Basically we can conduct any operations on OACIS. OACIS web-UI is implemented based on these methods. • creating • finding • getting info • deleting • Simulator • ParameterSet • Run • Host • Analyzer • Analysis x
61. Basic usage of APIs 83 Set "OACIS_ROOT" environment variable to the path where OACIS is installed. $ export OACIS_ROOT="$HOME/oacis" $ python3 my_script.py import os,sys sys.path.append( os.environ['OACIS_ROOT'] ) import oacis Load "oacis" module. (In the docker container, PYTHON_PATH is already set. You can skip the first two lines.) •Our sample is available at https://github.com/yohm/sim_ns_model/blob/master/oacis_find_optimum_r ho.ipynb
62. OACIS + Jupyter 84
63. Demo: Searching optimum rho 85 l Let us search optimum density for NS model l Amount of traffic flow shows non-monotonic behavior against rho.
64. Algorithm 86 1. Take the PS having the largest "flow". 2. Create two ParameterSets at the centers between its neighboring PSs. 3. Go back to 1 if we do not have an enough resolution. rho flow
65. Async & Await 87 l We want to iteratively determine parameters based on the results of finished jobs. (2) is executed when “ps1” have finished Use “OacisWatcher” class for asynchronous calls start an event-loop to monitor the completion of jobs “f1” and “f2” are asynchronously executed (4) is executed when all PS in “ps_list” have finished http://crest-cassia.github.io/oacis/en/api_watcher.html
66. Result 88
67. Conclusion 89 l We have shown how to register your Simulators on OACIS. l This might be a bit complicated, but once you have done the registration, the remaining workflow becomes much more productive and reproducible. l What we skipped includes l APIs l installation l Host and HostGroup l Analyzers l backup http://crest-cassia.github.io/oacis/
68. 90 l If you have any questions or feedbacks l oacis-dev@googlegroups.com l or @yohm13 l OACIS is still an on-going project. l Try using it in your research. Your feedbacks are really appreciated. l We are looking for users, collaborators and contributors.