Singularity 3 新機能

Singularity がHCP世界でよく使われるContainerソリューションで、シングルファイル・再現環境構築しやすい・セキュリティ高い・MPIとうまく連携などいろいろ素晴らしい特性を持っています。
この資料がSingularityの最新版の3の新機能を説明し、これからのロードマップもあります。

Transcript

2. • : • .. . . • . .
3. 0. • 3 A • 1 093 1 • 1 093 1 H • 1 1 . 2 A
4. 0685 1 6 d • 45 4 l . s Loz o • PH Gi y • G • g g i gre Gs i u re l y o • G 45l 8 6 SK y oeN gre0685 1 6 PH rgyn • 0 12 PH b a u t re G Cs0685 1 6 g 3 y o • 1 8 1 1 1
5. d G • Cce d . :/ . 6 3 3 3 mNI Sk • b • 3 0:d . 06 g nad pw y E l lids m k h urt m 0 : 3 ba • 2 3 2 . 6 3 :3
6. • il ge a W F S n • I F n 0 3 V • S b OS V • 0 . 0. 3W V S i f Ext3 SquashFS
   2.6) SIF(3.0) Partitioned SquashFS
7. ) ( ) ( 7 F ) C ( ED E D D B ( B A D ) C B D B D B G ( )) ( ) ) ( C B AD BC DEB
8. • . - F D / • / 82 - / / 2 1 / . 8 . 8 / / 2 8 1 • 82 8 : . / 2 1 - / • 82 8 : . B / 2 1 8 . : • / 1 8 8 F / • / 82 - / / 1 1 - / . : • 82 8 : . B/ / 1 1
9. C H 0 kh 3 3. • 3 . / . R 3 c 0B D a kh b a 3 3 m i 3 0 • LK o . G a 0Bpn i 3 0 a k h Pa KeyStore Service • 3. c D a kh
10. / • / . • DC • ( • :
11. • Create token - https://cloud.sylabs.io/auth • singularity push ./def.sif library://jialipassion/default/demo1.sif:latest • singularity pull demo1.sif library://jialipassion/default/demo1.sif:latest • singularity search demo1
12. ./ / / 1 • : •
13. • Create token - https://cloud.sylabs.io/auth • singularity build --remote library://jialipassion/default/demo1.sif:1.0 ./build.recipt • Build Definition File
     • https://www.sylabs.io/guides/3.0/user-guide/definition_files.html
14. / / . • / • //
15. • Create token - https://cloud.sylabs.io/auth • singularity keys newpair • singularity keys list • singularity keys push • singularity keys search (name/email)  
    
16. • ( 3 9 ad : Nf V C -. : 3 0 3 0 9 : 93 • 09 • nslookup sylabs.io | grep Server • sudo singularity exec --dns 8.8.8.8 ubuntu.sif cat /etc/resolv.conf • : 9- • 0: 93 - C B / : 9- - . 9 : 9- • 9 03 bpnotu a e • hostname -I • sudo singularity exec --net ubuntu.sif hostname -I • 9 : • . 03 • • -9 • -/ -9 • sulu Ibpnotu mirf e c P • 9 : - 3 • sudo singularity instance start --writable-tmpfs --net --network-args "portmap=8080:80/tcp" docker://nginx web2 • sudo singularity exec instance://web2 nginx • curl localhost:8080 : - - :9 ( 3 . /: /:9 - 9 9 : 93 /9
17. • A / E _p_p AD / 3 Sdkwt xs U fh • / / / • / C 3 A • 3 - go • 33 B • A / E L fh_ I eky v y L co • 3 A E • AD : 3 : k U co • am 3 : b ::/ / fh_ I CCC E /0 A 3 A 3 A 3 3 A E. : k L cng 3 A E
18. . . • • N . [memory] limit = 524288000 [cpu] share = 512 period = 100000 quota = 20000 cpus = “0-1” mems = “0-1” [blockIO] weight = 1000 leftWeight = 1000 [[devices]] access = “rwm” allow = false
19. O • / C I C bootstrap: docker-daemon from: ubuntu:latest
20. / ) • V S : • / . / / • / . ) / • ().) ) ) rm ~/.singularity/sypgp/* / . ) /
21. • I FS F • sudo singularity shell --overlay my_overlay/ ubuntu.sif • • sudo singularity build --sandbox ubuntu/ library://ubuntu • sudo singularity exec --writable ubuntu touch /foo • W F F W F F
22. : • / . / / / - .- export SINGULARITY_DOCKER_PASSWORD=<token> export SINGULARITY_DOCKER_USERNAME=$oauthtoken sudo singularity build digits.sif docker://nvcr.io/nvidia/digits:18.12-tensorflow singularity inspect --runscript digits.sif ./digits.sif export SINGULARITY_BIND=/home/david/digits:/workspace/jobs ./digits.sif # localhost:5000 sudo singularity build --sandbox digits.sand digits.sif sudo singularity shell --writable digits.sand/ cd /.singularity.d/ cp runscript startscript sudo -E singularity instance start --net --network-args "portmap=8888:5000/tcp" digits.sand/ digits sudo singularity instance list sudo singularity instance stop --all
23. bootstrap: docker from: nvcr.io/nvidia/digits:18.12-tensorflow %startscript OCI_ENTRYPOINT='"/usr/local/bin/nvidia_entrypoint.sh" "python" "-m" "digits"' OCI_CMD='' # ENTRYPOINT only - run entrypoint plus args if [ -z "$OCI_CMD" ] && [ -n "$OCI_ENTRYPOINT" ]; then SINGULARITY_OCI_RUN="${OCI_ENTRYPOINT} $@" fi # CMD only - run CMD or override with args if [ -n "$OCI_CMD" ] && [ -z "$OCI_ENTRYPOINT" ]; then if [ $# -gt 0 ]; then SINGULARITY_OCI_RUN="$@" else SINGULARITY_OCI_RUN="${OCI_CMD}" fi fi # ENTRYPOINT and CMD - run ENTRYPOINT with CMD as default args # override with user provided args if [ $# -gt 0 ]; then SINGULARITY_OCI_RUN="${OCI_ENTRYPOINT} $@" else SINGULARITY_OCI_RUN="${OCI_ENTRYPOINT} ${OCI_CMD}" fi eval ${SINGULARITY_OCI_RUN}
24. • . - / - /. .
25. Sylabs and SingularityPRO Stable, Secure, Production ready • Fully committed to open source • SingularityPRO is to Singularity as RHEL is to Fedora • LTS for several releases • Stability through backported bug fixes and features • Security though advanced security patches • Full access to Sylabs cloud infrastructure and option for on-prem hosting
26. vxmrio • () 2 93 twhas • -2 xu KI S R otdaw g s M . • 9 2 3 9 . • bghs( ) 0 9 nxu 3 9 2 9 . • 9C P Q C R C9 2 W19 e xl • ( 29 O
27. C • 0 ./ /: 0 ./ -3 : • . /: -/ - . : : • : 0 - : 0 /::/A 0 : : • -3 0 : - /: -3 -