DevOpsDays Cuba 2017: Ignite - Build and install scientific software with EasyBuild in HPC systems

Author: Javier Antonio Ruiz Bosch
Summary:

Transcript

1. Universidad Central "Marta Abreu" de Las Villas Dirección de Informatización Build and install scientific software with EasyBuild in HPC systems www.uclv.hpc.cu Ing. Javier Antonio Ruiz Bosch
2. What is High Performance Computing? www.uclv.hpc.cu
3. HPC software environments can be quite diverse HPC System Diverse users base Diverse software enviroment
4. Scientific software installations for a diverse user base can be • Tedious • Repetitive • Error-prone • Time-consuming Only source code Installation procedure Scientific software
5. To maintain the software in an HPC environment, a tool is required that offers: • Flexibility • Co-existence of versions.
6. To maintain the software in an HPC environment, a tool is required that offers: • Dependency handling. • Sharing implementations of installation procedures
7. Introducing EasyBuild http://easybuilders.github.io/easybuild/
8. What is EasyBuild? http://easybuilders.github.io/easybuild/
9. Installation procedure broken down into steps, as performed by EasyBuild 1: read easyconfig 2: fetch sources 3: check readinees 4: unpack sources 5: apply patches 6: prepare 7: configure build 8: build 9: test 11: extensions 12: sanity check 13: cleanup 14: env. module 15: test cases 10: install
11. Lmod: based on Lua, provides a convenient way to dynamically change the user environment through modules. This includes easily adding or removing directories to environment variablesPATH, LIBRARY_PATH, LD_LIBRARY_PATH, etc. prepend_path("CPATH","/opt/easybuild/software/GCCcore/6.3.0/include") prepend_path("LD_LIBRARY_PATH","/opt/easybuild/software/GCCcore/6.3.0/lib") prepend_path("LD_LIBRARY_PATH","/opt/easybuild/software/GCCcore/6.3.0/lib64") prepend_path("LIBRARY_PATH","/opt/easybuild/software/GCCcore/6.3.0/lib") prepend_path("LIBRARY_PATH","/opt/easybuild/software/GCCcore/6.3.0/lib64") prepend_path("MANPATH","/opt/easybuild/software/GCCcore/6.3.0/share/man") prepend_path("PATH","/opt/easybuild/software/GCCcore/6.3.0/bin") $ module load GCC https://www.tacc.utexas.edu/research-development/tacc-projects/lmod HPC: Environment modules (Lmod)
12. $ module load <module> $ module unload <module> $ module avail $ module spider $ module whatis <module> $ module list $ module purge $ module show <module> HPC: Environment modules (Lmod)
13. www.uclv.hpc.cu Configuration: • /etc/easybuild.d/*.cfg, global configuration file for EasyBuild. modules-tool: Lmod prefix: /opt/easybuild sourcepath: /opt/easybuild repositorypath: /opt/develop/easybuild-easyconfig Basic usage and configuration
14. eb --show-default-configfiles $ mkdir -p $HOME/.easybuild $ eb --confighelp > $HOME/.easybuild/config.cfg (available since EasyBuild v2.1.0) www.uclv.hpc.cu Basic usage and configuration
15. The user can install their own applications using the Easybuild module or using the traditional Linux compilation / installation mode. $ module load EasyBuild $ eb GROMACS-2016.3-foss-2016b-GPU-enabled-build -r www.uclv.hpc.cu $ eb GROMACS-2016.3-foss-2016b -r Basic usage and configuration
18. Related Work Traditional packages management tools • RPM • yum • APT Ports frameworks • Portage • Pacman • Pkgutils
19. Related Work Environment modules • TCL/C • TCL-variante única • LMOD Modern packages management tools • Nix • Maali • SWTools • Smithy • Spack • EasyBuild
20. Well-known supercomputing centers with EasyBuild ugent.be/hpc www.fz-juelich.de www.viennabiocenter.org scicore.unibas.ch