Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DevOpsDays Cuba 2017: Ignite - Build and install scientific software with EasyBuild in HPC systems

DevOpsDays Cuba
October 26, 2017
310

DevOpsDays Cuba 2017: Ignite - Build and install scientific software with EasyBuild in HPC systems

Author: Javier Antonio Ruiz Bosch
Summary:

DevOpsDays Cuba

October 26, 2017
Tweet

Transcript

  1. Universidad Central "Marta Abreu" de Las Villas Dirección de Informatización

    Build and install scientific software with EasyBuild in HPC systems www.uclv.hpc.cu Ing. Javier Antonio Ruiz Bosch
  2. Scientific software installations for a diverse user base can be

    • Tedious • Repetitive • Error-prone • Time-consuming Only source code Installation procedure Scientific software
  3. To maintain the software in an HPC environment, a tool

    is required that offers: • Flexibility • Co-existence of versions.
  4. To maintain the software in an HPC environment, a tool

    is required that offers: • Dependency handling. • Sharing implementations of installation procedures
  5. Installation procedure broken down into steps, as performed by EasyBuild

    1: read easyconfig 2: fetch sources 3: check readinees 4: unpack sources 5: apply patches 6: prepare 7: configure build 8: build 9: test 11: extensions 12: sanity check 13: cleanup 14: env. module 15: test cases 10: install
  6. Lmod: based on Lua, provides a convenient way to dynamically

    change the user environment through modules. This includes easily adding or removing directories to environment variablesPATH, LIBRARY_PATH, LD_LIBRARY_PATH, etc. prepend_path("CPATH","/opt/easybuild/software/GCCcore/6.3.0/include") prepend_path("LD_LIBRARY_PATH","/opt/easybuild/software/GCCcore/6.3.0/lib") prepend_path("LD_LIBRARY_PATH","/opt/easybuild/software/GCCcore/6.3.0/lib64") prepend_path("LIBRARY_PATH","/opt/easybuild/software/GCCcore/6.3.0/lib") prepend_path("LIBRARY_PATH","/opt/easybuild/software/GCCcore/6.3.0/lib64") prepend_path("MANPATH","/opt/easybuild/software/GCCcore/6.3.0/share/man") prepend_path("PATH","/opt/easybuild/software/GCCcore/6.3.0/bin") $ module load GCC https://www.tacc.utexas.edu/research-development/tacc-projects/lmod HPC: Environment modules (Lmod)
  7. $ module load <module> $ module unload <module> $ module

    avail $ module spider $ module whatis <module> $ module list $ module purge $ module show <module> HPC: Environment modules (Lmod)
  8. www.uclv.hpc.cu Configuration: • /etc/easybuild.d/*.cfg, global configuration file for EasyBuild. modules-tool:

    Lmod prefix: /opt/easybuild sourcepath: /opt/easybuild repositorypath: /opt/develop/easybuild-easyconfig Basic usage and configuration
  9. eb --show-default-configfiles $ mkdir -p $HOME/.easybuild $ eb --confighelp >

    $HOME/.easybuild/config.cfg (available since EasyBuild v2.1.0) www.uclv.hpc.cu Basic usage and configuration
  10. The user can install their own applications using the Easybuild

    module or using the traditional Linux compilation / installation mode. $ module load EasyBuild $ eb GROMACS-2016.3-foss-2016b-GPU-enabled-build -r www.uclv.hpc.cu $ eb GROMACS-2016.3-foss-2016b -r Basic usage and configuration
  11. Related Work Traditional packages management tools • RPM • yum

    • APT Ports frameworks • Portage • Pacman • Pkgutils
  12. Related Work Environment modules • TCL/C • TCL-variante única •

    LMOD Modern packages management tools • Nix • Maali • SWTools • Smithy • Spack • EasyBuild