DevOpsDays Cuba 2017: Ignite - Build and install scientific software with EasyBuild in HPC systems

D5db2dc3cc883df3479797edb63b581b?s=47 DevOpsDays Cuba
October 26, 2017
110

DevOpsDays Cuba 2017: Ignite - Build and install scientific software with EasyBuild in HPC systems

Author: Javier Antonio Ruiz Bosch
Summary:

D5db2dc3cc883df3479797edb63b581b?s=128

DevOpsDays Cuba

October 26, 2017
Tweet

Transcript

  1. Universidad Central "Marta Abreu" de Las Villas Dirección de Informatización

    Build and install scientific software with EasyBuild in HPC systems www.uclv.hpc.cu Ing. Javier Antonio Ruiz Bosch
  2. What is High Performance Computing? www.uclv.hpc.cu

  3. HPC software environments can be quite diverse HPC System Diverse

    users base Diverse software enviroment
  4. Scientific software installations for a diverse user base can be

    • Tedious • Repetitive • Error-prone • Time-consuming Only source code Installation procedure Scientific software
  5. To maintain the software in an HPC environment, a tool

    is required that offers: • Flexibility • Co-existence of versions.
  6. To maintain the software in an HPC environment, a tool

    is required that offers: • Dependency handling. • Sharing implementations of installation procedures
  7. Introducing EasyBuild http://easybuilders.github.io/easybuild/

  8. What is EasyBuild? http://easybuilders.github.io/easybuild/

  9. Installation procedure broken down into steps, as performed by EasyBuild

    1: read easyconfig 2: fetch sources 3: check readinees 4: unpack sources 5: apply patches 6: prepare 7: configure build 8: build 9: test 11: extensions 12: sanity check 13: cleanup 14: env. module 15: test cases 10: install
  10. None
  11. Lmod: based on Lua, provides a convenient way to dynamically

    change the user environment through modules. This includes easily adding or removing directories to environment variablesPATH, LIBRARY_PATH, LD_LIBRARY_PATH, etc. prepend_path("CPATH","/opt/easybuild/software/GCCcore/6.3.0/include") prepend_path("LD_LIBRARY_PATH","/opt/easybuild/software/GCCcore/6.3.0/lib") prepend_path("LD_LIBRARY_PATH","/opt/easybuild/software/GCCcore/6.3.0/lib64") prepend_path("LIBRARY_PATH","/opt/easybuild/software/GCCcore/6.3.0/lib") prepend_path("LIBRARY_PATH","/opt/easybuild/software/GCCcore/6.3.0/lib64") prepend_path("MANPATH","/opt/easybuild/software/GCCcore/6.3.0/share/man") prepend_path("PATH","/opt/easybuild/software/GCCcore/6.3.0/bin") $ module load GCC https://www.tacc.utexas.edu/research-development/tacc-projects/lmod HPC: Environment modules (Lmod)
  12. $ module load <module> $ module unload <module> $ module

    avail $ module spider $ module whatis <module> $ module list $ module purge $ module show <module> HPC: Environment modules (Lmod)
  13. www.uclv.hpc.cu Configuration: • /etc/easybuild.d/*.cfg, global configuration file for EasyBuild. modules-tool:

    Lmod prefix: /opt/easybuild sourcepath: /opt/easybuild repositorypath: /opt/develop/easybuild-easyconfig Basic usage and configuration
  14. eb --show-default-configfiles $ mkdir -p $HOME/.easybuild $ eb --confighelp >

    $HOME/.easybuild/config.cfg (available since EasyBuild v2.1.0) www.uclv.hpc.cu Basic usage and configuration
  15. The user can install their own applications using the Easybuild

    module or using the traditional Linux compilation / installation mode. $ module load EasyBuild $ eb GROMACS-2016.3-foss-2016b-GPU-enabled-build -r www.uclv.hpc.cu $ eb GROMACS-2016.3-foss-2016b -r Basic usage and configuration
  16. None
  17. None
  18. Related Work Traditional packages management tools • RPM • yum

    • APT Ports frameworks • Portage • Pacman • Pkgutils
  19. Related Work Environment modules • TCL/C • TCL-variante única •

    LMOD Modern packages management tools • Nix • Maali • SWTools • Smithy • Spack • EasyBuild
  20. Well-known supercomputing centers with EasyBuild ugent.be/hpc www.fz-juelich.de www.viennabiocenter.org scicore.unibas.ch