HPC Crash Course April, 2014 

(you can’t do it on your PC) Research computing in a nutshell 

(you can’t do it on Windows) Research computing in a nutshell 

You need Linux power! 

ILRI Magic Research Computing Cloud 

ILRI Not-Magic Research Computing Cloud 

ILRI Not-Magic Research Computing Cloud: demystified You HPC 

Brief intro to Linux commands cd – change directory mkdir – make directory mv – move a file or directory cp – copy a file or folder rm - remove a file whoami – print the name of the current user who – print a list of other users who are logged in date – print the current date and time on the server echo – print text to the screen 

Linux command structure Linux commands come in various forms. Some are simple, and can be used by themselves: ls rm date 

Linux command structure ... other times you can add “arguments” to change the behavior of the command. Arguments are separated by one or more spaces: mkdir data Some commands require arguments (they don’t make sense to run by themselves, like mkdir). 

Common pitfalls with Linux commands - Case sensitive (Ls vs ls)! - Attention to detail (ls -l vs ls --l)! - Pasting from Word (“pretty” quotes, etc)! - Missing spaces - Using Windows-isms (\ vs /)! Google is your friend! 

Basic HPC Workflow - Log into hpc.ilri.cgiar.org - Request a resource allocation - ??? - Profit 

Basic HPC Workflow Demonstrated [aorth@noma: ~]$ ssh hpc.ilri.cgiar.org [aorth@hpc: ~]$ interactive salloc: Granted job allocation 5462 [aorth@compute2: ~]$ 

SLURM: What you need to know - SLURM is the Simple Linux Utility for Resource Management - Manages CPUs and jobs on the cluster - Ensures everyone plays fair - Everyone gets ~20 CPUs by default - More info: http://hpc.ilri.cgiar.org/using-slurm Some commands: interactive - allocates resources for interactive shell (1 CPU) squeue - see a list of running / queued jobs scancel - cancel a job sbatch - submit a “batch” job (ie, not interactive) 

Accessing Software on HPC - Lots of software installed here! - Here’s a pretty up-to-date list: http://hpc.ilri.cgiar.org/list-of-software - Also, see the list using modules: module avail 

Accessing Software on HPC You can load a specific version of software module load blast/2.2.29+ blastn -query alan.fna -db nt List and unload modules with: module unload blast 

Rules of HPC Club - Use SLURM (interactive is your friend) - Try not to write to your home folder - Don’t run jobs on the head node (HPC) - If you crash the cluster you buy cookies for Alan - Don’t talk about HPC Club