and
`#SBATCH --ntasks=N` | ntasks specifies the number of tasks (or processes) the job will run. By default, you will be allocated one CPU/task. This can be increased by including the additional directive --cpus-per-task. | | `#SBATCH --mem=Ngb` | Select N gb of memory per node. If "gb" is not included, this value defaults to MB. | After setting your directives, you can instruct the HPC to do what you require similar to a bash script. Here's an example of a batch job: ``` #!/bin/bash #SBATCH --job-name=blast_job # Job name #SBATCH --partition=standard # Sets the job priority to standard #SBATCH --nodes=1 # Number of nodes #SBATCH --ntasks=1 # Number of tasks (processes) per node #SBATCH --cpus-per-task=4 # Number of CPU cores per task #SBATCH --mem=8G # Memory per node (in this case, 8GB) #SBATCH --time=02:00:00 # Time limit (HH:MM:SS) # Load necessary modules module load blast/2.12.0 # Load BLAST module (adjust version as needed) # Change to the directory where the job will run cd $SLURM_SUBMIT_DIR # Define input and output files query_file="query.fasta" # Input query file (FASTA format) database_file="database.fasta" # BLAST database file (FASTA format) output_file="blast_results.out" # Output file for BLAST results # Run BLAST command blastp -query $query_file -db $database_file -out $output_file -evalue 0.001 -num_threads $SLURM_CPUS_PER_TASK ``` !!! tip "Submitting a Batch Script" - To submit jobs you need to use `sbatch`, such as `sbatch script.slurm` - To cancel your job you do `scancel`, such as `scancel $JOBID` or `scancel -u $NETID` This will submit your job to the queue. Execution will depend on your submission type (partition). #### Launching an Interactive Node An **interactive node**, unlike batch jobs which are run asynchronously, allows immediate access to compute. Similar to batch jobs, interactive nodes are submitted to the queue, but once available, you will receive a prompt for a node with the selected resources. Read more on how to launch interactive jobs in [the official documentation](https://uarizona.atlassian.net/wiki/spaces/UAHPC/pages/75989825/Interactive+Jobs){target=_blank}. !!! tip "The Quick and Dirty" Don't need a lot of resources and just want access to the compute? Just type `interactive`. Disclaimer: you may require to wait longer as your job is going to fall in the `windfall` queue. Following are a list of useful flags (options) for setting up the interactive node. |Flag|Default value| Description| Example| |-|-|-|-| | `-n` | 1 | Number of CPUs requested per node | interactive -n 8 | | `-m` | 4GB | Memory per CPU | interactive -m 5GB | | `-a` | none | Account (group) to charge | interactive -a datalab | | `--partition=` | windfall | Partition to determine CPU time charges and is set to windfall when no account is specified, and is **set to standard when an account is provided.** | interactive --partition=windfall | | `-t` | 01:00:00 | Time allocated to session. | interactive -t 08:00:00 | | `-N` | 1 | Number of nodes. | There is no reason to exceed 1 node unless the number of CPUs requested is greater than the number of CPUs per node on a given cluster. | (in el gato, where number of CPU per node is 16 max ) interactive -n 32 -N 2 | An example for an interactive node is: ``` interactive -n 8 -m 16GB -a datalab -t 02:00:00 ``` The above example will request an interactive node with 8 cores, 16GB RAM, "charging" the datalab, running for 2 hours. Try it! !!! note "Modules" There are 100s of tools installed on the HPC, few of which are available on the login screen. These tools are available only during a batch job submission or within interactive jobs. To see what tools are already running, or which are available, you will need to use the `module` command. !!! tip "Helpful `module` commands" |Command|Purpose| |-|-| |`module list`| Lists loaded modules| |`module avail`| Lists available modules| |`module spider`| Lists ALL modules| |`module load`| Loads a module| |`module help`| Help command!| --- ## Singularity/Apptainer In 2021, the Sylabs, the developers behind the original Singularity, made a fork of the original project and renamed it [SingularityCE (Community Edition)](https://github.com/sylabs/singularity){target=_blank}. This would allow for the SingularityCE to be compliat with FOSS and allowing for the community to contribute to the builds. The Singularity team then joined the [Linux Foundation](https://en.wikipedia.org/wiki/Linux_Foundation){target=_blank} and decided to rename their effor to [**Apptainer**](https://github.com/apptainer/apptainer){target=_blank}. The technology behind Singularity/Apptainer is similar to the one of Docker, but, as mentioned before, it was created with the HPC in mind, and therefore bypasses the requirement of sudo. !!! note Until now we have used Singularity/Apptainer to refer to the same software. Onwards, you can decide whether to use Singularity *OR* Apptainer; in order to keep up with the latest release, we are going to be executing apptainer commands. Apptainer also installs a `singularity` command as an alias, so older instructions that use `singularity` still work. ??? info "Docker vs SingularityCE & Apptainer in the blink of an eye" **:material-open-source-initiative: Apptainer and SingularityCE are 100% compatible with Docker but they do have some distinct differences** **:material-docker: Docker** :octicons-container-24: Docker containers run as `root` - This privilege is almost never supported by administrators of High Performance Computing (HPC) centers. Meaning Docker is not, and will likely never be, installed natively on your HPC cluster. :octicons-container-24: uses compressed layers to create one image **:material-open-source-initiative: SingularityCE & Apptainer**: :octicons-container-24: Same user and group identity inside as outside the container :octicons-container-24: User only has `root` privileges if elevated with `sudo` when the container is run :octicons-container-24: Can run and modify any existing Docker image - These key differences allow Singularity to be installed on most HPC centers. Because you can run virtually all Docker containers in Singularity, you can effectively run Docker on an HPC. ### General Executable Commands Resources: - https://unm-carc.github.io/container-camp/singularity/intro/ - https://unm-carc.github.io/container-camp/singularity/hpc/ - https://unm-carc.github.io/container-camp/singularity/advanced/ Apptainer’s [command line interface](https://apptainer.org/docs/user/main/cli.html){target=_blank} allows you to build and interact with containers transparently. You can run programs inside a container as if they were running on your host system. You can easily redirect IO, use pipes, pass arguments, and access files, sockets, and ports on the host system from within a container. #### :octicons-container-24: help The `help` command gives an overview of Apptainer options and subcommands as follows: ``` $ apptainer help pull Pull an image from a URI Usage: apptainer pull [pull options...] [output file]
Adapted from [CyVerse FOSS](https://github.com/CyVerse-learning-materials/foss/blob/86d4cc384976a59bd19cce2a6fdcf994b19051bd/docs/09_HPC.md){target=_blank} (last source update 2025-10-21), CC BY 4.0. Spotted a problem? [Open an issue](https://github.com/UNM-CARC/foss/issues){target=_blank}.