Spack

Spack is a package management tool designed to support multiple versions and configurations of software on a wide variety of platforms and environments. It was designed for large supercomputing centers, where many users and application teams share common installations of software on clusters with exotic architectures, using libraries that do not have a standard ABI. Spack is non-destructive: installing a new version does not break existing installations, so many configurations can coexist on the same system. Most importantly, Spack is simple. It offers a simple spec syntax so that users can specify versions and configuration options concisely. Spack is also simple for package authors: package files are written in pure Python, and specs allow package authors to maintain a single file for many different builds of the same package.

https://spack.readthedocs.io/en/latest/

Search packages online

https://packages.spack.io/

Installation

Getting Spack is easy. You can clone it from the github repository using this command:

git clone -c feature.manyFiles=true https://github.com/spack/spack.git {:.info}

This will create a directory called spack.

Once you have cloned Spack, we recommend sourcing the appropriate script for your shell:

. spack/share/spack/setup-env.sh {:.info}

For bash/zsh/sh
. spack/share/spack/setup-env.sh
(add into .bashrc)
For tcsh/csh
source spack/share/spack/setup-env.csh

Sourcing these files will put the spack command in your PATH, set up your MODULEPATH to use Spack’s packages, and add other useful shell integration for certain commands, environments, and modules. For bash and zsh, it also sets up tab completion.

• spack env create yourEnvironmentName
• spack env activate yourEnvironmentName
• spack env list # You can see a list of your available environments
• spack env status # you can see which environment you are currently
  
• spack env deactivate
• spack find # to see what packages are installed in the current environment
• spack add python@3.10
• spack add compilerName@compilerVersion
• spack install --add xxx
• compilerPath=$(spack location -i compilerName@compilerVersion)
• spack add python@3.10%gcc@9.5.0 # As a general rule, you should use the same compiler for installing all of your packages within an environment
• spack env remove yourEnvironmentName

Compiler configuration

Spack has the ability to build packages with multiple compilers and compiler versions. Compilers can be made available to Spack by specifying them manually in compilers.yaml or packages.yaml, or automatically by running spack compiler find, but for convenience Spack will automatically detect compilers the first time it needs them.

You can see which compilers are available to Spack by running

spack compilers or spack compiler list {:.info}

Any of these compilers can be used to build Spack packages.

spack compiler add {:.info}

An alias for spack compiler find.

spack compiler find {:.info}

Lists the compilers currently available to Spack. If you do not see a compiler in this list, but you want to use it with Spack, you can simply run spack compiler find with the path to where the compiler is installed. For example:

$ spack compiler find /usr/local/tools/ic-13.0.079
==> Added 1 new compiler to ~/.spack/linux/compilers.yaml
    intel@13.0.079

Or you can run spack compiler find with no arguments to force auto-detection. This is useful if you do not know where compilers are installed, but you know that new compilers have been added to your PATH. For example, you might load a module, like this:

$ module load gcc/4.9.0
$ spack compiler find
==> Added 1 new compiler to ~/.spack/linux/compilers.yaml
    gcc@4.9.0

This loads the environment module for gcc-4.9.0 to add it to PATH, and then it adds the compiler to Spack.

spack compiler info {:.info}

If you want to see specifics on a particular compiler, you can run spack compiler info on it:

$ spack compiler info intel@15
intel@15.0.0:
  paths:
    cc  = /usr/local/bin/icc-15.0.090
    cxx = /usr/local/bin/icpc-15.0.090
    f77 = /usr/local/bin/ifort-15.0.090
    fc  = /usr/local/bin/ifort-15.0.090
  modules = []
  operating_system = centos6

This shows which C, C++, and Fortran compilers were detected by Spack. Notice also that we didn’t have to be too specific about the version. We just said intel@15, and information about the only matching Intel compiler was displayed.

Manual compiler configuration

If auto-detection fails, you can manually configure a compiler by editing your ~/.spack//compilers.yaml file. You can do this by running spack config edit compilers, which will open the file in your favorite editor.

Build Your Own Compiler

If you are particular about which compiler/version you use, you might wish to have Spack build it for you. For example:

$ spack install gcc@4.9.3

Once that has finished, you will need to add it to your compilers.yaml file.

spack add gcc@4.9.3
spack install
spack compiler add `spack location -i gcc@4.9.3`

You can then set Spack to use it by default by adding the following to your packages.yaml file:

packages:
  all:
    compiler: [gcc@4.9.3]

Specs & dependencies

We know that spack install, spack uninstall, and other commands take a package name with an optional version specifier. In Spack, that descriptor is called a spec. Spack uses specs to refer to a particular build configuration (or configurations) of a package. Specs are more than a package name and a version; you can use them to specify the compiler, compiler version, architecture, compile options, and dependency options for a build. In this section, we’ll go over the full syntax of specs.

Here is an example of a much longer spec than we’ve seen thus far:

mpileaks @1.2:1.4 %gcc@4.7.5 +debug -qt target=x86_64 ^callpath @1.1 %gcc@4.7.2

If provided to spack install, this will install the mpileaks library at some version between 1.2 and 1.4 (inclusive), built using gcc at version 4.7.5 for a generic x86_64 architecture, with debug options enabled, and without Qt support. Additionally, it says to link it with the callpath library (which it depends on), and to build callpath with gcc 4.7.2. Most specs will not be as complicated as this one, but this is a good example of what is possible with specs.

More formally, a spec consists of the following pieces:

• Package name identifier (mpileaks above)
• @ Optional version specifier (@1.2:1.4)
• % Optional compiler specifier, with an optional compiler version (gcc or gcc@4.7.3)
• + or - or ~ Optional variant specifiers (+debug, -qt, or ~qt) for boolean variants. Use ++ or -- or ~~ to propagate variants through the dependencies (++debug, --qt, or ~~qt).
• name= Optional variant specifiers that are not restricted to boolean variants. Use name== to propagate variant through the dependencies.
• name= Optional compiler flag specifiers. Valid flag names are cflags, cxxflags, fflags, cppflags, ldflags, and ldlibs. Use name== to propagate compiler flags through the dependencies.
• target=<value> os=<value> Optional architecture specifier (target=haswell os=CNL10)
• ^ Dependency specs (^callpath@1.1)

套件表示法 @ 開頭表示版本號 % 開頭表示編譯器 ^ 開頭表示依賴項

Spack load

There are several different ways to use Spack packages once you have installed them. As you’ve seen, spack packages are installed into long paths with hashes, and you need a way to get them into your path. The easiest way is to use spack load, which is described in this section.

spack load mpich %gcc@4.4.7

Slurm

#!/bin/sh
# This file is called submit-script.sh
#SBATCH --partition=shared       # default "shared", if not specified
#SBATCH --time=0-04:30:00        # run time in days-hh:mm:ss
#SBATCH --nodes=1                # require 1 nodes
#SBATCH --ntasks-per-node=64     # cpus per node (by default, "ntasks"="cpus")
#SBATCH --mem-per-cpu=4000       # RAM per node in megabytes
#SBATCH --error=job.%J.err
#SBATCH --output=job.%J.out
# v---Remember to activate your Spack environment!! 
spack env activate yourEnvironmentName
srun --mpi=pmix -n 64 /home/username/mpiprogram

References

1. Spack 指令教學