ARPACK
Introduction
From the ARPACK website:
The package is designed to compute a few eigenvalues and corresponding eigenvectors of a general n by n matrix A. It is most appropriate for large sparse or structured matrices A where structured means that a matrix-vector product w <- Av requires order n rather than the usual order n2 floating point operations. This software is based upon an algorithmic variant of the Arnoldi process called the Implicitly Restarted Arnoldi Method (IRAM). When the matrix A is symmetric it reduces to a variant of the Lanczos process called the Implicitly Restarted Lanczos Method (IRLM). These variants may be viewed as a synthesis of the Arnoldi/Lanczos process with the Implicitly Shifted QR technique that is suitable for large scale problems. For many standard problems, a matrix factorization is not required. Only the action of the matrix on a vector is needed.
Use Arpack on HPC cluster
Parallel Arpack was compiled for GNU, Intel and PGI compilers for both openmpi and mvapich2.
The corresponding libraries are located in
/opt/hpc/[gnu|intel|pgi]/[openmpi|mvapich2]/lib64
For example, to use libarpack.so and libparpack.so built from gnu-openmpi Fortran compiler, add the following options to your command line:
-L/opt/hpc/gnu/openmpi/lib64 -lparpack -larpack
Note that arpack is based upon blas and lapack library, and parpack calls for arpack. Therefore, we have to link to both the lapack and blas package, and specify the path to these two packages according to the compiler you are using.
For Gnu compiler, add
-L/usr/lib64 -lblas -llapack
For Intel compiler, we use the lapack and blas from the intel-MKL library, you need add
-L/gpfs/research/software/intel/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread -lm -lmkl_blas95_lp64 -lmkl_lapack95_lp64
For PGI compiler, add
-L/gpfs/research/software/pgi/linux86-64/14.3/lib -llapack -lblas
To run simple examples provided by the Arpack developers, following the following instructions:
# copy serial and parallel examples to your home directories
$ cd $HOME
$ mkdir arpack
$ cd arpack
$ mkdir SERIAL
$ mkdir PARALLEL
$ cp -r /opt/hpc/share/doc/arpack-doc-3.1.5/EXAMPLES SERIAL
$ cp -r /opt/hpc/share/doc/arpack-doc-3.1.5/MPI PARALLEL
To compile and run a serial example (e.g., `sssimp.f``, a serial f77 program provided by the software developer) using Gnu compiler:
$ cd $HOME/arpack/SERIAL/EXAMPLES/SIMPLE
$ gfortran sssimp.f -L/opt/hpc/gnu/lib64 -larpack -lblas -llapack -o sssimp
$ ./sssimp
To complile and run a parallel example on the HPC cluster using Gnu compiler:
$ cd $HOME/arpack/PARALLEL
$ module load gnu-openmpi
$ mpif77 pssdrv1.f -L/opt/hpc/gnu/openmpi/lib64 -larpack -lparpack -lblas -llapack -o pssdrv1
$ sbatch sub_parpack.sh
If you use intel mvapich2
$ module load intel-mvapich2
$ mpif77 pssdrv1.f -L/opt/hpc/intel/mvapich2/lib64 -larpack -lparpack -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread -lm -lmkl_blas95_lp64 -lmkl_lapack95_lp64 -L/gpfs/research/software/intel/mkl/lib/intel64 -o pssdrv1
Here is an example submit script sub_parpack.sh:
$ cat sub_parpack.sh
#!/bin/bash
#SBATCH --job-name="parpack_test"
#SBATCH -N 4
#SBATCH -p backfill
#SBATCH --mail-type="ALL"
#SBATCH -t 00:05:00
module purge
module load gnu-openmpi
cd $SLURM_SUBMIT_DIR
mpirun -np 4 ./pssdrv1
You will see the following outputs upon a successful run:
_saupd: number of update iterations taken
-----------------------------------------
1 - 1: 5
_saupd: number of "converged" Ritz values
-----------------------------------------
1 - 1: 4
_saupd: final Ritz values
-------------------------
1 - 4: 6.350E-01 3.985E-01 3.985E-01 1.620E-01
_saupd: corresponding error bounds
----------------------------------
1 - 4: 1.381E-13 8.283E-12 1.017E-10 3.955E-21
==========================================
= Symmetric implicit Arnoldi update code =
= Version Number: 2.1 =
= Version Date: 3/19/97 =
==========================================
= Summary of timing statistics =
==========================================
Total number update iterations = 5
Total number of OP*x operations = 81
Total number of B*x operations = 0
Total number of reorthogonalization steps = 81
Total number of iterative refinement steps = 0
Total number of restart steps = 0
Total time in user OP*x operation = 0.001000
Total time in user B*x operation = 0.000000
Total time in Arnoldi update routine = 0.001999
Total time in p_saup2 routine = 0.001999
Total time in basic Arnoldi iteration loop = 0.001999
Total time in reorthogonalization phase = 0.000000
Total time in (re)start vector generation = 0.000000
Total time in trid eigenvalue subproblem = 0.000000
Total time in getting the shifts = 0.000000
Total time in applying the shifts = 0.000000
Total time in convergence testing = 0.000000
Ritz values and direct residuals
--------------------------------
Col 1 Col 2
Row 1: 1.62029E-01 1.44471E-06
Row 2: 3.98507E-01 1.35166E-06
Row 3: 3.98508E-01 1.75542E-06
Row 4: 6.34986E-01 1.71304E-06
_SDRV1
======
Size of the matrix is 100
The number of processors is 4
The number of Ritz values requested is 4
The number of Arnoldi vectors generated (NCV) is 20
What portion of the spectrum: SM
The number of converged Ritz values is 4
The number of Implicit Arnoldi update iterations taken is 5
The number of OP*x is 81
The convergence criterion is 5.96046448E-08
Use Arpack on Spear nodes
The serial version of Arpack was compiled for Intel, GNU, and PGI compilers, respectively.
To compile applications based on arpack, add following options to your command line:
-L/opt/hpc/intel/lib64 -larpack -lblas -llapack
For example,
$ ifort -o a.out -L/opt/hpc/intel/lib64 -larpack -lblas -llapack myprog.f
The examples provided by the software developers are located in
/opt/hpc/share/doc/arpack-serial-3.1.5/EXAMPLES