SoftwareInstallation

Can I install my own software?

The answer is: yes, (of course) you can. There is a group-specific directory that is made available for this purpose:

/hpc/local/osversion/group/

Note that osversion is the current Linux version and group is your own group name. Here, you can install any software you like and maintain it yourself. See here for explanations how to install C, Perl or R packages.

Software of general interest

If you find a software to be of general interest to HPC users, let us know. We can provide is as an software module or install it as a system package (rpm) and update it on a regular basis.

However, if you are dependent on a specific version of a software package and don't want regular updates, we advise you to install it yourself and encourage you to take the benefits of #Making software available using LMOD.

How to install your own software

A specific directory is made available for group-specific software to be installed:

/hpc/local/osversion/group/

C software

You can download your C software of interest and unpack it in this directory. Typically, a pre-installation configuration is done by executing:

./configure --prefix /hpc/local/osversion/group/package

This will create a "Makefile" which explains to the "make" utility how the software should be compiled, and where the software will be installed. The 'prefix' is the top directory under which the whole package will be installed. You may want to include the package version number in the name of this directory; that way you can install more than one version next to each other. Use symlinks to point to the 'default' installation.

After the configure step, you compile the software using:

make

The Makefile will be read, building the application binaries. To install these binaries, use:

make install

That is it! You can check the user documentation of the installed software for details of how to run the application.

Python modules

use pip to install python modules.
module load python
module list
pip list
pip install --user <module>

Create your virtual software environment for a specific Python project with Python Virtual Environment :
see : http://docs.python-guide.org/en/latest/dev/virtualenvs

Your own R version

Go to https://cran.r-project.org/mirrors.html, download the latest "R-3.0.whatever.tar.gz" file to a submit host (hpcs01/hpcs02). Copy this file to a directory that has enough space (about 300M). In this case, let's assume /tmp, but your homedir probably has enough space as well.

cd /tmp
wget http://cran-mirror.cs.uu.nl/src/base/R-3/R-3.0.2.tar.gz
tar -zxvf R-3.0.2.tar.gz
cd R-3.0.2

Now, we'll have to "configure" and "make" this:

./configure --prefix=/hpc/local/CentOS7/bofh/R-3.0.2
make
make install

Where you replace "/hpc/local/CentOS7/bofh/R-3.0.2" with some path where you have write-access. Something like "/hpc/local/CentOS7/YOURGROUP/R-3.0.2" would probably be a good choice.

Now, you would start this version of R by entering the full path:

/hpc/local/CentOS7/bofh/R-3.0.2/bin/R

Or by adding the directory /hpc/local/CentOS7/YOURGROUP/R-3.0.2/bin to your own PATH (e.g. in your $HOME/.bash_profile).

R packages

Packages can easily be installed inside R by providing a local path:

R
install.packages( "yourLibrary", lib = "/hpc/local/osversion/group/path" )
library( "yourLibrary", lib.loc = "/hpc/local/osversion/group/path" )

Alternatively, you can customize your Linux environment variables and set R_LIBS to /hpc/local/osversion/group/path. This way, you can leave out the path specification in R.

Check
module load R
R
find.package('<mypackage>')

Perl libraries

To install CPAN perl libraries, you first have to instruct CPAN which directory to use. You can do this by modifying your CPAN configuration, from within the CPAN shell:

cpan
o conf mbuildpl_arg "installdirs=site install_base=/hpc/local/osversion/group"
o conf makepl_arg "INSTALLDIRS=site INSTALL_BASE=/hpc/local/osversion/group"
o conf prefer_installer MB
o conf prerequisites_policy follow
o conf commit

After this, CPAN should install all perl libraries in the appropriate directory such that they are available on the entire cluster. Your PERLLIB environment variable should be set to include /hpc/local/OSVERSION/GROUP/lib/perl5.

In the past, the environment variable PERL_INSTALL_ROOT used to be used for cpan installs, but that doesn't work anymore and wreaks havoc on the install if you use the cpan configuration shown above. I.o.w., be sure not to define PERL_INSTALL_ROOT.

If you later just wish to install packages, you can use the cpan -i command from the command line. No need to startup the CPAN shell.

Making software available using `GUIX`

Guix is a package manager and is installed for general use.

You can find information in the

Wiki .

and

Guix website in the UMC-DBG domain .

Add these lines in your ~/.bash_profile file to enable guix .

# guix environment 
# see : https://hpcguix.op.umcutrecht.nl/getting-started 

export PATH=$PATH:"/gnu/profiles/base/bin"

GX_PROFILE="$HOME/.guix-profile"
if [ -f "$GX_PROFILE/etc/profile" ]; then
  . "$GX_PROFILE/etc/profile"
fi

Making software available using `LMOD`

#Making software available using LMOD.

Making software available using `SINGULARITY`

Singularity

(docker compatible but more secure) is available on the compute-nodes (n00XX).
You can download docker- or singularity- images and run your programs in singularty.
p.e. R and SAIGE module :

You can execute this in a qlogin session.

SAIGE on singularity(image)
# Download a singularty image for SAIGE:
singularity pull shub://singularity-hub.org/statgen/singularity-saige

# Login into the container
singularity shell ./statgen-singularity-saige-master-latest.simg

# Check if SAIGE is available

R
installed.packages() 
q()
n

# run a R script in the container
# create 1.R

library("SAIGE") 
print("Hello World!")

# You can run this in a qsub script
# Run your code by :
singularity exec ./saige-0.35.8.1.simg Rscript 1.R

SAIGE on singularity with docker image

singularity pull docker://wzhou88/saige:0.35.8.1
singularity shell ./saige:0.35.8.1
R
installed.packages()
q()
n

# run
singularity exec ./saige-0.35.8.1.simg Rscript 1.R

This topic: HPC > WebHome > SoftwareInstallation
Topic revision: r16 - 2020-08-10 - ReneJanssen