Access the Cluster

To connect, you will need to ssh to the login node of the cluster:

  • Connect from MacOS, use ‘Terminal’ that comes with MacOS
  • Connect from Windows, use Putty which can be downloaded for free
  • Off-campus access to the cluster requires VPN
  • Your HPCC user id is the same as your user id for other network services (e.g. email, wireless, network printing etc) but its password is not in sync with Active Directory. You can reset your password of your HPCC account at (ssh [email protected]).

Submit Jobs to the Scheduler

To submit a job, you first write a simple shell script that wraps your job called a “submission script”. A submission script is comprised of two parts, the “directives” that tell the scheduler how to setup the computational resources for your job and the actual “script” portion, which are the commands you want executed during your job.

The directives are comprised of “#PBS” followed by a flag. Most commonly used flags (should be in just about every submission script) include:

-N job_name                        Custom job name
-q queue_name                   Queue where your job is sent to
-l nodes=<N>:ppn=8            Total number of nodes (N):number of processors (cores)
-l walltime=<HH:MM:SS>      Walltime of the job in Hours:Minutes:seconds
-l mem=<M>gb                     Memory requested for job.
-m abe -M <email address> Sends you job status reports when you job starts (b), aborts (a), and/or finishes (e)
-o file_name                        Specify file name for standard output from job
-e file_name                        Specify file name for standard error from job

Here is what a sample job submission script looks like
#PBS -N MyJob
#PBS -q hpcc
#PBS -l nodes=1:ppn=12
#PBS -l walltime=72:00:00
#PBS -l mem=48GB
#PBS -m abe -M [email protected]
#PBS -j oe
#PBS -k o

echo “”
echo “***Ignore Warnings for access to tty***”
echo “”
echo -n “Changing to directory… ”
echo “”
echo “Our List of nodes:”
echo “”
echo “”
echo -n “Loading module mpi/mpich-x86_64…. ”
module load mpi/mpich-x86_64
echo “Done!”
echo “”
echo -n “Finished program at: ”
echo “”
stata-mp -b do

To submit

If you save your job submission file as MyJobscript.pbs, you can just type qsub < your_job_submission_script> at the command line

$ qsub myjobscript.pbs

More PBS commands that you can use to monitor and manage your jobs (e.g. check job status, delete queued and/or running jobs, show jobs queued ……).

Transfer files to a Cluster

You can transfer files between your local computer and the cluster using an FTP client, such as FileZilla, WinSCP. You will need to configure the client with your hpcc account as User, the cluster login node as Host. Set the Protocol to SFTP – SSH File Transfer Protocol. Here is an example configuration of FileZilla on Windows as shown below.

In addition to SFTP, the cluster also supports SSHFS (tutorial on how to use SSHFS to mount remote file systems), SCP and SMB protocol

Use the Cluster Interactively

Interactive jobs can be used for testing and troubleshooting code. By requesting an interactive job, you will get a shell on the requested node to yourself.

$ qsub -I -q hpcc -l nodes=1:ppn=2 -l walltime=00:60:00

This will assign a free node with the resources you have requested, 2 cores on one of the free nodes and the walltime of 60 minutes. It will put your account within a shell on that node. You can run any number of commands within that shell. To free the allocated resources, exit from the shell. The environment variable $PBS_NODEFILE is set to the name of a file containing the names of the node you were allocated. You can also create a job submission script (e.g. interactive.pbs) and submit using qsub command (e.g. $ qsub interactive.pbs). When using an interactive shell, your job is vulnerable to being killed if you lose your network connection.