Williams HPC cluster is a shared Linux computing resource supporting research and teaching. It is available for all faculty, staff and students who need high performance computing in their work and study at Williams. To request an account please email us at [email protected].
Hardware
1 Head Node
4 Compute Nodes – Total of 256 Cores and 896 GB RAM
64 Cores w/ 128 GB RAM (1)
64 Cores w/ 256 GB RAM (3)
Queues and Scheduler
The cluster uses TORQUE for resources management and MAUI to schedule jobs. The policy has a fair-share component and backfill implementation to provide all users fair access to cluster resources. The current setup:
| name | max nodes/cores | max walltime | base priority | description |
|---|---|---|---|---|
| hpcc | 4 / 256 | up to 720 hours | normal | for all normal jobs |
| matlab | 4 / 96 | up to 720 hours | normal | dedicated for Matlab MDCS |
| debug | 2 / 32 | 1 hour | highest | for debugging |
Software
- Mathematica
- Matlab MDCS
- R
- Stata/MP
- MrBayes, Topcom, Macaulay2, Polymake, IMa3, SageMath and more ……
- Multipe versions of a particular software are managed through Modules
- To see a list of modules that are available to be loaded, type “module avail”
- To load a module, type “module load ”
Feel free to install things for yourself. If you would prefer we take care of things for you, contact [email protected].
Request an Account
The first step in gaining access to our clusters is requesting an account. Please email [email protected] for more information.
Guidelines
Before you begin using the cluster, here are some important guidelines:
- Do not run jobs or do real work on the head node (aka login node). Always allocate a compute node and run programs there
- Never give your password or ssh key to anyone else.
- Clean up after yourself by releasing unused jobs and removing unneeded files.
Log in
hpcc.williams.edu is accessed via a protocol called secure shell (ssh). You can use ssh directly. From a Mac, use Mac Terminal. On Windows, you can use Putty. If you want to access the cluster from outside Williams, you must use the Williams VPN. For more information on ssh and how to connect to the cluster with your application and operating system of choice, please see getting-started for more information.
Transfer Your Files
You will likely find it necessary to copy files between your local machines and the clusters. Just as with logging in, there are different ways to do this, depending on your local operating system. We support SFTP, SSHFS, SCP and SMP protocol. Please see getting-started for more information.
Use Software
To best serve the diverse needs of all the software that you need in your work in an HPCC environment, we use a module system to manage software. This allows you to swap between different application and versions of those applications with relative ease and focus on getting your work done, not compiling software. Please see Software Guide for more information. If you find software that you’d like to use that isn’t available, feel free to contact [email protected].
Schedule a Job
You control your jobs using a job scheduling system that dedicates and manages compute resources for you. Basically this is done in one of two ways. For testing and debugging you may want to run your job interactively. This way you can directly interact with the compute node(s) in real time to make sure your code works and your jobs will run as expected. The other way, which is the preferred way for large and long-running jobs, involves writing your job commands in a script and submitting that to the job scheduler. Please see Getting-started for more information.
Current Status of the Cluster
The cluster is monitored using Ganglia (cluster monitoring system). You can check the status of the cluster and the and its load live from this link.
New to Linux?
You don’t need to be a Linux expert to use the cluster but familiarity with Linux commands is required for interacting with the cluster. We have a Unix Commands Cheat Sheet that can help you get started.
Megamenu Social