Usage

Can I run non-parallel (serial) jobs on the clusters?

Yes, Many of the applications run on our clusters are serial jobs.

Connecting to a cluster

How to I connect to your clusters?

First, you need an account ( CAC account application ) . Once you have your account, you can connect to a cluster using ssh.

I'm having problems connecting with scp?

If you are using a calling a different shell for login, you should check that it is an interactive shell. You can do so w/ the following test:

if [ ! -z $PS1 ] ; then
if [ -x /usr/bin/tcsh ] ; then
exec /usr/bin/tcsh
fi
fi

The PS1 is set for interactive shell usage, so only when you are logging in will it attempt to invoke tcsh.

Limits on the number of jobs submitted?

Is there any limit on the number of jobs someone can submit? Isn't submitting too many jobs abusive?

There is no limit to the number of jobs one person can submit. However, this has no effect on how jobs are actually scheduled so doesn't affect when your jobs run. There is some information on this at: http://engin.umich.edu/resources/systems/nyxV2/pbs.html#policy

If you want to see the real state of jobs that are running, waiting, and blocked you can type: showq | less and you'll see the running jobs in order of last-to-finish to first-to-finish, the eligible jobs in order of start priority, and the blocked jobs in the order they were submitted.

Additionally, the more jobs any one person runs, the lower the priority is for their following jobs (this is called "fairshare" and is addressed somewhat at the URL above) so when their jobs do become eligible for scheduling, they are low on the list to start.

Because of all of those controls, the number of jobs any one person submits doesn't affect the other people's jobs on the system and it is often more convenient for people to submit large amounts of jobs at a time and let the system manage them. In fact, the more jobs the scheduler has to work with, the more efficient the system is and the more fair the overall usage is.

C++ runtime abort: internal error: static object marked for destruction more than once

I'm writing C++ MPI code and when I run it I get the error C++ runtime abort: internal error: static object marked for destruction more than once; what do I do?

This is a known problem with C++ code that you can work around by adding the -fPIC option to the end of your compile line, for example:

mpiCC -o myProg.exe mySub1.cpp mySub2.cpp mySub3.cpp -fPIC

SIGILL When running on nyx

When running code in batch on nyx some executables will fail with SIGILL (4).

When code built with the PGI compilers is ran on a machine with older cpus than the one the executable was created on, the compiler may place instructions that are not valid on that cpu. To work around this tell the PGI compilers exactly what (multiple) cpu models to support. List of PGI CPU types

pgf90 -tp x64,amd64e,barcelona-64 source.f90