This page contains frequently asked questions about the Quest high performance computing cluster at Northwestern.
Allocations And Accounts
How do I get an account on Quest?
To access Quest, you will need to be part of an active research allocation. There are two ways to obtain one:
- Submit a new allocation request form for either a Research I or Research II allocation.
- Join an existing research allocation.
The application forms can be found at Request Research Allocation Forms
The Research Allocation I is suitable for projects requiring 35,000 compute hours or less. This allocation provides a 500 GB project directory. The resources provided by Research I will fit the needs of the majority of our users. A Research II allocation is suitable for projects requiring up to 500,000 compute hours. This allocation provides a 2 TB project storage. Research II allocations are for projects with a large computational need, however you will need to submit a detailed proposal. Both Research I and II allocations are available free of charge, but we do request that a chartstring is provided for your research if possible. This chartstring would be used for internal tracking purposes so that we can see the impact Quest is having on research done on campus.
How do I retain access to Quest after I leave the University?
As long as you have a valid Northwestern NetID/password, you can access to Quest. When your NetID is deactivated by the University, your Quest access will also end. If you want to continue using Quest you should obtain a guest or affiliate NetID. There are individuals in departments/schools who can request guest NetIDs. You or your supervisor can reach out to them.
Please review the following documentation for more information: Affiliate, Departmental, and Guest NetIDs.
How do I use GPUs on Quest?
There are two ways to use GPUs on Quest. You can either purchase your own GPU nodes or use a general access allocation (i.e. Research I, Research II, and Education) to access shared GPU nodes.Please see GPUs on Quest for more information.
How do I transfer files to and from Box on Quest?
The best way to transfer data between Quest storage and OneDrive would be to connect to Quest with the FastX client and start a Gnome desktop session or a Gnome terminal session. Then launch a terminal (or in the terminal that is already launched, if you pick a Gnome terminal session) and type
firefoxto launch the firefox browser. In that browser you can log into OneDrive and transfer files.
Documentation on how to use FastX to connect to Quest is available here: Connecting to Quest with FastX.
You can also use the OneDrive client if you would like to automate your transfers. The details are available here: Transferring Files to and from Quest.
How do I use Amazon AWS on Quest?
The Amazon CLI is a python package install and is installed system wide on Quest. To load this package, run:
module load awscli/2
How do I use the Google Cloud SDK on Quest?
The Google Cloud SDK is installed system-wide on Quest. To load this package, run:
module load gcloud/322.0.0Following the instructions on this page, Google Cloud SDK Quickstart Linux, you can configure the Google Cloud SDK with your credentials, etc.
How do I get access to the Globus endpoint for RDSS (RESFILES/FSMRESFILES)?
To be able to use Globus to transfer data to and from your RDSS (also known as RESFILES or FSMRESFILES), open a service request by emailing firstname.lastname@example.org.
How do I transfer files to and from RDSS (RESFILES/FSMRESFILES) via Globus?
If you lose the ability to connect to a previously mounted resfiles or fsmresfiles share via Globus, you can take the following steps to re-establish a connection:
- SSH to qglobus02.it.northwestern.edu using your NetID as your username. This operation will automatically mount your RDSS share. There is no need to keep the SSH connection open, so you may exit at any time.
- Open https://app.globus.org/file-manager in the browser and log in as a Northwestern user.
- The Quest endpoint name is "Northwestern Quest". the RDSS endpoint (for RESFILES and FSMRESFILES shares) is "Northwestern Quest RDSS".
More information regarding Globus transfers is available here: Globus Transfer FAQ.
How do I share the data on Quest with my collaborators outside Northwestern?
You can use Globus to share data with external collaborators who do not have Northwestern affiliation. Please follow the instructions here: Sharing files with Globus. Login to Globus with Northwestern affiliation. The collection name (i.e. the endpoint) you should select is "Northwestern Quest" to access your files/folders on Quest. You can share data with collaborators who have personal Globus IDs or accounts through their institutions' subscription.
Scheduler and Job Submissions
I get the error "Unable to allocate resources: Requested time limit is invalid (missing or exceeds some limit)" when trying to submit a job.
This error indicates that you have specified your job to run for longer than a given queue will allow. To allow this job to run, you will need to either reduce the amount of walltime for the job to be within the selected queue's limits, or define a larger queue with a higher walltime.
You can find a list of all queues and their walltime limits at Quest Partitions/Queues.
My job was killed on a login node.
From time to time, we encounter errors on the login nodes that require killing all running jobs on that node to prevent the node from crashing. Unfortunately, your job may have been one of those that were killed.
It is recommended that users submit interactive or batch jobs to the compute nodes to avoid such job cancellations. Login nodes are shared resources are intended as entry points to Quest for all users. Submitting CPU or memory heavy jobs will affect everyone trying to access Quest. Please see Submitting a Job On Quest for more information about submitting interactive jobs.
I get the error "sbatch: error: Batch job submission failed: Invalid qos specification" when trying to submit a job.
This error is commonly observed if your allocation has expired. Slurm does not allow job submission if you are using an expired allocation. You can run the command checkproject <allocation ID> to see the expiration date of your allocation. If your research project continues and you want to continue using the same allocation, you will need to renew it by Requesting a Research Allocation
The software I want to use is not available on Quest.
If the software you require for your research is not available on Quest, there are a few options you can try. The first is to perform a local software installation to your home directory or project directory following instructions specific to the software you are trying to install. If a local install does not work, you can fill out the Software Installation Request Form. We will then assist you with the software installation.
How can I install and use Tensorflow with GPUs on Quest?
Please see the What GPU software is available on QUEST? section of GPUs on Quest for more information.