Transitioning from the SSCC to Quest

Quest and the SSCC are similar systems, but there are some differences that those moving from the SSCC to Quest should be aware of.


NOTICE

The SSCC has been retired.  You can no longer connect to it.  If you need to retrieve files from the SSCC, please make a request by emailing quest-help@northwestern.edu with your netid and where you'd like the files transferred to.


Infrastructure

Like the SSCC, Quest consists of login nodes that you connect to directly, and compute nodes that run batch or interactive jobs submitted to the job scheduler. Quest also has Analytics Nodes that allow you to use RStudio and SAS Studio via a web browser.

Unlike the SSCC, the compute nodes are a mix of general access nodes that support all researchers, and full access nodes that have been purchased by individual researchers for their own use. The compute nodes on Quest have more cores (CPUs) and more memory (RAM) than the compute nodes on the SSCC. See Quest Technical Specifications for more information.

Like the SSCC's seldon, hardin, and dornick, Quest has several login nodes. You can connect to them individually if needed, but usually you will connect to quest.northwestern.edu and then be redirected automatically to one of the login nodes, which are named quser10, quser11, quser12, and quser13.

You use your NetID and NetID password to log in to Quest, although you need to request a Quest account before you can log in.


Accounts and Allocations

SSCC accounts did not expire. In contrast, accounts on Quest are associated with an allocation, which is active for 1 year. A Quest allocation provides you with access to Quest, as well as a specific amount of storage and compute hours (for submitting jobs) on Quest. Quest allocations can be renewed yearly or extended.

An allocation provides you with compute hours to submit jobs on Quest as well as file storage. Users can be associated with more than one allocation at a time. Multiple users can be associated with a single allocation; this is how multiple collaborators work on a project together on Quest.

A compute hour is the use of one CPU/core for one hour. Unlike the SSCC, Quest keeps track of how many compute hours your jobs use. The compute hours you use are deducted from your allocation.  Your allocation must have sufficient compute hours to submit a job.

Types of Quest Allocations

There are two types of Quest general access allocations: Research Allocation I and Research Allocation II. The Research Allocation I is sized to meet the computational and storage needs of most SSCC users and only requires a short paragraph-length description of your research. Those users needing more than 35,000 compute hours or 500GB of storage should apply for a Research Allocation II to request these additional resources. The application for a Research Allocation II requires more information about your research and justification for the amount of hours and storage you've requested. Research Allocation II requests are typically approved quarterly, but applications from SSCC users will be approved on a rolling basis during Summer 2017.

You can apply for a Quest allocation at: HPC Allocation Request Form.

NOTE: If you have questions about compute hours or which allocation type you might need, please contact quest-help@northwestern.edu


File Storage and Transfer

Each Quest user has an 80GB home directory (/home/<netid>). Files in your home directory cannot be shared with others. Home directories are backed up for disaster recovery; users do not have direct access to multiple backup versions of files like was possible on the SSCC.

In addition, each allocation has a project directory that is accessible to everyone associated with the allocation (/projects/<allocationID>). The size of the project directory is determined by the type of allocation (Research I, Research II, or purchased storage). Project storage is not backed up.

Job submission scripts, code files, and smaller output files are appropriate for storage in your home directory. Large data files should be stored in the projects space, as it is optimized for data transfer to and from the compute nodes. For more information, see File Systems and Storage on Quest.

You can transfer files to and from Quest and your personal computer using an SFTP client as you could on the SSCC. See the SFTP section of Transferring Files to and from Quest for connection details.


Software

Unlike the SSCC, nearly all software on Quest is available through modules and must be loaded before use. In some cases a system library or package may be available without first loading a module, such as with Python, but it is not a good idea to use these default versions. Use a module to load software instead. For example:

module load python/anaconda3.6

Software Availability Differences

Mplus was available on the SSCC, but it is not available on Quest. Mplus is available on NUworkspace.  If you need to use Mplus with more computational resources than are available on your personal computer or NUworkspace, please contact quest-help@northwestern.edu for options.

Nearly all other software and programming languages available on the SSCC are also available on Quest. If you find that a program you need is missing, please let us know.


Login Nodes/Interactive Jobs

Unlike the SSCC, you should not run jobs directly on the Quest login nodes. You may run small jobs for quickly testing a script or installing packages. Any jobs that are longer than an hour, use more than a single core, or use more than a few GB of RAM should instead be run as an interactive job on a Quest compute node. You issue a command to the system scheduler to request resources from a compute node to use. See Running Interactive Jobs on Quest.


Batch Jobs

Batch jobs are submitted to compute nodes on Quest in a similar manner to the SSCC. However the job scheduler is a slightly different system, so the job options have slightly different names. Note that the "Submit" utility available on the SSCC is not available on Quest; you will have to write your job submission script yourself. For more information beyond what's below, see Submitting a Job on Quest and Example Jobs.

Accounting

As mentioned above, unlike the SSCC, Quest keeps track of the number of compute hours you use.  A compute hour is one hour use of one CPU/core.  Quest deducts the compute hours used by your job from your allocation.  If you request more walltime for your job than it uses, you're only charged for the time that your job actually ran.  You will always be charged for the full number of cores you request – regardless of your utilization of them – as you have a lock on those resources while your job is running.

Queues

Unlike the SSCC, Quest has multiple queues for jobs. The general access queues are distinguished by the amount of time your job will take to run: less than 4 hours, less than 48 hours (2 days), or less than 7 days. When possible, keep your job times to 4 hours or less, as there are many more compute nodes that can run these short jobs than there are nodes that can run longer jobs.

Jobs on Quest are limited to a run time of 7 days. If you have jobs that need to run longer than 7 days, contact quest-help@northwestern.edu to discuss options.

Job Submission Script Mapping

Most PBS options in SSCC job submission scripts become MSUB options in Quest job submission scripts. Below are some examples of how options or application lines from an SSCC script might look in a Quest script.

SSCCQuestNote
#!/bin/bash#!/bin/bashIndicate it is a bash script
#PBS -j oe#MSUB -j oeJoin output and error streams
#PBS -l walltime=08:00:00#MSUB -l walltime=08:00:00Set job runtime to 8 hours
#PBS -l mem=2gb#MSUB -l mem=2gbRequest 2GB of RAM; note that on Quest, there is more memory per core on the compute nodes. If you need less than 4GB of RAM per core, you can omit this option.
#PBS -l ncpus=1#MSUB -l procs=1
or
#MSUB -l nodes=1:ppn=1
Request 1 core

#MSUB -A <allocation ID> On Quest, you should specify your allocation ID
#PBS -q A#MSUB -q <queuename> On Quest, you should specify the queue name based on the walltime
#PBS -m ae#MSUB -m aeWhen to send email notifications
#PBS -M <email>#MSUB -M <email>Where to send email notifications
#PBS -l select=1:mem=2gb:ncpus=4#MSUB -l nodes=1:ppn=4,mem=2gbRequest 4 cores on a single node and 2GB of RAM total

module load <modulename>On Quest, make sure to load the module for your software in your script before the application line. After the MSUB options, leave a blank line then load modules, change the working directory, and call your script.

On the SSCC, you would submit your script with the qsub command. On Quest, you use msub to submit a job. Note that the commands for Managing Jobs are also different.


Other Options

Researchers who used the SSCC may also be interested in NUworkspace, which offers a virtual Windows desktop with software for quantitative and qualitative research.

Getting More Help

This page only provides an overview of differences between the systems. You can find more details in the Quest documentation. See the Quest User Guide for links to Quest help pages. For more help with Quest, or to set up an individual consultation, email quest-help@northwestern.edu.




Keywords:sscc, quest, research computing, pbs, msub   Doc ID:70708
Owner:Research Computing .Group:Northwestern
Created:2017-02-15 16:21 CDTUpdated:2018-05-02 16:31 CDT
Sites:Northwestern
CleanURL:https://kb.northwestern.edu/sscc-to-quest
Feedback:  0   0