Getting Started on the Genomics Compute Cluster (b1042) on Quest

After applying for access to the genomics compute cluster, and receiving an orientation on using the genomics compute cluster, users can submit jobs to the compute nodes.

About the Genomics Compute Cluster on Quest 

The Feinberg School of Medicine, in an initiative spearheaded by the Feinberg School of Medicine Office of the Dean, the Center for Genetic Medicine, and the Department of Biochemistry and Molecular Genetics provides 100 nodes on Northwestern’s High Performance Computing Cluster, Quest, to be used for genomics research. This University resource has been made available to the greater genomics community in an effort to foster genomics research and empower the computational genomics community at Northwestern University.

How do I start using the Genomics Compute Cluster?

You will first need to log in to Quest.

Moving files onto Quest

If you have sequencing done by NUSeq Core, they will deliver your FASTQ files via Northwestern Box. To get files from Box onto Quest:

  1. Log in to Quest. Northwestern IT recommends you use FastX to connect, but experienced users may opt for an SSH connection with X11 forwarding enabled.
  2. Run Firefox by typing the following commands in the terminal window: module load firefox, followed by firefox to launch the Firefox browser. A Firefox browser window will open.
  3. In the Firefox browser that opened, go to https://northwestern.box.com and log in with your NetID credentials.
  4. Select the file you wish to move onto Quest and click Download in the upper right corner. By default, the file will be placed into the Downloads directory in your home directory.

Note: If you want the file to go directly into /projects/b1042, before downloading click on the menu icon in the upper right of the Box browser, and click Preferences. From there you can select which directory to download into on Quest, including the /projects directories.

For information about transferring files from your local machine, RDSS (via smbclient), or Globus to Quest please visit the Transferring Files on Quest web page.

Using bioinformatics software available on Quest

To see a complete list of software installed on Quest by the Quest administrators, type module avail. Modules are scripts that make it possible to run software in your shell; if you would like to run a particular software package, use the module load command, and then run the software.

If you would like to use additional software packages not currently installed on Quest, please see Installing Software on Quest.

Running jobs on the Genomics Compute Cluster: Submission Scripts

To submit, monitor, modify, and delete jobs on Quest you must use Slurm commands as outlined in the Quest Slurm Quick Start Guide.

Genomics Compute Cluster Partitions

Partitions are defined for different types of jobs and have different resource limits.  Note that Partitions are also called queues.
  • genomics: For jobs that require less than 48 hours to run. Individual jobs in the genomics partition must run on fewer than 5 nodes. Individual users may submit multiple jobs to the genomics partition at one time and utilize many cores simultaneously.
    Script example: #SBATCH -p genomics
  • genomicsguest: a lower-priority version of the genomics partition used by non-Feinberg users.
    Script example: #SBATCH -p genomicsguest
  • genomicslong: For jobs that require between 48 - 240 hours to run.  Priority is lower on this partition than the genomics partition to reserve most of the resource for shorter jobs.
    Script example: #SBATCH -p genomicslong
  • genomicsguestex: a lower-priority version of the genomicslong partition for non-Feinberg users.
    Script example: #SBATCH -p genomicsguestex
  • genomics-burst: For special projects, large jobs requiring more than 5 nodes or that run for more than 48 hours may use the genomics-burst partition by contacting quest-help@northwestern.edu. Before using genomics-burst partition, users must meet with a Research Computing Services consultant to confirm that code has been reviewed for efficiency. These appointments will be set up after reservations are requested and are intended to ensure best practices for this shared resource. Because of the resources involved, jobs may need to wait for availability when requesting the genomics-burst partition. It is advised to schedule at least three (3) weeks in advance.

For more example job scripts see the GitHub repository of example jobs.

See Also:




Keywords:genomics, quest, research computing, b1042, bioinformatics, genetics, bwa, rna-seq, gatk   Doc ID:78602
Owner:Research Computing .Group:Northwestern
Created:2017-12-07 14:38 CDTUpdated:2019-04-23 15:28 CDT
Sites:Northwestern
CleanURL:https://kb.northwestern.edu/genomics-compute-cluster
Feedback:  0   0