Quest Troubleshooting: Insufficient Hours or Expired Allocation

This page demonstrates troubleshooting jobs that may have been cancelled due to an expired allocation or insufficient hours remaining on an allocation. Use the checkproject utility to check allocation resources.

If your job is cancelled by the scheduler or is listed with a STATE of SystemHold or Canceling in the "blocked jobs" section of the showq output, you may have submitted a job requesting more compute hours than are available in your allocation. This may be because your allocation is running low on hours, or it has expired.

For an expired allocation, at the end of the checkjob output (use the -v option with checkjob), you will see information like:

Job Messages -----
Label   CreateTime ExpireTime    Owner Prio Num Message
0         -00:02:04   23:57:55      N/A    0   5 Failure registering job Start (19935860) with accounting manager -- 
Unable to invoke AM request - server rejected request with status code 740 - Insufficient funds: There are no valid 
allocations against which to make the lien (Instance 19935860)
1         -00:00:52   23:59:08      N/A    0   1 job cancelled - job was rejected​

In the case where the allocation hasn't expired, but there are insufficient compute hours as requested for the job, the end of the checkjob output (again, use the -v option with checkjob) will look like:

Message[0] server rejected request with status code 782 - Insufficient balance to reserve job
Message[1] cannot debit job account - no funds

Use checkproject to check your allocation resources and expiration.

If you are out of compute hours or your allocation has expired, you can request an allocation extension or renewal. See Types of Allocations and Application Submission Guidelines for more information.

See Also:

Keywords:research computing, quest, troubleshooting, systemhold, allocation, "status code 782", "Insufficient balance"   Doc ID:78617
Owner:Research Computing .Group:Northwestern
Created:2017-12-07 17:06 CDTUpdated:2019-02-04 11:59 CDT
Feedback:  0   0