Quest Troubleshooting: Insufficient Hours or Expired Allocation
This page demonstrates troubleshooting jobs that may have been cancelled due to an expired allocation or insufficient hours remaining on an allocation. Use the checkproject utility to check allocation resources.
If your job is cancelled by the scheduler or is listed with a STATE of SystemHold or Canceling in the "blocked jobs" section of the showq output, you may have submitted a job requesting more compute hours than are available in your allocation. This may be because your allocation is running low on hours, or it has expired.
For an expired allocation, at the end of the checkjob output (use the -v option with checkjob), you will see information like:
Job Messages ----- Label CreateTime ExpireTime Owner Prio Num Message 0 -00:02:04 23:57:55 N/A 0 5 Failure registering job Start (19935860) with accounting manager -- Unable to invoke AM request - server rejected request with status code 740 - Insufficient funds: There are no valid allocations against which to make the lien (Instance 19935860) 1 -00:00:52 23:59:08 N/A 0 1 job cancelled - job was rejected
In the case where the allocation hasn't expired, but there are insufficient compute hours as requested for the job, the end of the checkjob output (again, use the -v option with checkjob) will look like:
Message server rejected request with status code 782 - Insufficient balance to reserve job Message cannot debit job account - no funds
Use checkproject to check your allocation resources and expiration.
If you are out of compute hours or your allocation has expired, you can request an allocation extension or renewal. See Types of Allocations and Application Submission Guidelines for more information.