Job Dependencies

PBS allows you to specify dependencies between two or more jobs. Dependencies are useful for a variety of tasks, such as:

  • Specifying the order in which jobs in a set should execute
  • Requesting a job run only if an error occurs in another job
  • Holding jobs until a particular job starts or completes execution

Syntax for Job dependencies

Use the “-W depend=<dependency list>” option to qsub to define dependencies between jobs.

The dependency list has the format: <type>:<arg list>

where except for the on type, the arg list is one or more PBS job IDs in the form: <job ID>[:<job ID> ...]

The depend job attribute controls job dependencies. You can set it using the qsub command line or a PBS directive:

qsub -W depend=...

#PBS -W depend=...

These are the available dependency types:

          after: <arg list>

          This job may start only after all jobs in arg list have started execution.

          afterok: <arg list>

         This job may start only after all jobs in arg list have terminated with no errors.

          afternotok: <arg list>

          This job may start only after all jobs in arg list have terminated with errors.

          afterany: <arg list>

          This job may start after all jobs in arg list have finished execution, with or without errors. This job will not run if a job inthe arg list was deleted without ever having been run.

          before: <arg list>

          Jobs in arg list may start only after specified jobs have begun execution. You must submit jobs that will run before other jobs with a type of on.

          beforeok: <arg list>

          Jobs in arg list may start only after this job terminates without errors.

          beforenotok: <arg list>

          If this job terminates execution with errors, the jobs in arg list may begin.

          beforeany: <arg list>

          Jobs in arg list may start only after specified jobs terminate execution, with or without errors. Requires use of on dependency for jobs that will run before other jobs.

          on:count

          This job may start only after count dependencies on other jobs have been satisfied. This type is used in conjunction with one of the before types. count is an integer greater than 0.

To determine whether a job has run with errors, PBS will use the exit status of the last command executed as the exit status of the job.

Job dependency examples

Example 1

You have three jobs, job1, job2, and job3, and you want job3 to start after job1 and job2 have ended:

qsub job1
    16394.pbs
 
qsub job2
    16395.pbs
 
qsub -W depend=afterany:16394:16395 job3
    16396.pbs

Example 2

You want job2 to start only if job1 ends with no errors:

qsub job1
    16397.qsub
 
qsub -W depend=afterok:16397 job2
    16396.pbs

Example 3

job1 should run before job2 and job3. To use the before any dependency, you must use the on dependency:

qsub -W depend=on:2 job1
    16397.pbs
 
qsub -W depend=beforeany:16397 job2
    16398.pbs
 
qsub -W depend=beforeany:16397 job3
    16399.pbs

 

Job Array Dependencies:

Job dependencies are supported:

  • Between jobs and jobs
  • Between job arrays and job arrays
  • Between job arrays and jobs
  • Between jobs and job arrays

Job dependencies are not supported for subjobs or ranges of subjobs.

Subjobs in arrays run completely independent of each other and in no particular order.