The IDLAsyncSpawnTaskJob class encapsulates the combined behaviors of its base classes IDLAsyncSpawnJob and IDLTaskJob. It allows the user to specify a single IDLTask that will executed by the TaskEngine when there are available resources. The IDLTask must have all of its input parameters specified before creating this job, as it will be cloned and dehydrated for passage to IDLTaskEngine via STDIN. Any subsequent changes to the task's parameter values will not be reflected when this job executes the task in the IDLTaskEngine. When the IDLTaskEngine is spawned, STDERR will be redirected to a temporary file that is read on completion and stored in the STDERR property, allowing access to the task error messages when the job fails.

Superclasses

Examples

; Use S3 folder for a dataset covering Boulder, CO, taken April 4, 2017
folderUrl = 'https://s3-us-west-2.amazonaws.com/landsat-pds/L8/033/032/LC80330322017104LGN00'
; Dataset name is same as folder name, and is root of all files
datasetName = File_BaseName(folderUrl)
 
; List of suffixes to apply to dataset name for all files needed to
; comprise a full Landsat 8 dataset.  Band 8 is the panchromatic band,
; which is the largest, so start downloading it first.
suffixes = [ '_B8.TIF', $
             '_B1.TIF', $
             '_B2.TIF', $
             '_B3.TIF', $
             '_B4.TIF', $
             '_B5.TIF', $
             '_B6.TIF', $
             '_B7.TIF', $
             '_B9.TIF', $
             '_B10.TIF', $
             '_B11.TIF', $
             '_BQA.TIF', $
             '_MTL.txt' ]
 
; create a folder in the temp folder for the local files
localFolder = FilePath(datasetName, /TMP)
if (~File_Test(localFolder)) then File_MkDir, localFolder
 
; construct IDLAsyncJoin object to use for waiting on all jobs to complete
oJoin = IDLAsyncJoin()
; construct IDLAsyncQueue to manage parallel execution of jobs
oQueue = IDLAsyncQueue(CONCURRENCY=6)
; create object array to hold all jobs
jobs = ObjArr(N_Elements(suffixes))
; create task to be used by the jobs
task = IDLTask('Download_S3_URL')
 
; construct a unique job for each remote file to download
foreach suf, suffixes, i do begin
    ; update task parameters to current file suffix
    file = datasetName + suf
    task.S3_URL = folderUrl + '/' + file
    task.LOCAL_FILE = FilePath(file, ROOT=localFolder)
    ; Construct IDLAsyncSpawnTaskJob that clones the task, passing in the join object.
    ; Set the COMPILE keyword since the task uses PRO code not a save file.
    jobs[i] = IDLAsyncSpawnTaskJob(task, JOIN=oJoin, /COMPILE)
endforeach
 
tic
; submit all the jobs to the queue for execution
oQueue.SubmitJob, jobs
; wait for all jobs to be done
oJoin.WaitForJoin
toc

Syntax

Result = IDLAsyncSpawnTaskJob(Task [, /COMPILE] [, JOIN=IDLAsyncJoin] [, WAIT=Float])

Arguments

Task

A scalar IDLTask object owned by this job, which will be executed inside IDLTaskEngine when Start is called.

Keywords

COMPILE (optional)

Specify an optional Boolean flag that indicates whether the "--compile" flag should be added to the IDLTaskEngine invocation. Use this if your task uses PRO code instead of save files.

JOIN (optional)

Specify an IDLAsyncJoin object that is passed to the IDLAsyncJob base class Init() method for handling.

WAIT (optional)

Specify a value in seconds to pass to IDLAsyncSpanJob::Init for handling.

Methods

Properties

IDLAsyncBridgeTaskJob inherits all properties from IDLAsyncSpawnJob and IDLTaskJob.

STDERR

A List containing all the lines of text the spawned external process wrote to STDERR.

IDLAsyncSpawnTaskJob::GetReturnValue

The IDLAsyncSpawnTaskJob::GetReturnValue method is an override of the IDLAsyncSpawnJob method. This override adds access to STDERR, which is a List of the strings read from a temporary file that STDERR was redirected into when the IDLTaskEngine was spawned.

Syntax

IDLAsyncSpawnJob.GetReturnValue, STDERR=list [, Variables=Value]

Arguments

None

Keywords

STDERR

The contents read from STDERR from the external application. This is stored as a list of strings, to allow users to insert carriage returns between lines as needed.

Variables

Named variables accessible from the base class IDLAsyncSpawnJob.

IDLAsyncSpawnTaskJob::OnDone

The IDLAsyncBridgeTaskJob::OnDone method is an implementation override of the IDLAsyncJob callback invoked on this job by the Done event handler. The basic sequence of operations follows:

  • Call IDLAsyncSpawnJob::OnDone to make sure the pipe is closed, exit status retrieved.
  • Check if the temporary file into which STDERR was redirected exists and has any text.
  • Read that text into STDERR property if it exists and delete the temporary file.
  • Call GetReturnValue to get serialized output parameters.
  • Rehydrate the output parameter values and update the TASK property.

Syntax

IDLAsyncSpawnTaskJob.OnDone

Arguments

None

Keywords

None

IDLAsyncSpawnTaskJob::OnStart

The IDLAsyncSpawnTaskJob::OnStart method is an implementation override of the IDLAsyncSpawnJob callback invoked on this job by the Start event handler. This override ensures that the IDL_PATH environment variable is set, so that the Task Engine process uses the same preference value as the main IDL session. The basic sequence of operations is:

  • Cache current IDL_PATH environment variable.
  • Set IDL_PATH environment variable to value of IDL_PATH preference.
  • Call IDLAsyncSpawnJob::OnStart to spawn the Task Engine.
  • Restore the IDL_PATH environment variable to original value

Note: If you need to update the IDL path to include your code, use PREF_SET for IDL_PATH instead of modifying !PATH directly, otherwise the Task Engine will not inherit those changes.

Syntax

IDLAsyncSpawnJob.OnStart

Arguments

None

Keywords

None

Version History

8.7

Introduced

See Also

IDLAsyncSpawnJob, IDLAsyncJob, IDLAsyncJoin, IDLTaskJob