X

NV5 Geospatial Blog

Each month, NV5 Geospatial posts new blog content across a variety of categories. Browse our latest posts below to learn about important geospatial information or use the search bar to find a specific topic or author. Stay informed of the latest blog posts, events, and technologies by joining our email list!



14932 Rate this article:
4.5

Queuing ESE Tasks in a Loop

Anonym

When using desktop ENVI and IDL, it is useful to setup processing that you have to do many times in a batch script. The easiest way to do this is with an IDL for loop that does processing on one file at a time. However, for an instance running ENVI Services Engine (ESE), it is best to call each task on an individual file. This makes the processing more robust, as an error in processing on a single file will not halt processing for every file. This leaves the question though - how would one loop over every file that needs processing?

One solution is to use IDL to create a loop that goes over each file, then launch every task that needs to be preformed. This builds up a list of tasks queued for execution. To do this, create a list of input files and output files much like you would in a batch process, then call the HTTP address required to submit the processing request to ESE one file at a time.

This can be done using IDL's built in HTTP client, IDLNetURL. As an example, if the ESE process to be called is an asynchronous task named "apply_color_table", the full HTTP call to start the task will be:

http://(host):8181/ese/services/AsyncService/apply_color_table/submitJob?inFile=file&outputFile=file

where (host) is the name or IP address of the server, and the keywords "file" are the actual input and output file names. One way to set up this call so that it occurs on multiple files is as follows, where inFiles is a variable containing all of the files to be processed.

 oURL = Obj_New('IDLnetUrl')

 oUrl.SetProperty, URL_SCHEME='http'

 oUrl.SetProperty, URL_HOST = !SERVER.HOSTNAME

 oUrl.SetProperty, URL_PORT='8181'

 oUrl.SetProperty, $

   URL_PATH='ese/services/AsyncService/apply_color_table/submitJob'

 foreach inFile, inFiles do begin

   oUrl.SetProperty, URL_QUERY='inputFile=' + inFile + $

     '&outputFile=' + 'ct_' + inFile

   result = oURL.Get()

   json = JSON_Parse(result)

   print, 'status file: ' + json['jobStatusURL']

 endforeach

The task that is called will contain the processing, in this case the call that would be made using IDL would be:

apply_color_table, inputFile=inputFile, outputFile=outputFile

This procedure will take in the input file and output file names as arguments, which are passed in through the queuing script.

Once the queuing script completes, ESE will begin running through the tasks one at a time, distributing the workload across CPUs and across any workers that are set up.

Please login or register to post comments.
«May 2025»
SunMonTueWedThuFriSat
27282930123
45678910
11121314151617
18192021222324
25262728293031
1234567
Digital Number, Radiance, and...

Number of views (169543)

Push Broom and Whisk Broom Sensors

Number of views (150805)

The Many Band Combinations of Landsat 8

Number of views (118725)

What Type of Loop Should I Use?

Number of views (78689)

Mapping Coastal Erosion Using LiDAR

Number of views (58475)