Using IDL_IDLBridge to run multiple processes in parallel
Anonym
I noticed a question on the newsgroup about running multiple time consuming IDL commands in parallel processes. This is a topic that we in the Custom Solutions group come accross fairly frequently. Below is a simple example of how to manage a number of processes running at the same time.
Notice in the code, that it is often important to control the working directory as well as the search path for all of the bridge processes so that the bridge processes can find the necessary IDL code as well as any data needed.
pro multi_process
compile_opt idl2, logical_predicate
cd, current=dir
n_proc = 10
tlb = widget_base(title='Initializing',/column, xsize=500, ysize=500)
t = widget_text(tlb, value='Initializing...')
widget_control, tlb, /realize
br = objarr(n_proc)
for i=0, n_proc-1 do begin
br[i] = IDL_IDLbridge()
br[i]->SetVar,'dir',dir
br[i]->Execute,'cd,dir'
br[i]->SetVar,'!path',!path
br[i]->Execute,'wait,50*randomu(seed)',/nowait
endfor
widget_control, tlb, /destroy
tlb = widget_base(title='Processing progress',/column)
t = widget_table(tlb, xsize=2, ysize=n_proc, value=strarr(2,n_proc), $
column_labels=['Process #','Status'], column_width=[250,250])
widget_control, tlb, /realize
; Monitor progress
status = replicate(1, n_proc)
table = strarr(2,n_proc)
table[0,*] = strtrim(indgen(n_proc),2)
codes = ['Idle','Executing','Completed','Error','Aborted']
while max(status) gt 0 do begin
wait, 1
for i=0, n_proc-1 do begin
status[i] = br[i]->Status()
endfor
table[1,*] = codes[status]
widget_control, t, set_value=table
endwhile
widget_control, tlb, /destroy
end
This code example will display a simple GUI showing the status of the IDL processes.
This is a great approach if you want to speed up some time consuming IDL processes that can run individually on the same system. In more complex situations, you want to instead look into ENVI Services Engine as it provides much more functionality for configuring how "jobs" are being run, and also supports distributing jobs over multiple systems (nodes). Despite the word "ENVI" here, it also supports running pure IDL code.