Note: I am using ENVI Classic and IDL
I'm looking for some tips on the best way to complete some spatial processing on very large single band images.
I'm investigating using ENVI's tiling routines to accomplish this, but have some issues.
One example given in ENVI's help uses the tiling routines to process an image and save it in memory. It is something along the lines of:
envi_file_query, fid, ns=ns, nl=nl, data_type=data_type
pos=[0]
mem_res = make_array(ns, nl, type=data_type)
tile_id = envi_init_tile(fid, pos, num_tiles=num_tiles)
for i = 0L, num_tiles-1 do begin
data = envi_get_tile(tile_id, i, ys=ys)
; process the data here
mem_res[0,ys] = data
endfor
envi_enter_data, data, ....
This method is not working for me because the image that I am using is ~30k x 30k pixels. So when I try to make a blank array of this size using make_array(), it error out due to not enough memory. Is there a way to allocate more memory so that arrays this large can be created? This is the hangup I face with this method.
The other method would be to write out the data to a file as it is being processed. Something like this:
envi_file_query, fid, ns=ns, nl=nl, data_type=data_type
pos=[0]
openw, unit, outname, /get_lun
tile_id = envi_init_tile(fid, pos, num_tiles=num_tiles)
for i = 0L, num_tiles-1 do begin
data = envi_get_tile(tile_id, i, ys=ys)
; process the data here
writeu, unit, data
endfor
free_lun, unit
envi_setup_head...
This method would solve my memory issue. However, it is too slow for such a large image. To increase the processing speed I am thinking about using multiple IDL bridges to process multiple tiles at the same time. However, if I'm writing the file out, the parallel processing wouldn't work due to having to write out the data in the correct order.
Any thoughts?
|