19 Oct 2009 11:12 AM |
|
Hi,
Having some trouble with ENVI calls that I'm making from an IDL procedure. I am attempting to call ENVI_DOIT 'ENVI_GRID_DOIT' many times (> 200) in order to grid a series of irregular points data files. For reasons that aren't clear to me, ENVI_GRID_DO_IT will only work this way for 28 files/iterations. On the 29th, the value of r_fid is -1, which indicates an I/O error. A subsequent IDL call to OPENR in another procedure (but within the same overall IDL session) returns ''OPENR: All available logical units are currently in use." It appears that ENVI is grabbing LUNs and not returning them to the pool. It also might be that there's some fid limit that I'm exceeding. Either way, I can't seem to figure this out based on the available documentation.
Has anyone out there seen something like this before? I don't think that this is a particularly strange application of ENVI or IDL, so I'm optimistic that there is an easy solution or workaround (and processing the images individually is not an easy workaround in this case because I'm accumulating some stats on the collective set of data). Thanks for any advice in advance...
|
|
|
|
MariM Veteran Member
Posts:2396  
19 Oct 2009 11:29 AM |
|
ENVI does not use LUN's, it uses FIDs and they are not the same. You cannot pass a LUN as a FID to an ENVI routine. You can get a FID by using ENVI_OPEN_FILE or similar routine, or if you use ENVI_SETUP_HEAD to create a new ENVI file header. It sounds like you are opening files using IDL's OPENR which would require a GET_LUN and then the LUN is not being freed later. Freeing the LUN would not be done at the end of an ENVI routine, which would instead return a FID, so you have to handle this yourself, usually before you pass your file to ENVI. Is there some reason you are using OPENR rather than ENVI's file open routines? Does ENVI not recognize the file type you are opening? If so, you may need to use ENVI_SETUP_HEAD to create an associated header for the binary datafile which would return a FID. This FID could be passed to ENVI_GRID_DOIT and it does not need to be freed later on.
|
|
|
|
Deleted User New Member
Posts:  
19 Oct 2009 12:28 PM |
|
Thank you for your response. My original post may have been confusing because I mentioned both IDL and ENVI calls. I am aware that ENVI does not use LUNs. I am not passing LUNs to ENVI or FIDs to IDL. My code is almost exclusively made up of IDL calls, and I only use ENVI for a very small percentage of what I'm doing. My point was that the IDL session from which I'd called ENVI had run out of LUNs, causing OPENR to fail with that error message.
The IDL calls do not appear to be the problem. I removed all ENVI calls from my code, and it worked correctly (without gridding of course) for hundreds of files. I next began to reinsert the ENVI calls, and as soon as I put the ENVI_GRID_DOIT call back into the procedure, the code bombed on the 29th file. As a quick debugging diagnostic, I printed the returned FID to the terminal immediately following each call to ENVI_GRID_DOIT. Its value starts at 2 and increments by one on every iteration of the code until it reaches 29 (this is the 28th file). On the next iteration/file, the value of the returned FID is -1, and then the code dies when this invalid FID gets used by a subsequent call to ENVI_GET_DATA.
So, I'm pretty sure that the problem lies within ENVI somewhere. Is there some kind of ENVI debugger that can be used to see what's going on inside those routines? The debugging mode of IDL Workbench can't see into the ENVI routines, which seems like a poor design to me. After all, both products are the property of ITT! Thanks again for your response...
|
|
|
|
Deleted User New Member
Posts:  
19 Oct 2009 12:28 PM |
|
oops, ignore dbl post.
|
|
|
|
MariM Veteran Member
Posts:2396  
19 Oct 2009 12:32 PM |
|
So how are you passing data into ENVI_GET_DATA? You can have a long integer number of FIDs, so you are not running out of FIDs. Can you post a small example code that exhibits this behavior? All the ENVI routines are compiled sav files, so you cannot view the source code.
You may just want to go through Technical Support to work on this issue.
|
|
|
|
Deleted User New Member
Posts:  
19 Oct 2009 01:28 PM |
|
Below is a stripped-down piece of code that demonstrates the behavior that I'm seeing. Sorry if it's a little too much for posting, but I wanted to make sure that I was replicating the situation. Compile it in IDL and run as:
print,TEST_ENVI_FIDS(29)
The minimum number of iterations that is required to cause ENVI_GRID_DOIT to fail is 29, and only takes about 30 seconds. The screen output in IDL shows that the ENVI routine is failing on the 29th call because it returns an FID of -1. If you execute it twice in the same IDL session with arguments that add up to 29 (say, 15 on first run and 14 on second run), it will fail on the second run's last call to ENVI_GRID_DOIT.
I hope something jumps out at you in this code listing as being an obvious mistake, because I'm really scratching my head over this. Your advice has been helpful, and I appreciate any more that you can provide. Thanks!
forward_function ENVI_PROJ_CREATE,ENVI_GET_DATA,ENVI_GET_MAP_INFO
;
;
FUNCTION TEST_ENVI_FIDS, num_iterations
;
;
; Loop "num_iterations" times on ENVI_GRID_DOIT
FOR l = 1, num_iterations DO BEGIN
;
;
; First restore the ENVI base save files
envi, /restore_base_save_files
; Initialize ENVI and send all errors and warnings to file
envi_batch_init, log_file = 'batch_log.txt'
; Make some fake data to test the gridding functionality
outpointdata_x = [-75.0,-76.0,-77.0]
outpointdata_y = [37.0,38.0,39.0]
outpointdata_z = [10.0,20.0,30.0]
; Set input and output projection info
iproj_name = 'My Geographic'
iproj_type = 1
oproj_name = 'My LAZEA'
oproj_params = [6370997.0, 38.000000, -76.000000, 0.0, 0.0]
oproj_type = 36 ; LAMBERT AZIMUTHAL EQUAL AREA
; Create the input and output projections based on the info
my_latlon_proj = ENVI_PROJ_CREATE(/GEOGRAPHIC,name=iproj_name,type=iproj_type)
my_lazea_proj = ENVI_PROJ_CREATE(name=oproj_name,params=oproj_params,type=oproj_type)
; Call envi_grid_doit to rasterize (grid) the point data.
; (first set some argument values)
out_dt = 4 ; floating point
pixel_size = [50.,50.] ; 50 m X 50 m
ENVI_DOIT, 'ENVI_GRID_DOIT', /IN_MEMORY, $
x_pts = outpointdata_x, y_pts = outpointdata_y, $
z_pts = outpointdata_z, i_proj = my_latlon_proj, $
out_dt = out_dt, $
pixel_size = pixel_size, o_proj = my_lazea_proj, $
interp = 0, r_fid = initial_gridded_file
print,"RETURNED FID",initial_gridded_file
print,"FILE/ITERATION", l
ENDFOR
END
|
|
|
|
MariM Veteran Member
Posts:2396  
19 Oct 2009 02:10 PM |
|
I am not sure why your code didn't post but I noticed that you have your RESTORE and BATCH_INIT inside your loop. I actually think it is the batch init causing the problem because I believe a LUN is used to open the log text file. There is no reason why these should be inside the loop, so moving them up before the FOR statement allowed me to run your code up to 35 times with a FID returned:
RETURNED FID 33
FILE/ITERATION 32
RETURNED FID 34
FILE/ITERATION 33
RETURNED FID 35
FILE/ITERATION 34
RETURNED FID 36
FILE/ITERATION 35
0
==========================
forward_function ENVI_PROJ_CREATE,ENVI_GET_DATA,ENVI_GET_MAP_INFO
;
;
FUNCTION TEST_ENVI_FIDS, num_iterations
;
;
; Loop "num_iterations" times on ENVI_GRID_DOIT
FOR l = 1, num_iterations DO BEGIN
;
;
; First restore the ENVI base save files
envi, /restore_base_save_files
; Initialize ENVI and send all errors and warnings to file
envi_batch_init, log_file = 'batch_log.txt'
; Make some fake data to test the gridding functionality
outpointdata_x = [-75.0,-76.0,-77.0]
outpointdata_y = [37.0,38.0,39.0]
outpointdata_z = [10.0,20.0,30.0]
; Set input and output projection info
iproj_name = 'My Geographic'
iproj_type = 1
oproj_name = 'My LAZEA'
oproj_params = [6370997.0, 38.000000, -76.000000, 0.0, 0.0]
oproj_type = 36 ; LAMBERT AZIMUTHAL EQUAL AREA
; Create the input and output projections based on the info
my_latlon_proj = ENVI_PROJ_CREATE(/GEOGRAPHIC,name=iproj_name,type=iproj_type)
my_lazea_proj = ENVI_PROJ_CREATE(name=oproj_name,params=oproj_params,type=oproj_type)
; Call envi_grid_doit to rasterize (grid) the point data.
; (first set some argument values)
out_dt = 4 ; floating point
pixel_size = [50.,50.] ; 50 m X 50 m
ENVI_DOIT, 'ENVI_GRID_DOIT', /IN_MEMORY, $
x_pts = outpointdata_x, y_pts = outpointdata_y, $
z_pts = outpointdata_z, i_proj = my_latlon_proj, $
out_dt = out_dt, $
pixel_size = pixel_size, o_proj = my_lazea_proj, $
interp = 0, r_fid = initial_gridded_file
print,"RETURNED FID",initial_gridded_file
print,"FILE/ITERATION", l
ENDFOR
END
|
|
|
|
Deleted User New Member
Posts:  
19 Oct 2009 02:40 PM |
|
Hey, that fixed it! I was wondering about those statements. The reason why I had them in the loop in the sample piece of code was to replicate their (repeated) execution whenever the real piece of code was called from a batch file. So all I have to do is pull those statements out of the original function and include them at the top of the batch file that calls the function.
This betrays my relative lack of experience with ENVI's batch mode. I'm much better acquainted with the way IDL works. Thanks so much for your help -- I'd already begun to think about how to code the ugly throwaway workaround I was going to have to write if this problem couldn't be solved. Can't think of a better way to begin the work week than with a coding victory...
|
|
|
|