X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 19 Oct 2009 11:12 AM by  anon
ENVI calls and maximum LUNs/fids
 7 Replies
Sort:
You are not authorized to post a reply.
Author Messages

anon



New Member


Posts:
New Member


--
19 Oct 2009 11:12 AM
    Hi, Having some trouble with ENVI calls that I'm making from an IDL procedure.  I am attempting to call ENVI_DOIT 'ENVI_GRID_DOIT' many times (> 200) in order to grid a series of irregular points data files.  For reasons that aren't clear to me, ENVI_GRID_DO_IT will only work this way for 28 files/iterations.  On the 29th, the value of r_fid is -1, which indicates an I/O error.  A subsequent IDL call to OPENR in another procedure (but within the same overall IDL session) returns ''OPENR: All available logical units are currently in use."  It appears that ENVI is grabbing LUNs and not returning them to the pool.  It also might be that there's some fid limit that I'm exceeding.  Either way, I can't seem to figure this out based on the available documentation. Has anyone out there seen something like this before?  I don't think that this is a particularly strange application of ENVI or IDL, so I'm optimistic that there is an easy solution or workaround (and processing the images individually is not an easy workaround in this case because I'm accumulating some stats on the collective set of data).  Thanks for any advice in advance...    

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    19 Oct 2009 11:29 AM
    ENVI does not use LUN's, it uses FIDs and they are not the same.  You cannot pass a LUN as a FID to an ENVI routine.  You can get a FID by using ENVI_OPEN_FILE or similar routine, or if you use ENVI_SETUP_HEAD to create a new ENVI file header.  It sounds like you are opening files using IDL's OPENR which would require a GET_LUN and then the LUN is not being freed later.  Freeing the LUN would not be done at the end of an ENVI routine, which would instead return a FID, so you have to handle this yourself, usually before you pass your file to ENVI.   Is there some reason you are using OPENR rather than ENVI's file open routines?  Does ENVI not recognize the file type you are opening?  If so, you may need to use ENVI_SETUP_HEAD to create an associated header for the binary datafile which would return a FID.  This FID could be passed to ENVI_GRID_DOIT and it does not need to be freed later on.

    Deleted User



    New Member


    Posts:
    New Member


    --
    19 Oct 2009 12:28 PM
    Thank you for your response.  My original post may have been confusing because I mentioned both IDL and ENVI calls.  I am aware that ENVI does not use LUNs.  I am not passing LUNs to ENVI or FIDs to IDL.  My code is almost exclusively made up of IDL calls, and I only use ENVI for a very small percentage of what I'm doing.  My point was that the IDL session from which I'd called ENVI had run out of LUNs, causing OPENR to fail with that error message. The IDL calls do not appear to be the problem.  I removed all ENVI calls from my code, and it worked correctly (without gridding of course) for hundreds of files.  I next began to reinsert the ENVI calls, and as soon as I put the ENVI_GRID_DOIT call back into the procedure, the code bombed on the 29th file.  As a quick debugging diagnostic, I printed the returned FID to the terminal immediately following each call to ENVI_GRID_DOIT.  Its value starts at 2 and increments by one on every iteration of the code until it reaches 29 (this is the 28th file).  On the next iteration/file, the value of the returned FID is -1, and then the code dies when this invalid FID gets used by a subsequent call to ENVI_GET_DATA. So, I'm pretty sure that the problem lies within ENVI somewhere.  Is there some kind of ENVI debugger that can be used to see what's going on inside those routines?  The debugging mode of IDL Workbench can't see into the ENVI routines, which seems like a poor design to me.  After all, both products are the property of ITT!  Thanks again for your response...  

    Deleted User



    New Member


    Posts:
    New Member


    --
    19 Oct 2009 12:28 PM
    oops, ignore dbl post.  

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    19 Oct 2009 12:32 PM
    So how are you passing data into ENVI_GET_DATA?  You can have a long integer number of FIDs, so you are not running out of FIDs.  Can you post a small example code that exhibits this behavior?  All the ENVI routines are compiled sav files, so you cannot view the source code.  You may just want to go through Technical Support to work on this issue.

    Deleted User



    New Member


    Posts:
    New Member


    --
    19 Oct 2009 01:28 PM
    Below is a stripped-down piece of code that demonstrates the behavior that I'm seeing.  Sorry if it's a little too much for posting, but I wanted to make sure that I was replicating the situation.  Compile it in IDL and run as:  print,TEST_ENVI_FIDS(29) The minimum number of iterations that is required to cause ENVI_GRID_DOIT to fail is 29, and only takes about 30 seconds.  The screen output in IDL shows that the ENVI routine is failing on the 29th call because it returns an FID of -1.  If you execute it twice in the same IDL session with arguments that add up to 29 (say, 15 on first run and 14 on second run), it will fail on the second run's last call to ENVI_GRID_DOIT.  I hope something jumps out at you in this code listing as being an obvious mistake, because I'm really scratching my head over this.  Your advice has been helpful, and I appreciate any more that you can provide.  Thanks!   forward_function ENVI_PROJ_CREATE,ENVI_GET_DATA,ENVI_GET_MAP_INFO ; ; FUNCTION TEST_ENVI_FIDS, num_iterations ; ; ; Loop "num_iterations" times on ENVI_GRID_DOIT FOR l = 1, num_iterations DO BEGIN ; ; ; First restore the ENVI base save files envi, /restore_base_save_files ; Initialize ENVI and send all errors and warnings to file envi_batch_init, log_file = 'batch_log.txt' ; Make some fake data to test the gridding functionality outpointdata_x = [-75.0,-76.0,-77.0] outpointdata_y = [37.0,38.0,39.0] outpointdata_z = [10.0,20.0,30.0] ; Set input and output projection info iproj_name = 'My Geographic' iproj_type = 1 oproj_name = 'My LAZEA' oproj_params = [6370997.0, 38.000000, -76.000000, 0.0, 0.0] oproj_type = 36 ; LAMBERT AZIMUTHAL EQUAL AREA ; Create the input and output projections based on the info my_latlon_proj = ENVI_PROJ_CREATE(/GEOGRAPHIC,name=iproj_name,type=iproj_type) my_lazea_proj = ENVI_PROJ_CREATE(name=oproj_name,params=oproj_params,type=oproj_type) ; Call envi_grid_doit to rasterize (grid) the point data.  ; (first set some argument values) out_dt = 4 ; floating point pixel_size = [50.,50.] ; 50 m X 50 m ENVI_DOIT, 'ENVI_GRID_DOIT', /IN_MEMORY, $   x_pts = outpointdata_x, y_pts = outpointdata_y, $   z_pts = outpointdata_z, i_proj = my_latlon_proj, $   out_dt = out_dt, $   pixel_size = pixel_size, o_proj = my_lazea_proj, $   interp = 0, r_fid = initial_gridded_file print,"RETURNED FID",initial_gridded_file print,"FILE/ITERATION", l ENDFOR END

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    19 Oct 2009 02:10 PM
    I am not sure why your code didn't post but I noticed that you have your RESTORE and BATCH_INIT inside your loop.  I actually think it is the batch init causing the problem because I believe a LUN is used to open the log text file.  There is no reason why these should be inside the loop, so moving them up before the FOR statement allowed me to run your code up to 35 times with a FID returned: RETURNED FID          33 FILE/ITERATION      32 RETURNED FID          34 FILE/ITERATION      33 RETURNED FID          35 FILE/ITERATION      34 RETURNED FID          36 FILE/ITERATION      35        0 ========================== forward_function ENVI_PROJ_CREATE,ENVI_GET_DATA,ENVI_GET_MAP_INFO ; ; FUNCTION TEST_ENVI_FIDS, num_iterations ; ; ; Loop "num_iterations" times on ENVI_GRID_DOIT FOR l = 1, num_iterations DO BEGIN ; ; ; First restore the ENVI base save files envi, /restore_base_save_files ; Initialize ENVI and send all errors and warnings to file envi_batch_init, log_file = 'batch_log.txt' ; Make some fake data to test the gridding functionality outpointdata_x = [-75.0,-76.0,-77.0] outpointdata_y = [37.0,38.0,39.0] outpointdata_z = [10.0,20.0,30.0] ; Set input and output projection info iproj_name = 'My Geographic' iproj_type = 1 oproj_name = 'My LAZEA' oproj_params = [6370997.0, 38.000000, -76.000000, 0.0, 0.0] oproj_type = 36 ; LAMBERT AZIMUTHAL EQUAL AREA ; Create the input and output projections based on the info my_latlon_proj = ENVI_PROJ_CREATE(/GEOGRAPHIC,name=iproj_name,type=iproj_type) my_lazea_proj = ENVI_PROJ_CREATE(name=oproj_name,params=oproj_params,type=oproj_type) ; Call envi_grid_doit to rasterize (grid) the point data.  ; (first set some argument values) out_dt = 4 ; floating point pixel_size = [50.,50.] ; 50 m X 50 m ENVI_DOIT, 'ENVI_GRID_DOIT', /IN_MEMORY, $   x_pts = outpointdata_x, y_pts = outpointdata_y, $   z_pts = outpointdata_z, i_proj = my_latlon_proj, $   out_dt = out_dt, $   pixel_size = pixel_size, o_proj = my_lazea_proj, $   interp = 0, r_fid = initial_gridded_file print,"RETURNED FID",initial_gridded_file print,"FILE/ITERATION", l ENDFOR END

    Deleted User



    New Member


    Posts:
    New Member


    --
    19 Oct 2009 02:40 PM
    Hey, that fixed it!  I was wondering about those statements.  The reason why I had them in the loop in the sample piece of code was to replicate their (repeated) execution whenever the real piece of code was called from a batch file.  So all I have to do is pull those statements out of the original function and include them at the top of the batch file that calls the function.  This betrays my relative lack of experience with ENVI's batch mode.  I'm much better acquainted with the way IDL works.  Thanks so much for your help -- I'd already begun to think about how to code the ugly throwaway workaround I was going to have to write if this problem couldn't be solved.  Can't think of a better way to begin the work week than with a coding victory...  
    You are not authorized to post a reply.