X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 01 Aug 2017 11:03 AM by  Miriam Johnston
ENVI_GLT_DOIT OPENR Error
 8 Replies
Sort:
You are not authorized to post a reply.
Author Messages

Miriam Johnston



New Member


Posts:4
New Member


--
28 Jul 2017 12:09 PM
    I am unable to get ENVI_GLT_DOIT to complete; it finishes "Build GLT File Part 1" and then aborts at the end of "Build GLT File Part 2 with the following error:

    Build GLT File: An error has occurred during processing. Error: "OPENR: Error opening file. Unit:102, File: /tmp/envFriJul2812222220171926558349_1.tmp" The result may be invalid.

    I am using IDL version 8.4 on Ubuntu 14.04 and trying to create GLTs from lat/long files associated with MASTER remote sensing datasets. Here is an example of the code:

    ;Open ENVI:
    e = envi()

    ;Open location raster that holds the x/y information:
    Loc = 'location.dat'
    Location = e.OpenRaster(Loc)
    Latitude = ENVISubsetRaster(Location,BANDS=0)
    Longitude = ENVISubsetRaster(Location,BANDS=1)

    ;Give x and y FIDs:
    fidx = ENVIRastertoFID(Longitude)
    fidy = ENVIRastertoFID(Latitude)

    ;Define input projection:
    iproj=envi_proj_create(/geographic)

    ;Determine dimensions
    envi_file_query, fidx, dims=dims

    ;Run the GLT routine (and don't change the projection):
    ENVI_DOIT, 'ENVI_GLT_DOIT', DIMS=dims, i_proj=iproj, o_proj=iproj, out_name="testGLT", rotation=0, x_fid=fidx, y_fid=fidy, x_pos=0, y_pos=0

    Thank in advance.

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    28 Jul 2017 02:26 PM
    I don't see anything wrong with your code. Unfortunately, the error is rather generic. If you dismiss it, does the process stop or complete? Are you able to build the GLT using the interface? If not, then there might be something unusual about the files. Does it work with the newer 'Reproject GLT with Bowtie correction'?

    Miriam Johnston



    New Member


    Posts:4
    New Member


    --
    31 Jul 2017 08:27 AM
    Thanks for the response.

    (1) Regarding whether the process stops or completes when I dismiss the error: I have only one choice when the OPENR error pops up, which is to press 'Ok.' Then, the Error dialog box goes away and the output GLT, 'testGLT', appears in my working directory. If I attempt to use 'testGLT' in ENVI_GEOREF_FROM_GLT_DOIT to georeference the data, I receive another error (please see below for details). Dismissing the second error results in a final file ("tempRefGLT") which is full of zeros.

    The code I used is (continuation of code in my previous post):

    testGLT='testGLT'
    ENVI_OPEN_FILE, testGLT, R_FID=OUT
    envi_file_query, OUT, dims=GLTdims
    ENVI_DOIT, 'ENVI_GEOREF_FROM_GLT_DOIT', BACKGROUND=0, FID=fid, GLT_DIMS=GLTdims, GLT_FID=OUT, OUT_NAME="tempRefGLT", POS=0, R_FID=tempRef_fid

    And the error is:
    'ENVI Retrieve Data: An error has occurred during processing. Error: "Attempt to subscript <PtrHeapVar239192> with <INT(1)> is out of range." The result may be invalid.'

    Given the error, I tried including the POS argument with and without brackets. Here is the information from the data file I am trying to georeference (fid=fid):
    ENVIRASTER <238826>
    AUXILIARY_SPATIALREF = !NULL
    AUXILIARY_URI = <Array[2]>
    DATA_TYPE = 'float'
    INTERLEAVE = 'bip'
    METADATA = <ObjHeapVar239575(ENVIRASTERMETADATA)>
    NBANDS = 1
    NCOLUMNS = 716
    NROWS = 9999
    PYRAMID_EXISTS = 1
    READ_ONLY = 1
    SPATIALREF = !NULL
    TIME = <ObjHeapVar239576(ENVITIME)>
    URI = 'MASTERL2_0460500_10_20031017_0116_0143_V01-surface_temp.dat'

    (2) Regarding whether I am able to build the GLT using the interface: Yes, in the ENVI classic interface I have no issues either building the GLT or using it to georeference the data. The output is as I would expect and appears correct.

    Thanks.




    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    31 Jul 2017 10:15 AM
    Where did you get the lat/lon bands? It looks like they were saved to a separate file? So when you do this in the GUI, you build the GLT from the file called 'location.dat' and then try to Georeference the original MASTER file with this GLT?

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    31 Jul 2017 11:54 AM
    I downloaded a L2 data set where the location bands and surface temp come as .dat bands in the archive. Is this the sort of data you have?
    I believe the issue is when you use ENVISubsetRaster, it creates a virtual raster which is a special case for ENVI that Classic doesn't seem to manage. It may be a requirement to save the subset raster bands first, then pass them to ENVIRasterToFID. Alternately, you can avoid creating the virtual rasters by passing the original location data as a 2 band file, then pass the x_pos/y_pos. For example:

    Location = e.OpenRaster(Loc)
    loc_fid = ENVIRastertoFID(location)

    ;Define input projection:
    iproj=envi_proj_create(/geographic)

    ;Determine dimensions
    envi_file_query, loc_fid, dims=loc_dims

    ENVI_DOIT, 'ENVI_GLT_DOIT', DIMS=loc_dims, i_proj=iproj, o_proj=iproj, r_fid=glt_fid, $
    out_name="testGLT", rotation=0, x_fid=loc_fid, y_fid=loc_fid, x_pos=0, y_pos=1

    Does this work for you?

    Miriam Johnston



    New Member


    Posts:4
    New Member


    --
    31 Jul 2017 11:59 AM
    Yes, exactly. The lat/lon bands are in the file location.dat. The data to be georeferenced (i.e. the MASTER data) are in the file surface_temp.dat, with the FID 'fid'. From the GUI, here are the steps I follow to successfully create the georeferenced dataset:

    (1) File --> Open Image File --> location.dat & surface_temp.dat
    (2) Map --> Georeference from Input Geometry --> Build GLT
    (2a) Input X Geometry Band: select Band 1 of location.dat
    (2b) Input Y Geometry Band: select Band 0 of location.dat
    (2c) Input Projection of Geometry Bands: Geographic Lat/Lon (Datum WGS-84, Units Degrees)
    (2d) Output Projection for Georeferencing: Geographic Lat/Lon (Datum WGS-84, Units Degrees)
    (2e) GLT parameters: Output Pixel size [ Leave default = 0.000376 ]; Output Rotation: 0; Output to memory (but it works to output to file as well)
    (3) Map --> Georeference from Input Geometry --> Georeference from GLT
    (3a) Select GLT file from memory as Input GLT
    (3b) Input data file: surface_temp.dat
    (3c) Georeference from GLT parameters: Subset to Output Image Boundary? yes; Background Value = 0; Output to file.

    Miriam Johnston



    New Member


    Posts:4
    New Member


    --
    31 Jul 2017 02:59 PM
    Yes, thank you very much for looking into the data -- that is indeed what I'm using. I have now coded the GLT in two new ways, given your insight about virtual rasters. Both worked, mostly (no errors, very exciting!), but there's one last issue: the output is oriented strangely. I'll return to this at the bottom of this posting. Here are the two ways I coded the GLT:

    (1) Using the code you suggested in your last response. This did not throw any errors, but it took about 2 hours just to build the GLT.

    (2) By saving the latitude & longitude from location.dat, re-loading them into ENVI, and then proceeding to the ENVI_GLT_DOIT line. This second method was speedier (~5 minutes):

    Latitude = ENVISubsetRaster(Location,BANDS=0)
    Latitude.Export, "fullpath/latexp", "tiff"
    LatRast = e.OpenRaster('latexp')
    lat_fid = ENVIRastertoFID(LatRast)

    Longitude = ENVISubsetRaster(Location,BANDS=1)
    Longitude.Export, "fullpath/lonexp", "tiff"
    LongRast = e.OpenRaster('lonexp')
    lon_fid = ENVIRastertoFID(LongRast)

    ENVI_DOIT, 'ENVI_GLT_DOIT', DIMS=loc_dims, i_proj=iproj, o_proj=iproj, r_fid=glt_fid, $
    out_name="testGLT", rotation=0, x_fid=lon_fid, y_fid=lat_fid, x_pos=0, y_pos=0

    The only remaining issue (I think...) is that when I set rotation = 0 in ENVI_GLT_DOIT, the output is a vertical rectangle that includes only part of the georeferenced data (presumably because now the georeferenced data doesn't lie in the rectangular window of the non-georeferenced data). This is not the case when I specify rotation =0 using the ENVI classic GUI. In contrast, when rotation is left blank, the program chooses a seemingly-random rotation (=3 -- perhaps to minimize blank space?)

    Thank you again! This is huge progress and I really appreciate it!

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    01 Aug 2017 08:28 AM
    There should be no difference using the location.dat directly or passing the bands individually. It takes about a minute to process the surface_temp.dat, which is 716x1933. Is your file significantly larger?

    I just noticed that I had my lat/lon bands wrong for x_pos, y_pos. Here is the full code I use which results in the exact same results as the Georeference from IGM tool in ENVI:
    ----------------------------
    pro master_georeference_test
    compile_opt idl2

    ;Open ENVI:
    e = envi()

    ; open master data
    file="C:\Testing\Test Data\MASTER\MASTERL2_1762600_01_20170224_0049_0054_V01-surface_temp.dat"
    raster=e.openraster(file)
    dataFID=ENVIRastertoFID(raster)

    ;Open location raster that holds the x/y information:
    Loc = 'C:\Testing\Test Data\MASTER\MASTERL2_1762600_01_20170224_0049_0054_V01-location.dat'
    Location = e.OpenRaster(Loc)
    loc_fid = ENVIRastertoFID(location)

    ;Define input projection:
    iproj=envi_proj_create(/geographic)

    ;Determine dimensions
    envi_file_query, loc_fid, dims=loc_dims

    ;Run the GLT routine (and don't change the projection):
    ENVI_DOIT, 'ENVI_GLT_DOIT', DIMS=loc_dims, i_proj=iproj, o_proj=iproj, r_fid=glt_fid, $
    out_name="C:\Testing\Test Data\MASTER\testGLT", rotation=0, x_fid=loc_fid, y_fid=loc_fid, x_pos=1, y_pos=0

    envi_file_query, glt_fid, dims=GLTdims
    ENVI_DOIT, 'ENVI_GEOREF_FROM_GLT_DOIT', BACKGROUND=0, FID=DataFID, GLT_DIMS=GLTdims, $
    GLT_FID=glt_fid, OUT_NAME="C:\Testing\Test Data\MASTER\tempRefGLT", POS=0, R_FID=tempRef_fid

    end

    Miriam Johnston



    New Member


    Posts:4
    New Member


    --
    01 Aug 2017 11:03 AM
    My files aren't any larger, but I was pulling them from a hard drive -- that seems to be the speed culprit. Everything it working well now, thanks very much for your help!
    You are not authorized to post a reply.