X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 06 Nov 2018 12:18 PM by  MariM
ENVI API GLTreproject
 14 Replies
Sort:
You are not authorized to post a reply.
Author Messages

Dulci Avouris



New Member


Posts:51
New Member


--
26 Oct 2018 09:15 AM
    Hello!

    I am trying to open and then re-project Sentinel 3 data using the ENVI API commands in IDL. The reprojectGLT ENVITask requires raster bands with the lat/lon values, but this is not how the Sentinel-3 data is opened in IDL. How do I do this? The GUI in ENVI works beautifully, but I am hoping to streamline the processing.

    Thanks so much,
    Dulci

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    28 Oct 2018 03:49 PM
    If you can open the data using the Scientific Data browser in ENVI, you can save the template and reuse it in the API by using the template keyword with ENVI::OpenRaster.
    https://www.harrisgeospatial.com/docs/ENVI__OpenRaster.html

    Dulci Avouris



    New Member


    Posts:51
    New Member


    --
    29 Oct 2018 01:21 PM
    Hello,

    Thanks for the reply. I am trying to not use the ENVI interface so I can batch process my imagery. The issue is not opening Sentinel 3A imagery - that is easily done in IDL. But I don't know how to access the lat/lon information in the sentinel 3A raster when it open in ENVI so that I can retroject the GLT and have a file with a standard projection.

    I can do this in ENVI - the Retroject GLT with Bowtie Correction tool works beautifully. I am looking for the IDL equivalent. The GLTreproject ENVITask asks for specific raster inputs of latitude & longitude. This info is not in separate bands when the S3A image is opened directly in IDL. So I am wondering how to retroject the S3A file in IDL without using the ENVI interface.

    Is that more clear?

    Thanks so much!
    -dulci

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    30 Oct 2018 05:44 AM
    If you first open the file using File->Open As->Scientific Formats, you can select the data and geometry bands and then save this as a template to use to open it using OpenRaster. This will show you the dataset_name to select and pass to OpenRaster.

    Alternately, if you open the data in ENVI (first test in the GUI) and you can see that the data open as 'GLT Projected', then ENVI is extracting the geometry on open. You can access the geometry bands from the array of files that are opened.

    Dulci Avouris



    New Member


    Posts:51
    New Member


    --
    30 Oct 2018 11:46 AM
    Thank you, as always for your help!

    I think we are still not communicating well. I know how to open the file in ENVI. I also know how to open the file in IDL. Sentinel 3A data needs to be reprojected, just like MODIS data, so the coordinate system is in a standard format. When I use the ENVI GUI to do that, I select the Retroject GLT with Bowtie correction, and the tool does the operation, and everything is good.

    I want to write a line of code in IDL that does the GLT reprojection. I do not want to have to do that step in ENVI for each individual image that I work with. There is an ENVItask ReprojectGLT (https://www.harrisgeospatial.com/docs/ENVIReprojectGLTTask.html) that will perform this process. However, the ENVItask needs the latitude and longitude data for the S3A image as rasters. I am asking how to point to that latitude and longitude data in IDL so the reprojection can be accomplished. Or - is there another way to do this using code in IDL?

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    30 Oct 2018 01:42 PM
    Yes, I do understand. I am just not explaining myself well. What I mean is if you can do it in the GUI, then if you just open the Sentinel-3 raster using OpenRaster, the lat/lon information will also be extracted as the spatialreference. You have to look through the spatialref object that opens with the data. For example, using the Sentinel-3 data from our tutorial:

    ENVI> file='C:\Testing\ENVI Tutorial Data\Sentinel-3\S3A_Marine_GulfOman\xfdumanifest.xml'
    ENVI> raster=e.openraster(file)

    ENVI> print, raster[0]
    ENVIRASTER <462153>
    AUXILIARY_SPATIALREF = !NULL
    AUXILIARY_URI = <Array[16]>
    DATA_TYPE = 'double'
    INTERLEAVE = 'bsq'
    METADATA = <ObjHeapVar493409(ENVIRASTERMETADATA)>
    NBANDS = 16
    NCOLUMNS = 4865
    NROWS = 4091
    PYRAMID_EXISTS = 1
    READ_ONLY = 1
    SPATIALREF = <ObjHeapVar493410(ENVIGLTRASTERSPATIALREF)>
    TIME = <ObjHeapVar493411(ENVITIME)>
    URI = 'C:\Testing\ENVI Tutorial Data\Sentinel-3\S3A_Marine_GulfOman\xfdumanifest.xml'

    ENVI> print, raster[0].spatialref
    ENVIGLTRASTERSPATIALREF <493410>
    XMAP_GRID = <ObjHeapVar493444(ENVIRASTER)>
    YMAP_GRID = <ObjHeapVar493445(ENVIRASTER)>

    You can pass the XMAP and YMAP as the lat/lon. I assume your data is similar.

    Dulci Avouris



    New Member


    Posts:51
    New Member


    --
    30 Oct 2018 07:42 PM
    Oh! This makes sense. Thank you!

    OK. So this is what I did:

    s3a = e.openraster('xfdumanifest.xml')

    xmap_grid = s3a[0].spatialref
    ymap_grid = s3a[0].spatialref

    Task = ENVITask('ReprojectGLT')

    Task.Input_Raster = s3a
    Task.Latitude_Raster = xmap_grid
    Task.Longitude_Raster = ymap_grid

    Task.Output_Raster_URI = e.GetTemporaryFilename()

    Task.Execute

    end

    and this is the error I got. What am I missing now?

    Parameter LATITUDE_RASTER failed validation: Value must be a scalar ENVIRASTER.

    Do I need to convert the map_grid & map_grid?

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    31 Oct 2018 08:34 AM
    Can you to a help or print on your:
    xmap_grid = s3a[0].spatialref
    ymap_grid = s3a[0].spatialref

    Are they grids and of the same size as the data?

    Dulci Avouris



    New Member


    Posts:51
    New Member


    --
    01 Nov 2018 11:26 AM
    when I print map_grid is or directly print s3a[0].spatialref the result is !NULL


    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    01 Nov 2018 12:00 PM
    So the question is, where is the spatial reference stored in your file. There could be different formats that store it in different locations I suppose but if you open the file in the GUI and you are able to use the Reproject GLT tool on it directly, ENVI must be extracting and locating the spatial reference.Does your file also open using a manifest.xml?
    It is difficult to test without having access to the data. Would you be able to point me to a similar data set that I could download?



    Dulci Avouris



    New Member


    Posts:51
    New Member


    --
    01 Nov 2018 12:36 PM
    Hello!

    Yes - that is exactly the essence of the issue. The data that I am using is the Sentinel 3A OLCI Level 2 WFR product. I have an account here:
    https://eoportal.eumetsat.int/userMgmt/login.faces?goto=http://coda.eumetsat.int/#/home

    If possible, I can send you my exact dataset.All the imagery that I use from Sentinel 3A would be in the same original format. I get it as a folder with all the data, but I open the controlling file using Open As -> Optical Sensors -> European Space Agency -> Sentinel -3 OLCI.

    The controlling file is an xfdumanifest.xml

    Maybe I need to use a different command to open it to begin with? NOT "s3a = e.openraster('xfdumanifest.xml')"

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    02 Nov 2018 09:44 AM
    I found a similar data set in our test data - S3A_OL_2_WFR____20161001T102832_20161001T103132_20161003T122944_0179_009_222_1980_MAR_O_NR_001.SEN3. I find this data has a geometry file called tie_geo_coordinates.nc, which is a "low resolution georeferencing data" that is not a 1-to-1 lat/lon raster that matches the data dimensions of 4865x4091. So ENVI will not open this data with this spatial reference since all rasters (lat/lon) must match the data resolution. This is why your spatialref is null.
    Does this data also come with the full resolution geometry?

    Dulci Avouris



    New Member


    Posts:51
    New Member


    --
    06 Nov 2018 06:47 AM
    It also comes with a tie_geometries.nc and a geo_coordinates.nc. Will those help?

    When I use the ENVI GUI to open these files, I only have to open the xfdumanifest.xml file, and ENVI reads in the all the data, and gives me a raster with the 16 image bands, and also a number of calculated products. I then use the "Reproject GLT with Bowtie Correction" tool (under Geometric Correction) on the 16 band image raster. This tool automatically reads in whatever geo information is in the sentinel 3A file and produces a raster image with a standard WGS-84. The original file says 'proj *GLT* geographic lat/lon, WGS-84' when I look at the spatial information in the metadata.

    So I don't know how to get at all that info when I am using the command lines in IDL to mimic what happens when I am using the ENVI GUI interface.

    Do I need to use a different command to open the original xfdumanifest.xml file? How do I print a list of everything in the file so I can see where it is?

    Thanks for the help!!

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    06 Nov 2018 07:42 AM
    You should be able to open the xfdumanifest.xml using OpenRaster just like you can in the GUI. The test file I had did *not* open with GLT georeferencing so your file must be a little different and if it opens in the GUI, it should also have the geometry when you open it with the API.
    I think I need to find a file just like yours. Perhaps come through tech support directly so we can work on the same file.

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    06 Nov 2018 12:18 PM
    I believe this is a syntax error and the data does contain the xmap and ymap information. You can access it in the following way and run the task:

    ENVI> file='S3A_OL_2_WFR____20181102T153657_20181102T153957_20181102T173406_0179_037_282_2520_MAR_O_NR_002.SEN3\xfdumanifest.xml'

    ENVI> raster=e.openraster(file)

    ENVI> print, raster[0]
    ENVIRASTER <458484>
    AUXILIARY_SPATIALREF = !NULL
    AUXILIARY_URI = <Array[16]>
    DATA_TYPE = 'double'
    INTERLEAVE = 'bsq'
    METADATA = <ObjHeapVar459329(ENVIRASTERMETADATA)>
    NBANDS = 16
    NCOLUMNS = 4865
    NROWS = 4091
    PYRAMID_EXISTS = 1
    READ_ONLY = 1
    SPATIALREF = <ObjHeapVar459330(ENVIGLTRASTERSPATIALREF)>
    TIME = <ObjHeapVar459331(ENVITIME)>
    URI = 'C:\Users\mminari\Downloads\S3A_OL_2_WFR____20181102T153657_20181102T153957_20181102T173406_0179_037_282_2520_MAR_O_NR_002.SEN3\xfdumanifest.xml'

    ENVI> print, raster[0].spatialref
    ENVIGLTRASTERSPATIALREF <459330>
    XMAP_GRID = <ObjHeapVar459332(ENVIRASTER)>
    YMAP_GRID = <ObjHeapVar459333(ENVIRASTER)>

    ENVI> Task = ENVITask('ReprojectGLT')

    ENVI> Task.Input_Raster = Raster[0]

    ENVI> Task.Latitude_Raster = raster[0].spatialref.ymap
    ENVI> Task.Longitude_Raster = raster[0].spatialref.xmap

    Task.Execute
    You are not authorized to post a reply.