X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 23 Jun 2017 08:44 AM by  MariM
IDL batch export to tiff from problematic hdf files
 5 Replies
Sort:
You are not authorized to post a reply.
Author Messages

PBreeze123



New Member


Posts:4
New Member


--
22 Jun 2017 07:13 AM
    I have a large number of (nearly a thousand) rasters in .hdf (hdf4) format from the GLASS albedo product. Unfortunately there is a problem with the projection information/header data which means ARCGIS, GDAL, QGIS, SAGA, and GRASS all seem to have problems with the data/will not recognise the projection. This problem has been acknowledged by the product support team as a known issue, but will not be resolved until the next release, until which time they suggest manually editing the headers (see below).

    The same is true in ENVI, but I correct the issue and use the data by opening a HDF file in ENVI, editing the envi header for that single file so the projection works, then copying the contents of that header to new text files (saved as .hdr) matching the names of all other files in the folder. This lets me open all those other data successfully in ENVI with the correct projection, however in other software/options it remains either entirely unrecognised/reads erroneously/causes a crash or appears without projection or even inverted. However, if data is opened using this approach in ENVI, and then exported as a Geotiff, it will then work in other software also.

    For this reason I am looking for a way to batch export all multiband files in a folder and/subfolders to multiband geotiffs, using ONLY IDL or ENVI GUI, so I can then integrate the data successfully for use in processes/models we have in other software. I wish. It seems previous answers to this batch export question have suggested using GDAL translate, but as I mentioned, this does not seem to work for this data. Obviously exporting each of the hundreds of files individually in the ENVI GUI is not feasible either. Unfortunately I have no coding experience, so please forgive in advance any questions I have which may seem obvious to those used to coding/IDL.

    Thanks for any help you can give,

    Paul

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    22 Jun 2017 09:10 AM
    You might try the ENVI Program Generator to help you write a program to export your files to TIFF using the ENVI API.
    http://www.harrisgeospati...enerator-ENVI-5.aspx

    PBreeze123



    New Member


    Posts:4
    New Member


    --
    22 Jun 2017 09:37 AM
    Hi Mari,

    thanks for the reply- that is tantalisingly close to a solution (I did not know about the program generator), but when I try, it will not read the files. For that matter, ENVI 5 will not open them either (though it sees them as HDF4), but Envi classic will? I don't suppose you know how to get program manager to access it using the ENVI classic as the back end do you?

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    22 Jun 2017 11:25 AM
    What version of ENVI are you running? It is strange that ENVI will not open the files but classic will.. The code generator actually uses classic API calls so you could work from there but I think a better option might be to use the toolkit Devin White developed (https://github.com/dawhite/MCTK) to import to classic. There are API examples as well that could be adapted for your data.

    PBreeze123



    New Member


    Posts:4
    New Member


    --
    23 Jun 2017 07:43 AM
    Hi Mari,

    I was running ENVI 5.1 , but having noticed the error I was getting seemed to match one you had a previous post on that was resolved by reinstalling, I have now updated to 5.3 to see if that helped (although I have just been told we dont have the IDL licence). Unfortunately I am still getting the following error from the EnviProgram Generator:

    RESTORE: Error opening file. Unit: 100, File: C:\Users\Paul\AppData\Loxal\Temp\myENVIProgram.sav

    Immediately after which, ENVI crashes.

    I have tried the Modis toolkit extension also, but as the GLASS albedo product is not one of the Modis own products (and possibly given the problem with the georeferencing) it refuses to open the files.

    Do you have any thoughts on why the above might be happening?

    Thanks again for your help- it is much appreciated!

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    23 Jun 2017 08:44 AM
    I am not sure what this is:
    C:\Users\Paul\AppData\Loxal\Temp\myENVIProgram.sav

    Does it exist in that directory? You might try removing it as well as deleting/moving the .idl directory from your home:

    C:\Users\mminari\.idl

    Then restart ENVI. It will recreate this directory fresh. This directory contains many configuration files that can sometimes become corrupted or conflicting.
    You are not authorized to post a reply.