X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 09 Apr 2009 07:52 AM by  anon
read DEM files in IDL
 1 Replies
Sort:
You are not authorized to post a reply.
Author Messages

anon



New Member


Posts:
New Member


--
09 Apr 2009 07:52 AM
    I'm trying to read some DEM files that I downloaded from the Utah GIS website (ftp://ftp.agrc.state.ut.us/DEM/10meter_dem/).  The problem I'm having is that these DEMs do not seem to exactly follow the USGS standard for the format.  According to the published USGS standard that I found each record is supposed to be a multiple of 1024 bytes long, with the last 3 bytes being spaces.  However, the Utah GIS DEMs don't comply.  Instead, each record is only 1021 bytes or shorter and terminated by a new line character.  I'd like the code to be able to read in files with the correct standard but also maintain the ability to read the Utah files.  Currently I'm using readf, LUN, recordvariable which opens the Utah files just fine as the new line character seems to terminate the the command and return everything up to that point.   But since the "correct" format does not use these newline characters to indicate the end of the record loading a correctly formatted file does not seem to work.  Even if I modify the command to be something like: readf,LUN,recordvariable,FORMAT='(A1024)' attempting to get one record at a time, it will return a string of 1024 characters but for some reason it puts the file pointer at the end of the file instead of leaving it at the end of the record. So, the behavior that I need is for 1024 characters to be read at a time (leaving the file pointer at the end of that 1024 character block) unless a newline character is found, in which case stop reading at that point.  Can someone help me do that? Thanks.

    Deleted User



    New Member


    Posts:
    New Member


    --
    10 Apr 2009 09:09 AM
      Have you contacted AGRC to find out why there are LFs strewn throughout their products? From my past experiences with dealing with USGS DEMs, WINZIP would add LF/CR during decompression. WinZip has this feature “Smart CR/LF Conversion” where it would add CR/LF during decompression. I’m wondering if AGRC had this turned on when they created these self-extracting DEM files and LFs were added during compression. The way I see it, you have three choices. Choice 1: External from IDL, you can use sed or awk to remove the LF, Choice 2: Internal to IDL, read the DEM file one byte at a time and remove the LF (ASCII 10B), then follow the USGS format description to extract the DEM information, Choice 3: Use ENVI to read the DEM and use IDL to read the hdr and img files that ENVI creates. From my brief review of the data, I see LF at various spots – every 439, 893, and 1021 bytes. For (choice 2) example, file =  ‘36109h1.dem’ b = 0B oldpos = 0L cnt = 0L info = FILE_INFO( file ) HELP, info, /STRUCT barr = BYTarr( info.size ) OPENR, lun, file, /GET_LUN WHILE ( NOT EOF(lun) ) DO BEGIN   READU, lun, b   IF ( b EQ 10B ) THEN BEGIN     POINT_LUN, -lun, pos     PRINT, ' #### ', pos, ( pos - oldpos )     oldpos = pos   ENDIF ELSE BEGIN     barr[cnt] = b     cnt = cnt + 1   ENDELSE ENDWHILE CLOSE, lun FREE_LUN, lun Then to get Record A, which according to the USGS documentation is 144 bytes, is = 0L ie = 143L PRINT, STRING( [ barr[is:ie] ] ) I’ll leave the rest of your imagination to extract Record B and Record C. Kelly Dean
    You are not authorized to post a reply.

    Forums Products IDL