X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 28 Dec 2011 09:47 PM by  anon
landsat TM batch
 11 Replies
Sort:
You are not authorized to post a reply.
Author Messages

anon



New Member


Posts:
New Member


--
28 Dec 2011 09:47 PM
    hi,every one , I recently need to process lots of landsat TM and MSS data, firstly I want the batch calibrate these files. I try to write this pro file expected to finish the job ,but failed, it shows 'FID is always =-1' .so ,I try to used"envi_open_file" or "envi_open_data_file", both donot work.the pro is gived below, I really hope anyone give any suggestion. thanks many. Pro landsat_batch_calibration compile_opt IDL2 ; First restore all the base save files. envi, /restore_base_save_files ENVI_BATCH_INIT, LOG_FILE = 'batch.log' FilePath = 'H:\LT\' OUTFile = 'H:\LToutput\' FileSearch = FILE_SEARCH(FilePath, COUNT = ProductCount, '*MTL.txt', /TEST_READ, /FULLY_QUALIFY_PATH) IF(ProductCount LE 0) THEN BEGIN Print, 'There is no landsat metedata to be processed.' RETURN ENDIF FOR i=0, ProductCount-1 DO BEGIN File = FileSearch[i] Filename = FILE_BASENAME(File) ;Open Landsat data ENVI_OPEN_DATA_FILE, 'file', /LANDSAT_METADATA, R_FID=fid ;Envi_open_data_file, File,/landsat_metedata,r_fid=fid print, fid ;the fid in my screen is always equal -1 fids=ENVI_GET_FILE_IDS() FOR i=0, N_ELEMENTS(fids) -1 DO BEGIN ENVI_FILE_QUERY, fids[i], NB=nb, SNAME=sname IF nb EQ 6 THEN mb_fid = fids[i] ENVI_FILE_QUERY, fid, ns=ns, nl=nl, nb=nb dims = [-1, 0, ns-1, 0, nl-1] pos = lindgen(nb) bands_present = [1,2,3,4,5,7] ENDFOR n = FILE_LINES(File) metadata = STRARR(1,n) OPENR, unit, File, /GET_LUN READF, unit, metadata FREE_LUN, unit ; find position of sun_angle in metadata and read value pStartSE = STRPOS(metadata[99],'=') sun_elevation = STRMID(metadata[99], pStartSE + 2,10) PRINT, 'sun elevation=', sun_elevation ; find position of sun_azimuth in metadata and read value pStartSA = STRPOS(metadata[98], '=') sun_azimuth = STRMID(metadata[98], pStartSA + 2,10) PRINT, 'sun_azimuth=', sun_azimuth sun_angel=90.0-sun_elevation ; Perform the TM Calibration ENVI_DOIT, 'TMCAL_DOIT',fid=fid, pos=pos, dims=dims,bands_present=bands_present,sat=1, cal_type=1, date=0,sun_angle=sun_angel, out_name=outfile+filename, r_fid=r_fid ENDFOR end

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    29 Dec 2011 07:23 AM
    Try some print statements to see what is returned when you do File_search and what the value of 'ProductCount' is. Do these return valid values (does it find files)? You are passing a string ('file') to ENVI_OPEN_DATA_FILE but it looks like file is a variable. Do a print on 'file' and see what its value is. Is it a file name?

    Deleted User



    New Member


    Posts:
    New Member


    --
    30 Dec 2011 04:02 AM
    it is strange. today I try the pro again. except the ENVI_DOIT, 'TMCAL_DOIT' looks like not run, the other are work well, and the fid =10 ,any one can tell why the ENVI_DOIT, 'TMCAL_DOIT' function does not run?

    Deleted User



    New Member


    Posts:
    New Member


    --
    30 Dec 2011 06:58 AM
    I can think of a lot of reasons it wouldn't run. But they would all have different behavior. Can you tell us exactly what you are seeing when it doesn't run? Do you get an error message? Peg

    Deleted User



    New Member


    Posts:
    New Member


    --
    30 Dec 2011 09:48 PM
    actually, as a vegetation bird, I write this pro file which is modified from the discussion "http://www.exelisvis.com/...posts.aspx",and I may have not enough exprience to finish it. hoep anyone give me help./ the result is show like this : "H:\LT51430302010232IKR00\L5143030_03020100820_MTL.txt 10 10 sun elevation=53.6730895 sun_azimuth=140.908264 IDL> "

    Deleted User



    New Member


    Posts:
    New Member


    --
    03 Jan 2012 07:48 AM
    It's hard to understand exactly what is happening from your description, but it looks to me like your code ran successfully. You show the program returning to an IDL prompt, and no error message, which is the behavior I would expect if everything ran successfully. Have you checked to see whether the output image you specified in your code was created, and if so, whether it looks good? - Peg

    Deleted User



    New Member


    Posts:
    New Member


    --
    10 Jan 2012 03:28 AM
    For Landsat 7 (Landsat 5 too), code will look something like this: [QUOTE]Code example PRO LANDSAT_CALIBRATION_BATCHMODE COMPILE_OPT IDL2 ENVI_BATCH_INIT InputPath = 'c:\LANDSAT\' ListFiles = FILE_SEARCH(InputPath, COUNT = ProductCount, '*MTL.txt', /FOLD_CASE, /TEST_READ, /FULLY_QUALIFY_PATH) IF(ProductCount LE 0) THEN BEGIN Print, 'There are no valid landsat metadata to be processed.' RETURN ENVI_BATCH_EXIT ENDIF FOR i=0, ProductCount-1 DO BEGIN InputFile = ListFiles[i] Filename = FILE_BASENAME(InputFile, '_MTL.txt', /FOLD_CASE) Envi_open_data_file, InputFile, /landsat_metadata fids = envi_get_file_ids() IF (fids[0] EQ -1) THEN BEGIN Print, 'There are no valid landsat metadata to be processed.' RETURN ENVI_BATCH_EXIT ENDIF FOR j = 0, n_elements(fids)-1 DO BEGIN envi_file_query, fids[j], fname = fname, nb=nb IF (nb EQ 6) OR (nb EQ 4) THEN BEGIN image_fid=fids[j] envi_file_query, image_fid, fname = fname_image, dims=dims_image, nb=nb_image print, fname_image , ' nb: ', nb_image, ' fid: ', image_fid ENDIF ENDFOR pos=lindgen(nb_image) ; Metadata parser n = FILE_LINES(InputFile) metadata = STRARR(1,n) OPENR, unit, InputFile, /GET_LUN READF, unit, metadata FREE_LUN, unit ; Find position of sun_angle in metadata and read value ind=WHERE(STRPOS(metadata,'SUN_ELEVATION') ge 0) IF (ind NE [-1]) THEN BEGIN searching_line = metadata[ind] searching_line_data=strsplit(strcompress(searching_line, /remove_all),'=', /extract) sun_elevation=float(searching_line_data[1]) sun_angel=90.0-sun_elevation ENDIF ; Find position of date in metadata and read value ind=WHERE(STRPOS(metadata,'ACQUISITION_DATE') ge 0) IF (ind NE [-1]) THEN BEGIN searching_line = metadata[ind] searching_line_data=strsplit(strcompress(searching_line, /remove_all),'=', /extract) date_line=searching_line_data[1] date_array=fix(strsplit(strcompress(date_line, /remove_all),'-', /extract)) date=intarr(3) date[0]=date_array[1] date[1]=date_array[2] date[2]=date_array[0] print, date ENDIF ; Determine Satellite IF ((STRPOS(Filename,'L7')) NE -1) THEN BEGIN sat=0 bands_present=[0,1,2,3,4,7] ENDIF IF ((STRPOS(Filename,'L5')) NE -1) THEN BEGIN sat=1 bands_present=[0,1,2,3,4,6] ENDIF PRINT, 'sun elevation=', sun_elevation PRINT, 'sat_ind=', sat gain=fltarr(nb_image) bias=fltarr(nb_image) min_max_rad=WHERE(STRPOS(metadata,'MIN_MAX_RADIANCE') ge 0) lmax_min=fltarr(min_max_rad[1]-min_max_rad[0]-1) searching_line = metadata[min_max_rad[1]+ 2] searching_line_data=strsplit(strcompress(searching_line, /remove_all),'=', /extract) qcalmax=fix(searching_line_data[1]) searching_line = metadata[min_max_rad[1]+ 3] searching_line_data=strsplit(strcompress(searching_line, /remove_all),'=', /extract) qcalmin=fix(searching_line_data[1]) FOR b=0, n_elements(lmax_min)-1 DO BEGIN searching_line = metadata[min_max_rad[0]+b+1] searching_line_data=strsplit(strcompress(searching_line, /remove_all),'=', /extract) lmax_min[b]=float(searching_line_data[1]) ENDFOR IF (sat EQ 0) OR (sat EQ 1) THEN BEGIN lmax_min=reform(lmax_min, 2, n_elements(lmax_min)/2, /overwrite) gain[0]=(lmax_min[0,0]-lmax_min[1,0])/(qcalmax-qcalmin) bias[0]=lmax_min[1,0]-((lmax_min[0,0]-lmax_min[1,0])/(qcalmax-qcalmin))*qcalmin gain[1]=(lmax_min[0,1]-lmax_min[1,1])/(qcalmax-qcalmin) bias[1]=lmax_min[1,1]-((lmax_min[0,1]-lmax_min[1,1])/(qcalmax-qcalmin))*qcalmin gain[2]=(lmax_min[0,2]-lmax_min[1,2])/(qcalmax-qcalmin) bias[2]=lmax_min[1,2]-((lmax_min[0,2]-lmax_min[1,2])/(qcalmax-qcalmin))*qcalmin gain[3]=(lmax_min[0,3]-lmax_min[1,3])/(qcalmax-qcalmin) bias[3]=lmax_min[1,3]-((lmax_min[0,3]-lmax_min[1,3])/(qcalmax-qcalmin))*qcalmin gain[4]=(lmax_min[0,4]-lmax_min[1,4])/(qcalmax-qcalmin) bias[4]=lmax_min[1,4]-((lmax_min[0,4]-lmax_min[1,4])/(qcalmax-qcalmin))*qcalmin gain[5]=(lmax_min[0,6]-lmax_min[1,6])/(qcalmax-qcalmin) bias[5]=lmax_min[1,6]-((lmax_min[0,6]-lmax_min[1,6])/(qcalmax-qcalmin))*qcalmin ENDIF print, qcalmax print, gain print, bias ENVI_DOIT, 'TMCAL_DOIT', fid=image_fid, bands_present=bands_present, pos=pos, dims=dims_image, sat=sat, $ cal_type=1, sun_angle=sun_angel, out_name=InputPath+Filename, r_fid=r_fid, date=date, $ gain=gain, bias=bias FOR index=0, n_elements(fids)-1 DO BEGIN envi_file_mng, /remove, id=fids[index] ENDFOR ENDFOR envi_batch_exit END [/QUOTE] I'm not sure how well TMCAL_DOIT works, but the description in the help causes serious though . [QUOTE]ENVI help result=Input * GAIN + BIAS gain = (lmax - lmin) / 255 bias = lmin[/QUOTE] This is correct only for QCALMAX=0. Alex

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    10 Jan 2012 07:42 AM
    TMCAL_DOIT has been re-written to pass the metadata file directly to the routine so that you no longer need to parse the metadata file. It can now be called using something as simple as: envi_doit, 'tmcal_doit', fid=fid, pos=pos, dims=dims, cal_type=1, out_name='tm_refl.dat', /use_metadata Note that you don’t need to pass in any gain/bias/date/sat/sun_angle or bands_present and you can use pos if you want to calibrate a spectral subset. We have a patch that can be applied to ENVI 4.8 that will allow for you to use the new routine. If you are interested, please contact Technical Support directly.

    Deleted User



    New Member


    Posts:
    New Member


    --
    06 Feb 2012 05:41 PM
    If you have multiple "MTL.txt" files the code provided by Alex will not work well. If you replace (almost at the end): FOR index=0, n_elements(fids)-1 DO BEGIN envi_file_mng, /remove, id=fids[index] ENDFOR with: fids=envi_get_file_ids() FOR index=0, n_elements(fids)-1 DO BEGIN envi_file_mng, /remove, id=fids[index] ENDFOR that will solve the problem (works for me). Otherwise, youll be always using the first file. Regardless this small "bug", the code works perfectly (thank you Alex for your code, was exactly what I was looking for!) Im also not sure if you have to use the sun elevation raw value, or you have to convert it to sun angle (like Alexs code does)... Im always confused about it. It seems to me that when you do it manually (through the ENVIs menu), ENVI uses the sun elevation value; Am I correct? Álvaro.

    Deleted User



    New Member


    Posts:
    New Member


    --
    07 Feb 2012 09:03 AM
    Yes, the SUN_ANGLE keyword is for the solar elevation angle, which is the angle of the sun above the horizon, extending from 0 degrees to 180 degrees. I think 0 to 90 to 180 degrees is when it's over the western horizon. So, 90 degrees is straight overhead. - Peg

    Deleted User



    New Member


    Posts:
    New Member


    --
    28 Oct 2013 09:31 PM
    The code works well in TM5 visible bands calibration. But when I calibration the TM5 thermal band to brightness temperature, I have set band_present=[5], the value is always between 0-1, not in Kelvin. I hope you give my any suggestion, thanks very much.

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    29 Oct 2013 06:41 AM
    The documentation states: Set BANDS_PRESENT=[0] to process Landsat-5 TM band 6 thermal data. This band is handled separately in the Available Bands List and has its own file ID (FID). http://www.exelisvis.com/docs/TMCAL_D...
    You are not authorized to post a reply.