Calculating Class statistics for batch files

Forums
Search Forum
Advanced SearchTopicsPosts

Prev Go to previous topic

Go to next topic

Last Post 02 Jun 2016 03:14 PM by anon

Calculating Class statistics for batch files

4 Replies

Sort:


You are not authorized to post a reply.

Author

Messages

anon

New Member

Posts:

02 Jun 2016 03:14 PM
Hello, I have a question about calculating class statistics. My current code performs an unsupervised classification on a batch of 25 images. I would now like to add the ability to calculate class statistics for the outputs but have so far been unsuccessful in getting this additional step to run. I believe I need to add another foreach command but am not sure how to pull my new classified images back in without having to somehow list them as I have for the initial batchfiles. In addition to this BIG issue, I have a couple of subsequent questions which I below. Following these questions is a copy of my current code. I recognize this is probably not the most elegant way to have performed this procedure, but for my experience and IDL skill level it is what has been working for me. Any input would be greatly appreciated. QUESTIONS: 1.) Do I have my new fid and dims correctly labeled as being based on my classification output (i.e. CLASS_FID=classifi01_fid) where classifi01 is the R_fid from the classification run? 2.) Do I need the line for the "envi_file_query"? CODE: pro ISODATA_classification_and_ClassStats_batch25 compile_opt idl2 on_error, 2 ; General error handler Catch, error if (error ne 0) then begin Catch, /CANCEL if obj_valid(envi) then $ envi.ReportError, "Error: " + !error_state.msg message, /RESET return endif envi=ENVI() batchfiles = ["C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-10-12_38383C.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-10-12_39390A.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-10-12_40392D.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-10-19_36078C.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-10-19_36079A.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-10-19_40392A.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-10-19_BARED.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-10-26_38785C.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-11-04_28043C.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-11-16_35275C.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-11-16_38382E.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-11-16_39990C.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-11-30_36078B.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-12-06_25631A.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-12-06_27233B.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2013-01-23_26837C.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2013-01-23_27234A.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2013-01-23_38785B.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2013-02-21_25631A.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2013-04-01_27233B.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2013-04-01_36078A.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2013-04-01_38782C.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2013-04-01_40389C.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2013-04-26_26430A.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2013-04-26_36078C.JPG"] foreach file, batchfiles do begin rasterloop = envi.OpenRaster(file) rasterloop_fid = ENVIRasterToFID(rasterLoop) envi_file_query, rasterloop_fid, DIMS=rasterloop_dims, NB=rasterloop_nb, BNAMES=rasterloop_bnames, FNAME=rasterloop_fname ; Perform unsupervised classification. envi_doit, 'class_doit', FID=rasterloop_fid, DIMS=rasterloop_dims, POS=Lindgen(rasterloop_nb), $ METHOD=4, NUM_CLASSES=10, MIN_CLASSES=10, ITERATIONS=10, $ ISO_MERGE_DIST=5.0, ISO_SPLIT_STD=1, ISO_MIN_PIXELS=30, ISO_MERGE_PAIRS=2, CHANGE_THRESH=5.0, $ OUT_BNAME='Classification ('+File_basename(rasterloop_fname)+')', $ R_FID=classif01_fid, OUT_NAME=envi.getTemporaryFilename() envi_file_query, classif01_fid, DIMS=classif01_dims, NB=classif01_nb, BNAMES=classif01_bnames, FNAME=classif01_fname ; Output image classification to ENVI format. basename = (strsplit(file_basename(file), '.', /extract))[0] envi_output_to_external_format, FID=classif01_fid, DIMS=classif01_dims, POS=Lindgen(classif01_nb), $ /ENVI, OUT_NAME='C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\ENVI\IDL\Batch\RGB\25 image\10 Classes\'+basename+'_ISO_10.dat' endforeach for each file, ?????????? do begin ; Perform class statistics calculation. ENVI_Doit, 'Class_Stats_Doit', CLASS_FID=classif01_fid, CLASS_DIMS=classifi01_dims, CLASS_PTR = [0,1,2,3,4,5,6,7,8,9,10], $ COMP_FLAG = 0, FID=stats01_fid, REPORT_FLAG = 0, POS = Lindgen(classif01_nb), $ REP_BNAME='Class Statistics ('+File_basename(rasterloop_fname)+')', $ envi_file_query, stats01_fid, DIMS=stats01_dims, NB=stats01_nb, BNAMES=stats01_bnames, FNAME=stats01_fname ; Output image statistics to .txt format. basename = (strsplit(file_basename(file), '.', /extract))[0] envi_output_to_external_format, FID=stats01_fid, DIMS=stats01_dims, POS=Lindgen(stats01_nb), $ /ENVI, OUT_NAME='C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\ENVI\IDL\Batch\RGB\25 image\10 Classes\'+basename+'_ISO_10_class_report.txt' endforeach end

MariM

Veteran Member

Posts:2396
Veteran Member

07 Jun 2016 08:43 AM
I noticed a few things: - you do not need to output the result from the class_doit to ENVI format. The output file is already in ENVI format. Instead of output to a temporary file name: OUT_NAME=envi.getTemporaryFilename() Just specify the output file name you want in Out_name. From there, I don't see the need to restart the foreach loop. Why not just go ahead and calculate the statistics from the returned class_doit file as part of the initial loop? If you want to save a text file from the class_stats_doit, you can use REP_NAME. ENVI_OUTPUT_TO_EXTERNAL_FORMAT will not output a text file. It is intended to output a raster to various formats including ascii raster. These few changes should simplify your code and hopefully give you the output you need.

Deleted User

New Member

Posts:

07 Jun 2016 02:54 PM
Hi Mari, Thank you for your response. I tried following your recommendations but am still having trouble getting the code to run. I agree I do not need the temporary file name and, in fact, I have actually been looking for a way to get rid of that step, but every time I try and remove this component the code fails. Because I am trying to do these calculations in batch mode my goal is to tell the program the exact location to output my files, and have each individual output file named with it's basename followed by my designated text. That is why I have the extra step of defining the basename and designating a second Out_name command. (example below) basename = (strsplit(file_basename(file), '.', /extract))[0] envi_output_to_external_format, FID=classif01_fid, DIMS=classif01_dims, POS=Lindgen(classif01_nb), $ OUT_NAME='C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\ENVI\IDL\Batch\RGB\25 image\2 Classes\'+basename+'_ISO_10.dat' I am sure there is a better way of "telling" the program where to save the files, and how to name them so each is differentiated, but I am not that clever with IDL yet. At this point, if I try to replace the envi.getTemporaryFilename(), and just specify my convoluted work text posted above I get errors. If I only replace the Out_name with the directory and '+basename+...' it says "basename is not defined. If I do not use any of that and simply put in my directory location and a file name (i.e. Out_name= 'C:\ENVI\TEST\''_ISO10.dat') it looks as if it is running all the images, but it is only outputting one file because the output name is the same. It is basically continuously writing over the output and just giving me one result file even though I have 25 separate input files that need to be run. Beyond that issue, if I were to keep everything as is in regard to assigning the file to an ENVI format by using the temporary file (which makes the classification portion of the code work as it should), I still can not get the class_stats_doit function to work. I guess I am not sure how to tell it to use the returned class_doit file. I tried to designate the new FID as the output R_FID from the classification but that results in an error stating that my FID is undefined. I have tried numerous other ways of writing this part but obviously my IDL knowledge is rather limited and when I simply try to apply general logic based on the examples given in the ENVI help docs, I am not successful. Ugh. Below is the relevant portion of what I have for the code. It still includes the envi.getTemporaryFile() because as of now that is the only way I can make the code run and output all my files. Of course I have not even gotten to how to designate the individual names for my stats outputs. I apologize for the length and messiness of my response. I was really hoping to figure it out on my own, but after hours of trying I have decided I still need help. envi=ENVI() batchfiles = ["C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-10-12_38383C.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-10-12_39390A.JPG", $ "C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\Input Files\25_image RGB\2012-10-12_40392D.JPG"] foreach file, batchfiles do begin rasterloop = envi.OpenRaster(file) rasterloop_fid = ENVIRasterToFID(rasterLoop) envi_file_query, rasterloop_fid, DIMS=rasterloop_dims, NB=rasterloop_nb, BNAMES=rasterloop_bnames, FNAME=rasterloop_fname ; Perform unsupervised classification. envi_doit, 'class_doit', FID=rasterloop_fid, DIMS=rasterloop_dims, POS=Lindgen(rasterloop_nb), $ METHOD=4, NUM_CLASSES=2, MIN_CLASSES=2, ITERATIONS=10, $ ISO_MERGE_DIST=5.0, ISO_SPLIT_STD=1, ISO_MIN_PIXELS=30, ISO_MERGE_PAIRS=2, CHANGE_THRESH=5.0, $ OUT_BNAME='Classification ('+File_basename(rasterloop_fname)+')', $ R_FID=classif01_fid, OUT_NAME=envi.getTemporaryFilename() envi_file_query, classif01_fid, DIMS=classif01_dims, NB=classif01_nb, BNAMES=classif01_bnames, FNAME=classif01_fname ; Output image classification to ENVI format. basename = (strsplit(file_basename(file), '.', /extract))[0] envi_output_to_external_format, FID=classif01_fid, DIMS=classif01_dims, POS=Lindgen(classif01_nb), $ /ENVI, OUT_NAME='C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\ENVI\TEST\'+basename+'_ISO_2test.dat' ; Perform class statistics calculation. ENVI_Doit, 'Class_Stats_Doit', FID=classif01_fid, DIMS=dims, PTR = [0,1,2,3,4,5,6,7,8,9,10], $ COMP_FLAG = 0, R_FID=stats01_fid, REPORT_FLAG = 0, POS = Lindgen(classif01_nb), $ REP_NAME='C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\ENVI\TEST\''_statistics.txt'', $ envi_file_query, stats01_fid, DIMS=stats01_dims, NB=stats01_nb, BNAMES=stats01_bnames, FNAME=stats01_fname endforeach end

MariM

Veteran Member

Posts:2396
Veteran Member

08 Jun 2016 09:18 AM
You only need to pass the r_fid from the classification to the statistics calculation. In this case, r_fid would be the input class_fid and the fid would be the fid of the original image. It looks like you are attaching the file_query to your stats calculation here: ; Perform class statistics calculation. ENVI_Doit, 'Class_Stats_Doit', FID=classif01_fid, DIMS=dims, PTR = [0,1,2,3,4,5,6,7,8,9,10], $ COMP_FLAG = 0, R_FID=stats01_fid, REPORT_FLAG = 0, POS = Lindgen(classif01_nb), $ REP_NAME='C:\Users\ahudson-dunn\Desktop\CLASSIFICATION\ENVI\TEST\''_statistics.txt'', $ envi_file_query, stats01_fid, DIMS=stats01_dims, NB=stats01_nb, BNAMES=stats01_bnames, FNAME=stats01_fname That might be causing your issue with calculating the stats. I created a small example using our sample dataset that outputs a classification with a unique name and the stats file to a specified directory. This might help you alter your example to work. I would suggest you utilize a lot of PRINT statements when creating variables to determine if they are valid. ---------- PRO batch_envi_class_doit compile_opt IDL2 envi=ENVI(/current) ;search for files files=file_search('C:\temp\test', '*.dat') print, files ; Process each file foreach element, files do begin rasterloop = envi.OpenRaster(files) fid = ENVIRasterToFID(rasterLoop) ; Set the keywords. Perform the isodata and class stats calculation envi_file_query, fid, dims=dims, nb=nb, sname=sname, bname=bname pos = lindgen(nb) ;this increments the output file names to be unique - alter as needed inc = strtrim(string(n_elements(files)), 2) print, 'inc is ', inc ;set output location outfolder = 'c:\temp\test\' cd, outfolder ;name output files newname = file_basename(sname, '.dat') out_name = newname +'_iso_'+ inc +'.dat' print, 'out_name is ', out_name out_bname = 'Classification ('+newname+')' print, 'out_bname is ', out_bname rep_name = 'Class Statistics ('+newname +')'+ '_stats.txt' print, 'out rep_name is ', rep_name ; Run the DOIT envi_doit, 'class_doit', FID=fid, DIMS=dims, POS=pos, $ METHOD=4, NUM_CLASSES=10, MIN_CLASSES=10, ITERATIONS=10, $ ISO_MERGE_DIST=5.0, ISO_SPLIT_STD=1, ISO_MIN_PIXELS=30, ISO_MERGE_PAIRS=2, CHANGE_THRESH=5.0, $ OUT_BNAME=out_bname, R_FID=r_fid, OUT_NAME=out_name ENVI_Doit, 'Class_Stats_Doit', CLASS_FID=r_fid, CLASS_DIMS=dims, CLASS_PTR = [0,1,2,3,4,5,6,7,8,9,10], $ COMP_FLAG = 0, FID=fid, POS = pos, REP_NAME=rep_name endforeach END

Deleted User

New Member

Posts:

19 Jul 2016 03:58 PM
Thank you Mari! Sorry for the delay in responding. With your help and some tweaking to fit my required output I was able to get my code running properly. Yay!

You are not authorized to post a reply.

Forums Products IDL