I believe if you pass a mask to ENVI_STATS_DOIT, it will exclude those pixels from the total pixels considered. Have you tried passing the mask (not applying it to the data) using the m_fid, m_pos option? Alternately, you can create an ROI that excludes those pixels and pass the ROI using the first element of the DIMS.

Hi Mari, Thank you for the reply. I have attempted to mask out the region but of a single image with varied success. The main issue here is that this .pro is a batch process looking at hundreds of images where the area that needs to be masked out varies in placement and size. Can I create a mask based on a range of values (which should stay the same) and have it apply to each image independently? Also, I am not sure what you mean by "pass a mask" is it possible to see an example of this? Here is the current set-up of this particular portion of the code: envi_doit, 'envi_stats_doit', dims=rasterloop_dims, fid=r_fid, pos=[0], $ comp_flag=2, dmin=dmin, dmax=dmax, mean=mean, stdv=stdv, hist=hist, $ rep_name=rep_name, report_flag=0 ;Use these arrays defined above to 'collect' the stats for each file that comes through the loop. This also defines the mathematical procedure and column headers for the .csv output. all_min[index,*] = dmin all_max[index,*] = dmax hist_total[index,*] = hist[1]/total(hist)*100 stats_name[index,*] = out_name stats_name2[index,*] = image_name + '.JPG'

Using a mask works just like it does in the GUI where you select a file and then select a mask band. The mask is "passed in" to the DOIT using the m_fid and m_pos keywords: https://www.harrisgeospatial.com/docs/envi_stats_doit.html The mask building routine for classic is not documented so it may be easier to pass in an ROI using the DIMS array. You could use ROI_THRESH_DOIT to create an ROI of your areas of interest and then pass in the ROI using the first element of the DIMS array. https://www.harrisgeospatial.com/docs/ROI_THRESH_DOIT.html

Here is an example using ROI thresholds: ----------------------------- pro test_roi_stats compile_opt idl2 ;open the sample dataset and query file='C:\Program Files\Exelis\ENVI53\classic\data\bhdemsub.img' envi_open_file, file, r_fid=fid envi_file_query, fid, dims=dims pos=[0] ;create an ROI using the ROI thresholds and save the ROI envi_doit, 'roi_thresh_doit', fid=fid, pos=pos, dims=dims, /no_query, $ max_thresh=2000, min_thresh=1500, roi_color=5, roi_name='test.roi', roi_id=roi_id envi_save_rois, 'c:\testing\dataout\test.roi', roi_id ;set up the statistics DIMs to use the ROI pointer stats_dims=[envi_get_roi_dims_ptr(roi_id[0]), 0, 0, 0, 0] ;calculate stats on the ROI regions only envi_doit, 'envi_stats_doit', fid=fid, pos=pos, dims=stats_dims, $ comp_flag=2, mean=roi_mean, stdv=roi_stdv, hist=roi_hist print, 'ROI mean is ', roi_mean print, 'ROI stdv is ', roi_stdv print, 'Image hist is ', roi_hist ;calculate stats on the full image DIMS envi_doit, 'envi_stats_doit', fid=fid, pos=pos, dims=dims, $ comp_flag=2, mean=roi_mean_full, stdv=roi_stdv_full, hist=roi_hist_full print, 'Image mean is ', roi_mean_full print, 'Image stdv is ', roi_stdv_full print, 'Image hist ', roi_hist_full end

What is the data type of the image you are using? If they are floating point values, you may need to be more specific and use -251.0 or something similar.

I got that one figured out :)

Bigger issue unfortunately: I see how the ROI and stats calculations work and I think it could be a good solution to my problem but the real challenge comes with how I fit these commands in with the current program. I have pasted my code below. it must run in batch mode as I am processing hundreds of images in one run. You will see that it currently looks at a two band image, performs a band math function determining which pixels meet a set criteria and then creates an array where all pixels that do meet it are assigned a 1 and all others are given a zero. Finally, envi stats are calculated on the new output and that histogram is used to calculate what percentage of the total number of pixels are equal to 1. NOTE: This worked great until I began getting images with this blacked out box on them. Now I have to figure out a way to not mess up my current code but still somehow remove "bad" pixels from the total count before the percentage calculation is performed. As I see it, the ROI_THRESHOLD command will have to run before band math, and it needs to only be based on band 1 (pos = 0). Of course when I try this I get the following error. Array dimensions must be greater than 0, Execution halted at: ENVI_ROI_PTR_FROM_ID 259, ENVI_SAVE_ROIS 992 Once this component does work, the next issue is making sure the ENVI_STATS commands are setup so they are using the proper outputs for the calculations. Here is my (non-working) attempt: PRO VegetativeCover_Percent_v2 compile_opt IDL2 envi=ENVI(/current) ; Search for files. batchfiles=file_search('C:\Users\test', '*.dat') ;print, batchfiles ;Establish location of output .csv and create arrays to hold the stats that will be stored and eventually written out to this .csv ;the number of elements in the arrays should match the number of files report_name = 'C:\Users\ahudson-dunn\stats.csv' all_min = fltarr(n_elements(batchfiles)) all_max = all_min hist_total = all_min stats_name = strarr(n_elements(batchfiles)) stats_name2 = strarr(n_elements(batchfiles)) ; Process each file. foreach file, batchfiles, index do begin rasterloop = envi.OpenRaster(file) rasterloop_fid = ENVIRasterToFID(rasterLoop) ; Set the keywords for the ... percent calculation. envi_file_query, rasterloop_fid, DIMS=rasterloop_dims, NB=rasterloop_nb, BNAMES=rasterloop_bnames, FNAME=rasterloop_fname, ROI_ID=rasterloop_roi_id ; Increments the output file names to be unique. inc = strtrim(string(n_elements(batchfiles)), 2) ; print, 'Input file number is ', inc ; Name output files. newname = file_basename(file, '.dat') image_name = newname.Remove(-15) out_name = image_name +'_VegetativeCoverPercent' out_bname = 'Vegetative Cover Percent ('+newname+')' ; Create an ROI using only BAND 1 and designated threshold values for the "blacked out box" envi_doit, 'roi_thresh_doit', fid=rasterloop_fid, pos=[0], dims=rasterloop_dims, /no_query, $ max_thresh= -251, min_thresh= -251, roi_color=2, roi_name='test.roi', roi_id=rasterloop_roi_id ; Mari -- not sure how this will work for multiple images as will need to save them independtly for each envi_save_rois, 'C:\Users\ahudson-dunn\test\roi\test.roi', roi_id ;set up the statistics DIMs to use the ROI pointer stats_dims=[envi_get_roi_dims_ptr(roi_id[0]), 0, 0, 0, 0] ; Perform the calculation for .. percent. envi_doit, 'math_doit',$ FID= [rasterloop_fid,rasterloop_fid], $ DIMS=rasterloop_dims, $ POS=[0,1], $ EXP='b2 gt 30 and b3 lt 1.02', $ OUT_BNAME=out_bname, R_FID=r_fid, OUT_NAME=out_name ;calculate stats on the ROI regions only envi_doit, 'envi_stats_doit', fid=fid, pos=[0], dims=stats_dims, $ comp_flag=2, mean=roi_mean, stdv=roi_stdv, hist=roi_hist print, 'ROI mean is ', roi_mean print, 'ROI stdv is ', roi_stdv print, 'Image hist is ', roi_hist ; Perform statistical calculation on full image output. envi_doit, 'envi_stats_doit', dims=rasterloop_dims, fid=r_fid, pos=[0], $ comp_flag=2, dmin=dmin, dmax=dmax, mean=mean, stdv=stdv, hist=hist, $ rep_name=rep_name, report_flag=0 ; Use these arrays defined above to 'collect' the stats for each file that comes through the loop. all_min[index,*] = dmin all_max[index,*] = dmax hist_total[index,*] = hist[1]/total(hist)*100 hist_total_masked[index,*] = hist[1]/roi_hist*100 stats_name[index,*] = out_name stats_name2[index,*] = image_name + '.JPG' print, hist_total print, hist_total_masked ; print, stats_name ; print, stats_name2 endforeach end

I am not sure what this statement is supposed to do but it causes my session to crash: image_name = newname.Remove(-15) So I removed it. I believe the error with the dims is because you are not passing the correct ROI_ID. When you run the threshold, you set the ROI_ID to be called 'rasterloop_roi_id' so this is what needs to be passed to the DIMS: stats_dims=[envi_get_roi_dims_ptr(rasterloop_roi_id[0]), 0, 0, 0, 0]

So I believe that I am almost there with this code. Currently it is calculating correctly but I am getting an error message saying the ENVI ROIS are undefined. I am guessing it is some small coding error I am overlooking. Would you mind taking a quick look to see if anything jumps out at you? PRO VegetativeCover_Percent_v2 compile_opt IDL2 envi=ENVI(/current) ; Search for files. batchfiles=file_search('C:\Users\ahudson-dunn\Desktop\RGB testing\Quadrat\Set 2\test', '*.dat') ;print, batchfiles ;Establish location of output .csv and create arrays to hold the stats that will be stored and eventually written out to this .csv ;the number of elements in the arrays should match the number of files report_name = 'C:\Users\ahudson-dunn\Desktop\RGB testing\Quadrat\Set 2\test\VegetativeCoverPercent_stats2.csv' all_min = fltarr(n_elements(batchfiles)) all_max = all_min hist_total = all_min hist_total_masked = all_min stats_name = strarr(n_elements(batchfiles)) stats_name2 = strarr(n_elements(batchfiles)) ; Process each file. foreach file, batchfiles, index do begin rasterloop = envi.OpenRaster(file) rasterloop_fid = ENVIRasterToFID(rasterLoop) ; Set the keywords for the Cover crop percent calculation. envi_file_query, rasterloop_fid, DIMS=rasterloop_dims, NB=rasterloop_nb, BNAMES=rasterloop_bnames, FNAME=rasterloop_fname, ROI_ID=rasterloop_roi_id ; Increments the output file names to be unique. inc = strtrim(string(n_elements(batchfiles)), 2) ; Name output files. newname = file_basename(file, '.dat') image_name = newname.Remove(-15) out_name = image_name +'_VegetativeCoverPercent' out_bname = 'Vegetative Cover Percent ('+newname+')' ; Create an ROI using only BAND 1 and designated threshold values for the "red box" envi_doit, 'roi_thresh_doit', fid=rasterloop_fid, pos=[0], dims=rasterloop_dims, /no_query, $ max_thresh= -251, min_thresh= -251, roi_color=2, roi_name='test.roi', roi_id=rasterloop_roi_id envi_save_rois, 'C:\Users\ahudson-dunn\Desktop\RGB testing\Quadrat\Set 2\test\roi\test.roi', rasterloop_roi_id ;set up the statistics DIMs to use the ROI pointer stats_dims=[envi_get_roi_dims_ptr(rasterloop_roi_id[0]), 0, 0, 0, 0] ; Perform the calculation for .. percent. envi_doit, 'math_doit',$ FID= [rasterloop_fid,rasterloop_fid], $ DIMS=rasterloop_dims, $ POS=[0,1], $ EXP='b1 gt 24 and b2 lt 1.02', $ OUT_BNAME=out_bname, R_FID=r_fid, OUT_NAME=out_name ;calculate stats on the ROI regions only envi_doit, 'envi_stats_doit', fid=r_fid, pos=[0], dims=stats_dims, $ comp_flag=2, mean=roi_mean, stdv=roi_stdv, hist=roi_hist print, 'Image hist is ', roi_hist ; Perform statistical calculation on raster output. The only relevant output here will be the histogram. envi_doit, 'envi_stats_doit', dims=rasterloop_dims, fid=r_fid, pos=[0], $ comp_flag=2, dmin=dmin, dmax=dmax, mean=mean, stdv=stdv, hist=hist, $ rep_name=rep_name, report_flag=0 ; Use these arrays defined above to 'collect' the stats for each file that comes through the loop. This also defines the mathematical procedure and column headers for the .csv output. all_min[index,*] = dmin all_max[index,*] = dmax hist_total[index,*] = hist[1]/total(hist)*100 hist_total_masked[index,*] = hist[1]/(total(hist) - roi_hist)*100 stats_name[index,*] = out_name stats_name2[index,*] = image_name + '.JPG' print, hist_total print, hist[1] print, hist_total_masked

At what point does it fail in the code? What is the full error? I would do some print statements on the ROIs that are created to make sure they are being created and valid. You don't actually have to save the ROI you create but you can. If you do not save the ROI from thresholding, it is held in memory until deleted or ENVI is closed.

Removing the "envi_save_rois" statement removed the error. I am not sure how I had it setup wrong but I do not need to save them for it to work so removing it is fine. With that gone the program runs great and outputs the correct calculations - YAY! The only remaining concern is that it hangs up on any images that do not have the masked pixels. This is due to the fact that no pixels are found meeting that threshold range. Is there a way to to tell it to still move forward in the code but ignore the ROI_threshold if no pixels are found that meet that criteria? I can run the images as separate batches (those with the box and those without) but that is not ideal. Thanks for all your help!

You could test if the ROI_ID returned from ROI_THRESH_DOIT is valid. Something like: help, roi_id if roi_id eq !NULL then begin stats_dims=rasterloop_dims endif else begin stats_dims=[envi_get_roi_dims_ptr(roi_id[0]), 0, 0, 0, 0] endelse

Hi. I have been testing the suggestion you gave me and am still unable to get the code to run through when the ROI_THRESH_DOIT command results in values outside of the range specified. In addition to getting past the above issue, I believe I will also need some kind of statement specifying which histogram calculation should be used depending on the result of the thresh_doit command. For example, if the ROI_THRESH_DOIT command finds pixels that fall within the specified range then it will need to output the array based on "hist_total_masked[index,*]". Following similar logic, if ROI_THRESH_DOIT does not find pixels in the range specified, then the final array should be based on the "hist_total[index,*]" Does that make sense? I have pasted the code below: PRO VegetativeCover_Percent_FOR_MASKING compile_opt IDL2 envi=ENVI(/current) ; Search for files. batchfiles=file_search('C:\Users\', '*.dat') ; print, batchfiles ; Establish location of output .csv and create arrays to hold the stats that will be stored and eventually written out to this .csv ; the number of elements in the arrays should match the number of files report_name = 'C:\Users\VegetativeCoverPercent_stats.csv' all_min = fltarr(n_elements(batchfiles)) all_max = all_min hist_total_masked = all_min stats_name = strarr(n_elements(batchfiles)) stats_name2 = strarr(n_elements(batchfiles)) ; Process each file. foreach file, batchfiles, index do begin rasterloop = envi.OpenRaster(file) rasterloop_fid = ENVIRasterToFID(rasterLoop) ; Set the keywords for the Cover crop percent calculation. envi_file_query, rasterloop_fid, DIMS=rasterloop_dims, NB=rasterloop_nb, BNAMES=rasterloop_bnames, FNAME=rasterloop_fname, ROI_ID=rasterloop_roi_id ; Increments the output file names to be unique. inc = strtrim(string(n_elements(batchfiles)), 2) ; Name output files. newname = file_basename(file, '.dat') image_name = newname.Remove(-15) out_name = image_name +'_VegetativeCoverPercent' out_bname = 'Vegetative Cover Percent ('+newname+')' ; Create an ROI using only BAND 1 and designated threshold values for the "red box" envi_doit, 'roi_thresh_doit', fid=rasterloop_fid, pos=[0], dims=rasterloop_dims, /no_query, $ max_thresh= -251, min_thresh= -251, roi_color=2, roi_name='redbox.roi', roi_id=rasterloop_roi_id help, rasterloop_roi_id if rasterloop_roi_id eq !NULL then begin dims=rasterloop_dims endif else begin stats_dims=[envi_get_roi_dims_ptr(rasterloop_roi_id[0]), 0, 0, 0, 0] endelse ; set up the statistics DIMs to use the ROI pointer ; stats_dims=[envi_get_roi_dims_ptr(rasterloop_roi_id[0]), 0, 0, 0, 0] ; Perform the calculation for Cover crop percent. Number of "rasterloop_fid" texts should correspond to the number of bands your image has. ; CHANGE THRESHOLD VALUES HERE envi_doit, 'math_doit',$ FID= [rasterloop_fid,rasterloop_fid], $ DIMS=rasterloop_dims, $ POS=[0,1], $ EXP='', $ OUT_BNAME=out_bname, R_FID=r_fid, OUT_NAME=out_name ; calculate stats on the ROI (the redbox) regions only. This gives us the number of pixels that need to be subtracted from the image total. envi_doit, 'envi_stats_doit', fid=r_fid, pos=[0], dims=stats_dims, $ comp_flag=2, mean=roi_mean, stdv=roi_stdv, hist=roi_hist ; print, 'ROI pixel # (redbox) ', roi_hist ; Perform statistical calculation on the new vegetated/non-vegetated raster output. This gives us the total number of pixels for the image as well as the number of just vegetated pixels. envi_doit, 'envi_stats_doit', dims=rasterloop_dims, fid=r_fid, pos=[0], $ comp_flag=2, dmin=dmin, dmax=dmax, mean=mean, stdv=stdv, hist=hist, $ rep_name=rep_name, report_flag=0 ; Use these arrays defined above to 'collect' the stats for each file that comes through the loop. This also defines the mathematical procedure and column headers for the .csv output. all_min[index,*] = dmin all_max[index,*] = dmax hist_total[index,*] = hist[1]/total(hist)*100 hist_total_masked[index,*] = hist[1]/(total(hist) - roi_hist)*100 stats_name[index,*] = out_name stats_name2[index,*] = image_name + '.JPG' print, 'Vegetated pixel # (B1) ', hist[1] print, 'Total number of pixels (B1+B2) ', hist print, 'Vegetated percent (mask) ', hist_total_masked ; print, stats_name ; print, stats_name2 endforeach