16 Dec 2015 12:01 PM |
|
I'm trying to determine correlation values between 2 histograms that I have created with ENVI/IDL. The histograms I have created are made from single band images and my IDL script outputs stat reports but I am having trouble figuring out a process to run correlation between these 2 histograms.
The idea is to have one standard image histogram that I would check many other image histograms against in order to determine correlation between them. Can anyone push me in the right direction?
|
|
|
|
Zachary Norman Basic Member
Posts:173  
16 Dec 2015 01:28 PM |
|
Hi,
Do you know what kind of correlation you are trying to do between your two histograms? There is the P_CORRELATE function ( http://www.exelisvis.com/docs/P_CORRELATE.html) that you can use for two different data sets. If you want to use this, you would probably need to make sure that each histogram has the same binsize, min, and max value. Is that what you are looking for?
-Zach (VIS)
|
|
|
|
Deleted User New Member
Posts:19  
17 Dec 2015 11:01 AM |
|
That may be what I'm looking for. Is there a way to adjust the bin size using the envi_stast_doit function? Currently the bin sizes come out different but if I were able to make the bin size 1 that would be ideal.
This is my current histogram script:
pro HISTO_BATCH2
;
; First restore all the base save files.
;
envi, /restore_base_save_files
;
; Initialize ENVI and send all errors
; and warnings to the file batch.txt
;
envi_batch_init, log_file='C:\error_log.txt'
;
;Search all files in folder
;Create Input array containing file & path
file_array = file_search('D:\Test\Sum_Test\', '*dat', count=num_file)
for k=0, num_file-1 do begin
file=file_array(k)
print, file
;
openr, lun, /get_lun, file
;
; Open the input file
;Error handler if file not ex or wrong
envi_open_file, file, r_fid=fid
if (fid eq -1) then begin
envi_batch_exit
return
endif
;
;
free_lun, lun
; Get the samples, lines and # bands
; for the input file.
;
envi_file_query, fid, dims=dims, nb=nb
;
; Set the dims and pos to calculate
; statistics for all data (spatially and
; spectrally) in the file.
;
pos = [0]
;
;
stat_ext=".sta"
rep_ext=".stat_rep.txt"
;
;
stat_filename=string('D:\Test\Sum_Test\Final\'+file_basename(file))+stat_ext
report_filename=string('D:\Test\Sum_Test\Final\'+file_basename(file))+rep_ext
;
print, stat_filename
print, report_filename
;
; Calculate the basic statistics and the
; histogram for the input data file. Print
; out the calculated information& generate graphic plot
;
envi_doit, 'envi_stats_doit', fid=fid, pos=pos, $
dims=dims, comp_flag=2, dmin=dmin, dmax=dmax, $
mean=mean, STA_NAME=stat_filename,stdv=stdv, $
hist=hist, rep_name=report_filename
;
;print, STA_NAME
;print, rep_name
;
;print, 'Minimum ', dmin
;print, 'Maximum ', dmax
;print, 'Mean ', mean
;print, 'Standard Deviation ', stdv
;
;for i=0,nb-1 do begin
;print, 'Histogram Band ', i+1
;print, hist[*,i]
;endfor
print,'Printed ', k+1, ' out of ',num_file
;
;stop
;
endfor
;
;
end
|
|
|
|
Zachary Norman Basic Member
Posts:173  
|
Deleted User New Member
Posts:19  
18 Dec 2015 10:12 AM |
|
I'm having troubles getting that to work. Not getting errors but nothing happens. I'm far from an expert with scripting so I'm not really sure where to go from here.
I'm running it without the correlation part also. Just trying to get it to create a histogram with a binsize of 1 right now. Do I have an option of outputting a text report of the histogram somewhere? I don't see where I would include that in the description of the 'histogram' function
I have summed data bands that I am working with, which outputs a .dat file. Will the openraster and getdata functions still work with this file type?
|
|
|
|
Zachary Norman Basic Member
Posts:173  
18 Dec 2015 11:47 AM |
|
So you can actually do this within ENVI without needing to create a .dat file. To read teh .dat file, it really depends on how you created the data. You can modify my original example to get only a certain band from an image by changing the call to getdata like so :
e = envi(/headless)
file1 = "C:\Path\to\file"
file2 = "C:\Path\to\other\file"
raster1 = e.openraster(file1)
raster2 = e.openraster(file2)
data1 = raster1.GetData(BANDS = [0])
data2 = raster2.GetData(BANDS = [0])
;sum data
datasum = data1 + data2
;histogram set up
minval = 0
maxval = 255
binsize = 1
;calculate histograms (for 2D array)
hist1 = histogram(data1, MIN = minval, MAX = maxval, BISIZE = binsize)
hist2 = histogram(data2, MIN = minval, MAX = maxval, BISIZE = binsize)
And can you clarify what you mean by creating a text report? By default there is not an ability to automatically generate a text file, that is something you will have to write yourself, but here is a small example for how to do that:
;string array to print to a file
output = ['Array of strings', 'print them all out to the desktop!']
;this works for windows (USERPROFILE is an environment variable formateed as C:\Users\UserName)
file = GETENV('USERRPROFILE') + path_sep() + 'test_out.txt'
;get logical unit number (file ID basically) and open file for writing
; if you use the APPEND keyword below, then we add contents to the file,
; otherwise the file when openend with OPENW is erased first
OPENW, lun, file, /GET_LUN
;send our text to the file
printf, lun, output
;free the logical unit number
; this step is very important otherwise IDL will put a lock on the file until is it closed and restarted
FREE_LUN, lun
|
|
|
|
Deleted User New Member
Posts:19  
18 Dec 2015 12:16 PM |
|
Thanks for the help. What if I wanted to sum the 3 bands from the first image. Would it just look like this?
data1 = raster1.GetData(BANDS = [0])
data2 = raster1.GetData(BANDS = [1])
data3 = raster1.GetData(BANDS = [2])
datasum = data1+data2+data3
Then I could create a histogram from 'datasum'?
Another question is where is this histogram created? I would like to output the histogram data as a text file (bin number, pixel count, percentage of pixels; just like the statistics function in ENVI outputs) but if I didn't do that where would I view this created histogram? Is that something I have to write into the IDL script as well? I'm really more interested in a text report than just visuals though. This is something the 'envi_stats_doit' function had that was very useful but it doesn't seem like I can control the binsize using it.
|
|
|
|
Zachary Norman Basic Member
Posts:173  
18 Dec 2015 12:54 PM |
|
For your first question, yep! That's just how you would add the bands together from different rasters. As a follow up to that, I haven't ever seen the text data format for the ENVI classic statistics, but you should be able to determine that from the result of the histogram. The histogram function will return an array of the counts for each bin (doesn't return the information regarding the bin location). However, if you just have an image, then you can figure out your bin location by thinking about the data because your 'bins' will be pixel values. So if you have a binsize of 1 and you have 256 bins, then you are gathering all of the pixel values between 0 and 255 (IDL is 0 based).
If you want the actual bin locations, then you can do:
hist_data = histogram(data, MIN = min, MAX = max, BINSIZE = binsize, LOCATIONS = xbin)
the last keyword, LOCATIONS will fill the variable xbin with the corresponding x locations. With this information you can create plot like:
p = plot(xbin, hist_data)
There is a complete example on this web page for the histogram and the cumulative histogram, which I think you might be looking for:
http://www.exelisvis.com/docs/HISTOGRAM.html
-Zach (VIS)
|
|
|
|
Deleted User New Member
Posts:19  
18 Dec 2015 02:31 PM |
|
Currently, just working on summing bands from a single raster and creating a histogram my script looks like this:
e = envi(/headless)
file1 = "D:\Test\09bcd14205_904_30_8bit_rgb.tif"
;file2 = "C:\Path\to\other\file"
raster1 = e.openraster(file1)
data1 = raster1.GetData(BANDS = [0])
data2 = raster1.GetData(BANDS = [1])
data3 = raster1.GetData(Bands = [2])
;sum data
datasum = data1 + data2 + data3
;histogram set up
minval = 0
maxval = 765
binsize = 1
;calculate histograms (for 2D array)
hist1 = histogram(datasum, MIN = minval, MAX = maxval, BINSIZE = binsize, LOCATIONS = xbin)
p=plot(xbin,hist1)
;
End
This returns a histogram but the data value range is from 0-255. If the data sum works correctly it should be 0-765 (255+255+255). So I'm not sure this sum is working correctly. I guess I really need to get a text report with bin numbers and counts to confirm that the sum is working the way I need it to. A visual plot isn't sufficient for what I need to do.
Unless this method does not add bin numbers but only bin counts? That would be fine but comparing this histogram to one generated in ENVI using the sum data bands and statistics tool it does not seem to be correct. I'm not even sure what the y-axis numbers are
|
|
|
|
Zachary Norman Basic Member
Posts:173  
20 Dec 2015 02:57 PM |
|
As far as your results from adding together your bands, my guess is that the actual data range is not going to be 0-765 for three bands which range from 0 to 255. The reasoning for this is that you would only have 0 to 755 if your bands all have 255 in the exact same pixel locations, which will vary from image to image. If you want to examine the numbers, you can always just print out the information to the IDL console, provided there aren't too many bins (if there are too many elements then you won't be able to see the top of the data set). To do that you would just type:
print, transpose([[indgen(n_elements(hist1))],[hist1]])
You can also add that to the bit of code that will print out to a text file as long as you replace print with "printf, lun, transpose([[indgen(n_elements(hist1))],[hist1]])". That will print your bin number and the corresponding count for that bin to either the IDL console or the text file from the previous example for printing to a text file.
|
|
|
|
Deleted User New Member
Posts:19  
21 Dec 2015 11:43 AM |
|
I should have mentioned that my image does have pixel values that contain the value 255 in the same locations. This is a custom image I have made in order to test the scripts I use and ensure it is doing what I want. Summing the bands with 'envi_sumdata_doit' returns the proper summed values (Data range from 86-765) but this method you showed me does not. The max value of pixels seems to be ~255 by examining the histogram.
The print, transpose function also returns a syntax error and does not work. I ensured the correct number and placement of brackets but it still does not work.
|
|
|
|
Zachary Norman Basic Member
Posts:173  
22 Dec 2015 09:38 AM |
|
Looks like I had a missing parenthesis in my example when I typed it, which I fixed on the post and it should work now. My guess for the data range you have is that you are using byte data for your arrays(To check this you can do "help, added"). If you use byte arrays and add them together, then the result will also be a byte array. This is important to know because the maximum data range for byte data is 0 to 255. In order to get the actual data range, you will need to convert your data to a different type. Below is an example to show you how to convert your byte data to integers before adding the arrays together using the FIX function:
datasum = fix(data1) + fix(data2) + fix(data3)
Converting to integers expands your data range to -32,768 to +32,767. If you need to have a larger data range than this, you can convert to other data types as well. Here is a link to a page with all of the IDL datatypes and how you can convert between them:
http://www.exelisvis.com/.../IDL_Data_Types.html
-Zach (VIS)
|
|
|
|