X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 24 Jan 2018 09:12 AM by  Careli Caballero
Radiometric correction- WorldView 3 images
 20 Replies
Sort:
You are not authorized to post a reply.
Page 1 of 212 > >>
Author Messages

Prabha Rupasinghe



New Member


Posts:22
New Member


--
20 Oct 2017 02:16 PM
    Hi,
    I have a WorldView 3 image (Standard 2A product, 8 bands) and I am trying to do radiometric correction with ENVI 5.4.1. The image do not have the gain and offset values and Digital Globe said they do not give these data along with the image. Therefore, I can not do the regular radiometric correction for this image. Can anyone help me with this problem?

    Thank you in advance!
    Prabha

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    23 Oct 2017 07:35 AM
    Why would DigitalGlobe not provide gains and offsets with the data so you can calibrate it? In the WV-3 data that I have worked with, they do provide this information in the .IMD files:
    absCalFactor = 1.432870e-02;
    effectiveBandwidth = 4.050000e-02;

    ENVI will use this information to perform the Radiometric Calibration if you open the data using the .TIL file.

    If your data do not have the gain and offset information to perform Radiometric Calibration, I wold ask DG to provide it like other data providers do. I have also run across some of their calibration information here:
    https://dg-cms-uploads-production.s3.amazonaws.com/uploads/document/file/207/Radiometric_Use_of_WorldView-3_v2.pdf

    You can then save your data as ENVI format and use View Metadata->Edit Metadata and add in the gains and offset values to the header so the data can be calibrated to radiance. You will also need the Irradiance values and Earth-sun distance if you want to calibrate to TOA reflectance.

    Prabha Rupasinghe



    New Member


    Posts:22
    New Member


    --
    23 Oct 2017 10:03 AM
    Thank you for the reply!

    I opened the .TIL image using open as-> optical sensors -> Digital Globe -> World View 3. My metadata file have absCalFactor = -9.990000e+02 and effectiveBandwidth = 3.810000e-02. But in the metadata -> spectral table, radiance gains, reflectance gains, reflectance offset and bad bands fields are empty. It only has values for radiance offsets and Irradiance. When I try to perform the radiometric correction, it sends me an error message saying "Calibration requires gain and offset for each band".

    I contacted the re-seller where we bought the image and they resend the images three times, but the images were all same. I also contacted DG directly and their reply said "There is no calibration issues. As I mentioned before .imd files doesn't include gain and offset values, they are auto generated with the same format for every order."

    The calibration information you send have some values for gain and offset, I'll try to enter them into the image file as you instructed, hopefully it will work.. But I do not understand and just so curious why these values do not automatically appear in the metadata as other sensors do..


    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    23 Oct 2017 10:57 AM
    I think it would be a good idea to have a look at the product you received. Perhaps there was a change in format and ENVI is unable to read the calibration information.
    Could you please submit this question through support services? If you are a student using a lab license, it should be submitted by the license administrator.
    Alternatively, if you have or know the gains and offsets to use, I would save the file to ENVI format and add the information to the header as described previously. This may be the work around if there is an issue reading the metadata.

    Prabha Rupasinghe



    New Member


    Posts:22
    New Member


    --
    18 Nov 2017 09:57 AM
    Hi, Thank you so much your your advise. I was able to get DRA off, 16 bit product and it works perfectly for the radiometric correction. But I am having a new trouble now. My image have 8 bands (coastal blue, blue, green, yellow, red, red edge, NIR 1 and NIR2), no SWIR bands. So I can not perform the FLAASH correction. I tried QUAC correction as WV2, RGB, NIR and Landsat options as it does not provide option for WV3. But every time, I get blank pixels in the output, it appears like wholes in the image. It is worst when I used WV2 option. Is there any other atm correction method or option in QUAC correction that might work for this dataset? Thank you!

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    20 Nov 2017 05:23 AM
    You can still process the data in FLAASH using Multispectral Settings. You will not be able to calculate aerosols or do a water retrieval so those should be off.
    For QUac, you can use Unknown sensor. Be sure to create a mask of background pixels and use it when processing in FLAASH. If you do not, the background will be considered an endmember and can affect the results.

    Prabha Rupasinghe



    New Member


    Posts:22
    New Member


    --
    19 Dec 2017 10:25 AM
    Hi, Thank you for your reply. I was able to process FLAASH without aerosol retrieval, but still there are lots of blank pixels, I just noticed that some blank pixels are created during the radiometric correction as well. Then there are lots of blank pixels after the atmospheric correction. I tried to apply a blank pixel mask, but FLAASH did not give the option to apply mask. I tried to use mask for the QUAC correction and process as unknown sensor. But it gives me error message saying it can't extract the endmembers and ask to try a different sensor. I tried several other options, but received the same error message. Can you please help me to fix this problem? Thank you!

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    19 Dec 2017 11:45 AM
    What do you mean by 'blank'? If you put your cursor over one of these pixels, what value does it show? It is possible that they are negative or very low. This occurs frequently in areas of water and shadow. Can you calculate and post the basic statistics of the result from Radiometric Calibration (using FLAASH Settings) and then after FLAASH correction?

    Prabha Rupasinghe



    New Member


    Posts:22
    New Member


    --
    19 Dec 2017 12:19 PM
    When opened true color, the cursor value shows as 'No Data' for all 8 bands after the radiometric correction. They appear as holes in the image. Most of these pixels are in shaded areas and few in water.
    Basic stat after radiometric correction:
    Min Max Mean StdDev
    Band 1 0.009543 60.408104 0.644900 1.085968
    Band 2 0.001946 62.044239 1.204353 1.297036
    Band 3 0.015647 40.522682 1.600582 1.074362
    Band 4 0.016796 34.853352 1.458623 1.078179
    Band 5 0.012294 49.201557 1.735617 1.628500
    Band 6 0.016195 33.391163 5.387143 1.929354
    Band 7 0.006135 33.457809 10.306771 3.883123
    Band 8 0.007536 24.913122 8.170882 2.852122

    After the FLAASH correction, there are more of these pixels, but they appear in blue color. Cursor value is 0 for all 8 bands. All most all the shaded pixels and most of the water pixels have 0 after the FLAASH.
    Basic stat after the FLAASH correction:
    Min Max Mean StdDev
    Band 1 -2235 24847 -1552.295251 535.641585
    Band 2 -1310 17972 -784.258209 357.980847
    Band 3 -851 11483 -300.630367 262.516010
    Band 4 -598 10214 -138.393889 256.406434
    Band 5 -508 15120 4.129498 385.559903
    Band 6 -584 12530 1335.838476 803.633796
    Band 7 -707 13455 3498.917128 1870.665882
    Band 8 -664 15441 4359.234572 2172.485202


    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    19 Dec 2017 12:35 PM
    Your means after FLAASH correction are mostly negative. Can you post your template.txt that is created after FLAASH processing?

    Prabha Rupasinghe



    New Member


    Posts:22
    New Member


    --
    19 Dec 2017 12:49 PM
    Below is the template.txt file;
    ;
    ;ENVI FLAASH PARAMETERS TEMPLATE (5.4.1)
    ;Written Tue Dec 19 14:35:13 2017
    ;
    ; Project Parameters
    enviacc.prj.radiance_file = I:\Worldview 3 new data\processed\preprocessing_Dec_2017\radiometric_correction.dat
    enviacc.prj.reflect_file = I:\Worldview 3 new data\processed\preprocessing_Dec_2017\FLAASH_correction2
    enviacc.prj.filter_func_file = C:\Program Files\Harris\ENVI54\classic\filt_func\worldview3.sli
    enviacc.prj.filter_func_file_index = 1
    enviacc.prj.water_band_choice = 1.13
    enviacc.prj.red_channel = 5
    enviacc.prj.green_channel = 3
    enviacc.prj.blue_channel = 1
    enviacc.prj.water_abs_channel = 0
    enviacc.prj.water_ref_channel = 0
    enviacc.prj.kt_upper_channel = 0
    enviacc.prj.kt_lower_channel = 5
    enviacc.prj.kt_cutoff = 0.0800
    enviacc.prj.kt_ratio = 0.5000
    enviacc.prj.cirrus_channel = 0
    enviacc.prj.water_retrieval = 0
    enviacc.prj.modtran_directory = I:\Worldview 3 new data\processed\preprocessing_Dec_2017\New folder\
    ;
    ; MODTRAN Parameters
    enviacc.modtran.visvalue = 40.0000
    enviacc.modtran.f_resolution = 5.0000
    enviacc.modtran.day = 12
    enviacc.modtran.month = 7
    enviacc.modtran.year = 2016
    enviacc.modtran.gmt = 16.6106
    enviacc.modtran.latitude = 42.7668
    enviacc.modtran.longitude = -80.5959
    enviacc.modtran.sensor_altitude = 617.0000
    enviacc.modtran.ground_elevation = 0.0000
    enviacc.modtran.view_zenith_angle = 180.0000
    enviacc.modtran.view_azimuth = 0.0000
    enviacc.modtran.atmosphere_model = 2
    enviacc.modtran.aerosol_model = 1
    enviacc.modtran.multiscatter_model = 2
    enviacc.modtran.disort_streams = 8
    enviacc.modtran.co2mix = 390.0000
    enviacc.modtran.water_column_multiplier = 1.0000
    ;
    ; Image Parameters
    enviacc.img.nspatial = 3795
    enviacc.img.nlines = 2773
    enviacc.img.data_type = 4
    enviacc.img.margin1 = 0
    enviacc.img.margin2 = 0
    enviacc.img.nskip = 0
    enviacc.img.pixel_size = 0.8800
    enviacc.img.sensor_name = WorldView-3
    ;
    ; Analysis Parameters
    enviacc.ana.aerosol_scaleht = 1.5000
    enviacc.ana.use_adjacency = 1
    enviacc.ana.output_scale = 10000.0000
    enviacc.ana.output_diagnostic_files = 0
    enviacc.ana.auto_save_template = 1
    enviacc.ana.polishing_res = 9
    enviacc.ana.aerosol_retrieval = 0
    enviacc.ana.calc_wl_correction = 0
    enviacc.ana.reuse_modtran_calcs = 0
    enviacc.ana.use_square_slit_function = 0
    enviacc.ana.convolution_method = fft
    enviacc.ana.use_tiling = 1
    enviacc.ana.tile_size = 100.0000
    ;
    ; GParameters Parameters
    enviacc.gparameters.gfact = 0
    enviacc.gparameters.gval = 0.0000
    enviacc.gparameters.use_lr_method = 0
    enviacc.gparameters.multikernel = 1
    enviacc.gparameters.autoroi = auto
    enviacc.gparameters.selarg = a
    ;
    ; Spectral Parameters
    enviacc.spc.wavelength_units = nanometer
    enviacc.spc.lambda = [
    425.0000, 480.0000, 545.0000, 605.0000, 660.0000, 725.0000, 832.5000,
    950.0000]
    enviacc.spc.fwhm = [
    50.000000, 60.000000, 70.000000, 40.000000, 60.000000, 40.000000, 125.000000,
    180.000000]
    enviacc.img.p_input_scale = [
    1.0000, 1.0000, 1.0000, 1.0000, 1.0000, 1.0000, 1.0000,
    1.0000]

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    19 Dec 2017 01:19 PM
    You have me stumped without having access to the data. It is not unusual that you have negative values in your FLAASH results in dark areas so it might be correct. I just don't see anything inherently wrong in your FLAASH settings or input stats. Your output stat means look very low in the first few bands. If your image looks fairly clear without a lot of water vapor or haze, you might try setting the Initial visibility to 80-100km to boost the visible bands.
    You mentioned you would get an error in Quac regarding the inability to find enough endmembers in the image. Is your image very homogeneous? Quac does require that an image have a minimum of 10 unique materials to characterize the image accurately.

    Prabha Rupasinghe



    New Member


    Posts:22
    New Member


    --
    28 Dec 2017 12:09 PM
    My image is very clear, no visible haze at all and covers a large area, So I always tried to do a smaller area to save processing time. I tried everything you suggested, it reduced the No Data and 0 value pixels, but I still had loads of them.. I was able to use QUAC when I used the entire image, but there were many No Data pixels (more than 80% of the image).

    I managed to do something, but I am not sure whether that method is acceptable. I checked the shadow areas and they have very low pixel values in the original image. So I multiplied the entire image by 10 using band math (float (b1)*10). Then transferred gain and offset from the original image to the resultant image and did radiometric correction with FLAASH settings. Then did FLAASH correction with initial visibility of 40 km. There is no more No data or 0 value pixels in the image and it looks so good! The histograms after the FLAASH is little weird though, below are the basic stat;
    Basic Stats Min Max Mean StdDev
    Band 1 -32768 32767 2188.468643 3274.561761
    Band 2 -32768 32767 2472.035491 3252.697032
    Band 3 -32768 32767 3406.993476 2913.007473
    Band 4 -32768 32767 3561.009254 3400.422862
    Band 5 -32768 32767 5330.570552 5767.738096
    Band 6 -32768 32767 13907.716940 6676.264984
    Band 7 -32768 32767 -686.028568 22913.114711
    Band 8 -32768 32767 -4222.349394 22408.476623

    Is this method correct/accepable?
    I know it is really hard to help me without the data. Unfortunately, I can not send the data files and the forum do not allow me to add any screenshots, I can email you if you like to check. I really appreciate all your help and suggestions! Thank you.


    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    28 Dec 2017 02:02 PM
    I would not boost the pixel values before FLAASH correction. FLAASH expects specific physical units of microW/(sr*cm^2*nm) and the output pixel values can be poorly modeled if you do not use the correct units. If you can submit your question through a POC at your institution, it may be best to work with your data directly.

    Prabha Rupasinghe



    New Member


    Posts:22
    New Member


    --
    28 Dec 2017 07:24 PM
    That is so unfortunate.. I am not sure who is POC in my university. But the lisence administrator is my my supervisor. I'll send an email through her. Thank you!

    Prabha Rupasinghe



    New Member


    Posts:22
    New Member


    --
    28 Dec 2017 07:47 PM
    I'm sorry, I forgot to mention in my previeous reply, is it same for thr QUAC correction? QUAC also works well after multiplying the image with 10.

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    04 Jan 2018 10:56 AM
    No, you can provide DN, radiance or even TOA reflectance to Quac for correction. A boost of 10x should not affect it in a similar way.

    Prabha Rupasinghe



    New Member


    Posts:22
    New Member


    --
    23 Jan 2018 09:01 AM
    Thank you so much! My image is doing great with the QUAC. I really appreciate your help.

    Careli Caballero



    New Member


    Posts:2
    New Member


    --
    23 Jan 2018 02:39 PM
    Hello, I read the conversation because I have the same problem with my WV3 images. But, I would like to confirm if my understood about the solution is the correct one please: Using my original images (they are in DN values) I have to create a mask for background pixels and then apply QUAC using unknown sensor option. So, I do not have to run a radiometric calibration before using Flaash settings, right?
    Furthermore, if I want to do a pan sharpening, can I use the result obtained after applying QUAC?
    Finally, I have WV3, WV2 and GE1 images and I was wondering if I can apply the same solution for all my images.
    Thank you in advance for your help!

    MariM



    Veteran Member


    Posts:2396
    Veteran Member


    --
    24 Jan 2018 06:20 AM
    You do not have to run Radiometric Calibration. However, I think it is good practice to convert the data to radiance using FLAASH Settings and run both FLAASH and Quac on the data to compare results. You are correct that when using Quac, it is important to use a mask on background pixels. FLAASH is not affected by the background pixels so it is not necessary for that tool.
    For Pan sharpening, I would use the results from Quac but do realize that pan sharpening affects the reflectance values so if you are most concerned with spectral integrity, it is best to do analysis on products that are not pan sharpened.
    Yes, you can use a similar process on other images as well. If you do not see your sensor listed in Quac, you can use Unknown.
    You are not authorized to post a reply.
    Page 1 of 212 > >>