How to exclude/ignore specific values when computing MEAN_FILTER in IDL?

MEAN_FILTER is an IDL routine to apply mean filter noise reduction to images. By default, the mean filter is applied to the whole input image. However, you may want to exclude specific input values during the computation.

The help is listing the INVALID keyword which can be assigned to a given value so it will be ignored during the computations. However, INVALID values are not fully ignored in IDL 8.8.1 and previous versions: they are replaced by a value of 0 instead.

 

Reproduce Case:

 

IDL> c=float(fix(dist(6)))*100.

IDL> print,c

      0.00000      100.000      200.000      300.000      200.000      100.000

      100.000      100.000      200.000      300.000      200.000      100.000

      200.000      200.000      200.000      300.000      200.000      200.000

      300.000      300.000      300.000      400.000      300.000      300.000

      200.000      200.000      200.000      300.000      200.000      200.000

      100.000      100.000      200.000      300.000      200.000      100.000

IDL> print, MEAN_FILTER(c,3,INVALID=300.)

      75.0000      116.667      100.000      133.333      100.000      150.000

      116.667      144.444      111.111      133.333      111.111      166.667

      100.000      111.111      122.222      133.333      122.222      116.667

      133.333      133.333      133.333      133.333      133.333      133.333

      100.000      111.111      122.222      133.333      122.222      116.667

      150.000      166.667      116.667      133.333      116.667      175.000

Let’s look at the value c[3,2] in blue bold in the above outputs.

The mean value at this position, with a green kernel (3*3), and with an INVALID/ignored value set to 300, should be

200+200+200+200+400=1200/5=240

However, the output value from MEAN_FILTER with INVALID keyword set to 300 is 133.33. It corresponds to the following computation:

200+200+200+200+400=1200/9=133.33

It means that INVALID keyword does not ignore the value 300 but it replaces it by 0.

 

Workaround 1: use SMOOTH routine instead of MEAN_FILTER

 

One workaround is to use the SMOOTH routine instead of MEAN_FILTER. SMOOTH routine can only ignore NaN values so a first step will consist to replace values to be ignored by NaN.

IDL> c=float(fix(dist(6)))*100.

IDL> print,c

      0.00000      100.000      200.000      300.000      200.000      100.000

      100.000      100.000      200.000      300.000      200.000      100.000

      200.000      200.000      200.000      300.000      200.000      200.000

      300.000      300.000      300.000      400.000      300.000      300.000

      200.000      200.000      200.000      300.000      200.000      200.000

      100.000      100.000      200.000      300.000      200.000      100.000

IDL> pt=where(c eq 300.)

IDL> c[pt]=!VALUES.F_NAN

IDL> print,smooth(c,3,/Nan)

      0.00000      100.000      200.000          NaN      200.000      100.000

      100.000      144.444      166.667      200.000      166.667      100.000

      200.000      166.667      220.000      240.000      220.000      200.000

          NaN          200.000      240.000      240.000      240.000          NaN

      200.000      166.667      220.000      240.000      220.000      200.000

      100.000      100.000      200.000          NaN      200.000      100.000

The output at position c[3,2]is now the expected one: 240. One side effect of SMOOTH routine is to keep input values on end points - in black bold in the above outputs. You may need to manage them independently and/or to use EDGE_* keywords to have them processed.

 

Workaround 2: use the undocumented NORMALIZE keyword and a scaling factor with MEAN_FILTER

 

• It consists in adding the NORMALIZE undocumented keyword to MEAN_FILTER
• In addition, a scale factor needs to be applied which is
  • N^2 when using a square kernel NxN
  • M * N when using a MxN rectangle kernel

 

 

IDL> c=float(fix(dist(6)))*100.

IDL> print,c

      0.00000      100.000      200.000      300.000      200.000      100.000

      100.000      100.000      200.000      300.000      200.000      100.000

      200.000      200.000      200.000      300.000      200.000      200.000

      300.000      300.000      300.000      400.000      300.000      300.000

      200.000      200.000      200.000      300.000      200.000      200.000

      100.000      100.000      200.000      300.000      200.000      100.000

IDL> print, MEAN_FILTER(c,3,INVALID=300.,/NORMALIZE)*9

      75.0000      116.667      128.571      171.429      128.571      150.000

      116.667      144.444      166.667      200.000      166.667      166.667

      128.571      166.667      220.000      240.000      220.000      150.000

      171.429      200.000      240.000      240.000      240.000      171.429

      128.571      166.667      220.000      240.000      220.000      150.000

      150.000      166.667      150.000      171.429      150.000      175.000

With the NORMALIZE keyword and a scale factor of 3^2=9, the mean value at position c[3,2], with a green kernel (3*3), and an INVALID/ignored value set to 300, is now the expected one:

200+200+200+200+400=1200/5=240

 

 

----------------------------------------------

created by BC on 12/3/2021

reviewed by BC(US)