X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 04 Jan 2016 10:33 AM by  anon
trouble using "where" command in IDL
 1 Replies
Sort:
You are not authorized to post a reply.
Author Messages

anon



New Member


Posts:81
New Member


--
04 Jan 2016 10:33 AM
    IDL user question: Hi: I'm running the following simple script to use "where" command. It works for all values but 0.9. Any thoughts why this is happening? I tried this on both IDL 8.2.3 on Mac, and IDL 8.2.2 linux version on our server. Thank you. IDL> x=indgen(21)*.05 IDL> print, where(x eq .8) 16 IDL> print, where(x eq .95) 19 IDL> print, where(x eq 1) 20 IDL> print, where(x eq .9) -1

    Deleted User



    New Member


    Posts:81
    New Member


    --
    04 Jan 2016 11:39 AM
    Answer: Hello, This type of problem can be caused when trying to represent real decimal data values on a computer, which uses a limited precision binary representation for floating point values. As a rule of thumb, single precision floating point values can be trusted to approximately 6 or 7 significant digits (depending on the specific binary representation of the number). Further, limited precision can propagate through calculations in prescribed ways such that results from parallel calculations that go through different computational paths can appear to give a different answers -- perhaps even different from the expected theoretical answer. However, in such cases, if you take into account only the proper number of "propagated* significant digits for those particular calculations, the results can be interpreted to be substantially the same as that expected from theoretical precise calculation. This seems to be the case for your example statement: print, where(x eq .9) To illustrate: IDL> x=indgen(21)*.05 IDL> print, x, format='(7f12.8)' 0.00000000 0.05000000 0.10000000 0.15000001 0.20000000 0.25000000 0.30000001 0.34999999 0.40000001 0.45000002 0.50000000 0.55000001 0.60000002 0.65000004 0.69999999 0.75000000 0.80000001 0.85000002 0.90000004 0.94999999 1.00000000 IDL> print, where(x eq .8) 16 IDL> print, .8, format='(7f12.8)' 0.80000001 IDL> print, where(x eq .95) 19 IDL> print, .95, format='(7f12.8)' 0.94999999 IDL> print, where(x eq 1) 20 IDL> print, 1, format='(7f12.8)' 1.00000000 IDL> print, where(x eq .9) -1 IDL> print, .9, format='(7f12.8)' 0.89999998 IDL> print, 18*.05, format='(f10.8)' 0.90000004 IDL> print, .9, format='(f10.8)' 0.89999998 To compensate for the imprecise representation of floating point value when doing value comparisons, one approach that can be used is to include a"buffer" value to ensure that the floating point value approximations will fall within acceptable ranges. For example: IDL> buf = .0000001 IDL> print, where(x le .9+buf and x ge .9-buf ) 18 IDL> .9+.0000001 0.90000010 IDL> .9-.0000001 0.89999986 Another approach can be to work with the significant digit comparisons in the integer realm, where computer value representations can be trusted to be exact: IDL> multiplier = 100000L IDL> print, where(round(x*multiplier) eq round(.9*multiplier)) 18 For additional information on this topic, please also see the following IDL help page on our Documentation Center: http://www.exelisvis.com/docs/Accurac... I hope this can behelpful to you. Kind regards,
    You are not authorized to post a reply.