X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 10 Aug 2016 07:57 PM by  anon
MOD operator unexpected behavior
 1 Replies
Sort:
You are not authorized to post a reply.
Author Messages

anon



New Member


Posts:2
New Member


--
10 Aug 2016 07:57 PM
    even if A is an integer multiple of B, (A MOD B) = B if B is too small. A MOD B =0 would be correct. i am not sure what the threshold value for this behavior is, but it is between B = 0.2 and B = 0.25. this is for IDL 8.4 on the mac, so perhaps it has been corrected in later versions. i may not check back here for replies... i just wanted to bring this to people's attention in case it's helpful. here is some sample output showing the behavior (and the workaround on the last line): IDL> print, 100d mod 2d 0.0000000 IDL> print, 100d mod 1d 0.0000000 IDL> print, 100d mod .5d 0.0000000 IDL> print, 100d mod .25d 0.0000000 IDL> print, 100d mod .2d 0.20000000 IDL> print, ((100d * 10) mod (0.2d * 10)) / 10 0.0000000

    Deleted User



    Basic Member


    Posts:143
    Basic Member


    --
    17 Aug 2016 04:29 PM
    I found an old bug report (IDL-5276) dealing with an issue very similar to this. The bug report states that this behavior is occurring because .2 cannot be exactly represented as a floating point or double precision number. Essentially, the conclusion from the report is that floating point arithmetic is not exact and should not be relied upon.
    You are not authorized to post a reply.