X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 26 Sep 2013 03:05 AM by  anon
Last line of file problems
 3 Replies
Sort:
You are not authorized to post a reply.
Author Messages

anon



New Member


Posts:
New Member


--
26 Sep 2013 03:05 AM
    When I import data from a text file into IDL and uses it, then the data from the last line in the text file ends up giving errors. So there is some kind of end of file thing going on, that makes reading that last line of data go wrong. I however can't see what is wrong, because when I get my script to show me the numbers from that line, they look as they should, but when I try to use them in calculations or put my data through filters, they don't acts as the number shown. I am using one of two scripts for the read-ins of the files; a professional script from NASA -readpds- and a colleagues script, and both have given me the same type of problems on different files; while other people don't have problems with them, I am assuming since that would quickly have been discovered. I wonder if it can be something to do with that I am using a Mac (I work on IDL 8.2.0 on a Mac IOS 10.8.5.), as that have cause problems one time before with a different script. That time I had however made the text file myself on the mac, while now I am using a data file from NASA, which the readpds was made to read. Do anyone knows how what can be wrong and ways to fix it? I am also having the weird problem with the same file, that when I write less than (LT) a certain number, then the data shown includes that number, just as if I wrote LE. I have used LT and LE hundreds of times, but never before have I had this problem. It can be seen here: print, n_elements(where(a le 0.1)) 64 print, n_elements(where(a lt 0.1)) 64 print, n_elements(where(a lt 0.099999)) 55 The numbers in the array looks in the text file like 0.16 and are in the array shown as 0.16000000. But that don't explain why 0.10000000 < 0.1. Can anyone explain to me what is wrong? I would like to trust my calculations and results that I get from this, but it is hard when I find such obvious errors.

    Deleted User



    New Member


    Posts:
    New Member


    --
    26 Sep 2013 03:52 PM
    Ah, yes, I am sure you will be interested in the following articles: http://www.idlcoyote.com/math_tips/sk... http://www.idlcoyote.com/math_tips/ra... I don't know how to advise you on your text file reading problem, because you don't indicate what the error is. But I agree, if the problem was with IDL or the script, the problem would have been well-known by now. That fact that is isn't suggests that, uh, something else is probably wrong. Did you pay attention to line endings when you FTPed the data file to your machine?

    Deleted User



    New Member


    Posts:
    New Member


    --
    27 Sep 2013 02:24 AM
    Thanks, for the links. The results from articles there is that it is rounding off errors: "This binary representation does not let IEEE 754 exactly represent numbers that are not simple exact multiples of powers of two. This results in the second surprise to many users: most simple decimal floating point numbers (e.g., 0.3 or 0.1) cannot be exactly representedand any attempt to store such numbers results in errors. E.g., 0.1 might become 0.1000000005 when stored." Irritating all the same. The easiest is to use LT 0.099 in stead of LT 0.1. I discovered the explaination to my problem with the last point of the data. I was using the 'where' function to make sure all bad data was put to NaN: a[where(a GT 9.9)]=!Values.F_NAN The problem came about when the where function didn't find anything, because it then gives the value -1, which when used as an index indicate the last number in the array (personally I think IDL should change this, so it gave an error in stead). It was therefore necessary to change it to: b=where(a GT 9.9,count) if count gt 0 then a(b)=!Values.F_NAN

    Deleted User



    New Member


    Posts:
    New Member


    --
    27 Sep 2013 07:52 AM
    The other alternative is to set the NULL keyword on the Where function.
    You are not authorized to post a reply.