X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 18 Mar 2008 07:03 AM by  anon
Handling large arrays in IDL
 1 Replies
Sort:
You are not authorized to post a reply.
Author Messages

anon



New Member


Posts:
New Member


--
18 Mar 2008 07:03 AM
    Hello, Can you give me some tips on how to handle large arrays in IDL. The size of the arrays that I am working with are around (100000 x 100000). One way to get around is to create many smaller arrays but I would like to keep this option as the last one. The RAM size of the server is also very large, 8 GB, which I guess is the maximum that could fit onto the machine. I would be grateful to have some suggestions from you. Thank you. Regards, Vijayakumar

    Deleted User



    New Member


    Posts:
    New Member


    --
    18 Mar 2008 07:03 AM
    See ITT Visual Information Solutions' Tech Tip article #3346 ("Overcoming Windows Memory Allocation Limitations") at URL: http://www.ittvis.com/ser...echtip.asp?ttid=3346 for a clearer understanding of how IDL memory allocation is performed on a 32-bit operating system. (The points in that article are also relevant for 32-bit Linux, Mac or UNIX as well, by the way.) A 100000 x 100000 element array needs somewhere between 10 GB and 80GB of storage, depending on its datatype. If you really want to process such an array stored in one variable, then you absolutely need a 64-bit operating system. On such a system I think that IDL would have no trouble allocating such an array size. On any 32-bit system, however, IDL could only store and process such an array in subsets (called "tiles" if the array is an image array), and, even in tiles, a single 32-bit IDL process could probably not hold more than 1.5 GB of the array in virtual memory at any one point in time, and little more than 1GB in any single tile. James Jones
    You are not authorized to post a reply.