X
304 Rate this article:
No rating

[Internal] IDL 8.X 32bit Windows : Unable to allocate memory

Anonym
Topic:
*Note: The information in this Tech Tip is internal information. There is an external customer-facing version of this Tech Tip located here at Tech Tip #4603.


Users will encounter the error message below when creating moderate to small sized IDL variables on 32bit Windows systems running IDL 8.0.

% Unable to allocate memory: to make array.
Discussion:

Prior to IDL 8.0, when running the IDL Workbench, IDL existed as two separate processes:

idlde.exe
idl_opserver.exe


Starting with IDL 8.0, IDL now runs in a single in-process server which allows the Eclipsed based IDL Workbench and the underlying core IDL process to better communicate and better integrate. This solves many problems that IDL versions prior to IDL 8.0 had. However, for 32bit Windows systems, this has introduced an unintended consequence.

32bit Windows IDL 8.0 users now have a greatly reduced memory space for IDL to create variables in. Users will only be able to create an array 10% to 20% of the size they could in IDL 7.1.

This only appears to affect 32bit IDL 8.0 for Windows, and does not seem to affect 64 bit Windows systems or other platforms.

Error messages encountered are likely to look like this:

% Unable to allocate memory: to make array.


Under the best of circumstances (a 32bit Windows machine with 4GB of ram and only IDL running) this typically means that users can only allocate about 200MB worth of variables before IDL will run out of memory, compared to about 1GB for IDL 7.1

CR 58077 is the active bug report for this issue. Please refer to the CR for updates on the progress towards resolution to this issue.

32bit Windows users may have to rewrite their IDL code to more carefully clean up variables once they are not needed any longer. However in certain situations, when creating large arrays, it may not be possible to allocate the needed memory. Some possible workarounds:

Primary Workaround for 32bit Windows Users



32bit Windows users in particular may notice a reduced amount of RAM that can be allocated for an array. If this is encountered follow the steps below to increase the amount of RAM available to the IDL session:

1.   Open this file in a plain text editor:
C:\Program Files\ITT\IDL\IDL80\bin\bin.x86\idlde.ini


2.   Change the file to look like the following, containing only these lines:
-vm
C:\Program Files\ITT\IDL\IDL80\bin\bin.x86\jre\bin
-vmargs
-Xms128M
-Xmx128M


This will increase the available memory to IDL. You may also see good results by removing all but the first two lines to look like this:

-vm
C:\Program Files\ITT\IDL\IDL80\bin\bin.x86\jre\bin



Secondary Workarounds relative to all platforms



!null



The !null system variable was introduced in IDL 8.0 to allow users to either set a variable equal to null, or to destroy objects or other types of data. !null essentially allows the programmer to un-define an IDL variable, freeing up the memory that it occupies. This will allow users to destroy variables they don't need, freeing up memory for other variables to be created. For example a user could do the following:

Array   = findgen(1000,1000)   ;Create an array
newArray= Array^2    ;Use the array in a calculation
Array   = !null      ;Set the array equal to null


Now Array is undefined, and only newArray is left in memory, leaving open memory for new variables to be created. !null can also be used to destroy objects and pointers. The !null variable can be used at any point in your program like so:

;sets a variable equal to null
Variable = !null      

Or

;sets the output of the function to null
!null = functionname()



Automatic Garbage Collection



Starting with IDL 8.0, IDL now features Automatic Garbage Collection. When a heap variable becomes inaccessible with no existing references to it, IDL will automatically de-allocate the memory it occupies, freeing it up for IDL to later use. For example, if you do something like this:

pro myprogram
   p = ptr_new( findgen(1000) )
end



when the program is finished executing, p is lost and then IDL will then remove the heap variable, the 1000 element float array in this case, from memory. Prior to IDL 8.0, when either a pointer or an object reference was lost the pointer p would be lost, out of scope, but the heap variable would remain in memory, inaccessible and therefore not usable thus needlessly taking up memory space.
However, in IDL 8.0, as soon as there are no more references (pointers, variable names, etc.) IDL removes the heap variable from memory, giving it back to the system for later use.

Automatic Garbage Collection is enabled by default, so there is no need to configure it. I can also be disabled/enabled at will by the use of the HEAP_REFCOUNT() function, see the IDL Help document for details.


IDL_IDLBridge



The IDL_IDLBridge object class can be used to create a spawned separate IDL session, which will have its own memory space. This will allow you to run an IDL program outside of the parent IDL Workbench session, potentially giving you more memory to work with. For example, to execute a procedure called "myprogram" with a parameter called "par" within the IDL_IDLBridge session, you could do something like this:

osession = IDL_IDLBridge()
osession.execute, 'myprogram, par'


The string given to the IDL_IDLBridge::Execute method can be any IDL statement that you wish to be executed. This session will remain open until the IDL_IDLBridge object is destroyed. For more information search for "IDL_IDLBridge" in the IDL Help browser.


IDL Command Line



The command line version of IDL generally will have more memory available to it because it does not have to create a graphical user interface. A GUI takes up memory when it is created and rendered to the screen. Running the command line version of IDL avoids this memory overhead, which will potentially allow you more memory for your IDL program. If your IDL program consumes a lot of memory, you could create and edit the application in the Workbench, but then run the program from the command line version. This would allow you to have the development features of the workbench, while then allowing you more memory when executing your program.

The command line version of IDL 8.0 can be accessed as follows:

  • Windows: Selecting the "IDL 8.0" > "Tools" > "IDL command line" start menu item.
  • MacOS: Double clicking on the "IDLCommandLine.app" in the /Applications/itt/idl/idl80 directory.
  • Linux/Unix: Issuing the "idl" command at a terminal prompt.


IDL 8.1 Update


With IDL 8.1, the workbench and IDL are once again separate processes on Win 32-bit systems. There is a IDLDE switch '-inprocess' that can be used to combine them into one process. In addition, on 64 bit windows systems there is a IDLDE switch '-outofprocess' that can be used to launch IDL with the workbench as a separate process.Solution:
[Edit this field in the IDL-based Tech Tip Editor, v80]