Is There Better Memory Allocator Which IDL Can Use?
Exelis VIS receives many questions about how IDL manages memory use. Since memory is a central issue for any program, these questions are not very surprising.
This Help Article answers the question on "Is The System Allocator Usually the Best Solution Or Is There A Better Memory Allocator Which IDL Can Use? "
Please see additional Help Article for detailed information on how IDL uses memory:
Discussion:
To put it simply: The system allocator is the best solution. There isn't a better memory allocator to use. At least, not one that is without significant drawbacks of its own.
There are, of course, alternative memory allocators. GNU malloc is sometimes mentioned as a better malloc because it will under some circumstances release memory to the operating system. There are also commercial alternatives (such as SmartHeap). Through experimentation and hard experience we have found that it is best to simply stay with the system supplied memory allocator for the following reasons:
- All known mallocs perform poorly under some pathological circumstances. This is a direct result of the use of ad-hoc algorithms (due to the fact that optimal memory allocation is an NP-complete problem). The system malloc tends to suffer less from this hazard in practice because it is used by a wide variety of programs.
- It is absolutely vital that memory allocation be 100% correct, as a small error can be cause disastrous problems. Such problems are also difficult to debug. The fact that 99% of all programs on a given system use the system malloc and free functions means that they are the best tested and most reliable implementation available.
- A program like IDL relies on a huge amount of system library and other third party code. This code often uses the system malloc. Mixing memory allocators can cause difficulties in reliability and performance, whereas using the system allocator avoids the issue entirely.
- The system allocator takes special requirements of the system into account. For example, threaded system libraries require a thread safe malloc, but many replacement mallocs are not thread safe.
- An allocator needs to be tightly integrated with the implementation of the operating system kernel for best results.
Although we have investigated other alternatives, the system allocator is usually the best solution. Memory allocation is at best a statistical game, and no allocator is the best in all situations, but we find that the system allocator is the best more often than not, and more often than the competition. It is hard to see how we can do better than this, although it is an inescapable fact that there will always be a small number of people who see poor performance that a different malloc might avoid.
Review on 12/31/2013 MM