8766
Background: What is Virtual Memory?
Exelis VIS receives many questions about how IDL manages memory use. Since memory is a central issue for any program, these questions are not very surprising.
This Help Article gives you a background of Virtual Memory. The IDL user must be aware of virtual memory and how it works to get the best performance from their system.
Please see additional Help Articles for detailed information on how IDL uses memory:
Memory is usually thought of as a series of consecutive cells, each of which can hold a fixed amount of data. In the past, the size of this memory unit has varied, but over the last 30 years, the industry has settled on 8-bit bytes and other sizes are now very rare. This is in part due to the influence of the C programming language, which is difficult to implement on machines that are not byte addressable. Each memory unit has an address, which is denoted by a non-negative integer. The address of the first memory cell is 0, and following cells have increasing addresses.
In early computers, all programs running on a given computer shared a single physical address space. There were many problems with this approach, including:
- To run, a given program has to be loaded into memory. Since other programs are also running, the program has to fit in whatever spot is currently unoccupied. Memory would therefore quickly become cut up into inconvenient spans. Fixing this requires moving running programs in memory. Managing the relocation of such programs is complex.
- Programs could see and interfere with each other. This leads to unstable and insecure systems.
- Programs often require access to more memory that actually exists on a given machine. On a physical memory machine, this is not possible in a reliable way that is transparent to the program. Instead, programmers resort to a series of complex and error prone tricks, such as memory overlays and swapping data to and from files, to get around these limitations.
In the 1960s and 1970s, a technique called Virtual Memory was developed that solves these problems. The basic idea is that memory is considered to be a series of fixed size units called a page (Common sizes are powers of two, such as 512 bytes, 2K, 4K, and 8K and up). Special address translation hardware allows a given page (with a given physical address) to appear at a different arbitrary address to a given program. The operating system uses this ability to create the illusion that every running program has its own private memory space starting at address 0. Since a given program can only access a small number of pages at a given instant (a single page in the case uni-processor computers), the operating system can keep the data for pages not currently being used in a disk file commonly called a Pagefile or Swapfile. When a program tries to access a page that is not currently in physical memory, the operating system stops it, swaps in the missing page, and then restarts the program. This scheme is called Demand Paged Virtual Memory. Virtual memory also makes it trivial to protect programs from each other.
With the notable exception of the Apple Macintosh and Microsoft operating systems, there hasn't been a non-virtual memory machine running IDL since the PDP-11. The support for virtual memory is probably the single biggest step forward in the move from Windows 3.1 to Win95/NT/2000, and from the Classic Macintosh to MacOSX.
In conclusion, virtual memory is an elegant scheme for giving programs a private memory space (starting at address 0) with size limited only by available disk space, and with security between running programs. All systems running IDL today are virtual memory systems. Therefore, the IDL user must be aware of virtual memory and how it works to get the best performance from their system.