Techniques for improving IDL code
Topic
This article describes techniques that can be used to improve the performance of IDL programs. (NOTE: Most of the text in this article was taken from Mark Piper's "Scientific Programming with IDL" manual)
Discussion
Some techniques to improve the speed and memory usage of IDL programs is shown below:
1) Vectorization
Vectorization is the process of removing loops from code.Vectorization is important in all interpreted languages because looping creates a lot of overhead. This is a general IDL technique and nearly all IDL operators and routines work on arrays. For example:
a = fltarr(10)
;slower
for ind=0, 9 Doa[ind]= a[ind]+1
;faster
a = a + 1
2) Workbench Code Profiler
There is a tool in the IDL workbench called the Workbench Code Profiler that can help you analyze the performance of your applications.Information about how to use this tool can be found using the link below:
The Profiler View
3) Order of operations
The speed of arithmetic operations can be increased by a factor of two or more by performing scalar operations first, then array operations. For example:
ans = array * (scalar1 / scalar2)
is faster than
ans = array * scalar1 / scalar2
4) Row versus column array access
Rows are contiguous in memory and therefore are faster to access than columns. If you need to loop through every element of an array,loop over the rows in the outer loop and columns in the inner loop. For example:
for irow=0, nrows-1 do begin
for icol=0,ncols-1, do begin
element =array[icol, irow]
endfor
endfor
5) Array creation with NOZERO
If an array will be initialized with non-constant values then IDL's default initialization is wasted. You can use the NOZERO keyword toallocate memory without initialization
new = fltarr(n, /nozero)
6) Array assignment
An * on the left side of an equation generates an indexed array which slows down the program. A sub-array can be copied to a larger array without indexing by specifying an insertion point (cf. Excel).
array = bytarr(200, 200) + 127B
sub = indgen(50,50)
array[50,75] = sub
7) Variable type
Automatic type conversion in IDL is handy but it's slow.Avoid it if possible. Type conversion in loops is particularly inefficient -each loop iteration requires a type conversion.
Example:
array = fltarr(n)
for i=0, n-1, do array[i] + 5
8) Concatenation
Concatenation is the process of appending elements to adata container (e.g., array, structure, list, hash). Lists, in particular aredesigned for concatenation. Arrays are not. More information about LIST can befound using the link below:
LIST
list.add, number
is faster than
array = [array,number]
9) Memory Management
Some suggestions to improve the memory usage of IDL are shown below:
- Passing parameters by reference is more memory efficient than passing by value.
- Use the smallest variable type that can accurately represent your data.
- Use TEMPORARY or compound operators when possible.
- Let IDL allocate a big chunk of memory from the OS.
- Perform operations on subsets of data and loop over the subsets
- Increase the amount of RAM on the system
Written by DS 9/5/2014
Updating links by CS 3/13/2025