Techniques for improving IDL code

Topic

This article describes techniques that can be used to improve the performance of IDL programs. (NOTE: Most of the text in this article was taken from Mark Piper's "Scientific Programming with IDL" manual)

Discussion

Some techniques to improve the speed and memory usage of IDL programs is shown below:

1) Vectorization

Vectorization is the process of removing loops from code.Vectorization is important in all interpreted languages because looping creates a lot of overhead. This is a general IDL technique and nearly all IDL operators and routines work on arrays. For example:

a = fltarr(10)

;slower
for ind=0, 9 Doa[ind]= a[ind]+1

;faster
a = a + 1

2) Workbench Code Profiler

There is a tool in the IDL workbench called the Workbench Code Profiler that can help you analyze the performance of your applications.Information about how to use this tool can be found using the link below:

http://harrisgeospatial.com/docs/ProfilerView.html

3)  Order of operations

The speed of arithmetic operations can be increased by a factor of two or more by performing scalar operations first, then array operations. For example:

ans = array * (scalar1 / scalar2)

is faster than

ans = array * scalar1 / scalar2

4) Row versus column array access

Rows are contiguous in memory and therefore are faster to access than columns. If you need to loop through every element of an array,loop over the rows in the outer loop and columns in the inner loop. For example:

for irow=0, nrows-1 do begin
  for icol=0,ncols-1, do begin
        element =array[icol, irow]
   endfor
endfor

5) Array creation with NOZERO

If an array will be initialized with non-constant values then IDL's default initialization is wasted. You can use the NOZERO keyword toallocate memory without initialization

new = fltarr(n, /nozero)

6) Array assignment

An * on the left side of an equation generates an indexed array which slows down the program. A sub-array can be copied to a larger array without indexing by specifying an insertion point (cf. Excel).

array = bytarr(200, 200) + 127B
sub = indgen(50,50)
array[50,75] = sub

7) Variable type

Automatic type conversion in IDL is handy but it's slow.Avoid it if possible. Type conversion in loops is particularly inefficient -each loop iteration requires a type conversion.

Example:

array = fltarr(n)
for i=0, n-1, do array[i] + 5

8) Concatenation

Concatenation is the process of appending elements to adata container (e.g., array, structure, list, hash). Lists, in particular aredesigned for concatenation. Arrays are not. More information about LIST can befound using the link below:

http://harrisgeospatial.com/docs/LIST.html

list.add, number

is faster than

array = [array,number]

9) Memory Management

Some suggestions to improve the memory usage of IDL are shown below:

- Passing parameters by reference is more memory efficient than passing by value.
- Use the smallest variable type that can accurately represent your data.
- Use TEMPORARY or compound operators when possible.
- Let IDL allocate a big chunk of memory from the OS.
- Perform operations on subsets of data and loop over the subsets
- Increase the amount of RAM on the system

Written by DS 9/5/2014