List extensions, part I

(Note: Ron Kneusel, a Senior Consultant in the Exelis VIS Professional Services Group is covering for me with a pair of posts while I’m on vacation. Thanks, Ron!  –MP) The List class was introduced with IDL 8.0.  In this post we'll talk about some extensions to the List class that might come in handy from time to time.  Specifically, we will talk about three new methods for the List class (really four, one comes in a function and a procedure form) and a couple of new functions that operate on lists.  Next, there will be something for anyone familiar with Lisp/Scheme and finally a demonstration of how lists allow IDL to perform symbolic differentiation.  But first, the new methods. One nice feature of IDL is the ability to add new methods to any class at any time.  In this case, we will simply add new methods to the existing List class:

function list::show

pro list::show

function list::depth

function list::search, target, within=within

These methods, and their associated routines, are in the file list_extensions.pro.  To load all of these into IDL, download this file, put it in your path and type:

IDL> list_extensions

Lists often become quite complex and it is useful to be able to visualize them from time to time.  This is the purpose of the Show method.  The functional form returns the list representation as a string while the procedural form prints it on the screen. Let's see it in action:

IDL> t = list(1,2, list(3,4,5), 6, list(list(7,8), 9), 10)
IDL> print, t
       1
       2

       6

      10
IDL> t->show
(1 2 (3 4 5) 6 ( (7 8) 9) 10)

Printing the list does not recurse into the sublists but Show uses Lisp-style S-expressions to clearly indicate the contents and nesting of the list.  Items between matched parentheses are at the same nesting level or depth. How deep is the list t?  Let's see:

IDL> print, t->depth()
       2

Why 2?  Because the top level of the list is depth zero. Let's come back to Show since there are some options.  The full syntax of Show, either form, is:

pro list::show, full=full, form=form

As one might expect, setting FORM will show only the structure of the list:

IDL> t->show, /form
(# # (# # #) # ( (# #) #) #)

While setting FULL will show the entire list, expanding all elements:

IDL> w = list(dist(3), ['xyzzy','plugh'], indgen(10))
IDL> w->show, /full
([[0.000000,1.00000,1.00000],[1.00000,1.41421,1.41421],[1.00000,1.41421,1.41421]] ['xyzzy','plugh'] [0,1,2,3,4,5,6,7,8,9])

Of course, one must be a bit careful with the FULL keyword:

IDL> y = list(indgen(1000), dist(4), 1.0d, 'the end')
IDL> y->show, /full
     …do you really want to see the whole thing?
IDL> y->show
({2,1000} {4,4 4} 1.0000000 'the end')

By itself, Show uses some heuristics to decide whether to show the entire array in an element of the list or whether to abbreviate it using the notation {, } as is seen here with {2, 1000} for a 1000 element short integer vector and {4, 4 4} for a 4x4 array of floats. Also in list_extensions.pro are two new functions: LIST_SPLIT and LIST_EXPR. The first builds a new list from an array by iterating along one of the dimensions of the array:

IDL> a = [[1,2,3],[4,5,6],[7,8,9],[10,11,12]]
IDL> t = list_split(a, dimension=1)
IDL> t->show
([1,4,7,10] [2,5,8,11] [3,6,9,12])
IDL> t = list_split(a, dimension=2)
IDL> t->show
([1,2,3] [4,5,6] [7,8,9] [10,11,12])

The second, LIST_EXPR, creates an array from an S-expression.  This is the format used by Lisp and Scheme to represent lists and consists of items between sets of possibly nested parentheses.  By no coincidence, this is exactly how the Show method returns the list.

IDL> t = list_expr('(1 2 (3 4) ((5 6) 7) 8 9)')
IDL> t->show
(1 2 (3 4) ( (5 6) 7) 8 9)

With LIST_EXPR, one could easily use S-expressions as a preferences file for an application, for example.  It could be read as a string, passed to LIST_EXPR, used and updated in the application, and then output back to disk using Show with the FULL keyword set. The List class includes the method FindValue (renamed Where in IDL 8.2) which looks for a value in the list.  However, this method will not search the list recursively for a value. This is what the new Search method does.  Additionally, Search accepts a keyword, WITHIN, which will search within IDL arrays inside of the list for the value, if set.  Search returns 1 if the target value is found in the list, at any depth, or 0 if not.  Here are some examples using the LIST_EXPR function:

IDL> l = list_expr("(1 ('two' [1,2,3]) 5 ('six' ['seven','eight']) 9)")
IDL> l->show
(1 ('two' [1,2,3]) 5 ('six' ['seven','eight']) 9)
IDL> print, l->search(5)
       1
IDL> print, l->search('seven')
       0
IDL> print, l->search('seven', /within)
       1

Notice the last two examples above.  In the first, Search returns false because the string 'seven' is not in the list by itself, it is not an item in the list but an element of a string array within the list.  In the next example, WITHIN is set which searches within the IDL string array and there it finds the target value. The file lisp.pro contains a set of standard list access functions, both as functions and as methods added to the List class.  If you are familiar with Lisp or Scheme, these might be useful to you:

IDL> lisp
IDL> t = list_expr('(1 2 (3 4) ((5 6) 7) 8 9)')
IDL> print, car(t)
       1
IDL> print, t->cadr()
       2

and so on up to all combinations of cxxxxr. Tune in next week for part II of Ron’s discussion of list extensions, including an example of symbolic math in IDL (!).