One of the new features of IDL 8.4 that I've been digging is the Lambda() function. While there are some good examples in the help docs, I thought I'd share a few of my real world examples and some lessons learned.
One of the simpler examples I had is to take a Hash that I know has all string keys and create a List containing them all in uppercase form. There are a couple ways to accomplish this. The pre-8.4 way to do this requires a few lines of code:
h = Hash('foo', 1, 'bar', 2, 'baz', 3)
keys = h.Keys()
upperArray = StrArr(keys.Count())
foreach key, keys, i do upperArray[i] = StrUpCase(key)
upperKeys = List(upperArray, /EXTRACT)
Not too bad, but with the static methods introduced in IDL 8.4 we can do better, sorta:
h = Hash('foo', 1, 'bar', 2, 'baz', 3)
keys = h.Keys()
keyArray = keys.ToArray()
upperArray = keyArray.ToUpper()
upperKeys = List(upperArray, /EXTRACT)
At least this is using the ToUpper() static method to vectorize the transform of the entire array in one line instead of using the foreach loop on each element. But with Lambda functions and the List::Map () method we can do even better:
h = Hash('foo', 1, 'bar', 2, 'baz', 3)
keys = h.Keys()
upperKeys = keys.Map(Lambda(str: str.ToUpper()))
The Lambda function is applied to each element of the List, and its output is used as the corresponding element in the output List. In this case the Lambda function takes the input List element and calls the ToUpper() static method on it. Care does have to be taken that the source List contains only strings, or bad things ensue:
IDL> h = Hash('foo', 1, 2, 'bar', 'baz', 3)
IDL> keys = h.Keys()
IDL> upperKeys = keys.Map(Lambda(str: str.ToUpper()))
% Attempt to call undefined method: 'IDL_INT::TOUPPER'.
% Execution halted at: $MAIN$
Here the Hash has an integer key and the Lambda function tried to call ToUpper() on an IDL_Int variable, not an IDL_String variable.
Let's move on to a pair of Lambda functions I created when working on ENVITask and the ability of SetProperty and GetProperty to disambiguate _REF_EXTRA keywords against the list of ENVITask class properties and parameters owned by the task. The first Lambda function is used to identify all List elements that start with a given string, and the second is used to identify all List elements that are initial substrings of a given string. I'll start with a specific input string, and then generalize it to work with any string. Here is the Lambda function that can be used to find all strings in a List that start with the substring 'ENVI':
IDL> substringMatch = Lambda(listElem: listElem.StartsWith('ENVI'))
IDL> l = List('ENVIRaster', 'Gaussian', 'ENVIVector', 'Laplacian')
IDL> l2 = l.Filter(substringMatch)
IDL> l2
[
"ENVIRaster",
"ENVIVector"
]
The Lambda function uses the new static method StartsWith() to return a Boolean state of whether the current List element does or does not start with the string 'ENVI'. We could have used the /FOLD_CASE keyword in the StartsWith() call to make the string compare case insensitive.
Here is the Lambda function that can be used to find all strings in a List that are substrings of the string 'ENVIRaster':
IDL> startsWithSubstring = Lambda(listElem: ('ENVIRaster').StartsWith(listElem))
IDL> l = List('EN', 'ENVIRas', 'ENVIVector', 'ENVIRaster', 'ENVIRasterSpatialRef')
IDL> l2 = l.Filter(startsWithSubstring)
IDL> l2
[
"EN",
"ENVIRas",
"ENVIRaster"
]
Here the Lambda function uses the trick of putting the string literal inside parentheses so that it can invoke the static method on it.
This is all well and good, but if I had a number of different strings I wanted to compare against I'd have to write new Lambda function definitions for each one. Instead I want to have a parameterized Lambda function that uses a variable instead of a string literal for the search string. We can't just add a second parameter to the Lambda function definition however:
IDL> substringMatch = Lambda(listElem, substring: listElem.StartsWith(substring))
IDL> l = List('ENVIRaster', 'Gaussian', 'ENVIVector', 'Laplacian')
IDL> l2 = l.Filter(substringMatch('ENVI'))
% IDL_STRING::STARTSWITH: String expression required in this context: SUBSTRING.
% Execution halted at: IDL$LAMBDAF4 1 <Command Input Line>
% $MAIN$
The Lambda() function will create a valid lambda function for you, but it doesn't work as expected, and definitely doesn't work in the List::Filter() method. The problem is that the occurrence of 'substring' before the semicolon is treated as an input variable, so it expects the variable to be defined when it invokes the StartsWith() static method. Also you can't treat the substringMatch variable like a function of one variable, passing in the string value you want, it was defined with two parameters. But there is hope, as the Lambda function help demonstrates in its "More Examples" section you can create a Lambda function that in turn generates another Lambda function with the string literals you want plugged in. Here is the generalized substring matching Lambda function in action:
IDL> substringMatch = Lambda('subString: Lambda("listElem: listElem.StartsWith(''"+subString+"'')")')
IDL> l = List('ENVIRaster', 'Gaussian', 'ENVIVector', 'Laplacian')
IDL> l2 = l.Filter(substringMatch('ENVI'))
IDL> l2
[
"ENVIRaster",
"ENVIVector"
]
This new Lambda looks messy, but it can be broken down and analyzed. The first thing we notice is that that the outer Lambda is passed in a single string literal, not unquoted code. This is needed for the nested Lambda syntax to work. Now let's look at what that string is when printed to the console:
IDL> print, 'subString: Lambda("listElem: listElem.StartsWith(''"+subString+"'')")'
subString: Lambda("listElem: listElem.StartsWith('"+subString+"')")
The outer Lambda function takes its input parameter, which I named 'subString', and passes that into another call to Lambda(). This inner Lambda function concatenates three strings, two string literals that use double quotes and the subString value. We'll also note that the two string literals include a single quote so that the value of the subString variable is turned into a string literal when the inner Lambda function is evaluated. Looking back at the original declaration of the outer Lambda, we see a single quoted string literal with some double quoted string literals inside, each of which include the single quote character, which can be escaped as a pair of single quotes. A little messy, but it works when sit down and pull it apart.
Here then is the generalized superstring matching Lambda function in action:
IDL> startsWithSubstring = Lambda('superString: Lambda("listElem: (''"+superString+"'').StartsWith(listElem)")')
IDL> l = List('EN', 'ENVIRas', 'ENVIVector', 'ENVIRaster', 'ENVIRasterSpatialRef')
IDL> l2 = l.Filter(startsWithSubstring('ENVIRaster'))
IDL> l2
[
"EN",
"ENVIRas",
"ENVIRaster"
]
A similar analysis as above will prove how this nested Lambda function works.
There are two big caveats in using this nested Lambda function concept in your code. First, if you want to play with them in the console window, you're initially going to get weird errors like this:
IDL> substringMatch = Lambda('subString: Lambda("listElem: listElem.StartsWith(''"+subString+"'')")')
IDL> l = List('ENVIRaster', 'Gaussian', 'ENVIVector', 'Laplacian')
IDL> l2 = l.Filter(substringMatch('ENVI'))
% Type conversion error: Unable to convert given STRING to Long64.
% Detected at: $MAIN$
The reason for this is that the interpreter will think that the substringMatch variable is a string, not a function, so it thinks you are using the old school parenthesis array indexing syntax. The error message is rather cryptic in this regard, but that is what it means. You can fix this by simply typing compile_opt idl2 in the console window, then rerunning the l2 line of code. This shows how the compile_opt syntax can be used to change the behavior of the main interpreter session, not just inside routines. It also demonstrates that you need to make sure you include compile_opt in any routines where you use Lambda functions.
The second caveat is that if you want to use nested Lambda functions like this in code that you are compiling into save files, then you need to do a little extra work. If you try to use the code above, compile it and then call RESOLVE_ALL before calling SAVE, then you'll get error messages like this:
Attempt to call undefined function: 'STARTSWITHSUBSTRING'.
Attempt to call undefined function: 'SUBSTRINGMATCH'.
The solution to this is to add any variable names like this that you use to hold the nested Lambda function values to an array that you pass into the SKIP_ROUTINES keyword in RESOLVE_ALL. Once you do this, the save file will build and it should work properly when the code is executed.