Array Uniqueness in IDL

 

When a person finds oneself in pursuit of retrieving information from data, it is often imperative to inspect every unique element - but why waste time on inspecting EVERY element when you can weed out the duplicates?

IDL has a uniq function built for just such a purpose. It goes in to an array, and removes any duplicates adjacent to one another.

Let's take an array:

IDL> array = [1,2,2,5,1,4,4,2]

When we run uniq on this array, it returns the indices that are NOT duplicates of an adjacent entry. This prints out:

IDL> print, uniq(array)

          0        2       3        4        6        7

In order to get back the original array with the elements removed, we can subset the array with these indices:

IDL> print, array[uniq(array)]

       1       2      5       1       4      2

The duplicate 2 and 4 have been removed, however there are still more duplicates in the array. To get just one of each unique element, you first have to use the sort function. This function also returns indices; in this case the indices that put the array in ascending order:

IDL> print, sort(array)

          4        0        7       2        1        6       5        3

Just like with uniq, these can be used to re-order the original array to get the sorted array:

IDL> print, array[sort(array)]

       1       1      2       2       2      4       4       5

Now for the final step - since this sorted array has all ofthe similar elements adjacent to each other, we can now use the uniq function to pull out all of the unique elements of the array.

IDL> s = array[sort(array)]

IDL> print, s[uniq(s)]

       1       2      4       5

Or for those that like to do it in one line:

IDL> print, (array[sort(array)])[uniq(array[sort(array)])]

       1       2      4       5

Now instead of looping over and entire array to check every element, IDL will be able to look through and array that is half the size of its original.