Using List and Hash Parameters with GSF 2.0
Anonym
We recently released the Geospatial Services Framework (GSF) 2.0 product, a Node.JS based successor to ENVI Services Engine (ESE). Early adopters have run into some questions when they had tasks with output parameters of type List or Hash, as the form returned from the server was different then what they expected.
One of the risks you run when you use Lists and Hashes is that their contents may include objects which can’t be serialized automatically by the system to be returned to the REST client. So some validation is now performed to make sure that each List or Hash element is a primitive type (string, number), or is an object that supports the Dehydrate() method, which returns a collection of primitives. This guarantees that the serialization to JSON for output will be successful. Note that all the ENVI API object classes inherit the ENVIHydratable interface and support Dehydrate(), as I blogged about here.
Another thing we wanted to ensure was that the correct datatypes were preserved when a List or Hash was serialized. JSON and JavaScript only have one numeric type, which is the same as IDL’s Double type. When using an IDL client to GSF, the most likely code path includes a call to JSON_Parse(), which will return either a Double, Long64, or ULong64 depending on the contents of the JSON string. One of the many benefits of using ENVITasks with GSF is that we provide a stronger type checking system than standard IDL. So if you wanted to take the output of one task and pass it into another task, it makes sense to maintain the proper datatypes that were returned from the first task. To that end, the serialized form of List and Hash objects include metadata describing each element along with its dehydrated form, so that an exact clone of the original collection can be reconstituted.
Now that I’ve covered the why, let me explain the how. We’ll look at List objects first, as they are a little simpler than the Hash classes. When a List A is dehydrated, a Hash is created with one element with the “elements” key associated with another List B. There Is a 1-1 correlation between elements of List A and B, where each element of A maps to a Hash in B. The Hash elements of B each have 2 keys “type” and “dehydratedForm”. For each index i, the “type” key of B[i] is set to the string returned by calling the IDL TypeName() function on A[i], and “dehydratedForm” is set to the value returned by calling Dehydrate() on A[i]. For primitive values like strings and numbers, this is the exact same scalar or array value. This is easier to understand with some code examples than a verbal description of the process:
IDL> l = List(1, 'foo', !pi, ['bar', 'baz'], List(5))
IDL> l
[
1,
"foo",
3.1415927,
["bar", "baz"],
[
5
]
]
IDL> l.Dehydrate()
{
"elements": [
{
"type": "INT",
"dehydratedForm": 1
},
{
"type": "STRING",
"dehydratedForm": "foo"
},
{
"type": "FLOAT",
"dehydratedForm": 3.1415927
},
{
"type": "STRING",
"dehydratedForm": ["bar", "baz"]
},
{
"type": "LIST",
"dehydratedForm": {
"elements": [
{
"type": "INT",
"dehydratedForm": 5
}
]
}
}
]
}
Here we can see how a 5 element List results is a dehydrated form with 5 Hashes in the “elements” List, and how the process can involve recursion when you have a List in the List.
The Dehydrate() process for Hash and its subclasses OrderedHash and Dictionary is similar, with a couple notable exceptions. When a Hash A is dehydrated, a Hash is created with two keys “elements” and “fold_case”. The “fold_case” key is set to the return value of calling the IsFoldCase() method on A, and the “elements” key is set to another Hash B (or OrderedHash or Dictionary, so that it has the same type as A). We use the same class for B as A to make sure that order is preserved when needed for OrderedHashes, but we avoid the memory and performance hit when it is unnecessary for normal Hash and Dictionary. There is again a 1-1 correlation between Hash A and B, as they will have the exact same set of keys. The value assigned to each key in B is a Hash based on the corresponding value in A. These inner Hashes are the same as they are for List B above, with the “type” and “dehydratedForm” keys created the same way. Again a code example should make this easier to understand:
IDL> h = OrderedHash(1, 'foo', 'two', Hash(2.2, !pi), 3, ['bar', 'baz'], 'four', List(5))
IDL> h
{
1: "foo",
"two": {
2.20000: 3.1415927
},
3: ["bar", "baz"],
"four": [
5
]
}
IDL> h.Dehydrate()
{
"fold_case": false,
"elements": {
1: {
"type": "STRING",
"dehydratedForm": "foo"
},
"two": {
"type": "HASH",
"dehydratedForm": {
"fold_case": false,
"elements": {
2.20000: {
"type": "FLOAT",
"dehydratedForm": 3.1415927
}
}
}
},
3: {
"type": "STRING",
"dehydratedForm": ["bar", "baz"]
},
"four": {
"type": "LIST",
"dehydratedForm": {
"elements": [
{
"type": "INT",
"dehydratedForm": 5
}
]
}
}
}
}
For symmetry purposes, the List and Hash classes now also have static Hydrate() methods that can take one of these dehydrated Hashes and rebuild a clone of the original List or Hash. These methods will properly typecast the numeric values to the correct type, even if the dehydrated Hash has gone through conversion to and from JSON. The Hydrate() methods will also account for the possibility that arrays of numbers and strings were turned into Lists by JSON_Parse(), and restore them back to the correct array form.