Why Can’t I “Get” My Data

While working on a recent project, I needed to pull some data off of an ESRI server using a REST API. This is by no means a novel task, but just like many of you, this is not something I do on a daily basis. In preparation for this task, I brushed up on my IDLnetURL objects and read through the subsequent documentation. I felt as though I was adequately prepared and it was off to the races. First, I coded up my “get” call to obtain the geometry of the region I was working in (for this example I will use Utah).

oURL = obj_new('IDLnetURL')

oURL->SetProperty, $

  url_scheme='https', $

  url_host = 'sampleserver6.arcgisonline.com',$

  url_path = 'arcgis/rest/services/USA/MapServer/2/query?
text=Utah&geometryType=esriGeometryEnvelope&spatialRel
=esriSpatialRelIntersects&f=pjson'

Cords_Json = oUrl->Get(/string_array, /BUFFER)

Cords_Json = JSON_PARSE(strjoin(Cords_Json))

GeomJson = JSON_SERIALIZE(((Cords_Json['features'])[0])['geometry'])

This worked great and the next step was to find all the vectors that fell within the given region (for this example I will use counties). However, when I went back to my trusty “get” command it fell apart. 

oURL = obj_new('IDLnetURL')

oURL->SetProperty, $

  url_scheme='https', $

  url_host = 'sampleserver6.arcgisonline.com',$

  url_path = 'arcgis/rest/services/USA/MapServer/3/query?geometry='+GeomJson+'&geometryType=esriGeometryPolygon&spatialRel
=esriSpatialRelContains&outFields=*&f=json'

County_Json = oUrl->Get(/string_array)

Instead of nicely formed JSON it was HTML, and incomplete HTML at that. I found this perplexing until I realized that the “get” request has a return character limit of 2,048. This posed a problem given that in my region there are 29 vectors and each one has complex geometry with hundreds of vertices. So it was back to the drawing boards.

When performing a request-response from a client to a server, there are two options for returning data.  There is the “get” method and the “post” method. There are several subtle differences when it comes to these two methods, but for our purpose the main thing to note is that “get” returns the data you request embedded within the URL and because of that there is a limit on how long it can be. On the other hand, “post” returns it with the body of the request and thus does not have a limit. So as I’m sure you’ve already figured out, the “post” method was the answer to my problem.

oURL = obj_new('IDLnetURL')

oURL->SetProperty, $

  url_scheme='https', $

  url_host = 'sampleserver6.arcgisonline.com',$

  url_path = 'arcgis/rest/services/USA/MapServer/3/query?f
=json'

data = 'geometry='+GeomJson+'&geometryType=esriGeometryPolygon&spatialRel
=esriSpatialRelContains&outFields=*'

County_Json = oURL->Put(data, /POST, /BUFFER)

By using the “post” method over the “get” method, I was able to retrieve all of the counties and their names.

; get county names

County_Json = JSON_PARSE(County_Json, /TOSTRUCT)

County_Names = strarr(n_elements(County_Json.features))

for i = 0, n_elements(County_Json.features)-1 do County_Names[i] = (County_Json.features)[i].Attributes.name

print, County_Names, FORMAT='(a)'