An example of how to use IDLnetURL to download data secured through OAuth 2.0 authorization protocol

QUESTION:

How can IDL be used to programmatically download data secured through OAuth 2.0 authorization protocol?

 

ANSWER / EXAMPLE:

This is doable with IDLnetURL!

In this example, we will download VIIRS satellite data from the Colorado School of Mines in this format:

https://eogdata.mines.edu/nighttime_light/monthly/v10/2014/201404/vcmcfg/SVDNB_npp_20140401-20140430_00N060E_vcmcfg_v10_c201507201613.tgz

which has been secured using the OAuth 2.0 authorization protocol, as described HERE. 

Here are the steps:

1 )  A POST call to the authorization server to obtain the access token (which expires every 24 hours). In the body of the POST call, you must provide the following information:

client_id=eogdata_oidc 
client_secret=2677ad81-521b-4869-8480-6d05b9e57d48
username=<your_username>
password=<your_password>

It's not intuitive, but to do POST calls with IDLnetURL, you need to use the PUT method with the POST keyword:

Then just parse the result to obtain the access token. 

See this webpage for additional information about the token strings needed for programmed download access from the data server in this example.

2) Use the access token with a GET call to the data server to download the data file. The GET request must be delivered with the access token in the header and in this format -- Authorization: Bearer $accessToken

 

 

Here is example IDL PRO code that demonstrates this workflow:

 

;GET THE TOKEN =========================
;Build POST request body
  username='joe.user@l3harris.com'
  password='joeUsersPassword'
  ;NOTE the '&' is needed at the end of the line to prevent IDL from adding a rogue line break...
  tokenRequestBody = 'client_id=eogdata_oidc&client_secret=2677ad81-521b-4869-8480-6d05b9e57d48&username='+username+'&password='+password+'&grant_type=password&' 

;Submit POST request (via PUT method...) to obtain token response data
 oUrl = OBJ_NEW('IDLnetUrl')
result = oUrl->Put(tokenRequestBody,/POST,/BUFFER,/STRING, $
             URL='https://eogauth.mines.edu/auth/realms/master/protocol/openid-connect/token')
OBJ_DESTROY, oUrl

;Parse result to get access token
oHash=json_parse(result) ;convert JSON return to orderedhash
Values = oHash.Values()
accessToken = Values[0]

;USE TOKEN TO DOWNLOAD DATA============================
dataURL='https://eogdata.mines.edu/nighttime_light/monthly/v10/2014/201404/vcmcfg/SVDNB_npp_20140401-20140430_00N060E_vcmcfg_v10_c201507201613.tgz'
outputFile = 'test.tgz'
oUrl = OBJ_NEW('IDLnetUrl')
oURL->SetProperty, HEADERS= 'Authorization: Bearer '+accessToken

;GET request to download ~500MB file
result = oUrl->Get(filename=outputFIle,URL=dataURL)
OBJ_DESTROY, oUrl

 

NOTE: This specific example may or may not still be functional since it relies on a third-party data endpoint. Nonetheless, the concepts will still apply to other OAuth 2.0 secured data. 

 

 

 

 

Created by BC-NA on 12/14/22, Reviewed by JU on 12/27/2022