Note: The IDLnetURL class is deprecated. We recommend that you use the HttpRequest class instead, as it offers more options and a cleaner interface.

The IDLnetURL class allows IDL to act as a client to an HTTP or FTP server. The object handles:

• Opening a connection to a remote HTTP or FTP server.
• Creating and sending HTTP requests to the remote HTTP or FTP server.
• Receiving responses from the remote HTTP or FTP server.
• Receiving and writing large data files to disk at a user-specified location.

For additional information, see the following sections:

• Overview of the IDLnetURL Object
• Using Callbacks with the IDLnetURL Object

Creation

See IDLnetURL::Init.

Properties

Objects of this class have the following properties. See IDLnetURL Properties for details on individual properties.

Methods

This class has the following methods:

• IDLnetURL::Cleanup
• IDLnetURL::CloseConnections
• IDLnetURL::Delete
• IDLnetURL::FtpCommand
• IDLnetURL::Get
• IDLnetURL::GetFtpDirList
• IDLnetURL::GetProperty
• IDLnetURL::Init
• IDLnetURL::Put
• IDLnetURL::SetProperty
• IDLnetURL::URLDecode
• IDLnetURL::URLEncode

Examples

See the following methods for examples:

• IDLnetURL::FtpCommand 
• IDLnetURL::Get 
• IDLnetURL::GetFtpDirList
• IDLnetURL::Put

Version History

Overview of the IDLnetURL Object

The following sections give a general overview of the IDLnetURL object’s capabilities.

HTTP Functionality

The IDLnetURL object enables the following HTTP-related tasks:

• Allows IDL to act as a HTTP client node.
• Allows IDL to act as a HTTP SSL client node.
• Issues Get requests to retrieve data from a remote HTTP server and place it in a buffer or file.
• Issues Put requests to move data from a buffer or file to a remote HTTP server.
• Issues Post requests to move data from a buffer or file to a remote HTTP server.
• Encrypts all data to and from a remote HTTP server with SSL.
• Compresses and decompresses data to and from a remote HTTP server.
• Supports Basic and Digest authentication modes.
• Supports customizing HTTP headers that are later sent to the remote HTTP server.
• Returns the HTTP response code and HTTP response header to the user.
• Provides status callbacks so the user can monitor call progress.
• Requests can be cancelled if the user chooses not to wait for a complete reply.

FTP Functionality

The IDLnetURL object enables the following FTP-related tasks:

• Allows IDL to act as a FTP client node.
• Allows IDL to act as a FTP SSL client node.
• Issues Get requests to retrieve data from a remote FTP server to a buffer or file.
• Issues Put requests to move data from a buffer or file to a remote FTP server.
• Issues Dir requests to retrieve directory listings from a remote FTP server.
• Issues commands (Delete, Make dir, and so on) to a remote FTP server.
• Encrypts all data to and from a remote FTP server with SSL.
• Supports authentication (logging into a remote FTP server).
• Returns the FTP response code and FTP response replies to the user.
• Provides status callbacks so the user can monitor call progress.
• Requests can be cancelled if the user chooses not to wait for a complete reply.
• Supports custom commands that are sent to the remote FTP server.
• Supports both Active and Passive mode connections.

HTTP Authentication Modes

The IDLnetURL object supports Disabled, Basic, and Digest HTTP authentication modes. It also supports authenticating with a remote HTTP or FTP server, and with proxy servers. The username and password can be set for both a remote HTTP or FTP server, and for a proxy server.

FTP Authentication

The IDLnetURL object supports logging in to a remote FTP server. A username and password can be set prior to connecting to a remote FTP server.

Secure HTTP

The IDLnetURL object supports the HTTPS protocol by implementing a secure socket layer (SSL). Secure HTTP ensures that all requests and responses are encrypted. To activate SSL, set the URL_SCHEME property to ‘https’.

Secure FTP

The IDLnetURL object supports implicit and explicit SSL, which ensure that all requests and responses are encrypted. An FTP server may support implicit SSL, explicit SSL, or both.

Note: Different FTP servers provide different responses when SSL authentication is not available. IDL attempts to catch these responses and generate errors if the FTP server does not support SSL, but there may be cases when IDL makes a non-SSL connection even when an SSL connection is requested. If you are not sure that the FTP server you are connecting to supports SSL, view the complete response headers to check that the connection is being made using SSL.

Explicit SSL is activated by setting the FTP_EXPLICIT_SSL keyword in the Get, Put, GetFtpDirList, and FtpCommand methods. Implicit SSL is activated by setting the scheme to ‘ftps’.

Note: Implicit SSL has been deprecated.

SSL Protocol Versions

By default, the IDLnetURL object supports automatic determination of the SSL protocol that is supported by the remote HTTP or FTP server. If the remote server does not support automatic determination of the SSL protocol version, you can direct the object to use a specific version with the SSL_VERSION property. The IDLnetURL object supports protocol versions TLS v1, SSL v2, and SSL v3. TLS v1 supersedes all previous versions of the SSL protocols.

HTTP Encoding

The IDLnetURL object supports compressed/encoded responses from a remote HTTP server, as well as the deflate and GZIP encoding formats. Refer to the description of the ENCODE property for details.

Proxies

The IDLnetURL object supports proxies. When the object utilizes a proxy, the PROXY_HOSTNAME and PROXY_PORT properties must be set.

Connection Caching

The IDLnetURL object caches up to five existing connections to remote HTTP or FTP servers. Once a connection is established to a remote server, the connection is reused on subsequent transactions with the remote server. Reusing a connection can save time because the connect, login, and SSL handshake information does not need to be repeated.

Use the IDLnetURL::CloseConnections method to close a connection. Typically, a remote server closes an existing connection after a period of no activity. When this occurs, the closed connection is removed from the existing connections list. Closing a connection using the CloseConnections method is useful for changing login information, enabling or disabling SSL, starting a new set of transactions on the FTP or HTTP server, and so on for the next transaction with the server.

User Agent

If you receive an Http ErrCode(403) or Http Err(Forbidden) the url host may require 'User-Agent:' value in the http header.

A 'User Agent:' field can be added to the http header, for example:

Obj->SetProperty, HEADERS = 'User-Agent: <user agent value>'

Mapping URL Components to IDLnetURL Properties

The following table shows how the components of a URL map to IDLnetURL properties. Set these properties by calling the SetProperty method.

Note: You can use the PARSE_URL function to break up a URL into its component strings, which can then be passed to IDLnetURL::SetProperty method.

A URL has the following composition:

URL_SCHEME://URL_USERNAME:URL_PASSWORD@URL_HOST:URL_PORT/URL_PATH?URL_QUERY

For example:

http://me:mypw@host.com:8080/project/data/get_data.cgi?dataset=climate&date=01012006

Using Callbacks with the IDLnetURL Object

Callbacks from the IDLnetURL object provide a way to transmit information back to the caller based on the status of a request made to a HTTP or FTP server. The value returned by the callback function is then used by the IDLnetURL object to determine whether to continue or abort the request.

The caller must set the callback function property to receive callbacks from a method. The callback is made to an IDL function, which takes IDL variables as input parameters.

Note: The callback method is intended to provide status and progress information. The callback should not be used to process the incoming or outgoing data. Any data processing must occur after the Get, Put, GetFtpDirList, or FtpCommand methods have returned.

The IDLnetURL object lets you define functions that are called during a Get or Put method call. If a callback function is specified with the CALLBACK_FUNCTION property, it is called in the following circumstances:

• For the IDLnetURL::Get method, the callback function is called each time a packet is received from the HTTP or FTP server.
• For the IDLnetURL::Put method, the callback function is called each time a packet of data is sent to the HTTP or FTP server.
• For the IDLnetURL::GetFtpDirList method, the callback function is called each time a list of files is retrieved.
• For the IDLnetURL::FtpCommand method, the callback function is called each time an FTP command is executed.

All methods invoke the callback function with an array of strings indicating status as the first parameter, an array of integers containing progress information as the second parameter, and with the value (if any) specified by the CALLBACK_DATA property as the third parameter. A callback function returns a value of type Int.

Cancelling a Request

The return value of the callback function should be an integer zero or one. If the return value is zero, a cancel message will be sent to the remote node. If the return value is one, the operation will continue.

Callback Routine Signature

A callback function is written in IDL and has the following format:

Function Callback_Function_Name, StatusInfo, ProgressInfo,    CallbackData

...

return vCancelFlag

where

• Callback_Function_Name is the name of the callback function. This value is specified by the CALLBACK_FUNCTION property.
• StatusInfo is the first input parameter. It is an array of IDL strings that contains status information about the Get or Put. When the caller is a user interface (UI), the UI typically displays the string array in a text widget.
• ProgressInfo is the second input parameter, and is an array of signed 64-bit values (type Long64) that contains progress information:
• ProgressInfo[0] – Contains 1 (one) when the ProgressInfo array contains valid data, or 0 (zero) when the array does not contain valid data.
• ProgressInfo[1] – The download total used when doing a HTTP Get or FTP Get. This value is 0 when doing a HTTP Get and the transfer encoding is chunked.
• ProgressInfo[2] – Contains the number of bytes that have been downloaded during the current HTTP or FTP get.
• ProgressInfo[3] – Contains the total number of bytes to be uploaded during a HTTP Put, HTTP Post, or FTP Put.
• ProgressInfo[4] – Contains the number of bytes currently uploaded during the HTTP Put, HTTP Post, or FTP Put.
• CallbackData is the CALLBACK_DATA property that can be set prior to calling the Get, Put, GetFtpDirList, or FtpCommand method. This property is passed into the callback function unmodified, and the contents of the property are determined by the user. When the caller is a UI, the callback data typically contains the ID of a text widget that displays the status information strings. If the CALLBACK_DATA property is not set, its value is undefined.
• The return value is of type Int, and can be used to cancel the Get, Put, GetFtpDirList, or FtpCommand operation. See Cancelling a Request for more information.

HTTP and FTP Response Codes

The following tables list the response codes stored in the RESPONSE_CODE property for the last HTTP or FTP request sent to the remote server. See RESPONSE_CODE for more information about the property that holds these values.

The following table lists the error response codes generated by the IDLnetURL object:

UNSUPPORTED_PROTOCOL

FAILED_INIT

URL_MALFORMAT

URL_MALFORMAT_USER

COULDNT_RESOLVE_PROXY

COULDNT_RESOLVE_HOST

COULDNT_CONNECT

FTP_WEIRD_SERVER_REPLY

FTP_ACCESS_DENIED

FTP_USER_PASSWORD_INCORRECT

FTP_WEIRD_PASS_REPLY

FTP_WEIRD_USER_REPLY

FTP_WEIRD_PASV_REPLY

FTP_WEIRD_227_FORMAT

FTP_CANT_GET_HOST

FTP_CANT_RECONNECT

FTP_COULDNT_SET_BINARY

PARTIAL_FILE

FTP_COULDNT_RETR_FILE

FTP_WRITE_ERROR

FTP_QUOTE_ERROR

HTTP_RETURNED_ERROR

WRITE_ERROR

MALFORMAT_USER

FTP_COULDNT_STOR_FILE

READ_ERROR

OUT_OF_MEMORY

OPERATION_TIMEOUTED

FTP_COULDNT_SET_ASCII

FTP_PORT_FAILED

FTP_COULDNT_USE_REST

FTP_COULDNT_GET_SIZE

HTTP_RANGE_ERROR

HTTP_POST_ERROR

SSL_CONNECT_ERROR

AD_DOWNLOAD_RESUME

FILE_COULDNT_READ_FILE

LDAP_CANNOT_BIND

LDAP_SEARCH_FAILED

LIBRARY_NOT_FOUND

FUNCTION_NOT_FOUND

ABORTED_BY_CALLBACK

BAD_FUNCTION_ARGUMENT

BAD_CALLING_ORDER

INTERFACE_FAILED

BAD_PASSWORD_ENTERED

TOO_MANY_REDIRECTS 

UNKNOWN_TELNET_OPTION

TELNET_OPTION_SYNTAX 

OBSOLETE

SSL_PEER_CERTIFICATE

GOT_NOTHING

SSL_ENGINE_NOTFOUND

SSL_ENGINE_SETFAILED

SEND_ERROR

RECV_ERROR

SHARE_IN_USE

SSL_CERTPROBLEM

SSL_CIPHER

SSL_CACERT

BAD_CONTENT_ENCODING

LDAP_INVALID_URL

FILESIZE_EXCEEDED

FTP_SSL_FAILED

SEND_FAIL_REWIND

SSL_ENGINE_INITFAILED

LOGIN_DENIED

TFTP_NOTFOUND

TFTP_PERM

TFTP_DISKFULL

TFTP_ILLEGAL

TFTP_UNKNOWNID

TFTP_EXISTS

TFTP_NOSUCHUSER

The following table lists all the HTTP response codes. Refer to https://www.w3.org/ for detailed specifications.

Informational                            

Continue

Switching Protocols

Success                            

OK

Created

Accepted

Non-Authoritative Information

No Content

Reset Content

Partial Content

Multi-Status

Redirection

Multiple Choices

Moved Permanently

Found

See Other (since HTTP/1.1)

Not Modified

Use Proxy (since HTTP/1.1)

Temporary Redirect (since HTTP/1.1)

Client Error                            

Bad Request

Unauthorized

Payment Required

Forbidden

Not Found

Method Not Allowed

Not Acceptable

Proxy Authentication Required

Request Timeout

Conflict

Gone

Length Required

Precondition Failed

Request Entity Too Large

Request-URI Too Long

Unsupported Media Type

Requested Range Not Satisfiable

Expectation Failed

Retry

Server Error                            

Internal Server Error

Not Implemented

Bad Gateway

Service Unavailable

Gateway Timeout

HTTP Version Not Supported

Bandwidth Limit Exceeded

The following table lists all the FTP response codes. Refer to https://www.w3.org/ for detailed specifications.

Restart marker reply.

Service ready in nnn minutes.

Data connection already open; transfer starting.

File status okay; about to open data connection.

Command okay.

Command not implemented, superfluous at this site.

System status, or system help reply.

Directory status.

File status.

Help message.

NAME system type.

Service ready for new user.

Service closing control connection.

Data connection open; no transfer in progress.

Closing data connection.

Entering Passive Mode (h1,h2,h3,h4,p1,p2).

User logged in, proceed.

Requested file action okay, completed.

"PATHNAME" created.

User name okay, need password.

Need account for login.

Requested file action pending further information.

Service not available, closing control connection.

Can't open data connection.

Connection closed; transfer aborted.

Requested file action not taken.

Requested action aborted: local error in processing.

Requested action not taken.

Syntax error, command unrecognized.

Syntax error in parameters or arguments.

Command not implemented.

Bad sequence of commands.

Command not implemented for that parameter.

Not logged in.

Need account for storing files.

Requested action not taken.

Requested action aborted: page type unknown.

Requested file action aborted.

Requested action not taken. File name not allowed.

Example of Using Widgets with IDLnetURL

The following example creates a widget-based dialog that connects to a OGC WCS server and allows you to download files located on the server. The dialog automatically points to the NV5 Geospatial Solutions FTP server, which contains several sample files for download.

See idlneturl_widget.pro, located in the examples/doc/objects subdirectory of the IDL distribution, for the example code listing. View the file in an IDL editor window by entering .EDIT idlneturl_widget.pro. at the IDL Command Line.

Version History

See Also

HttpRequest