X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 14 Feb 2024 08:18 AM by  Ben Castellani
SPAWN : version mismatch
 2 Replies
Sort:
You are not authorized to post a reply.
Author Messages

Jan H. Schween



New Member


Posts:2
New Member


--
01 Feb 2024 05:37 AM
    I use SPAWN to call curl to get data from a password protected webpage.
    curl produces an error message - with verbose output:
    curl: /opt/nv5/idl90/bin/bin.linux.x86_64/libcurl.so.4: no version information available (required by curl)
    ...
    curl: (77) error setting certificate file: /etc/pki/tls/certs/ca-bundle.crt

    if I do the same call in the bash shell the output is different and there is no error.
    Main difference is:
    ...
    * successfully set certificate verify locations:
    * CAfile: /etc/ssl/certs/ca-certificates.crt
    CApath: /etc/ssl/certs
    ...

    => SPAWN, 'curl ...' tries to store certificates at a location which is not available.
    => it seems as if this is a different verfsion of curl.

    checking the version of curl in SPAWN I get
    curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/8.4.0-DEV ...
    Release-Date: 2020-01-08
    ...
    WARNING: curl and libcurl versions do not match. Functionality may be affected.


    Doing the same in bash I get
    curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/7.68.0 ...
    Release-Date: 2020-01-08
    ...
    and no warning about the curl and curllib - as they are the same version.

    => libcurl version under idl/spawn is different - how can be that ?

    I tried to do an update of curl, but ubuntu apt tells me that I have the newest version of curl (at least what ubuntu apt thinks to be the latest)

    From the very first line of the SPAWN output it seems as if curl is not loaded by the system but by IDL from :
    /opt/nv5/idl90/bin/bin.linux.x86_64/libcurl.so.4 ...

    How do I get around this ?

    ((I used simple formatting of this ext with linebreaks an indentation - but seems as if this formatting is ignored and erased))

    Jan H. Schween



    New Member


    Posts:2
    New Member


    --
    02 Feb 2024 02:09 AM
    I forgot to mention i have IDL 9.0 and Ubuntu 20.04.6 LTS
    And all worked fine with the previous IDL version.

    The porblem is the libcurl version 8.4.0-DEV which is somehow provided by IDL and a certificate path which does not fit on my system.

    Solutions 1 ;
    curl can be forced to ignore certificates with option -k
     
    But of course this is not safe.

    Solution 2:
    curl can be provided with a path for the certificates with option --capath . You can find the path for the certificates by investigating the verbose output of curl from the shell and insert the path you find there
    [CODE] SPAWN, curl --capath /etc/ssl/certs/ ...

    It remains the danger that curl and this version of libcurl do not work together.

    Solution 3:
    As IDL is using libcurl it mus do somthing with it. Searching on the IDL doc pages for curl points me a new class HTTPRequest introduced with IDL version 9.0 - so that is the reason why it stopped working with this IDL version. One can use HTTPRequest to do the call, nevertheless one has to provide the name of the certificate file (IDL developpers preset a MS-Windows system path - although I am working on Linux ...).

     
      URL='...'
      options = { USERPWD : '...', CAINFO : '/etc/ssl/certs/ca-certificates.crt' }
      params = hash( 'lomin', bbox[0], ... )
      data = HTTPREQUEST.GET( URL, param=params, options=options )
      print, 'data='
      print, data.text
      print, data.json()
     


    I used here a structure for the options and a hash for the paramters - but this can be mixed...
    For details see https://www.nv5geospatialsoftware.com/docs/HttpRequest.html

    Ben Castellani



    Basic Member


    Posts:130
    Basic Member


    --
    14 Feb 2024 08:18 AM
    The reason why you see different behavior between using cURL in a plain bash terminal vs. using SPAWN in IDL is due to your PATH environment variable. When IDL is launched, this variable is updated to make IDL work properly.

    Using SPAWN in IDL will create a linked child process --- not a completely separate process. Thus it is expected and by design that the entire environment in IDL will be carried over into the child process's environment. That is the behavior you are seeing which causes the libcurl that ships inside of IDL to be found first on your PATH.

    Here is a potential work-around (this is IDL .pro code):

    ;=========
    ;Capture initial PATH variable state inside IDL
    initialPath = GETENV('PATH')
    ;Append to the PATH the location of your system cURL library. On Ubuntu 22, it is /usr/bin/curl
    updatedPath = '/usr/bin:'+initialPath
    SETENV,'PATH='+updatedPath
    ;SPAWN using cURL, which will now use system library
    SPAWN,'which curl',curlUsed
    SPAWN,'curl -V',versionInfo
    print,curlUsed
    print,versionInfo
    ;Reset PATH inside of IDL to be default
    SETENV,'PATH='+initialPath

    You are not authorized to post a reply.