X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 07 Feb 2023 11:30 AM by  Ben Castellani
python to IDL bridge on Mac
 3 Replies
Sort:
You are not authorized to post a reply.
Author Messages

Stephane Erard



New Member


Posts:2
New Member


--
25 Mar 2020 07:43 AM
    Hello

    I have a hard time trying to make the python bridge to work on Mac (OS X 14.2 localized, IDL 8.7.1, python 2.7 and 3.6)
    I'm using either the Apple install of python 2.7 or a homebrew install of 3.6.
    The IDL to python seems OK with python in the PATH (I can call python from IDL command line, although not in the workbench), but not the other way around.


    - Defining a PYTHONHOME logical only results in complete blocking of python (exits with message ImportError: No module named site)
    - Launching setup.py without PYTHONHOME results in
    *** PYTHONHOME environment variable must be set to your Python installation.
    Setup failed. Exiting.


    - I understand this is a path issue. I've tried a solution found on a forum, I think from IDL support:
    cd /Applications/harris/idl87/bin/bin.darwin.x86_64/
    sudo install_name_tool -change libidl.8.7.dylib @loader_path/libidl.8.7.dylib pythonidl36.so
    sudo install_name_tool -change libidl_ips.8.7.dylib @loader_path/libidl_ips.8.7.dylib pythonidl36.so
    sudo install_name_tool -change libpython3.6m.dylib /usr/bin/libpython3.6m.dylib pythonidl36.so
    At this point, DYLD_LIBRARY_PATH is still needed in the .login to specify IDL’s bin directory. To eliminate that, the following updates can be made to tell various libraries where to find their dependencies:
    sudo install_name_tool -change libMesaGL6_2.dylib @loader_path/libMesaGL6_2.dylib libidl.8.7.dylib
    sudo install_name_tool -change libMesaGLU6_2.dylib @loader_path/libMesaGLU6_2.dylib libidl.8.7.dylib
    sudo install_name_tool -change libOSMesa6_2.dylib @loader_path/libOSMesa6_2.dylib libidl.8.7.dylib
    sudo install_name_tool -change libXm.3.0.2.dylib @loader_path/libXm.3.0.2.dylib libidl.8.7.dylib
    sudo install_name_tool -change libMesaGL6_2.dylib @loader_path/libMesaGL6_2.dylib libMesaGLU6_2.dylib
    sudo install_name_tool -change libMesaGL6_2.dylib @loader_path/libMesaGL6_2.dylib libOSMesa6_2.dylib
    sudo install_name_tool -change libidl.8.7.dylib @loader_path/libidl.8.7.dylib libidl_ips.8.7.dylib


    - But this doesn't fix it:
    When typing: from idlpy import *
    from any dir I get
    ModuleNotFoundError: No module named 'idlpy'
    - Launching from /Applications/harris/idl87/lib/bridges with an alias of pythonidl.36 put there, I get:
    ImportError: dlopen(/Applications/harris/idl87/lib/bridges/pythonidl36.so, 2): no suitable image found. Did find:
    /Applications/harris/idl87/lib/bridges/pythonidl36.so: file too short
    /Applications/harris/idl87/lib/bridges/pythonidl36.so: stat() failed with errno=13
    - and by the way, the date oof pythonidl.36 is changed in the process - as if the file were modified.


    This is probably a small thing, but a very annoying one. Can you provide any clue, or ideally a fix to setup.py? I'd love to use IDL in Jupyter notebooks…
    (you may also want to fix the Mac path to setup.py in https://www.harrisgeospatial.com/docs/Python.html)

    Regards

    Stephane Erard



    New Member


    Posts:2
    New Member


    --
    31 Mar 2020 04:24 AM
    After many tries, I came up with a ~ stable solution:
    - the Apple install is special and can't be handled by the setup.py script provided (this is 2.7 anyway)
    - For some reason the brew install is also difficult to handle at least for calling python from IDL (I personally need to keep this one up and running for other reasons)
    - a convenient solution is to install an appropriate version of Anaconda - this will leave other installs alone, but paths have to be adapted (there may be a more convenient solution using environments)
    For IDL 8.7.1 and Mac OS 10.14, the appropriate download (python 3.6) is
    https://repo.continuum.io/archive/Anaconda3-5.2.0-MacOSX-x86_64.pkg
    this is installed at root level, not in user's home as indicated in IDL doc

    I'm launching the setup from terminal only (not .tcshrc), so that I can revert to my previous python install (brew) in a new terminal:
    PYTHONPATH is retrieved from this python with:
    import sys
    sys.path
    and the two IDL dir must be added:

    setenv PYTHONPATH /anaconda3/lib/python36.zip:/anaconda3/lib/python3.6:/anaconda3/lib/python3.6/lib-dynload:/anaconda3/lib/python3.6/site-packages:/anaconda3/lib/python3.6/site-packages/aeosa:$IDL_DIR/bin/bin.darwin.x86_64:$IDL_DIR/lib/bridges

    setenv PATH /anaconda3/bin:${PATH}

    setenv PYTHONHOME /anaconda3

    alias python /anaconda3/bin/python

    cd /Applications/harris/idl87/lib/bridges
    python setup.py install

    + copy the "kernels" folder
    from /lib/bridges/
    to ~/Library/Jupyter/

    This gives access to the two-way bridge and IDL support in Jupyter notebooks.
    There is a remaining issue with python windows under IDL - they make IDLDE crash, and can't be closed when IDL is run from terminal (but the prompt is still there)

    stephane.erard@obspm.fr



    New Member


    Posts:2
    New Member


    --
    06 Dec 2022 02:16 AM
    Follow up for IDL 8.8:

    • The supported Python versions are 3.7 to 3.9
    - Install an appropriate version of Anaconda, see here: https://docs.anaconda.com/anaconda/reference/release-notes/
    An adapted version is Anaconda3-2021.11-MacOSX-x86_64.pkg, available on https://repo.continuum.io/archive
    ** Beware that this version of Anaconda is installed in a different location from the previous one, in /opt/anaconda3

    • In your startup file:
    source /Applications/harris/idl88/bin/idl_setup
    set path= ($path /opt/anaconda3/bin)
    • do this only once from the command line, will add something in tcshrc file
    conda init tcsh

    • To make it clean, define an environment with required libraries: 
    conda create -n pyast2 python=3.9 astropy matplotlib numpy pandas pyvo
    # activate environment
    conda activate pyast2

    ** There are library conflicts between these versions of Anaconda and IDL (on Mac and Linux), see:
    https://www.l3harrisgeospatial.com/Support/Self-Help-Tools/Help-Articles/Help-Articles-Detail/ArtMID/10220/ArticleID/24118/Workaround-when-IDL-to-python-bridge-is-crashing-due-to-MKL-library-conflict
    Solution: 
    conda deactivate
    conda remove mkl
    conda install nomkl
    conda install numpy
    conda activate astpy2 

    • copy the IDL "kernels" folder
    from wherever/lib/bridges/
    to ~/Library/Jupyter/ 

    • Check local path with
    cd /opt/anaconda3/bin
    ./python
    >>import sys
    >>sys.path
    Copy the result in PYTHONPATH and add the required IDL directories - in pratice I'm doing this in a script to preserve access to other versions of python, it reads like this:
    # (idlpython2 script)
    setenv PYTHONPATH /opt/anaconda3/envs/pyast2/lib/python39.zip:/opt/anaconda3/envs/pyast2/lib/python3.9:/opt/anaconda3/envs/pyast2/lib/python3.9/lib-dynload:/opt/anaconda3/envs/pyast2/lib/python3.9/site-packages:/Applications/harris/idl88/lib/bridges:/Applications/harris/idl88/bin/bin.darwin.x86_64
    setenv PATH /opt/anaconda3/bin:${PATH}
    setenv PYTHONHOME /opt/anaconda3
    # launch 3.9
    alias python /opt/anaconda3/envs/pyast2/bin/python3.9
    conda activate pyast2
    # Then do this only the first time:
    cd /Applications/harris/idl88/lib/bridges
    python setup.py install

    • To use this:
    # activate adapted environment:
    conda activate pyast2
    # launch the IDL-python script above:
    source idlpython2


    Ben Castellani



    Basic Member


    Posts:130
    Basic Member


    --
    07 Feb 2023 11:30 AM
    Thanks for the follow-up Stephane. Glad you got the Python bridge working. It definitely can be challenging on Mac due to OS restrictions that have been implemented by Apple over the years and some nuances to the bridge itself.

    With that said, IDL 8.9 will have improved Python bridge functionality which no longer requires a setup script to run as well as overhauled bridge documentation. Please give it a try when it releases later in 2023!
    You are not authorized to post a reply.

    Forums Products IDL