Workaround when IDL to python bridge is crashing due to MKL library conflict

Example of configuration that is failing

IDL 8.8.1, IDL 8.8, 8.7.3

Python 3.8.5 – Anaconda distribution

Mac or Linux Operating System

 

Description of the issue

• The IDL to python bridge is crashing IDL with the below error

INTEL MKL ERROR: dlopen(/Users/username/opt/anaconda3/lib/libmkl_intel_thread.1.dylib, 9): Library not loaded: @rpath/libiomp5.dylib

Referenced from: /Users/username/opt/anaconda3/lib/libmkl_intel_thread.1.dylib

Reason: image not found.

Intel MKL FATAL ERROR: Cannot load libmkl_intel_thread.1.dylib.

• The python to IDL bridge may work - but could also crash in some cases.

The issue is most probably due to the MKL library provided with the Conda distribution, which badly interacts with the IDL Lapack library.

Note: The Lapack library is present only in IDL 8.7.3+ versions.

 

Possible workaround: it consists in removing MKL package from the main python installation

• Launch the command below

conda remove mkl

It will spit out a big list of Python modules that are built against MKL, and thus will be removed when removing MKL. We recommend noting all packages that are removed so they can be reinstalled afterwards.

•  Confirm the removal.

• Then run the below command

conda install nomkl

This command will set a flag in the Python installation to only install non-MKL versions when installing new modules. Confirm to install it.

Note: The nomkl package is not available on Windows. Because of the way DLLs are handled on Windows however, this specific crash related to MKL should be uncommon on Windows.

• It is then mandatory to reinstall numpy afterwards to be able to launch the bridge

conda install numpy

This will reinstall numpy, now with no MKL.

• Test the IDL to Python 3.8 bridge which should now work.

•  Re- install all needed modules/packages that were removed at step 1, such as pandas or scipy.

Most modules have non-MKL versions available to install.

 

Another option:

Since the above option will alter the main python installation, another work-around would be to call the IDL to Python bridge from within an IDL-IDL Bridge. For example:

b = OBJ_NEW('IDL_IDLBridge')

b->Execute,"Python.Run('import sys')"

Despite not being convenient, this will prevent the MKL conflict without altering the main python installation, especially for users who absolutely need MKL support in Python.

 

 

-------------------------------------------

created by BC & BC(US) on 10/23/2021

Reviewed by BC(US)