X

NV5 Geospatial Blog

Each month, NV5 Geospatial posts new blog content across a variety of categories. Browse our latest posts below to learn about important geospatial information or use the search bar to find a specific topic or author. Stay informed of the latest blog posts, events, and technologies by joining our email list!



Not All Supernovae Are Created Equal: Rethinking the Universe’s Measuring Tools

Not All Supernovae Are Created Equal: Rethinking the Universe’s Measuring Tools

6/3/2025

Rethinking the Reliability of Type 1a Supernovae   How do astronomers measure the universe? It all starts with distance. From gauging the size of a galaxy to calculating how fast the universe is expanding, measuring cosmic distances is essential to understanding everything in the sky. For nearby stars, astronomers use... Read More >

Using LLMs To Research Remote Sensing Software: Helpful, but Incomplete

Using LLMs To Research Remote Sensing Software: Helpful, but Incomplete

5/26/2025

Whether you’re new to remote sensing or a seasoned expert, there is no doubt that large language models (LLMs) like OpenAI’s ChatGPT or Google’s Gemini can be incredibly useful in many aspects of research. From exploring the electromagnetic spectrum to creating object detection models using the latest deep learning... Read More >

From Image to Insight: How GEOINT Automation Is Changing the Speed of Decision-Making

From Image to Insight: How GEOINT Automation Is Changing the Speed of Decision-Making

4/28/2025

When every second counts, the ability to process geospatial data rapidly and accurately isn’t just helpful, it’s critical. Geospatial Intelligence (GEOINT) has always played a pivotal role in defense, security, and disaster response. But in high-tempo operations, traditional workflows are no longer fast enough. Analysts are... Read More >

Thermal Infrared Echoes: Illuminating the Last Gasp of a Dying Star

Thermal Infrared Echoes: Illuminating the Last Gasp of a Dying Star

4/24/2025

This blog was written by Eli Dwek, Emeritus, NASA Goddard Space Flight Center, Greenbelt, MD and Research Fellow, Center for Astrophysics, Harvard & Smithsonian, Cambridge, MA. It is the fifth blog in a series showcasing our IDL® Fellows program which supports passionate retired IDL users who may need support to continue their work... Read More >

A New Era of Hyperspectral Imaging with ENVI® and Wyvern’s Open Data Program

A New Era of Hyperspectral Imaging with ENVI® and Wyvern’s Open Data Program

2/25/2025

This blog was written in collaboration with Adam O’Connor from Wyvern.   As hyperspectral imaging (HSI) continues to grow in importance, access to high-quality satellite data is key to unlocking new insights in environmental monitoring, agriculture, forestry, mining, security, energy infrastructure management, and more.... Read More >

1345678910Last
11849 Rate this article:
.1

The Amazing Race!

Zachary Norman
It wasn't so long ago that the IDL-Python bridge was introduced to IDL 8.5. It was with this new version, that I got my first experience programming with Python and testing the IDL-Python bridge. Through the past year it has been exciting to see the new changes and improvements that have become a part of the bridge.



Some of these new features include:
-Better error catching with the IDL-Python bridge

-Enhanced Jupyter notebook that allows for the development of full IDL programs and Python code in the same environment

-Improved support for variables passing back and forth


With all the time I have spent with Python, I have always wondered what some of the advantages are between Python and IDL. One thing that I have commonly heard several engineers say was that IDL was much faster than Python. For this blog, I decided to put that to the test and see how Python and IDL really compared to one another.



Before talking about the test, I do just want to explain things a bit about how it was set up and some potential caveats about the processing times that will be shown. With the tests I created, I did my best to choose tests that were comparable between IDL and Python. Since I'm no expert at Python, there very well may have been other methods that could be faster than what I will show. Most of the pieces I included in the test are things I found easily by doing a web search - meaning that most of the approaches I used were the most common programming methods that people are likely using. This shows how much faster IDL might be than a general program than something that someone might write in Python.





The test:

Here is what was actually tested between IDL and Python with an array of [10000,10000] or 10000*10000 elements

-Array creation time
-Type conversion times
-Index array creation times (i.e. [0,1,2,3,4...,n-1])

-Incrementing array values of all elements by 1

-Complex math expression with array (exact equation: sqrt(sin(arr*arr)))

-Single threaded for IDL and multithreaded

-Array element access times (i.e. setting y = arr[i])

-Simple image processing filter times (filters: sobel, roberts, prewitt)




The results:

Average array creation time (seconds):

    Python : 0.213000 +/- 0.00953933

    IDL    : 0.0936666 +/- 0.0155028

Total time (seconds):

    Python : 0.639000

    IDL    : 0.281000

Python/IDL time ratio: 2.27402

Average array data type conversion time (seconds):

     Python : 0.171333 +/- 0.0155028

    IDL    : 0.0730000 +/- 0.00866031

Total time (seconds):

     Python : 0.514000

     IDL    : 0.219000

Python/IDL time ratio: 2.34703





Average index array creation time (seconds):

     Python : 0.229000 +/- 0.00866031

     IDL    : 0.124667 +/- 0.0160104

Total time (seconds):

    Python : 0.687000

    IDL    : 0.374000

Python/IDL time ratio: 1.83690





Average increasing array value time (seconds):

     Python : 0.0933333 +/- 0.000577446

     IDL    : 0.0313334 +/- 0.000577377

Total time (seconds):

    Python : 0.280000

    IDL    : 0.0940001

Python/IDL time ratio: 2.97872





Average complex math statements (1 thread) time (seconds):

    Python : 6.36967 +/- 0.0645319

    IDL    : 8.34667 +/- 0.0155028

Total time (seconds):

    Python : 19.1090

    IDL    : 25.0400

Python/IDL time ratio: 0.763139





Average complex math statements (8 thread) time (seconds):

    Python : 6.34400 +/- 0.0321871

    IDL    : 1.93933 +/- 0.00923762

Total time (seconds):

    Python : 19.0320

    IDL    : 5.81800

Python/IDL time ratio: 3.27123





Average loop through array element time (seconds):

    Python : 11.5290 +/- NaN

    IDL    : 3.29100 +/- NaN

Total time (seconds):

    Python : 11.5290

    IDL    : 3.29100

Python/IDL time ratio: 3.50319





Average image processing routines time (seconds):

    Python : 15.3660 +/- 0.0829635

    IDL    : 1.39900 +/- 0.0238955

Total time (seconds):

    Python : 46.0980

    IDL    : 4.19700

Python/IDL time ratio: 10.9836







Conclusion:



In short, IDL significantly outperformed Python will all the speed tests apart from the complex math statement. However, IDL has access to built in multithreading for large arrays and, with multithreading enabled, IDL outperforms Python significantly when using all available cores.



Below is the IDL code used to compare the processing speed of IDL and Python. To use it you will need a few Python modules which can be found at the beginning of the procedure "python_test". 

IDL Code:
pro printTimes, pythontimes, idltimes, TIMED = timed
  compile_opt idl2
  
  ;check if we have a name for the process to print with the mean
  if ~keyword_set(timed) then begin
    add = ' time (seconds):'
  endif else begin
    add = timed + ' time (seconds):'
  endelse
  
  print, 'Average ' + add 
  print, '  Python : ' + strtrim(pythontimes.mean(),2) + ' +/- ' + strtrim(stddev(pythontimes),2)
  print, '  IDL    : ' + strtrim(idltimes.mean(),2) + ' +/- ' + strtrim(stddev(idltimes),2)
  print, 'Total time (seconds):'
  print, '  Python : ' + strtrim(total(pythontimes),2)
  print, '  IDL    : ' + strtrim(total(idltimes),2)
  print, 'Python/IDL time ratio: ' + strtrim(total(pythontimes)/total(idltimes),2)
  
end

pro python_test
  compile_opt idl2
  
  
  ;initialize Python
  >>> 'import numpy as np'
  >>> 'import time'
  >>> 'import math'
  >>> 'from idlpy import *'
  >>> 'from skimage.filters import roberts, sobel, prewitt'
  
  ;number of times we want to run each test
  nloops = 3
  
  ;array dimensions for iterator tests
  dims = 10000
  ;>>>'dims = ' + strtrim(dims,2)
  python.dims = dims
  
  ;array dimensions for filter tests
  dims_filter = 10000
  python.dims_filter = dims_filter
  
  ;initialize arrays to hol dpython times
  pythontimes = fltarr(nloops)
  idltimes = fltarr(nloops)
  
  ;test array creation in Python and IDL
  for i=0,nloops-1 do begin
    tStart = systime(/seconds)
    >>> 'arr = np.ones((dims,dims))'
    tEndPython = systime(/seconds)
    arr = fltarr(dims,dims)
    tEndIDL = systime(/seconds)
    
    ;save the times
    pythontimes[i] = tEndPython - tStart
    idltimes[i] = tEndIDL - tEndPython
  endfor
  print
  printTimes, pythontimes, idltimes, TIMED = 'array creation'

  ;check type conversion times
  for i=0,nloops-1 do begin
    tStart = systime(/seconds)
    >>> 'arr2 = arr.astype(int)'
    tEndPython = systime(/seconds)
    arr2 = fix(arr)
    tEndIDL = systime(/seconds)
    
    ;save the times
    pythontimes[i] = tEndPython - tStart
    idltimes[i] = tEndIDL - tEndPython
  endfor
  print
  printTimes, pythontimes, idltimes, TIMED = 'array data type conversion'

  ;check index array creation times
  for i=0,nloops-1 do begin
    tStart = systime(/seconds)
    >>> 'arr = np.arange(0, dims*dims, dtype=long)'
    tEndPython = systime(/seconds)
    arr = lindgen(dims*dims)
    tEndIDL = systime(/seconds)

    ;save the times
    pythontimes[i] = tEndPython - tStart
    idltimes[i] = tEndIDL - tEndPython
  endfor
  print
  printTimes, pythontimes, idltimes, TIMED = 'index array creation'

  ;check adding values to array
  for i=0,nloops-1 do begin
    tStart = systime(/seconds)
    >>> 'arr += 1'
    tEndPython = systime(/seconds)
    arr += 1
    tEndIDL = systime(/seconds)

    ;save the times
    pythontimes[i] = tEndPython - tStart
    idltimes[i] = tEndIDL - tEndPython
  endfor
  print
  printTimes, pythontimes, idltimes, TIMED = 'increasing array value'

  ;check complex math expressions with a single thread
  pref_set, 'IDL_CPU_TPOOL_NTHREADS', 1, /commit
  for i=0,nloops-1 do begin
    tStart = systime(/seconds)
    >>> 'y = np.sin(arr*arr)**.5'
    tEndPython = systime(/seconds)
    y = sqrt(sin(arr*arr))
    tEndIDL = systime(/seconds)

    ;save the times
    pythontimes[i] = tEndPython - tStart
    idltimes[i] = tEndIDL - tEndPython
  endfor
  print
  printTimes, pythontimes, idltimes, TIMED = 'complex math statements (' + strtrim(!CPU.TPOOL_NTHREADS,2) + ' thread)'
  pref_set, 'IDL_CPU_TPOOL_NTHREADS', /default, /commit
  
  ;check complex math expressions with all threads
  for i=0,nloops-1 do begin
    tStart = systime(/seconds)
    >>> 'y = np.sin(arr*arr)**.5'
    tEndPython = systime(/seconds)
    y = sqrt(sin(arr*arr))
    tEndIDL = systime(/seconds)

    ;save the times
    pythontimes[i] = tEndPython - tStart
    idltimes[i] = tEndIDL - tEndPython
  endfor
  print
  printTimes, pythontimes, idltimes, TIMED = 'complex math statements (' + strtrim(!CPU.TPOOL_NTHREADS,2) + ' thread)'

  ;check array element access times
  nhere = 1
  for i=0,nhere-1 do begin
    tStart = systime(/seconds)
    >>>'for x in np.nditer(arr):\n    y=x'
    tEndPython = systime(/seconds)
    foreach x, arr do y = x
    tEndIDL = systime(/seconds)

    ;save the times
    pythontimes[i] = tEndPython - tStart
    idltimes[i] = tEndIDL - tEndPython
  endfor
  print
  printTimes, pythontimes[0:nhere-1], idltimes[0:nhere-1], TIMED = 'loop through array element'

  ;check times for image processing
  im_dat = lonarr(dims_filter, dims_filter)
  ;set a square in the middle of the image to 1
  im_dat[.4*dims_filter:.6*dims_filter,.4*dims_filter:.6*dims_filter] = 1
  ;send the array to Python as well
  python.im_dat = im_dat
  
  for i=0,nloops-1 do begin
    tStart = systime(/seconds)
    >>> 'edge_sobel = sobel(im_dat)'
    >>> 'edge_roberts = roberts(im_dat)'
    >>> 'edge_prewitt = prewitt(im_dat)'
    tEndPython = systime(/seconds)
    edge_sobel = sobel(im_dat)
    edge_roberts = roberts(im_dat)
    edge_prewitt = prewitt(im_dat)
    tEndIDL = systime(/seconds)

    ;save the times
    pythontimes[i] = tEndPython - tStart
    idltimes[i] = tEndIDL - tEndPython
  endfor
  print
  printTimes, pythontimes, idltimes, TIMED = 'image processing routines'

  stop
end
Please login or register to post comments.