X
13656 Rate this article:
3.7

Jazzing Up Your Python-Embedded IDL Graphics With Gesture Interactivity

Jim Pendleton

In October 2015, my colleague Atle Borsholm posted a blog on the topic of Embedding an IDL graphic window in a Python GUI based on the IDL Python bridge added in IDL 8.5.

His example shows how to render IDL graphics into a GUI created on the Python side of the bridge, and how to animate the objects in the scene based on a timer fired within Python and a Python slider GUI element to adjust the animation's update rate. 

Because I'm not yet proficient in Python, I wanted to see how much effort it would take to add user interactivity to his example.

Specifically, I wanted to enable mouse button and drag events within the Python GUI, and pass them back to IDL where the graphics scene could be updated based on those events.

It turned out to be a surprisingly simple task, given that I'd never before worked with Qt directly and I needed to learn a few new concepts on the Python side as well. For reference, it took significantly longer to write this blog article than the code itself.

Here is a simplified animation of the result:

(I used an "AccumulatinggrBuffer" object described in an earlier blog to generate the animated GIF  data on the fly.)

 

New Behavior

Initially, the "planet" orbits the "sun".  The deviation occurs in the animation when I clicked and dragged the mouse. Interactive scene manipulation is driven in tandem with the timer-based "orbit". The effects are cumulative on the final scene.

IDL updates the scene via button event information passed to a Trackball object, a handy utility that keeps track of quaternions associated with objects or scenes. In this example, it can be conceptualized as a virtual mouse trackball that covers the entire display, with its origin at the center of the 3D scene. Click and drag to "roll" the "ball" causing a change in the orientation of the objects in the graphics scene.

There are only a few steps needed for this new functionality.

  1. Mouse button press, release, and drag event handlers are added to the original code.  These manage events on the Python side, within the context of the QtGUI.
  2. The Python event information is repackaged in a format suitable for the IDL Trackball object.
  3. The model transformation matrix is updated based on the mouse event information.
  4. The scene is redrawn.

Updated Code

Here is the full source.  Copy and paste it to a file, then execute the file from Python.

# GUI widgets
import PyQt4
from PyQt4 import QtGui
from PyQt4 import QtCore

import sys

# numpy needed for IDL
import numpy
from numpy import array
from idlpy import IDL

# OS is needed to handle Linux vs. Windows
import os

class MainWin(QtGui.QMainWindow):
  def __init__(self, win_parent=None):
    QtGui.QMainWindow.__init__(self,win_parent)

    # widget layout
    wcol = QtGui.QVBoxLayout()
    wrow = QtGui.QHBoxLayout()
    wcol.addLayout(wrow)
    self.check = QtGui.QCheckBox('Animate')
    self.slider = QtGui.QSlider(1)
    self.text = QtGui.QLabel('Speed')
    # EmbedContainer only works on Linux, for Windows use QWidget
    if os.name== 'nt':
      self.idldraw = QtGui.QWidget()
    else:
      self.idldraw = QtGui.QX11EmbedContainer()
    self.idldraw.setFixedHeight(500)
    self.idldraw.setFixedWidth(500)
    wrow.addWidget(self.check)
    wrow.addWidget(self.slider)
    wrow.addWidget(self.text)
    wcol.addWidget(self.idldraw)
    w = QtGui.QWidget()
    w.setLayout(wcol)
    self.setCentralWidget(w)
    self.setWindowTitle('IDL-Python Viewer Mark II')
    self.timer = QtCore.QTimer()
    # Set the inter-frame timer delay in milliseconds
    self.delay = 10
    self.slider.setValue(90)
    self.timer.start(self.delay)
    self.hasDraw = False

    # connect event handlers for slider and timer
    QtCore.QObject.connect(self.check, 
        QtCore.SIGNAL('clicked()'),
        self.on_check_clicked)
    QtCore.QObject.connect(self.timer, 
        QtCore.SIGNAL('timeout()'),
        self.on_timeout)

  def on_check_clicked(self):
    # Animate checkbox was clicked for the first time
    if self.hasDraw == False:
      # Create the IDL graphics objects only once
      # Windows comes up as 'nt'
      if os.name == 'nt':
        winid = int(self.idldraw.winId())
      else:
        winid = self.idldraw.effectiveWinId()
      # Create the IDLgrWindow, using software rendering
      # connect to the Python widget using EXTERNAL_WINDOW
      self.win = IDL.obj_new('IDLgrWindow', RENDERER=1, EXTERNAL_WINDOW=winid)
      self.outermodel = IDL.obj_new('IDLgrModel')
      # We want copies of the references to the draw window and top-level model on the IDL side, too.
      IDL.outermodel = self.outermodel
      IDL.owindow = self.win
      # We use IDL.run when we need the values of returned keyword parameters in Python
      IDL.run('owindow.getproperty, DIMENSIONS = d')
      self.winsize = IDL.d
      # Create a graphics hierarchy.  An "outer" model is controlled by the trackball.
      self.view = IDL.obj_new('IDLgrView',COLOR=array([128,255,128]),
          PROJECTION=2,DEPTH_CUE=array([0,1]),EYE=1.5)
      self.view.add(self.outermodel)
      # A system model within the outer model has the animated motion of the orbit.
      self.systemmodel = IDL.obj_new('IDLgrModel')
      self.outermodel.add(self.systemmodel)
      self.planet = IDL.obj_new('orb',RADIUS=0.1,COLOR=array([20,20,255]))
      self.moon = IDL.obj_new('orb',RADIUS=0.03,POS=array([0.75,0,0]),
          COLOR=array([128,128,128]))
      self.sun = IDL.obj_new('IDLgrLight',TYPE=1,LOCATION=array([0,18,8]))
      self.systemmodel.rotate(array([1,0,0]), 10)
      self.outermodel.add(self.sun)
      self.outermodel.add(self.planet)
      self.systemmodel.add(self.moon)
      self.hasDraw = True
      # Create a trackball that manages left mouse button events
      trackball = IDL.obj_new('trackball',
        array([self.winsize[0]/2,self.winsize[1]/2]), self.winsize[0]/2, MOUSE=1)
      IDL.trackball = trackball
      # We are not running XMANAGER in IDL, so we need to synthesize the event
      # structures needed by the Trackball::Update method.  Create a proxy
      # structure and populate it with some default values.
      IDL.run('proxyEvent = {WIDGET_DRAW}')
      IDL.run('proxyEvent.press = 1')
      IDL.run('proxyEvent.release = 1')
      self.start = None

    if self.hasDraw:
      self.win.draw(self.view)

  def on_timeout(self):
    # handle timer event, draw the scene, rotate by 2 degrees
    # and check the slider value to determine the new
    # timer interval
    if self.hasDraw & self.check.isChecked():
      self.systemmodel.rotate(array([0,0.9848,0.1736]),-2)
      self.win.draw(self.view)
      # slider goes from 0-99, so delay goes from 1 ms to 100 ms
      speed = 100-self.slider.value()
      if self.delay != speed:
        self.delay = speed
        self.timer.setInterval(self.delay)

  def mousePressEvent(self, event):
    # handle mouse button press events, filtering for only
    # the left button.
    if event.button() == QtCore.Qt.LeftButton:
      mouseEvent = QtGui.QMouseEvent(event)
      # start of a drag gesture, which will rotate the scene
      self.start = mouseEvent;
      self.updateIDLTrackball(mouseEvent, 0)
 
  def mouseMoveEvent(self, event):
    # handle mouse motion events, but only watch if
    # a click but no release has occurred
    if self.start is not None:
      mouseEvent = QtGui.QMouseEvent(event)
      self.updateIDLTrackball(mouseEvent, 2)

  def mouseReleaseEvent(self, event):
    # handle mouse relese events, filtering for only
    # the left button
    if event.button() == QtCore.Qt.LeftButton:
      mouseEvent = QtGui.QMouseEvent(event)
      self.updateIDLTrackball(mouseEvent, 1)
      self.start = None
  
  def updateIDLTrackball(self, mouseEvent, eventType):
    # transfer information from the Python mouse event object into
    # the proxy structure passed to the Trackball::Update method
    # then update the scene
    IDL.run('proxyEvent.x = ' + str(mouseEvent.x()))
    # Python's coordinate system has an origin at the upper left
    # of the draw window.  The Y axis coordinate is reversed relative to IDL.
    IDL.run('proxyevent.y = ' + str(self.winsize[1] - 1 - mouseEvent.y()))
    IDL.run('proxyevent.type = ' + str(eventType))
    # The Trackball::Update method does not require other WIDGET_DRAW
    # structure tags to be populated, such as TOP, ID, or HANDLER.
    IDL.run('hasTransform = trackball.update(proxyevent, TRANSFORM=updatetransform)')
    if IDL.hasTransform == 1:
       IDL.run('outermodel.getproperty, TRANSFORM = oldtransform')
       IDL.run('outermodel.setproperty, TRANSFORM = oldtransform # updatetransform')
       self.win.draw(self.view)  

if __name__ == '__main__':
# Someone is launching this directly
# Create the QApplication
    app = QtGui.QApplication(sys.argv)
# The Main window
    main_window = MainWin()
    main_window.show()
# Enter the main loop
    app.exec_()

Click on the "Animate" button in the upper left of the UI to kick things off.

I've slightly rearranged the graphics model hierarchy relative to Atle's original example. There is now a separate container model for the orbiting "planet" and another outer model that contains all the inner objects, including the planet model. It is the outer model that will be updated with the trackball gestures while the inner planet model continues to be updated by the timer event.

Choosing the Appropriate Technique For Calling an IDL Routine

In the initialization section of the application, I create an IDL trackball object.

As with other IDL functions, when creating an IDL object within Python, we frequently have a choice for the calling convention.

First, we may construct an IDL command as a string then execute it via the Python to IDL Bridge's idlpy.run() method. Second, we may call IDL's OBJ_NEW function using the Python dot notation and pass arguments that are Python variables.

For example, which of these two variants is more appropriate?

IDL.run("otrackball = obj_new('trackball', [0,0], .5)")

vs.

otrackball = IDL.obj_new('trackball', array([0, 0]), .5)

As with most things, the answer is "it depends". The general rule of thumb is to use the idlpy.run method if you either do not intend to share the IDL variable data with Python, or if the function you are calling generates output parameters other than the function return value.

An example of the latter is as an object's ::GetProperty method. The dot notation style provides no mechanism for returning output keyword values back to Python from IDL. If that's the desired goal, a two-step process requiring the idlpy.run() notation is needed.

IDL.run("otrackball.getproperty, radius = radius")
pyrthonradius = IDL.radius

All scalar parameters and keywords are passed by value. The following version will not return the value you want, even if the variable "r" is already defined. 

IDL.otrackball.getproperty, radius = r

 

In the provided code, although I neither need the reference to the Trackball object on the Python side nor does the object creation pass back output parameters, it was syntactically simpler to construct the object using the dot notation and Python variables then pass the reference to the object back to IDL. The code will need a local $MAIN$-level variable reference to the object in its execution.

 

# Create a trackball that manages left mouse button events
trackball = IDL.obj_new('trackball',
   array([self.winsize[0]/2,self.winsize[1]/2]), self.winsize[0]/2, MOUSE=1)
IDL.trackball = trackball

Capturing Python Events

Three overriding methods have been added to Atle's original example.

def mousePressEvent(self, event):
def mouseMoveEvent(self, event):
def mouseReleaseEvent(self, event):

These methods implement functionality exposed by the QWidget class. In each case, we access the methods of the QMouseEvent object stored in the parameter "event" to ensure it's coming from the left mouse button.

A fourth new method is then responsible for repackaging the position information from the event into a proxy event structure.

def updateIDLTrackball(self, mouseEvent, eventType):

This method is custom to our task. Unlike the other three it doesn't override any abstract methods on this class. It will be described in a later section.

Translating Python Events Into IDL

The trackball object's Update method takes as input a WIDGET_DRAW event structure. (Scroll down to the section Widget Events Returned by Draw Widgets on the latter page for more information.)

If this were a pure IDL widget application, this event structure would be created for us within the context of IDL's event handling in a draw widget. The XMANAGER procedure would dispatch the structure we need to our widget's event handler.

Because Python is managing the events in its window, we are not using XMANAGER. We are therefore responsible for cooking up our own WIDGET_DRAW event structure.

During application initialization we create on the IDL side a {WIDGET_DRAW} structure and populate a couple of default tags. In a more general case we would populate all the tags on the fly in the Python event handlers based on the Python event data. In order to simplify the example code, only the left mouse button is being monitored.

IDL.run('proxyEvent = {WIDGET_DRAW}')
IDL.run('proxyEvent.press = 1')
IDL.run('proxyEvent.release = 1')

For our purposes, we do not need to populate the event structure's standard .TOP, .ID, or .HANDLER tags.  The Trackball::Update method doesn't use them.

When a Structure Needs to Live In IDL Only

I had originally attempted to use the convenient Python syntax to populate the event structure.

IDL.run('proxyEvent = {WIDGET_DRAW}')
proxy = IDL.proxy
proxy['PRESS'] = 1
proxy['RELEASE'] = 1

This is incorrect.  Although the Python bridge conveniently translates IDL structure data into a Python dict type, it's actually making a copy of each tag's data. The transform is a one-way street. There is no way from Python to then automatically repopulate the original structure's data. Attempting to pass a Python dict to an IDL routine expecting an IDL structure will result in Very Bad Things (TM).

Updating the Trackball

The updateIDLTrackball method extracts the position data from the Python mouse event and populates the position information in the proxy event created during initialization.

IDL.run('proxyEvent.x = ' + str(mouseEvent.x()))
# Python's coordinate system has an origin at the upper left
# of the draw window.  The Y axis coordinate is reversed relative to IDL.
IDL.run('proxyevent.y = ' + str(self.winsize[1] - 1 - mouseEvent.y()))
IDL.run('proxyevent.type = ' + str(eventType))

The updated proxy event is passed to the Trackball::Update method via idlpy.run. 

IDL.run('hasTransform = trackball.update(proxyevent, TRANSFORM=updatetransform)')  

In this case there are two reasons to use idlpy.run instead of dot notation.  We need to pass a structure as input, and the TRANSFORM keyword is an output value that we need subsequently in IDL.

If the mouse action has caused an update of the transform, we apply the transformation matrix to the outer model.

IDL.run('outermodel.getproperty, TRANSFORM = oldtransform')
IDL.run('outermodel.setproperty, TRANSFORM = oldtransform # updatetransform')

Finally, the scene is re-rendered.

self.win.draw(self.view)

Going Forward

It is interesting to consider the possibilities of adding new capabilities to IDL graphics via Python interfaces.  For example, PyQt provides a QTouchEvent class which gives us access to multi-touch environments.  An IDL application written on a traditional platform could in theory be extended via the Python bridge to incorporate events that are not currently supported in IDL directly.