Hacking Animations on Jupyter

All the world's a vine, and all the graphics merely players

The Python bridge introduced in IDL 8.5 includes support for the IDL IPython Notebook Kernel.

I don't claim to be an expert at IPython or Jupyter notebooks. In fact I'd never used either before this week.

Of course, I decided the first thing I had to try with IDL would be relatively challenging.

But first, a little background on the fundamentals is required.

Jupyter and IDL

A Jupyter notebook is a very rich web-based interactive computational environment, as described by Wikipedia. Though designed as an interface for Python programming, through the IDL bridge the functionality is expanded to our orbit of influence as well.

In summary, you enter IDL commands in a web browser text entry box and display the resulting static text and graphics output in the same browser tab.

When you execute the commands via a UI element in the browser, the text is sent from the web server in the IPython notebook application through the Python bridge to an IDL Runtime process where the commands are executed sequentially, as if manually entered at the IDL command line.

 

The commands you enter into a notebook persist as a stand-alone file that can be shared with other users, provided they have installed both IDL and their notebook kernel.

Text output from IDL, for example from HELP or PRINT, will be echoed to the web page, as will certain types of graphics. Output is queued until an entire code cell has executed.

IDL Graphical Output

When graphics are generated by IDL, the image data is compressed into a PNG-format graphics buffer which is in-lined through BASE64 encoding into the HTML that populates the output Jupyter canvas in your browser. No stand-alone image file is needed. If you inspect the image element from above in your browser, you will see it represented like this:

By "in-line" the effect is as if you had written a PNG file to disk, then opened the file using OPENR and then executed READU on the file into a byte buffer. It's treated as a "blob" of data. The Base-64 encoding converts the binary data into ASCII for easy transmission over HTML. The web browser decodes the data back into its byte form, then reads it as if from an image file on disk.

Internally, the IDL interpreter is aware that it's running in the context of the Python bridge and does some special tricks to redirect your graphics output.

When graphics output is generated with Direct Graphics or function graphics and sent to the default display device, it's routed through a special pipeline that (in the end) creates the PNG in memory. As discussed below, Object Graphics output requires a slightly different idiom.

An upshot is that only the final graphics output that's been generated at the end of your command sequence will be sent back to the web page.

For example, if you were to execute a loop to animate through frames using the TV Procedure you will not get an animation in your web browser. Only the final display frame will be returned. and your browser will not receive any of the intermediate frames.

For comparison, execute the following in a Jupyter IDL notebook and compare the output against the same command run from a standard IDL command prompt.

In [0]: window,0&for i=0,3 do begin&tvscl,shift(dist(256),i*64,i*64)&wait,1.&endfor

(Special note:  In IDL 8.5, you will want to explicitly create a WINDOW prior to generating any Direct Graphics within a single code cell. Later versions may not require this.)

Object Graphics Tricks

For Object Graphics output, additional trickery lies under the hood.

If you attempt to use an IDLgrWindow or one of its family as your destination, you will find that a floating graphics window outside your web page will be created. Generally, this isn't what you want.

Instead, in the Python bridge idiom you must use an IDLgrBuffer object as your destination. This requirement is indeed documented but it takes some effort to find it.

Somewhere deep inside IDL it recognizes you're running a bridge object so data drawn to any IDLgrBuffer will be scraped at the completion of your commands then compressed into a PNG.

Each IDLgrBuffer will produce a separate canvas in your Jupyter web page. You should keep this in mind if the code you are calling uses other IDLgrBuffer objects internally and those IDLgrBuffer object references are not destroyed before your script completes.

Animations

Because the default action of the Python bridge is to display screen scrapes of static image windows, we need to pull a couple tricks to generate and display animation.

First of all, if we want to generate an embedded animation for display in the notebook we must write the frames to a format that we can embed in the HTML that the browser will interpret.

One option is to create an animated GIF that may be in-lined into the HTML. An example of this is shown below, a rotating sphere.