Running ENVI Routines thru the Java-IDL Export Bridge

Topic
The purpose of this Help Article is threefold:

1. Describe in detail steps to create a Java-IDL Export Bridge program using the Eclipse Java Development environment.

2. Show the special syntax that is required in both IDL and Java source code to make a Java-IDL Export Bridge program which uses ENVI "batch programming" API calls.

3. Provide tips to solitary programmers (as opposed to programmers sharing code in teams) on one reasonable strategy for storing the files needed by your independent Java Workbench and IDL DE projects.

This first example in our Help Articles is a Console Program, demonstrating only text output to a DOS or UNIX shell. The ENVI programming API can also be used to populate graphics windows, but that demonstration is for another Help Article or Code Contribution Library item in the future. We should mention also that this first example in our Help Articles demonstrates making the interface with a custom IDL class and with the IDL Export Bridge Assistant.

The functionality demonstrated in this example could also have been accomplished with the built-in "IDL Connector Object," a Java class named "java_IDL_connect" integrated in every IDL installation. That approach requires fewer code lines, but is not as flexible or powerful as the approach discussed in this Help Article.

Discussion
The specific demonstration that this example performs is, through Java, opening up an ENVI '.img' file and executing the example code lines cited in ENVI's Online Help for ENVI_STATS_DOIT. The underlying IDL/ENVI code then returns the stats it acquires to the Java caller, which sends the string values of the imported data to 'stdout' (i.e. to the terminal console). Involved in this demonstration are: 

a) project initialization and a small amount of coding in the Eclipse Java Developer Environment (presumably very similar to the Sun Java NetBeans Developer Enviroment), 

b) rudimentary object class creation in the IDLDE, which includes rudimentary calls to the ENVI programming API (an ENVI+IDL developer license is needed for this),

c) use of the IDL Export Bridge Assistant to prepare the IDL code for interface to Java, and, finally, 

d) recommended organization in your file system of the files that make up this application.

A typical Java-IDL Export Bridge ("JIDL") project starts in the IDL Development Environment, where the functionality you want to implement needs to be "cast" via a custom IDL Object Class. The IDL Export Bridge Assistant is only designed to interpret Object Class definitions and methods. The full code for the functionality in our example is shown at the bottom of this page. IDL's Online Help 'Contents tab page -> Programmer's Guides -> Object Programming -> Creating Custom Objects in IDL' has the details on this kind of coding, if you have never learned it before. In this Help Article we only present topics that are specific to implementing the ENVI interface.

SYNTAX OF THE IDL SOURCE CODE
The source code shown below performs the following functionality:

1) it stores data acquired both from Java and from ENVI in its main class structure defined in TEST_ENVI_OBJECT__DEFINE. By doing this it is easy to write "getter and setter" methods with one line of code, e.g. "return, self.myMeanValues".

2) It has an INIT method (the method triggered by OBJ_NEW in IDL, alternatively by 'createObject' in export bridge Java) which not only initializes ENVI, but also opens the file passed to it as an argument to 'createObject'. INIT also makes the ENVI_STATS_DOIT call that eventually populates the class structure. 3) It implements a few very generic "getter" methods to demonstrate passing of both scalar values and numeric array values back to Java.

Here are the main tips that can be learned from the Example Code below:

ENVI calls must be wrapped in CALL_PROCEDURE syntax.
The IDL Export Bridge Assistant has an undocumented dependence on RESOLVE_ALL, an IDL procedure with which ENVI batch programming is not entirely compatible. If you cannot run RESOLVE_ALL error/warning-message-free from the IDL> command prompt, then that same IDL> prompt will throw fatal errors at the next call to IDLEXBR_ASSISTANT and IDL will not be able to generate the essential bridging Java code. If your IDL class code has any direct ENVI_... calls in it, RESOLVE_ALL will always throw at least one error/warning message. However, if you wrap your ENVI API calls as string arguments to IDL's EXECUTE function or CALL_PROCEDURE procedure, their incompatibility with RESOLVE_ALL and IDLEXBR_ASSISTANT is fixed. The example code at the bottom of this page demonstrates the easy syntax for CALL_PROCEDURE

ROUTINE_INFO can be particularly useful in Java-IDL Export Bridge projects.
The current working directory for the IDL process triggered by the export bridge program is the '.../idl/bin/bin.[OS].[chipset]' directory. This is not a directory where a developer would normally want to store his/her custom program's data or resource files. The ROUTINE_INFO algorithm shown in the example code below shows a good way to get the IDL process to dynamically find the root directory of the IDL application.

Your own INIT implementations do not need to take any arguments. Arguments to INIT really complicate the eventual syntax needed for 'createObject' in the Java caller, discussed later in this article.


FILE SYSTEM STRATEGIES FOR THE IDL SOURCE CODE
The new Eclipse-based IDL Workbench recommends having all IDL applications stored in (or linked into) an "IDLWorkspace" directory tree in your home directory. At the same time, the Eclipse-based Java Developer Environment defaults to storing Java applications in a directory named just "workspace" in the same home directory. To confuse matters further, the JIDL tools in IDL produce not just '.sav' files, but '.java' and '.class' files as well in your IDL app's directory tree. What is a smart strategy for storage of JIDL export bridge projects for developers relatively new to Java, IDL or Eclipse? We think this is one reasonable approach:

1. Create two same-named project (or "application root") subdirectories, one in your IDL source code directory, the other in your main Java source code directory (let's call this the "Java workspace"). If you are working with Eclipse, let the Eclipse project wizard dialogs do this for you. "ExJavaEnviConsole" would be a good name for this project.

2. Put your IDL source code, 'test_envi_object__define.pro', in the IDL application root directory.

3. When you run the IDL Export Bridge Assistant (next section), let it create and populate its default Java "package" directory. In this example it will be called "test_envi_object".

4. Create, through the Eclipse Java Developer Environment ("JDE"), a same-named subdirectory in the Java workspace, e.g. '~/workspace/ExJavaEnviConsole/test_envi_object/'. The '.java' file produced by IDL in the last step should eventually be moved to this directory (see the 'Developing the Java Project in Eclipse' section below).

5. The '.class' file produced in the IDL application directory tree should be deleted. Eclipse/Java will automatically rebuild it in the location it is needed.

6. ***Make sure that your IDL Search Path includes this IDL application's root directory***!. Your Java interface program will find the IDL class '.pro' file that it needs if it is in your IDL Search Path. You do not need to move IDL '.pro' or '.sav' files into the Java application tree. Some developers may need to set their IDL Search Path through a system/shell environment variable named "IDL_PATH" to get the Java dvelopment environment to use the same IDL search path as the IDL development environment.

You are now ready to compile your IDL app and get started on Java programming.


RUNNING THE IDL EXPORT BRIDGE ASSISTANT

There are no special requirements here beyond what is instructed in the IDL Connectivity Bridges manual. You run IDLEXBR_ASSISTANT from the IDL> command line. You choose 'File->New Project->Java...' from the assistant menu and browse to the '.pro' you created (in this case 'test_envi_object__define.pro'). In this example, the only modifications that have to be made are in the properties of the three methods. GETDATAFILEPATH needs to be set to export a scalar JIDLString. GETMEANS and GETSTANDARDDEVIATIONS need to be set to export array JIDLNumber's; we don't care about the 'Convert Majority' property for the 1D arrays these methods return. 'File->Save' the project. That creates the 'test_envi_object_java_wrapdef.sav' in the application root directory. Then select 'Build->Build Object' in the assistant menu. That creates the package subdirectory ('test_envi_object'), generates in this subdirectory the Java source code for the interface to the Java program, and compiles it into a '.class' binary. As discussed in 'File System Strategies' above, if you are developing your Java program in Eclipse, throw away the '.class' file just generated by IDL. At this point you can exit IDL; its work is all done.


DEVELOPING THE JAVA PROJECT IN ECLIPSE

Readers who do not have a Java development environment loaded on their system will find that the main freeware options on the Internet install and work robustly and intuitively "out of the box". This Help Article provides instructions based on the Eclipse Java Developer Environment, but Sun Java NetBeans environment users will probably have little trouble translating the instructions below into a corresponding workflow in NetBeans. In any case, here is the URL for the freeware Eclipse Java download: http://www.eclipse.org/downloads/.

Our recommended routine for the Eclipse JDE workflow for this project is:

1. File -> New -> Project -> Select 'Java Project' and hit 'Next' to bring up the 'New Java Project' wizard.

2. On the first dialog window enter the same Project name you used in your IDL workspace ("ExJavaEnviConsole"). Use radio button settings 'Create new project in workspace', 'Use default compiler compliance' and 'Use project folder as root ...'. Hit 'Next>'.

3. On the 'Java Settings' dialog window select the 'Libraries' tab page and click on the 'Add External JARs...' button. Navigate to your IDL's '...\resource\bridges\export\java\' directory, select the 'javaidlb.jar' file and hit the 'Open' button. The 'javaidlb.jar' entry should appear in the JARs list now. Click 'Finish'.

4. Your 'Package Explorer' should now have an 'ExJavaEnviConsole' entry. Right-click on this entry and select 'New -> Package' from the context menu. Call this package by the name of your IDL class: 'test_envi_object'. We advise this because that is the default package name for the '.java' file that the IDL Export Bridge Assistant created. We want the JDE's auto-generated package to match IDL's auto-generated package.

5. Right-click on your new 'test_envi_object' Package entry in the JDE's Package Explorer and select 'New -> Class' from the context menu. On the ensuing dialog, you could call your class anything, but giving it the same name as the Eclipse project name ("ExJavaEnviConsole") is a perfectly good strategy. For this very basic Java program allowing the class to be 'Public' and setting just the 'public static void main(String[] args)' and 'Constructors from superclass' checkboxes is all that is needed. The template for the main Java source code file will immediately appear in your Eclipse editor window.

6. Before you compose the Java source code, import IDL's bridging '.java' file. Right-click on the project name in the JDE's Package Explorer, select 'Import... -> General -> File System', click 'Next>', 'Browse...' to your IDL workspace and select its 'test_envi_object' directory. This will populate the list boxes with a view of your 'test_envi_object.java' file. Check that file, check the 'Create selected folders only' radio button and click 'Finish'.

7. That last step may create a new package icon named "(default package)" in the JDE's Package Explorer window. Expand that package and click and drag the 'test_envi_object.java' icon into the 'test_envi_object' package node right below. This will do two things: a) Get rid of the "(default package)" reference, and b) Physically place a copy of the 'test_envi_object.java' in the right location in the JDE workspace's project directory.

Now we can take the final step of composing and building/compiling the Java program.


SYNTAX FOR THE JAVA CALLING PROGRAM

Right below this paragraph is the source code for the Java interface to IDL's bridging class (which produces the main executable for this application). Comments in the source explain what the most critical code lines are doing or why they are there. In general, besides showing the basic steps and syntax involved in calling IDL class methods in Java, this source code example demonstrates: a) Issues involved in initializing the bridge to IDL, particularly the complex syntax needed if an IDL class's Init method takes arguments, and b) the special syntax for passing the return values, both scalar and array, of IDL class functions back to the Java calling functions.

   package test_envi_object;
   import com.idl.javaidl.*; // Don't forget this!

   public class ExJavaEnviConsole {
      private test_envi_object enviObj;
      private String targetFile;

      // This constructor passes 'filename' to the IDL class's constructor
      // which has ENVI open this file and extract its image statistics.
      // In this particular demo the IDL constructor knows to look for
      // 'filename' in the directory that holds the IDL source code.
      // There is no output from this constructor; it just populates some
      // critical memory fields.
      public ExJavaEnviConsole(String filename) {
         enviObj = new test_envi_object();
   
         // JIDL routine for passing one arg to IDL class's Init
         JIDLString filenameArg = new JIDLString(filename);
         final int ARGC = 1;
         Object[] argv = new Object[ARGC];
         int[] argp = new int[ARGC];
         argv[0] = filenameArg;
         argp[0] = JIDLConst.PARMFLAG_CONST;
         enviObj.createObject(ARGC, argv, argp);
         // End of routine for passing one arg. Having in the IDL class
         // code an 'Init' that takes arguments is so much more complex
         // than having an arg-free 'Init'. If we had had an arg-free
         // 'Init' in the IDL class - which was an option - then I would
         // have needed just one code line above (for 'createObject') and
         // one line below to a new method in the IDL class that I might
         // call "processEnviFile". The above approach is demonstrated
         // only because you might need to pass an Init arg in your own
         // production source code in the future.
         this.targetFile =
            (new JIDLString(enviObj.GETDATAFILEPATH())).stringValue();
      }

      // The one function in this app that produces example output.
      // More specifically, it demonstrates passing vector array data from
      // IDL to Java.
      public void printMeanValues() {
         JIDLArray meansArray = (JIDLArray)enviObj.GETMEANS();
         double[] means = (double[])meansArray.getValue();
         System.out.println("Mean values of bands in " +
         this.targetFile + ":");
         for (int i = 0; i < means.length; i++) {
            System.out.println("Band " + i + ": " + means[i]);
         }
      }

      public static void main(String[] args) {
         // 'can_tmr.img' and 'can_tmr.hdr' must be in the same directory
         // with the IDL '.pro' source code.
         ExJavaEnviConsole example = new ExJavaEnviConsole("can_tmr.img");
         // Print out the mean values of 'can_tmr.img'
         example.printMeanValues();
      }
   }

When the above is completed in the Eclipse JDE editor, click 'File -> Save'. Then right-click on the 'ExJavaEnviConsole' node in the Package Explorer view and select 'Build Project' from the context menu. This will compile the Eclipse JDE's '.java' files will be compiled into their '.class' binaries. Copy the ENVI example data files 'can_tmr.img' and 'can_tmr.hdr' (in ENVI's 'data' subdirectory) into the IDL application root directory ('ExJavaEnviConsole' in the IDL workspace).

To run the application - for the first time, anyway - right-click again on the 'ExJavaEnviConsole' node in the JDE and select 'Run As -> Java Application' . In the ensuing 'Select Java Application' dialog select 'ExJavaEnviConsole'. This will eventually display in the JDE's Console View the path of the data file and its mean values, both imported from ENVI.

That's it. Source code files for the IDL class '.pro' (shown below) and for the '.java' caller (shown above) are in the these two links:   test_envi_object__define.pro   ExJavaEnviConsole.java. That is just the user-defined source code, though. You still have to go through steps above to generate the additional files needed to make this program run.

Solution:

; Save in file: 'test_envi_object__define.pro'
; Syntax for testing in IDL:
;     oEnviStats = obj_new('test_envi_object', 'can_tmr.img')
;     print, oEnviStats->GETMEANS()
; Example of "Getter" functions that get array data, data that
; was acquired, in this case, in the INIT() call below
FUNCTION test_envi_object::GETMEANS
return, *self.means     ; returns an array of doubles
END

FUNCTION test_envi_object::GETSTANDARDDEVIATIONS
return, *self.stdDevs   ; returns an array of doubles
END

FUNCTION test_envi_object::GETDATAFILEPATH
return, self.dataFile
END

; ... etc. ...

; INIT will not only initialize ENVI, but open up the target 'file'
; argument, calculate statistics about its bands, and store these
; in the class structure "member variables". This Init example is
; an implementation of the same functionality shown in the
; example in ENVI Online Help for ENVI_STATS_DOIT.
FUNCTION test_envi_object::INIT, file
compile_opt STRICTARR
; First thing let's set current working directory to the directory
; holding the source code ... otherwise the JIDL bridge will set it
; to the '.../bin/bin.[OS].[chipset]/' directory ... not very useful
; for finding linked in resource and data files by relative paths.
pathInfo = routine_info('TEST_ENVI_OBJECT__DEFINE', /SOURCE)
sourceCodeDir = file_dirname(pathInfo.path)
cd, sourceCodeDir   ; This new loc is not actually exploited in this example
self.dataFile = filepath(file, ROOT_DIR=sourceCodeDir)
; Basic ENVI startup routines. ***ALL*** ENVI routines must be called
enviLogFile = filepath('jidl_envi_session.log', ROOT_DIR=sourceCodeDir)
call_procedure, "ENVI", /RESTORE_BASE_SAVE_FILES
call_procedure, "ENVI_BATCH_INIT", LOG_FILE=enviLogFile
if n_elements(file) ne 0 then $
    call_procedure, "ENVI_OPEN_FILE", self.dataFile, R_FID=fid $
else begin
    call_procedure, 'ENVI_BATCH_EXIT'
    return, -1
endelse
if (fid eq -1) then begin
    call_procedure, 'ENVI_BATCH_EXIT'
    return, -2
endif
call_procedure, 'ENVI_FILE_QUERY', fid, DIMS=dims, NS=nSamples, $
    NL=nLines, NB=nBands
; Set the POS keyword to calculate stats for all bands
pos  = lindgen(nBands)
; Calculate the basic statistics and the histogram for the data file.
call_procedure, 'ENVI_DOIT', 'ENVI_STATS_DOIT', FID=fid, POS=pos, $
    DIMS=dims, COMP_FLAG=3, DMIN=dataMins, DMAX=dataMaxes, $
    MEAN=means, STDV=stdDevs, HIST=hists
self.dataMins = ptr_new(dataMins)
self.dataMaxes = ptr_new(dataMaxes)
self.means = ptr_new(means)
self.stdDevs = ptr_new(stdDevs)
self.hists = ptr_new(hists)
return, 1
END

; Implements routine pointer-freeing steps
PRO test_envi_object::CLEANUP
ptr_free, self.dataMins, self.dataMaxes, self.means, self.stdDevs, self.hists
END

; Set up to store most anything returned by ENVI_STATS_DO_IT
PRO test_envi_object__define
struct = { test_envi_object, $
    dataFile:'' , $
    dataMins:ptr_new(), $
    dataMaxes:ptr_new(), $
    means:ptr_new(), $
    stdDevs:ptr_new(), $
    hists:ptr_new() $
}
END