NCDF_DATA__DEFINE Name
NCDF_DATA__DEFINE
Purpose
This program is designed to make it easier to browse and read the
data and metadata in netCDF and HDF files. The user can browse files,
and read the data and metadata into main-level IDL variables. New netCDF
and HDF files can be opened at any time. The user interacts with the
program via a browser window (GUI) or directly through the methods of
the object. The program implements an IDL object.
Note that only HDF files with scientific datasets (SD) can be read currently.
There is no support for VDATA objects or other objects sometimes found in HDF
files. Also note that when variables are returned from HDF files, they are returned
in a calibrated form, if calibration information about the variable is present in the
file. Calibration information is presented as an extra variable attribute in the
browser.
calibratedData = calData.cal * (uncalibratedData - calData.offset)
Author
FANNING SOFTWARE CONSULTING
David Fanning, Ph.D.
1645 Sheely Drive
Fort Collins, CO 80526 USA
Phone: 970-221-0438
E-mail: david@idlcoyote.com
Coyote's Guide to IDL Programming: http://www.idlcoyote.com
Category
File I/o
Calling Sequence
IDL> nCDFObject = Obj_New('NCDF_DATA', filename)
Arguments
filename: The name of a netCDF or HDF file to open and browse.
Keyword Parameters
BROWSE: If this keyword is set, the Browse Window is invoked as soon
as the object is initiated.
DESTROY_FROM_BROWSER: As with all objects, this object is persistent until
it is destroyed. However, with this keyword set, the object will
be destroyed when the user closes the Browse Window.
EXTENSION: In general, netCDF and HDF files use *.nc, *.ncf, *.ncdf of *.hdf file extensions to
identify themselves as netCDF or HDF files. Some users have their own file extensions.
You can use this keyword to identify the file extension you wish to use. If
set here, it will be used as the file filter in place of the normal file
extensions in DIALOG_PICKFILE.
obj = ('NCDF_DATA', file, EXTENSION='*.bin')
NO_READ_ON_PARSE: Normally, when a file is opened it is parsed for information.
One piece of information is the minimum and maximum values of the variables.
This requires actually reading the variables. This can slow things down
considerably is the variable is large. Setting this keyword will suppress
the reading of the variables during the parsing of the data file, with the
result that no minimum or maximum values will be reported.
Notes
This program is designed to be flexible in how it is used, so it
can be used in both interactive and non-interactive (called directly)
ways. A less flexible way of interacting with the program is via the
NCDF_BROWSER program, which is a front-end to this object.
The netCDF and HDF file formats are thought to be "standards". And to
a large extent, they are. But files are not always created to standards,
and both netCDF and HDF files can be quirky. If you look carefully at the
code you will see places where I work around quirks in the files I typically
use on a daily basis. If you find you can't read a particular file, let me know
about it. I may be able to improve the program in such as way that it can be read.
This program is not meant to be the be-all and end-all of programs. Rather, it is
a tool I use, and improve upon whenever necessary, in my own work with netCDF and HDF
files. It will get better for all of us if you report problems to me directly.
REQUIRES:
The following programs are required from the Coyote Library. And it is always a
good idea to make sure you have the latest version of the Coyote Library code,
as updates are irregular and frequent.
http://www.idlcoyote.com/programs/netcdf_data__define.pro
http://www.idlcoyote.com/programs/error_message.pro
http://www.idlcoyote.com/programs/centertlb.pro
http://www.idlcoyote.com/programs/undefine.pro
http://www.idlcoyote.com/programs/textbox.pro
http://www.idlcoyote.com/programs/fsc_base_filename.pro
http://www.idlcoyote.com/programs/textlineformat.pro
Methods
The following methods can be used directly.
ncdfObject -> Browse ; Use GUI to browse file data and metadata.
ncdfObject -> OpenFile, filename ; Opens a new netCDF or HDF file.
globalAttr = ncdfObject -> ReadGlobalAttr() ; Return a structure containing global attributes.
attribute = ncdfObject -> ReadAttribute(attrname); Return an attribute, identified by name.
dim = ncdfObject -> ReadDimension(dimName) ; Return a dimension, identified by name.
variable = ncdfObject -> ReadVariable(varname) ; Return a variable, identified by name.
varstruct = ncdfObject -> ReadVariableWithAttr(varname) ; Return a variable, identified by
; name, along with its attributes.
allData = ncdfObject -> ReadFile(filename) ; Read all the data in the file, into structures.
Example
IDL> filename = 'example.nc'
IDL> ncdfObj = Obj_New('NCDF_DATA', filename)
IDL> ncdfObj -> Browse
IDL> Obj_Destroy, ncdfObj
Modification History
Written by: David W. Fanning, 03 Feb 2008. Used ideas from many
people, including Chris Torrence, Ken Bowman, Liam Gumely,
Andrew Slater, and Paul van Delst.
Added EXTENSION keyword, resizeable TLB, and ability to download
individual global attibutes. DWF. 04 Feb 2008.
Added ReadDimension and ReadVariableWithAttr methods. DWF. 05 Feb 2008.
Ill-formed attribute names giving me fits. Now doing checks with IDL_VALIDNAME
before creating structures. 06 February 2008. DWF.
Same problem. Wide use of IDL_VALIDNAME everywhere it seems wise. 06 Feb 2008. DWF.
Added functionality to read a variable with its attributes from the browser interface,
and fixed a problem with reading CHAR values. 2 March 2008. DWF.
Fixed a problem with changing variable name when reading variable plus attributes. 6 March 2008. DWF.
Fixed a problem with not setting GLOBAL keyword when inquiring about global attribute. 6 March 2008. DWF.
Made sure file was parsed before attempting to read variables and attributes to avoid errors. 7 March 2008. DWF.
Small bug with variable attributes fixed. 18 Dec 2008. DWF.
Added ability to read HDF files containing Scientific Datasets (SD). 21 February 2009. DWF.
Added error handling and protection for NCDF variables that have a dimension of length zero. 22 April 2009. DWF.
Added NO_READ_ON_PARSE keyword. 22 April 2009. DWF.
Now convert NCDF CHAR type variables to strings on output. 22 April 2009. DWF
Fixed a problem with the directory being correct when file name passed in. 11 May 2009. DWF.
Added COUNT, OFFSET, and STRIDE keywords to ReadVariable method. 25 June 2009. DWF.
When reading a netCDF variable by itself (without it's attributes), the program now looks for
a SCALE_FACTOR and ADD_OFFSET attribute, and if found will apply this to the variable before
it is returned to the user. 24 August 2009. DWF.
Added the methods GetAttrNames, GetVarNames, GetVarAttrNames, and ReadVarAttr to retrieve specfic
information from the data files. 16 November 2009. DWF.
Modified the ReadVariableWithAttr method to include the number of dimensions (in the NDIMS field,
and the dimensions (in the DIMS field) in the return structure. For HDF files, the DIMS field
is a vector of the dimensions of the variable. For netCDF files, the DIMS field is a vector
of dimension IDs for the dimensions of the variable. 27 Nov 2009. DWF.
Andy Meigs alerted me to a problem creating a structure when the ncdf variable name
is ill-formed according to IDL structure tag name rules. Fixed in the ReadFile method.
30 November 2009. DWF.
Added NO_NEW_FILE keyword to the BROWSE method. This keyword will suppress the OPEN FILE
button on the browse interface. 3 Feb 2010. DWF.
Made the default browser size a bit larger to accomodate longer variable names. 3 Feb 2010. DWF.
Add a check for HDF/netCDF file type in the INIT method to better accommodate reading data
from the file without first parsing the file. 16 March 2010. DWF.
Changed the ReadVariable for netCDF files to now check for missing data, using either the
depreciated missing_value attribute or the compliant _FillValue attribute. Missing data
is now identified via new output keywords MISSINGINDICES and FILLVALUE, and missing data
is not scaled or offset, if these operations are applied to the data prior to return.
21 March 2010. DWF. Problem with these changes, fixed 23 March 2010. DWF.
Fixed a problem with memory leakage when the input file cannot be read. 1 May 2010. DWF.
Fixed a problem with memory leakage from created structures. 1 May 2010. DWF.
Have done some work on parsing HDF-EOS swath files, but currently unused in code. 15 May 2010. DWF.
Modified the ReadVariable method to check for 0 length dimensions when reading variables
from HDF files. 21 July 2010. DWF.
Modified the global attribute structure so that the "filename" field, which holds the
name of the netCDF of HDF file is now named "ncdf_filename" or "hdf_filename". This
will avoid conflicts with global attributes with "filename". 20 January 2011. DWF.