FXBREADM
Name
FXBREADM
Purpose
Read multiple columns/rows from a disk FITS binary table file.
Explanation
A call to FXBREADM will read data from multiple rows and
multiple columns in a single procedure call. Up to forty-nine
columns may be read in a single pass; the number of rows is
limited essentially by available memory. The file should have
already been opened with FXBOPEN. FXBREADM optimizes reading
multiple columns by first reading a large chunk of data from
the FITS file directly, and then slicing the data into columns
within memory. FXBREADM can read variable-length arrays (see
below).
The number of columns is limited to 49 if data are passed by
positional argument. However, this limitation can be overcome
by having FXBREADM return the data in an array of pointers.
The user should set the PASS_METHOD keyword to 'POINTER', and an
array of pointers to the data will be returned in the POINTERS keyword.
The user is responsible for freeing the pointers; however,
FXBREADM will reuse any pointers passed into the procedure, and
hence any pointed-to data will be destroyed.
FXBREADM can also read variable-length columns from FITS
binary tables. Since such data is not of a fixed size, it is
returned as a structure. The structure has the following
elements:
VARICOL: ;; Flag: variable length column (= 1)
N_ELEMENTS: ;; Total number of elements returned
TYPE: ;; IDL data type code (integer)
N_ROWS: ;; Number of rows read from table (integer)
INDICES: ;; Indices of each row's data (integer array)
DATA: ;; Raw data elements (variable type array)
In order to gain access to the Ith row's data, one should
examine DATA(INDICES(I):INDICES(I+1)-1), which is similar in
construct to the REVERSE_INDICES keyword of the HISTOGRAM
function.
Calling Sequence
FXBREADM, UNIT, COL, DATA1, [ DATA2, ... DATA48, ROW=, BUFFERSIZE = ]
/NOIEEE, /NOSCALE, /VIRTUAL, NANVALUE=, PASS_METHOD = POINTERS=,
ERRMSG = , WARNMSG = , STATUS = , /DEFAULT_FLOAT]
Input Parameters
UNIT = Logical unit number corresponding to the file containing the
binary table.
COL = An array of columns in the binary table to read data
from, either as character strings containing column
labels (TTYPE), or as numerical column indices
starting from column one.
Outputs
DATA1, DATA2...DATA48 = A named variable to accept the data values, one
for each column. The columns are stored in order of the
list in COL. If the read operation fails for a
particular column, then the corresponding output Dn
variable is not altered. See the STATUS keyword.
Ignored if PASS_METHOD is 'POINTER'.
Optional Input Keywords
ROW = Either row number in the binary table to read data from,
starting from row one, or a two element array containing a
range of row numbers to read. If not passed, then the entire
column is read in.
/DEFAULT_FLOAT = If set, then scaling with TSCAL/TZERO is done with
floating point rather than double precision.
/NOIEEE = If set, then then IEEE floating point data will not
be converted to the host floating point format (and
this by definition implies NOSCALE). The user is
responsible for their own floating point conversion.
/NOSCALE = If set, then the output data will not be scaled using the
optional TSCAL and TZERO keywords in the FITS header.
Default is to scale.
VIRTUAL = If set, and COL is passed as a name rather than a number,
then if the program can't find a column with that name, it
will then look for a keyword with that name in the header.
Such a keyword would then act as a "virtual column", with the
same value for every row.
DIMENSIONS = FXBREADM ignores this keyword. It is here for
compatibility only.
NANVALUE= Value signalling data dropout. All points corresponding to
IEEE NaN (not-a-number) are converted to this number.
Ignored unless DATA is of type float, double-precision or
complex.
PASS_METHOD = A scalar string indicating method of passing
data from FXBREADM. Either 'ARGUMENT' (indicating
pass by positional argument), or 'POINTER' (indicating
passing an array of pointers by the POINTERS
keyword).
Default: 'ARGUMENT'
POINTERS = If PASS_METHOD is 'POINTER' then an array of IDL
pointers is returned in this keyword, one for each
requested column. Any pointers passed into FXBREADM will
have their pointed-to data destroyed. Ultimately the
user is responsible for deallocating pointers.
BUFFERSIZE = Raw data are transferred from the file in chunks
to conserve memory. This is the size in bytes of
each chunk. If a value of zero is given, then all
of the data are transferred in one pass. Default is
32768 (32 kB).
Optional Output Keywords
ERRMSG = If defined and passed, then any error messages will be
returned to the user in this parameter rather than
depending on the MESSAGE routine in IDL. If no errors are
encountered, then a null string is returned. In order to
use this feature, ERRMSG must be defined first, e.g.
ERRMSG = ''
FXBREAD, ERRMSG=ERRMSG, ...
IF ERRMSG NE '' THEN ...
WARNMSG = Messages which are considered to be non-fatal
"warnings" are returned in this output string.
Note that if some but not all columns are
unreadable, this is considered to be non-fatal.
STATUS = An output array containing the status for each
column read, 1 meaning success and 0 meaning failure.
Calls
IEEE_TO_HOST, FXPAR(), WHERENAN()
Common :
Uses common block FXBINTABLE--see "fxbintable.pro" for more
information.
Restrictions
The binary table file must have been opened with FXBOPEN.
The data must be consistent with the column definition in the binary
table header.
The row number must be consistent with the number of rows stored in the
binary table header.
Generaly speaking, FXBREADM will be faster than iterative
calls to FXBREAD when (a) a large number of columns is to be
read or (b) the size in bytes of each cell is small, so that
the overhead of the FOR loop in FXBREAD becomes significant.
Side Effects
If there are no elements to read in (the number of elements is zero),
then the program sets !ERR to -1, and DATA is unmodified.
Category
Data Handling, I/O, FITS, Generic.
Prev. Hist. :
C. Markwardt, based in concept on FXBREAD version 12 from
IDLASTRO, but with significant and
major changes to accomodate the
multiple row/column technique. Mostly
the parameter checking and general data
flow remain.
C. Markwardt, updated to read variable length arrays, and to
pass columns by handle or pointer.
20 Jun 2001
C. Markwardt, try to conserve memory when creating the arrays
13 Oct 2001
Handle case of GE 50 columns, C. Markwardt, 18 Apr 2002
Handle case where TSCAL/TZERO changes type of column,
C. Markwardt, 23 Feb 2003
Fix bug in handling of FOUND and numeric columns,
C. Markwardt 12 May 2003
Removed pre-V5.0 HANDLE options W. Landsman July 2004
Fix bug when HANDLE options were removed, July 2004
Handle special cases of TSCAL/TZERO which emulate unsigned
integers, Oct 2003
Add DEFAULT_FLOAT keyword to select float values instead of double
for TSCAL'ed, June 2004
Read 64bit integer columns, E. Hivon, Mar 2008
Add support for columns with TNULLn keywords, C. Markwardt, Apr 2010
Add support for files larger than 2 GB, C. Markwardt, 2012-04-17