READCOL
Name
READCOL
Purpose
Read a free-format ASCII file with columns of data into IDL vectors
Explanation
Lines of data not meeting the specified format (e.g. comments) are
ignored. By default, columns may be separated by commas or spaces.
Use READFMT to read a fixed-format ASCII file. Use RDFLOAT for
much faster I/O (but less flexibility). Use FORPRINT to write
columns of data (inverse of READCOL).
If you sure that all lines meet the specified format (excluding
commented and SKIPed lines) then the speed for reading large files
can be significantly improved by setting the /QUICK keyword.
Calling Sequence
READCOL, name, v1, [ v2, v3, v4, v5, ... v40 , COMMENT=, /NAN
DELIMITER= ,FORMAT = , /DEBUG , /SILENT , SKIPLINE = , NUMLINE =
COUNT =, STRINGSKIP=
Inputs
NAME - Name of ASCII data file, scalar string.
Optional Input Keywords
FORMAT - scalar string containing a letter specifying an IDL type
for each column of data to be read. Allowed letters are
A - string data, B - byte, D - double precision, F- floating
point, I - short integer, L - longword, LL - 64 bit integer,
U - unsigned short integer, UL - unsigned long integer
Z - longword hexadecimal, and X - skip a column.
Columns without a specified format are assumed to be floating
point. Examples of valid values of FMT are
'A,B,I' ;First column to read as a character string, then
1 column of byte data, 1 column integer data
'L,L,L,L' ;Four columns will be read as longword arrays.
' ' ;All columns are floating point
If a FORMAT keyword string is not supplied, then all columns are
assumed to be floating point.
/SILENT - Normally, READCOL will display each line that it skips over.
If SILENT is set and non-zero then these messages will be
suppressed.
/DEBUG - If this keyword is non-zero, then additional information is
printed as READCOL attempts to read and interpret the file.
COMMENT - single character specifying comment character. Any line
beginning with this character will be skipped. Default is
no comment lines.
/COMPRESS - If set, then the file is assumed to be gzip compressed.
There is no automatic recognition of compressed files
by extension type.
DELIMITER - Character(s) specifying delimiter used to separate
columns. Usually a single character but, e.g. delimiter=':,'
specifies that either a colon or comma as a delimiter.
Set DELIM = string(9b) to read tab separated data
The default delimiter is either a comma or a blank.
/NAN - if set, then an empty field will be read into a floating or
double numeric variable as NaN; by default an empty field is
converted to 0.0.
/PRESERVE_NULL - If set, then spaces are considered to be valid fields,
useful if the columns contain missing data. Note that between
April and December 2006, /PRESERVE_NULL was the default.
/QUICK - If set, then READCOL does not check that each individual line
matches the supplied format. This makes READCOL less
flexible but can provide a significant speed improvement when
reading large files.
SKIPLINE - Scalar specifying number of lines to skip at the top of file
before reading. Default is to start at the first line.
NUMLINE - Scalar specifying number of lines in the file to read.
Default is to read the entire file
STRINGSKIP - will skip all lines that begin with the specified string.
(Unlike COMMENT this can be more than 1 character.) Useful to
skip over comment lines.
Outputs
V1,V2,V3,...V40 - IDL vectors to contain columns of data.
Up to 40 columns may be read. The type of the output vectors
are as specified by FORMAT.
Optional Output Keywords
COUNT - integer giving the number of valid lines actually read
NLINES - integer giving the total number of lines in the file
(as returned by FILE_LINES)
Examples
Each row in a file position.dat contains a star name and 6 columns
of data giving an RA and Dec in sexagesimal format. Read into IDL
variables. (NOTE: The star names must not include the delimiter
as a part of the name, no spaces or commas as default.)
IDL> FMT = 'A,I,I,F,I,I,F'
IDL> READCOL,'position.dat',F=FMT,name,hr,min,sec,deg,dmin,dsec
The HR,MIN,DEG, and DMIN variables will be integer vectors.
Alternatively, all except the first column could be specified as
floating point.
IDL> READCOL,'position.dat',F='A',name,hr,min,sec,deg,dmin,dsec
To read just the variables HR,MIN,SEC
IDL> READCOL,'position.dat',F='X,I,I,F',HR,MIN,SEC
Restrictions
This procedure is designed for generality and not for speed.
If a large ASCII file is to be read repeatedly, it may be worth
writing a specialized reader.
Columns to be read as strings must not contain the delimiter character
(i.e. commas or spaces by default). Either change the default
delimiter with the DELIMITER keyword, or use READFMT to read such files.
Numeric values are converted to specified format. For example,
the value 0.13 read with an 'I' format will be converted to 0.
Procedures Called
GETTOK(), STRNUMBER()
The version of STRNUMBER() must be after August 2006.
Revision History
Written W. Landsman November, 1988
Modified J. Bloch June, 1991
(Fixed problem with over allocation of logical units.)
Added SKIPLINE and NUMLINE keywords W. Landsman March 92
Read a maximum of 25 cols. Joan Isensee, Hughes STX Corp., 15-SEP-93.
Call NUMLINES() function W. Landsman Feb. 1996
Added DELIMITER keyword W. Landsman Nov. 1999
Fix indexing typos (i for k) that mysteriously appeared W. L. Mar. 2000
Hexadecimal support added. MRG, RITSS, 15 March 2000.
Default is comma or space delimiters as advertised W.L. July 2001
Faster algorithm, use STRSPLIT if V5.3 or later W.L. May 2002
Accept null strings separated by delimiter ,e.g. ',,,'
Use SCOPE_VARFETCH instead of EXECUTE() for >V6.1 W.L. Jun 2005
Added compile_opt idl2 W. L. July 2005
Added the NaN keyword W. L August 2006
Added /PRESERVE_NULL keyword W.L. January 2007
Assume since V5.6 (FILE_LINES available ) W.L. Nov 2007
Added COUNT output keyword W.L. Aug 2008
Added NLINES output keyword W.L. Nov 2008
Added SKIPSTART keyword Stephane Beland January 2008
Renamed SKIPSTART to STRINGSKIP to keep meaning of SKIP W.L. Feb 2008
Assume since V6.1, SCOPE_VARFETCH available W.L. July 2009
Read up to 40 columns W.L. Aug 2009
Use pointers instead of SCOPE_VARFETCH. Fixes bug with
IDL Workbench and runs 20% faster Douglas J. Marshall/W.L. Nov 2009
Recognize LL, UL, and ULL data types, don't use 'val' output from
STRNUMBER() W.L. Feb 2010
Graceful return even if no valid lines are present D. Sahnow April 2010
Ability to read tab separated data WL April 2010
Free memory used by pointers WL July 2010
Added /QUICK keyword WL Sep 2010
Accept normal FORTRAN formats (e.g. F5.1) P. Noterdaeme/W.L Jan 2011
Add COMPRESS keyword, IDL 6 notation W. Landsman/J. Bailin Feb 2011
Allow filename to be 1 element array W.Landsman/S.Antonille Apr 2011
Feb 2010 change caused errors when reading blanks as numbers.
W.L. July 2012