What are the input map coordinates (i.e. values of xmap/ymap)? File coordinates should not be negative. For example, can you reproduce this with ENVI's sample dataset, bhtmref.img? I did the following:
ENVI> xmap=276165.00
ENVI> ymap=4903605.0
ENVI> envi_select, fid=fid
ENVI> envi_convert_file_coordinates, fid, xf, yf, xmap, ymap
ENVI> print, xf
46.000000
ENVI> print, yf
110.00000
|