X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 02 Jul 2014 12:03 PM by  anon
H5A_* increasing slowness
 1 Replies
Sort:
You are not authorized to post a reply.
Author Messages

anon



New Member


Posts:
New Member


--
02 Jul 2014 12:03 PM
    I am creating an object to parse the metadata in HDF5 files in a manner similar to H5_Parse. One difference, though, is that all groups, attributes, datasets, etc. remain open until the object is destroyed. I am parsing many files. If I parse them individually, closing each one before parsing the next, execution time increases linearly with the number of files. If I parse all files at once (closing only after all have been parsed), however, the H5A_* routines taking an increasing amount of time. Is there a reason for this? Below is profiler output for the H5A_* routines for 1) a single file, 2) 10 files parsed together, and 3) 10 files parsed individually. Note the difference between cases for H5A_GET_NUM_ATTRS, H5A_OPEN_NAME, and H5A_OPEN_IDX. The difference compounds for each additional file. Single File ------------------------------------------------------------------------------------------------------- H5A_CLOSE (S) 4452 0.004479 0.000001 0.004479 0.000001 H5A_GET_NAME (S) 3904 0.003998 0.000001 0.003998 0.000001 H5A_GET_NUM_ATTRS (S) 92 0.001094 0.000012 0.001094 0.000012 H5A_GET_SPACE (S) 548 0.001016 0.000002 0.001016 0.000002 H5A_GET_TYPE (S) 548 0.000923 0.000002 0.000923 0.000002 H5A_OPEN_IDX (S) 3904 0.454829 0.000117 0.454829 0.000117 H5A_OPEN_NAME (S) 548 0.066381 0.000121 0.066381 0.000121 10 Files -------------------------------------------------------------------------------------------------------- H5A_CLOSE (S) 44520 0.050385 0.000001 0.050385 0.000001 H5A_GET_NAME (S) 39040 0.040173 0.000001 0.040173 0.000001 H5A_GET_NUM_ATTRS (S) 920 0.033910 0.000037 0.033910 0.000037 H5A_GET_SPACE (S) 5480 0.009080 0.000002 0.009080 0.000002 H5A_GET_TYPE (S) 5480 0.008593 0.000002 0.008593 0.000002 H5A_OPEN_IDX (S) 39040 8.980389 0.000230 8.980389 0.000230 H5A_OPEN_NAME (S) 5480 1.308467 0.000239 1.308467 0.000239 10 Files Individually ---------------------------------------------------------------------------------------------------------- H5A_CLOSE (S) 44520 0.044785 0.000001 0.044785 0.000001 H5A_GET_NAME (S) 39040 0.039924 0.000001 0.039924 0.000001 H5A_GET_NUM_ATTRS (S) 920 0.027584 0.000030 0.027584 0.000030 H5A_GET_SPACE (S) 5480 0.008716 0.000002 0.008716 0.000002 H5A_GET_TYPE (S) 5480 0.008387 0.000002 0.008387 0.000002 H5A_OPEN_IDX (S) 39040 4.474848 0.000115 4.474848 0.000115 H5A_OPEN_NAME (S) 5480 0.665722 0.000121 0.665722 0.000121

    Deleted User



    New Member


    Posts:
    New Member


    --
    18 Jul 2014 05:22 PM
    Below is a link to a program that illustrates the compounding slowness of H5A_OPEN_IDX and H5A_OPEN_NAME. I am so far unable to reproduce the same problems with H5A_GET_NUM_ATTRS and H5A_GET_NAME; however, I have encountered the same problem with those as well. https://dl.dropboxusercontent.com/u/42944960/test_h5a_v2.pro Here is a link to a data file http://www.rbsp-ect.lanl....D_20130223_v2.1.0.h5 At the bottom of the program, there is a main-level script. Simply change the name of the data file and IDL> .run test_h5a_v2
    You are not authorized to post a reply.