X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 28 Jul 2010 12:43 PM by  anon
Data Structure Concatenation
 2 Replies
Sort:
You are not authorized to post a reply.
Author Messages

anon



New Member


Posts:
New Member


--
28 Jul 2010 12:43 PM
    Hello all. Sorry if this is going to be long. I know having to debug other peoples scripts, I prefer a good concise explanation to figure things out! And I added a copy of some of the code that is giving me errors. It's just a lot of fluff you need not worry about. I have been using IDL for a few years now, but my new job has forced me to start taking my more casual use of this language to a new degree. As such, I have found myself a bit baffled with how I can concatenate structures in IDL. Or more specifically, concatenate in a certain way to maintain the standards of a code that has been previously written (and perhaps without the proper foresight to prevent these issues.) Essentially, I have two pre-existing data structures. (Included below, named complete_set and data_set) and I need to place data_set into complete_set. Normally, this isn't an issue. create_struct will concatenate these two just fine. However, I am now working with a file that is well over a gigabyte in size. data_set stores the needed information for the some 2 million indices. On older computers, some other users in my group can't directly concatenate this structure into the other, as the resultant strain on the memory is a bit too large. I would delete other variables and try the use of temporary, but that's not really possible here. I'm not in the main function, and data_set is needed for other portions of the code before the step it dies at. My idea was to essentially pop off an element of the data_set structure at a time, and place that into complete_set. This way when I redefine complete_set after each iteration, it takes less memory and I can then increase the memory being taken from swap for complete_set. Only I can't seem to do this. When using create_struct, I am stuck naming things (by virtue of not wanting to change a few year old huge suite of scripts): complete_set = create_struct(complete_set,"data_set",data_set) Replacing the full structure data_set with data_set[0] works fine. But when I try to run the command: complete_set=create_struct(complete_set,"data_set",data_set[1]) I get issues because the sub structure data_set already exists. I would have thought it would concatenate an additional entry into that sub structure. So my general question is, how might I start adding new elements to complete_set directly, one at a time. Thanks for your help! Brian. ~~~~~~~~~~~~~~~~~ IDL> print,complete_set { 1} IDL> help,complete_set,/str ** Structure , 1 tags, length=2, data length=2, refs=2: FLAG INT 1 I'm only copying the first element of data_set. Each entry is large, and all 2 million entries are essentially identical. IDL> print,data_set[0] {{ 0 0 0 }{ 0 0 0 0 }{ 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 }{ 0 0 }{ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 }{ 0 0 0 0 0 0 0 0 0 } 0 0 } IDL> help,data_set,/str ** Structure , 48 tags, length=459, data length=459, refs=3: COLUMN1 STRUCT -> Array[1] COLUMN2 STRUCT -> Array[1] COLUMN3 STRUCT -> Array[1] COLUMN4 STRUCT -> Array[1] COLUMN5 STRUCT -> Array[1] COLUMN6 STRUCT -> Array[1] COLUMN7 STRUCT -> Array[1] COLUMN8 STRUCT -> Array[1] COLUMN9 STRUCT -> Array[1] COLUMN10 STRUCT -> Array[1] COLUMN11 STRUCT -> Array[1] COLUMN12 STRUCT -> Array[1] COLUMN13 STRUCT -> Array[1] COLUMN14 STRUCT -> Array[1] COLUMN15 STRUCT -> Array[1] COLUMN16 STRUCT -> Array[1] COLUMN17 STRUCT -> Array[1] COLUMN18 STRUCT -> Array[1] COLUMN19 STRUCT -> Array[1] COLUMN20 STRUCT -> Array[1] COLUMN21 STRUCT -> Array[1] COLUMN22 STRUCT -> Array[1] COLUMN23 STRUCT -> Array[1] COLUMN24 STRUCT -> Array[1] COLUMN25 STRUCT -> Array[1] COLUMN26 STRUCT -> Array[1] COLUMN27 STRUCT -> Array[1] COLUMN28 STRUCT -> Array[1] COLUMN29 STRUCT -> Array[1] COLUMN30 STRUCT -> Array[1] COLUMN31 STRUCT -> Array[1] COLUMN32 STRUCT -> Array[1] COLUMN33 STRUCT -> Array[1] COLUMN34 STRUCT -> Array[1] COLUMN35 STRUCT -> Array[1] COLUMN36 STRUCT -> Array[1] COLUMN37 STRUCT -> Array[1] COLUMN38 STRUCT -> Array[1] COLUMN39 STRUCT -> Array[1] COLUMN40 STRUCT -> Array[1] COLUMN41 STRUCT -> Array[1] COLUMN42 STRUCT -> Array[1] COLUMN43 STRUCT -> Array[1] COLUMN44 STRUCT -> Array[1] COLUMN45 STRUCT -> Array[1] COLUMN46 STRUCT -> Array[1] COLUMN47 STRUCT -> Array[1] CRLF BYTE Array[2]

    Deleted User



    New Member


    Posts:
    New Member


    --
    04 Aug 2010 03:11 PM
    I may not have understood the issue correctly, but why not simply replace the value of dataset in the new structure, rather than trying to add it again as a new element of the structure> i.e. instead of calling complete_set=create_struct(complete_set,"data_set",data_set[1]) on the next (and subsequent) iterations, do this: complete_set.data_set = dataset[ 1 ] This should be allowable since all of these structures are anonymous. r moss

    Deleted User



    New Member


    Posts:1
    New Member


    --
    25 Aug 2010 08:57 AM
    Within a structure? That's not an option unfortunately. You would need to use the new v8 LIST data type or use a pointer to an object or heap variable in the structure that you re-alllocate as needed when the size changes. Non-structure array variables can be re-assigned to arrays of any size, so you can change their size effectively whenever you need. -mh
    You are not authorized to post a reply.