X
PrevPrev Go to previous topic
NextNext Go to next topic
Last Post 10 Aug 2022 01:44 PM by  Ben Castellani
Need Optiminization help!
 3 Replies
Sort:
You are not authorized to post a reply.
Author Messages

Bill Biagiotti



New Member


Posts:1
New Member


--
05 Aug 2022 02:18 PM
    I've been working at this dilemma for quite a while and thought I would ask my fellow IDL experts. I am processing an image by taking sequential 3x3 2D subsets from it and calculating the following: 1) Subtracting the center point from the surrounding elements, 2) Calculating the absolute value of (1), and then summing all of those deltas. The problem is I can't figure out how to efficiently do this with IDL without loops. Here is some code:

    n = statboxsize - 2
    temp = INTARR(statboxsize, statboxsize)

    val = indgen(47, 47) ; the subset taken from the master image

    for y = 1, n DO $
    for x = 1,n DO $
    temp[x, y] = TOTAL(ABS(val[x-1:x+1, y-1: y+1] - val[x, y]))

    In a medium size image (800 x 600) , the number of TOTALs and ABSs are in the millions and kill the execution time. Does anyone know an IDL array-based method to make this fast? Thank you in advance!

    Ben Castellani



    Basic Member


    Posts:130
    Basic Member


    --
    08 Aug 2022 04:05 PM
    The SHIFT function would be helpful here: https://www.l3harrisgeospatial.com/docs/shift.html

    Here is an example below. Using SHIFT is about 15-20 times faster than your double loop in my testing.

    ;IDL PRO CODE
    dim = 4000 ;number of X/Y points in 2D array
    val = randomn(seed,dim,dim) ;generate random 2D float array
    sz = size(val)

    temp = fltarr(sz[1],sz[2])

    ;Existing method
    tic
    for y = 1, sz[2]-2 DO $
    for x = 1,sz[1]-2 DO $
    temp[x, y] = TOTAL(ABS(val[x-1:x+1, y-1: y+1] - val[x, y]))
    toc

    ;Another method using SHIFT instead
    tic
    temp2 = abs(val-shift(val,[1,0]))+abs(val-shift(val,[1,-1]))+abs(val-shift(val,[0,-1]))+abs(val-shift(val,[-1,-1]))+abs(val-shift(val,[-1,0]))+abs(val-shift(val,[-1,1]))+ $
    abs(val-shift(val,[0,1]))+abs(val-shift(val,[1,1]))
    toc

    ----

    IDL Prints:

    % Time elapsed: 9.2260001 seconds.
    % Time elapsed: 0.51099992 seconds.

    There might be some further room for improvement using

    Bill Biagiotti



    New Member


    Posts:1
    New Member


    --
    09 Aug 2022 11:56 AM
    Ben,

    That's exactly what I was looking for, thank you! I had to make a minor adjustment to ignore the wraparound when I sum the results, but that's ok. In my project, it gave me a 4x speed improvement and I'm looking to see where else I can use this. In your post, you left with saying: "There might be some further room for improvement using....." Was there something else to add?

    Thanks again!!

    Ben Castellani



    Basic Member


    Posts:130
    Basic Member


    --
    10 Aug 2022 01:44 PM
    Ah, it appears I forgot to finish my last sentence there! One more thing to consider is the IDL Threadpool. ABS and TOTAL calculations can both be shared across CPU threads in IDL. You might be able to get further improvement by tweaking the CPU settings if multi-threading is not already activating when running that line of code in either my method or your original method: https://www.l3harrisgeospatial.com/docs/controlling_the_idl_thre.html

    In my testing, any image larger than ~300x300 will trigger the threadpool under IDL's default setting of 100,000 calculations per operation. So this probably is already helping you.

    I also tried a few other routes to improve the overall speed (for example, WHERE can often replace loops in IDL and it is very fast!), but was not able to beat the SHIFT method. This is one iteration with a single loop instead of two, but this is twice as slow as your original method:

    temp3 = fltarr(sz[1],sz[2])
    for idx=1+dim,n_elements(val)-2-dim do begin
    temp3[idx] = total(abs([val[idx-1:idx+1],val[idx+dim-1:idx+dim+1],val[idx-dim-1:idx-dim+1]]-val[idx]))
    endfor


    Good luck!
    You are not authorized to post a reply.