[Carpet] cost of repeated calls on CCTK_SyncGroup()

[Erik Schnetter]

On Jun 13, 2006, at 09:54:44, Jonathan Thornburg wrote:

> Hi, Erik,
>
> Some algorithms Bela and I are working on typically need to make
> on the order of 10 sweeps-over-the-grid per time step, each of which
> requires calling CCTK_SyncGroup() on 20 or more grid functions.
> Together, all these synchronizations are taking a *lot* of time
> (over half the total run time in some of Bela's tests).
>
> My basic question is, how could this be optimized?
>
> The flesh provides a  CCTK_SyncGroupsI()  function to synchronize a
> whole set of groups, but this is implemented as a sequential loop over
> the groups, calling  CCTK_SyncGroup()  on each one.
>
> Looking in the Carpet source code, I see a  SyncProlongateGroups()
> function in  Carpet/Carpet/src/Comm.cc , but it also prolongates
> the boundaries, which I don't want here.
>
> Question:
> Is there some deep reason this function *must* prolongate as well as
> synchronize?  Looking at the code, it looks easy enough to turn off
> the prolongation where needed.  That is, suppose I were to add a
> Boolean parameter 'prolongate' to this function, defaulting to true
> for backwards compatability, and have the function only prolongate
> if this parameter is true.
>
> I could then call this function from my thorn
> 	[either as an aliased fn or as an overload of
> 	 CCTK_SyncGroupsI() ]
> to (hopefully) more efficiently synchronize my entire set of grid
> functions all at once.
>
> Would this be a reasonable thing to do?

Consider also alternatives, such as using hyperslabbing to move data  
about, or copying a region of data (if you only work on a small  
region of a set of grid functions) into a temporary array, maybe a 3D  
grid function.

Carpet offers aliased functions EnableProlongating and  
QueryProlongating to enable and disable prolongation.  You could use  
these.  Make sure to restore the previous state afterwards, because  
MoL also uses these functions when there are buffer zones.

If you need a certain functionality, we can also easily add a new  
aliased function that you can call.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : /archives/developers/attachments/20060613/4b57e753/attachment.pgp