[Carpet] Regridding & Recomposing

[Bela Szilagyi]

> > From what I see, the return value of the Regrid function is set to
> > "true" when
> > requesting a decrease in the number of refinement levels.
>
> There are two functions, Regrid and Recompose.  Regrid is a global
> function which determines the new grid structure.  It does not
> actually change the grid structure, it only decides how the new grid
> structure should look like.  This function returns true if the grid
> structure should be changed, not if the number of levels should
> decrease.  

Here is what I see in CarpetRegrid:

1) CarpetRegrid_Regrid() returns the value of "do_recompose" (assuming it does 
not quit earlier with a zero return in cases when, e.g., parameter settings 
imply no regridding).

2) This variable "do_recompose" is set, e.g., by the return value of 
ManualGridpointList()

3) ManualGridpointList() returns zero in case the  number of refinement levels 
has not changed and tracking is not active.  Otherwise this function returns 
one.

That is, if I have no tracking, and the only change I request is deactivation 
of the finest refinement level, then ManualGridpointList() returns one, 
which, in turn is the return value of CarpetRegrid_Regrid().

This in term implies a "true" return value for Carpet::Regrid(), which is
picked up by the variable "did_regrid" in Carpet::CallRegrid()

> This return value is not actually needed -- Carpet could 
> instead compare the old and the new grid structures, but the return
> values simplifies life.
>
> The function Recompose is called for a certain refinement level.  It
> actually changes this refinement level according to a pre-determined
> new grid structure.  The Recompose function should be called from
> coarsest to finest level, so that newly created fine regions can be
> filled by prolongation from coarser levels.  (The proper nesting
> property guarantees that this is always possible.)  Recompose returns
> whether this level was actually changed.  Thus the postregrid bin can
> be skipped if the grid did not actually change, which is an important
> optimisation.
>
> >   Also, the
> > Recompose() routine appears to return "true" only if the grid
> > definition on a
> > particular level has changed.
>
> Correct.

So, at this point, we are saying that in Carpet::CallRegrid() the following 
hold:

1) did_regrid is set to true if, e.g., I request nothing but a reflevel 
deactivation

2) did_recompose is set to true iff changes were made within a refinement 
level.

My complaint is  that, as far as I see, 

CTK_ScheduleTraverse("CCTK_POSTREGRID", cctkGH, CallFunction);

is only called if
1) both did_regrid and did_recompose is true
2) we are a refinement level where the grid changed (i.e., did_recompose was 
set to true)


>
> After writing my explanation above I understand what you mean with
> "decrease".  You want to deactivate a level.  In this case, the
> regridding routine should return a grid structure with fewer levels
> in it, in addition to returning "true".  Deactivating a grid should
> certainly be treated as any other change to the grid hierarchy, and
> the postregrid bin should be called.

I believe I argued now that the current code does not implement this.

> What mechanism are you using to deactivate levels?  It could be that
> this mechanism doesn't work properly.  Can you check the grid
> structure that is set up by the regridding routine?

I am using an aliased function that would give the requested number of 
refinement levels based on some criteria such as the AH radius, etc.  This 
number would only decrease in my cases.

> The postregrid bin should always be called on all levels.

It seems that right now the code does not implement this.  One easy way to 
achieve this, though, would be to declare "did_recompose" outside the loop 
over all refinement levels (in Carpet::CallRegrid()) and make it equal 
the "OR" composition of the return value from Recompose().  This way you'd 
still optimize away calls to the regrid bin on coarse levels where the grid 
did not change, while calling the same bin on all levels that have either 
changed or are above a level that has changed.

As I think through the issue, it may well be that one only wants to call the 
regrid bin, in case of a refinement level deactivation, on the new finest 
grid only.  Just to "convey" the evolution algorithm that the finest grid is 
now different.  This is not in the spirit of Berger Oliger algorithms, where 
codes should never care about the particular refinement level, but that would 
be a world  without optimization.

Please let me know if I am mis-interpreting the code.

Bests,

-- 
Bela Szilagyi

Max-Planck-Institut für Gravitationsphysik
Albert-Einstein-Institut
Tel: +49 331 567 7632
Fax: +49 331 567 7649