[Carpet] when does Carpet call scheduled routines?

[Jonathan Thornburg]

Hi, Erik,

Thanks for your prompt and informative responce to my questions.
I have a couple of further followup-questions:


You wrote:
> The basic algorithm in Carpet is the 
> Berger-Oliger time evolution scheme, which recursively evolves the 
> coarse levels before taking multiple steps on the finer levels. 
> There is a figure in CQG 21 1465 (gr-qc 0310042) which explains this 
> graphically.

Ok, I understand this part.


> There is also a latex document Carpet/Carpet/doc/ 
> schedule.tex which details which schedule bins are executed in what 
> order in that recursive scheme.  Note the difference between initial 
> data and the time evolution loop.

There is also a latex document  doc/internals.tex  which gives what
seems to be a somewhat different description.  For example:
* Section 3.2.1 "Initialization" gives BASEGRID (algorithm step 5)
   *inside* the loop-over-refinement-levels, whereas  Carpet/doc/schedule.tex
   shows BASEGRID *outside* the loop.
* Section 3.2.2 "Evolution" says (algorithm steps 6,7,8) that PRESTEP,
   EVOL, and POSTSTEP for a given level are executed in sequence with no
   other levels intervening, whereas  Carpet/doc/schedule.tex  says
   the recursion to other levels happens between EVOL and
   POSTRESTRICT-and-POSTSTEP.

>From its "May 3, 2003" date, and its lack of mention of anything other
than local and global modes, I suspect that  doc/internals.tex  is out
of date and/or incomplete in its description of scheduling.  Is this true?


This basic algorithm is exactly what happens to routines scheduled in 
level mode.  Routines in singlemap mode and local mode are wrapped in 
additional loops, in the same way as you assumed.  That is,

> loop over all models
>     walk over all levels using the Berger-Oliger scheme
>        foreach scheduled routine S encountered as we walk the  schedule tree
>           select case (S.schedule_options)
>           case (LEVEL)
>              call S on model M, level L, in LEVEL mode
>           case (SINGLEMAP)
>              foreach patch P in model M, level L
>                 call S on model M, level L, patch P, in SINGLEMAP mode
>           case (LOCAL)
>              foreach patch P in model M, level L
>                 foreach component C in model M, level L, patch P
>                    call S on model M, level L, patch P, component C
>                           in LOCAL mode

Ok, this makes perfect sense too.


> Global mode and meta mode work differently.  Routines scheduled in 
> global mode and meta mode should be called less often than the 
> recursive scheme dictates.  There are global "flags" which indicate 
> e.g. whether "this is the first level which is encountered on this 
> iteration", aka "do-global-mode".  Global mode and meta mode routines 
> are executed only when their corresponding flags are true.  Thus the
> complete algorithm is
> 
> do-meta-mode = true
> loop over all models
>     do-global-mode = true
>     walk over all levels using the Berger-Oliger scheme
>        foreach scheduled routine S encountered as we walk the  schedule tree
>           select case (S.schedule_options)
>           case (META)
>              if (do-meta-mode)
>                 call S in META mode
>           case (GLOBAL)
>              if (do-global-mode)
>                 call S on model M in GLOBAL mode
>           case (LEVEL)
>              call S on model M, level L, in LEVEL mode
>           case (SINGLEMAP)
>              foreach patch P in model M, level L
>                 call S on model M, level L, patch P, in SINGLEMAP mode
>           case (LOCAL)
>              foreach patch P in model M, level L
>                 foreach component C in model M, level L, patch P
>                    call S on model M, level L, patch P, component C 
>                           in LOCAL mode
>        do-global-mode = false
>     do-meta-mode = false

Ahh, now I see what's happening.

>From looking at  enter_global_mode()  in  Carpet/src/modes.cc  I
suspect that "models" in the above algorithm correspond to differing
values of mglevel, and that that's the same thing you meant by
"convergence levels" in
> Also, there are no "models" in Carpet, these are "convergence 
> levels", in case you want to run the same simulation with different 
> resolutions at the same time.
Right?


How does Carpet handle "multi-models" in the sense of

> INFO (Carpet): Multi-Model listing:
>    model 0: "world"
> INFO (Carpet): Multi-Model processor distribution:
>    processor 0: model 0 "world"
> INFO (Carpet): Multi-Model: This is processor 0, model 0 "world"

?  That is, how are the executions of the different "multi-model models"
interleaved?


thanks, ciao,

-- 
-- Jonathan Thornburg <jthorn at aei.mpg.de>
    Max-Planck-Institut fuer Gravitationsphysik (Albert-Einstein-Institut),
    Golm, Germany, "Old Europe"     http://www.aei.mpg.de/~jthorn/home.html
    "Washing one's hands of the conflict between the powerful and the
     powerless means to side with the powerful, not to be neutral."
                                       -- quote by Freire / poster by Oxfam