[Carpet] Carpet scheduling question

[Jonathan Thornburg]

Hi, Frank,

you asked [[about my "how to have A come before B when A is in local mode
and B is in global mode" query]]
> What prevents the scheduler / us to do that for all local routines, such 
> that the order is always as specified?

The basic problem is that there are two distinct pieces of software
here, the Cactus scheduler (which lives in the Cactus flesh, was designed
before Carpet even existed, and which Carpet can't change), and Carpet,
and neither of them knows everything.  In particular:
* the scheduler doesn't know anything about Carpet modes or refinement
  levels (only Carpet knows that)
* Carpet doesn't know which routines are scheduled before/after which
  other routines (only the flesh knows that)

This division of information is for historical reason -- the basic
design of Cactus -- including the scheduler -- predates Carpet, and
Carpet was designed to fit into that basic design and to be able to
work with the existing Cactus infrastructure (including thorns which
weren't written with mesh refinement in mind).

I suspect that if one were to write a Carpet-like system from scratch
(i.e. if one could design a new Cactus flesh as as a Berger-Oliger
mesh refinement driver), a cleaner/simpler design would be possible.

	[I should emphasize that I am *not* criticising Carpet's
	design here.  Rather, I'm hypothesizing an "alternate
	universe" in which Cactus didn't exist prior to Carpet,
	so the Cactus flesh and Carpet could be designed from
	scratch to work together as an integrated system, and
	in which there was no requirement that existing thorns
	continue to work when switching to Carpet.]

Thomas Radke and I have been working for several months on trying to
expand the Carpet documentation to better explain how Carpet scheduling
works.  We've made a lot of progress, and hope to something ready to
push to darcs later this week.  I plan to include the problem I posed
in my query to Erik as an example in this documentation, along with
a discussion of the various possible solutions.

ciao,

-- 
-- Jonathan Thornburg <jthorn at aei.mpg.de>      
   Max-Planck-Institut fuer Gravitationsphysik (Albert-Einstein-Institut),
   Golm, Germany, "Old Europe"     http://www.aei.mpg.de/~jthorn/home.html      
   "Washing one's hands of the conflict between the powerful and the
    powerless means to side with the powerful, not to be neutral."
                                      -- quote by Freire / poster by Oxfam