[Carpet] problems applying RotatingSymmetry90 when both CCTK_INT and CCTK_REAL gridfns are present

[Jonathan Thornburg]

Hi, Erik,

I have a wierd error appearing which looks like it may be a problem
with RotatingSymmetry90.
I have marked a single CCTK_INT gridfn, and 40 or so CCTK_REAL gridfns,
for the "none" BC (I did this all in level mode), and then scheduled
ApplyBCs.  The error message

WARNING level 0 in thorn Slab processor 0 host carbon.aei.mpg.de
  (line 841 of /home/jonathan/cactus/Cactus/configs/test-moving-excision/build/Slab/slab.c): 
  -> internal error
WARNING level 0 in thorn Slab processor 0 host carbon.aei.mpg.de
  (line 841 of /home/jonathan/cactus/Cactus/configs/test-moving-excision/build/Slab/slab.c): 
  -> internal error
[0] MPI Abort by user Aborting program !
[0] Aborting program!

and stack traceback from gdb

#0  0x0a32bdfa in exit () from /usr/lib/libc.so.38.2
#1  0x1cb0e89f in MPID_SHMEM_Abort (comm_ptr=0x82b29e80, code=0, msg=0x0)
    at shmeminit.c:118
#2  0x1cb0be2a in MPID_Abort (comm_ptr=0x82b29e80, code=0, 
    user=0x3c0b7062 "MPI Abort by user", str=0x0) at adi2init.c:201
#3  0x1caf29b0 in PMPI_Abort (comm=91, errorcode=0) at abort.c:54
#4  0x1c9678ef in Carpet::Abort(_cGH*, int) (cgh=0x0, retval=0)
    at /home/jonathan/cactus/Cactus/configs/test-moving-excision/build/Carpet/helpers.cc:99
#5  0x1c00dd94 in CCTK_VWarn (level=0, line=841, 
    file=0x3c0a1a80 "/home/jonathan/cactus/Cactus/configs/test-moving-excision/build/Slab/slab.c", thorn=0x3c0a1a6c "Slab", format=0x3c002c9b "%s")
    at /home/jonathan/cactus/Cactus/configs/test-moving-excision/build/Cactus/main/WarnLevel.c:604
#6  0x1c00d916 in CCTK_Warn (level=0, line=841, 
    file=0x3c0a1a80 "/home/jonathan/cactus/Cactus/configs/test-moving-excision/build/Slab/slab.c", thorn=0x3c0a1a6c "Slab", 
    message=0x3c0a1a5d "internal error")
    at /home/jonathan/cactus/Cactus/configs/test-moving-excision/build/Cactus/main/WarnLevel.c:395
#7  0x1ca641af in Slab_MultiTransfer (cctkGH=0x7e77ec00, dim=3, 
    xferinfo=0x86fd5e00, options=-1, nvars=65, srctypes=0x823cf400, 
    srcptrs=0x7da7ce00, dsttypes=0x823cf400, dstptrs=0x823cfa00)
    at /home/jonathan/cactus/Cactus/configs/test-moving-excision/build/Slab/slab.c:841
#8  0x1c8b38ee in BndRot90VI (cctkGH=0x7e77ec00, nvars=65, vis=0x81551e00)
    at /home/jonathan/cactus/Cactus/configs/test-moving-excision/build/RotatingSymmetry90/rotatingsymmetry90.c:493
#9  0x1c8b427c in Rot90_ApplyBC (cctkGH=0x7e77ec00)
    at /home/jonathan/cactus/Cactus/configs/test-moving-excision/build/RotatingSymmetry90/rotatingsymmetry90.c:599
#10 0x1c00a5e8 in CCTK_CallFunction (function=0x1c8b3ef0, fdata=0x7f55b50c, 
    data=0x7e77ec00)
    at /home/jonathan/cactus/Cactus/configs/test-moving-excision/build/Cactus/main/ScheduleInterface.c:258

suggest that the problem is spotted when the code in Slab tries to
verify that all the gridfns being operated on have the same data types.

Does this ring any obvious bells to you?  If not, tomorrow I will try to
reproduce the problem in a test thorn to make it easily debuggable....

ciao,

-- 
-- Jonathan Thornburg <jthorn at aei.mpg.de>      
   Max-Planck-Institut fuer Gravitationsphysik (Albert-Einstein-Institut),
   Golm, Germany, "Old Europe"     http://www.aei.mpg.de/~jthorn/home.html      
   "Washing one's hands of the conflict between the powerful and the
    powerless means to side with the powerful, not to be neutral."
                                      -- quote by Freire / poster by Oxfam