[Carpet] CCTK_{ORIGIN, DELTA}_SPACE() not set when entering local mode
Jonathan Thornburg
jthorn at aei.mpg.de
Tue May 23 18:53:14 CEST 2006
Hi, Erik,
I've run into what looks like a bug in Carpet: CCTK_ORIGIN_SPACE() and
CCTK_DELTA_SPACE() are sometimes not set (properly) when entering local
mode in a multipatch simulation.
In more detail:
I'm using a darcs pull of Carpet from 12.May.2006 (I haven't seen any
darcs messages since then that look relevant to this), and current-cvs
flesh and other thorns from a couple of days ago. (Notably, I'm using
CactusBase/CartGrid3D/src/CartGrid3D.c version 1.55, dated
Mon May 15 20:02:10 2006.)
I'm running on a single processor (my laptop with MPICH).
I've set up a 6-patch system. I've scheduled a routine in level mode
(in BASEGRID) which uses
Carpet::enter_singlemap_mode(GN, map_number)
...
Carpet::leave_singlemap_mode()
to enter singlemap mode for each map in turn. For each map, I then do a
BEGIN_LOCAL_COMPONENT_LOOP(GH, CCTK_GF)
{
...
}
END_LOCAL_COMPONENT_LOOP
to walk all components for that map. For each component, I verify that
I'm in local mode, then print CCTK_ORIGIN_SPACE() and CCTK_DELTA_SPACE().
The problem is that while CTK_ORIGIN_SPACE() and CCTK_DELTA_SPACE() are
fine for map #0, they're garbage for all the other maps.
Attached is a tar.gz of my toy test thorn to demonstrate this, a par file,
and stdout+stderr from running this on my laptop.
Am I doing something silly in changing modes this way?
Or is this a multipatch bug in Carpet?
thanks, ciao,
--
-- Jonathan Thornburg <jthorn at aei.mpg.de>
Max-Planck-Institut fuer Gravitationsphysik (Albert-Einstein-Institut),
Golm, Germany, "Old Europe" http://www.aei.mpg.de/~jthorn/home.html
"Washing one's hands of the conflict between the powerful and the
powerless means to side with the powerful, not to be neutral."
-- quote by Freire / poster by Oxfam
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-carpet-grid-info.tar.gz
Type: application/octet-stream
Size: 4500 bytes
Desc:
Url : /archives/developers/attachments/20060523/76cc5135/attachment.obj
-------------- next part --------------
# this parameter sets up a 6-patch Carpet grid
# and tests the grid info for it
cactus::highlight_warning_messages = false
# flesh
cactus::cctk_itlast = 0
########################################
ActiveThorns = "IOUtil IOBasic"
IOUtil::parfile_write = "no"
IO::out_dir = $parfile
########################################
#
# ***** multipatch system *****
#
# 6 patches, full-sphere, no symmetries
ActiveThorns = "Carpet CarpetLib CarpetReduce"
Carpet::num_maps = 6 # number of patches
Carpet::domain_from_coordbase = true
# specify the continuum (= nominal-grid) size/shape of each patch
# (i.e. this specification does *not* include ghost zones)
# ... n.b. we label the coordinate axes (x,y,z) = (rho,sigma,wr)
ActiveThorns = "SymBase"
ActiveThorns = "CoordBase"
CoordBase::domainsize = "minmax" # specify continuum patch size
CoordBase::xmin = -0.785398163397448 # -45 degrees
CoordBase::xmax = +0.785398163397448 # +45 degrees
CoordBase::ymin = -0.785398163397448 # -45 degrees
CoordBase::ymax = +0.785398163397448 # +45 degrees
CoordBase::zmin = 0.12 # inner wr (rmin = 1.37)
CoordBase::zmax = 1.12 # outer wr (rmax = 27.5)
# grid spacing
CoordBase::spacing = "numcells"
CoordBase::ncells_x = 21 # delta_rho = 4.286 degrees
CoordBase::ncells_y = 21 # delta_sigma = 4.286 degrees
CoordBase::ncells_z = 25 # delta_wr = 0.04
#
# ghost zones
#
driver::ghost_size_x = 2
driver::ghost_size_y = 2
driver::ghost_size_z = 3 # for radial upwind FD
# ghost zones are outside the nominal grid
CoordBase::boundary_internal_x_lower = false
CoordBase::boundary_internal_x_upper = false
CoordBase::boundary_internal_y_lower = false
CoordBase::boundary_internal_y_upper = false
CoordBase::boundary_internal_z_lower = false
CoordBase::boundary_internal_z_upper = false
# ghost zone widths
CoordBase::boundary_size_x_lower = 2
CoordBase::boundary_size_x_upper = 2
CoordBase::boundary_size_y_lower = 2
CoordBase::boundary_size_y_upper = 2
CoordBase::boundary_size_z_lower = 2
CoordBase::boundary_size_z_upper = 3
# ghost zones start 1 point outside nominal grid boundary
CoordBase::boundary_shiftout_x_lower = 1
CoordBase::boundary_shiftout_x_upper = 1
CoordBase::boundary_shiftout_y_lower = 1
CoordBase::boundary_shiftout_y_upper = 1
CoordBase::boundary_shiftout_z_lower = 1
CoordBase::boundary_shiftout_z_upper = 1
ActiveThorns = "CartGrid3D"
CartGrid3D::type = "CoordBase"
ActiveThorns = "TestCarpetGridInfo"
TestCarpetGridInfo::N_maps = 6
-------------- next part --------------
--------------------------------------------------------------------------------
10
1 0101 ************************
01 1010 10 The Cactus Code V4.0
1010 1101 011 www.cactuscode.org
1001 100101 ************************
00010101
100011 (c) Copyright The Authors
0100 GNU Licensed. No Warranty
0101
--------------------------------------------------------------------------------
Cactus version: 4.0.b16
Compile date: May 23 2006 (18:36:59)
Run date: May 23 2006 (18:40:19)
Run host: carbon.aei.mpg.de
Working directory: /home/jonathan/cactus/Cactus/arrangements/ThornburgCVS/TestCarpetGridInfo/par
Executable: cactus_test-carpet-grid-info
Parameter file: 6patch.par
--------------------------------------------------------------------------------
Activating thorn Cactus...Success -> active implementation Cactus
Activation requested for
--->IOUtil IOBasic<---
Activating thorn IOBasic...Success -> active implementation IOBasic
Activating thorn IOUtil...Success -> active implementation IO
Activation requested for
--->Carpet CarpetLib CarpetReduce<---
Activating thorn Carpet...Success -> active implementation Driver
Activating thorn CarpetLib...Success -> active implementation CarpetLib
Activating thorn CarpetReduce...Success -> active implementation reduce
Activation requested for
--->SymBase<---
Activating thorn SymBase...Success -> active implementation SymBase
Activation requested for
--->CoordBase<---
Activating thorn CoordBase...Success -> active implementation CoordBase
Activation requested for
--->CartGrid3D<---
Activating thorn CartGrid3D...Success -> active implementation grid
Activation requested for
--->TestCarpetGridInfo<---
Activating thorn TestCarpetGridInfo...Success -> active implementation TestCarpetGridInfo
--------------------------------------------------------------------------------
if (recover initial data)
Recover parameters
endif
Startup routines
[CCTK_STARTUP]
Carpet: Multi-model Startup routine
Carpet: Startup routine
CarpetReduce: Startup routine
CartGrid3D: Register GH Extension for GridSymmetry
CoordBase: Register a GH extension to store the coordinate system handles
IOUtil: Startup routine
IOBasic: Startup routine
SymBase: Register GH Extension for SymBase
Startup routines which need an existing GH
[CCTK_WRAGH]
CartGrid3D: Register coordinates for the Cartesian grid
GROUP SymBase_Wrapper: Wrapper group for SymBase
GROUP SymmetryRegister: Register your symmetries here
CartGrid3D: Register symmetry boundaries
SymBase: Print symmetry boundary face descriptions
Parameter checking routines
[CCTK_PARAMCHECK]
CartGrid3D: Check coordinates for CartGrid3D
Initialisation
[CCTK_BASEGRID]
GROUP MaskBase_SetupMask: Set up the weight function
CarpetReduce: Initialise the weight function
GROUP SetupMask: Set up the weight function (schedule other routines in here)
CarpetReduce: Set up the outer boundaries of the weight function
CarpetReduce: Set up the weight function for the restriction regions
CartGrid3D: Set up ranges for spatial 3D Cartesian coordinates
CartGrid3D: Set up spatial 3D Cartesian coordinates on the GH
SymBase: Check whether the driver set up the grid consistently
TestCarpetGridInfo: test grid info
if (NOT (recover initial data AND recovery_mode is 'strict'))
[CCTK_INITIAL]
IOBasic: Initialisation routine
[CCTK_POSTINITIAL]
[CCTK_POSTSTEP]
endif
if (recover initial data)
[CCTK_RECOVER_VARIABLES]
[CCTK_POST_RECOVER_VARIABLES]
endif
if (checkpoint initial data)
[CCTK_CPINITIAL]
endif
if (analysis)
[CCTK_ANALYSIS]
CarpetLib: Print timing statistics if desired
CarpetLib: Print memory statistics if desired
endif
Do periodic output of grid variables
do loop over timesteps
Rotate timelevels
iteration = iteration + 1
t = t+dt
[CCTK_PRESTEP]
[CCTK_EVOL]
[CCTK_POSTSTEP]
if (checkpoint)
[CCTK_CHECKPOINT]
endif
if (analysis)
[CCTK_ANALYSIS]
CarpetLib: Print timing statistics if desired
CarpetLib: Print memory statistics if desired
endif
Do periodic output of grid variables
enddo
Termination routines
[CCTK_TERMINATE]
Shutdown routines
[CCTK_SHUTDOWN]
Routines run after restricting:
[CCTK_POSTRESTRICT]
Routines run after changing the grid hierarchy:
[CCTK_POSTREGRID]
GROUP MaskBase_SetupMask: Set up the weight function
CarpetReduce: Initialise the weight function
GROUP SetupMask: Set up the weight function (schedule other routines in here)
CarpetReduce: Set up the outer boundaries of the weight function
CarpetReduce: Set up the weight function for the restriction regions
CartGrid3D: Set Coordinates after regridding
--------------------------------------------------------------------------------
INFO (Carpet): Multi-Model listing:
model 0: "world"
INFO (Carpet): Multi-Model processor distribution:
processor 0: model 0 "world"
INFO (Carpet): Multi-Model: This is processor 0, model 0 "world"
--------------------------------------------------------------------------------
AMR driver provided by Carpet
--------------------------------------------------------------------------------
INFO (Carpet): Carpet is running on 1 processors
INFO (Carpet): CoordBase boundary specification for map 0:
nboundaryzones: [[2,2],[2,2],[2,3]]
is_internal : [[0,0],[0,0],[0,0]]
is_staggered : [[0,0],[0,0],[0,0]]
shiftout : [[1,1],[1,1],[1,1]]
INFO (Carpet): CoordBase domain specification for map 0:
physical extent: [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
interior extent: [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
exterior extent: [-0.934998,-0.934998,0.04] : [0.934998,0.934998,1.24] ([1.87,1.87,1.2])
base_spacing : [0.0747998,0.0747998,0.04]
INFO (Carpet): Adapted domain specification for map 0:
convergence factor: 2
convergence level : 0
physical extent : [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
interior extent : [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
exterior extent : [-0.934998,-0.934998,0.04] : [0.934998,0.934998,1.24] ([1.87,1.87,1.2])
spacing : [0.0747998,0.0747998,0.04]
INFO (Carpet): Base grid specification for map 0:
number of grid points : [26,26,31]
number of ghost points: [2,2,3]
INFO (Carpet): Grid structure (grid points):
[0][0][0][0] exterior extent: [0,0,0] : [25,25,30] ([26,26,31])
INFO (Carpet): CoordBase boundary specification for map 1:
nboundaryzones: [[2,2],[2,2],[2,3]]
is_internal : [[0,0],[0,0],[0,0]]
is_staggered : [[0,0],[0,0],[0,0]]
shiftout : [[1,1],[1,1],[1,1]]
INFO (Carpet): CoordBase domain specification for map 1:
physical extent: [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
interior extent: [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
exterior extent: [-0.934998,-0.934998,0.04] : [0.934998,0.934998,1.24] ([1.87,1.87,1.2])
base_spacing : [0.0747998,0.0747998,0.04]
INFO (Carpet): Adapted domain specification for map 1:
convergence factor: 2
convergence level : 0
physical extent : [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
interior extent : [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
exterior extent : [-0.934998,-0.934998,0.04] : [0.934998,0.934998,1.24] ([1.87,1.87,1.2])
spacing : [0.0747998,0.0747998,0.04]
INFO (Carpet): Base grid specification for map 1:
number of grid points : [26,26,31]
number of ghost points: [2,2,3]
INFO (Carpet): Grid structure (grid points):
[0][0][1][0] exterior extent: [0,0,0] : [25,25,30] ([26,26,31])
INFO (Carpet): CoordBase boundary specification for map 2:
nboundaryzones: [[2,2],[2,2],[2,3]]
is_internal : [[0,0],[0,0],[0,0]]
is_staggered : [[0,0],[0,0],[0,0]]
shiftout : [[1,1],[1,1],[1,1]]
INFO (Carpet): CoordBase domain specification for map 2:
physical extent: [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
interior extent: [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
exterior extent: [-0.934998,-0.934998,0.04] : [0.934998,0.934998,1.24] ([1.87,1.87,1.2])
base_spacing : [0.0747998,0.0747998,0.04]
INFO (Carpet): Adapted domain specification for map 2:
convergence factor: 2
convergence level : 0
physical extent : [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
interior extent : [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
exterior extent : [-0.934998,-0.934998,0.04] : [0.934998,0.934998,1.24] ([1.87,1.87,1.2])
spacing : [0.0747998,0.0747998,0.04]
INFO (Carpet): Base grid specification for map 2:
number of grid points : [26,26,31]
number of ghost points: [2,2,3]
INFO (Carpet): Grid structure (grid points):
[0][0][2][0] exterior extent: [0,0,0] : [25,25,30] ([26,26,31])
INFO (Carpet): CoordBase boundary specification for map 3:
nboundaryzones: [[2,2],[2,2],[2,3]]
is_internal : [[0,0],[0,0],[0,0]]
is_staggered : [[0,0],[0,0],[0,0]]
shiftout : [[1,1],[1,1],[1,1]]
INFO (Carpet): CoordBase domain specification for map 3:
physical extent: [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
interior extent: [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
exterior extent: [-0.934998,-0.934998,0.04] : [0.934998,0.934998,1.24] ([1.87,1.87,1.2])
base_spacing : [0.0747998,0.0747998,0.04]
INFO (Carpet): Adapted domain specification for map 3:
convergence factor: 2
convergence level : 0
physical extent : [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
interior extent : [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
exterior extent : [-0.934998,-0.934998,0.04] : [0.934998,0.934998,1.24] ([1.87,1.87,1.2])
spacing : [0.0747998,0.0747998,0.04]
INFO (Carpet): Base grid specification for map 3:
number of grid points : [26,26,31]
number of ghost points: [2,2,3]
INFO (Carpet): Grid structure (grid points):
[0][0][3][0] exterior extent: [0,0,0] : [25,25,30] ([26,26,31])
INFO (Carpet): CoordBase boundary specification for map 4:
nboundaryzones: [[2,2],[2,2],[2,3]]
is_internal : [[0,0],[0,0],[0,0]]
is_staggered : [[0,0],[0,0],[0,0]]
shiftout : [[1,1],[1,1],[1,1]]
INFO (Carpet): CoordBase domain specification for map 4:
physical extent: [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
interior extent: [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
exterior extent: [-0.934998,-0.934998,0.04] : [0.934998,0.934998,1.24] ([1.87,1.87,1.2])
base_spacing : [0.0747998,0.0747998,0.04]
INFO (Carpet): Adapted domain specification for map 4:
convergence factor: 2
convergence level : 0
physical extent : [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
interior extent : [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
exterior extent : [-0.934998,-0.934998,0.04] : [0.934998,0.934998,1.24] ([1.87,1.87,1.2])
spacing : [0.0747998,0.0747998,0.04]
INFO (Carpet): Base grid specification for map 4:
number of grid points : [26,26,31]
number of ghost points: [2,2,3]
INFO (Carpet): Grid structure (grid points):
[0][0][4][0] exterior extent: [0,0,0] : [25,25,30] ([26,26,31])
INFO (Carpet): CoordBase boundary specification for map 5:
nboundaryzones: [[2,2],[2,2],[2,3]]
is_internal : [[0,0],[0,0],[0,0]]
is_staggered : [[0,0],[0,0],[0,0]]
shiftout : [[1,1],[1,1],[1,1]]
INFO (Carpet): CoordBase domain specification for map 5:
physical extent: [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
interior extent: [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
exterior extent: [-0.934998,-0.934998,0.04] : [0.934998,0.934998,1.24] ([1.87,1.87,1.2])
base_spacing : [0.0747998,0.0747998,0.04]
INFO (Carpet): Adapted domain specification for map 5:
convergence factor: 2
convergence level : 0
physical extent : [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
interior extent : [-0.785398,-0.785398,0.12] : [0.785398,0.785398,1.12] ([1.5708,1.5708,1])
exterior extent : [-0.934998,-0.934998,0.04] : [0.934998,0.934998,1.24] ([1.87,1.87,1.2])
spacing : [0.0747998,0.0747998,0.04]
INFO (Carpet): Base grid specification for map 5:
number of grid points : [26,26,31]
number of ghost points: [2,2,3]
INFO (Carpet): Grid structure (grid points):
[0][0][5][0] exterior extent: [0,0,0] : [25,25,30] ([26,26,31])
INFO (Carpet): Group and variable statistics:
INFO (Carpet): There are 5 grid functions in 2 groups
INFO (Carpet): There are 5 grid scalars in 3 groups
INFO (Carpet): There are 0 1-dimensional grid arrays in 0 groups
INFO (Carpet): There are 0 2-dimensional grid arrays in 0 groups
INFO (Carpet): There are 0 3-dimensional grid arrays in 0 groups
INFO (Carpet): (The number of variables counts all time levels)
INFO (IOBasic): I/O Method 'Scalar' registered: output of scalar quantities (grid scalars, reductions) to ASCII files
INFO (IOBasic): I/O Method 'Info' registered: output of scalar quantities (grid scalars, reductions) to screen
INFO (CartGrid3D): Grid Spacings:
INFO (CartGrid3D): dx=>7.4799825e-02 dy=>7.4799825e-02 dz=>4.0000000e-02
INFO (CartGrid3D): Computational Coordinates:
INFO (CartGrid3D): x=>[-0.935, 0.935] y=>[-0.935, 0.935] z=>[ 0.040, 1.240]
INFO (CartGrid3D): Indices of Physical Coordinates:
INFO (CartGrid3D): x=>[0,25] y=>[0,25] z=>[0,30]
map_number=0 component_number=0
CCTK_ORIGIN_SPACE()=(-0.934998,-0.934998,0.04)
CCTK_DELTA_SPACE()=(0.0747998,0.0747998,0.04)
map_number=1 component_number=0
CCTK_ORIGIN_SPACE()=(2.6196e-314,1.53813e-173,0)
CCTK_DELTA_SPACE()=(4.97601e-315,1.53813e-173,0)
map_number=2 component_number=0
CCTK_ORIGIN_SPACE()=(4.74159e-314,1.53813e-173,0)
CCTK_DELTA_SPACE()=(4.97601e-315,1.53813e-173,0)
map_number=3 component_number=0
CCTK_ORIGIN_SPACE()=(6.86359e-314,1.53813e-173,0)
CCTK_DELTA_SPACE()=(4.97601e-315,1.53813e-173,0)
map_number=4 component_number=0
CCTK_ORIGIN_SPACE()=(8.98558e-314,1.53813e-173,0)
CCTK_DELTA_SPACE()=(4.97601e-315,1.53813e-173,0)
map_number=5 component_number=0
CCTK_ORIGIN_SPACE()=(1.11076e-313,1.53813e-173,0)
CCTK_DELTA_SPACE()=(4.97601e-315,1.53813e-173,0)
--------------------------------------------------------------------------------
Done.
More information about the developers
mailing list