[Carpet] component geometry for multi-core processor chips

[Steve White]

Erik,

Further thoughts on this.

First, I think the 
	ppn=n
syntax requests a certain number of processes on each node.

Second, it would be better not to ask the specific batch system (PBS) about 
which processes are running on which nodes.  I would prefer a means that
is independent of batch system.

How about:

	if this is not the MPI root process
		* send to MPI root process the result of
			system( "uname -n" )
	otherwise
		* make a hash of 
			node_name, mpi_rank
		* wait for message from each other process
		* for each message,
			add to the hash the the message body with mpi_rank
		* add to the hash the present node name with mpi_rank = 0

Now you have an easy association of nodes to MPI rank numbers.

Cheers!


On 22.08.06, Erik Schnetter wrote:
> On Aug 16, 2006, at 07:23:16, Steve White wrote:
> 
> >Erik & al.
> >
> >Does Carpet do anything regarding clumping together geometrically
> >processes that are running on the same node?
> >
> >If 8 processes are running per node, it makes a very big performance
> >difference if the 8 on each node are arranged in a cube, because much
> >less network communication need be done.
> >
> >We already have low cost clusters with dual-core dual-chip  
> >motherboards,
> >which would benefit, and the quad core chips are soon to come.
> >
> >Any ideas on this?
> 
> Carpet splits the domain onto the processors approximately in the  
> following way: First the domain is cut in the z direction, the  
> resulting slabs are cut in their y directions, and the resulting  
> bricks are cut in the x direction.  These regions are the numbered  
> according to that hierarchy.  This leads to a numbering scheme that  
> is very similar to how Fortran lays out a 3D array in memory.
> 
> In particular, there are no 2x2x2 blocks that would be arranged in a  
> cube.
> 
> A question comes to my mind: How can one find out from PBS and from  
> MPI which processors are "close" to each other?  You seem to assume  
> that the processors within a node are numbered sequentially.  That  
> does not seem to be so on Supermike, which enumerates first the first  
> processor on each node, then the second.  That seems to be a PBS  
> property, and I don't know whether MPI is clever enough to rearrange  
> the processors.
> 
> In order to experiment with this, I would first play with manual  
> processor decompositions and measure the speed differences.
> 
> -erik
> 
> -- 
> Erik Schnetter <schnetter at cct.lsu.edu>
> 
> My email is as private as my paper mail.  I therefore support encrypting
> and signing email messages.  Get my PGP key from www.keyserver.net.
> 
> 
> 



> _______________________________________________
> developers mailing list
> developers at lists.carpetcode.org
> http://lists.carpetcode.org/listinfo/developers


-- 
Steve White : Programmer
Max-Planck-Institut für Gravitationsphysik      Albert-Einstein-Institut
Am Mühlenberg 1, D-14476 Golm, Germany                  +49-331-567-7625