[Carpet] component geometry for multi-core processor chips

[Erik Schnetter]

On Aug 16, 2006, at 07:23:16, Steve White wrote:

> Erik & al.
>
> Does Carpet do anything regarding clumping together geometrically
> processes that are running on the same node?
>
> If 8 processes are running per node, it makes a very big performance
> difference if the 8 on each node are arranged in a cube, because much
> less network communication need be done.
>
> We already have low cost clusters with dual-core dual-chip  
> motherboards,
> which would benefit, and the quad core chips are soon to come.
>
> Any ideas on this?

Carpet splits the domain onto the processors approximately in the  
following way: First the domain is cut in the z direction, the  
resulting slabs are cut in their y directions, and the resulting  
bricks are cut in the x direction.  These regions are the numbered  
according to that hierarchy.  This leads to a numbering scheme that  
is very similar to how Fortran lays out a 3D array in memory.

In particular, there are no 2x2x2 blocks that would be arranged in a  
cube.

A question comes to my mind: How can one find out from PBS and from  
MPI which processors are "close" to each other?  You seem to assume  
that the processors within a node are numbered sequentially.  That  
does not seem to be so on Supermike, which enumerates first the first  
processor on each node, then the second.  That seems to be a PBS  
property, and I don't know whether MPI is clever enough to rearrange  
the processors.

In order to experiment with this, I would first play with manual  
processor decompositions and measure the speed differences.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : /archives/developers/attachments/20060822/76a797a7/attachment.pgp