[Carpet] Too many HDF5 files?

[Thomas Radke]

Christian David Ott wrote:
>>>Yes, I think so. I'll take a closer look on the one_file_per_group
>>>implementation of IOScalar.
>>
>>If it is not too difficult, then I can also try that myself.

I think I have a patch already implementing that.

> Note that a number of our analysis tools (for example: Amira) may 
> potentially (or will probably) choke on multiple variables in a Carpet 
> hdf5 output file. So grouping multiple variables into a single output file 
> might not be the best way to do things.

DV would not have a problem. The ImportCarpetHDF5 import module for 
OpenDX would need to be extended by a new input tab where you can select 
the variables to be imported by name. That's a small thing to do. Amira 
probably needs a similar modification. Any other tools which we need to 
take care of ?

>>the basic problem is that there is one file per processor -- 
>>on 1000 processors the output directory will just overflow.  Maybe the 
>>correct approach would be to have different output subdirectories on 
>>different processors?  This should also improve performance a bit, since 
>>changes to the subdirectories are then interesting only for a single 
>>processor, whereas with a global output directory each processor has to be 
>>informed about newly created files, resulting in lock contention.  (If I 
>>understood Maciek correctly.)
> 
> 
> I agree with this. The number of cpus (which is increasing giving better 
> scaling of future/experimental Carpet versions) is going to be the major 
> issue here and having the option of using a separate output directory for
> each cpu sounds good.

CactusPUGHIO/IOHDF5 implements this feature: if the out_dir parameter 
contains a '%u' template it will be substituted by the processor ID. Is 
this what we also want for CarpetIOHDF5 ? The same logic would then also 
need to be added to all the visualisation and postprocessing utility 
tools, currently they expect all chunked files in the same directory.

-- 
Cheers, Thomas.