[Carpet] Too many HDF5 files?

[Thomas Radke]

Erik Schnetter wrote:
> Thomas,
> 
> I find that my output directories contain many HDF5 files.  There are  
> many more HDF5 than ASCII files because each processor writes its own  
> HDF5 file.  I assume that the performance win of this approach is  worth 
> it, although I haven't tested it.

The feature is there, just set IO::out_mode = "onefile" and you will get 
exactly that per variable for a parallel run. You can even set the mode 
for individual variables, I think.
I haven't recently tested the performance of this I/O mode either but 
I'd guess that processor 0 would still turn out to become an I/O 
serialisation bottleneck for any reasonably parallel run.

One idea I had was that each variable could be output by a different 
processor, then at least the writing to disk could happen in parallel.

> Should we implement another  method 
> to reduce the number of files?  We could e.g. write one output  file per 
> group, per thorn, or even per iteration.  Do you have  another suggestion?

One file per group shouldn't be difficult to implement, one just needs 
to copy the logic which is already there in IOScalar or IOASCII, right ?
One file per thorn sounds a little strange to me. One per iteration 
basically amount to writing a checkpoint.

> If we combine different variables into the same file, is there  anything 
> special that needs to be added?  I think it should be  possible to just 
> write several variables into the same file, adding  the meta-information 
> (grid structure etc.) only once.  Is that correct?

Yes, I think so. I'll take a closer look on the one_file_per_group 
implementation of IOScalar.

Would that be enough in order to reduce the total number of HDF5 output 
files ?

-- 
Cheers, Thomas.