[Carpet] [Benzstrasse] hdf5 compression

[Thomas Radke]

Luca Baiotti wrote:
> Hi,
> 
> I may bring this up again at the group meeting, so here are some data I 
> gathered with a whisky+BSSN run with PUGH and different compression 
> levels for hdf5 output:
> 
> du -s TestTime_*/rho_3d
> 
> 263M    TestTime_1Compression/rho_3d
> 259M    TestTime_2Compression/rho_3d
> 254M    TestTime_3Compression/rho_3d
> 249M    TestTime_4Compression/rho_3d
> 247M    TestTime_5Compression/rho_3d
> 245M    TestTime_6Compression/rho_3d
> 244M    TestTime_7Compression/rho_3d
> 242M    TestTime_8Compression/rho_3d
> 242M    TestTime_9Compression/rho_3d
> 607M    TestTime_noCompression/rho_3d
> 
> 
> 
> du -s TestTime_*/gxx_3d
> 
> 379M    TestTime_1Compression/gxx_3d
> 375M    TestTime_2Compression/gxx_3d
> 375M    TestTime_3Compression/gxx_3d
> 374M    TestTime_4Compression/gxx_3d
> 370M    TestTime_5Compression/gxx_3d
> 365M    TestTime_6Compression/gxx_3d
> 363M    TestTime_7Compression/gxx_3d
> 363M    TestTime_8Compression/gxx_3d
> 363M    TestTime_9Compression/gxx_3d
> 607M    TestTime_noCompression/gxx_3d
> 
> 
> 
> Contrary to what I had anticipated, the total time of the simulations 
> are similar in all cases. This tests were made under real-life disc 
> usage, and this is seen in the non monotonicity of the curve
> simulation-time vs. compression-level. The variance due to file-system 
> load seems to be larger than the overhead of compression.
> 
> 
> time  compression level
> 
> 35    TestTime_1Compression
> 43    TestTime_2Compression
> 30    TestTime_3Compression
> 35    TestTime_4Compression
> 32    TestTime_5Compression
> 35    TestTime_6Compression
> 35    TestTime_7Compression
> 37    TestTime_8Compression
> 39    TestTime_9Compression
> 
> 44    TestTime_noCompression   *
> 35    TestTime_noCompression   *
> 
> * two different runs of the same parfile & executable
> 
> 
> 
> I would then suggest to use Compression1 routinely (once it is 
> implemented in carpet).

Hi Luca,

thanks a lot for testing automatic HDF5 dataset compression !

I just pushed the corresponding patch to also support this feature in 
Carpet. However, I kept the default setting to disable compression (as 
in CactusPUGHIO/IOHDF5); users have to set
IOHDF5::compression_level to a value between 1 and 9 explicitely to 
enable gzip compression.

-- 
Cheers, Thomas.