write fails for custom structs of length >3

[FG-TUM]

Bug Report

Description

I was looking at example-02d-stdvector-struct.cpp.

Suppose we want to have even more than 3 fields, so I extended the struct Double3 by a fourth value a and called the resulting type Double4:

struct Double4 {
    double                    x, y, z, a;
    [[nodiscard]] std::string string() const { return h5pp::format("x: {} y: {} z: {} a: {}", x, y, z, a); }
};

If I now create a data vector and then call writeThenRead() with it, the writing of the data fails (error message see bottom)

    std::vector<Double4> coord4d = {{1.0, 2.0, 3.0, 4.}, {4.0, 5.0, 6.0, 7.}, {7.0, 8.0, 9.0, 10.}};
    writeThenRead(file, coord4d, "CoordinatesInFourDimensions");

Seems weird that this fails as soon as the struct is larger than 4 elements (happens for double or float and even for three when the struct is annotated with __attribute__((aligned(32))).)

I'm happy to provide further info if needed.

Error Message

HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140392769433792:
  #000: ../../../src/H5D.c line 147 in H5Dcreate2(): unable to create dataset
    major: Dataset
    minor: Unable to initialize object
  #001: ../../../src/H5Dint.c line 494 in H5D__create_named(): unable to create and link to dataset
    major: Dataset
    minor: Unable to initialize object
  #002: ../../../src/H5L.c line 1697 in H5L_link_object(): unable to create new link to object
    major: Links
    minor: Unable to initialize object
  #003: ../../../src/H5L.c line 1941 in H5L_create_real(): can't insert link
    major: Symbol table
    minor: Unable to insert object
  #004: ../../../src/H5Gtraverse.c line 869 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #005: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #006: ../../../src/H5L.c line 1744 in H5L_link_cb(): unable to create object
    major: Object header
    minor: Unable to initialize object
  #007: ../../../src/H5O.c line 3178 in H5O_obj_create(): unable to open object
    major: Object header
    minor: Can't open object
  #008: ../../../src/H5Doh.c line 297 in H5O__dset_create(): unable to create dataset
    major: Dataset
    minor: Unable to initialize object
  #009: ../../../src/H5Dint.c line 1231 in H5D__create(): datatype is not sensible
    major: Invalid arguments to routine
    minor: Inappropriate type
terminate called after throwing an instance of 'std::runtime_error'
  what():  h5pp: Failed to create dataset  | size 3 | bytes 96 | rank 1 | dims {3} | layout H5D_COMPACT | resize mode OFF | dset path [CoordinatesInFourDimensions] | c++ type [H5T_COMPOUND] | c++ size [32] bytes
[1]    28882 abort (core dumped)  examples/h5pp-example-02d-stdvector-struct

Additional Info

• h5pp Version: 1.9.0
• libhdf5-dev/bionic 1.10.0-patch1+docs-4

[DavidAce]

Thank you for a clear bug report!

The error you are getting is due to h5pp only having support for structs with 2 or 3 elements "out of the box", called Scalar2 and Scalar3 (template classes defining that define HDF5 types). These are defined here.

The list of "ScalarN" types could in principle be expanded to N=4,5..., but as they are initialized at the start of the first call to an h5pp file, I'd like to keep them to a minimum.
Perhaps a better approach would be to detect that the user is trying to do this and give a hint in the error message.

In any case, the approach you should look at for now is the one in example 04a. There, you have to manually register the struct layout with HDF5 before passing it to h5pp (these are essentially the steps I automated for Scalar2 and Scalar3).

[FG-TUM]

Thanks for the reply. I see that expanding these manually built types does not scale, however, I think that it should be documented more explicitly that the whole mechanism shown in the example is built upon these internally defined types and does not work with anything else out of the box. Currently, the example says:

h5pp supports writing and reading simple POD structs such as coordinate structs

which sounds like the shown technique works for any POD struct.

Also thanks for pointing me to example 04a, which showed exactly what I needed :)

[DavidAce]

Yes that example is confusing, thanks for pointing that out! I'll edit the examples affected.