Thread Subject: HDF5 multidimensional order: Inconsistent with h5utils?

Hello,
I have a minor problem, or rather question regarding the order of hdf5 datasets dimensions.

It seems that the dimensions as shown by size(A) in Matlab is reversed compared to the output by "h5ls foo.h5" (h5ls from the hdf5 utilities http://www.hdfgroup.org/HDF5/doc/RM/Tools.html#Tools-Ls ).

See the following example:

In Matlab:
a = zeros(1, 2, 3, 4);
disp(size(a)) % 1 2 3 4
hdf5write('test_order.h5', '/a', a);

On the command line:
$ h5ls test_order.h5
a Dataset {4, 3, 2, 1}

To me this looks as if the order was reversed when saving, the same behaviour can be observed when loading a DataSet via hdf5read.

The documentation talks about an option called 'V71Dimensions', which according to the documentation affects only the first two dimensions and resolves this behaviour only partially:

Matlab:
a = zeros(1, 2, 3, 4);
disp(size(a)) % 1 2 3 4
hdf5write('test_order.h5', '/a', a, 'V71Dimensions', true);
info_order = hdf5info('test_order.h5');
b=hdf5read(info_order.GroupHierarchy.Datasets, 'V71Dimensions', true);
disp(size(b)) % 3 4 2

Shell:
$ h5ls input/test_order.h5
a Dataset {1, 2, 3, 4}

So saving works here (at least the order is correct, and not just the first two dimensions were swapped), but loading the data in Matlab again fails, because the (wrong) order "4 3 2 1" is changed (still wrong) into "3 4 2 1" (the last, singleton dimension is silently removed).

Now my questions: How can I read and save multidimensional hdf5 arrays in the correct order (assuming that the hdf5 libraries and utilities show the correct order), or is Matlab right here? The workaround I currently use is to simply reshape the resulting matrix, which results in correct data.

Note: This is with Matlab 2010.b (64 bit), on Mac OS X.

Thanks for your insights,
Daniel

Hi Daniel, the dimensions look reversed because MATLAB uses column-major order while h5ls is a C application and therefore uses row-major order for arrays. The in-memory ordering of the data is the same for both, though.


"Daniel" wrote in message <io73d1$kvl$1@fred.mathworks.com>...
> Hello,
> I have a minor problem, or rather question regarding the order of hdf5 datasets dimensions.
>
> It seems that the dimensions as shown by size(A) in Matlab is reversed compared to the output by "h5ls foo.h5" (h5ls from the hdf5 utilities http://www.hdfgroup.org/HDF5/doc/RM/Tools.html#Tools-Ls ).
>
> See the following example:
>
> In Matlab:
> a = zeros(1, 2, 3, 4);
> disp(size(a)) % 1 2 3 4
> hdf5write('test_order.h5', '/a', a);
>
> On the command line:
> $ h5ls test_order.h5
> a Dataset {4, 3, 2, 1}
>
> To me this looks as if the order was reversed when saving, the same behaviour can be observed when loading a DataSet via hdf5read.
>
> The documentation talks about an option called 'V71Dimensions', which according to the documentation affects only the first two dimensions and resolves this behaviour only partially:
>
> Matlab:
> a = zeros(1, 2, 3, 4);
> disp(size(a)) % 1 2 3 4
> hdf5write('test_order.h5', '/a', a, 'V71Dimensions', true);
> info_order = hdf5info('test_order.h5');
> b=hdf5read(info_order.GroupHierarchy.Datasets, 'V71Dimensions', true);
> disp(size(b)) % 3 4 2
>
> Shell:
> $ h5ls input/test_order.h5
> a Dataset {1, 2, 3, 4}
>
> So saving works here (at least the order is correct, and not just the first two dimensions were swapped), but loading the data in Matlab again fails, because the (wrong) order "4 3 2 1" is changed (still wrong) into "3 4 2 1" (the last, singleton dimension is silently removed).
>
> Now my questions: How can I read and save multidimensional hdf5 arrays in the correct order (assuming that the hdf5 libraries and utilities show the correct order), or is Matlab right here? The workaround I currently use is to simply reshape the resulting matrix, which results in correct data.
>
> Note: This is with Matlab 2010.b (64 bit), on Mac OS X.
>
> Thanks for your insights,
> Daniel

"John " <com.works.math@evans.john> wrote in message <io76mg$stc$1@fred.mathworks.com>...
> Hi Daniel, the dimensions look reversed because MATLAB uses column-major order while h5ls is a C application and therefore uses row-major order for arrays. The in-memory ordering of the data is the same for both, though.

Hi John,
thanks for the fast reply. You are right of course, also reading http://www.hdfgroup.org/ftp/HDF5/examples/examples-by-api/matlab/HDF5_M_Examples/README.txt would have helped me already, I think.

Thanks,
Daniel