Path: news.mathworks.com!not-for-mail
From: "Ashish Uthama" <first.last@mathworks.com>
Newsgroups: comp.soft-sys.matlab
Subject: Re: reading HDF5 dataset subset
Date: Tue, 03 Nov 2009 15:10:23 -0500
Organization: TMW
Lines: 102
Message-ID: <op.u2t4zlc7a5ziv5@uthamaa.dhcp.mathworks.com>
References: <hcnllp$lv2$1@fred.mathworks.com>
 <op.u2tjkwbva5ziv5@uthamaa.dhcp.mathworks.com>
 <hcpv87$6gq$1@fred.mathworks.com>
NNTP-Posting-Host: uthamaa.dhcp.mathworks.com
Mime-Version: 1.0
Content-Type: text/plain; charset=iso-8859-15; format=flowed; delsp=yes
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1257279023 15090 172.31.57.126 (3 Nov 2009 20:10:23 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Tue, 3 Nov 2009 20:10:23 +0000 (UTC)
User-Agent: Opera Mail/10.01 (Win32)
Xref: news.mathworks.com comp.soft-sys.matlab:582153


On Tue, 03 Nov 2009 14:11:03 -0500, Tess  <tess.brandon@noaa.gov> wrote:

> Thanks; I've seen these examples but haven't been able to get them to  
> apply to my problem successfully.  Is 'hyperslab' just a fancy word for  
> subset or chunk of the data?  Does it matter at all how the data set is  
> chunked?
>
> I ran the following code and received the following error:
> fileID=H5F.open(file);
> dset=H5D.open(fileID,'/WeeklySST'); % this is a 53x512x512 data set  
> which I
> % read successfully as a whole using H5D.read
> space=H5D.get_space(dset);
> start=[0 0 0];
> stride=[50 50 50] % making it symmetrical b/c I'm still figuring out how  
> HDF5 flips
> % dimensions
> count=[]; block=[]; % "If count and block are specified as [], the count  
> and block
> % size default to a single element."
> H5S.select(space,'H5S_SELECT_SET',start,stride,count,block);
>
> ??? Error using ==> hdf5lib2
> The HDF5 library encountered an error:  "unable to set hyperslab  
> selection"
>
> Error in ==> H5S.select_hyperslab at 35
> H5ML.hdf5lib2('H5Sselect_hyperslab', spaceID, op, start, stride, count,  
> block);
>
> I am running 7.8.0 R2009a.
>
> Thanks again,
> Tess

1.
As I understand it, chunking is defined on the internals of the file, i.e  
how the data is stored/organized internally in the file. This appears to  
be something required to enable 'extendible' data sets.
http://www.zib.de/zibdoc/zib/mpp/prog/hdf5/Tutor/extend.html

A hyperslab is what you appear to think of as a 'chunk of data'. It is a  
subset of the data. I dont think the chunking of the file matters when you  
are reading a hyperslab.

I am sure there is more info somewhere in the HDF documentation.

2. Using [] results in the arguments defaulting to a single element. Thats  
not what you want if you want to select a single element (look at code  
below)

3. You need to use H5S.select_hyperslab

4. You need to specify a memory space ID which matches the size of the  
data selected (see code)



Heres something which might help you understand better and get started  
with your own code:



%hdf5write('o.h5','/DS1',reshape(1:53*512*512,[53 512 512])); %used to  
create sample data, each element ought to be unique


fileID=H5F.open('o.h5','H5F_ACC_RDONLY','H5P_DEFAULT');
dset=H5D.open(fileID,'/DS1'); % this is a 53x512x512 data set
space=H5D.get_space(dset);


start=[7 18 26]; %zero based!
stride=[1 1 1];
count=[1 1 1]; block=[ ];


H5S.select_hyperslab(space,'H5S_SELECT_SET',fliplr(start-1),...   %-1 to  
take care of 0 based indexing
     fliplr(stride),fliplr(count),fliplr(block));                  %fliplr  
to take care of row/column major reversal (please see the ReadMe.txt in my  
previous post)
mem_space = H5S.create_simple(3, fliplr(count), []);  %see Readme.txt for  
explanation on need for flipping


rdata = H5D.read(dset,'H5T_NATIVE_INT',mem_space,space,'H5P_DEFAULT');

H5D.close(dset);
H5S.close(space);
H5S.close(mem_space);
H5F.close(fileID);

disp(rdata)


%Cross check with the expected element:
r=hdf5read('o.h5','/DS1');
r(7,18,26)