Thread Subject: Low Level HDF5 Functions

I have written a compiled script in 2007b to read and modify hdf5 data
using the low level functions. I am unable to use the basic HDF5
utilities due to some company specifc modifcations. The script runs
fairly slow and according to the profiler strmatch called in H5ML
functions is taking a lot of the processing time. Does anyone know a
way to speed this up by chance?
Thanks

Hi Jer,

Yes I found the same issue, masive slow downs due to the
way Matlab have implemented error handling and syntax
checking for HDF5 commands. What's actually happening in
the background is Matlab matches you hdf5 comand to a list
of posible/alowed functions and then procedes to process it
eventually calling an external mex function which actuallyl
does the fancy low-level HDF5 work.

What you need to do is call the mex functions directly.
However you should be confident as to the syntax you need
to use as you will get very little in the way of error help
and you may even harm your data.

I have written a basic class to handle hdf5 files in a
manner similar to memmapfile, where you can asign value to
a hdf5 class variable as if it were a basic matlab scalar
matrix.

Please email me if you want this implementation.

Maria

Jer <jcrobbinsster@gmail.com> wrote in message <faf91b13-
17e2-40cf-bafd-4607f97d29b7@l64g2000hse.googlegroups.com>...
> I have written a compiled script in 2007b to read and
modify hdf5 data
> using the low level functions. I am unable to use the
basic HDF5
> utilities due to some company specifc modifcations. The
script runs
> fairly slow and according to the profiler strmatch called
in H5ML
> functions is taking a lot of the processing time. Does
anyone know a
> way to speed this up by chance?
> Thanks

I have been experiancing the same problem with the slow down especially when pulling small amounts of data from many different places such as multipule attributes. So you say that you have written function to call the mex functions directly. I was unable to find the mex functions in my matlab root directory. Do you mean that you wrote mex functions to call the c - code possibly. I have been thinking of this as an alternative since the c code is aviable on the hdf5 website. I have been also looking at python as a possible place to proccess this data but the only interface to the c code files I have found is pytables and as far as I can tell it is too restrictive in the way it writes data. I would be interested in your functions if you could post them to the exchange

"Christopher " <christopher.f.nyland@us.army.mil> wrote in message <h163k2$b6a$1@fred.mathworks.com>...
> I have been experiancing the same problem with the slow down especially when pulling small amounts of data from many different places such as multipule attributes. So you say that you have written function to call the mex functions directly. I was unable to find the mex functions in my matlab root directory. Do you mean that you wrote mex functions to call the c - code possibly. I have been thinking of this as an alternative since the c code is aviable on the hdf5 website. I have been also looking at python as a possible place to proccess this data but the only interface to the c code files I have found is pytables and as far as I can tell it is too restrictive in the way it writes data. I would be interested in your functions if you could post them to the exchange

The Mex functions are called, I guess, by hdf5lib2. My code, for example:

function [sObj,sAttr,oEx] = ReadTable(dLocId,strds,varargin)
% +PHdf.PH5ts/ReadTable 读取全部成员的全部记录。
%
% 使用说明：
% sObj = PHdf.PH5ts.ReadTable(dLocId,strds);
% [sObj,sAttr,oEx] = PHdf.PH5ts.ReadTable(dLocId,strds,clAttr);
% 输入参数：
% dLocId——文件或群组辨识符。
% strds——数据集(表格)的名称。
% clAttr——需要读取的属性名称，如果为空则不读取属性。
% 输出参数：
% sObj——读取数据的结构。
% sAttr——属性结构。
% oEx——执行成功返回[]，否则返回MException实例。
%
% 例如：
%{
    clear classes;
    tic
    strf = [PDir.EtfGoRoot(),'EtfGo\Dbase\+PEtfH\fTest.H5'];
    oGath = PEtfH.HCodeTs(strf,'/Qut','/fCode');
    oGath.dLocId = H5F.open(oGath.strf,'H5F_ACC_RDONLY','H5P_DEFAULT');
% 读取数据。
    oCode = PEtfH.HCode(); oCode.oGath = oGath;
    ReadTable(oCode);
    %[num2cell(oCode.dCode),oCode.clChNm]
    H5F.close(oGath.dLocId);
    toc
%}

% 作者： 卢晨曦，2010/7/1
% Copyright 2004-2010，仟斯数码
% $修订版：4.0 $ $日期：2010/7/1 16:57:38 $


try
% 定位辨识符：
%{
    strf = H5F.get_name(dLocId)
    sInfo = hdf5info(strf,'ReadAttributes',true)
    sInfo.GroupHierarchy.Attributes.Value(2)
%}

% 1. 数据集

% sInfo = PHdf.PH5d.GetInfo(dDsetId)
    dDsetId = H5D.open(dLocId,strds,'H5P_DEFAULT');

% 2. 读取数据

% dMTId = H5ML.get_mem_datatype(dDsetId)
% sInfo = PHdf.PH5t.GetMemberInfo(dMTId)
    sObj = H5D.read(dDsetId,'H5ML_DEFAULT','H5S_ALL','H5S_ALL','H5P_DEFAULT');
%{
% 逐列读取已经通过测试。
dFTId = H5D.get_type(dDsetId);
dNoCols = H5T.get_nmembers(dFTId);
for dNoC = 1:dNoCols
    strFld = H5T.get_member_name(dFTId,dNoC-1);
    dBscId = H5ML.get_mem_datatype(H5T.get_member_type(dFTId,dNoC-1));
    dColTId = H5T.create('H5T_COMPOUND',H5T.get_size(dBscId));
    H5T.insert(dColTId,strFld,0,dBscId);
    sTmp = H5D.read(dDsetId,dColTId,'H5S_ALL','H5S_ALL','H5P_DEFAULT');
    sObj.(strFld) = sTmp.(strFld);
    H5T.close(dBscId);
    H5T.close(dColTId);
end
%}
% 3. 读取属性

    if(isempty(varargin) || isempty(varargin{1}))
        sAttr = [];
    else
        sAttr = PHdf.PH5ts.PH5a.ReadMsStruct(dDsetId,varargin{1});
    end

% 4. 关闭

    H5D.close(dDsetId);
    oEx = [];
    
catch oEx
    %PFig.ShowWarn(PMException.GetReport(oEx),'警告','modal');
    try H5D.close(dDsetId); catch, end
    sObj = [];
end