Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Low Level HDF5 Functions

Subject: Low Level HDF5 Functions

From: Jer

Date: 5 Aug, 2008 17:40:00

Message: 1 of 4

I have written a compiled script in 2007b to read and modify hdf5 data
using the low level functions. I am unable to use the basic HDF5
utilities due to some company specifc modifcations. The script runs
fairly slow and according to the profiler strmatch called in H5ML
functions is taking a lot of the processing time. Does anyone know a
way to speed this up by chance?
Thanks

Subject: Low Level HDF5 Functions

From: Maria Pavlou

Date: 2 Sep, 2008 11:10:20

Message: 2 of 4

Hi Jer,

Yes I found the same issue, masive slow downs due to the
way Matlab have implemented error handling and syntax
checking for HDF5 commands. What's actually happening in
the background is Matlab matches you hdf5 comand to a list
of posible/alowed functions and then procedes to process it
eventually calling an external mex function which actuallyl
does the fancy low-level HDF5 work.

What you need to do is call the mex functions directly.
However you should be confident as to the syntax you need
to use as you will get very little in the way of error help
and you may even harm your data.

I have written a basic class to handle hdf5 files in a
manner similar to memmapfile, where you can asign value to
a hdf5 class variable as if it were a basic matlab scalar
matrix.

Please email me if you want this implementation.

Maria

Jer <jcrobbinsster@gmail.com> wrote in message <faf91b13-
17e2-40cf-bafd-4607f97d29b7@l64g2000hse.googlegroups.com>...
> I have written a compiled script in 2007b to read and
modify hdf5 data
> using the low level functions. I am unable to use the
basic HDF5
> utilities due to some company specifc modifcations. The
script runs
> fairly slow and according to the profiler strmatch called
in H5ML
> functions is taking a lot of the processing time. Does
anyone know a
> way to speed this up by chance?
> Thanks

Subject: Low Level HDF5 Functions

From: Christopher

Date: 15 Jun, 2009 18:24:02

Message: 3 of 4

I have been experiancing the same problem with the slow down especially when pulling small amounts of data from many different places such as multipule attributes. So you say that you have written function to call the mex functions directly. I was unable to find the mex functions in my matlab root directory. Do you mean that you wrote mex functions to call the c - code possibly. I have been thinking of this as an alternative since the c code is aviable on the hdf5 website. I have been also looking at python as a possible place to proccess this data but the only interface to the c code files I have found is pytables and as far as I can tell it is too restrictive in the way it writes data. I would be interested in your functions if you could post them to the exchange

Subject: Low Level HDF5 Functions

From: meng long

Date: 25 Jun, 2009 08:44:02

Message: 4 of 4

"Christopher " <christopher.f.nyland@us.army.mil> wrote in message <h163k2$b6a$1@fred.mathworks.com>...
> I have been experiancing the same problem with the slow down especially when pulling small amounts of data from many different places such as multipule attributes. So you say that you have written function to call the mex functions directly. I was unable to find the mex functions in my matlab root directory. Do you mean that you wrote mex functions to call the c - code possibly. I have been thinking of this as an alternative since the c code is aviable on the hdf5 website. I have been also looking at python as a possible place to proccess this data but the only interface to the c code files I have found is pytables and as far as I can tell it is too restrictive in the way it writes data. I would be interested in your functions if you could post them to the exchange

The Mex functions are called, I guess, by hdf5lib2. My code, for example:

function [sObj,sAttr,oEx] = ReadTable(dLocId,strds,varargin)
% +PHdf.PH5ts/ReadTable 读取全部成员的全部记录。
%
% 使用说明:
% sObj = PHdf.PH5ts.ReadTable(dLocId,strds);
% [sObj,sAttr,oEx] = PHdf.PH5ts.ReadTable(dLocId,strds,clAttr);
% 输入参数:
% dLocId——文件或群组辨识符。
% strds——数据集(表格)的名称。
% clAttr——需要读取的属性名称,如果为空则不读取属性。
% 输出参数:
% sObj——读取数据的结构。
% sAttr——属性结构。
% oEx——执行成功返回[],否则返回MException实例。
%
% 例如:
%{
    clear classes;
    tic
    strf = [PDir.EtfGoRoot(),'EtfGo\Dbase\+PEtfH\fTest.H5'];
    oGath = PEtfH.HCodeTs(strf,'/Qut','/fCode');
    oGath.dLocId = H5F.open(oGath.strf,'H5F_ACC_RDONLY','H5P_DEFAULT');
% 读取数据。
    oCode = PEtfH.HCode(); oCode.oGath = oGath;
    ReadTable(oCode);
    %[num2cell(oCode.dCode),oCode.clChNm]
    H5F.close(oGath.dLocId);
    toc
%}

% 作者: 卢晨曦,2010/7/1
% Copyright 2004-2010,仟斯数码
% $修订版:4.0 $ $日期:2010/7/1 16:57:38 $


try
% 定位辨识符:
%{
    strf = H5F.get_name(dLocId)
    sInfo = hdf5info(strf,'ReadAttributes',true)
    sInfo.GroupHierarchy.Attributes.Value(2)
%}

% 1. 数据集

% sInfo = PHdf.PH5d.GetInfo(dDsetId)
    dDsetId = H5D.open(dLocId,strds,'H5P_DEFAULT');

% 2. 读取数据

% dMTId = H5ML.get_mem_datatype(dDsetId)
% sInfo = PHdf.PH5t.GetMemberInfo(dMTId)
    sObj = H5D.read(dDsetId,'H5ML_DEFAULT','H5S_ALL','H5S_ALL','H5P_DEFAULT');
%{
% 逐列读取已经通过测试。
dFTId = H5D.get_type(dDsetId);
dNoCols = H5T.get_nmembers(dFTId);
for dNoC = 1:dNoCols
    strFld = H5T.get_member_name(dFTId,dNoC-1);
    dBscId = H5ML.get_mem_datatype(H5T.get_member_type(dFTId,dNoC-1));
    dColTId = H5T.create('H5T_COMPOUND',H5T.get_size(dBscId));
    H5T.insert(dColTId,strFld,0,dBscId);
    sTmp = H5D.read(dDsetId,dColTId,'H5S_ALL','H5S_ALL','H5P_DEFAULT');
    sObj.(strFld) = sTmp.(strFld);
    H5T.close(dBscId);
    H5T.close(dColTId);
end
%}
% 3. 读取属性

    if(isempty(varargin) || isempty(varargin{1}))
        sAttr = [];
    else
        sAttr = PHdf.PH5ts.PH5a.ReadMsStruct(dDsetId,varargin{1});
    end

% 4. 关闭

    H5D.close(dDsetId);
    oEx = [];
    
catch oEx
    %PFig.ShowWarn(PMException.GetReport(oEx),'警告','modal');
    try H5D.close(dDsetId); catch, end
    sObj = [];
end

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us