File Exchange

image thumbnail

MDF Import Tool and Function

version 1.7.0.1 (38.2 KB) by

App and function to import signal data from MDF (Measure Data Format) files

143 Downloads

Updated

View License

This tool and function imports data from v3.0 MDF (.dat or .mdf) files to the MATLAB® workspace or MAT-file. Each signal comes in as a separate variable. Associated time vectors are also generated.
Calling 'mdfimport' without parameters brings up a interactive GUI. Calling 'mdfimport' with parameters reads the signals in directly.
Instructions: Unzip attached files and put on your path or in the current directory. Type mdfimport to run. Access help from the tools Help menu of the GUI for more information or by directly reading the two HTML files included.
Tested in releases from R13 to R2006A. Tested more with MDF files generated by ETAS INCA. Does work with some files VECTOR CANtech CANape.

Comments and Ratings (82)

it works fine! thank you very much!

Tim Spaeder

@ Romano
since I ran into the same problem here is a quick fix:
replace
test= ~isempty(varargin{2}) && isgraphics(varargin{2});
with
 test= ~isempty(varargin{2}) && max(isgraphics(varargin{2}));

The error is caused because you called the mdfimport via command line and supplied a specific .mat file (a string). The function isgraphics will test if this string is a graphics handle, which is of course not the case. The returned data type is a logic array and thus can not be combined with a single logic value. The max function should circumvent this. It definitely worked for me.

Regards,
Tim

Stuart McGarrity

Hi Kirankumar, Contact me via my profile page and give me your email address. I will send you the documentation I used. I tried contacting you but your email filed.

Kirankumar

Thank you very much for this tool. API helped me to integrate. Can you please provide the reference documents like DAT file structure and parsing mechanism?

News:
Matlab 2016b with the vehicle_network_toolbox supports mdf format!
:D

Stuart McGarrity

Sorry Roman, I'm not actively supporting this tool any more. I can only suggesting trying to debug it yourself i.e. setting "stop if error" and looking at the variables. You could then set a break point just before that location and run it again with the parameters which work and see what the variables look like there.

Hi Stuart

first of all many thanks for this great tool and function. However if try to use the mdfimport function I get an error. I try to load the selected signals into a MAT file but get this error message:

Operands to the || and && operators must be convertible to logical scalar values.

Error in mdfimport (line 55)
            test= ~isempty(varargin{2}) && isgraphics(varargin{2});

If use 'workspace' or [] it will import all signals without any problems but when I use 'Auto MAT-file' or 'mymatfile.mat' I get the error message.

Regards,
Romnano

Stuart McGarrity

Hi Antonello, The tool does not support arbitrary numbers of bits. I'm assuming one of your signals has a number bits that are not supported. It only supports 1,2,8,16 and 32 for unsigned ints (you can see the code there in mdfread).

Problem in mdfimport, Can you help me?

Error using mdfimport>datatypeformat (line 1953)
Unsupported number of bits for unsigned int

Error in mdfimport>mdfread (line 1692)
    datatype=datatypeformat(signalDataType,numberOfBits); %Get signal data type (e.g. 'int8')

Error in mdfimport>mdfload (line 1106)
            [data signalNames]=mdfread(MDFInfo,dataBlock,selectedChannels); % Load specified signals

Error in mdfimport>importdatawithoptions (line 2271)
                mdfload(options.MDFInfo,dataBlock,channelIndices,options.blockDesignation,ws,options.additionalText);

Error in mdfimport>importdata_Callback (line 354)
    importdatawithoptions(options);

Error in gui_mainfcn (line 95)
        feval(varargin{:});

Error in mdfimport (line 87)
    gui_mainfcn(gui_State, varargin{:});

Stuart McGarrity

Sorry Jake, I'm not actively supporting this tool any more. It would take a me a while to refresh my memory and I would need your data file to analyze. Our consulting group knows a bit about this file format.

Jake Silhavy

Great tool, thanks Stuart.

A problem I have been having: when I import two signals with the same idealized sample rate from the same MDF file, the matrices generated by matlab are different sizes. Usually they differ by either 2 or 16 entries. And one of the corresponding time vectors will be slightly closer (16 * 12.5 ms closer) to the actual max time. This is a significant problem for me, since I am comparing the two signals very closely. Any suggestions?

Stuart McGarrity

The tool makes pretty efficient use of the low-level function fread. Multiple MDF Signals are "interlaced" together in the file, so if you do not load them all, fread will need to skip over thr bytes from the other signals. It could be that skipping is not much faster than reading.

All I can suggest is not saving the other signals you don't need (if you own making the file)

fengbo li

Hallo,

This function is cool. Thank you Stuart.

By using this function I am importing .dat to Matlab but importing time(Computational time) is very high. How can I reduce importing time?

For example : It is not so much big difference when I import all signals from one data (.dat) and 4 signals from same data.

Tank you in advance

Regards
Mani

Josef Laumer

Very great tool! I use it nearly every day.

Regarding the post of Julian: I build an additional parameter for the function, to access the function "mdfinfo" and put the output in the varargout statement. Maybe this functionality, to get the info, a signal list, ... could be introduced in this version.

Also the submisson from Daniel Findeisen (http://www.mathworks.com/matlabcentral/fileexchange/52952-mdf-import-tool-and-function-updated-for-mdf-file-format-from-inca-7-1-9) could be a upgrade of this version here. This would be great!

And of course, the support of MDF4 would be indeed great! Maybe i take a look in the python implementation in the next vacation.

Regards

Stuart McGarrity

Hi Julian, There is a subfunction called mdfinfo, inside the main function you can extract and save separately that returns information about the file contents:

function [MDFsummary, MDFstructure, counts, channelList]=mdfinfo(fileName)
% MDFINFO Return information about an MDF (Measure Data Format) file
%
% MDFSUMMARY = mdfinfo(FILENAME) returns an array of structures, one for
% each data group, containing key information about all channels in each
% group. FILENAME is a string that specifies the name of the MDF file.
%
% [..., MDFSTRUCTURE] = mdfinfo(FILENAME) returns a structure containing
% all MDF file information matching the structure of the file. This data structure
% match closely the structure of the data file.
%
% [..., COUNTS] = mdfinfo(FILENAME) contains the total
% number of channel groups and channels.

Julian

Julian (view profile)

Very good tool, have been using this for years.
Question: Is there a way to query the mdf file, prior to loading in the data, to get a list of signals available? This would be very helpful.
Thanks,
Julian

Hey Gsab,
MDF4 uses completely different headers and structure compared to MDF3. You will most probably be not able to read MDF4 with this tool.
MDF4 can store more kind of data and introduces metadata, variable length data, not byte aligned date, etc. but it is also much more complex.
You can have an idea of it by looking at this code:
https://github.com/ratal/mdfreader
Regards

Daniel F.

Daniel F. (view profile)

Hey Gsab,

I have not tried to use it with MDF4 - maybe you can try wether it works or not. If not, feel free to provide a file for me and I can try to update the function according to MDF4 file format.

Best regards.

gsab

gsab (view profile)

Can we use this tool to import MDF 4 files in Matlab ?

Daniel F.

Daniel F. (view profile)

Hello everyone,

there is indeed a new format formula. I did some debugging on "mdfimport" to check how it worked and found the following:

1. I checked the new format in ETAS MDA
2. the new identifier is "9"
3. the representation I found up to now matched the following format:
[P1] = 0
[P2] = factor
[P3] = Offset
[P4] = 0
[P5] = 0
[P6] = 1

Therefor I modified my "mdfimport" to suit these new requirements and it works so far. How can I distribute this updated file? Can anyone tell me?

Stuart McGarrity

I assume the new .dat file is using a conversion formula that I did not implement. Sorry, but I am not maintaining this submission any more.

Hi Stuart.
I've used the mdfimport as m-function.
Up to INCA V7.1.8, there are not any problems to import measurement file(*.dat). but in case of INCA V7.1.9, it can't import with this error message.

=====================
Error using mdfimport>mdfread (line 2009)
Conversion Formula Identifier not supported

Error in mdfimport>mdfload (line 1201)
            [data signalNames]=mdfread(MDFInfo,dataBlock,selectedChannels); % Load specified signals

Error in mdfimport>importdatawithoptions (line 2511)
                    [size names] =
                    mdfload(options.MDFInfo,dataBlock,channelIndices,options.blockDesignation,ws,options.additionalText,options.newSampleRate);
                    
Error in mdfimport (line 99)
            importdatawithoptions(options);
====================
Could you help me ?

Stuart McGarrity

Is "facCorrnElup_Cprbe_CddElecUndrPpmp_tElupBoardEstimrprbe_CddElecUndrPPmp_HWEprbe_CddElecUndrPPmp_18" the name of one of your signals? The genvarname function in the tool should shorten it to less than 63 characters for MATLAB to handle. Any way to make the signal name shorter? Or try setting the code to dbstop if error and look at the code when it errors-out and see why genvarname is failing. Perhaps you have multiple similar long-name signals?

Denis Ott

Hi Stuart,

when I try to open my .dat file with a lot of signals. It produce this error:

Warning: Ignoring modified signal name
'facCorrnElup_Cprbe_CddElecUndrPpmp_tElupBoardEstimrprbe_CddElecUndrPPmp_HWEprbe_CddElecUndrPPmp_18'.

Cannot be turned into a variable name.

> In mdfimport>mdfload at 1158
  In mdfimport>importdatawithoptions at 2259
  In mdfimport>importdata_Callback at 354
  In gui_mainfcn at 96
  In mdfimport at 87

Can you help me?
Thanks

Denis

Stuart McGarrity

Hi Claudio, Are you trying to get handles to the graphical objects? You can't do it with an output argument. The GUI is designed to also be called as a function and return data (see the documentation).

claudio

Hi Stuart,

tool works well but, when I try to get handles, Matlab (v.7.10.0) produces an error

--> h=mdfimport(filename)

Created 39 signal variable(s) appended with '_1' for '12.5ms' rate
... and 1 actual time vector 'time_1'

 .
 .
 .
 .
 .

Created 19 signal variable(s) appended with '_10' for '3.125ms' rate
... and 1 actual time vector 'time_10'
Error in ==> mdfimport at 50
if nargin % If arguments past in

??? Output argument "varargout" (and maybe others) not assigned during call to
"C:\Users\***********************************
mdfimport.m>mdfimport".

Can you help me?
Thanks

Claudio

Ahmad Fadel

Hi Stuart,

Ran into an error if variables in MDF file contain German special characters. ['ä','ö','ü','ß','Ä','Ö','Ü']

would be nice to add the following line to the code:
varName = stripchars(varName,['ä','ö','ü','ß','Ä','Ö','Ü']);

thanks,
ahmad

jaidev

jaidev (view profile)

Hi Stuart,

This is great tool for INCA generated MDF (*.dat) files. Though I am unable to comprehend the functionaliy of Ideal Uniform time vector creation from the GUI.

Once Import Signals is pressed, all the variables generated in workspace have no change in the vector lengths. The unequal lengths of several parameters remain unchanged for the actual time and even for the ideal uniform time vector selection. To my understanding when I choose an ideal time of say 10 msec, then all the data should have been interpolated on 1 uniform time axis threby providing all vectors of equal length, i dont see this happening.

Can you please comment on the expected behavior.

Regards,
Jai
 

Stuart McGarrity

Sorry, I don't have plans to.

Michael H

Hi Stuart,
will you update your import tool for MDF 4.0/4.1/4.1 with compression?
I can provide sample files if needed.

Thanks

Michael

Stuart McGarrity

I'm afaid I can't rememeber for sure. I think there was at least one text only data format it could inport. I would try.

Felix

Felix (view profile)

Hi Stuart,

is it possible to export signals' units and comments from the INCA Files to MATLAB?

Thanks for your help

Felix

Gabor

Gabor (view profile)

Hi Stuart,

I get following error on importing an mdf-file:

??? Error using ==> mdfimport>datatypeformat at 1945
Unsupported number of bits for unsigned int

Error in ==> mdfimport>mdfread at 1684
    datatype=datatypeformat(signalDataType,numberOfBits); %Get signal data type (e.g.
    'int8')

Error in ==> mdfimport>mdfload at 1098
            [data signalNames]=mdfread(MDFInfo,dataBlock,selectedChannels); % Load
            specified signals

Error in ==> mdfimport>importdatawithoptions at 2251
                mdfload(options.MDFInfo,dataBlock,channelIndices,options.blockDesignation,ws,options.additionalText);
                
Error in ==> mdfimport at 59
        importdatawithoptions(options);

Any ideas how to solve this?

Thanks!

Gabor

Stuart McGarrity

This tool does not provide support for writing to MDF files.

Vaijanath

Nice tool,
is it possible to convert back Matlab files like .mat or .fig into .mdf format?

Stuart McGarrity

The variable used in the ASAM-MCD2 text conversion formula is usually 'x'. Looks like it is 'X' in your MDF file.

Try editing the line before where the error happened:

x=double(Blockcell{selectedChannel});

...to replace the lower case x with upper case.

James

James (view profile)

Tool seems to work great, but I'm getting the following error sometimes when I call it as a function, can't seem to figure out why...

Error using eval
Undefined function or variable 'X'.

Error in mdfimport>mdfread (line 1825)
                data{selectedChannel}=eval(textFormula);
                % Evaluate string

Error in mdfimport>mdfload (line 1098)
            [data
            signalNames]=mdfread(MDFInfo,dataBlock,selectedChannels);
            % Load specified signals

Error in mdfimport>importdatawithoptions (line
2251)
                mdfload(options.MDFInfo,dataBlock,channelIndices,options.blockDesignation,ws,options.additionalText);
                
Error in mdfimport (line 54)
            importdatawithoptions(options);

I have following error.... Can someone help me in fixing it....

Subscripted assignment dimension mismatch.

Error in mdfimport>mdfblockread (line 1918)
            Block.(fieldname)(record,:)=fread(fid,count,['*' precision])';

Error in mdfimport>mdfinfo (line 1230)
    TXBlockTemp2=mdfblockread(formatstxtext(TXBlockTemp.blockSize),fid,current,1);

Error in mdfimport>populate_GUI (line 401)
[MDFsummary, MDFInfo, counts, channelList]=mdfinfo(handles.fullFileName);

Error in mdfimport>Load_MDF_File_Callback (line 607)
        handles=populate_GUI(handles); % Populate GUI (list box)

Error in gui_mainfcn (line 96)
        feval(varargin{:});

Error in mdfimport (line 78)
    gui_mainfcn(gui_State, varargin{:});
 
Error while evaluating uimenu Callback

how do I use this tool ? I am so new to Matlab... I have (.dat) files and I want them to make it compatible with Matlab for some program

satish

satish (view profile)

Thank you

Sebastian

Ok, I do it one by one after driving ;)
Didnt you find another script with google ? I also heard about a Toolbox doing this. But I dont know any details

shilpa

shilpa (view profile)

Thanks Stuart :)

Hello Sebestian ,

But in CANape I couldnt convert more than 40 mdf files at a time ..
Can you pls tell how did you execute it

Thanks in advance :)

Sebastian

Yes, I do it with CANape.

Stuart McGarrity

I believe the INCA or CANape tools do this.

shilpa

shilpa (view profile)

thanks Stuart,

is there any way to convert 2.14 v mdf files to mat files?
thanks in advance :)

Stuart McGarrity

Unfortunately only v3.0 of the standard is supported (and not all of it)

shilpa

shilpa (view profile)

This Tool is not working for mdf version of 2.14.. any help ?

Thanks :)

AlainMil

is it possible to recover automatically the measurement file comments?
- date
- hour
- project
- subject
- user comment

these informations are available with ETAS-MDA or by opening the file in a text editor

AlainMil

sometime it's necessary to adapt the subfunction "mygenvarname" in "mdfimport.m" for having valid variable names

by adding some lines like :

varName = strrep(varName,'à','a'); % Replace 'à' with 'a'

I use this method also to shorten long variable names :

example :
varName = strrep(varName,'ETCSignals_ES1000_ES1135_ES1232_ETK_','');

Jaco

Jaco (view profile)

Very helpful but doesn't allow variable list (VAR) import option: mdfimport(fullFileName,'workspace',VAR,'ideal'). First time raster (time1) is incorrectly populated and error when looking for txt file if VAR>3. This error was noticed when used as a function in the command line.

I am trying to import some INCA data into the workspace. The problem I am running into is the the length of the signal name is more than 64 characters. Is there a way to change this limit.

What alternate options do I have to get this data into MATLAB?

Thanks,
Nilesh

Nikhil

Nikhil (view profile)

AlainMil

Great Tool. But it doesn't take older files with MDF-Version 2.14 - any help what can I do?

Adrian

Adrian (view profile)

Nice tool :)

Hi all,

my name is Andreas Patzer and I'm the business development manager of CANape (Vector Informatik GmbH).

Thanks for the development of the import possibility of MDF-files into Matlab.
Please keep in mind that there is not ‘THE’ MDF file. In the early 90’ the MDF format was developed by Vector and Bosch and meanwhile there are some different MDF versions that are supported by CANape. Since years every development step was done in corporation with ETAS, to make sure that their and our customers have the same format.

One of the comments was like “ … not any more dependent of INCA MDA ..”. From my opinion ;-)) the very best tool to analyze MDF files is CANape or the analyzing subset of CANape, called CANgraph.

If you want to handle the MDF content in the Matlab workspace, please use the export possibilities in CANape. File | Converter options | Matlab Export Converter …

Please do not hesitate to contact me Andreas.patzer@vector-informatik.de

Thanks
Andreas

Ingmar Brandt

Hello!

I've got some problems with the tool. Some signals in my CANape generated mdf-files (MDF - Version 3.1) are importable, some others aren't. The error message is the following one:

??? Error using ==> mdfimport>datatypeformat at 1944
Unsupported number of bits for unsigned int

Error in ==> mdfimport>mdfread at 1683
    datatype=datatypeformat(signalDataType,numberOfBits); %Get signal
    data type (e.g. 'int8')

Error in ==> mdfimport>mdfload at 1097
            [data
            signalNames]=mdfread(MDFInfo,dataBlock,selectedChannels);
            % Load specified signals

Error in ==> mdfimport>importdatawithoptions at 2250
                mdfload(options.MDFInfo,dataBlock,channelIndices,options.blockDesignation,ws,options.additionalText);
                
Error in ==> mdfimport>importdata_Callback at 345
    importdatawithoptions(options);

Error in ==> gui_mainfcn at 96
        feval(varargin{:});

Error in ==> mdfimport at 78
    gui_mainfcn(gui_State, varargin{:});
 
??? Error while evaluating uicontrol Callback

Does anybody have an idea if this problem is fixable? And if, how?

Thx Ingmar

J. Wang

excellent tool!

Andreas .

Thanks a lot!

Sven P.

Sunil Lengare

Thank you Great Material

KC YANG

It works well with INCA data. Next interation must be import all files at the same time.

firstname lastname

fantastic!

Jedediah Frey

There is a problem with some Canape created files. The 'longSignalName' is generated from the comment field on line 1274:
CNBlockTemp.longSignalName=TXBlockTemp.comment;

And when the signal names are used, the program only checks to see if longSignalName is not empty (Line 1578), a better method would be to use 'isvarname' to determine if the longSignalName is a valid Matlab variable name. Currently we have a few channels where the longSignalNames come back as gibberish.

This is my suggested fix:
Line 1578 to 1582:
        if isvarname(longSignalNames{signal}) % If no long name, use signal name
            useNames(signal)=longSignalNames(signal); % Use Long name
        else
            useNames(signal)=signalNames(signal);
        end

Line 1599 to 1603:
        if length(signalNames{signal})>length(longSignalNames{signal})||~isvarname(longSignalNames{signal}) % If signal name longer use it
            useNames(signal)=signalNames(signal);
        else
            useNames(signal)=longSignalNames(signal); % Use Long name
        end

jerome M

Doesn't work with my mdf from CANApe. And doesn't display well on Matlab for Linux

Mark Zurawski

It works well on all of my older MDF's. However, some of the newer MDF's from the Vector CANcaseXL Log seem to have an issue.

Here is a detail of the error:

??? Reference to non-existent field 'CNBlock'.

Error in ==> mdfimport>mdfinfo at 1569
    pos=zeros(length(CGBlockTemp.CNBlock),1);

Error in ==> mdfimport>populate_GUI at 456
[MDFsummary, MDFInfo, counts, channelList]=mdfinfo(handles.fullFileName);

Error in ==> mdfimport>Load_MDF_File_Callback at 664
        handles=populate_GUI(handles); Populate GUI (list box)

Error in ==> gui_mainfcn at 75
        feval(varargin{:});

Error in ==> mdfimport at 124
    gui_mainfcn(gui_State, varargin{:});

??? Error using ==> mdfimport('Load_MDF_File_Callback',gcbo,[],guidata(gcbo))
Reference to non-existent field 'CNBlock'.

??? Error while evaluating uimenu Callback

Manohar Chenekala

Sir,
Im using mdfimport to convert mdf files(CANAPE) to mat and m file. Im succeeding for some files. for some files Im unable to get M-file. Can you tell me what the limitations and compatibilities with MDF files(version,size etc). my id is manoharc@cmcltd.com

With regards,
Manohar.ch

john coppola

sweet

jerome edwards

G D

 Doesn't work with formula conversion type 9, hence could not use for data import

Elmar Riesmeier

Very useful tool - saves time and works just fine as function or via GUI.

Timo Schmidt

Great!
Good feeling to be not dependent on ETAS-MDA any more.

Gene DiValentin

I've been using this tool for a couple of months and it's fast, easy, and intuative to use. Postprocessing INCA data has become much eaiser. Thank you.

Liu Qiang

Great!
After a month of analysis of the binary file, I find this tool just before I begin to write a similar tool.
Am I lucky or unlucky? :-P

Xiao Zhang

Very nice job. Save us a lot of time. Thank you.

Sebastien LAURENT

Very good job

Douglas Basberg

Works seamlessly. Intuitive interface.
Help file is clear, but could use some grammer and typo checking. A big well done. This package will save many people hours. Thank you.

Andrew Herman

A great asset to our algorithm development and validation efforts!

Brian Gebby

Very worthwhile tool!

Updates

1.7.0.1

Updated license

1.7

Updated to work in R2014b

1.6

Added copyright

1.4

n/a

1.3

Copyright work.

1.1

Added trademarks.

* Added support for string data types
* Added support from formula conversion types 10 and 12

Modified Summary

Removed version number

* Now callable as a command.
* Generate the equivalent command and parameters
* Save imported signals to a MAT-File.
* Append variable names with rate strings
* Append variable names with additional text

* Ensure that long signal names are used for variables. Previously, signal names longer than 32 characters were truncated

* Changed "Remove device names" check box text to "Hide device names"

Add missing "Create signal selection file" menu item. Change version number from .2 to 1.0

MATLAB Release
MATLAB 6.5 (R13)

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video