TDMS Reader

Read TDMS files v1 & v2 without DLL Current Version: 2.5

Description

Reads TDMS files into Matlab.

Advantages:
   - supports reading v2 files
   - doesn't require the NI DLL, thus it doesn't require 32bit windows
   - supports interleaved data
   - allows only reading names & properties to get a quick feel for what is in the file
   - allows reading specific subsets of the data for limited memory usage

• Required Products: MATLAB
• MATLAB release: MATLAB 7.10 (R2010a)

[Jim Hokanson]

Amrita,

Please add the tdmsSubfunctions directory to your path as well.

Jim

[Amrita]

I get the following error when I try to read a tdms file:

??? Undefined function or method 'TDMS_handleGetDataOption' for input
arguments of type 'struct'.

Error in ==> TDMS_readTDMSFile at 216
TDMS_handleGetDataOption('check',paramsStruct)

Error in ==> TDMS_getStruct at 57
[temp,metaStruct] = TDMS_readTDMSFile(filePath,readOptions{:});

How do I solve this?
Thanks!

[Robert]

I am trying to use the file to convert tdms file but I have a following error:

Error using TDMS_preprocessFile (line 450)
The remaining data doesn't split evently into chunks, estimated # of chunks: 8.181818e-01

Error in TDMS_readTDMSFile (line 226)
metaStruct = TDMS_preprocessFile(fid,tdmsFileName,paramsStruct);

Could someone help me to solve it, please?
Thanks

[Joe Cuschieri]

I am using your algorithm posted on the MATLAB Central to read data from a TDMS file, however I am getting an error which I do not quite understand why.

This is a section of my code:
getStruct = struct;
getStruct.group = {'6120'};
getStruct.channel = {'Untitled 3'};
getStruct.indices = [2 3];
tempOutput4 = TDMS_readTDMSFile(fname,'GET_INDICES',getStruct,'SUBSET_IS_LENGTH',false)

However I get these errors:
Error using cell/ismember>cellismemberlegacy (line 132)
Input A of class cell and input B of class cell must be cell arrays of strings,
unless one is a string.

Error in cell/ismember (line 76)
[varargout{1:nlhs}] = cellismemberlegacy(varargin{:});

Error in TDMS_handleGetDataOption (line 167)
[~,loc] = ismember(objPaths_getIndex,objectPaths);

Error in TDMS_readTDMSFile (line 228)
optionStruct = TDMS_handleGetDataOption('getArray',paramsStruct,metaStruct); %Option
handling

Error in Test1 (line 21)
tempOutput4 =
TDMS_readTDMSFile(fname,'GET_INDICES',getStruct,'SUBSET_IS_LENGTH',false)

I am using MATLAB R2012b.

Any suggestions?

[Jim Hokanson]

René,

TDMS doesn't contain time unless you explicitly save it as a channel. This is most often only done if you are sampling something periodically, such that the timing between events is not constant. Otherwise time is usually implied from a known sampling rate. If you are saving the time as a timestamp, I currently return the timestamp array as a numeric vector in Matlab's time format instead of converting each element to a string (like datestr). This makes it easier to do computations on the time (note that a difference of 1 in this vector represents a difference of 1 day, divide by 86400 to go to seconds). If you still have questions feel free to contact me via email.

Jim

[Rene]

Jim,

Like your work a lot. It will save a lot of disk space, since it will be no longer required to save as lvm.

The output format is pretty straightforward, but I however can't figure out what variable contains the array with sample times.
I figure the tdms-file must hold the time since excel can grab it from there.

Could you tell me whether the sample time is also imported into matlab?

Best regards,
René

[Jim Hokanson]

Hi Anthony,

Yes this problem is solved. I tend to send emails offline.

This is the email I sent to Kevin. Let me know if you need anything else:

EMAIL TO KEVIN:
Whoever is giving you the data is most likely not closing the files properly. In addition they are writing the files really inefficiently.

In general every time you write to the file a additional index header is created. It is more efficient to write 100k values to a file than to write 1 value, 100k times with 100k headers. In this case the NAV info is being written in one sample, then the TANDEM is being written for 200 samples. It would be more efficient to write these to some queue in memory and then dump them to file occasionally.

That being said you can read a portion of your files if you change the following:
Assuming you are not doing any fancy reading:
- TDMS_readFileHelper_v1 - comment out near line 187 which says if eofPosition ~= ftell(fid)
- In TDMS_processLeadIn - around line 34, comment out the error and add the following:
flags = [];
info = [];
return
- In TDMS_preprocessFile - around line 180, write the following:
if isempty(flags)
nSegs = nSegs - 1;
break
end

Essentially you are just stopping the code when you get to the error and returning only what you have up until that point. What you have in that file is alot already. My guess is you are only missing a bit of data at the end but I haven't bothered to check that ...

A few others have had similar errors but there is a flag in the header which is specifically for ending too early. This is the first time I have seen the header itself be corrupt like this. Perhaps that is due to working with the tdms file and not the index file.

Good luck and let me know if those changes get things working for you or not.

[Anthony]

Jim,

Was there ever a solution to Kevin's error? I seem to be getting the same error when i run the Matlab code.

[Harm]

@uwe: I have used this function on 64bit Matlab (linux and win64) without any problem.

[Uwe Lelke]

Can I also read simple TDM-files with 64-bit Matlab?

[Jim Hokanson]

Farhan: Sorry about the delayed response. I'm getting ready for a conference and I'm in a bit of a rush to finish things up. I'm glad you got things working. I'll add a check for your error so that others get a more obvious error message. Thanks!

Kevin: File size shouldn't be an issue. In fact I wrote the code specifically to support very large files in which you can do partial reads and process the data before reading more. See my private message on how we might be able to resolve your issue.

[farhan]

Jim,
I resolved the issue, there were some empty Group names that our LabView code was forgetting to assign. We fixed that problem in the LabView code and now the TDMS converts beautifully into a data structure.
Thank you!

[Kevin]

Many thanks for this work it's great but I have an issue when I try to read a bigger file : few MB works well but with a 80 MB file I get this error :
"
??? Error using ==> TDMS_processLeadIn at 33
Unexpected lead in header

Error in ==> TDMS_preprocessFile at 179
[flags,info,eof_error] = TDMS_processLeadIn(fid,lastLetter);

Error in ==> TDMS_readTDMSFile at 226
metaStruct = TDMS_preprocessFile(fid,tdmsFileName,paramsStruct);

Error in ==> TDMS_getStruct at 57
[temp,metaStruct] = TDMS_readTDMSFile(filePath,readOptions{:});
"
Anyway it's working great for the smaller files.

[farhan]

Jim,
I successfully used your code with an older version of our TDMS, but now I am getting an error. The file is intact in that it can open in the Excel reader.

>> TDMS_readTDMSFile('LOG_2012-10-08_10-53-27.tdms')
??? Improper assignment with rectangular empty matrix.

Error in ==> TDMS_readTDMSFile at 275
groupIndices(curGroupCount) = I_groupObject;

Any help would be greatly appreciated!

[Pian Xiaochuan]

Jim, I really appreciate your work. It helps me a loooooooooooooooooooot!!!

Marshall

[Larry]

This code was a god send for me. Very well written and documented, functional and efficient. Thanks so much for contributing this to the public, Jim.

[Jim Hokanson]

Dedric Xu recently reported observing a problem with the code which turned out to be a bug. If I really think hard, I convince myself that it might be possible to have the bug he was observing occur without any noticeable code failure. I would recommend updating the code to V2.5 when you can (once it posts). The bug came from interpreting NI documentation too literally. They have acknowledged the poorly worded documentation and said they will update it. My apologies to anyone else this effected, although I sincerely expect that it was basically no one.

More details can be found in the version info file.

Although I am currently working on other things, I still have in mind the following updates:

1) Port of the code to an object oriented nature. Part of this involves creating more functions. I stayed away from this due to speed concerns with function call overheads and data passing but my decision obviously makes debugging and updating more difficult, as well as potential "optional" mex substitutions at key bottleneck points.
2) A TDMS viewer GUI
3) Support to optionally read data directly into a double vector, instead of the format it was written in and converting after the fact.

[Toby scientis]

After lots of frustration with the code that NL wrote, this worked perfectly on the first try.

[Jim Hokanson]

You need to add the subfunction folder to your path as well. I eventually want to make this a class which makes some methods private and hidden, but in the mean time the sub functions folder just helps delineate between functions you should call and functions you shouldn't.

[Hari Patel]

I tried to read .tdms file with what you have in readMe.txt and I got following error:
>> filename = 'Digital_Input.tdms';
>> my_tdms_struct = TDMS_getStruct(filename)
data = TDMS_readTDMSFile ('Digital_input.tdms')

??? Undefined function or method 'TDMS_handleGetDataOption' for input arguments of type
'struct'.

Error in ==> TDMS_readTDMSFile at 213
TDMS_handleGetDataOption('check',paramsStruct)

Error in ==> TDMS_getStruct at 57
[temp,metaStruct] = TDMS_readTDMSFile(filePath,readOptions{:});

??? Undefined function or method 'TDMS_handleGetDataOption' for input arguments of type
'struct'.

Error in ==> TDMS_readTDMSFile at 213
TDMS_handleGetDataOption('check',paramsStruct)

[Laura]

 

[Laura]

Thanks a lot for the reply and indeed also again the solution Jim, working perfectly!!
(I didn't realized I had done such a change, my mistake)
Thanks again

[Jim Hokanson]

John:

Thanks for the helpful comments. I started writing this file in late 2010 since v2 TDMS files weren't supported by the NI driver, as well as because it meant Mac users in the lab were asking me to read files. I thought for sure NI would be rolling out a driver for v2 but after enough waiting I decided to create my own version. I was quite surprised when I was done with writing the code that there was another version already available. I am not sure how I missed this is in my numerous online searches. There are actually multiple versions of the code you mentioned available. The first version I tried I believe was v2 (still available online) and the enumeration of data types was (is?) wrong. That's a HUGE mistake and the code is still posted. I've looked at the other files and I believe with v6 those implementing the code didn't realize that multiple reads of data are possible within a single "header" (for lack of a better word). This means you may write 2000 values to file and only get 100 back, when in reality you need to read the file 19 more times to get all 2000 samples. Version 9 is much better although it doesn't have the same functionality that mine does, particularly the ability to read only a part of a file, or to read interleaved data. At the time of this writing I believe that is the case, obviously this may change.

I think there are a few key points here though:
1) Posting multiple versions of the same file is not a good idea. Versions 2 & 6 shouldn't exist on the file exchange, unless perhaps as links to some website for archival purposes, or with big warning signs saying "This code doesn't work."
2) More importantly, I think this points out the need for better code integration between files on file exchange, both in terms of shared authorship and in terms of the ability to create entry points to data. Imagine if on the Mathworks website we could create a section titled "File processors" and under it there could be a link to TDMS (National Instruments). On that page there could be a short description about the format along with the various solutions that people have offered. In general I think the File Exchange is outdated and not effective. I believe there have been some attempts to write wrappers around the file exchange but that these have been shut down by TMW. Even now I'm moving my new code to github where although it loses its association with the language/program, I feel there is a better chance that someone can contribute to my code and help to make community software.

[John F]

Ladies and gents, this file is working great. If you're using an older version of Matlab, you'll have to change a couple of the "~" placeholders with what I'm calling "dummy variables" because the ~ is not recognized.

IF ANYONE IS INTERESTED, there is another TDMS conversion program called convertTDMS (v6). I was spinning my tires like mad trying to figure out why that code wasn't working properly.

Unfortunately, convertTDMS would only read part of my file. This one works like a dream.

If anyone has any idea why convertTDMS was only reading part of my file, I'd love to hear your theories.

Best,

JF
Former Noob, Current Novice

[Laura]

P.S.: the different kinds of data is only a problem when working with the files modified to have double precision. It does work with the original version, but then, with single

[Laura]

Hi again Jim!,

I continue making use of your TDMS converter (really a good work!) after your successful recommendations to convert the tdms data in double format.

However I have found another small issue that prevents me to successfully convert my new tdms files. I have been taking a look, but again, I have not been able so far to fix it. When trying to run the converter, I find the following error:

??? Attempted to access dataTypes(10); index out of bounds because numel(dataTypes)=3.

Error in ==> TDMS_readFileHelper_v1 at 76
if any(dataTypes ~= dataTypes(10))

Error in ==> TDMS_readTDMSFile at 249
data = TDMS_readFileHelper_v1(fid,optionStruct,metaStruct,paramsStruct);

Error in ==> TDMS_getStruct at 57
[temp,metaStruct] = TDMS_readTDMSFile(filePath,readOptions{:});

and indeed my tdms files have two types of data: DT_FLOAT, (the only existing one in the first tdms that I was dealing with), but also the last group of data which is now DT_DOUBLE.

What should I do if my tdms file has two types of data? Can I handle it with this converter?

Thanks a lot in advance for whenever you have the time to reply to this request!

Best regards,
Laura

[Richard]

Thank you very much, Jim.

[Brian Kaul]

 

[Jim Hokanson]

Dedric, please check my email. Thanks.
Jim

[Dedric Xu]

Hi Jim,
I've encounteered a maddening problem. I have tried to use your function to read my tdms file,but failed. And my file is a little larger, about 60MB.

When I ran your function, it crashed.

The function I used is [output,metaStruct] = TDMS_getStruct(filePath,4);

And the program has ran about 20min,then it crashed.

I gtt the following error:

Warning: Warning: FOR loop index is too large. Truncating to 2147483647.
> In TDMS_readFileHelper_v1 at 66
In TDMS_readTDMSFile at 249
In TDMS_getStruct at 57
Error using TDMS_readFileHelper_v1 (line 184)
Catastrophic error detected, code probably has an error somewhere

Error in TDMS_readTDMSFile (line 249)
data = TDMS_readFileHelper_v1(fid,optionStruct,metaStruct,paramsStruct);

Error in TDMS_getStruct (line 57)
[temp,metaStruct] = TDMS_readTDMSFile(filePath,readOptions{:});

Is it a Matlab version issue or am I missing something else?

Best regards,
Dedric

[Michael Kung]

 

[Laura]

It worked perfectly!Thanks a lot for the program and for the fast help!
It is really the best work I have found online for tdms conversion

[Laura]

 

[Jim Hokanson]

Laura, you are pointing out an issue that I thought would be good to address but that I just haven't gotten to yet. In general I have found it is good to respect the data type so as to prevent memory overflow for a user, but there are obviously times when you want things read in as a double.

I eventually wanted to handle this more gracefully but you can try the following hack to get things to work:
1) in TDMS_initData, remove the string specification from zeros, in your case it sounds like you want case {9 25} to be:
output = zeros(1,nSamples) instead of output = zeros(1,nSamples,'single')
2) in TDMS_getTypeSeekSizes change the read specification to be "type" instead of "type=>type", in your case you want 'single' instead of 'single=>single'

These two changes should make it so that all doubles are read as singles. I eventually want to make a cleaner fix but I've been too busy lately with other things.

[Laura]

It is some time that I am playing with your routines. It is a fantastic work. However, although it is working fine to convert to matlab a complex .tdms file, I didn't succeed in modifying the precision of the converted values. I guess is specified in the readOptions (different from default), and specifically called in the subroutines TDMS_getTypeSeekSizes and TDMS_readFileHelper_v1. I am not a advanced matlab user. I would really appreciate if you could help me to change those settings.
My problem consists in that I would like to have the record the structured data in double precision instead of in single.
Thanks for your work and attention

LV

[Sergiy]

 

[John Vazey]

I get an error
??? Error using ==> TDMS_readFileHelper_v1 at 188
Catastrophic error detected, code probably has an error somewhere

Has anyone else had this problem?

JV

[Blaine]

I'm new to TDMS files I just assumed that they can all be read the same. The test file I have is just an example file I downloaded from NI. I wasn't aware that Raw DAQ MX was different than any other TDMS file.

[Jim Hokanson]

Regarding improvements in documentation, that is probably one area to improve. Raw DAQ MX data is a proprietary Labview format and I've been turned down in requests to get specifications on that type. I've managed to parse a couple of examples but since I have had so few examples to go off of I haven't tried implementing it. In my experience the raw format is really just read as a normal data type (int16 in my experience??) with properties that specify how to scale that raw data into measured data. I don't know how to extract the real data type from their meta information. I could technically just treat it as some data type, say uint8, but to parse the data generally (not always) relies on knowing the size of the data type. In addition there is some strange buffering of the meta information which I can manually hard code around but I am not sure if it is always present.

That being said, if you have an example file I would be interested in looking at it so that I could eventually offer some workarounds. As I said I have had some luck in extracting the data.

[Blaine]

I tried to open my large data file with:

[tmp,metaStruct] = TDMS_readTDMSfile('C:\Users\bjmarti4.NDC\Desktop\FIRST_LIGHT_CO2.tdms','GET_DATA_OPTION','getnone')

I get the following error:

??? Error using ==> TDMS_processLeadIn at 50
Currently code is unable to ignore/handle Raw Daq MX data

Error in ==> TDMS_preprocessFile at 179
[flags,info] = TDMS_processLeadIn(fid,lastLetter);

Error in ==> TDMS_readTDMSFile at 223
metaStruct = TDMS_preprocessFile(fid,tdmsFileName,paramsStruct);

Error in ==> TDMS_exampleFunctionCalls at 95
[tmp,metaStruct] =
TDMS_readTDMSfile('C:\Users\bjmarti4.NDC\Desktop\FIRST_LIGHT_CO2.tdms','GET_DATA_OPTION','getnone')

I was able to open a small test file, but when I try to open my data file is when I get the error

[Jim Hokanson]

You can be as specific about reading data as you would like. See the documentation file TDMS_retrievingSubsets. You can get as fine grained in reading of the data as you would like, specifying down to the channel and sample numbers if you like. Please let me know if there is anything about the documentation you think should be clarified.

Looking over this more carefully it appears that I have made an error in my documentation related to the meta_struct. I will be updating that soon.

The main error is that subsequent calls is that if you read more more data and want to use the metaStruct from a previous call to reduce parsing time, you still need to pass in the filename again:

TDMS_readTDMSfile(filename,'META_STRUCT',metaStruct,etc)

I tend to recommend using TDMS_getStruct, in which case you would use:

TDMS_getStruct(filename,versionNumber,{'META_STRUCT' metaStruct etc})

[Blaine]

Is it possible to only read sections of a large (~3.5 GB) TDMS file with your code? I would like to work with pieces of the data at a time. If so, how would I do it?

[Jim Hokanson]

Andreas, I am unaware of any bugs in the code at this time. Please check the message I sent you.

[Andreas]

Does the Code work with Matlab 7.11 ?

I've tried it on a 32bit (XP) and a 64bit (Windows7) System with various TDM/TDMS files.
Depending on the type of input file I get 2 different error messages. For native sample files from NI the reaction is:

>> my_tdms_struct = TDMS_getStruct(filePath);
??? Error using ==> TDMS_processLeadIn at 33
Unexpected lead in header

Error in ==> TDMS_preprocessFile at 179
[flags,info] = TDMS_processLeadIn(fid,lastLetter);

Error in ==> TDMS_readTDMSFile at 223
metaStruct = TDMS_preprocessFile(fid,tdmsFileName,paramsStruct);

Error in ==> TDMS_getStruct at 57
[temp,metaStruct] = TDMS_readTDMSFile(filePath,readOptions{:});

My self-generated TDMS files (read with NI Diadem and Excel importer without problems) gave

??? Subscripted assignment dimension mismatch.

Error in ==> TDMS_preprocessFile at 456
rawDataInfo(curIndex).dataMatrix(indices,2) = ...

Error in ==> TDMS_readTDMSFile at 223
metaStruct = TDMS_preprocessFile(fid,tdmsFileName,paramsStruct);

Error in ==> TDMS_getStruct at 57
[temp,metaStruct] = TDMS_readTDMSFile(filePath,readOptions{:});

Is it a Matlab version issue or am I missing something else?

Best regards,
Andreas
[I'm a different Andreas than the one who posted here before..]

[Jim Hokanson]

Hi Andreas,

I've known about that bug for a while I just haven't uploaded a fix yet. I had been a bit slow since I wanted to incorporate some other changes and it is an error that will be thrown as opposed to failing silently and pretending to work. I'll upload a modification very soon but for now you just need to replace in that particular case structure (starting around line 100) in TDMS_readFileHelper_v1 the following:
dataType -> change to -> dataTypes(1)

Jim

[Andreas]

Hi Jim,

It looks like your script does exactly what I need. But when I tried to use it today, I get the following error with my tdms file:

>> test=TDMS_readTDMSFile('test.tdms')

??? Undefined function or variable "dataType".

Error in ==> TDMS_readFileHelper_v1 at 106
temp = fread(fid,numValuesRead*nChans,precisionType{dataType});

Error in ==> TDMS_readTDMSFile at 258
data = TDMS_readFileHelper_v1(fid,optionStruct,metaStruct,paramsStruct);

Reading the index file is working fine and i get the following result:

>>test=TDMS_readTDMSFile('test.tdms_index')

test =
rootIndex: 1
groupIndices: 2
groupNames: {'Analog Input'}
chanIndices: {[3 4 5 6 7]}
chanNames: {{1x5 cell}}
data: {[] [] [] [] [] [] []}
propNames: {{1x1 cell} {1x0 cell} {1x1 cell} {1x1 cell} {1x1 cell} {1x1 cell} {1x1 cell}}
propValues: {{1x1 cell} {1x0 cell} {1x1 cell} {1x1 cell} {1x1 cell} {1x1 cell} {1x1 cell}}
objectPathsOrig: {'/' '/'Analog Input'' '/'Analog Input'/'T'' '/'Analog Input'/'U'' '/'Analog Input'/'I'' '/'Analog Input'/'B'' '/'Analog Input'/'H''}
numberDataPointsRaw: [0 0 28000 28000 28000 28000 28000]
dataType: [0 0 10 10 10 10 10]
dataTypeName: {'void' 'void' 'double' 'double' 'double' 'double' 'double'}

Maybe you have a tip for me what I have to change in the script to read my tdms file?

Thank you in advance.

Best regards

Andreas

[Jim Hokanson]

That shouldn't be a problem. The raw format is currently unsupported because its type is unspecified.

[Jeffrey Fletcher]

Great piece of work - thanks.
I get an erro when trying to read by TDMS file created directly by the NI-DAQmx .NET TdmsLogging function. The error stack is:

??? Error using ==> TDMS_processLeadIn at 50
Currently code is unable to ignore/handle Raw Daq MX data
Error in ==> TDMS_preprocessFile at 175
[flags,info] = TDMS_processLeadIn(fid,lastLetter);
Error in ==> TDMS_readTDMSFile at 225
metaStruct = TDMS_preprocessFile(fid,tdmsFileName,paramsStruct);
Error in ==> TDMS_getStruct at 57
[temp,metaStruct] = TDMS_readTDMSFile(filePath,readOptions{:});

It appears that "Raw DAQmx data" is not currently supported. It's too bad because you can create a data logger in MATLAB (with ONLY NI-DAQmx and base MATLAB installed) with a few lines of code. I'll share this with anyone interested.

Any chance of an update that supports this data type?
Jeff

[Jim Hokanson]

Craig has identified a 2nd bug that is present in v2.2. This bug effects all previous versions. Group names with "/" characters are currently incorrectly parsed. An update to v2.2 will be released soon that addresses this bug, and the getSubset,IgnoreSubset bug. NOTE: Both of these bugs will cause errors instead of failing silently. Thanks guys for letting me know you had a problem so I could improve the code.

Jim

[Craig Smith]

Jim, Please ignore my earlier comment. I have resolved (stupid user error by me). I now get to the following error:
my_tdms_struct = TDMS_getStruct(filename);
??? In an assignment A(I) = B, the number of elements in B and
I must be the same.

Error in ==> TDMS_readTDMSFile at 255
groupIndices(curGroupCount) = I_groupObject;

Error in ==> TDMS_getStruct at 57
[temp,metaStruct] = TDMS_readTDMSFile(filePath,readOptions{:});

The value of curGroupCount is 1 but the I_groupObject is an array of length 14 with integer values of 2 through 15. Not sure why the mismatch here

[Craig Smith]

Jim, I've just downloaded your files and tried to read a TDMS file v2 astarting with what you have in the readMe.txt file. When I run TDMS_getStruct(filename) and select my file I get the following error messages:
??? Undefined function or method 'TDMS_handleGetDataOption' for input arguments of type 'struct'.

Error in ==> TDMS_readTDMSFile at 192
TDMS_handleGetDataOption('check',paramsStruct)

Error in ==> TDMS_getStruct at 57
[temp,metaStruct] = TDMS_readTDMSFile(filePath,readOptions{:});

Any ideas how I should proceed?

[Jim Hokanson]

Found bug for v2.2 effecting use of GET_DATA_OPTION with values 'getSubset' or 'ignoreSubset'
This bug will already throw an error to let you know the read was incorrect

If you get the error:
The # of requested values does not equal the # of returned values, error in code likely
Then this should be fixed by adding this line:
numValuesToGetActual(~keepDataArray) = 0;

after these lines: (around line 133)
useSubset = false;
subsetInfo = struct([]);
numValuesToGetActual = metaStruct.numberDataPoints;

This will be fixed in the next version.

[Jim Hokanson]

Hmmm,

In general the code looks fine. I would throw in the already parsed meta structure back into the function call to save processing time. I'll send some instructions on that later. As the code states when a problem like that happens it is most likely my fault. In general I try and throw in checks where possible because I'd rather have things not work and tell you than pretend to work. I'll send you an email with more instructions on how we can find a fix to this problem.

Jim

[Juha Suomalainen]

Hi,
Great code you have here but I think there is still a bug somewhere inside it. I am able to load my whole TDMS file using you function. That works fine. Unfortunately some of my datasets will be huge and due to memory issues must be loaded and processed in parts. If I try to read a subset of data using your function, I get an error.

Here is my code to read and process data group by group:
---
% Get list what is inside the TDMS
params = {'GET_DATA_OPTION','getnone'};
TDMS_Structure = TDMS_readTDMSFile(TDMS_File, params{:});
% Read and process groups from the TDMS file one by one
for GroupNumber = 1:numel(TDMS_Structure.groupNames)
params = {...
'GET_DATA_OPTION', 'getSubset',...
'OBJECTS_GET', struct('groupsKeep',{{TDMS_Structure.groupNames{GroupNumber}}})};
TempData = TDMS_readTDMSFile(TDMS_File, params{:});
% process subset of data here
ProcessData(TempData)
end
---

I can read the TDMS_Structure without problems, but the subset loading call of 'TDMS_readTDMSFile' produces an error:

---
??? Error using ==> TDMS_readFileHelper_v1 at 211
The # of requested values does not equal the # of returned values, error in code likely

Error in ==> TDMS_readTDMSFile at 229
data = TDMS_readFileHelper_v1(fid,optionStruct,metaStruct,paramsStruct);
---

Is it a bug or am I just using your code with wrong parameters?

[Jim Hokanson]

Hi Fiona,

Thanks for pointing out the issue. I'm really busy early this week but I might be able to look at this issue Thursday or Friday. See me email for more details.

Jim

[Fiona]

Thanks for uploading!
When I tried to read my TDMS file, I got the errors below. I wonder if anyone can help. Thanks!

??? Error using ==> TDMS_preprocessFile at 375
The remaining data doesn't split evently into chunks, estimated # of chunks: 5.000000e-001

Error in ==> TDMS_readTDMSFile at 198
metaStruct = TDMS_preprocessFile(fid,tdmsFileName,paramsStruct);

Error in ==> Test at 11
[finalOutput,metaStruct] = TDMS_readTDMSFile(tdmsFileName)

[Jim Hokanson]

Yes it does. I use it with 64 bit Windows and it has been tested with a 64 bit Mac.

[Kevin]

please advise if this works with 64bit matlab

[Jim Hokanson]

@Guosong Zhang
In 2009b the ~ was introduced as a way to ignore outputs. This can be especially useful if that output requires a lot of memory. I've gotten in the habit of putting those in although I realize not everyone may be using 2009b or later. I might upload a fix in which I put dummy variables back, although I'd be tempted instead to write a small script that allows checking the version and doing a replace in all the files if needed.

[Guosong Zhang]

??? Error: File: TDMS_readTDMSFile.m Line: 453 Column: 11
Expression or statement is incorrect--possibly unbalanced (, {, or [.

[Harm]

Works great! The NI tool with the DLLs failed on me (64bit version does not seem to work, 32 bit version was OK).
But this one is better; and I think it will work on Linux systems as well.

[Harm]

 

[Edward Zechmann]

 

[Ray]

 

[Jim Hokanson]

Joacim,

I fixed the unicode bug. Ver. 1.1 should be up soon. I had misread some documentation regarding # of bytes vs # of characters. Thanks for pointing out my mistake.

Jim

[Jim Hokanson]

Hi all,

In response to Joacim's comments:

1) There should be no file size limitation (other than the users memory size), the out of memory indicates a parsing error.

2) I am still quite unsure as to how well the code handles unicode. The error seems to indicate an ability to properly parse a string ...

3) If anyone sees a similar error please feel free to email an example file to me and I'll gladly try and fix the code so that it is handled.

Thanks,
Jim

[Joacim]

Impressive amount of work you've put into this. Good job and well documented.
Tested the function on my data saved by Signal Express 2009. Works for small files (<6 MB) but when trying to load larger files (>50 MB) it crashes.
I'm lazy so I downloaded the matlab_tdm_example_sp2010.zip from NI. Using the nilibddc.dll and approx. 200 lines of code, including comments, I'm reading even larger files (approx 320 MB) in less than 4 seconds.

Below is some info from the crash. Maybe it can help you fix the problem:

testTDMSreading
266 propName = fread(fid,propNameLength,'*char')';
??? Error using ==> fread
Out of memory. Type HELP MEMORY for your options.

Error in ==> TDMS_preprocessFile at 266
propName = fread(fid,propNameLength,'*char')';

Error in ==> TDMS_readTDMSFile at 176
[rawDataInfo,numberDataPoints,objectPaths,segInfo] =
TDMS_preprocessFile(fid2,isIndexFID2,params);

Error in ==> testTDMSreading at 34
output = TDMS_readTDMSFile(fullfileName,optionCA{:});

>> testTDMSreading
266 propName = fread(fid,propNameLength,'*char')';
K>> propNameLength
propNameLength =
1.9629e+009
K>> fid
fid =
12
K>> curProps
curProps =
Columns 1 through 8
'format-string' [1x22 char] 'unit_string' [] [] [] [] []
Columns 9 through 20
[] [] [] [] [] [] [] [] [] [] [] []
K>> iProp
iProp =
4
K>> propNameLength
propNameLength =
1.9629e+009