Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Matfile - spped up I/O

Subject: Matfile - spped up I/O

From: Yavuz kaya

Date: 1 Sep, 2013 04:28:09

Message: 1 of 2

I have an application in Matlab that reads one text file, generated by a data acquisition system, every one second. The data is then processed, and the results are stored in a mat file using "matfile" function. Below is an example of my code:

matObj = matfile('Kasim.mat','Writable',true);
a1=1; a2=200;
for j=1:1:300
    for k=1:1:5
        matObj.Ch1_RawData(a1:a2, k) = rand(200, 1);
        matObj.Ch2_RawData(a1:a2, k) = rand(200, 1);
        matObj.Ch3_RawData(a1:a2, k) = rand(200, 1);
        matObj.Ch4_RawData(a1:a2, k) = rand(200, 1);
        matObj.Ch5_RawData(a1:a2, k) = rand(200, 1);
        matObj.Ch6_RawData(a1:a2, k) = rand(200, 1);
        matObj.Ch7_RawData(a1:a2, k) = rand(200, 1);
        matObj.Ch8_RawData(a1:a2, k) = rand(200, 1);
        matObj.Ch9_RawData(a1:a2, k) = rand(200, 1);
        matObj.Ch10_RawData(a1:a2, k) = rand(200, 1);
        .
        .
        .
        matObj.Ch59_RawData(a1:a2, k) = rand(200, 1);
end
    a1=a1+200; a2=a2+200;
end

The randomly generated numbers represent the output that my Matlab application generates, and they need to stored in local hard disk of the PC.

Now my problem: I have a lot of output generated every one second and all of which must be stored in real-time in a mat file. If I try to store all of the output in a mat file, it takes more than 2 seconds to write all output into mat file, but I have to speed it up as I have another data coming every one second.

Do you have any idea about how to speed up writing into mat file? Or is there a mistake in my code above? May be there is an efficient way of doing it.

I was also thinking of the following:
Is the data, which is stored into matfile, compressed before writing to mat file? If so, is there a way to disable the compression using "matfile" function? Maybe it is the compression process that slow downs the process.

Thanks in advance and any help would be greatly appreciated.
Yavuz

Subject: Matfile - spped up I/O

From: Yair Altman

Date: 1 Sep, 2013 08:34:10

Message: 2 of 2

"Yavuz kaya" wrote in message <kvufop$bmd$1@newscl01ah.mathworks.com>...
> I have an application in Matlab that reads one text file, generated by a data acquisition system, every one second. The data is then processed, and the results are stored in a mat file using "matfile" function. Below is an example of my code:
[snip]
> The randomly generated numbers represent the output that my Matlab application generates, and they need to stored in local hard disk of the PC.
>
> Now my problem: I have a lot of output generated every one second and all of which must be stored in real-time in a mat file. If I try to store all of the output in a mat file, it takes more than 2 seconds to write all output into mat file, but I have to speed it up as I have another data coming every one second.
>
> Do you have any idea about how to speed up writing into mat file? Or is there a mistake in my code above? May be there is an efficient way of doing it.
>
> I was also thinking of the following:
> Is the data, which is stored into matfile, compressed before writing to mat file? If so, is there a way to disable the compression using "matfile" function? Maybe it is the compression process that slow downs the process.
>
> Thanks in advance and any help would be greatly appreciated.
> Yavuz


There are several things you could try:
1. Use a solid-state hard disk (SSD) - this will significantly speed-up the save/load
2. Use save() directly rather than the matfile() function. matfile() is slower as it is a wrapper to a class with multiple checks, that ultimately calls save()
3. Use save '-v6' rather than the default (much slower) '-v7.3' - see http://undocumentedmatlab.com/blog/improving-save-performance/
4. Modify your logic to save every N seconds, rather than every second

Yair Altman
http://UndocumentedMatlab.com
 

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us