MATLAB Examples

Using MDF Files Via MDF Datastore

This example shows you how to use the MDF Datastore feature of Vehicle Network Toolbox to quickly and efficiently process a data set spread across a collection of multiple MDF files. This workflow is also valuable when there is too much data to fit into available memory.

Contents

Access MDF Files in a Datastore

Find the collection of MDF files representing logged information from multiple test sequences.

dir('File*.mf4')
File01.mf4  File02.mf4  File03.mf4  File04.mf4  File05.mf4  

Create an MDF Datastore

You create an MDF datastore by selecting a folder location containing a collection of MDF files. In this case, target all files in the current working directory.

mds = mdfDatastore(pwd)
mds = 

  MDFDatastore with properties:

                         Files: {
                                ' ...\home\jpyle\documents\MATLAB\examples\vnt-ex10761765\File01.mf4';
                                ' ...\home\jpyle\documents\MATLAB\examples\vnt-ex10761765\File02.mf4';
                                ' ...\home\jpyle\documents\MATLAB\examples\vnt-ex10761765\File03.mf4'
                                 ... and 2 more
                                }
                 ChannelGroups: 
                                ChannelGroupNumber    AcquisitionName        Comment        ... and 4 more columns
                                __________________    _______________    _______________        

                                1                     ''                 'Integer Types'      
                                2                     ''                 'Float Types'        


                      Channels: 
                                ChannelGroupNumber                ChannelName                 DisplayName    ... and 9 more columns
                                __________________    ____________________________________    ___________        

                                1                     'Sigend_Int16_LE_Offset_32'             ''               
                                1                     'Unsigend_UInt32_LE_Master_Offset_0'    ''               
                                2                     'Float_32_LE_Offset_64'                 ''               

                                ... and 1 more rows
          SelectedChannelNames: {
                                'Sigend_Int16_LE_Offset_32';
                                'Unsigend_UInt32_LE_Master_Offset_0'
                                }
    SelectedChannelGroupNumber: 1
                      ReadSize: 'file'

Configure the MDF Datastore

Multiple options allow control of what data is read from the MDF files and how the reads are performed. In this case, the first channel group is used by default. Note that only one channel group may be selected by the datastore at a time. You can also specify certain channels within the selected channel group to read. In this case, all channels are read by default.

mds.SelectedChannelGroupNumber
mds.SelectedChannelNames
ans =

     1


ans = 

  2×1 string array

    "Sigend_Int16_LE_Offset_32"
    "Unsigend_UInt32_LE_Master_Offset_0"

MDF Datastore preview

Using the preview function, you can obtain a quick view of the data available in the file set. Preview always returns up to eight data points from the first file in the datastore.

preview(mds)
ans =

  8×2 timetable

     Time    Sigend_Int16_LE_Offset_32    Unsigend_UInt32_LE_Master_Offset_0
    _____    _________________________    __________________________________

    0 sec    0                            0                                 
    1 sec    1                            1                                 
    2 sec    2                            2                                 
    3 sec    3                            3                                 
    4 sec    4                            4                                 
    5 sec    5                            5                                 
    6 sec    6                            6                                 
    7 sec    7                            7                                 

Using MDF Datastore readall

You can use the readall function to read the entire contents of data in a single call. This is an efficient way to read from many files when the data set fits into available memory. Aftering readall, the datastore resets to the beginning of the data set.

data = readall(mds);
data(1:5,:)
ans =

  5×2 timetable

     Time    Sigend_Int16_LE_Offset_32    Unsigend_UInt32_LE_Master_Offset_0
    _____    _________________________    __________________________________

    0 sec    0                            0                                 
    1 sec    1                            1                                 
    2 sec    2                            2                                 
    3 sec    3                            3                                 
    4 sec    4                            4                                 

Using MDF Datastore read

You can use the read function to obtain data from the file set. By default, reads from the MDF datastore will read an entire file's worth of data per call. The power of a datastore comes from reading through multiple files sequentially within the file set. As you read, the datastore automatically bridges from one file to the next until all data from all files is read.

for ii = 1:3
    data = read(mds);
    whos('data')
    data(1:5,:)
end
  Name          Size             Bytes  Class        Attributes

  data      10000x2             241767  timetable              


ans =

  5×2 timetable

     Time    Sigend_Int16_LE_Offset_32    Unsigend_UInt32_LE_Master_Offset_0
    _____    _________________________    __________________________________

    0 sec    0                            0                                 
    1 sec    1                            1                                 
    2 sec    2                            2                                 
    3 sec    3                            3                                 
    4 sec    4                            4                                 

  Name          Size             Bytes  Class        Attributes

  data      10000x2             241767  timetable              


ans =

  5×2 timetable

     Time    Sigend_Int16_LE_Offset_32    Unsigend_UInt32_LE_Master_Offset_0
    _____    _________________________    __________________________________

    0 sec    0                            0                                 
    1 sec    1                            1                                 
    2 sec    2                            2                                 
    3 sec    3                            3                                 
    4 sec    4                            4                                 

  Name          Size             Bytes  Class        Attributes

  data      10000x2             241767  timetable              


ans =

  5×2 timetable

     Time    Sigend_Int16_LE_Offset_32    Unsigend_UInt32_LE_Master_Offset_0
    _____    _________________________    __________________________________

    0 sec    0                            0                                 
    1 sec    1                            1                                 
    2 sec    2                            2                                 
    3 sec    3                            3                                 
    4 sec    4                            4                                 

MDF Datastore reset

At any time, you can call the reset function to start over again at the beginning of the data set.

reset(mds)

Configure MDF Datastore read By Data Record

You can use the ReadSize property to specify how much data to read on each call. ReadSize can be specified as a numeric value to read a fixed number of data points. ReadSize lets you control how much data is loaded into memory when you have a data set larger than available memory. We recommend using custom read sizes that are small enough to fit in memory, but still as large as possible to reduce processing overhead and improve performance.

mds.ReadSize = 5
for ii = 1:3
    data = read(mds)
end
mds = 

  MDFDatastore with properties:

                         Files: {
                                ' ...\home\jpyle\documents\MATLAB\examples\vnt-ex10761765\File01.mf4';
                                ' ...\home\jpyle\documents\MATLAB\examples\vnt-ex10761765\File02.mf4';
                                ' ...\home\jpyle\documents\MATLAB\examples\vnt-ex10761765\File03.mf4'
                                 ... and 2 more
                                }
                 ChannelGroups: 
                                ChannelGroupNumber    AcquisitionName        Comment        ... and 4 more columns
                                __________________    _______________    _______________        

                                1                     ''                 'Integer Types'      
                                2                     ''                 'Float Types'        


                      Channels: 
                                ChannelGroupNumber                ChannelName                 DisplayName    ... and 9 more columns
                                __________________    ____________________________________    ___________        

                                1                     'Sigend_Int16_LE_Offset_32'             ''               
                                1                     'Unsigend_UInt32_LE_Master_Offset_0'    ''               
                                2                     'Float_32_LE_Offset_64'                 ''               

                                ... and 1 more rows
          SelectedChannelNames: {
                                'Sigend_Int16_LE_Offset_32';
                                'Unsigend_UInt32_LE_Master_Offset_0'
                                }
    SelectedChannelGroupNumber: 1
                      ReadSize: 5


data =

  5×2 timetable

     Time    Sigend_Int16_LE_Offset_32    Unsigend_UInt32_LE_Master_Offset_0
    _____    _________________________    __________________________________

    0 sec    0                            0                                 
    1 sec    1                            1                                 
    2 sec    2                            2                                 
    3 sec    3                            3                                 
    4 sec    4                            4                                 


data =

  5×2 timetable

     Time    Sigend_Int16_LE_Offset_32    Unsigend_UInt32_LE_Master_Offset_0
    _____    _________________________    __________________________________

    5 sec    5                            5                                 
    6 sec    6                            6                                 
    7 sec    7                            7                                 
    8 sec    8                            8                                 
    9 sec    9                            9                                 


data =

  5×2 timetable

     Time     Sigend_Int16_LE_Offset_32    Unsigend_UInt32_LE_Master_Offset_0
    ______    _________________________    __________________________________

    10 sec    10                           10                                
    11 sec    11                           11                                
    12 sec    12                           12                                
    13 sec    13                           13                                
    14 sec    14                           14                                

Configure MDF Datastore read By Data Timestamp

You can also specify ReadSize as a duration to read data points by elapsed time. Note that when the read type is changed, the datastore resets to the beginning of the data set.

mds.ReadSize = seconds(5)
for ii = 1:3
    data = read(mds)
end
mds = 

  MDFDatastore with properties:

                         Files: {
                                ' ...\home\jpyle\documents\MATLAB\examples\vnt-ex10761765\File01.mf4';
                                ' ...\home\jpyle\documents\MATLAB\examples\vnt-ex10761765\File02.mf4';
                                ' ...\home\jpyle\documents\MATLAB\examples\vnt-ex10761765\File03.mf4'
                                 ... and 2 more
                                }
                 ChannelGroups: 
                                ChannelGroupNumber    AcquisitionName        Comment        ... and 4 more columns
                                __________________    _______________    _______________        

                                1                     ''                 'Integer Types'      
                                2                     ''                 'Float Types'        


                      Channels: 
                                ChannelGroupNumber                ChannelName                 DisplayName    ... and 9 more columns
                                __________________    ____________________________________    ___________        

                                1                     'Sigend_Int16_LE_Offset_32'             ''               
                                1                     'Unsigend_UInt32_LE_Master_Offset_0'    ''               
                                2                     'Float_32_LE_Offset_64'                 ''               

                                ... and 1 more rows
          SelectedChannelNames: {
                                'Sigend_Int16_LE_Offset_32';
                                'Unsigend_UInt32_LE_Master_Offset_0'
                                }
    SelectedChannelGroupNumber: 1
                      ReadSize: 5 sec


data =

  5×2 timetable

     Time    Sigend_Int16_LE_Offset_32    Unsigend_UInt32_LE_Master_Offset_0
    _____    _________________________    __________________________________

    0 sec    0                            0                                 
    1 sec    1                            1                                 
    2 sec    2                            2                                 
    3 sec    3                            3                                 
    4 sec    4                            4                                 


data =

  5×2 timetable

     Time    Sigend_Int16_LE_Offset_32    Unsigend_UInt32_LE_Master_Offset_0
    _____    _________________________    __________________________________

    5 sec    5                            5                                 
    6 sec    6                            6                                 
    7 sec    7                            7                                 
    8 sec    8                            8                                 
    9 sec    9                            9                                 


data =

  5×2 timetable

     Time     Sigend_Int16_LE_Offset_32    Unsigend_UInt32_LE_Master_Offset_0
    ______    _________________________    __________________________________

    10 sec    10                           10                                
    11 sec    11                           11                                
    12 sec    12                           12                                
    13 sec    13                           13                                
    14 sec    14                           14