Documentation

This is machine translation

Translated by Microsoft
Mouseover text to see original. Click the button below to return to the English verison of the page.

Note: This page has been translated by MathWorks. Please click here
To view all translated materals including this page, select Japan from the country navigator on the bottom of this page.

TallDatastore

Datastore for checkpointing tall arrays

Description

TallDatastore objects are for recreating tall arrays from binary files written to disk by the write function. You can use the object to recreate the original tall array, or you can access and manage the data by specifying TallDataStore properties and using the object functions.

Creation

Create TallDatastore objects using the datastore function. For example, tds = datastore(location,'Type','tall') creates a datastore from a collection of files specified by location.

Properties

expand all

Files included in the datastore, resolved as character vector or a cell array of character vectors, where each character vector is a full path to a file. The location argument of the datastore function defines the Files property when the datastore is created.

The files must be either MAT-files or Sequence files generated by the write function.

Example: {'C:\dir\data\file1.ext';'C:\dir\data\file2.ext'}

Data Types: char | cell

File type, specified as either 'mat' for MAT-files or 'seq' for sequence files. By default, the type of file in the provided location determines the FileType.

Maximum number of data rows to read in a call to the read or preview functions, specified as a positive integer. When the datastore function creates a TallDatastore, it determines and assigns the best possible value for ReadSize.

Object Functions

hasdataDetermine if data is available to read
numpartitionsNumber of datastore partitions
partitionPartition a datastore
previewSubset of data in datastore
readRead data in datastore
readallRead all data in datastore
resetReset datastore to initial state

Examples

expand all

Use TallDatastore objects to reconstruct tall arrays directly from files on disk rather than re-executing all of the commands that produced the tall array. Create a tall array and save it to disk using write function. Retrieve the tall array using datastore and then convert it back to tall.

Create a simple tall double.

t = tall(rand(500,1))
t =

  500×1 tall double column vector

    0.8147
    0.9058
    0.1270
    0.9134
    0.6324
    0.0975
    0.2785
    0.5469
    :
    :

Save the results to a new folder named ExampleData on the C:\ disk. (You might want to specify a different write location, especially if you are not using a Windows® computer.)

location = 'C:\ExampleData';
write(location, t);
Writing tall data to folder C:\ExampleData
Evaluating tall expression using the Parallel Pool 'local':
Evaluation completed in 0 sec

To recover the tall array that was written to disk, first create a new datastore that references the same directory. Then convert the datastore into a tall array.

tds = datastore(location);
t1 = tall(tds)
t1 =

  M×1 tall double column vector

    0.8147
    0.9058
    0.1270
    0.9134
    0.6324
    0.0975
    0.2785
    0.5469
    :
    :

See Also

|

Introduced in R2016b

Was this topic helpful?