Documentation

This is machine translation

Translated by Microsoft
Mouseover text to see original. Click the button below to return to the English verison of the page.

Note: This page has been translated by MathWorks. Please click here
To view all translated materals including this page, select Japan from the country navigator on the bottom of this page.

partition

Partition a datastore

Syntax

subds = partition(ds,n,index)
subds = partition(ds,'Files',index)
subds = partition(ds,'Files',filename)

Description

example

subds = partition(ds,n,index) partitions datastore ds into the number of parts specified by n and returns the partition corresponding to the index index.

example

subds = partition(ds,'Files',index) partitions the datastore by files and returns the partition corresponding to the file of index index in the Files property.

subds = partition(ds,'Files',filename) partitions the datastore by files and returns the partition corresponding to the file specified by filename.

Examples

collapse all

Create a datastore from the sample file, airlinesmall.csv, which contains tabular data.

ds = tabularTextDatastore('airlinesmall.csv');

Partition the datastore into three parts.

subds = partition(ds,3,1)
subds = 

  TabularTextDatastore with properties:

                      Files: {
                             ' ...\matlab\toolbox\matlab\demos\airlinesmall.csv'
                             }
               FileEncoding: 'UTF-8'
          ReadVariableNames: true
              VariableNames: {'Year', 'Month', 'DayofMonth' ... and 26 more}

  Text Format Properties:
             NumHeaderLines: 0
                  Delimiter: ','
               RowDelimiter: '\r\n'
             TreatAsMissing: ''
               MissingValue: NaN

  Advanced Text Format Properties:
            TextscanFormats: {'%f', '%f', '%f' ... and 26 more}
                   TextType: 'char'
         ExponentCharacters: 'eEdD'
               CommentStyle: ''
                 Whitespace: ' \b\t'
    MultipleDelimitersAsOne: false

  Properties that control the table returned by preview, read, readall:
      SelectedVariableNames: {'Year', 'Month', 'DayofMonth' ... and 26 more}
            SelectedFormats: {'%f', '%f', '%f' ... and 26 more}
                   ReadSize: 20000 rows

Create a datastore from the sample file, mapredout.mat, which is the output file of the mapreduce function.

ds = datastore('mapredout.mat');

Get the default number of partitions for ds.

n = numpartitions(ds);

Partition the datastore into the default number of partitions and return the datastore corresponding to the first partition.

subds = partition(ds,n,1);

Read the data in subds.

while hasdata(subds)
	data = read(subds);
end

Create a datastore that contains three image files.

ds = imageDatastore({'street1.jpg','peppers.png','corn.tif'})
ds = 

  ImageDatastore with properties:

       Files: {
              ' ...\matlab\toolbox\matlab\demos\street1.jpg';
              ' ...\matlab\toolbox\matlab\imagesci\peppers.png';
              ' ...\matlab\toolbox\matlab\imagesci\corn.tif'
              }
    ReadSize: 1
      Labels: {}
     ReadFcn: @readDatastoreImage

Partition the datastore by files and return the part corresponding to the second file.

subds = partition(ds,'Files',2)
subds = 

  ImageDatastore with properties:

       Files: {
              ' ...\matlab\toolbox\matlab\imagesci\peppers.png'
              }
    ReadSize: 1
      Labels: {}
     ReadFcn: @readDatastoreImage

subds contains one file.

Create a datastore from the sample file, mapredout.mat, which is the output file of the mapreduce function.

ds = datastore('mapredout.mat');

Partition the datastore into three parts on three workers in a parallel pool.

n = 3;
p = parpool('local',n);

parfor ii=1:n
    subds = partition(ds,n,ii);
    while hasdata(subds)
        data = read(subds);
    end
end

Input Arguments

collapse all

Input datastore. You can use the datastore function to create a datastore object from your data.

Number of partitions, specified as a positive integer.

Example: 3

Data Types: double

Index, specified as a positive integer.

Example: 1

Data Types: double

File name, specified as a character vector.

Example: 'file1.csv'

Example: '../dir/data/file1.csv'

Example: 'hdfs://myserver:7867/data/file1.txt'

Data Types: char

Output Arguments

collapse all

Output datastore. The output datastore is of the same type as the input datastore ds.

See Also

|

Topics

Introduced in R2015a

Was this topic helpful?