Documentation

This is machine translation

Translated by
Mouseover text to see original. Click the button below to return to the English version of the page.

partition

Partition a datastore

Syntax

``subds = partition(ds,n,index)``
``subds = partition(ds,'Files',index)``
``subds = partition(ds,'Files',filename)``

Description

example

````subds = partition(ds,n,index)` partitions datastore `ds` into the number of parts specified by `n` and returns the partition corresponding to the index `index`.```

example

````subds = partition(ds,'Files',index)` partitions the datastore by files and returns the partition corresponding to the file of index `index` in the `Files` property.```
````subds = partition(ds,'Files',filename)` partitions the datastore by files and returns the partition corresponding to the file specified by `filename`.```

Examples

collapse all

Create a datastore from the sample file, `airlinesmall.csv`, which contains tabular data.

```ds = tabularTextDatastore('airlinesmall.csv'); ```

Partition the datastore into three parts.

```subds = partition(ds,3,1) ```
```subds = TabularTextDatastore with properties: Files: { ' ...\matlab\toolbox\matlab\demos\airlinesmall.csv' } FileEncoding: 'UTF-8' ReadVariableNames: true VariableNames: {'Year', 'Month', 'DayofMonth' ... and 26 more} Text Format Properties: NumHeaderLines: 0 Delimiter: ',' RowDelimiter: '\r\n' TreatAsMissing: '' MissingValue: NaN Advanced Text Format Properties: TextscanFormats: {'%f', '%f', '%f' ... and 26 more} TextType: 'char' ExponentCharacters: 'eEdD' CommentStyle: '' Whitespace: ' \b\t' MultipleDelimitersAsOne: false Properties that control the table returned by preview, read, readall: SelectedVariableNames: {'Year', 'Month', 'DayofMonth' ... and 26 more} SelectedFormats: {'%f', '%f', '%f' ... and 26 more} ReadSize: 20000 rows ```

Create a datastore from the sample file, `mapredout.mat`, which is the output file of the `mapreduce` function.

`ds = datastore('mapredout.mat');`

Get the default number of partitions for `ds`.

`n = numpartitions(ds);`

Partition the datastore into the default number of partitions and return the datastore corresponding to the first partition.

`subds = partition(ds,n,1);`

Read the data in `subds`.

```while hasdata(subds) data = read(subds); end```

Create a datastore that contains three image files.

```ds = imageDatastore({'street1.jpg','peppers.png','corn.tif'}) ```
```ds = ImageDatastore with properties: Files: { ' ...\matlab\toolbox\matlab\demos\street1.jpg'; ' ...\matlab\toolbox\matlab\imagesci\peppers.png'; ' ...\matlab\toolbox\matlab\imagesci\corn.tif' } ReadSize: 1 Labels: {} ReadFcn: @readDatastoreImage ```

Partition the datastore by files and return the part corresponding to the second file.

```subds = partition(ds,'Files',2) ```
```subds = ImageDatastore with properties: Files: { ' ...\matlab\toolbox\matlab\imagesci\peppers.png' } ReadSize: 1 Labels: {} ReadFcn: @readDatastoreImage ```

`subds` contains one file.

Create a datastore from the sample file, `mapredout.mat`, which is the output file of the `mapreduce` function.

`ds = datastore('mapredout.mat');`

Partition the datastore into three parts on three workers in a parallel pool.

```n = 3; p = parpool('local',n); parfor ii=1:n subds = partition(ds,n,ii); while hasdata(subds) data = read(subds); end end```

Input Arguments

collapse all

Input datastore. You can use the `datastore` function to create a datastore object from your data.

Number of partitions, specified as a positive integer.

Example: `3`

Data Types: `double`

Index, specified as a positive integer.

Example: `1`

Data Types: `double`

File name, specified as a character vector.

Example: `'file1.csv'`

Example: `'../dir/data/file1.csv'`

Example: `'hdfs://myserver:7867/data/file1.txt'`

Data Types: `char`

Output Arguments

collapse all

Output datastore. The output datastore is of the same type as the input datastore `ds`.