Using datastore for a matrix

8 views (last 30 days)
Chuck Noise
Chuck Noise on 9 Sep 2023
Commented: Catalytic on 9 Sep 2023
Hi
I am trying to set up a datastore in order to train a feedforward neural network with a trainingset that is too large for my workspace.
I have a training set, X, that is 80002x10000 which I stored in a .mat file called ¨TrainingData'. I am now trying to set up a datastore so I can load mini batches and train on those, but I can't get it working.
I have tried datastore and arrayDatastore.
datastore cannot determine the datastore type for the specified location and arrayDatastore just output the name of the .mat-file whenever i read, i.e.
>> read(ds)
ans =
1×1 cell array
{'X.mat'}
What am I doing wrong here?
EDIT
I just read that the arrayDatastore is only for in-memory data which makes it useless in my case
Thanks

Accepted Answer

Catalytic
Catalytic on 9 Sep 2023
If you're not going to store your data in memory, then the training samples have to be split up into separate .mat files. You could then use a fileDatastore to read from and manage them.

More Answers (0)

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!