Taking the Inverse of a Tall Array To Solve a Linear System of Equations

4 views (last 30 days)
Hi, I'm working with some large data sets and have begun using tall arrays for the first time. I was happy to hear that the inverse function (\) was available for tall arrays, but am having some trouble on implementation. Below is a none-working example (you may need to set your own output directory), if F is tall or not doesn't change anything.
function [SOL] = MinWorkExample
outDIR = 'M:\project\MATLAB Folder\PhD\TESTAREA\InterpolationForPaper\PHFtall';
fileIDX = 1;
rowsPerFile = 1e3;
rowsWritten = 0;
totalRows = 1e5;
while rowsWritten<totalRows
rowsToWrite = min(rowsPerFile, totalRows-rowsWritten);
datas = rand(1,totalRows);
fname = fullfile(outDIR,sprintf('datas_%05d.mat',fileIDX));
save(fname,'datas');
fileIDX = 1 + fileIDX;
rowsWritten = rowsToWrite + rowsWritten;
end
ds = fileDatastore(fullfile(outDIR, '*.mat'), ...
'ReadFcn', @(fname) getfield(load(fname), 'datas'), ...
'UniformRead', true);
CONMAT = tall(ds);
% Either F does not work.
% F = rand(totalRows,1);
F = tall(rand(totalRows,1));
SOL = CONMAT\F;

Answers (1)

Ayden Clay
Ayden Clay on 23 May 2020
So! I'm a collossal idiot. The error note was incredibly well written, and described my exact problem. Turns out that if I repeat the process but write in columns rather than rows, we're all good.
In the same nomenclature as the question, here is a working version:
function [SOL] = MinWorkExample
outDIR = '\PHFtall';
fileIDX = 1;
ColsPerFile = 1e3;
ColsWritten = 0;
totalCols = 1e5;
while ColsWritten<totalCols
colsToWrite = min(ColsPerFile, totalCols-ColsWritten);
datas = rand(totalCols,1);
fname = fullfile(outDIR,sprintf('datas_%05d.mat',fileIDX));
save(fname,'datas');
fileIDX = 1 + fileIDX;
ColsWritten = colsToWrite + ColsWritten;
end
ds = fileDatastore(fullfile(outDIR, '*.mat'), ...
'ReadFcn', @(fname) getfield(load(fname), 'datas'), ...
'UniformRead', true);
CONMAT = tall(ds);
F = tall(rand(1,totalCols));
SOL = CONMAT\F;
  1 Comment
Ayden Clay
Ayden Clay on 23 May 2020
Edited: Ayden Clay on 23 May 2020
Nevermind. This causes a further issue in that the data is vertically concatenated, producing a large Nx1 vector. Still working on a fix.

Sign in to comment.

Categories

Find more on Large Files and Big Data in Help Center and File Exchange

Products


Release

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!