File Exchange

image thumbnail

Handling Large Data Sets Efficiently in MATLAB

version 1.8.0.1 (1.23 MB) by Stuart McGarrity
Handling Large Data Sets Efficiently in MATLAB®

2 Downloads

Updated 01 Sep 2016

View Version History

View License

Editor's Note: This file was a File Exchange Pick of the Week

Example code from "Handling Large Data Sets Efficiently in MATLAB " webinar describing strategies for handling large amounts of data in MATLAB and avoiding "out-of-memory" errors. It will provide you with an understanding of the causes of memory limitations in MATLAB and a set of techniques to increase the available memory in MATLAB. It will also show techniques for minimizing memory usage in MATLAB while accessing, storing, processing, and plotting data.

Specific topics include:
* Understanding the maximum size of an array and the workspace in MATLAB
* Using undocumented features to show you the available memory in MATLAB
* Setting the 3GB switch under Windows XP to get 1GB more memory for MATLAB
* Using textscan to read large text files and memory mapping feature to read large binary files

Cite As

Stuart McGarrity (2021). Handling Large Data Sets Efficiently in MATLAB (https://www.mathworks.com/matlabcentral/fileexchange/9060-handling-large-data-sets-efficiently-in-matlab), MATLAB Central File Exchange. Retrieved .

Comments and Ratings (22)

Anthony

Good, but the presentation stops after processing and doesn't get to plotting.

Stuart McGarrity

Please watch the webinar for which this is the code. The first 40 min, as you correctly say, focuses on maximizing the available memory for MATLAB. The latter 25 min focuses on minimizing MATLAB's memory usage, processing a large data set in a few ways. BTW MATLAB's out of memory errors are not directly affected by available RAM.

Aldemar

Okay... the title says: "Handling Large Data Sets Efficiently in MATLAB".
Actually, it just shows information on how to "Handle" the system memory in order to provide more RAM for MATLAB to work with.
Not even one word is spent on an arbitrary "Data Set".
Nevertheless, this guide is useful for the above.

Conclusion:
Good work, wrong topic.

Suggestion:
Replace the title by something like "Providing MATLAB with more RAM in order not to run out of memory" or something similar.

Danila

Chamane

Helpfull and really interesting

Danila

Steen Johansen

Very interesting. This should be a part of the standard documentation.
I only wish I had found this when it was posted.

Lin Dong

great

Lizhu Qi

Very helpful. I really appreciate your work.

Gideon Alon

Very good and useful information.

Chinara Griffith

The information here is timely and accurate. I learnt a lot from this presentation.

Kurt Sanger

Video for setting 3GB Switch does not play. What do I do now?

mohamed el_amawy

anu puri

Roland May

syaidin dien

me again

setting the 3GB switch too stupid slow/big download/view

Lorena patiño

Parul Gupta

I found this webinar tremendously useful - there were lots of tips and tricks I didn't know that can help better memory management

hamed mehdizadeh

Gurk Peter

Gives some information on how to resolve out of memory errors, but all the tips are more or less straightforward for anyone knowing basic things about computers and operating systems.

John D'Errico

A superb review of memory management for those really
big problems. He explains what the limits are, why those
limits exist, and what to do if its possible to do anything.
Its well worth your time to read.

MATLAB Release Compatibility
Created with R14SP2
Compatible with any release
Platform Compatibility
Windows macOS Linux

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!