Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Large Array Issues

Subject: Large Array Issues

From: Kevin

Date: 10 May, 2013 22:43:11

Message: 1 of 3

Hi there,

I'm working with an arbitrary waveform generator with 2 GSa of memory. Each sample is 14 bit but is padded to 16 bit meaning I need to generate a 1 x 2^30 array of int16. I'd like to do this using MATLAB but I'm running into some issues.

The following is a simplified example of what I'm trying to do (copy a 1024 long sample 2^20 times to make a 1 GSa array).

>> clear; tic; blah = int16(ones(1,1024)); repmat(blah,1,2^18); toc
Elapsed time is 3.753619 seconds.
>> clear; tic; blah = int16(ones(1,1024)); repmat(blah,1,2^19); toc
Elapsed time is 7.488963 seconds.
>> clear; tic; blah = int16(ones(1,1024)); repmat(blah,1,2^20); toc

When I get to 2^20 my computer locks up and if I'm lucky I can end the task but usually I have to power cycle my computer. Trying something like 2^30 immediately gives me an out of memory error so I don't think that's the problem.

This is also confirmed by running the memory command.

>> memory
Maximum possible array: 19232 MB (2.017e+10 bytes) *
Memory available for all arrays: 19232 MB (2.017e+10 bytes) *
Memory used by MATLAB: 1619 MB (1.698e+09 bytes)
Physical Memory (RAM): 12286 MB (1.288e+10 bytes)

* Limited by System Memory (physical + swap file) available.

Any advice would be greatly appreciated. I have run the memory profiler and confirmed that when doing repmat(blah,1,2^19), MATLAB is using about 1 Gigabytes of RAM, this is correct as an int16 is 2 bytes and 2^19 is half of a Giga. Since I have 12 GB of RAM it doesn't make sense why it can't handle an array that's 2 GB?

Subject: Large Array Issues

From: Yair Altman

Date: 11 May, 2013 17:46:07

Message: 2 of 3

"Kevin" wrote in message <kmjt5v$t6u$1@newscl01ah.mathworks.com>...
> Hi there,
>
> I'm working with an arbitrary waveform generator with 2 GSa of memory. Each sample is 14 bit but is padded to 16 bit meaning I need to generate a 1 x 2^30 array of int16. I'd like to do this using MATLAB but I'm running into some issues.
>
> The following is a simplified example of what I'm trying to do (copy a 1024 long sample 2^20 times to make a 1 GSa array).
>
> >> clear; tic; blah = int16(ones(1,1024)); repmat(blah,1,2^18); toc
> Elapsed time is 3.753619 seconds.
> >> clear; tic; blah = int16(ones(1,1024)); repmat(blah,1,2^19); toc
> Elapsed time is 7.488963 seconds.
> >> clear; tic; blah = int16(ones(1,1024)); repmat(blah,1,2^20); toc
>
> When I get to 2^20 my computer locks up and if I'm lucky I can end the task but usually I have to power cycle my computer. Trying something like 2^30 immediately gives me an out of memory error so I don't think that's the problem.
>
> This is also confirmed by running the memory command.
>
> >> memory
> Maximum possible array: 19232 MB (2.017e+10 bytes) *
> Memory available for all arrays: 19232 MB (2.017e+10 bytes) *
> Memory used by MATLAB: 1619 MB (1.698e+09 bytes)
> Physical Memory (RAM): 12286 MB (1.288e+10 bytes)
>
> * Limited by System Memory (physical + swap file) available.
>
> Any advice would be greatly appreciated. I have run the memory profiler and confirmed that when doing repmat(blah,1,2^19), MATLAB is using about 1 Gigabytes of RAM, this is correct as an int16 is 2 bytes and 2^19 is half of a Giga. Since I have 12 GB of RAM it doesn't make sense why it can't handle an array that's 2 GB?


Did you try simply blah=ones(1,2^30,'int16') ?

This removes the need for type conversion and repmat. This is important since internally repmat uses index values that are double (=8 bytes) in size, which explains why your computer starts churning and appears to be hung (it's not - it simply pages memory in and out between RAM and disk so much).

Yair Altman
http://UndocumentedMatlab.com
 

Subject: Large Array Issues

From: Bruno Luong

Date: 12 May, 2013 08:13:08

Message: 3 of 3

"Kevin" wrote in message <kmjt5v$t6u$1@newscl01ah.mathworks.com>...

> >> clear; tic; blah = int16(ones(1,1024)); repmat(blah,1,2^20); toc
>
> When I get to 2^20 my computer locks up and if I'm lucky I can end the task but usually I have to power cycle my computer. Trying something like 2^30 immediately gives me an out of memory error so I don't think that's the problem.
an array that's 2 GB?

It happens to me too.

Bruno

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us