Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
growdata and growdata2 - mathworks problem, still ?

Subject: growdata and growdata2 - mathworks problem, still ?

From: someone

Date: 23 Oct, 2012 13:25:20

Message: 1 of 7

Hi,

This is from 2007:

http://www.mathworks.com/matlabcentral/fileexchange/8334


Is it still necessary/advisable/required to do something like "growdata"
if you want a speed increase and have (very) long increasing arrays ?

Or did Mathworks see this problem and fixed the problem by using a
clever compiler that automatically optimizes its way out of this problem?



I'm asking, because I have a program and it LOOKS to me, like the
problem still persists. So I think maybe I should use growdata now...
What do you think, mr. Steven Lord or others ?

Subject: growdata and growdata2 - mathworks problem, still ?

From: Steven_Lord

Date: 23 Oct, 2012 13:41:31

Message: 2 of 7



"someone" <newsboost@gmail.com> wrote in message
news:k665s0$eg2$1@dont-email.me...
> Hi,
>
> This is from 2007:
>
> http://www.mathworks.com/matlabcentral/fileexchange/8334
>
>
> Is it still necessary/advisable/required to do something like "growdata"
> if you want a speed increase and have (very) long increasing arrays ?

It is impossible to answer that question without knowing the specific way
your code is written.

> Or did Mathworks see this problem and fixed the problem by using a clever
> compiler that automatically optimizes its way out of this problem?

In release R2011a there is an entry in the Release Notes that reads:

"This release improves the performance of growing an array in the trailing
dimension if that array has not been preallocated."

> I'm asking, because I have a program and it LOOKS to me, like the problem
> still persists. So I think maybe I should use growdata now... What do you
> think, mr. Steven Lord or others ?

Try to preallocate first. If that doesn't improve the performance, or if you
can't do that for some other reason, send a section of code for which you
believe you'll need to use GROWDATA on to Technical Support so they can
investigate if there's additional improvement the development staff can make
to how arrays are grown in MATLAB.

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

Subject: growdata and growdata2 - mathworks problem, still ?

From: dpb

Date: 23 Oct, 2012 13:42:59

Message: 3 of 7

On 10/23/2012 8:25 AM, someone wrote:
> Hi,
>
> This is from 2007:
>
> http://www.mathworks.com/matlabcentral/fileexchange/8334
>
>
> Is it still necessary/advisable/required to do something like "growdata"
> if you want a speed increase and have (very) long increasing arrays ?
>
> Or did Mathworks see this problem and fixed the problem by using a
> clever compiler that automatically optimizes its way out of this problem?
...

My suggestion if you can't figure out how to preallocate is "try it and
see".

Better JIT compilation can fix many things but poor coding practice or
algorithms that require much general rearranging of the source code to
handle memory allocation (and particularly _re_allocation aren't likely
to be solvable that way.

--

Subject: growdata and growdata2 - mathworks problem, still ?

From: Bruno Luong

Date: 23 Oct, 2012 17:21:08

Message: 4 of 7

> I'm asking, because I have a program and it LOOKS to me, like the
> problem still persists. So I think maybe I should use growdata now...

Growing data issue exists in any language (C, C++) not only in MATLAB, and IMO there is no universal way to carry such task smartly.

My own preference is automatically doubling the array each time there is an overflowed assigment then removing the exceed tail at the end. I usually do it on the fly within a loop without calling external function (such as growdata), since calling function inside the loop is always costly.

Bruno

Subject: growdata and growdata2 - mathworks problem, still ?

From: dpb

Date: 23 Oct, 2012 17:36:54

Message: 5 of 7

On 10/23/2012 9:48 AM, someone wrote:
...

> I have a lot of calls to ODE45 and then I save a lot of things to some
> global growing arrays, both growing vectors, growing matrices and
> growing cell arrays. After the timeloop ends, I investigate the results
> using data from inside ODE45.
>
...

> > Try to preallocate first. If that doesn't improve the performance, or if
>
> But I don't know how big I should preallocate to before, using ODE45.
> It's impossible to say.
...
> But *I* think that the matlab compiler should be clever enough to see
> and think that:
>
> "hey! there's an array that keeps growing bigger and bigger. Even though
> the user didn't ask me, then: Let me use the brain my developers at
> mathworks gave me and then maybe don't preallocate a single array
> element at a time, but preallocate some extra elements based on a clever
> guess I've learned - to save time allocating data the whole time..."
...

Are you talking about memory internal to ODE45 or your application memory?

If the latter, _I'd_ think then if you're doing this kind of thing more
than once (as you apparently are) you would fairly quickly learn what
kind of memory growth to expect and could do the above pretty yourself
specifically for your own situation.

Let your application use the brain you build into it to not allocate an
element at a time... :)

I can see as Steven L says some special cases where the JIT
compiler/code analyzer can do some special things but in general if you
write code that isn't clever in it's memory usage I think it's both
unreasonable and unfair to expect the compiler/TMW to take care of it
for you automagically.

--

Subject: growdata and growdata2 - mathworks problem, still ?

From: someone

Date: 24 Oct, 2012 08:23:32

Message: 6 of 7

On 2012-10-23 19:36, dpb wrote:> On 10/23/2012 9:48 AM, someone wrote:
 > ...
 >
 >> I have a lot of calls to ODE45 and then I save a lot of things to some
 >> global growing arrays, both growing vectors, growing matrices and
 >> growing cell arrays. After the timeloop ends, I investigate the results
 >> using data from inside ODE45.
 >>
 > ...
 >
 >> > Try to preallocate first. If that doesn't improve the performance,
 >> or if
 >>
 >> But I don't know how big I should preallocate to before, using ODE45.
 >> It's impossible to say.
 > ...
 >> But *I* think that the matlab compiler should be clever enough to see
 >> and think that:
 >>
 >> "hey! there's an array that keeps growing bigger and bigger. Even though
 >> the user didn't ask me, then: Let me use the brain my developers at
 >> mathworks gave me and then maybe don't preallocate a single array
 >> element at a time, but preallocate some extra elements based on a clever
 >> guess I've learned - to save time allocating data the whole time..."
 > ...
 >
 > Are you talking about memory internal to ODE45 or your application
memory?

Ofcourse my own application memory. What ODE45 does is not my concern.

 > If the latter, _I'd_ think then if you're doing this kind of thing more
 > than once (as you apparently are) you would fairly quickly learn what
 > kind of memory growth to expect and could do the above pretty yourself
 > specifically for your own situation.

Yes, I can do it myself, using growdata. That was my idea.

 > Let your application use the brain you build into it to not allocate an
 > element at a time... :)

Yes, exactly. Using growdata, was my idea...

 > I can see as Steven L says some special cases where the JIT
 > compiler/code analyzer can do some special things but in general if you
 > write code that isn't clever in it's memory usage I think it's both
 > unreasonable and unfair to expect the compiler/TMW to take care of it
 > for you automagically.

Don't know about that. If it don't take care of it, I'll try to use
growdata. If it does, then it can see where the loops are and it already
today comes up with a lot of warnings (yellow marks to the right in the
editor) about numerous things.

In any case, I think I should use growdata.

Subject: growdata and growdata2 - mathworks problem, still ?

From: someone

Date: 24 Oct, 2012 08:24:18

Message: 7 of 7

On 2012-10-23 19:21, Bruno Luong wrote:
>> I'm asking, because I have a program and it LOOKS to me, like the
>> problem still persists. So I think maybe I should use growdata now...
>
> Growing data issue exists in any language (C, C++) not only in MATLAB,
> and IMO there is no universal way to carry such task smartly.

Growdata is a smart way, right?

> My own preference is automatically doubling the array each time there is
> an overflowed assigment then removing the exceed tail at the end. I
> usually do it on the fly within a loop without calling external function
> (such as growdata), since calling function inside the loop is always
> costly.

I think I'll begin to use growdata...

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us