Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Call by reference / references

Subject: Call by reference / references

From: Philipp

Date: 8 Nov, 2007 13:11:46

Message: 1 of 9

Hi,

since I want to do 'extended' programming in Matlab, I'm
missing an important feature that all common programming
languages offer: Call by reference and references (or
pointer). There are two situations, where I don't know how
to handle it in Matlab except the brute force (and slow)
way:

1) I want a function that operates and changes a data
structure. For example, lets have a cell array <map>
storing mapEntry structs which each struct having a .key
and .value entry. Now I want to write a function that adds
a key/value pair to the map cell array if it is not there
otherwise add it. In Matlab the function call would be like:
map = {};
map = addEntry( map, 'foo', 'test' )
Now the question arises, what happens if map gets very
large ? As far as I understood Matlab, there are now three
copy operations involved. (two there and one in the
functions to copy the map coming in to the map going out).

This is just an example there are other examples, too.

2) I have two lists, lets say a list of bananas and a list
of juices. Every banana has certain attributes and every
juice as well. More than that, every juice has a list of
bananas it is made of. I see only one possibility: every
banana is given a unique ID and every juice stores a list
of these IDs. When I have a juice now and want to access
the attributes of its bananas, I get the ID and then I have
to search my banana list for this ID. See, this is very
slow. I want the juice to refer to the banana (like C
pointers or Java references).

Thanks a lot !

Subject: Call by reference / references

From: Peter Boettcher

Date: 8 Nov, 2007 14:43:52

Message: 2 of 9

"Philipp " <philipp.maurer@ocag.ch> writes:

> 1) I want a function that operates and changes a data
> structure. For example, lets have a cell array <map>
> storing mapEntry structs which each struct having a .key
> and .value entry. Now I want to write a function that adds
> a key/value pair to the map cell array if it is not there
> otherwise add it. In Matlab the function call would be like:
> map = {};
> map = addEntry( map, 'foo', 'test' )
> Now the question arises, what happens if map gets very
> large ? As far as I understood Matlab, there are now three
> copy operations involved. (two there and one in the
> functions to copy the map coming in to the map going out).

Copying only happens when you modify a piece of data. So if addEntry
had only a line that returned map, there would be NO data copying. In
each case, a smart reference would happen instead.

When you DO modify an array, that is when it is "unshared", and the
allocation and copy occurs. But even here, the copy is not necessarily
a "deep" copy. If you modify a container object (a cell or a struct),
only the top level list of pointers is copied. Each "contained" object
is now shared between two cell arrays. When you modify one of those
objects, its top level is copied. And so on.

So in your case, adding a single cell element to map, there is exactly
one allocation and one memcpy. This memcpy is exactly
length(map)*sizeof(void*), that is, one pointer for every element in the
cell array.

Then the MATLAB accelerator comes into play. It is able to optimize
some functions of the form you just listed (x = fun(x)), by performing
the operation in place. I don't know exactly what conditions the
function must meet for this to happen. But certainly it won't work if
you are expanding the size of your variable. You should definitely be
preallocating map to be big enough, and keep track of the number of
current entries with another variable.

> 2) I have two lists, lets say a list of bananas and a list
> of juices. Every banana has certain attributes and every
> juice as well. More than that, every juice has a list of
> bananas it is made of. I see only one possibility: every
> banana is given a unique ID and every juice stores a list
> of these IDs. When I have a juice now and want to access
> the attributes of its bananas, I get the ID and then I have
> to search my banana list for this ID. See, this is very
> slow. I want the juice to refer to the banana (like C
> pointers or Java references).

Instead of storing a list of unique banana identifiers and searching for
them, store a list of banana indices. This will end up
(algorithmically, anyway) about as complex as a pointer list.

for i=1:length(juice(j).banana_list)
  banana_idx = juice(j).banana_list(i);
  current_banana = bananas(banana_idx);
end


-Peter

Subject: Call by reference / references

From: Philipp

Date: 8 Nov, 2007 14:53:28

Message: 3 of 9

Peter Boettcher <boettcher@ll.mit.edu> wrote in message
<muy4pfwlrvb.fsf@G99-Boettcher.llan.ll.mit.edu>...
> "Philipp " <philipp.maurer@ocag.ch> writes:
>
> > 1) I want a function that operates and changes a data
> > structure. For example, lets have a cell array <map>
> > storing mapEntry structs which each struct having
a .key
> > and .value entry. Now I want to write a function that
adds
> > a key/value pair to the map cell array if it is not
there
> > otherwise add it. In Matlab the function call would be
like:
> > map = {};
> > map = addEntry( map, 'foo', 'test' )
> > Now the question arises, what happens if map gets very
> > large ? As far as I understood Matlab, there are now
three
> > copy operations involved. (two there and one in the
> > functions to copy the map coming in to the map going
out).
>
> Copying only happens when you modify a piece of data. So
if addEntry
> had only a line that returned map, there would be NO data
copying. In
> each case, a smart reference would happen instead.
>
> When you DO modify an array, that is when it
is "unshared", and the
> allocation and copy occurs. But even here, the copy is
not necessarily
> a "deep" copy. If you modify a container object (a cell
or a struct),
> only the top level list of pointers is copied.
Each "contained" object
> is now shared between two cell arrays. When you modify
one of those
> objects, its top level is copied. And so on.
>
> So in your case, adding a single cell element to map,
there is exactly
> one allocation and one memcpy. This memcpy is exactly
> length(map)*sizeof(void*), that is, one pointer for every
element in the
> cell array.
>
> Then the MATLAB accelerator comes into play. It is able
to optimize
> some functions of the form you just listed (x = fun(x)),
by performing
> the operation in place. I don't know exactly what
conditions the
> function must meet for this to happen. But certainly it
won't work if
> you are expanding the size of your variable. You should
definitely be
> preallocating map to be big enough, and keep track of the
number of
> current entries with another variable.
>
> > 2) I have two lists, lets say a list of bananas and a
list
> > of juices. Every banana has certain attributes and
every
> > juice as well. More than that, every juice has a list
of
> > bananas it is made of. I see only one possibility:
every
> > banana is given a unique ID and every juice stores a
list
> > of these IDs. When I have a juice now and want to
access
> > the attributes of its bananas, I get the ID and then I
have
> > to search my banana list for this ID. See, this is very
> > slow. I want the juice to refer to the banana (like C
> > pointers or Java references).
>
> Instead of storing a list of unique banana identifiers
and searching for
> them, store a list of banana indices. This will end up
> (algorithmically, anyway) about as complex as a pointer
list.
>
> for i=1:length(juice(j).banana_list)
> banana_idx = juice(j).banana_list(i);
> current_banana = bananas(banana_idx);
> end
>
>
> -Peter

a) Ok, I didn't know how Matlab handles parameters in
function calls (by value or by reference). Anyway, is there
any documentation when Matlab copies an object and when it
is a deep copy then ? How do I know if Matlab works with
pointers or with real objects ?

b) ok, the problem is that the indices might change in my
specific case, so there is too much effort to keep the link
via indices. That's why I used unique ids.

Thanks so far !

Subject: Call by reference / references

From: Loren Shure

Date: 9 Nov, 2007 17:44:20

Message: 4 of 9

In article <muy4pfwlrvb.fsf@G99-Boettcher.llan.ll.mit.edu>,
boettcher@ll.mit.edu says...
> "Philipp " <philipp.maurer@ocag.ch> writes:
>
> > 1) I want a function that operates and changes a data
> > structure. For example, lets have a cell array <map>
> > storing mapEntry structs which each struct having a .key
> > and .value entry. Now I want to write a function that adds
> > a key/value pair to the map cell array if it is not there
> > otherwise add it. In Matlab the function call would be like:
> > map = {};
> > map = addEntry( map, 'foo', 'test' )
> > Now the question arises, what happens if map gets very
> > large ? As far as I understood Matlab, there are now three
> > copy operations involved. (two there and one in the
> > functions to copy the map coming in to the map going out).
>
> Copying only happens when you modify a piece of data. So if addEntry
> had only a line that returned map, there would be NO data copying. In
> each case, a smart reference would happen instead.
>
> When you DO modify an array, that is when it is "unshared", and the
> allocation and copy occurs. But even here, the copy is not necessarily
> a "deep" copy. If you modify a container object (a cell or a struct),
> only the top level list of pointers is copied. Each "contained" object
> is now shared between two cell arrays. When you modify one of those
> objects, its top level is copied. And so on.
>
> So in your case, adding a single cell element to map, there is exactly
> one allocation and one memcpy. This memcpy is exactly
> length(map)*sizeof(void*), that is, one pointer for every element in the
> cell array.
>
> Then the MATLAB accelerator comes into play. It is able to optimize
> some functions of the form you just listed (x = fun(x)), by performing
> the operation in place. I don't know exactly what conditions the
> function must meet for this to happen. But certainly it won't work if
> you are expanding the size of your variable. You should definitely be
> preallocating map to be big enough, and keep track of the number of
> current entries with another variable.
>
> > 2) I have two lists, lets say a list of bananas and a list
> > of juices. Every banana has certain attributes and every
> > juice as well. More than that, every juice has a list of
> > bananas it is made of. I see only one possibility: every
> > banana is given a unique ID and every juice stores a list
> > of these IDs. When I have a juice now and want to access
> > the attributes of its bananas, I get the ID and then I have
> > to search my banana list for this ID. See, this is very
> > slow. I want the juice to refer to the banana (like C
> > pointers or Java references).
>
> Instead of storing a list of unique banana identifiers and searching for
> them, store a list of banana indices. This will end up
> (algorithmically, anyway) about as complex as a pointer list.
>
> for i=1:length(juice(j).banana_list)
> banana_idx = juice(j).banana_list(i);
> current_banana = bananas(banana_idx);
> end
>
>
> -Peter
>

also consider using nested functions for some of your purposes. There
is a single instance of the data available to the nested function
workspace and it can be updated by calling the nested function (many
examples in the doc).

also, look at my blog (reference below) to see discussions of memory
copying.

--
Loren
http://blogs.mathworks.com/loren/

Subject: Call by reference / references

From: Aaron Schurger

Date: 5 Mar, 2010 15:11:26

Message: 5 of 9

Dear Loren,
The burning question I have is, why not just allow the programmer to force call-by-reference is desired? I can't think of a good reason not to have this feature in the language (although you may know of the reason). I am constantly up against this problem, and constantly getting "out of memory errors" in spite of my thorough knowledge of how matlab tries to intelligently handle the allocation of memory in the local context of a function. Sometimes you just want to call by reference, and no amount cleverness on the part of the programmer or MatLab will help. I have a cell array with many large matrices in it (signal data). I have a function that goes through the cell array one element at a time and applies a lowpass filter to the matrix. MatLab is smart enough to make a local copy of each element of the cell array only when it is modified (in this case when the filter is applied). So the
programs starts to iterate through the elements of the cell array, and then at a certain point... "out of memory". In this case, I'd rather have the out of memory error as soon as I call the function since eventually it is guaranteed to halt (a case where smart copying does not pay off). I could iterate through the cell array at top level, but that defeats the purpose since the idea was to encapsulate the processing (which has a few other steps) in a function. "My kingdom for a pointer!" Is there any chance that a future version of MatLab will offer call by reference? I would certainly like to have it!

Thanks,
Aaron Schurger

Loren Shure <loren@mathworks.com> wrote in message <MPG.219e667db96ef9889897e6@news.mathworks.com>...
> In article <muy4pfwlrvb.fsf@G99-Boettcher.llan.ll.mit.edu>,
> boettcher@ll.mit.edu says...
> > "Philipp " <philipp.maurer@ocag.ch> writes:
> >
> > > 1) I want a function that operates and changes a data
> > > structure. For example, lets have a cell array <map>
> > > storing mapEntry structs which each struct having a .key
> > > and .value entry. Now I want to write a function that adds
> > > a key/value pair to the map cell array if it is not there
> > > otherwise add it. In Matlab the function call would be like:
> > > map = {};
> > > map = addEntry( map, 'foo', 'test' )
> > > Now the question arises, what happens if map gets very
> > > large ? As far as I understood Matlab, there are now three
> > > copy operations involved. (two there and one in the
> > > functions to copy the map coming in to the map going out).
> >
> > Copying only happens when you modify a piece of data. So if addEntry
> > had only a line that returned map, there would be NO data copying. In
> > each case, a smart reference would happen instead.
> >
> > When you DO modify an array, that is when it is "unshared", and the
> > allocation and copy occurs. But even here, the copy is not necessarily
> > a "deep" copy. If you modify a container object (a cell or a struct),
> > only the top level list of pointers is copied. Each "contained" object
> > is now shared between two cell arrays. When you modify one of those
> > objects, its top level is copied. And so on.
> >
> > So in your case, adding a single cell element to map, there is exactly
> > one allocation and one memcpy. This memcpy is exactly
> > length(map)*sizeof(void*), that is, one pointer for every element in the
> > cell array.
> >
> > Then the MATLAB accelerator comes into play. It is able to optimize
> > some functions of the form you just listed (x = fun(x)), by performing
> > the operation in place. I don't know exactly what conditions the
> > function must meet for this to happen. But certainly it won't work if
> > you are expanding the size of your variable. You should definitely be
> > preallocating map to be big enough, and keep track of the number of
> > current entries with another variable.
> >
> > > 2) I have two lists, lets say a list of bananas and a list
> > > of juices. Every banana has certain attributes and every
> > > juice as well. More than that, every juice has a list of
> > > bananas it is made of. I see only one possibility: every
> > > banana is given a unique ID and every juice stores a list
> > > of these IDs. When I have a juice now and want to access
> > > the attributes of its bananas, I get the ID and then I have
> > > to search my banana list for this ID. See, this is very
> > > slow. I want the juice to refer to the banana (like C
> > > pointers or Java references).
> >
> > Instead of storing a list of unique banana identifiers and searching for
> > them, store a list of banana indices. This will end up
> > (algorithmically, anyway) about as complex as a pointer list.
> >
> > for i=1:length(juice(j).banana_list)
> > banana_idx = juice(j).banana_list(i);
> > current_banana = bananas(banana_idx);
> > end
> >
> >
> > -Peter
> >
>
> also consider using nested functions for some of your purposes. There
> is a single instance of the data available to the nested function
> workspace and it can be updated by calling the nested function (many
> examples in the doc).
>
> also, look at my blog (reference below) to see discussions of memory
> copying.
>
> --
> Loren
> http://blogs.mathworks.com/loren/

Subject: Call by reference / references

From: Matt J

Date: 5 Mar, 2010 15:36:28

Message: 6 of 9

"Aaron Schurger" <aaron.schurger_removeThisText@gmail.com> wrote in message <hmr6ut$2fg$1@fred.mathworks.com>...

 "My kingdom for a pointer!" Is there any chance that a future version of MatLab will offer call by reference? I would certainly like to have it!
=======

It already does, although it requires a modest amount of wrapping.
Essentially, what you need to do is make yourself a generic handle class as follows

classdef myClass < handle
  properties
     
     data;
        
  end
end


Now you can create an object to hold your cell array data as in the following example

>> ob=myClass; ob.data={1,2,3};
>> ob.data

ans =

    [1] [2] [3]

Now you can make changes to ob.data purely by reference, as illustrated next

>> test(ob); ob.data

ans =

    [2] [4] [6]

where I have defined test.m as follows

function test(ob)

 ob.data=cellfun(@(a)2*a, ob.data,'uni',0); %double all cell elements

end

Subject: Call by reference / references

From: Matt J

Date: 5 Mar, 2010 15:51:21

Message: 7 of 9

"Matt J " <mattjacREMOVE@THISieee.spam> wrote in message <hmr8ds$7uh$1@fred.mathworks.com>...

>
> function test(ob)
>
> ob.data=cellfun(@(a)2*a, ob.data,'uni',0); %double all cell elements
>
> end

Sorry, bad example. You would need to implement test.m this way, to avoid a complete new copy of ob.data inside its workspace

 function test(ob)

  for ii=1:length(ob.data)
 
      ob.data{ii}=ob.data{ii}*2;
     
  end

end

Anyway, the point is, the copy of ob.data being manipulated inside test.m is exactly the one in your base workspace. So, you wouldn't get an out-of-memory error in larger sized examples.

Subject: Call by reference / references

From: Mansour Khaleghi

Date: 8 Jun, 2012 17:14:13

Message: 8 of 9

"Philipp " <philipp.maurer@ocag.ch> wrote in message <fgv1uh$hj2$1@fred.mathworks.com>...
> Hi,
>
> since I want to do 'extended' programming in Matlab, I'm
> missing an important feature that all common programming
> languages offer: Call by reference and references (or
> pointer). There are two situations, where I don't know how
> to handle it in Matlab except the brute force (and slow)
> way:
>
> 1) I want a function that operates and changes a data
> structure. For example, lets have a cell array <map>
> storing mapEntry structs which each struct having a .key
> and .value entry. Now I want to write a function that adds
> a key/value pair to the map cell array if it is not there
> otherwise add it. In Matlab the function call would be like:
> map = {};
> map = addEntry( map, 'foo', 'test' )
> Now the question arises, what happens if map gets very
> large ? As far as I understood Matlab, there are now three
> copy operations involved. (two there and one in the
> functions to copy the map coming in to the map going out).
>
> This is just an example there are other examples, too.
>
> 2) I have two lists, lets say a list of bananas and a list
> of juices. Every banana has certain attributes and every
> juice as well. More than that, every juice has a list of
> bananas it is made of. I see only one possibility: every
> banana is given a unique ID and every juice stores a list
> of these IDs. When I have a juice now and want to access
> the attributes of its bananas, I get the ID and then I have
> to search my banana list for this ID. See, this is very
> slow. I want the juice to refer to the banana (like C
> pointers or Java references).
>
> Thanks a lot !


Hi
if you use the function "addEntry()" only for the variable "map" you can use MATLAB global variables as follows:
1- define the variable "x" which you want to edit as a global variable;
2- define and call the function "f()" without input argument "x". in the body of function "f()", define "x" as a global variable;
Now your function doesn't copy "x" and it will work with the same variable "x" which is already in MATLAB work space. for example, for the function you mentioned do it as follows:
global map = {};
addEntry( 'foo', 'test' )

and edit the function "addEntry()" :
#########################
function addEntry(string1,string2)
global map; % so "map" is the same variable as exist in MATLAB work space
.
.
.
#########################

Good luck !

Subject: Call by reference / references

From: Yair Altman

Date: 9 Jun, 2012 17:54:08

Message: 9 of 9

The following article may alleviate some of your worries:

http://undocumentedmatlab.com/blog/internal-matlab-memory-optimizations/

Yair Altman
http://UndocumentedMatlab.com
 

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us