Thread Subject:

why are these two matrices the same size ??

Hello,
I am wondering why b and c take up the same amount of memory, even though b has way less elements:

b=sprand(5*10^3,5*10^3,0.1);
c=b;
b(b<0.99)=0;
length(find(c))

ans =
     2378848

length(find(b))

ans =
       23625

whos
  Name Size Bytes Class Attributes

  b 5000x5000 38101576 double sparse
  c 5000x5000 38101576 double sparse

Considering that b has way less elements than c, I would hope that it would take up much less memory!

Juliette

> Considering that b has way less elements than c, I would hope that it would take up much less memory!
>

doc sparse
and look at the nzmax input argument.

"Juliette Salexa" <juliette.physicist@gmail.com> wrote in message <ibk99t$hg5$1@fred.mathworks.com>...
> Hello,
> I am wondering why b and c take up the same amount of memory, even though b has way less elements:
>
> b=sprand(5*10^3,5*10^3,0.1);
> c=b;
> b(b<0.99)=0;
> length(find(c))
>
> ans =
> 2378848
>
> length(find(b))
>
> ans =
> 23625
>
> whos
> Name Size Bytes Class Attributes
>
> b 5000x5000 38101576 double sparse
> c 5000x5000 38101576 double sparse
>
> Considering that b has way less elements than c, I would hope that it would take up much less memory!
>
> Juliette

Which version of MATLAB are you using, and what operating system? There is a bug in the sparse memory reallocation for some versions of MATLAB where the background realloc function does not allocate a new smaller block. According to communications with TMW it is only for some recent versions of MATLAB and only on Windows 32-bit. Here is a related thread with the bug report:

http://www.mathworks.com/matlabcentral/newsreader/view_thread/294841#790790

James Tursa

"Juliette Salexa" <juliette.physicist@gmail.com> wrote in message <ibk99t$hg5$1@fred.mathworks.com>...
> Hello,
> I am wondering why b and c take up the same amount of memory, even though b has way less elements:
>
> b=sprand(5*10^3,5*10^3,0.1);
> c=b;
> b(b<0.99)=0;
=======

As Sean said, it looks like an nzmax issue. I've checked this both on 32-bit and 64-bit machines and the behavior is the same.

However, I would have expected any assignment statement like

b(b<0.99)=0;

to force a memory reallocation. This, for example, will crunch b down to its minimal size

b(:)=b(:);

>> whos b c
  Name Size Bytes Class Attributes

  b 5000x5000 308220 double sparse
  c 5000x5000 28562664 double sparse

"James Tursa" <aclassyguy_with_a_k_not_a_c@hotmail.com> wrote in message <ibkd5j$rqk$1@fred.mathworks.com>...
> "Juliette Salexa" <juliette.physicist@gmail.com> wrote in message <ibk99t$hg5$1@fred.mathworks.com>...
> > Hello,
> > I am wondering why b and c take up the same amount of memory, even though b has way less elements:
> >
> > b=sprand(5*10^3,5*10^3,0.1);
> > c=b;
> > b(b<0.99)=0;
> > length(find(c))
> >
> > ans =
> > 2378848
> >
> > length(find(b))
> >
> > ans =
> > 23625
> >
> > whos
> > Name Size Bytes Class Attributes
> >
> > b 5000x5000 38101576 double sparse
> > c 5000x5000 38101576 double sparse
> >
> > Considering that b has way less elements than c, I would hope that it would take up much less memory!
> >
> > Juliette
>
> Which version of MATLAB are you using, and what operating system? There is a bug in the sparse memory reallocation for some versions of MATLAB where the background realloc function does not allocate a new smaller block. According to communications with TMW it is only for some recent versions of MATLAB and only on Windows 32-bit. Here is a related thread with the bug report:
>
> http://www.mathworks.com/matlabcentral/newsreader/view_thread/294841#790790
>
> James Tursa

P.S. If you need to recover that memory you can use this workaround:

Download my MTIMESX submission on the FEX:

http://www.mathworks.com/matlabcentral/fileexchange/25977-mtimesx-fast-matrix-multiply-with-multi-dimensional-support

Edit the file mtimesx_RealTimesReal.c
Change the following line 6911 from this:

    if( ar == one && ai == zero ) {

to this:

    if( ar == one && ai == zero && !mxIsSparse(B) ) {

That will force MTIMES to return a deep copy of B with my own reallocation code in the background instead of returning a shared data copy, and MTIMESX will recover the memory. e.g.,

>> edit mtimesx_RealTimesReal.c
     (then change line 6911 ala above and save the file)
>> mtimesx_build
 
... Build routine for mtimesx
... Checking for PC
... Finding path of mtimesx C source code files
... Found file mtimesx.c in C:\Documents and Settings\tursaj\My Documents\MATLAB\mtimesx.c
... Found file mtimesx_RealTimesReal.c in C:\Documents and Settings\tursaj\My Documents\MATLAB\mtimesx_RealTimesReal.c
... Opened the mexopts.bat file in C:\Documents and Settings\tursaj\Application Data\MathWorks\MATLAB\R2008a\mexopts.bat
... Reading the mexopts.bat file to find the compiler and options used.
... LCC is the selected compiler
... OpenMP compiler not detected ... you may want to check this website:
    http://openmp.org/wp/openmp-compilers/
... Using BLAS library lib_blas = 'C:\Program Files\MATLAB\R2008a\extern\lib\win32\lcc\libmwblas.lib'
... Now attempting to compile ...
 
mex('C:\Documents and Settings\tursaj\My Documents\MATLAB\mtimesx.c',lib_blas,'-DCOMPILER=LCC')
 
... mex mtimesx.c build completed ... you may now use mtimesx.
 
>> S = sparse(rand(4000));
>> T = S;
>> T(T<.99)=0;
>> whos
  Name Size Bytes Class Attributes
  S 4000x4000 192016004 double sparse
  T 4000x4000 192016004 double sparse

>> V = mtimesx(1,T);
>> whos
  Name Size Bytes Class Attributes
  S 4000x4000 192016004 double sparse
  T 4000x4000 192016004 double sparse
  V 4000x4000 1928840 double sparse

>> isequal(T,V)
ans =
     1

You can see that V has recovered all that unused memory from T.

I may consider making this change permanent in the next release of MTIMESX just to have this particular bug workaround available for 32-bit Windows users.

James Tursa

"Matt J " <mattjacREMOVE@THISieee.spam> wrote in message <ibkeq4$f1v$1@fred.mathworks.com>...
> "Juliette Salexa" <juliette.physicist@gmail.com> wrote in message <ibk99t$hg5$1@fred.mathworks.com>...
> > Hello,
> > I am wondering why b and c take up the same amount of memory, even though b has way less elements:
> >
> > b=sprand(5*10^3,5*10^3,0.1);
> > c=b;
> > b(b<0.99)=0;
> =======
>
> As Sean said, it looks like an nzmax issue. I've checked this both on 32-bit and 64-bit machines and the behavior is the same.
>
> However, I would have expected any assignment statement like
>
> b(b<0.99)=0;
>
> to force a memory reallocation. This, for example, will crunch b down to its minimal size
>
> b(:)=b(:);
>
> >> whos b c
> Name Size Bytes Class Attributes
>
> b 5000x5000 308220 double sparse
> c 5000x5000 28562664 double sparse

Not on my 32-bit R2008a Windows machine. That is the nature of the bug. e.g.,

>> b=sprand(5*10^3,5*10^3,0.1);
>> c=b;
>> b(b<0.99)=0;
>> whos
  Name Size Bytes Class Attributes
  b 5000x5000 28573176 double sparse
  c 5000x5000 28573176 double sparse

>> b(:)=b(:);
>> whos
  Name Size Bytes Class Attributes
  b 5000x5000 28573176 double sparse
  c 5000x5000 28573176 double sparse

>> nzmax(b)
ans =
     2379431
>> nzmax(c)
ans =
     2379431

>> a = mtimesx(1,b);
>> whos
  Name Size Bytes Class Attributes
  a 5000x5000 303264 double sparse
  b 5000x5000 28573176 double sparse
  c 5000x5000 28573176 double sparse

>> isequal(a,b)
ans =
     1

>> nzmax(a)
ans =
       23605


James Tursa

"James Tursa" <aclassyguy_with_a_k_not_a_c@hotmail.com> wrote in message <ibkg90$i05$1@fred.mathworks.com>...
>
> >> b(b<0.99)=0;
> >> whos
> Name Size Bytes Class Attributes
> b 5000x5000 28573176 double sparse
> c 5000x5000 28573176 double sparse
>
> >> b(:)=b(:);
> >> whos
> Name Size Bytes Class Attributes
> b 5000x5000 28573176 double sparse
> c 5000x5000 28573176 double sparse
=========

That is interesting indeed, however, let's just note that

(1) It wouldn't necessarily coincide with that the OP is seeing because
b(b<0.99)=0 was the only thing she tried, and that apparently fails to compress nzmax on _all_ versions of MATLAB, both 32-bit and 64-bit.

(2) It is still puzzling to me that in the bug-free versions of MATLAB, these two statements

b(b<0.99)=0;
b(:)=b(:);

don't both compress nzmax...

It's well known that cash can make us disembarrass. But how to act when one does not have money? The one way only is to get the <a href="http://bestfinance-blog.com/topics/credit-loans">credit loans</a> or bank loan.