Path: news.mathworks.com!newsfeed-00.mathworks.com!nlpi057.nbdc.sbc.com!prodigy.net!news.glorb.com!postnews.google.com!i28g2000prd.googlegroups.com!not-for-mail
From: Mike <SulfateIon@gmail.com>
Newsgroups: comp.soft-sys.matlab
Subject: Re: filtering data problem
Date: Thu, 23 Apr 2009 21:09:23 -0700 (PDT)
Organization: http://groups.google.com
Lines: 75
Message-ID: <882d3e8b-6dad-4338-b9e1-9dc7328e1974@i28g2000prd.googlegroups.com>
References: <gsoc01$eog$1@fred.mathworks.com> <gsp1tf$bno$1@fred.mathworks.com>
NNTP-Posting-Host: 140.130.155.210
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Trace: posting.google.com 1240546164 456 127.0.0.1 (24 Apr 2009 04:09:24 GMT)
X-Complaints-To: groups-abuse@google.com
NNTP-Posting-Date: Fri, 24 Apr 2009 04:09:24 +0000 (UTC)
Complaints-To: groups-abuse@google.com
Injection-Info: i28g2000prd.googlegroups.com; posting-host=140.130.155.210; 
	posting-account=_SkeuAoAAAC009f9YVFWbpiyXNzGH2zw
User-Agent: G2/1.0
X-HTTP-UserAgent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.9) 
	Gecko/2009040821 Firefox/3.0.9 GTB5 (.NET CLR 3.5.30729),gzip(gfe),gzip(gfe)
Xref: news.mathworks.com comp.soft-sys.matlab:535115


On Apr 23, 2:30=A0pm, "Jos " <#10...@fileexchange.com> wrote:
> "Mike " <sulfate...@gmail.com> wrote in message <gsoc01$eo...@fred.mathwo=
rks.com>...
> > Hi
> > =A0 =A0 I usually need to filter data. =A0Sometimes I use logical-index=
ing, but I found there are many temporary arrays I created. =A0Does that wa=
y waste memory? =A0How to avoid this?
>
> > Mike
>
> Logical indexing is very powerful but indeed creates a (temporary) array.=
 However, they use 1/8 of the memory occupied by =A0double array of the sam=
e size:
> A =3D rand(100,1) ; B =3D A > 0.5 ; whos
>
> You can clear out, or overwrite, any variables you no longer need:
> A =3D rand(10,1) ;
> q =3D A > 0.5 ;
> A =3D A(q) ; % overwrite
> clear q ;
>
> If this is not an option, you can turn to very inefficient loops, like th=
is one, which does not occupy (a lot of) additional memory ...
>
> A =3D rand(10,1) ;
> idx =3D 1 ;
> while (idx <=3D numel(A)),
> =A0 if A(idx) > 0.5,
> =A0 =A0 idx =3D idx + 1 ; % goto next element
> =A0 else
> =A0 =A0 A(idx) =3D [] ; % remove element from list
> =A0 end
> end
>
> hth
> Jos

Thank you very much, Jos.
Allow me to add some cputime elapse time statements.

N=3D100000;
t =3D cputime;
A =3D rand(N,1) ;
q =3D A > 0.5 ;
A =3D A(q) ; % overwrite
clear q ;
e=3Dcputime-t

t =3D cputime;
A =3D rand(N,1) ;
idx =3D 1 ;
while (idx <=3D numel(A)),
  if A(idx) > 0.5,
    idx =3D idx + 1 ; % goto next element
  else
    A(idx) =3D [] ; % remove element from list
  end
end
e=3Dcputime-t

e =3D

    0.0156


e =3D

   46.5781

The latter one is very inefficient as you say.
In practice, I filter data using the first way continuously.
The inefficiency would propagate one by one using element-by-element
way.

Mike