MATLAB Central Newsreader - Vectorizing different parts of the same array

Vectorizing different parts of the same array

Theodor Zouk — Mon, 06 Oct 2008 15:10:20 -0400

Hi

Im using a file containing data approximately the size of 1 Giga Byte. Now i gona partition this file into blocks (fragments) of the size of 4KB. (Then make some calculation on each fragment..). I know that i can use a for loop to read each fragment from the file and make my calculations.. but that resolves in an iteration of 250 000.
Now i was thinking: "ok, lets read say 10Mb of the file each iteration and then partition it into 4kb of fragments"

So I will now have an array k=1x10 240 000 elements long.
I want to get k(1:n),where n=4096, make some calculation and then store the result in the first row in an matrix..... then jump to k(n+1:2*n),calculated, store the result in the second row in the matrix and then jump to k(2*n+1:3*n) and so on... Can this be vectorized? I have tried... but i dont think u can avoid an for loop here...

//TEO

Re: Vectorizing different parts of the same array

Adam — Mon, 06 Oct 2008 16:34:02 -0400

"Theodor Zouk" <rebet4@hotmail.com> wrote

> I want to get k(1:n),where n=4096, make some calculation and then store the result in the first row in an matrix..... then jump to k(n+1:2*n),calculated, store the result in the second row in the matrix and then jump to k(2*n+1:3*n) and so on... Can this be vectorized? I have tried... but i dont think u can avoid an for loop here...

can you reshape to 4096xN and work on each column?

~Adam

Re: Vectorizing different parts of the same array

Adam — Mon, 06 Oct 2008 16:36:02 -0400

"Theodor Zouk" <rebet4@hotmail.com> wrote

ok, that wasn't very clear.

reshape, then you should be able to apply functions that work down the columns. Since you didn't say what calculations you're doing it's hard to offer advice towards a solution.

~Adam

Re: Vectorizing different parts of the same array

Walter Roberson — Mon, 06 Oct 2008 16:58:30 -0400

Theodor Zouk wrote:

> So I will now have an array k=1x10 240 000 elements long.
> I want to get k(1:n),where n=4096, make some calculation and then store the result
> in the first row in an matrix..... then jump to k(n+1:2*n),calculated,
> tore the result in the second row in the matrix and then jump to k(2*n+1:3*n) and so on...
> Can this be vectorized? I have tried... but i dont think u can avoid an for loop here...

mat2cell() can break it up into cells of the appropriate size; then you can
use cellfun() (but cellfun is not always faster than a for loop.)

Re: Vectorizing different parts of the same array

Theodor Zouk — Mon, 06 Oct 2008 17:18:02 -0400

"Adam " <not.real@email.com> wrote in message <gcdelq$5he$1@fred.mathworks.com>...
> "Theodor Zouk" <rebet4@hotmail.com> wrote
>
> > I want to get k(1:n),where n=4096, make some calculation and then store the result in the first row in an matrix..... then jump to k(n+1:2*n),calculated, store the result in the second row in the matrix and then jump to k(2*n+1:3*n) and so on... Can this be vectorized? I have tried... but i dont think u can avoid an for loop here...
>
> can you reshape to 4096xN and work on each column?
>
> ~Adam

No i can't reshape... because i have a vector.
My problem is more like: how can i access each element in a part of an vector then store those values (or at best be able to calculate things in parallell) and then continuing to read elements in the same vector from were i stopped reading the first time...

I want to vectorize this for example:

vec=(1:10);
mat= zeros(5,2);
k=1;

for i=1:2:10
    mat(k,:)= vec(i:i+1);
    k=k+1;
end

Re: Vectorizing different parts of the same array

Jos — Mon, 06 Oct 2008 21:37:01 -0400

"Theodor Zouk" <rebet4@hotmail.com> wrote in message <gcdh8a$3lm$1@fred.mathworks.com>...
> "Adam " <not.real@email.com> wrote in message <gcdelq$5he$1@fred.mathworks.com>...
> > "Theodor Zouk" <rebet4@hotmail.com> wrote
> >
> > > I want to get k(1:n),where n=4096, make some calculation and then store the result in the first row in an matrix..... then jump to k(n+1:2*n),calculated, store the result in the second row in the matrix and then jump to k(2*n+1:3*n) and so on... Can this be vectorized? I have tried... but i dont think u can avoid an for loop here...
> >
> > can you reshape to 4096xN and work on each column?
> >
> > ~Adam
>
> No i can't reshape... because i have a vector.

Wrong!

> My problem is more like: how can i access each element in a part of an vector then store those values (or at best be able to calculate things in parallell) and then continuing to read elements in the same vector from were i stopped reading the first time...
>
> I want to vectorize this for example:
>
> vec=(1:10);
> mat= zeros(5,2);
> k=1;
>
> for i=1:2:10
> mat(k,:)= vec(i:i+1);
> k=k+1;
> end

vec = 1:10 ;
mat = reshape(vec,2,[]).'

Jos

Re: Vectorizing different parts of the same array

Theodor Zouk — Tue, 07 Oct 2008 06:17:03 -0400

"Jos " <DELjos@jasenDEL.nl> wrote in message <gce0dt$9j6$1@fred.mathworks.com>...
> "Theodor Zouk" <rebet4@hotmail.com> wrote in message <gcdh8a$3lm$1@fred.mathworks.com>...
> > "Adam " <not.real@email.com> wrote in message <gcdelq$5he$1@fred.mathworks.com>...
> > > "Theodor Zouk" <rebet4@hotmail.com> wrote
> > >
> > > > I want to get k(1:n),where n=4096, make some calculation and then store the result in the first row in an matrix..... then jump to k(n+1:2*n),calculated, store the result in the second row in the matrix and then jump to k(2*n+1:3*n) and so on... Can this be vectorized? I have tried... but i dont think u can avoid an for loop here...
> > >
> > > can you reshape to 4096xN and work on each column?
> > >
> > > ~Adam
> >
> > No i can't reshape... because i have a vector.
>
> Wrong!
>
> > My problem is more like: how can i access each element in a part of an vector then store those values (or at best be able to calculate things in parallell) and then continuing to read elements in the same vector from were i stopped reading the first time...
> >
> > I want to vectorize this for example:
> >
> > vec=(1:10);
> > mat= zeros(5,2);
> > k=1;
> >
> > for i=1:2:10
> > mat(k,:)= vec(i:i+1);
> > k=k+1;
> > end
>
> vec = 1:10 ;
> mat = reshape(vec,2,[]).'
>
> Jos

It works.
thank you Jos

/TEO

Re: Vectorizing different parts of the same array

Bruno Luong — Tue, 07 Oct 2008 06:41:02 -0400

"Theodor Zouk" at time t1
>
> No i can't reshape... because i have a vector.

"Theodor Zouk" at time t2

> It works.
> thank you Jos

Conclusion at the time t2 "Theodor Zouk" is not the same than "Theodor Zouk" at the time t1: He has been reshaped. ;-)

Bruno

Re: Vectorizing different parts of the same array

Theodor Zouk — Tue, 07 Oct 2008 09:02:05 -0400

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <gcf09u$kjq$1@fred.mathworks.com>...
> "Theodor Zouk" at time t1
> >
> > No i can't reshape... because i have a vector.
>
> "Theodor Zouk" at time t2
>
> > It works.
> > thank you Jos
>
> Conclusion at the time t2 "Theodor Zouk" is not the same than "Theodor Zouk" at the time t1: He has been reshaped. ;-)
>
> Bruno

It's true :-) it seems that i didnt really understand the reshape function the first time...