Thread Subject: Importing selected column from a Csv files

Hi,
I have a very big CSV data archive.
Due to dimensions I'm not able to import all of it in the
Workspace.

As I don't need all the variables,
I just would like to read a group of selected columns.

Let's say I want to import columns 1 39 50 90.
I could do it with a cicle using "csvread" and importing one
column once a time, but I would prefer doing this with a
built in function (something like csvread(file, [1 39 50
90]). Is this possible???

Thank you very much in advance!
Best regards -Luca-

"Luca Cerone" <cerone@to-see.it> wrote in message
<fjm0gl$s1u$1@fred.mathworks.com>...
> Hi,
> I have a very big CSV data archive.
> Due to dimensions I'm not able to import all of it in the
> Workspace.
>
> As I don't need all the variables,
> I just would like to read a group of selected columns.
>
> Let's say I want to import columns 1 39 50 90.
> I could do it with a cicle using "csvread" and importing one
> column once a time, but I would prefer doing this with a
> built in function (something like csvread(file, [1 39 50
> 90]). Is this possible???
>
> Thank you very much in advance!
> Best regards -Luca-


This is a good case for using textscan.

Something like:

wanted_cols=[1 39 50 90];
format=[];
for I=1:num_cols
if any(I==wanted_cols)
format=[format '%n'];
else
format=[format '%*n'];
end
end

(There may be a more elegant way to create this format but
I've never been able to figure it out.)

Now we have a char array which we can use as the 'format' in
textscan.

Now to read the file:
fid=fopen('filename.csv','rt');

data=textscan(fid,format,'delimiter',',')

fclose(fid);



In my experience this method tends to be faster than using a
loop and csvread for large files since it only scans the
file once.

Hope this helps!

Thank you very much, it seems a smart idea!
(I do something similar to create MySql queries)

Thanks a lot!

"William Dampier" <walldo2@gmail.com> wrote in message
<fjpfbu$368$1@fred.mathworks.com>...
> "Luca Cerone" <cerone@to-see.it> wrote in message
> <fjm0gl$s1u$1@fred.mathworks.com>...
> > Hi,
> > I have a very big CSV data archive.
> > Due to dimensions I'm not able to import all of it in the
> > Workspace.
> >
> > As I don't need all the variables,
> > I just would like to read a group of selected columns.
> >
> > Let's say I want to import columns 1 39 50 90.
> > I could do it with a cicle using "csvread" and importing one
> > column once a time, but I would prefer doing this with a
> > built in function (something like csvread(file, [1 39 50
> > 90]). Is this possible???
> >
> > Thank you very much in advance!
> > Best regards -Luca-
>
>
> This is a good case for using textscan.
>
> Something like:
>
> wanted_cols=[1 39 50 90];
> format=[];
> for I=1:num_cols
> if any(I==wanted_cols)
> format=[format '%n'];
> else
> format=[format '%*n'];
> end
> end
>
> (There may be a more elegant way to create this format but
> I've never been able to figure it out.)
>
> Now we have a char array which we can use as the 'format' in
> textscan.
>
> Now to read the file:
> fid=fopen('filename.csv','rt');
>
> data=textscan(fid,format,'delimiter',',')
>
> fclose(fid);
>
>
>
> In my experience this method tends to be faster than using a
> loop and csvread for large files since it only scans the
> file once.
>
> Hope this helps!