Thread Subject: Keep every tenth input from .ascii-file

I have an .ascii-file with loads of input-data which I need to down-sample. The sample-rate is now 1000Hz and I want 100Hz so I thought I could write a MATLAB-program that only reads every tenth and put the values in a vector.

How do I do this?

"Niklas " <magnussn@student.chalmers.se> wrote in message <hnvvo9$5nu$1@fred.mathworks.com>...
> I have an .ascii-file with loads of input-data which I need to down-sample. The sample-rate is now 1000Hz and I want 100Hz so I thought I could write a MATLAB-program that only reads every tenth and put the values in a vector.
>
> How do I do this?

Hi Niklas, Do you have the Signal Processing Toolbox? If so, then why not just read the file and use downsample() in the MATLAB workspace?

Wayne

"Niklas " <magnussn@student.chalmers.se> wrote in message <hnvvo9$5nu$1@fred.mathworks.com>...
> I have an .ascii-file with loads of input-data which I need to down-sample. The sample-rate is now 1000Hz and I want 100Hz so I thought I could write a MATLAB-program that only reads every tenth and put the values in a vector.
>
> How do I do this?

Something like this, perhaps?

DownSampledData = Data(1:10:end)

If you need to skip values during reading, take a look at format strings like '%f%*f%*f', which does only work properly if your data is stored in a single row in the ascii file.

hth
Jos

"Wayne King" <wmkingty@gmail.com> wrote in message <ho00lp$l1a$1@fred.mathworks.com>...
> "Niklas " <magnussn@student.chalmers.se> wrote in message <hnvvo9$5nu$1@fred.mathworks.com>...
> > I have an .ascii-file with loads of input-data which I need to down-sample. The sample-rate is now 1000Hz and I want 100Hz so I thought I could write a MATLAB-program that only reads every tenth and put the values in a vector.
> >
> > How do I do this?
>
> Hi Niklas, Do you have the Signal Processing Toolbox? If so, then why not just read the file and use downsample() in the MATLAB workspace?
>
> Wayne

Thanks for the tip Wayne. The problem is that I have more than 888 000 input elements. This way I get every tenth sample, but only of a small piece of the original data-collection.

If you wonder what I am doing this is EEG-measurements, 1000 Hz during (about) 20 minutes. That gives me LOADS of data. I need to downsample at the same time I read the file. Or can I tell MATLAB to start reading after the 888 000:th data? That way I could do the down-sampling in separate steps.

Niklas wrote:
> I have an .ascii-file with loads of input-data which I need to
> down-sample. The sample-rate is now 1000Hz and I want 100Hz so I thought
> I could write a MATLAB-program that only reads every tenth and put the
> values in a vector.
>
> How do I do this?

 >> x=[1:20]; s = num2str(x); % make a short sample data string
 >> v = sscanf(s,'%d %*d')' % read every other one
v =
      1 3 5 7 9 11 13 15 17 19
 >> % salt to suit

Note you may find repmat() useful in making up the format string to
repeat the (in your example case) the formatting string that will need
the optional "*" formatting option repeated 9 times...

 >> fmt = strcat('%f ', repmat('%*f ',1,9))
fmt =
%f%*f %*f %*f %*f %*f %*f %*f %*f %*f
 >>



--

"Niklas " <magnussn@student.chalmers.se> wrote in message <hnvvo9$5nu$1@fred.mathworks.com>...
> I have an .ascii-file with loads of input-data which I need to down-sample. The sample-rate is now 1000Hz and I want 100Hz so I thought I could write a MATLAB-program that only reads every tenth and put the values in a vector.
>
> How do I do this?

do something like this (unfinished & untested code)

Data100=zeros(1e6,1); %pre-allocate
fid=fopen(fname,'r');
row=1;
n=0;
v=0;
while ~feof(fid)
    v=v+str2double(fgetl(s)); %read a line
    n=n+1;
    if n==10 %average groups of 10 samples
       if row>=length(Data100)
            Data100(end+1e6)=0; %allocate chunk
       end
       Data100(row)=v/n;
       row=row+1;n=0;
    end
end
Data100(row+1:end)=[]; %truncate
fclose(fid)

Niklas wrote:
...

> Thanks for the tip Wayne. The problem is that I have more than 888 000
> input elements. This way I get every tenth sample, but only of a small
> piece of the original data-collection.
...

See my earlier response that demonstrates the how of reading every n-th
value (from a string, but sscanf() has its brother fscanf()).

Depending on the form of the ascii file, there may be some other
specifics but this should be enough to let you figure out what you need
for the actual case.

There also is textread(), textscan() and friends...

doc % the above function(s) and proceed...

--

Thanks for all your help.

The problem, though, still remains the same. I can read every tenth, but only every tenth from the first 888 000 data samples. I presume that fopen only can open files with a limit of 888 000 entries and that is the problem. can you make MATLAB open data after the 888 000:st ? that way you could open the file in different "parts" and put the down-sampled parts together. Or is there another way around?

Niklas wrote:
> Thanks for all your help.
>
> The problem, though, still remains the same. I can read every tenth, but
> only every tenth from the first 888 000 data samples. I presume that
> fopen only can open files with a limit of 888 000 entries and that is
> the problem. ...

No, that certainly is _NOT_ a limitation of fopen()

It is something in the structure of the file which you haven't provided
enough detail about to be able to debug.

fopen() returns a handle to a file; it doesn't care what the size of the
file is.

Need more info.

--

> No, that certainly is _NOT_ a limitation of fopen()
>
> It is something in the structure of the file which you haven't provided
> enough detail about to be able to debug.
>
> fopen() returns a handle to a file; it doesn't care what the size of the
> file is.
>
> Need more info.
>
> --

The .ascii-file contains numeric data (both positive and negative, integer). Each numeric data is seperated with 'space'. e.g:

-6435 -1235 12312 -2324 5455 -1244

and so on...

Niklas wrote:
>> No, that certainly is _NOT_ a limitation of fopen()
>>
>> It is something in the structure of the file which you haven't
>> provided enough detail about to be able to debug.
>>
>> fopen() returns a handle to a file; it doesn't care what the size of
>> the file is.
>>
>> Need more info.
>>
>> --
>
> The .ascii-file contains numeric data (both positive and negative,
> integer). Each numeric data is seperated with 'space'. e.g:
>
> -6435 -1235 12312 -2324 5455 -1244
> and so on...

Well, at some point there's a record boundary or something at that point
  or something else going on.

 >> x=fix((rand(1E6,1)-0.5)*10E4); % make up some sample data
 >> fid=fopen('test.dat','wt');
 >> fprintf(fid, '%7d',x');
 >> fclose(fid)
ans =
      0
 >> fid=fopen('test.dat','rt');
 >> z(~(z<5))
 >> v = fscanf(fid,fmt);
 >> fmt = strcat('%f ', repmat('%*f ',1,9))
fmt =
%f%*f %*f %*f %*f %*f %*f %*f %*f %*f
 >> v = fscanf(fid,fmt);
 >> length(v)/length(y)
ans =
     0.1000
 >> whos x v
   Name Size Bytes Class

   v 100000x1 800000 double array
   x 1000000x1 8000000 double array

Grand total is 1100000 elements using 8800000 bytes

 >>

Or, try w/ x 10X as long...

 >> x=fix((rand(1E7,1)-0.5)*10E4); % make up some sample data
 >> fid=fopen('test.dat','wt');
 >> fprintf(fid, '%7d',x');
 >> fclose(fid)
ans =
      0
 >> fid=fopen('test.dat','rt');
 >> v = fscanf(fid,fmt);
 >> whos x v
   Name Size Bytes Class

   v 1000000x1 8000000 double array
   x 10000000x1 80000000 double array

Grand total is 11000000 elements using 88000000 bytes

 >>

Clearly it isn't anything fundamental; something is causing the problem
in your file.

How was the file written might be a starting point for clues.

Or, use a hex viewer and see if there are record markers, etc., in it.

Alternatively, did you forget to fopen w/ the 't' option to indicate ASCII?

--

"Niklas" wrote in message <hnvvo9$5nu$1@fred.mathworks.com>...
> I have an .ascii-file with loads of input-data which I need to down-sample. The sample-rate is now 1000Hz and I want 100Hz so I thought I could write a MATLAB-program that only reads every tenth and put the values in a vector.
>
> How do I do this?

Yikes...how about this:
 
 V_every10th = v(1:10:end)

http://www.mathworks.com/company/newsletters/articles/Matrix-Indexing-in-MATLAB/matrix.html;jsessionid=4JllTRLZ5VQnMFz11LCrzVDXr12nJTQTTNpbn7vFlGLpb4Twz4W5!-990726143