Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
How to delete the first 50 rows of individual files

Subject: How to delete the first 50 rows of individual files

From: Yan

Date: 23 May, 2013 04:05:11

Message: 1 of 5

Hi there,

I need to delete the first 50 rows and the last 200 rows from individual text files. I have over 400 individual tab-delimited files, all are matrix of 600 by 65.

Any help will be greatly appreciated.

Yan

Subject: How to delete the first 50 rows of individual files

From: dpb

Date: 23 May, 2013 13:31:52

Message: 2 of 5

On 5/22/2013 11:05 PM, Yan wrote:
> Hi there,
>
> I need to delete the first 50 rows and the last 200 rows from individual
> text files. I have over 400 individual tab-delimited files, all are
> matrix of 600 by 65.
>
> Any help will be greatly appreciated.

Basically, the quickest will be to use dir() to return the list of
files, process it sequentially to read in the array; do the necessary
elimination of rows in memory and then save the result either
overwriting the original file if not needed/wanted or in a new file if
that's the choice wanted.

One could use textscan() w/ the 'headerlines' option to skip the first
50 lines on the reading part and also only read the next 200 lines, but
I suspect the overhead of doing it that way would make the
straightforward way noticeably quicker.

--

Subject: How to delete the first 50 rows of individual files

From: Yan

Date: 23 May, 2013 14:03:25

Message: 3 of 5

Thanks a lot for your response!
I am shamefully a beginning user of Matlab with no programming knowledge.
My friend wrote me the following script to take off the first two lines.
I guess that just a few changes to the script will make it read files from line 51 to line 450. However, I do not know how to twist it :(

Below is the script:


function filt
    % Define parameters
    path = '/Users/yuy3/Documents/MATLAB/PCA_CSD/Yan_diss_CSD_mul_all/mms'; % The folder contains data
    overwrite = true; % Whether overwrite the data file, if not, a new data file with surffix '.new' will be written

    % Search the directory to find all files
    namelist = dir_search( path, char( '') );
    
    for it = 1 : size(namelist,1)
        % Form the name of the source file and the destination file
        source = deblank( namelist( it, : ) );
        if isempty( source )
            continue
        end
        if overwrite
            dest = source;
        else
            dest = sprintf( '%s.new', source );
        end
        % Read file, and filter it
        file_filt( source, dest );
    end
end

% Search directory
function newlist = dir_search( path, namelist )
    % Read directory
    direnv = dir(path);
    newlist = namelist;
    for it = 1 : size(direnv,1)
        fname = direnv(it).name;
        % Skip the parent directory and the directory itself
        if strcmp( fname, '.' )
            continue
        end
        if strcmp( fname, '..' )
            continue
        end
        fullname = sprintf( '%s/%s', path, fname );
        if isdir( fullname )
            % If this is a directory, search it recursively
            newlist = char( newlist, dir_search( fullname, char('') ) );
        else
            % If this is a file, add the name into list
            newlist = char( newlist, fullname );
        end
    end
end

% File filter
function file_filt ( source, dest )
    % Read source file
    fid = fopen( source, 'rb' );
    data = char( '' );
    dummy = fgets( fid ); % Skip the first line
    dummy = fgets( fid ); % Skip the second line
    x = fgets( fid );
    while ischar( x )
        % Read the following lines
        data = char( data, x );
        x = fgets( fid );
    end
    fclose( fid );
    
    % Write destination file
    fid = fopen( dest, 'wb' );
    for it = 2 : size( data, 1 )
        x = deblank( data( it, : ) );
        % Write each line, add return and enter at the end of line except
        % the last line
        if ( it < size( data, 1 ) )
            fprintf( fid, '%s\r\n', x );
        else
            fprintf( fid, '%s', x );
        end
    end
    fclose( fid );
end


Again, thanks a ton!

Yan



dpb <none@non.net> wrote in message <knl5or$f3i$1@speranza.aioe.org>...
> On 5/22/2013 11:05 PM, Yan wrote:
> > Hi there,
> >
> > I need to delete the first 50 rows and the last 200 rows from individual
> > text files. I have over 400 individual tab-delimited files, all are
> > matrix of 600 by 65.
> >
> > Any help will be greatly appreciated.
>
> Basically, the quickest will be to use dir() to return the list of
> files, process it sequentially to read in the array; do the necessary
> elimination of rows in memory and then save the result either
> overwriting the original file if not needed/wanted or in a new file if
> that's the choice wanted.
>
> One could use textscan() w/ the 'headerlines' option to skip the first
> 50 lines on the reading part and also only read the next 200 lines, but
> I suspect the overhead of doing it that way would make the
> straightforward way noticeably quicker.
>
> --

Subject: How to delete the first 50 rows of individual files

From: dpb

Date: 23 May, 2013 19:07:35

Message: 4 of 5

On 5/23/2013 9:03 AM, Yan wrote:

...[Please don't toppost...hard conversation follow makes]...

> Thanks a lot for your response!
> I am shamefully a beginning user of Matlab with no programming knowledge.
> My friend wrote me the following script to take off the first two lines.
> I guess that just a few changes to the script will make it read files
> from line 51 to line 450. However, I do not know how to twist it :(
...

Pretty complicated for the job it seems...you _could_ fix it up to skip
the 50 lines one at a time as is done simply by changing a loop counter
and the same thing for the reading, but I'd go at it more like

L1=51; % set the two line limits desired to keep
L2=450;
d=dir('suitableWildCardNameString.txt'); % get the files desired
for i=length(d)
   x=dlmread(d(i).name,'\t'); % read the tab-delimited file
   x=x(L1:L2,:); % keep desired lines
   dlmwrite(d(i).name,'\t'); % overwrite the existing file
   % fixup the above by making a new file name first if need, obviously
end

The above does assume the files are in the working directory and don't
need to search subdirectories, etc., etc., etc., ... but those are
details. The guts of how (at least one relatively efficient way) to
truncate an existing file to a selected portion is herein above.

--

Subject: How to delete the first 50 rows of individual files

From: dpb

Date: 23 May, 2013 19:14:45

Message: 5 of 5

On 5/23/2013 2:07 PM, dpb wrote:
> On 5/23/2013 9:03 AM, Yan wrote:
>
> ...[Please don't toppost...hard conversation follow makes]...
>
>> Thanks a lot for your response!
>> I am shamefully a beginning user of Matlab with no programming knowledge.
>> My friend wrote me the following script to take off the first two lines.
>> I guess that just a few changes to the script will make it read files
>> from line 51 to line 450. However, I do not know how to twist it :(
> ...
>
> Pretty complicated for the job it seems...you _could_ fix it up to skip
> the 50 lines one at a time as is done simply by changing a loop counter
> and the same thing for the reading, but I'd go at it more like
>
> L1=51; % set the two line limits desired to keep
> L2=450;
> d=dir('suitableWildCardNameString.txt'); % get the files desired
> for i=length(d)
> x=dlmread(d(i).name,'\t'); % read the tab-delimited file
> x=x(L1:L2,:); % keep desired lines
> dlmwrite(d(i).name,'\t'); % overwrite the existing file
> % fixup the above by making a new file name first if need, obviously
> end
>
> The above does assume the files are in the working directory and don't
> need to search subdirectories, etc., etc., etc., ... but those are
> details. The guts of how (at least one relatively efficient way) to
> truncate an existing file to a selected portion is herein above.

I blew syntax here by cut 'n pasting from read instead of thinking,
sorry..hard to output anything w/o anything in the call list.

dlmwrite(d(i).name,x,'\t'); % overwrite the existing file

--

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us