Thread Subject: Need File I/O help for reading and rewriting string data

Subject: Need File I/O help for reading and rewriting string data

From: "G.A.M.

Date: 16 Sep, 2007 02:42:18

Message: 1 of 1

I need to read from a CSV file of mixed data types (mostly strings). I
need to process each record (such as adding a few more values to it).
Then I need to write (or append) the data to another CSV file.

I can't seem to find a decent way to do this. The code below is super
slow on large files (and this code is a drastic simplification of what
I really need to do). I actually have to look for and remove duplicate
records before writing out the modified data.

I would appreciate any suggestions for improving the code. (For now
I'm stuck using ASCII files, so I can't go to a database or binary
files.)

filedata = textread(myFilename, '%s','delimiter','\n');
charRec = '';
outputData = cell(length(filedata), 1);%preallocate
for k = 1 : length(filedata)
record = strread(char(filedata(k)),'%s','delimiter',',');
for m = 1 : length (record)
temp = strcat(record{m}, ',');
charRec = strcat(charRec, temp);
end%for
charRec = strcat(charRec, 'newData');
outputData(k) = {charRec};
charRec = '';
end

testFile = fopen (testFilename,'wt');
if testFile ~= -1
    for k = 1 : length(outputData)
     fprintf(testFile,'%s\n', char(outputData(k)));
    end
    fclose(testFile);
end


3 lines of sample data:
lastname1,firstname,DOB,Male,Caucasian,,,sp = 0 ti = 26,22,ox(1),03-
Aug-2007 10:36:48,13.15,5.85,189.058,18.9,5.8
lastname2,firstname,DOB,Male,Caucasian,,,sp = 0 ti = 33,22,ox(2),03-
Aug-2007 10:37:20,16.54,6.35,213.073,21.3,7.3
lastname3,firstname,DOB,Male,Caucasian,,,sp = 0 ti = 27,22,ox(3),03-
Aug-2007 10:53:16,15.86,7.68,192.082,19.2,8.2

I hope all this code and data comes out readable.

Tags for this Thread

Everyone's Tags:

Add a New Tag:

Separated by commas
Ex.: root locus, bode

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Tag Activity for This Thread
Tag Applied By Date/Time
fileio G.A.M. 16 Sep, 2007 09:52:05
fprintf G.A.M. 16 Sep, 2007 09:52:05
ascii G.A.M. 16 Sep, 2007 09:52:05
import G.A.M. 16 Sep, 2007 09:52:05
export G.A.M. 16 Sep, 2007 09:52:05
rssFeed for this Thread
 

MATLAB Central Terms of Use

NOTICE: Any content you submit to MATLAB Central, including personal information, is not subject to the protections which may be afforded information collected under other sections of The MathWorks, Inc. Web site. You are entirely responsible for all content that you upload, post, e-mail, transmit or otherwise make available via MATLAB Central. The MathWorks does not control the content posted by visitors to MATLAB Central and, does not guarantee the accuracy, integrity, or quality of such content. Under no circumstances will The MathWorks be liable in any way for any content not authored by The MathWorks, or any loss or damage of any kind incurred as a result of the use of any content posted, e-mailed, transmitted or otherwise made available via MATLAB Central. Read the complete Terms prior to use.

Contact us at files@mathworks.com