Error while reading the .csv file

While reading the .txt files, the below code runs good. But when I read .csv files, I end up with the following error.

Error using fopen

File identifier must be an integer-valued scalar of type double.

Error in check (line 12)

fid = fopen(fn, 'r');

Can someone help me to resolve this part.

clear
clc
 
%f=dir('*.txt');                     % finding all txt files
f=dir('*.csv');                     % finding all csv files
n_snapshots=size(f,1);              % finding number of snapshots
 
for i=1:n_snapshots
    %fn=f(i).name;                   % for .txt file
    fn = readmatrix(f(i).name);      % for .csv file
    fid = fopen(fn, 'r');             % reading files
    % scaning files into 4 columns - gives in rows
    data=fscanf(fid, '%f %f %f %f' , [4, inf]);               
    % exporting columns
    x=data(4,:);
    y=data(3,:);
    v(i,:)=data(1,:);
    u(i,:)=data(2,:);
    fclose('all');
end 

Thank you

Answer 1

[Stephen23]

Explanation: As its documentation makes clear, the first input to FOPEN must be the fiename. However, instead of calling it with a filename you call it with some matrix of data that you imported using READMATRIX. That will not work.

Solution: replace this

%fn=f(i).name;
fn = readmatrix(f(i).name);
fid = fopen(fn, 'r');

with

fid = fopen(f(i).name, 'r');

You should also replace

fclose('all');

with

fclose(fid);

But why are you calling FOPEN & FSCAN & FCLOSE when you could simply call READMATRIX ?

P = 'absolute or relative path to where the files are saved'; 
S = dir(fullfile(P,'*.csv'));
for k = 1:numel(S)
    F = fullfile(P,S(k).name);
    M = readmatrix(F, 'FileType','text');
    ...
end

CSV files are text files. There is no reason why you cannot use READMATRIX to import both .CSV and .TXT files. You should use READMATRIX unless there is a really good reason why you cannot use it.

[Kumaresh Kumaresh]

Thank you for your answer. With the software I work, I could able to export only .csv files. But my senior who work with experiments could able to extract .txt files and make use of below Matlab code for post-processing. With the corrections you mentioned, I still find a bug and I couldn't able to fix as I am not good in programming. I need to read 4 columns given in the attachment, but Matlab reads only 1 column:

Index in position 1 exceeds array bounds
(must not exceed 1).

Error in OOO_9004 (line 17)
    x=data(2,:);

Here is the complete code:

    clear
clc
 
%f=dir('*.txt');                     % finding all txt files
f=dir('*.csv');                     % finding all csv files
%f=dir('*.csv','Row_offset','column_offset');  
n_snapshots=size(f,1);              % finding number of snapshots
 
for i=1:n_snapshots
    %fn=f(i).name;                   % Listing filenames
    %fn = readmatrix(f(i).name);      % https://in.mathworks.com/matlabcentral/answers/2053302-how-to-import-multiple-csv-file-in-matlab
    fid=fopen(f(i).name, 'r');             % reading files
    %fid=input(fn, 'r');    
    % scaning files into 4 columns - gives in rows
    data=fscanf(fid, '%f %f %f %f' , [4, inf]);               
    % exporting columns
    x=data(4,:);
    y=data(3,:);
    v(i,:)=data(1,:);
    u(i,:)=data(2,:);
    fclose(fid);
end
 
% Correlation Matrix C
 
c1=v*v';
c2=u*u';
C=(c1+c2)/n_snapshots;
 
% Cbeta=lambdaB - eigenvalue problem
 
[beta,lmd]=svd(C);
 
% basis functions
 
phix=u'*beta;
phiy=v'*beta;
 
% normalization
 
Gridnumber=size(x,2);

for j=1:n_snapshots
    
    phinor=0;
    
    for i=1:Gridnumber
        phinor=phinor+power(phix(i,j),2)+power(phiy(i,j),2);
    end
    phinor=sqrt(phinor);
                
    phix(:,j)=phix(:,j)/phinor;
    phiy(:,j)=phix(:,j)/phinor;
    
end
 
TimCoeU=v*phix;
TimCoeV=u*phiy;
PrimitiveTimeCoefficientMatrix=TimCoeU+TimCoeV;
 
% Sorting
 
O=PrimitiveTimeCoefficientMatrix';
J=sort(O, 'descend');
TimeCoefficient=J';
Ensambleaverage=mean(TimeCoefficient);
TimeCoeffEnsambleaveAppend=[TimeCoefficient;Ensambleaverage];
 
%Extracting modes
 
for a=1:n_snapshots
    FilNamPhi=1000+a;
    PhiOut = fopen([num2str(FilNamPhi), '.txt'], 'wt');
    fprintf(PhiOut,'');
    phia =[x;y;phix(:,a)';phiy(:,a)'];
    fprintf(PhiOut, '%20.9f %20.9f %20.9f %20.9f\n',phia);
    fclose(PhiOut);
end
 
xlswrite('TimeCoefficient.xlsx', TimeCoefficient);
dlmwrite('Ensambleaverage.txt', Ensambleaverage);
dlmwrite('TimeCoeffEnsambleaveAppend.txt', TimeCoeffEnsambleaveAppend);
 

Kindly check it. Thank you

[Walter Roberson]

Your error message says

Error in OOO_9004 (line 17)
    x=data(2,:);

Your code says

    x=data(4,:);

Your posted code disagrees with the error message.

Anyhow, if the file is empty, then the fscanf() will return an empty array, [], which will not have four rows.

Likewise, fscanf() will return the empty matrix if there is text in the file before the numeric data. Which I suspect is the case here. You probably need to insert a line

  fgetl(fid);    %read and discard text header line

[Walter Roberson]

Note that fscanf() tries to apply the given input format from where-ever it happens to be positioned in the file, and will fail returning [] if the input format does not match what is in the file at the current position.

fscanf() does not skip input words or input lines looking for a place in the input file that happens to match the input format.

[Kumaresh Kumaresh]

Thank you for your answer. Sorry for the typo error message. To keep it simple, the following code works for .txt file. Can you guide me, how to make it work for .csv file.

clear
clc
 
f=dir('*.txt');                     % finding all txt files
%f=dir('*.csv');                     % finding all csv files
n_snapshots=size(f,1);              % finding number of snapshots
 
for i=1:n_snapshots
    fn=f(i).name;                   % Listing filenames
    %fn = readmatrix(f(i).name);      % https://in.mathworks.com/matlabcentral/answers/2053302-how-to-import-multiple-csv-file-in-matlab
    %fid=fopen(f(i).name, 'r');             % reading files
    fid=fopen(fn, 'r');             % reading files
    %fid=input(fn, 'r');    
    % scaning files into 4 columns - gives in rows
    data=fscanf(fid, '%f %f %f %f' , [4, inf]);
    %data = fgetl(fid);
    % exporting columns
    x=data(4,:);
    y=data(3,:);
    v(i,:)=data(1,:);
    u(i,:)=data(2,:);
    fclose(fid);
end
 
% Correlation Matrix C
 
c1=v*v';
c2=u*u';
C=(c1+c2)/n_snapshots;
 
% Cbeta=lambdaB - eigenvalue problem
 
[beta,lmd]=svd(C);
 
% basis functions
 
phix=u'*beta;
phiy=v'*beta;
 
% normalization
 
Gridnumber=size(x,2);

for j=1:n_snapshots
    
    phinor=0;
    
    for i=1:Gridnumber
        phinor=phinor+power(phix(i,j),2)+power(phiy(i,j),2);
    end
    phinor=sqrt(phinor);
                
    phix(:,j)=phix(:,j)/phinor;
    phiy(:,j)=phix(:,j)/phinor;
    
end
 
TimCoeU=v*phix;
TimCoeV=u*phiy;
PrimitiveTimeCoefficientMatrix=TimCoeU+TimCoeV;
 
% Sorting
 
O=PrimitiveTimeCoefficientMatrix';
J=sort(O, 'descend');
TimeCoefficient=J';
Ensambleaverage=mean(TimeCoefficient);
TimeCoeffEnsambleaveAppend=[TimeCoefficient;Ensambleaverage];
 
%Extracting modes
 
for a=1:n_snapshots
    FilNamPhi=1000+a;
    PhiOut = fopen([num2str(FilNamPhi), '.txt'], 'wt');
    fprintf(PhiOut,'');
    phia =[x;y;phix(:,a)';phiy(:,a)'];
    fprintf(PhiOut, '%20.9f %20.9f %20.9f %20.9f\n',phia);
    fclose(PhiOut);
end
 
xlswrite('TimeCoefficient.xlsx', TimeCoefficient);
dlmwrite('Ensambleaverage.txt', Ensambleaverage);
dlmwrite('TimeCoeffEnsambleaveAppend.txt', TimeCoeffEnsambleaveAppend);
 

Thank you

[Walter Roberson]

Change

f=dir('*.txt');                     % finding all txt files

to

f=dir('*.csv');                     % finding all csv files

Change

    data=fscanf(fid, '%f %f %f %f' , [4, inf]);

to

    data = fscanf(fid, '%f,%f,%f,%f' , [4, inf]);

[Kumaresh Kumaresh]

Thank you so much. It worked. I have one more concern. I have a folder with multiple .csv files, similar to the one attached here. I am trying to ignore the top row of the csv files and read the remaining rows through the script. In order to do it, I did some research in Matlab forums, where I could find row and column offsets (that ignores top row containing strings), which is not straight forward to fix with my above code to work on.

x=csvread('filename.csv','Row_offset','column_offset')
% f=dir('*.csv','Row_offset','column_offset'); 
% M(1:3,:) = []; -- Trim the first three rows

Kindly give some insights.

Thank you

[Walter Roberson]

Unless you are using a rather old version of MATLAB, we recommend,

x = readmatrix('filename.csv');

readmatrix() automatically skips headers (if any)

[Kumaresh Kumaresh]

Thank you.

Is there any other way you can recommend to resolve this issue in current MATLAB version ?

[Walter Roberson]

R2019a to present: recommend you use readmatrix()

R2013b to R2018b: recommend you use readtable() followed by array2table()

Older but not super-super-super old versions of MATLAB: use fileread() on the file. Examine the character vector you get back to figure out how many lines of headers there are. Use textscan() on the character vector with 'HeaderLines' option telling it how many header lines to skip.

Super-super-super but not Super-super-super-super old versions of MATLAB: fopen() and use fgetl() to fetch lines and examine them, eventually finding the first numeric line. sscanf() to get the numeric content of that line. fscanf() the rest of the file to get the remaining numeric lines.

MATLAB 1.x and MATLAB 2.x: I don't know, those releases did not even have character vectors; I would need access to terribly old documentation to figure it out.

I simplified the lineage here. There were some versions in there that supported textread() before textscan() existed.