MATLAB Answers

Import multiple .csv files with different of variables and rows

26 views (last 30 days)
Benjamin Azrieli
Benjamin Azrieli on 11 Nov 2020
Edited: Jeremy Hughes on 18 Nov 2020
I have a large amount of .csv files that have basically the same format as the one I attached, except the number/type of variables ( and number of data points is different depending on the file. The function I attached is able to read in the current file I have and assign column headers, which Matlab gathered by reading the 3rd line of the .csv when I used the 'Import Data' then 'Generate Function' buttons. My question is how can I make this function general so that it reads the file, then creates a table with the same column headers as the current file. I do not need any of the information in the top line of the .csv's, I only need the 3rd line (variables) and below (data).
There is a space between the 3rd and 5th lines of the .csv's, which I would like not to change manually because I have so many files, which may affect reading variable names using alternative methods.
I tried removing the variable naming from line 95 of the importfile function I created, and then using textscan() to read the 3rd line of the .csv to use as variable names, but for some reason textscan stops after the first column (Date-time); most likely because it is recognizing the 'time' cell as NaN.
I have also tried importdata(), but all it did was import the very first cell of the .csv.
Any help is appreciated, thank you

Answers (2)

Mathieu NOE
Mathieu NOE on 13 Nov 2020
hello
following your first idea, I modified the import function so that it can extract the varaible names from the 3rd line of the header
so far it works , but if you test it with a csv file with another number of columns or differently organized we will have an issue
with formatSpec = '%s%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%C%f%C%f%f%f%f%C%f%C%f%C%f%f%f%f%f%f%f%C%[^\n\r]'; that also need to be addressed in case of change of dimensions of the csv file
maybe in a 2nd attempt ?
all the best
  2 Comments

Sign in to comment.


Jeremy Hughes
Jeremy Hughes on 17 Nov 2020
Edited: Jeremy Hughes on 18 Nov 2020
More recent versions of MATLAB generate code with the Import Tool which used the Import Options and readtable. The code is simpler, and has more configuration options available.
I added a line to sub-select variables to the base generated code.
opts.SelectedVariableNames = [...]
The code should still work in some older releases (those where import options are available).
  4 Comments
Jeremy Hughes
Jeremy Hughes on 18 Nov 2020
I can see it was fixed in R2019a. I'll make a report, and I've made some changes to work around the issue.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!