how to extract data from a range of (slightly) varying csv files

Hi there,

I'm trying to write a function that will import and plot relevant data from a csv file. This is because for every file I have, there is some settings above the data that vary in length. So for example; sometimes the data starts at row 20 and sometimes 50 . Also the length of the data column varies. The columns from which I want to extract data is always the same.

How can matlab recognise from what point to take the data?

I think this should be possible as the settings only take 2/3 columns and the data is 5 or 6 columns wide, so you can 'see' where the data starts if you let matlab 'search' in the 6th column if that is possible.

Thank you in advance!

Geard

[Jan]

We cannot know how to recognize the length of the header, if you do not explain, how the header can be distinguished from the data. Maybe the header lines start with a comment character? Or there is an empty line between header and data?

What does "the settings only take 2/3 columns" mean exactly? Please define uniquely, how the data can be identified.

[Gerard Kluivers]

The header values in the first column of the csv file are either empty or text, whilst the data contains numbers.

I included a screenshot from one of the files at the lines where the data starts to to give you an idea about the recorded settings in contrast to the data (plain numbers). You can see the data ranges +-12 columns whilst the upper info does not take more than 2 columns.

I hope this clarifies the issue,

Thank you in advance!

[Stephen23]

@Gerard Kluivers: screenshots are not very useful: we cannot load them as data, we cannot edit them, we cannot search them, we cannot test our code on them. Please provide us with a few sample files to work with by clicking the paperclip button.

[Gerard Kluivers]

My apologies, here I included a couple of files.

Thank you for commenting!

[Stephen23]

@Gerard Kluivers: contrary to the misleading file extensions, those files are not CSV formatted at all: they are really text files containing XML formatted data. Probably your best bet would be to import them using some XML importing function. You could try MATLAB's inbuilt xmlread:

https://www.mathworks.com/help/matlab/import_export/importing-xml-documents.html

or one of the third-party functions on FEX:

https://www.mathworks.com/matlabcentral/fileexchange?term=xml

[Nikhil Negi]

Hello Gerard,

from observing all the files you provided it looks like all your required data is between the < segment1 > and < /segment1 > of the text file. is it the case with all the files or just the 3 you provided.

if its the same with all the files you can use these strings as "milestones" to identify the data and then extract them.

[Guillaume]

Stephen is right that the files are absolutely not csv. The extensions are very misleading.

However, while structurally similar to XML, the files are also not valid XML and I suspect that an XML parser will choke on them (e.g. lines like Limits2,Direction=< will wreak havoc with the XML parsing)