MATLAB Central Newsreader - quick question about 'textscan'

quick question about 'textscan'

Apurva Dave — Mon, 14 Sep 2009 17:15:06 -0400

Hi,

I'm using textscan to read some tab-delimited data. I want to ignore the metadata as well as some comments. I'm using the following syntax, but I'm getting an 'unknown option' error when I try to use 'commentstyle':

fid = fopen('P980413A.DAT');

textfile = textscan(fid, '%f', 28, 'delimiter',' ', ...
'commentstyle', {'/begin_header /end_header@'}, ...
'commentstyle', '/begin_data', ...
'commentstyle', '/end_data@');

fclose(fid);

As far as I can tell, I've entered it all correctly. Parameters are not case-sensitive, although I also tried different versions just to check. Is this a bug in the matlab? I tried restarting.

'headerlines' isn't a good alternative for me, b/c the metadata for each data file aren't uniform. In any case I still need to find a way to ignore comments interspersed through the file. Any thoughts?

Thanks,
Apurva

Re: quick question about 'textscan'

dpb — Mon, 14 Sep 2009 17:30:33 -0400

Apurva Dave wrote:
> Hi,
>
> I'm using textscan to read some tab-delimited data. I want to
> ignore the metadata as well as some comments. I'm using the following
> syntax,but I'm getting an 'unknown option' error when I try to use 'commentstyle':
>
> fid = fopen('P980413A.DAT');
>
> textfile = textscan(fid, '%f', 28, 'delimiter',' ', ...
> 'commentstyle', {'/begin_header /end_header@'}, ...
> 'commentstyle', '/begin_data', ...
> 'commentstyle', '/end_data@');
...

textscan() postdates my version and don't have time to look at web doc
right now, but does it indicate can have multiple 'commentstyle' entries
as allowable syntax? That's unusual at least w/ any ML function I'm
aware of....

--

Re: quick question about 'textscan'

Apurva Dave — Mon, 14 Sep 2009 17:48:19 -0400

yup, that would be my problem. thank you!

does anyone have a suggestion for how to deal with metadata and comments by calling distinct paramaters (ie. without trying to call 'commentstyle' multiple times). once again, 'headerlines' isn't a good option for me b/c my metadata vary from file to file.

thanks,
apurva

Re: quick question about 'textscan'

Rune Allnor — Mon, 14 Sep 2009 17:59:54 -0400

On 14 Sep, 19:48, "Apurva Dave" <|remove.this|apurva.dave|
remove.th...@duke.edu> wrote:
> yup, that would be my problem. thank you!
>
> does anyone have a suggestion for how to deal with metadata and comments by calling distinct paramaters (ie. without trying to call 'commentstyle' multiple times). once again, 'headerlines' isn't a good option for me b/c my metadata vary from file to file.

Don't use TEXTSCAN.

It seems your data might be complicated enough to warrant
you writing a dedicated parser.

Rune

Re: quick question about 'textscan'

Apurva Dave — Mon, 14 Sep 2009 18:33:04 -0400

Thanks Rune, but that seems like something that's beyond my abilities right now...

Is there anyway I can make edits to my data files? E.g. within a loop, open each data file with read/write permissions, find and remove all metadata and comments? I know it's possible to fopen text files in read/write mode, but I don't know how to proceed from there...

I have ~200 data files, so opening them up in an editor and making changes myself is not an option...

apurva

Re: quick question about 'textscan'

Rune Allnor — Mon, 14 Sep 2009 18:49:55 -0400

On 14 Sep, 20:33, "Apurva Dave" <|remove.this|apurva.dave|
remove.th...@duke.edu> wrote:

> Is there anyway I can make edits to my data files? E.g. within a loop, open each data file with read/write permissions, find and remove all metadata and comments?

Yes, there is: Write a parser.

Rune

Re: quick question about 'textscan'

Apurva Dave — Mon, 14 Sep 2009 20:22:17 -0400

ok, will give it a shot. thanks!
apurva

Re: quick question about 'textscan'

Andres — Mon, 14 Sep 2009 21:33:01 -0400

It looks like you (just) want to read numerical data with a variable number of header lines. It may be worth giving txt2mat from the mathworks file exchange a try.