Thread Subject:

Read in a flat file, space delimited, NaNs as "."

Hi Everyone:

I am having some difficulty trying to read in a file with the
following entries (actually a flat file produced by SAS). Note that
SAS writes "." instead of NaN for null values. There is no header and
I cannot edit the files on disk to replace the nulls appropriately.

Here is an example:

XXX1LZ1 2.66923195712975
XXX1NS1 -.42258512166855
XXX1OT1 0.03028640299762
XXX1PF1 .
XXX1UG1 -.09154710877974
XXX1W51 .
XXX1X31 .
XXX24N1 .
XXX28Z1 .
XXXAAC1 0.83949752862828
XXXAAH1 0.79328676386255


Honestly, any help is appreciated.

Cheers,

t.

In article
<a4e6a593-cfa3-47d8-a10c-dfc37a5073b4@w4g2000vbc.googlegroups.com>,
 Tyler <hayes.tyler@gmail.com> wrote:

> Hi Everyone:
>
> I am having some difficulty trying to read in a file with the
> following entries (actually a flat file produced by SAS). Note that
> SAS writes "." instead of NaN for null values. There is no header and
> I cannot edit the files on disk to replace the nulls appropriately.
>
> Here is an example:
>
> XXX1LZ1 2.66923195712975
> XXX1NS1 -.42258512166855
> XXX1OT1 0.03028640299762
> XXX1PF1 .
> XXX1UG1 -.09154710877974
> XXX1W51 .
> XXX1X31 .
> XXX24N1 .
> XXX28Z1 .
> XXXAAC1 0.83949752862828
> XXXAAH1 0.79328676386255
>
>
> Honestly, any help is appreciated.
>
> Cheers,
>
> t.

If your file is in data.txt:

  fid = fopen('data.txt','rt');
  data = textscan(fid,'%s%f','TreatAsEmpty','.');
  fclose(fid);

data{1} will be a column vector of strings, data{2} a column vector of
numbers with NaNs where the periods are.

--
Doug Schwarz
dmschwarz&ieee,org
Make obvious changes to get real email address.

I'm hitting the same problem but I found that non-empty fields starting with '.' then cause problems. It looks like TreatAsEmpty does not match the whole field but only the leading part of the field!

To repeat, the solution given here doesn't work for lines like
XXX1LZ1 .234
What happens is that the '.' is treated as a NaN and then "234" is the next id.

I believe handling of TreatAsEmpty should be modified to match the whole field. If this cannot be done, then the rest of any field treated as empty should be discarded. As it is, textscan can split the input at a point not matching the delimiter.

>> s=sprintf('a .2\nb .\n')

s =

a .2
b .


>> A=textscan(s, '%s %f')

A =

    {2x1 cell} [0.2000]

>> A=textscan(s, '%s %f', 'TreatAsEmpty', '.')

A =

    {3x1 cell} [3x1 double]

>> A{1}

ans =

    'a'
    '2'
    'b'

>>


Doug Schwarz <see@sig.for.address.edu> wrote in message <see-3301CD.23483323012012@news.frontiernet.net>...
> In article
> <a4e6a593-cfa3-47d8-a10c-dfc37a5073b4@w4g2000vbc.googlegroups.com>,
> Tyler <hayes.tyler@gmail.com> wrote:
>
> > Hi Everyone:
> >
> > I am having some difficulty trying to read in a file with the
> > following entries (actually a flat file produced by SAS). Note that
> > SAS writes "." instead of NaN for null values. There is no header and
> > I cannot edit the files on disk to replace the nulls appropriately.
> >
> > Here is an example:
> >
> > XXX1LZ1 2.66923195712975
> > XXX1NS1 -.42258512166855
> > XXX1OT1 0.03028640299762
> > XXX1PF1 .
> > XXX1UG1 -.09154710877974
> > XXX1W51 .
> > XXX1X31 .
> > XXX24N1 .
> > XXX28Z1 .
> > XXXAAC1 0.83949752862828
> > XXXAAH1 0.79328676386255
> >
> >
> > Honestly, any help is appreciated.
> >
> > Cheers,
> >
> > t.
>
> If your file is in data.txt:
>
> fid = fopen('data.txt','rt');
> data = textscan(fid,'%s%f','TreatAsEmpty','.');
> fclose(fid);
>
> data{1} will be a column vector of strings, data{2} a column vector of
> numbers with NaNs where the periods are.
>
> --
> Doug Schwarz
> dmschwarz&ieee,org
> Make obvious changes to get real email address.