Thread Subject: Decimals from comma to dot

Hi,
I've got a file in ASCII format with decimal values comma seperated and columns are seperated by tab. The columns either have strings or floating numbers and my first column is in a ISO time-date format.
The file to be imported has column headers and titles which I don't need however, I do need all the string data and the floating number data from the general file.

I want to write a script to import this data, but as the floating points numbers are comma seperated I can't seem to manage. I previously wrote scripts to do this but up to now my decimals were always dot seperated. No matter what I do now I seem to be stuck. Here under some of the many functions I tried and what happened:
A) importdata - gives me a strucutre with one data field and one textdata field but almost all the strings have been trunckated and the numerical values are combined (i.e. 3,34 6,91 9,32 gives me 3 346 91932)
B) fopen - error message saying that the variable is undefined
C) textscan or textread - error message as the data is not recognised
D) strrep - I can't replace the string as I can't open the file

Does anyone have an idea???
Cheers
Fiona

"Fiona Buckley"
> I want to write a script to import this data, but as the floating points numbers are comma seperated I can't seem to manage...

a hint:
other CSSMers have proposed a PERL script to do the conversion before using any one of the ML input procedures...
look for PERL in this NG...

us

Look up the help for the fopen command to see how to use that, then read in
the string (fileread), then use strrep, followed by textscan.


"Fiona Buckley" <buckley_fiona@yahoo.com> wrote in message
news:gjvfqq$muv$1@fred.mathworks.com...
> Hi,
> I've got a file in ASCII format with decimal values comma seperated and
> columns are seperated by tab. The columns either have strings or floating
> numbers and my first column is in a ISO time-date format.
> The file to be imported has column headers and titles which I don't need
> however, I do need all the string data and the floating number data from
> the general file.
>
> I want to write a script to import this data, but as the floating points
> numbers are comma seperated I can't seem to manage. I previously wrote
> scripts to do this but up to now my decimals were always dot seperated. No
> matter what I do now I seem to be stuck. Here under some of the many
> functions I tried and what happened:
> A) importdata - gives me a strucutre with one data field and one textdata
> field but almost all the strings have been trunckated and the numerical
> values are combined (i.e. 3,34 6,91 9,32 gives me 3 346 91932)
> B) fopen - error message saying that the variable is undefined
> C) textscan or textread - error message as the data is not recognised
> D) strrep - I can't replace the string as I can't open the file
>
> Does anyone have an idea???
> Cheers
> Fiona

"us " <us@neurol.unizh.ch> wrote in message <gjvi08$qab$1@fred.mathworks.com>...
> "Fiona Buckley"
> > I want to write a script to import this data, but as the floating points numbers are comma seperated I can't seem to manage...
>
> a hint:
> other CSSMers have proposed a PERL script to do the conversion before using any one of the ML input procedures...
> look for PERL in this NG...
>
> us



Thanks for the hint ;o)
I found the program and it is very slick, a real savious. However, I still have a problem... as my file is ASCII, I get the following message:
            ??? Error using ==> replaceinfile at 63
            Can't do inplace edit on Down: File exists.
However, if I save my ASCII file as a txt file I then can run the script with no problem at all. Any idea what I could do to avoid doing the save as manipulation?

Cheers
Fiona

"Fiona Buckley" <buckley_fiona@yahoo.com> wrote in message <gk1uu1$8d7$1@fred.mathworks.com>...
> "us " <us@neurol.unizh.ch> wrote in message <gjvi08$qab$1@fred.mathworks.com>...
> > "Fiona Buckley"
> > > I want to write a script to import this data, but as the floating points numbers are comma seperated I can't seem to manage...
> >
> > a hint:
> > other CSSMers have proposed a PERL script to do the conversion before using any one of the ML input procedures...
> > look for PERL in this NG...
> >
> > us
>
>
>
> Thanks for the hint ;o)
> I found the program and it is very slick, a real savious. However, I still have a problem... as my file is ASCII, I get the following message:
> ??? Error using ==> replaceinfile at 63
> Can't do inplace edit on Down: File exists.
> However, if I save my ASCII file as a txt file I then can run the script with no problem at all. Any idea what I could do to avoid doing the save as manipulation?
>
> Cheers
> Fiona

Hi, can you give a more detailed example of what you are doing?
How do you call replaceinfile, which file names you have etc?
You can also email me directly (Assuming you use the replaceinfile function I posted), I'll be happy to try to fix any problems.

Perl is not included in my native languages, sorry.. It was originally made to do exactly what you are trying to do.

"Fiona Buckley" <buckley_fiona@yahoo.com> wrote in message <gjvfqq$muv$1@fred.mathworks.com>...
> Hi,
> I've got a file in ASCII format with decimal values comma seperated and columns are seperated by tab. The columns either have strings or floating numbers and my first column is in a ISO time-date format.
> The file to be imported has column headers and titles which I don't need however, I do need all the string data and the floating number data from the general file.
>
> I want to write a script to import this data, but as the floating points numbers are comma seperated I can't seem to manage. I previously wrote scripts to do this but up to now my decimals were always dot seperated. No matter what I do now I seem to be stuck. Here under some of the many functions I tried and what happened:
> A) importdata - gives me a strucutre with one data field and one textdata field but almost all the strings have been trunckated and the numerical values are combined (i.e. 3,34 6,91 9,32 gives me 3 346 91932)
> B) fopen - error message saying that the variable is undefined
> C) textscan or textread - error message as the data is not recognised
> D) strrep - I can't replace the string as I can't open the file
>
> Does anyone have an idea???
> Cheers
> Fiona

One possibility is to run this function as a separate step. It doesn't work with comma separated files and it will replace commas in the header (if there is one)
/ per

function comma2point_overwrite( filespec )
% replaces all occurences of comma (",") with point (".") in a text-file.
% Note that the file is overwritten, which is the price for high speed.
    file = memmapfile( filespec, 'writable', true );
    comma = uint8(',');
    point = uint8('.');
    file.Data( transpose( file.Data==comma) ) = point;
    delete(file)
end

> One possibility is to run this function as a separate step. It doesn't work with comma separated files and it will replace commas in the header (if there is one)
> / per
>
> function comma2point_overwrite( filespec )
> % replaces all occurences of comma (",") with point (".") in a text-file.
> % Note that the file is overwritten, which is the price for high speed.
> file = memmapfile( filespec, 'writable', true );
> comma = uint8(',');
> point = uint8('.');
> file.Data( transpose( file.Data==comma) ) = point;
> delete(file)
> end


Hi Per, thanks for this, I had exactly the same problem as Fiona. With this function I can now read in the .asc files without any problem using:

comma2point_overwrite( filespec )
[data] = importdata([filespec],'\t',headerlength);

You saved me (and surely others) from eternal hours of struggle! :-)

> One possibility is to run this function as a separate step. It doesn't work with comma separated files and it will replace commas in the header (if there is one)
> / per
>
> function comma2point_overwrite( filespec )
> % replaces all occurences of comma (",") with point (".") in a text-file.
> % Note that the file is overwritten, which is the price for high speed.
> file = memmapfile( filespec, 'writable', true );
> comma = uint8(',');
> point = uint8('.');
> file.Data( transpose( file.Data==comma) ) = point;
> delete(file)
> end


Hi Per, thanks for this, I had exactly the same problem as Fiona. With this function I can now read in the .asc files without any problem using:

comma2point_overwrite( filespec )
[data] = importdata([filespec],'\t',headerlength);

You saved me (and surely others) from eternal hours of struggle! :-)

just use notepad 2 (download http://www.flos-freeware.ch/notepad2.html)
it replaces commas with dots much faster an much simpler.

or simply in the international settings of your operating system, switch them to a comma-delimiting-decimal system. For example to the italian setting.

This method could be faster if u have to import many files and if they are big.

Oleg

This script is great!!

I also need to get rid of commas, but I'm hoping to find a solution that doesn't overwrite the original file. Does anyone know where I could find that?


"per isakson" <poi.nospam@bimDOTkthDOT.se> wrote in message <gk2tkh$64l$1@fred.mathworks.com>...
> "Fiona Buckley" <buckley_fiona@yahoo.com> wrote in message <gjvfqq$muv$1@fred.mathworks.com>...
> > Hi,
> > I've got a file in ASCII format with decimal values comma seperated and columns are seperated by tab. The columns either have strings or floating numbers and my first column is in a ISO time-date format.
> > The file to be imported has column headers and titles which I don't need however, I do need all the string data and the floating number data from the general file.
> >
> > I want to write a script to import this data, but as the floating points numbers are comma seperated I can't seem to manage. I previously wrote scripts to do this but up to now my decimals were always dot seperated. No matter what I do now I seem to be stuck. Here under some of the many functions I tried and what happened:
> > A) importdata - gives me a strucutre with one data field and one textdata field but almost all the strings have been trunckated and the numerical values are combined (i.e. 3,34 6,91 9,32 gives me 3 346 91932)
> > B) fopen - error message saying that the variable is undefined
> > C) textscan or textread - error message as the data is not recognised
> > D) strrep - I can't replace the string as I can't open the file
> >
> > Does anyone have an idea???
> > Cheers
> > Fiona
>
> One possibility is to run this function as a separate step. It doesn't work with comma separated files and it will replace commas in the header (if there is one)
> / per
>
> function comma2point_overwrite( filespec )
> % replaces all occurences of comma (",") with point (".") in a text-file.
> % Note that the file is overwritten, which is the price for high speed.
> file = memmapfile( filespec, 'writable', true );
> comma = uint8(',');
> point = uint8('.');
> file.Data( transpose( file.Data==comma) ) = point;
> delete(file)
> end

"per isakson" <poi.nospam@bimDOTkthDOT.se> wrote in message <gk2tkh$64l$1@fred.mathworks.com>...
> "Fiona Buckley" <buckley_fiona@yahoo.com> wrote in message <gjvfqq$muv$1@fred.mathworks.com>...
> > Hi,
> > I've got a file in ASCII format with decimal values comma seperated and columns are seperated by tab. The columns either have strings or floating numbers and my first column is in a ISO time-date format.
> > The file to be imported has column headers and titles which I don't need however, I do need all the string data and the floating number data from the general file.
> >
> > I want to write a script to import this data, but as the floating points numbers are comma seperated I can't seem to manage. I previously wrote scripts to do this but up to now my decimals were always dot seperated. No matter what I do now I seem to be stuck. Here under some of the many functions I tried and what happened:
> > A) importdata - gives me a strucutre with one data field and one textdata field but almost all the strings have been trunckated and the numerical values are combined (i.e. 3,34 6,91 9,32 gives me 3 346 91932)
> > B) fopen - error message saying that the variable is undefined
> > C) textscan or textread - error message as the data is not recognised
> > D) strrep - I can't replace the string as I can't open the file
> >
> > Does anyone have an idea???
> > Cheers
> > Fiona
>
> One possibility is to run this function as a separate step. It doesn't work with comma separated files and it will replace commas in the header (if there is one)
> / per
>
> function comma2point_overwrite( filespec )
> % replaces all occurences of comma (",") with point (".") in a text-file.
> % Note that the file is overwritten, which is the price for high speed.
> file = memmapfile( filespec, 'writable', true );
> comma = uint8(',');
> point = uint8('.');
> file.Data( transpose( file.Data==comma) ) = point;
> delete(file)
> end

Thank you Per! This is a great help!