unicode2ascii

Converts unicode endcoded files to ASCII encoded files

Description

UNICODE2ASCII Converts unicode endcoded files to ASCII encoded files
  UNICODE2ASCII(FILE)
   Converts the file to ASCII (overwrites the old file!)
  UNICODE2ASCII(SOURCEFILE, DESTINATIONFILE)
   Converts the contents of SOURCEFILE to ASCII and writes it to DESTINATIONFILE
  ASCIISTRING = UNICODE2ASCII('string', UTFSTRING)
   Converts the UTFSTRING to ASCII and returns the string.

The unicode header and all 0-bytes will be deleted. If there are characers with encoding > FF then the file will continue to contain garbage because there is no ASCII representation for those characters. But about 99% of the files should convert ok.

• MATLAB release: MATLAB 6.5 (R13)

[jf b]

 

[Gordon]

combined this with csvimport and it worked a treat

[Sergey]

Thanks! It's clean, fast, and most important - works )

[Mark Patterson]

Works well. The included isunicode function does not correctly release the file handle when it returns. The attached diff correct this.

--- isunicode.m 2010-09-20 18:25:56 +0000
+++ isunicode.m 2010-09-27 21:11:21 +0000
@@ -13,30 +13,22 @@
%
% (c) Version 1.0 by Stefan Eireiner (<a
href="mailto:[e-mail]?subject=isunicode">stefan.eirei
[e-mail]</a>)
% last change 10.04.2006
-%
-% Downloaded from:
-%
http://www.mathworks.com/matlabcentral/fileexchange/10686-unicode2ascii.
-%
-% See also: unicode2ascii.

isuc = false;
-if(nargin == 2)
- if(strcmpi(filename, 'string'))
- firstLine = varargin{1}(1:4);
+if nargin == 2 && strcmpi(filename, 'string')
+ firstLine = varargin{1}(1:4);
+else
+ fileInfo = dir(filename);
+ if(fileInfo.bytes < 4) % a unicode file incl. header can't be
smaller than 4 bytes if it shall display at least one char.
+ return;
    end
-end
-
-if(~exist('firstLine', 'var'))
    fin = fopen(filename,'r');
    if (fin == -1) %does the file exist?
        error(['File ' filename ' not found!'])
        return;
    end
- fileInfo = dir(filename);
- if(fileInfo.bytes < 4) % a unicode file incl. header can't be
smaller than 4 bytes if it shall display at least one char.
- return;
- end
    firstLine = fread(fin,4)';
+ fclose(fin) ;
end

% assign all possible headers to variables
@@ -58,7 +50,3 @@
elseif(strfind(firstLine, utf32leheader) == 1)
        isuc = 5;
end
-
-if(~exist('firstLine', 'var'))
- fclose(fin);
-end