Float "corruption" on read/write to .TXT

So I'm trying to create a permanent archive of some brainwaves in .TXT format. The raw data is recorded as in a proprietary format and the hardware manufacturer has provided an ActiveX control that lets you query the data in Matlab.

The query returns the waveform as a vector of single precision floats (because that's what the medical hardware records). I save these to .TXT fprintf and specifying %f for the data. Matlab write this as a double and all seems to be well.

Since these .TXT files will be a permanent archive on your laboratory server, I wanted to do some quality checks. Obvious, these things should match: 1) The waveform on the proprietary software 2) The waveform as queried by Matlab 3) The waveform from the .TXT files as read by in by Matlab

2 & 3 don't match. Numerous entries in every vector fail <eps checks and the plots() change in very subtle ways. What's going on and what might I try to play with?

My first thought is that it has to do with the single->double conversion. However, when I compare the query to double(query) the values are identical. I think the write/read is the source of the trouble.

Answer 1

[Jan]

Which fprintf format are you using to write the TXT file?

It would be much better to store the single data and single data instead of converting them to strings, which represent decimal values. There is no 100% accurate conversion from binary to decimal numbers, when a limite dnumber of digits is used. Therefore this conversion will introduce some noise and there is no chance to avoid this.

In opposite to this the single->double conversion is stable. Each value in single format can be represented exactly as a double, but not the other way around.

If you store the single values with 8 digits, you loose 1 ro 2 bit precision, but this should be still more precise than the AD-sampling from the EEG-electrodes - usually this sampling has 10 to 16 bits only. Therefore I would not expect this conversion to be a significant source of noise. Of course the displayed curves look a little bit different, but this is a porblem of the very coarse resultion on the monitor only:

plot(1:3, [-eps, 0, eps])

I cannot test this currently, but I assume the above command does not draw an exactly horizontal line. Perhapse "eps('single')" is required. But numerically the line is as good as horizontal. If your diagram has 1024 pixels, this means a resampling with 10 bits. This resampling introduces a noticable noice. But when you zoom into the diagram, this noise gets smaller.

However, storing the data in ASCII-format (with a high enough number of digits, which is determined by an numerical analysis of the noise) has an advantage compare to the accurate storing of the binary values: The ASCII files can be read in 10 years also even if the documentation of the file format has been lost. They can be read in Matlab 2022b and Fortran2032++, in D and D++ also. If you store the values in a binary format, be sure to create an extremly well documentation of the format.