Prevent rounding of a number.

5 views (last 30 days)
Matt
Matt on 15 Nov 2012
I'm using the following code to check and correct extreme values in a matrix. The maximum value can not be 1, it must be one quantisable segment below 1. In this example bitDepth is 16 and so maxValue should calculate to be the number commented after it. However, MATLAB just rounds maxValue up to 1 and so my code is failing further along. How can I make MATLAB keep the accuracy of this number?
%Check for clipping
maxValue = 1-(1/2^bitDepth); %0.99998474121093744448884876;
sampleValue = inputBuffer(currentFrameSample, currentChannel);
if sampleValue > maxValue
inputBuffer(currentFrameSample, currentChannel) = maxValue;
elseif sampleValue < (-1.0)
inputBuffer(currentFrameSample, currentChannel) = (-1.0);
end
Thanks in advance, Matt.

Accepted Answer

Matt Fig
Matt Fig on 15 Nov 2012
Edited: Matt Fig on 15 Nov 2012
Why do you think MATLAB is rounding 0.99998474121093744448884876 to 1?
F = 0.99998474121093744448884876;
F==1 % No
F<1 % Yes
% Let's look closely
fprintf('\n\n\t%17.17f\n\n',1-(1/2^16))

More Answers (1)

Matt
Matt on 15 Nov 2012
Feeling very stupid right now -_-
Thanks very much for you're help! I was convinced it was the rounding that was causing an issue but after you showed me the value was infact correct, I found it was actually a problem further down the code!

Categories

Find more on 3-D Scene Control in Help Center and File Exchange

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!