Thread Subject: Approximation problem???

Hi,

I am having a really weird problem that I cant find a reason. After a lot of calculation I was getting a wrong value for a variable, so I debbuged to find where was the problem, and I found a simple division mistake. A double h=0.0030 and another double hh=1/h .
Of course hh is suposed to be 333.3333 but I am getting hh=333.8898
Here is the piece of code:

                h=str2double(cell2mat(raw(nameLinNum,densColumn)));
                hh=1/h;

Does anyone hava an idea on how to fix it???

Thanks

Tales wrote:
...
> I am having a really weird problem that I cant find a reason. After a
> lot of calculation I was getting a wrong value for a variable, so I
> debbuged to find where was the problem, and I found a simple division
> mistake. A double h=0.0030 and another double hh=1/h .
> Of course hh is suposed to be 333.3333 but I am getting hh=333.8898
> Here is the piece of code:
>
> h=str2double(cell2mat(raw(nameLinNum,densColumn)));
> hh=1/h;
>
> Does anyone hava an idea on how to fix it???
...

Undoutedly h _isn't_ 0.00300000... but closer to 0.002995...

Try

disp(h-0.0030)

during a debugging session and see what is returned.

Why that is isn't determinable from the data provided but delving into
where the underlying data come from and how they're processed will
surely uncover same.

--