Floating Point

[Kathleen Bell]

Years ago I heard about the way Excel stores numbers. I wrote it off as
something accountants would never have to deal with since we weren't dealing
with high precision numbers. Well...

This past week I tried to upload a journal entry that had about 15K lines.
The computer rejected it as not balancing. In the ninth and tenth decimal
places there was a 15. We tried everything. It became a "class project".
We tried rounding up, rounding down, just plain rounding, truncating. We
copied as values, copied as text and then recopied as numbers. Finally we
found a website that talked about the floating point, something which
something that sort of came out of the recesses of my memory. There was no
other explanation.

I split the journal entry into three and somehow the new combination of
numbers got rid of the phantom number way out there and they went through.

 

[JE McGimpsey]

take a look here:

http://www.mcgimpsey.com/excel/pennyoff.html

 

[joeu2004]

Finally we found a website that talked about the
floating point, something which something that sort
of came out of the recesses of my memory. There
was no other explanation.

I split the journal entry into three and somehow the
new combination of numbers got rid of the phantom
number way out there and they went through.

Do you have a question? Perhaps it would help to read
the following KB articles:

http://support.microsoft.com/kb/214118/en-us
http://support.microsoft.com/kb/78113/en-us
http://support.microsoft.com/kb/57298/en-us