I think that I have found a second related bug that is in both Excel
2003 and Excel 2007. Namely, if you key in the following:
=DEC2HEX(2^16), it correctly gives 10000 (hex)
=DEC2HEX(2^16-1), it correctly gives FFFF (hex)
=DEC2HEX(65535), it correctly gives FFFF (hex)
=DEC2HEX(850*77.1), it incorrectly gives FFFE (hex)
That's no bug, as:
number value stored in FPU value of that
850 0x408a900000000000 850
77.1 0x4053466666666666 77.09999999999999431566
850*77.1 0x40efffdfffffffff 65534.99999999999272404239
65535 0x40efffe000000000 65535
As you see, 77.1 is stored as a periodic binary representation. You can
find the very same problem on your cheap 8-digit or 10-digit calculator
by entering 1 / 3 * 3 =, that is, (1/3) * 3: it will show 0.999999999
instead of 1. Just like it's impossible to represent 1/3 correctly in
the decimal system, 77.1 can't be represented correctly in binary. See
the series of 6666666666 in the number... they are just like the
33333333333 in 1/3 in decimal. When multiplying that so that it SHOULD
cancel that stuff out, a roundoff error causes it to end up as ffffffff
instead of 00000000.
And DEC2HEX apparently always rounds down, so
DEC2HEX(65534.99999999999999) = DEC2HEX(65534). One could argue that the
function should better round to the nearest number, but that decision is
up to the author of such a function.