Simple INT error?

[Roger]

In the debug window in Excel 2000, why does ? int( 0.6 * 100) give an answer of 59?

I know it's due to binary storage or something - but how do I correct it?

 

[keepitcool]

try:

? cint( 0.6 * 100) => 60
? .6*100 \1 => 60



keepITcool

< email : keepitcool chello nl (with @ and .) >
< homepage: http://members.chello.nl/keepitcool >

 

[J.E. McGimpsey]

I've never encountered that before. In XLv.X:

? int( 0.6 * 100)
60

 

[Tom Ogilvy]

It appears to be caused by come conversion that is being performed:

? int( 0.6 * 100)
59

? int( 0.6! * 100)
60
? int( 0.6# * 100)
59


0.6 is double
0.6# is double
0.6! is single.

 

[Harald Staff]

Another Intel error then. I haven't encountered this one either. Could anyone running an
Athlon processor please test this ?

It's easy enough to fix that particular calculation, problem is that INT is used a lot to
remove decimals from calculated big numbers (bigger than integer limits) and it would be
impossible to know when this happens or not. Hmmm. I have to proofread tons of code
because of this, buying Macs to my 3000 collegues is unfortunately no option <g>.

Best wishes Harald
Followup to newsgroup only please.

 

[J.E. McGimpsey]

buying Macs to my 3000 collegues is unfortunately no option <g>.

but have you tried??? Bet you could get a substantial discount!<g>