Field Definitions - problem with floating point calculations

[dhstein]

I had a field defined as a Long Integer which I realized needed to be a
"Double". So I changed it and in VBA code the values I'm using for the field
are also "Double". But I'm getting completely wrong results on the
calculations and I'm sure it's related to my field definitions. For example
a field which has a value of 1100 - I subtract 16.67 in the code and get a
result of 54166.5. So can someone help me to define what the field
definition should be and how the corresponding VBA variables should be
defined? Thanks for any help on this.

 

[Douglas J. Steele]

I suspect it's your code, not your field definitions. What does your code
look like?

 

[Peter Hibbs]

Perhapsyou should look at
http://www.fmsinc.com/tpapers/math/index.html
for an answer.

HTH

Peter Hibbs.

 

[Douglas J. Steele]

While Luke's essay on floating point numbers is good, it certainly wouldn't
explain how 1100 - 16.67 would result in 54166.5.

There has to be something wrong with the code.

 

[Peter Hibbs]

You are right, of course, but it might be useful to know when you
using floating point numbers in calculations.

Peter Hibbs.