I know I have to use integer math when working with variables but just want to clarify that I can still do something like: var1 = var2 * .3333  The integer portion of the answer should be correct but the decimal will be truncated?
Announcement
Collapse
No announcement yet.
Quick question regarding integer math.
Collapse
X

You can't write a constant as '.3333'.
You can write a variety of equivalent calculations:
Code:var1 = var2 / 3 var1 = var2 ** 21843 '(var1 = (var2 * 21843) / 65536) var1 = var2 */ 85 '(var1 = (var2 * 85) / 256)

Thanks, Charles
What I actually have is:
Atemp = Atemp + TempHistory[Hcount]*(Mincount/60)
Mincount can be between 0 and 60 so divided by 60 it will always be a fraction. So if Mincount = 20 it would equate to .333.... This seems to work within the equation. So I understand what your saying is the to use .3333 is not allowed because it is a constant but occurring in the course of a calculation it would work truncating everything after the decimal point?
Comment

You must consider each operation separately. Because of your placement of parentheses, (Mincount/60) will always be performed first, and will always yield 0 because it's always a fraction.
You should do the multiplication first in order to keep a workable value:
Atemp = Atemp + ((TempHistory[Hcount]*Mincount)/60)
The problem then becomes the maximum result of the multiplication, which can't be allowed to exceed the capability of the largest variable type. If you're compiling with LONG variables enabled, this usually isn't an issue. If compiling with only WORDs, then TempHistory[Hcount] can't exceed 1092, because 1093 * 60 = 65580.
The question is, what is the maximum possible value for TempHistory[Hcount]?
Comment

I'm using it like this where Atemp is a word that's 010000 and MinCount is a byte that can be 160 and Loopcount is a byte that's always less than 24.
I want:: Atemp = Atemp / ((Loopcount 1) + MinCount/60)
Use: Atemp = Atemp / ((Loopcount 1)*60 + MinCount
Comment

The 'want' and 'use' formulas are not mathematically equivalent.
The 'want' formula could use a scaling of 60 to remove the error attached to the MinCount division. This may be what you were getting at with the 'use' formula:
Code:Temp1 = ((Loopcount  1) * 60) + MinCount Temp2 = Atemp * 60 Atemp = DIV32 Temp1
Comment
Comment