Floating point maths

Description

Due to the Floating Point Error of the mathematical co-processor in your computer you will not be able to sum or compare a floating point variable to an "exact" amount. There is always a tiny little difference. This is not unique to Liberty, all programs have to deal with this.

Liberty keeps two types of numbers, integers, which have nothing after the decimal point and floats, which have an amount after the decimal point. If you see 2 listed you can be sure it is held as an integer. If you see 2.0 listed it is held as a float and there is a tiny amount after the decimal. Floats are held to 16 decimal places, right click in the debugger and click on the option to view the full number. Most often floats are truncated for display.

Example 1

Example code to demonstrate the bug.

a = 2.1 - 2
b = 0.1
print
print a, b
print
print "(2.1 - 2) = 0.1 (should print 1 (True)) ... "; a = b
print
print using("#.################################", a)
print using("#.################################", b)

Example code to work around the bug.

a = 2.1 - 2
b = 0.1
precision = 1e-16
print
print a, b, precision
print
print "(2.1 - 2) = 0.1 (should print 1 (True)) ... "; abs(a - b) < precision
print
print using("#.################################", a)
print using("#.################################", b)

You should never try to compare floating point numbers for being equal. If you must, compare them to a level of precision as shown. 1e-16 is the narrowest of margins, you might choose 1e-7 to allow a wider margin of error. Better still, don't try and compare floats, use integers.

Example 2

Example code to demonstrate the bug.

    for i = 1 to 2 step 0.1
        print i
    next

Expected result:

printed numbers from 1 to 2 with step 0.1

What actually happens:

printed numbers from 1 to 1.9 with step 0.1

Example code to work around the bug.

    for i = 0 to 10
        num = 1 + i * 0.1
        print num
    next

Looping using integer numbers provides an exact number of steps.