As you probably know, floating point mathematics can give you some really weird results.
SQL Server SQL Server does some fun things with approximation. Consider this example:
DECLARE @a FLOAT, @b FLOAT SELECT @a = 3.2, @b = 1.5 SELECT @a SELECT @b SELECT @a + @b |
Results:
3.2000000000000002 1.5 4.7000000000000002 |
What the ... ? Since FLOAT can't represent 3.2 exactly, it approximates its value. Trust me, this has thrown many people into bottomless pits of confusion, as it becomes extremely significant for scientific, mathematical and financial calculations. Luckily, we have recourse; if you use DECIMAL instead, no problems:
DECLARE @a DECIMAL(5, 2), @b DECIMAL(5, 2) SELECT @a = 3.2, @b = 1.5 SELECT @a SELECT @b SELECT @a + @b |
Results:
You should try to avoid the FLOAT datatype whenever possible, and opt for the more versatile, and precise, DECIMAL or NUMERIC datatypes instead. DECIMAL and NUMERIC are functionally equivalent, so you can interchange them — however, I use DECIMAL whenever I will be keeping decimal points, because I think it more accurately describes the data. If I don't care about decimal points, I'll use INT or BIGINT, depending on the requirements.
Another scenario you might come across is representing numbers in scientific notation. Let's say you have this:
DECLARE @num VARCHAR(30) SET @num = '3.1415926E-05' |
What if you want to present this in its true decimal form, without all the "weird" formatting?
| SELECT CONVERT(DECIMAL(15, 12), STR(@num, 15, 12)) |
Result:
----------------- .000031415926 |
VBScript Here is a simple example that displays how a mathematical result that ends up as a decimal (no whole number) takes on a mind of its own:
<% a = 30.754 - 30 response.write a %> |
Result: 0.754000000000001
This floating point inaccuracy is unresolved at this point, according to Microsoft (see
KB #165373). Here is a
discussion about this phenomenen.
To work around the issue, you can use formatnumber to eliminate extraneous decimal points:
<% a = cdbl(formatnumber(30.754 - 30, 3)) response.write a %> |
Result: 0.754