
Splitting Hairs
over Precision
There is a really interesting short
article in the February 13, 2006, edition of
EETimes titled,
"Humans
Still Thrive on Decimals," by Clive (Max) Maxfield, that illustrates
by example how binary mathematics can make simple numerical operations
troublesome. Drawing on the extensive collection of works by IBM Fellow
Mike Cowlishaw’s "General
Decimal Arithmetic" website, Max shows how, for instance, the "double"
floating point real number type cannot precisely represent 0.1. Multiplying
0.1 x 8 returns, to full precision, 0.8000000000000000444089209850062616169452667236328125.
However, adding 0.1 to itself 8 times yields an entirely different result,
0.7999999999999999333866185224906075 458209991455078125.
Theoretically,
an infinite number of binary bits is required to precisely represent
many seemingly simple decimal values. Having written a fair amount of
engineering software that requires carrying calculations to many decimal
places, I have run into this problem often. It can be a real headache
to resolve in a manner that does not cause users to question the validity
of results. For instance, consider the case of calculating frequency
conversions where both very high and very low frequencies are employed
in the same calculation. The two sets of numbers (maybe three sets,
depending on the relative frequency of the local oscillator) may be
separated by six or even nine orders of magnitude. Many decimal places
are required to maintain precision at both extremities. Nobody I work
with will accept an answer that turns what should be, say, 5.000000
kHz into 4.999999 kHz. It just doesn’t feel right even though we know
it is "close enough."
Because of the limitations imposed by
our finite systems, financial calculations are by law required to carry
out using binary coded decimal (BCD) operations. Doing so adding more
bits to represent each number, but the phenomenon of binary approximation
is eliminated. Financials can face the same number extremities as engineering
calculations when you consider the case of calculating the trillion
dollar gross domestic product of a country like the U.S. down to the
penny ($1,000,000,000,000.01). Calculation times increases significantly
when using BCD arithmetic, and can take 1000 times or more longer than
floating point operations. Of course, special circuitry could be used
(FPGAs) to implement calculators whose only function is to do binary
math and not floating point.
The story reminds me of a simple
test that was popular in the 1980s as a cursory means of determining
whether a particular handheld calculator was worthy of engineering pursuits.
Perform the operation 1 / 3 * 3 =. If the answer was 1.00000000, then
you could buy the calculator. If it was 0.9999999, then it was time
to scoff at it in an elitist way and move on – let the knaves use those.
A huge collection of my 'Factoids' can be accessed from my 'Kirt's Cogitations'
table of contents.
Topical Smorgasbord, another manifestation of Factoids,
are be found on these pages:
 1
 2 
3 
4  5
 6  7
 8  9
 10 
11  12 
13  14
 15 
16  17 
18  19
 20  
21  22
 23 
24  25 
26  27
 28 
29  30 
31  32
 33 
34  35 
All pertain to topics that are related to the general engineering and science theme
of RF Cafe.







Copyright:
1996  2018 Webmaster:
Kirt Blattenberger, BSEE  KB3UON 
RF Cafe began life in 1996 as "RF Tools" in an AOL screen name web space totaling
2 MB. Its primary purpose was to provide me with ready access to commonly needed formulas
and reference material while performing my work as an RF system and circuit design engineer.
The Internet was still largely an unknown entity at the time and not much was available
in the form of WYSIWYG
...
All trademarks, copyrights, patents, and other rights of ownership to images and text
used on the RF Cafe website are hereby acknowledged.
My Hobby Website: AirplanesAndRockets.com

