3

How many significant figures are in 6.0*5.0 - 6.0*4.0?

Let's multiply first:

6.0 * 5.0 - 6.0 * 4.0
(6.0 * 5.0) - (6.0 * 4.0)
(30.) - (24)
6

The final answer comes out to have one significant figure because the last step involves subtraction in which the smallest precision of the two numbers was the one's place.

Now let's distribute and subtract first

6.0 * 5.0 - 6.0 * 4.0
6.0 * (5.0 - 4.0)
6.0 * (1.0)
6.0

Now the final answer has two significant figures because the subtraction involved more precise numbers with the same number of significant figures.

Which is correct, or are they both correct? Shouldn't the rules of significant figures prevent this from happening?

Ethan Bolker
  • 103,433
ElliotThomas
  • 190

1 Answers1

1

This is a phenomenon well-known in computing circles; subtracting two nearly equal floating-point numbers always results in a loss of relative precision (because the absolute precision remains about the same, but the magnitude of the result is less than the magnitude of either operand).

Much effort has been put into crafting numerical algorithms that minimise this loss by rearranging the sequence of operations.

TonyK
  • 68,059