Significant Figures

Unfortunately, there is no hard-and-fast rule to use when determining the precision of the coefficients of a best-fit equation.  If we knew what individual operations were used to perform the regression, we could certainly determiner the sig-figs using the standard techniques.  We are not going to be learning the actual regression algorithm, however, so we must develop some other standard for determining how many significant figures to use when analyzing a regression.

Even though we will not concern ourselves with exactly how, it is nevertheless clear that the regression is based on the numerical data in the calculator's list variables.  So in order to determine the precision of the regression coefficients, we must turn our attention to the list elements themselves.  When analyzing a regression, we will follow this rule of thumb:
 

"The number if significant figures in the regression coefficients is the highest number which is less than or equal to the number of significant figures used in two thirds of the data set."
While this may seem cumbersome, it is not all that difficult to apply in practice.

For example, consider the sample data set:
 
 

Sample d vs. t Data
Time 
(s)		 Distance 
(m)
0.00 
0.25 
0.50 
0.75 
1.00 
1.25 
1.50 
1.75 
2.00 
2.25 
2.50 		  0.19 
 3.48 
 6.30 
 8.86 
 9.84 
11.39 
12.09 
11.82 
10.96 
 9.21 
7.20 
 

Of the 22 data points in this table:

So, 17 of the 22 have 3 or more sig-figs.  According to the rule, the regression coefficients would have 3 significant figures as well:
 
So:    a = -9.91 m/s2 
       vo = 15.3 m/s 
       do = 0.0181 m 

Remember, the 2/3 is just a rule of thumb.  It is not necessary to calculate the fraction exactly.  The idea is, we don't want to give up a sig-fig just because a couple of  points in a large data set are less well known than the others.  On the other hand, it would be misleading to suggest that our overall answer is known with as much precision as the two or three most precise numbers me measured.

Back to   Analysis Page