An Investigation of the Median-Median Method of Linear Regression

Journal of Statistics Education Volume 14, Number 2 (2006), jse.amstat.org/v14n2/morrell.html

Copyright © 2006 by Elizabeth J. Walters, Christopher H. Morrell, and Richard E. Auer, all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the authors and advance notification of the editor.

Key Words: Least squares line.

Abstract

1. Introduction

The second trend is the introduction of statistical ideas earlier in the school curriculum. Younger students are being exposed to the question of what truths lurk beneath the surface of data. These students are now being trained to use statistics as the primary tool for researching any topic within any field.

Evidence of both trends is seen when elementary and secondary school students use graphing calculators to perform data analysis on exams and homework assignments. Much of the early statistical training is being accomplished by teaching the basics of what has been termed exploratory data analysis (EDA).

As the first pioneer in exploratory data analysis, Tukey (1971, p. v) effectively defined EDA as “… looking at data to see what it seems to say. It concentrates on simple arithmetic and easy-to-draw pictures. It regards whatever appearances we have recognized as partial descriptions, and tries to look beneath them for new insights.” In his recommendations for what type of activity belongs in introductory statistics coursework, Hahn (1988, p. 27) said: “When we get to data analysis, we should stress graphical methods or exploratory data analysis, over formal statistical procedures.” Cobb and Moore (1997, p. 815) echoed the sentiments of Hahn saying: “Students like exploratory analysis and find that they can do it, a substantial bonus when teaching a subject feared by many. Engaging them early on in the interpretation of results, before the harder ideas come to their attention, can help establish good habits when you get to inference.”

Similar thinking was the basis for the Quantitative Literacy program of the mid-1980’s which was sponsored by the American Statistical Association (2005a) and the National Council of Teachers of Mathematics Joint Committee on the Curriculum in Statistics and Probability. Funded in part by the National Science Foundation, this program involves “… education workshops and written materials to help elementary and secondary school teachers make statistics more accessible to students … with an emphasis on graphical techniques” (American Statistical Association, 2005b). These written works were made available by Dale Seymour Publications through a series of four books. All four of these books focus on a “hands-on” approach that utilizes data very familiar to students from areas like sports, television, music, etc. One of these, Exploring Data by Landwehr and Watkins (1986), introduces students to a method of fitting a line through bivariate data using a simple application of the median.

One major goal of young statisticians as they learn exploratory data analysis is studying the relationship between two variables. When the data are bivariate with two numerical variables, this usually involves fitting a straight line through points on a scatterplot. The graphing calculator is programmed with two methods of line fitting for bivariate data. One method, fitting the least squares line, is a procedure well studied and routinely applied. But the median-median line, on the other hand, is not so well known or understood.

This paper considers the performance of these two line-fitting techniques as an aide to budding statisticians and their teachers as they encounter exploratory data analysis. We intend to illuminate the behavior of the relatively unknown median-median line with several data sets and through a small simulation study. Readers may compare how the median-median line reacts in various data settings to the performance of the least squares line.

2. Historical Evolution of the Median-Median Line

A similar procedure suggested by Nair and Shrivastava (1942) breaks up the points on a scatterplot into three regions with each region containing about the same number of points. The means of the x and y points in the left and right regions are used to find the slope of the line of fit much as Wald suggested.

Brown and Mood (1951) used the two-region approach but found the slope of the line of fit using medians in place of means. The primary advantage of using this measure of center comes from the median’s inherent ability to resist the strong effect of outliers. Most students of statistics know that the mean can be affected greatly by outliers since they are included with equal weight with the rest of the data in the sum of the scores. But the median takes on the same value whether the largest score in a data set is just somewhat larger than the rest of the data or is much larger than the second biggest score. In the context of fitting points on a scatterplot, this implies that a single point far from the general sloping trend of the rest of the points would not apply such a large “tug” on the location of the line of fit if the median is used to find the line.

Readers may recall that the least squares line is found by minimizing the sum of the squared distances that each point lies from the line. Since these distances are squared, Hartwig and Dearing (1979, p. 34) noted that “… cases lying farther and farther from the regression line increase the sum of the squared residuals at an increasing rate ... [and the line] will have to come reasonably close to them to satisfy the least squares criterion and, therefore, the least squares regression line will lack resistance to the excessive influence of a few atypical cases.” This means that Brown and Mood’s method is not only simple to apply, but also has the advantage of not allowing outlying cases to have undue impact on determining the line of fit.

Like Brown and Mood, Tukey (1971) utilized the medians in finding his line of fit, but he did so borrowing the three-region approach of Nair and Shrivastava. His line of fit, called the resistant line, is considered a basic methodology of exploratory data analysis. The median-median line provides the first iteration in the procedure to find Tukey’s resistant line. To obtain Tukey’s resistant line, the residuals are used to adjust the parameters in an iterative fashion. Appendix A provides the algorithm used by Texas Instruments to compute the median-median line. Tukey’s resistant line may be obtained using Minitab under the EDA submenu of Stat.

For those who wish to learn more about the broad methodology of exploratory data analysis, Velleman and Hoaglin (1981) provide a gentle review of the entire subject. To learn more about the median-median line at a more sophisticated mathematical level, readers are encouraged to consider Emerson and Hoaglin (1983) and Johnstone and Velleman (1985).

3. Examples

3.1 Manatees and Motor Boats

Once the data are divided into three regions, the median of the x- and median of the y-scores are calculated for each region. The resulting three points for these data are termed the median-median points = (481, 20), = (542.5, 28.5), = (675, 43). The slope of the median-median line is the slope of the line passing through and ; that is, . The position of the median-median line is determined by looking at the line passing through and and a parallel line passing through . Moving the line connecting the two outer points one third of the way to the line through the point in the center region yields the median-median line (see Figure 1). Note that the line that connects the two outer points is based on two of the median-median points, while the line through is based on only this median-median point. Thus, choosing to move the first line one third of the way towards the second effectively makes the median-median line a weighted average of the outer two lines. The resulting y-intercept is (See Appendix A. Note that TI calls the slope a and the y-intercept b.). This yields the equation of the median-median line: . Note that the median-median and least-squares lines have similar equations, as is evident from the scatterplot (see Figure 2).

Figure 1

Figure 1. Illustration of the median-median method for the manatee data.

Figure 2

Figure 2. Scatterplot of manatees killed by powerboats versus number of boat registrations.

3.2 Mental Aptitude and Age at First Word

Figure 3

Figure 3. Scatterplot of Gesell Adaptive Score versus Age at Which Child Began Speaking.

3.3 Meal Choices and Body Habits

Figure 4

Figure 4. Scatterplot of dietary energy density versus body mass index.

For illustration, we also consider DED as the explanatory variable and BMI as the response variable. In this scenario, the observation that was previously a high leverage point is now an observation with a large residual (Figure 5). Now the least squares line is not affected as much by the outlying observation as in Figure 4 where the observation has both high leverage and is far from the trend formed by the remaining observations. The data are provided in Appendix B.

Figure 5

Figure 5. Scatterplot of body mass index versus dietary energy density.

4. Simulation Study

The conditions under which the simulation is performed are:

1. two sets of values of the explanatory variable:
   Set 1: x-values = (1, 2, 3, …, 24), and
   Set 2: x-values = (2, 2, 4, 4, …, 24, 24),
2. two levels of the error standard deviation ( = 1 and 5), and
3. a number of outlier possibilities.

Without loss of generality, in most cases we assume that the outlier occurs in the upper region of X-values, that is, the outlier is one or two of Y₁₇, …, Y₂₄. Table 2 provides a summary of the outliers generated in the simulation study.

For each type of outlier generation, 1000 samples are simulated. For each sample the least squares and median-median estimates of the intercept and slope are computed. Tables 3 to 6 provide the following statistics on the 1000 simulated least squares and median-median slope estimates: means, standard deviations, and mean square errors (variance of simulated slopes plus bias squared where the bias = mean - 1) for each of the nine methods of generating the data. If the mean of the simulated slope estimates is 1, the estimator is considered unbiased. The standard deviation provides a measure of spread of the estimates of the slopes. And the mean square error provides a measure of precision of how far the estimates vary from the true slope. Since the main focus of this paper is on the slopes of the estimated line, we do not consider the estimated intercepts in this discussion of the simulation results. These tables also show the leverage value (see Neter, Kutner, Nachtsheim, and Wasserman (1996, p. 375-377)) for the design points at which the outlier is generated. Leverage values help to identify outlying X-values that, in conjunction with extreme Y-observations, may lead to data points with a large influence on the slope and intercept of the line. It is recommended to compare the leverage value with where p is the number of parameters in the linear regression model. In our case p = 2 (for the intercept and slope) and n = 24 so = 0.167. In our example, no design point has a leverage that exceeds this value though the leverage of the most extreme X-values (1 and 24) come close to .

Figures 6 to 8 provide density estimates for three of these examples for the first set of x-values (1, 2, 3, …, 24): no outliers, a moderate outlier example (high and low outliers at the end – Table 2, design (e)), and the most extreme outlier example (two high outliers at the end – Table 2, design (i)). The density estimates are simply the simulated distributions of the slope estimates and are computed from the 1000 slope estimates using the SPlus function density.

The simulation study comparing the performance of the two methods of regression when no outliers are present (Figure 6) shows that both the median-median and LS estimates are close to being unbiased. However, there is less variation in the least-squares slope estimates than in the median-median slope estimates.

Figure 6

Figure 6. Density estimates of the slopes for the 1000 replications for least squares and median-median estimates and for = 1 and = 5;
X’s: Set 1; no outliers.

In the moderate case (Figure 7, one high and one low outlier at the end – Table 2, design (e)), the least-squares method still tends to give unbiased estimates of the slope while the median-median estimates are slightly biased to the low side and exhibit smaller MSE than the LS line when = 5.

Figure 7

Figure 7. Density estimates of the slopes for the 1000 replications for least squares and median-median estimates and for = 1 and = 5;
X’s: Set 1; one high and one low outlier at the end – Table 2, design (e).

In the extreme case (Figure 8, two high outliers at the end – Table 2, design (i)), however, the median-median method performs much better, with less variation and more accuracy than the least-squares method, especially in the case of greater population standard deviation ( = 5 vs. = 1).

Figure 8

Figure 8. Density estimates of the slopes for the 1000 replications for least squares and median-median estimates and for = 1 and = 5;
X’s: Set 1; two high outliers at the end – Table 2, design (i).

5. Conclusion

The three examples provided in this paper show that, when an influential point is present, the median-median line may be less affected by the influential point than the least-squares line. The simulation study further supports the argument for using the median-median line as a resistant method of regression when outliers are present, especially in the most extreme cases (one or two high outliers at the end; see Table 2, designs (h and i)) and in cases with greater population variance when the outlier is on the most extreme x-value.

On the other hand, there exists a well-developed set of tools for inference and for detecting the presence of unusual observations when using least-squares regression under normal error assumptions. In contrast, inference about the median-median estimates would require bootstrapping to obtain approximate standard errors, confidence intervals/regions, and p-values for tests. Given this, in addition to the superior performance of least-squares estimation when either no outliers or moderate outliers are present, the median-median method of regression would not appear to be a valuable tool at either the professional or collegiate level.

However, in elementary and middle schools the median-median method of estimating a line may be a reasonable approach to describe a linear relationship in a scatterplot. The teacher may first introduce the idea of a scatterplot as a way of visualizing the association between a response and explanatory variable. If there is a straight-line trend in the plot, the straight line may be fit by eye using a ruler to try to capture the trend. While each student will likely be satisfied that their eye-balled line fits the data well, they will also recognize that each student would have chosen a slightly different line. This will suggest the need for a more objective and structured method of fitting a line. The median-median line will not only be simple to find, but may also feel very connected to the visual process each student had just undertaken.

Appendix A: Texas Instruments median-median algorithm

Median-Median Line Algorithm on Graphing Handhelds. 

Solution:

What method is used to calculate the median-median line?

The goal of the median-median line is to:

1) Divide the data into three parts with an equal number of data points
2) Define a summary point for each part
3) Use the three summary points to define the median-median line

How the TI Calculator does this:

1) The calculator will attempt to break the list into three equal parts without breaking up data groups of equal x values. 
   In this case, the algorithm we use is designed to include at least 1/3 of the points in the left and right groups. What 
   ever is left over is put into the center group, hopefully the remaining 1/3. The approach chosen was to fill the outside 
   groups first and allocate the remaining data points to the center group. If the center group is empty, an error message
   is generated.

   The reason equally x values are not split up is to ensure the same results are produced independent of the order the 
   data appears in the original list. Without this restriction, a different result could be produced depending on the 
   ordering in the data input lists.

2) A summary point is simply the median of all x's and y's in that part.  Let's call the summary points (x1,y1), (x2,y2), 
   and (x3,y3).

3) The median-median line will be parallel to the line going through the points (x1,y1) and (x3,y3)

                                                a = (y3 - y1) / (x3 - x1)

   and 1/3 the distance between the line through the two summary points and a parallel line going through the second 
   summary point (x2,y2).

                                        b = (y1 + y2 + y3 - a (x1 + x2 + x3)) / 3.

   An advantage of the median-median line over a least-squares line, is that stray data points do not affect the end result 
   very much.

Appendix B: Data sets used in the examples

Example 2. Gessell Scores and Age illustrating the 3 regions (data used with permission of W.H. Freeman and Company)

Example 3. Body Mass Index (BMI) and Dietary Energy Density (DED) illustrating the 3 regions (data used with permission of Thompson Learning)

From Statistics: The Exploration and Analysis of Data (with CD-ROM), 4^th Ed., by Devore/Peck, 2001. Reprinted with permission of Brooks/Cole, a division of Thompson Learning: www.thomsonrights.com. Fax 800 730-2215.

References

American Statistical Association (2005b), Materials for K-12 Statistics Education from jse.amstat.org/education/index.cfm?fuseaction=k12material

Brown, G. W. and Mood, A. M. (1951), “On Median Tests for Linear Hypotheses,” Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA: University of California Press, 159-166.

Cobb, G. W. and Moore, D. S. (1997), “Mathematics, Statistics, and Teaching,” The American Mathematical Monthly, 104(9), 801-823.

Devore, J. and Peck, R. (2001), Statistics: The Exploration and Analysis of Data, 4^th Ed., Belmont, CA: Brooks/Cole.

Emerson, J. D. and Hoaglin, D. C. (1983), “Resistant Lines for y-versus-x,” in Understanding Robust and Exploratory Data Analysis, eds. D. C. Hoaglin, F. Mosteller, and J. W. Tukey, New York: John Wiley.

Hahn, G.J. (1988) “What Should the Introductory Statistics Course Contain?,” College Mathematics Journal, 19(1), 26-29.

Hartwig, F. and Dearing, B. E. (1979), Exploratory Data Analysis, Beverly Hills, CA: Sage Publications.

Jiang, C. L. and Hunt, J. N. (1983), “The Relation Between Freely Chosen Meals and Body Habits,” American Journal of Clinical Nutrition, 38, 32-40.

Johnstone, I. M. and Velleman, P. F. (1985), “The resistant line and related regression methods,” Journal of the American Statistical Association, 80, 1041-1054.

Landwehr, J. A. and Watkins, A. E. (1986), Exploring Data, Palo Alto, CA: Dale Seymour Publications.

Moore, D. S. (1997), Statistics: Concepts and Controversies, 4^th Ed., New York: W. H. Freeman and Co.

Moore, D. S. and McCabe, G. P. (2003), Introduction to the Practice of Statistics, 4^th Ed., New York: W. H. Freeman and Co.

Nair, K. R. and Shrivastava, M. P. (1942), “On a Simple Method of Curve Fitting,” Sankhaya, 6, 121-132.

Neter, J., Kutner, M. H., Nachtsheim, C. J., and Wasserman, W. (1996), Applied Linear Statistical Models, 4^th Ed., Boston: McGraw Hill.

Tukey, J. W. (1971), Exploratory Data Analysis, Reading, MA: Addison-Wesley.

Velleman, P. F. and Hoaglin, D. C. (1981), Applications, Basics and Computing of Exploratory Data Analysis, Boston, MA: Duxbury Press.

Wald, A. (1940), “The Fitting of Straight Lines if Both Variables Are Subject to Error,” Annals of Mathematical Statistics, 11, 282-300.

Acknowledgements

Elizabeth J. Walters
Mathematical Sciences Department
Loyola College in Maryland
Baltimore, MD 21210-2699
U.S.A.
ewalters@loyola.edu

Christopher H. Morrell
Mathematical Sciences Department
Loyola College in Maryland
Baltimore, MD 21210-2699
U.S.A.
chm@loyola.edu

Richard E. Auer
Mathematical Sciences Department
Loyola College in Maryland
Baltimore, MD 21210-2699
U.S.A.
rea@loyola.edu

Volume 14 (2006) | Archive | Index | Data Archive | Information Service | Editorial Board | Guidelines for Authors | Guidelines for Data Contributors | Home Page | Contact JSE | ASA Publications