Using Simple Linear Regression to Assess the Success of the Montreal Protocol in Reducing Atmospheric Chlorofluorocarbons

Dean Nelson
University of Pittsburgh at Greensburg

Journal of Statistics Education Volume 17, Number 2 (2009), jse.amstat.org/v17n2/nelson.html

Copyright © 2009 by Dean Nelson all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the author and advance notification of the editor.

Key Words: Regression analysis; Introductory statistics; Ozone; GAISE recommendations.

Following the Guidelines for Assessment and Instruction in Statistics Education (GAISE) recommendation to use real data, an example is presented in which simple linear regression is used to evaluate the effect of the Montreal Protocol on atmospheric concentration of chlorofluorocarbons. This simple set of data, obtained from a public archive, can be used to tell a compelling story of success in international diplomacy solving a global environmental problem. A description of the use of these data and analyses are presented for a number of courses in applied statistics including introductory statistics.

1. Introduction

The recently adopted Guidelines for Assessment and Instruction in Statistics Education (GAISE) guidelines (jse.amstat.org/education/gaise/GAISECollege.htm) made six general recommendations for the teaching of introductory statistics.  The second recommendation was to "Use real data". We would like to suggest an expanded version, "Use real data that tell a compelling story". This paper is meant to provide an illustration of this amended recommendation. The data and accompanying analysis presented in this paper provide both a meaningful example of data analysis using simple linear regression and a story of remarkable success of international cooperation addressing a global environmental problem.

The Montreal Protocol was an international agreement undertaken in 1987 by 191 countries to reduce the levels of ozone depleting substances (ODSs) in the atmosphere. The agreement has met with unprecedented success. Kofi Annan, former Secretary General of the United Nations referred to the Montreal Protocol as "perhaps the single most successful international agreement to date."  The schedule for reductions in manufacturing and use of ODSs was not only met but accelerated. Manufacturers of ODSs, rather than resist the phase-out of ODSs, embraced the challenge and developed replacement compounds reversing possible detrimental economic consequences of the treaty (Shende, 2007).

In all of these qualitative ways, the Montreal Protocol has been uniformly successful, but this paper is not about qualitative measures.  Rather, it describes how students are introduced to this story using a set of data that was collected in relative obscurity until discovery of the "hole" in the ozone layer in 1985 (Farman, Gardiner, and Schanklin, 1985). Many of these data are made publicly available, on the website (gaw.kishou.go.jp/wdcgg/) of the World Data Centre for Greenhouse Gases (WDCGG) maintained by the Japanese Meteorological Agency in cooperation with the World Meteorological Organization (see detailed instructions on how to download the data used in this paper in the Appendix). Some of these data tell compelling stories, as do the data on atmospheric concentrations of chlorofluorocarbons (CFCs), measured monthly by the National Oceanic and Atmospheric Administration on Mauna Loa in Hawaii since 1977.

2. The Story

The story of the Montreal Protocol, describing the international cooperation of nearly the entire world, is both interesting and compelling. This global story, however, starts very small, at a molecular level.

2.1 There’s a Hole in the Ozone

Ozone is the name given to the molecule O₃. Ozone is formed in the stratosphere when regular oxygen molecules, O₂, are split by the ultraviolet radiation of the sun. The oxygen atoms become attached to molecular oxygen to form O₃. The amount of ozone in the atmosphere is very small. Even so, the ozone in the atmosphere, along with oxygen molecules, absorbs most of the ultraviolet radiation from the sun. Since ultraviolet radiation is harmful to animal and plant life, the ozone in the stratosphere has a very important protective role for life on earth.

In 1972, Lester Machta reported (Rowland and Molina, 2007) that James Lovelock had discovered a new compound in the earth’s atmosphere, trichlorofluoromethane, a chlorofluorocarbon (CFC). Researchers at the Atomic Energy Commission became interested in these new atmospheric components and asked the question "What happens to these compounds in the environment?"  The answer they found (which later became one of the reasons they were awarded the 1995 Nobel Prize in Chemistry) was that these compounds slowly rise into the stratosphere where they encounter ultraviolet light that is energetic enough to break them apart. When they break apart, they release chlorine atoms that can act as catalysts for destruction of ozone (Molina and Rowland, 1974).

Subsequent to the discovery of CFCs in the atmosphere, and an understanding of their potential for ozone destruction, scientists throughout the world began to investigate the problem. Even so, only sporadic measurement and analysis of the ozone layer was conducted until a comprehensive report summarizing the literature was released in 1986, (WMO, 1986). Even though the report was nearly 1,100 pages with 86 pages of reference, it did not claim to provide proof that CFCs were depleting the ozone layer. It was another three years before a joint NASA-NOAA report (WMO, 1988) was released that provided sound evidence that CFCs were depleting the ozone layer.

The most dramatic evidence of ozone depletion was the seasonal creation of an ozone "hole" over the Antarctic first reported in 1985 (Farman, Gardiner, and Schanklin, 1985). Particularly in the months of September and October, when Antarctic air is isolated from milder air, the extreme cold facilitates the breakdown of the ozone resulting in depletion of up to 80% in that region. Depletion at that level over population regions of the earth could have dramatic detrimental effects on human, animal, and plant life.

2.2  The Montreal Protocol

While scientists were hard at work trying to determine the extent of the CFC problem, governments and industry were debating what should be done about it. A series of international meetings were organized to address the problem. The end result was the Montreal Protocol of 1987, an agreement signed by 191 countries to begin the process of cooperative action toward a solution.

The Montreal Protocol created a schedule for phasing out manufacturing and use of CFCs by participating countries. Table 1 below shows the initial schedule and subsequent revisions. When the Montreal Protocol was first agreed upon, it was not certain that suitable safe replacements could be found for all the purposes for which CFCs were used. Even so, after the Montreal Protocol, CFC manufacturers were quick to dedicate themselves to developing new compounds that could be used for the same purposes as CFCs without ozone depletion potential. The success of these manufacturers to develop suitable replacements for CFCs allowed the acceleration of the phase-out schedule (also shown in Table 1).

Table 1. Original and Revised Phase-Out Schedules for CFS in Developed Countries

2.3 Evaluating the Effect of the Montreal Protocol

Every year since full implementation of the protocol, a yearly review has been produced to assess the success of global efforts. In 2007, a panel of scientists from countries around the world produced a 206-page report (UNEP 2007) that describes progress since the previous report in 2006.  We cannot hope to reproduce the detail necessary to assess the entire scope of the Montreal Protocol, but there are publicly available data that can be used to assess the effect of the Montreal Protocol on one very important aspect of the problem, the presence of CFCs in the atmosphere.

3. Presentation of the Montreal Protocol Story to Students

We have used this story in statistics courses of varying difficulty; a first introductory course, a second semester course, and an applied regression course. In each course, the presentation of the problem, the data, and the analysis follows the same pattern, although the expectations of the students differ according to the level of the course. The sequence below was used in these courses for this case study:

1. Present the environmental problem similar to the presentation in section 2 above, requiring some independent investigation by the students. The independent investigation provides the students with the opportunity to encounter, and struggle with, original source material. The purpose is two-fold, to increase their understanding and awareness of the immense amount of work required to address this problem and recognize their limits of understanding, and to enable groups of students to contribute the knowledge they have gained in subsequent discussions. We have found it useful to assign this as a group task, each group being asked to expand on some aspect of the problem that interests them. Some possible topics are
   
   • The dangers of ozone depletion
   • The 1995 Nobel Prize for Chemistry
   • A simple description of gas chromatography
   • Some aspect of the Montreal Protocol
   • Some aspect of the Montreal Protocol
   • Atmospheric measurements using the flask method
   • CFC measurements at other sites around the world
2. After a basic understanding of the environmental problem has been achieved, introduce the students to the data on the WDCGG web site. A detailed description of how to navigate the site to these data as well as the ftp URL for these data is provided in the Appendix. These data are provided in an ASCII file, which can be read using any standard analysis package. Only three numbers are read from each record; the year, month, and CFC concentration (labeled CFC-11). This provides a good opportunity to demonstrate how to read ASCII files with statistical software being used in the course.
3. After reading the data, ask the students  to think about the numbers they now have available for analysis as shown below:
    
   Year Month  CFC
   1977  01   139.90
   1977  02   139.50
   1977  03   139.00
   1977  04   134.10
   1977  05   135.00
   1977  06   143.40
   1977  07   140.30
   1977  08   142.60
   1977  09   144.70
   1977  10   144.20
   1977  11   144.80
   1977  12   145.70
   1978  01   148.60
   1978  02   144.80
   1978  03   148.40

   What does a value of 145 for CFC concentration mean?  This is not an easy question, and one that does not have a definitive answer. Abstractly, it means that 145 parts of every trillion parts are CFC molecules, even though "parts" could mean mass, volume, or molecular count.  Students should be able to provide this abstract answer, but they should also be able to tell us why that answer is not actually true. In practice, we can never measure anything without error, and the error in our measurement may have many sources. In order to identify our sources of error, we need to understand how the measurement was taken, what procedures and instruments were used. With a little forethought on our part in assigning independent investigation by students in part 1 above, descriptions of the procedures and instruments can be provided by students in the class.

   Although knowing how the measurement was taken allows us to ask the right questions in order to assess the accuracy of the measurement, it will not be possible to answer these questions fully in class. The chemistry, physics, and engineering behind the gas chromatography measurement instruments are too complex. Therefore, we are forced to rely on faith, that the scientists who designed, built, tested, used these instruments did so correctly. That is not too great a leap of faith, because those scientists used the same kind of science that resulted in other technology we rely on every day. Still, our aim here is not to arrive at a definitive answer to the questions, but to determine the right questions to ask, and to assess the degree to which we can rely on our answers, both to gauge the success of our modeling efforts and to determine when we need other expert assistance.

4. Ask the students to produce a graph similar to Figure 1 below. This entails a discussion on how to create the variable for the time axis since the data are provided by year and month, which are measured on different scales. In order to produce a graph, time is a single dimension measured on a single scale. For year, one unit equals one year. For month, one unit equals one month. To combine them into a single measure of time, either year or month has to be transformed into the scale of the other. Give this to students as an exercise. Some will transform years into months, and some will transform months into years. For instance, time in months is obtained by the formula (Year*12)+Month. Time in years is obtained by Year+Month/12. Notice that in the last formula, it is assumed that December of 1977, say, will become 1978. This is equivalent to assuming that the measurements were taken at the end of each month. However, the data seem to indicate that the measurements were taken at the beginning of each month. Therefore, time in years should be calculated as Year+(Month-1)/12.
   Figure 1. Monthly Atmospheric Concentrations of CFCs (parts per trillion) from January 1977 to September 2004
   

5. Before talking about the effect of the Montreal Protocol, it is necessary to identify and agree upon the time periods that will be used for ‘before’ and ‘after’ the Montreal Protocol. It can be seen from Table 1 that the original treaty called for a 50% reduction by the year 2000. However, because of the leadership role exhibited by the CFC industry in not only embracing the phasing out of CFCs but also in developing non-ozone-depleting alternatives, the schedule was accelerated by revisions to the treaty in 1990 and 1992. Although the rate of decline in CFC manufacturing declined quickly through 1995 and then began to decline more slowly, (Figure 2), a reversal in the trend in atmospheric CFC concentrations can be seen during the years during which the decreases were being enacted. By 1995, the cumulative effect of this 5-year, rapid decline in CFC manufacturing had established a new pattern of change. We usually define the period before the Montreal Protocol as before January 1990 and the period after the Montreal Protocol as after December 1994.

   After producing a graph like Figure 1, ask the students if these data could be useful in determining whether the Montreal Protocol had been successful in reducing atmospheric CFCs. They will, of course, say yes right away, in part because they are perceptive enough to reason that we wouldn’t be going through the trouble of presenting these data if the data did not show an effect. Because the reasoning for their answer was superficial, it will require some time before they can articulate why they think the data can be useful. For instance, one student said, "CFCs were rising, and after the Montreal Protocol CFCs were falling."  Our reply was "That is what the data show, but the data also show that the level of CFCs before and after the Montreal Protocol looks about the same, and may even be higher if we look at right before and right after the Montreal Protocol."  Discussions we have had generally result in a number of proposed definitions of the effect of the Montreal Protocol. They include
   a.       The difference between the mean CFC concentration before the Montreal Protocol and the mean CFC concentration after the Montreal Protocol.
   b.      The difference between the annual rate of CFC concentration change before the Montreal Protocol and the annual rate of CFC concentration change after the Montreal Protocol.
   c.       For some given time value, the difference between the predicted CFC levels using two models, one for before the Montreal Protocol and one for after the Montreal Protocol.
   After further discussion, students usually agree that a. falls short of capturing the most important feature of the data, that is the rate of change.

6. Ask the students to estimate and/or test their claim of an effect using the statistics that they know. They will have been introduced to both estimation and hypothesis testing. Our expectation is that after our discussion on what "effect" means in the context of our research question, students will translate that definition into a problem of estimation and/or a hypothesis test. Depending on the level of the course, they may choose the following corresponding to the two definitions of "effect" in 5b. and 5c. above.
   5b - The annual rates of change can be estimated by slope values obtained in a simple linear regression. This can be done using separate regressions, or by doing simultaneous regressions within a single model, depending on the level of the class. A point estimate of the difference is done by simply computing the difference in the slopes. An interval estimate and hypothesis test for the difference between the slopes may be presented in a more advanced course.
   5c - Point estimates for any selected value of time can be estimated by evaluating the model at the selected value of time. A point estimate of the difference between predicted values is obtained by subtracting one from the other. An interval estimate and hypothesis test for the difference between the predicted values  may be presented in a more advanced course.

4. Data Analyses

After understanding the underlying environmental problem and formulating an acceptable definition for "effect", the students are asked to conduct data analyses to determine the extent to which the data confirm an effect on atmospheric CFCs attributable to implementation of the Montreal Protocol. We have used this problem in an introductory course, in the second semester of an introductory course sequence, and in a regression course. With respect to the analysis of these data, a set of expectations for the different levels are listed below.

First Introduction Course
1.      Estimate separate regression equations for the time periods before and after the Montreal Protocol
2.      Interpret of the slopes and intercepts
3.      Calculate the predicted values, given some value for the independent variable.
4.      Create a graph with the regression lines and predicted values labeled.
5.      Produce point estimates of the effect of the Montreal Protocol, both a difference in slopes and a difference in predicted values.
Second Introduction Course
1.      Items listed for the First Introduction Course.
2.      Recognize that the point estimates of the effect of the Montreal Protocol should include a statement of uncertainty.
3.      Recognize the problem of extrapolation.
4.      A deeper understanding of the model, demonstrated by a questioning of the assumptions.
Advanced Course
1.      Items listed for the Second Introduction Course.
2.      Create interval estimates for the point estimates of the effect of the Montreal Protocol.
3.      Perform hypothesis tests for the effect of the Montreal Protocol.
4.      Assess assumptions of the model.

The time-series graph of monthly measurements of CFC concentrations from 1977 to 2004 in Figure 1 makes quickly apparent two systematic, yet distinctly different, trends before and after the Montreal Protocol implementation. In order to determine, and justify, the implementation period of the Montreal Protocol, data on CFC manufacturing and use are presented. The international community had agreed at the 1987 meeting to a phase-out schedule (see Table 1) beginning in 1990 to reach 50% by 2000, but this schedule was accelerated by agencies within specific countries or groups of countries so that by 1995, most countries had drastically reduced or eliminated the manufacture and use of CFCs (UNEP, 2007) resulting in dramatic decreases in the manufacture of CFCs worldwide (Figure 2).

The phase-out period of 1990-1995 corresponds closely to the period of trend reversal in the data depicted in Figure 1. Prior to 1990, the pattern of atmospheric CFC concentration showed a constant rate of increase over time. In contrast, the pattern after 1994 showed a constant rate of decrease in atmospheric concentrations of CFCs over time. The students should recognize that we can estimate and compare these rates of change before and after the Montreal Protocol using estimation from simple linear regression models in and introductory class or using hypothesis testing within a simultaneous regression model in a more advanced class.

4.1 Simple Linear Regression Models

Since the data show remarkably constant rates of change during periods that correspond to the times prior to and subsequent to implementation of the Montreal Protocol, these rates of change can be modeled with a simple linear equation. If we take y to be equal to the atmospheric concentration of CFCs in parts per trillion and x to be time in years, then we can use the simple linear regression models in (1) to predict atmospheric concentrations of CFCs over time for before, y₁, and after, y₂, the Montreal Protocol implementation respectively.

Using least squares criteria, we obtain estimates for the models in (1).

In addition, the Montreal Protocol provides a clear rationale for deciding what dates to use to partition our data. We fit this simple linear model separately for two distinct time periods; prior to1990, and subsequent to 1994, corresponding to data before and after the Montreal Protocol implementation. The results of these analyses are given in Tables 2 and 3.

After estimating the regression models, students are asked a series of questions that prompt reflection on the estimated models:

1. Interpret the intercepts for each of the regression lines. Do they make sense to you?  If so, why?  If not, why not?
   Many students will be confused by the intercept estimates until they reason out the correct interpretation as the estimated concentration at year 0. Even so, the negative value of the intercept in the ‘Before 1990’ model makes no logical sense with respect to the quantity being measured, because levels of concentration cannot be less than zero. This provides an opportunity to make the distinction between a mathematical model and reality. The model is an abstract conceptualization that can be useful in describing reality, but the model itself is not reality.  One of our primary aims is to provide our students with the appropriate mixture of respect and skepticism for statistical analysis of data, the ability to recognize the value while at the same time the limitations of our ability to describe our experience with abstract models.
2. CFCs are man-made. What year would you guess CFCs were first manufactured using the regression equation estimated with the data from 1977-1990?
   Using the parameter estimates in Table 3, the answer to this question is 1962.955, or around January 1963. This question requires an understanding of the relationship between time (x) and CFC concentration (y) described by the regression equation. Although we have discussed the problem as if knowing the value of time allows us to derive the level of concentrations, our mathematical representation can just as easily allow derivation of the value of time from a known level of concentration. Recognition of these symmetric interpretations of the model indicates a deeper understanding of the relationship described by the model.

3. CFCs were first manufactured in 1931. How would you explain the discrepancy between the actual year they were first manufactured and the year you predicted?
   Thinking about this question can provide students with several significant insights. First, they will recognize that the model predicts a time that is about 32 years off. That is a large error.  This may initially cause them to infer that the model is a poor model.  However, examination of a graph like Figure 3 (see Section 5.1) and/or interpretation of the R² value of .996 for this model, is convincing evidence that the regression model does a very good job of describing the data. Resolving this apparent contradiction provides an important insight, one that can be generalized to all of science, and if we want to be very philosophical to all human experience. The truth is that there is no logical reason to assume that the ability of our model to successfully describe the data observed between 1977 and 1990 guarantees that it can successfully describe data outside that time period.  The insight gained by recognizing this can be profound, because it tells us something about ourselves as humans. Our natural inclination is to assume that discovered patterns are extensible. This kind of reasoning is called induction, and even though it is not logically defensible, empirically it has been fabulously successful.
   Second, students will recognize that this is a case in which the inductive reasoning, so often used successfully, does not work. The model cannot be successfully used to make predictions into the past. Why not?  They will correctly surmise it is because the patterns we observe during the time period of the observed data were not true of CFC concentrations at all times since 1931.
   
4. What year would you predict the level of CFCs in the atmosphere will reach zero?  Based on their experience with the previous question, most students will not just solve the ‘After 1994’ regression model for x when y=0 and present that as the answer. They will realize that to do so requires a strong assumption that needs to accompany the prediction. The answer is not simply ‘the year 2143.851 or around October 2143’, but ‘the year 2143.851 if we assume that the rate of decline remains constant until that time’.

4.2 Simultaneous Linear Regression Model

Alternatively, a simultaneous regression could be used in a more advanced class to estimate parameters for both lines in the same model.  For this analysis, the following model is used,

%Equation (3)
\begin{equation}
y =\beta_{10}+\beta_{11}x_{1}+\beta_{20}+\beta_{21}x_{2}+e
\end{equation}


where x₁ is the CFC concentration prior to 1990 and equal to zero otherwise and x₂ is the CFC concentration after 1994 and equal to zero otherwise. Again using least squares criteria, we obtain estimates for the parameters in (3).

%Equation (4)
\begin{equation}
\hat{\mu}_{y \vert x}}}=\hat{\beta}_{10}+\hat{\beta}_{11}x_{1}+\hat{\beta}_{20}+\hat{\beta}_{21}x_{2}
\end{equation}

The results of this analysis are shown in Tables 4 and 5.

Table 4. ANOVA Table for the Simultaneous Linear Regression Models

Model	Source	Sums of Squares	df	Means Square	F	p
Simultaneous	Model	206180.294	2	103090.147	262.611	.000
Two Lines	Error	937.974	265	392.559

 

Table 5. Parameter Estimates for the Simultaneous Linear Regression Models

Period	Parameter	Estimate	Standard Error	t	p
Simultaneous	β10	-19064.219	80.409	-237.091	.000
Two Lines	β11	9.712	.041	239.554	.000
	β20	3929.678	124.635	31.530	.000
	β21	-1.833	.062	-29.410	.000

Estimating this model with standard software will require that the data be set up to explicitly define the design matrix. The data will consist of four columns, x_{1} and x_{2} as described and two additional indicator columns, each paired with an x column indicating with a value 1 that the x-value is not zero and a value of 0 if the x-value is zero. The data below show the setup of the data to explicitly define the design matrix:

Int1   x1           Int2   x2
1.00   1977.00              .00    .00
1.00   1977.08              .00    .00
1.00   1977.17              .00    .00
1.00   1977.25              .00    .00
1.00   1977.33              .00    .00
1.00   1977.42              .00    .00
.
.
.
1.00   1989.58              .00    .00
1.00   1989.67              .00    .00
1.00   1989.75              .00    .00
1.00   1989.83              .00    .00
1.00   1989.92              .00    .00
.00    .00           1.00   1995.00
.00    .00           1.00   1995.08
.00    .00           1.00   1995.17
.00    .00           1.00   1995.25
.00    .00           1.00   1995.33
.
.
.
.00    .00           1.00   2004.33
.00    .00           1.00   2004.42
.00    .00           1.00   2004.50
.00    .00           1.00   2004.58
.00    .00           1.00   2004.67


Using SPSS, we were unable to run the model using the Regression procedure without resorting to executing syntax since it is not possible to alter the collinearity tolerance within the interface dialogue boxes. The following syntax produced the correct parameter estimates for the model above:

REGRESSION
  /MISSING LISTWISE
  /STATISTICS COEFF OUTS R ANOVA
  /CRITERIA=PIN(.05) POUT(.10) tolerance(.00000000001)
  /ORIGIN
  /DEPENDENT CFC
  /METHOD=ENTER int1 x1 int2 x2  .


Notice that we explicitly declared that no overall intercept should be included in the model. This prevents the software from adding another column of ones to our design matrix.

Discussion of this model with students is centered on recognizing that the parameter estimates are identical to the estimates obtained when the models were fit separately. The model was parameterized specifically so that this would explicitly occur. The obvious question then arises as to whether there is any difference between estimating the models together or separately. The difference is easily explained by comparing the separate models in (1) to the combined model in (3). For the separate models, two error terms are used, e₁ and e₂. For the combined model, only a single error term is used, e. The difference between the models, therefore, is that the combined model estimates a single error variance rather than the separate error variances estimated by the separate models. This difference results in several questions:


1. Why would we choose to use one model instead of two?  The most compelling reason, and one that students will readily understand, is that combining the models increases our sample size. If we increase our sample size, we can obtain more accurate estimates, that is estimates with narrower confidence intervals.  However, this is only made possible because we have assumed that the error variance is constant across all the data. That brings up the second question.
2. Are we justified in assuming a single error variance?  If we examine both the mean squared error of the separate regressions (Table 2) and the residual plots from those models (Figures 4 and 5 in Section 6), it is obvious that the error variance for the data prior to 1990 is larger by about an order of magnitude than the error variance after 1994. Given such a large difference, are we justified assuming that they are equal?  It is outside of the scope of this paper to diverge into testing this assumption, but it is useful to have the students think about what might cause this order of magnitude difference. By examining the data file, it is clear that the monthly values we used are actually the mean of a sample of measurements taken each month. Our first inclination would be to assume that perhaps the sample sizes after 1994 were larger than before 1990. That would cause the sample means to have smaller variances. However, the sample sizes after 1994 were not larger than before 1990. Therefore, the difference in variances must be due to a decrease in variance between measurements. In other words, the measurement technique was improved. We have not found independent verification of this, but there is a period of time in which no observations were recorded. Prior to that hiatus, the variances were larger than afterwards. Perhaps an ambitious group of students could track down the necessary information to verify this surmise.
3. How does use of the simultaneous model change our estimates and hypothesis tests?  This is a question that the students should be able to answer in general after the discussion of point 1. Our point estimates are unchanged, but the margin of error will be different, and our t-values for our hypothesis tests will be different. In other words, anything computed from a formula containing the error variance will be affected.

5. Estimation, Inference, and Interpretation

After the discussion described in Section 3 item 5, students have developed an understanding of what the definition of an "effect" of the Montreal Protocol means with respect to the data. Depending on the level of the class, this understanding still needs to be translated into statements about population parameters and the corresponding sample statistics used to estimate those parameters. The effect defined in 5b of section 3 is written in terms of the difference between the slopes,

%Equation (5)
\begin{equation}
d_{S}= \beta_{11} - \beta_{21}
\end{equation}

Our estimate of this difference uses the slope estimates from our regression analyses,

%Equation (6)
\begin{equation}
\hat{d}_{S}= \hat{\beta}_{11} - \hat{\beta}_{21}
\end{equation}

The effect defined in 5c is written in terms of the difference between two predicted values,

%Equation (7)
\begin{equation}
d_{P}=\mu_{y_{1} \vert x}-\mu_{y_{2} \vert x}
\end{equation}

Our estimate of this difference uses the predicted values for some value of x from our regression analyses,

%Equation (8)
\begin{equation}
\hat{d}_{P}=\hat{\mu}_{y_{1} \vert x}-\hat{\mu}_{y_{2} \vert x}
\end{equation}

Depending on the level of the class, students are expected to provide point estimates for (6) and (8), and inference in the form of interval estimates and/or hypothesis tests using either the simple linear models estimated in Section 4.1 or the simultaneous regression model estimated in Section 4.2 or both.

5.1 Simple Linear Regression Models

Our estimates of the slopes, and , in the simple linear regression models for before and after the Montreal Protocol implementation are interpreted as rates of change per year in atmospheric CFCs concentration. Our model estimates that atmospheric concentration of CFCs of increased at a rate of 9.712 parts per trillion per year prior to the Montreal Protocol. Contrast that to the rate of change after the Montreal Protocol, a decrease of 1.833 parts per trillion per year. We have defined our point estimate of the effect of the Montreal Protocol as this difference in the rate of change in (6), a decrease of 9.712-(-1.833)=11.545 parts per trillion in the annual CFC atmospheric concentration rate of change.

We can construct a confidence interval for the rate of change difference as

%Equation (9)
\begin{equation}
\hat{d}_{S} \pm t_{\frac{\alpha}{2}} s_{\hat{d}_{S}}
\end{equation}

where

%Equation (10)
\begin{equation}
s_{\hat{d}_{S}}=\sqrt{V_{1}+V_{2}}
\end{equation}

and

%Equation (11)
\begin{equation}
V_{i}=s^2_{\hat{\beta}_{i1}}
\end{equation}

The degrees of freedom for the t-distribution are

%Equation (12)
\begin{equation}
d.f.=\frac{(V_{1}+V_{2})^2}{\left(\frac{V^2_{1}}{n_1-2}+\frac{V^2_{2}}{n_2-2}\right)}
\end{equation}

Thus we find our 95% confidence interval for the rate of change difference is 11.545±.113.

If we further ask whether this difference is more than we would expect by chance, a hypothesis test can be conducted comparing the two slopes. Although it is unlikely that the students will suggest the appropriate test, they can readily understand that we may use a t-test for this hypothesis, since the comparison of the two slopes is directly analogous to the comparison of two means. The null and alternative hypotheses are

%Equation (13)
\begin{equation}
\begin{array}{r{}l}
 H_{0}:  d_{S}=0 \\
 H_{A}:  d_{S}  \neq 0
\end{array}
\end{equation}

The t statistic is calculated as

%Equation (14)
\begin{equation}
t=\frac{\hat{d}_{S}}{s_{\hat{d}_{S}}}
\end{equation}

with the standard deviation of the difference and degrees of freedom given by (10) and (12) respectively. Calculation of the test statistic gives t=200.861 with a p-value smaller than .001.

The other way we define the effect of the Montreal Protocol is to determine the difference between what the atmospheric CFC concentration would have been (assuming the same trend were to continue) had the Montreal Protocol not been enacted and what the predicted the atmospheric CFC concentration will be now that the Montreal Protocol has been enacted.  Symbolically, the effect is defined as (7) and our estimate of this effect is given in (8)

In order to define the effect, we need to choose a date (a value for x) at which the comparison will be made, say January 2009. The predicted concentration levels of CFCs using the pre-Montreal Protocol and the post-Montreal Protocol regression models are shown in Figure 3 as the straight lines obtained by our regression estimates. For the pre-Montreal Protocol model, it was assumed that the CFC concentration levels would continue to rise at the same rate, 9.712 parts per trillion a year until 2009. The predicted level is 446.226 parts per trillion in January 2009. For the post-Montreal Protocol model, it is assumed the CFC concentration levels would continue to fall at the same rate, 1.833 parts per trillion a year until January 2009. The predicted level is 247.408 parts per trillion in 2009. It seems clear that the Montreal Protocol was responsible for this reversal from a yearly increase in CFC concentration levels to a yearly decrease in CFC concentration levels, since the pattern of CFC concentration change over time coincides exactly with the implementation of the protocol. When we calculate the difference between the predicted CFC levels for the two models, that the Montreal Protocol is responsible for a decrease of 198.818 parts per trillion in CFC concentration in January, 2009.

Figure 3. Comparing Predicted Levels of Atmospheric Concentrations of CFCs (parts per trillion) Using the Pre-Montreal Protocol and the Post Montreal Protocol Data


We can construct a confidence interval for this estimate of the difference between predicted values as

%Equation (15)
\begin{equation}
\hat{d}_{P} \pm t_{\frac{\alpha}{2}} s_{\hat{d}_{P}}
\end{equation}

where

%Equation (16)
\begin{equation}
s_{\hat{d}_{P}}=\sqrt{V_{1}+V_{2}}
\end{equation}

and

%Equation (17)
\begin{equation}
V_{i}=s^2_{i}\left[ \frac{1}{n_{i}}+\frac{(x-\bar{x}_i)^2}{\sum(x_{ij}-\bar{x}_i)^2}\right]
\end{equation}

The degrees of freedom for the t-distribution are computed by (12). Our 95% confidence interval for the difference between predicted values is 198.818±2.734.

A hypothesis test to determine whether the difference between the predicted values is due to chance is constructed similar to (13).

%Equation (18)
\begin{equation}
\begin{array}{r{}l}
 H_{0}:  d_{P}=0 \\
 H_{A}:  d_{P}  \neq 0
\end{array}
\end{equation}


The test statistic used for this hypothesis is calculated as

%Equation (19)
\begin{equation}
t=\frac{\hat{d}_{P}}{s_{\hat{d}_{P}}}
\end{equation}

with the standard deviation of the difference and degrees of freedom given by (16) and (12) respectively. Calculation of the test statistic gives t=328.020 with a p-value smaller than .001. Again, the p-value is very small, indicating that observing these data when the null hypothesis is true is highly unlikely.

5.2 Simultaneous Linear Regression Model

The parameter estimates for the linear terms are identical to those obtained in the separate regressions. Our interpretations of the estimated parameters in the simultaneous model are also identical to those of the simple linear models in 5.1. What differs is that the data are pooled to calculate a single error variance instead of two error variances as was done using separate models, one for each regression model. Consequently, the t-values testing whether the before and after the Montreal Protocol slopes are equal to zero are different, 239.554 and -29.410 respectively.

Because the estimated model parameters are the same as in the separate regressions, our point estimates of the slope difference and of predicted values difference are also the same. What differs when using the simultaneous regression model are our inferences based on the stochastic part of the model. With a single error variance for the entire model, we construct our confidence intervals and t-tests as before except with the assumption of homogeneous rather than heterogeneous variances.

The confidence interval for the difference between the slopes is calculated as before, using (9) with the standard deviation of the difference defined by (10). However, the slope variances, (11), do not have the same values as previously since the error term used in their calculation is estimated from all the data. The degrees of freedom for the critical t value are calculated using

%Equation (20)
\begin{equation}
d.f.=n-4
\end{equation}

Therefore, the 95% confidence interval for the slope difference is 11.545±.146.

The value of t-statistic used to test the hypothesis of no slope differences in  (13), is computed as in (14).  The standard deviation of the slope difference is calculated as in (10). The t-value is 155.278. Evaluated with 265 degrees of freedom, this t-value results in a p-value of 3.158x10^-262. Again, this p-value is very small indicating that the probability of observing these data when the null hypothesis is true is highly unlikely.

The confidence interval for the difference between the two predicted values is computed using (15) with the standard deviation of the difference defined in (16) and V_i defined as

%Equation (21)
\begin{equation}
V_{i}=s^2\left[ \frac{1}{n_{i}}+\frac{(x-\bar{x}_i)^2}{\sum(x_{ij}-\bar{x}_i)^2}\right]
\end{equation}

This results in a 95% confidence interval for the predicted values difference of 198.818±2.366.

The test statistic used to test the hypothesis is calculated as (19) with the standard deviation of the difference and degrees of freedom given by (16) and (20) respectively. Calculation of the test statistic gives t=371.288 with a p-value so small neither the algorithms used in either Excel or SPSS are able to calculate its value. Again, we reject the null hypothesis in favor of the alternative that there is a difference between predicted values that is not due to chance.

6. Evaluation of the Models

We introduce some form of evaluation of the models in classes at every level, although the extent of the evaluation varies dramatically. For introductory classes, we confine ourselves to defining, computing, reporting, and interpreting the R² value. We avoid calling it the coefficient of determination, since the words are not intrinsically enlightening. Instead we call it a measure of the goodness of fit, a phrase that is jargon, but also descriptive. We define it as both the ratio of the regression sum of squares divided by the total sum of squares and, in the case of simple linear regression, the square of the correlation coefficient between x and y. The students are taught to interpret the R² value as the proportion of the variance in the data explained by the model.

The goodness of fit, measured by the proportion of variability in the data accounted for by the model, is extremely good for each of the models. The R² values are .996 and .980 for the simple linear regression models fit to data observed before and after the Montreal Protocol respectively. When combining the models, a single error variance is estimated for the simultaneous model. Therefore, the R² value combines the model sums of squares of both individual models in the numerator and the total sums of squares of both individual models in the denominator, and is equal to .995. This is between the R² values of the two separate linear models. It, too, is extremely high.

Additional evaluation of the models is performed in the second introductory and advanced courses. In both courses, we examine the residuals plots shown in Figures 4 and 5.

Figure 4. Residual Values for the Simple Linear Model fit to Pre-Montreal Protocol Data

Figure 5. Residual Values for the Simple Linear Model fit to Post-Montreal Protocol Data


In both residual plots, there is a noticeable pattern that is not consistent with the assumption of the linear model that the errors are normally distributed about zero with a constant variance. In both plots, there is a curvilinear pattern in the residuals. Therefore, a quadratic term was added to both of the models to create the quadratic model

%Equation (22)
\begin{equation}
\begin{equation}
\begin{array}{r{}l}
\hat{\mu}_{y_{1} \vert x}} =\hat{\beta}_{10}+\hat{\beta}_{11}x +\hat{\beta}^2_{12}\\
\hat{\mu}_{y_{2} \vert x}} =\hat{\beta}_{20}+\hat{\beta}_{21}x +\hat{\beta}^2_{22}
\end{array}
\end{equation}
\end{equation}
 

The results are displayed in Tables 6 and 7.

Table 6. ANOVA Table for the Simple Linear Regression Models

Period	Source	Sums of Squares	df	Means Square	F	p
Before 1990	Model	203437.656	2	101718.828	27487.316	.000
	Linear	203118.741	1	203118.741
	Quadratic	318.915	1	318.915
	Error	555.086	150	3.70057333
After 1994	Model	3119.199	2	1559.5995	5086.140	.000
	Linear	3090.876	1	3090.876
	Quadratic	28.323	1	28.323
	Error	34.650	113	0.30663717

 

Table 7. Parameter Estimates for the Simple Linear Regression Models

Period	Parameter	Estimate	Standard Error	t	p
Before 1990	β10	427555.191	48109.963	8.887	.000
	β11	-440.634	48.511	-9.083	.000
	β12	.114	.012	9.283	.000
After 1994	β20	-280061.630	29041.620	-9.643	.000
	β21	282.180	282.180	29.044	.000
	β22	-.071	-.071	.007	.000

The R² values were .997 and .989 for the two periods respectively. The parameter estimates for the linear term in both regression equations were highly significant with t-values of -9.083 and 9.716 respectively. The parameter estimates for the quadratic terms were highly significant as well with t-values of 9.283 and -9.779 respectively. It is worthy of note that the signs of the quadratic terms indicate whether the change in atmospheric concentrations of CFCs is accelerating up, if the term is positive, or accelerating down, if the term is negative.

Review of the residual plots from the quadratic models shows that the expected pattern of error randomly distributed about zero with a constant variance is now plausible, and a deviation from that pattern is less noticeable. For a class that includes the topic of autocorrelation, connecting the errors from time to time will reveal that there is still a pattern of autocorrelation in the data that could be investigated by further analysis.  Although those analyses are not considered within the scope of this paper, these data could be a useful example in a course that covers autocorrelation.

Although the quadratic model is statistically justified in both models, i.e. the quadratic terms account for a statistically significant proportion of the variability in the data, the proportions of variance explained are very small compared to the linear terms. For the purposes of analyzing these data in introductory classes, the quadratic terms have not been used when investigating the primary research question, the impact of the Montreal Protocol. The justification for this is threefold. First, introduction of the quadratic terms makes interpretation of the model more difficult if the purpose of the class is to introduce specific concepts, like rates of change, rather than to be statistically exhaustive. Second, even if we felt compelled to select the best model statistically, the quadratic model would be prone to larger error when extrapolating outside the range of the data, hence requiring a greater degree of caution. In this case, using a linear model without the quadratic term will underestimate the difference between predicted values at some point in the future outside the range of the data. Even so, the differences are highly significant. Third, once a firm grasp of the linear model has been achieved, then further refinement of the model by adding the quadratic term can be introduced. Emphasis in this paper is placed on an understanding of the linear model.

7. Conclusion

This case study tells a compelling story of international cooperation resulting in the successful collaboration on a global environmental problem. Further, evidence of this success can be encapsulated in a simple set of data that is accessible to students in an introductory course, yet complex enough to allow for use in more advanced courses. This case study also provides a rich context within which to introduce and explore many of the concepts central to the statistical analysis of data, the assumptions entailed by methods of measurement, sampling, and analysis. Lastly, there is a moral to the story. Even though a complete evaluation of success of the Montreal Protocol with respect to ozone layer depletion is beyond the scope of this paper, it is clear that we can claim the Montreal Protocol had a real, positive effect on the levels of atmospheric CFC concentration. So we return to the beginning of the story and ask the students to ponder whether similar international efforts could have similar effects on other global problems.

---

Appendix

Instructions to Download Mauna Loa Monthly Atmospheric CFC Data

The data used in this paper can be found at the World Data Centre for Greenhouse Gases (WDCGG) web site maintained by the Japanese Meteorological Agency in cooperation with the World Meteorological Organization. Navigation steps to this file are:

1.  Go to the web site’s main page, gaw.kishou.go.jp/wdcgg/
2. On the left side of the page, click on the navigation rectangle labeled "Data/Quick Plot".
3. Scroll down through the list of collection sites to "Mauna Loa". This site is listed twice. Select the one that has "CFCs" in the list of parameters shown at the far right.
4. Scroll down through the list of parameters and click on "CFCs (flask)".
5. In the table entitled "MONTHLY Data Total", select the cell that has "cfc11 (flask)" as the row heading and "Data" as the column heading. The link is labeled "27.4K", which is the size of the file.
6. This link is an ftp URL that downloads the ASCII data file into your browser. The direct link is ftp://gaw.kishou.go.jp/pub/data/current/cfcs/cfc11/monthly/mlo519n00.noaa.as.fl.cfc11.nl.mo.dat.txt

---

Acknowledgment

The author would like to acknowledge the work of the National Oceanic and Atmospheric Administration for collection of the data used in this paper.

---

References

Farman, J. C., Gardiner, B. G. and Shanklin, J. D. (1985), "Large Losses of Total Ozone in Antarctica Reveal Seasonal CLOx/NOx Interaction", Nature, 315, 207-10.

Molina, M. J. and Rowland, F. S. (1974), "Stratospheric Sink for Chlorofluoro-methanes: Chlorine Atom Catalyzed Destruction of Ozone", Nature, 249, 810-14.

Ott, L. (2001) An Introduction to Statistical Methods and Data Analysis. Pacific Grove: Duxbury.

Rees, D. G. and Henry, J. K. (1988), "On comparing the predicted values from two simple linear regression lines", The Statistician, 37, 299-306.

Rowland, F. S. and Molina, M. J. (2007), "The CFC-ozone puzzle: environmental science in the global arena". In Kaniaru, D. (ed.) The Montreal Protocol Celebrating 20 Years of Environmental Progress Ozone Layer and Climate Protection.  London: Cameron May.

Satterthwaite, F. E. (1946), "An approximate distribution of estimates of variance components", Biometrics Bulletin, 2, No. 6, 110-114

Shende, R. (2007), "From Montreal to Kyoto: The Refrigeration Industry’s Journey Toward Sustainability". In Kaniaru, D. (ed.) The Montreal Protocol Celebrating 20 Years of Environmental Progress Ozone Layer and Climate Protection.  London: Cameron May.

UNEP (2007), Report of the UNEP Technology and Economic Assessment Panel, United Nations Environmental Programme, Montreal Protocol On Substances that Deplete the Ozone Layer. http://www.unep.ch/ozone/Assessment_Panels/TEAP/Reports/TEAP_Reports/Teap_progress_report_April2007.pdf

Welch, B. L. (1938) "The significance of the difference between two means when the population variances are unequal", Biometrika 29, 350-362

World Meteorological Organization (1988), Report of the International Ozone Trends Panel—1988 (Report 18, Global Ozone Research and Monitoring Project, World Meteorological Organization, Geneva).

World Meteorological Organization (1986), Atmospheric Ozone 1985 (Report 16, Global Ozone Research and Monitoring Project, World Meteorological Organization, Geneva).

---

Dean Nelson
University of Pittsburgh at Greensburg
150 Finoli Drive
Greensburg, PA 15601
E-mail: den@pitt.edu
Phone:724-838-8044
Fax: 724-836-7172

---

Volume 17 (2009) | Archive | Index | Data Archive | Resources | Editorial Board | Guidelines for Authors | Guidelines for Data Contributors | Home Page | Contact JSE | ASA Publications