The Role of Models in Variance Estimation
 for Establishment Surveys: 
An Introductory Overview Lecture

Phillip S. Kott
National Agricultural Statistics Service
June 21, 2007
pkott@nass.usda.gov
703 877-8000 x 102
Outline 

Why Do We Need Models?
The Role of Models in Establishment Surveys
The Role of Models in Variance Estimation
Prediction Models in Nonresponse Adjustment
Extensions
Concluding Remark

Why Do We Need Models?

To make statistically rigorous inferences from a convenience sample or when the sample size is too small for randomization- based (design-based) inference.

To choose the best among reasonable-sounding randomization-based estimation strategies. 

To simplify variance estimation under an unequal probability-of-selection scheme.

To speed up the asymptotics with middling-sized samples.   
This will often improves the accuracy of coverage intervals.

To put statistical structure on “ad-hoc” practices.
This provides us with tools for evaluating their strengths and weaknesses.

To estimate totals and their variances in the face of nonresponse 
or coverage errors. 

Model-based Estimators (for totals)

Goal:   Estimate a population (U) total, 
		with a sample, S, of n (out of N ) elements.

Given:  An auxiliary variable, xk  S, about which  is
 known. 

The auxiliary variable may come from administrative data or a census.  It could trivially be 1 for all elements in the population. 
A Motivating Example
 
A population of farms, where
xk is the total land (believed to be) on farm k, and
yk is the planted acres of a particular crop (say corn) on the farm. 

The total land value is assumed to be known before sampling and enumeration.  It cannot be changed by survey information even if found to be in error.  
If we assume that each yk U satisfies the linear prediction model: 

, then the ratio estimator,
  
,
is a (model) unbiased estimator (predictor) for Ty in the sense that 

when the sampling mechanism is ignorable:

.

This ignorability assumption is unnecessary for 
simple random samples and cutoff samples (so long as the population values are unchanged by survey information),  but is needed for convenience samples. 

Model Groups

Some populations naturally separate into G mutually exclusive groups.  If the group x-totals, , are known, 
then the separate ratio estimator :

is not only unbiased for Ty under the simple linear model, 

but also the group-ratio model: 

where (again) .

The simple ratio estimator, , is generally not unbiased under this model. 

When all the xk =1, the group-ratio model is called the “group- mean model.”  The separate ratio estimator has the familiar 
“poststratified” form:

Important distinction:  Although the G groups look like H design strata, the difference is that samples need not be selected independently across groups.

Prediction Form

Returning to the simple linear model, 
an estimator for Ty of the form: 

is model unbiased when  is an unbiased estimator for   
This estimator uses the model to predict the y-values for the nonsampled elements.

If we assume the are uncorrelated and each has variance ,
then the best linear unbiased (BLU) estimator for Ty is

This collapses into the ratio estimator if and only if 
.
Model-assisted Randomization-based Estimators

But models fails.

In particular,  may tend to increase or decrease along with xk.

Nevertheless, from a randomization-based point of view,   under simple random sampling is almost unbiased.  In fact, its relative randomization mean squared error will tend to zero as the sample size grows arbitrarily large:  it is randomization consistent.  

Moreover, if the ek = yk  Bxk, where B = ,  tend to be smaller than the , then the ratio estimator will have less randomization mean squared error than the expansion estimator, 

In other words, 
, 
however imperfect,  is a better implicit prediction model than 
.

Combined Mean Squared Error

Without a model, it is often impossible to choose among reasonable randomization-based estimation strategies with identical sample sizes.   As a result, it is often helpful to:
 
1. 	Restrict attention (if possible) to almost randomization-unbiased or randomization-consistent estimators, and 

2.	Look at the model expectation of the randomization mean squared error (mse) of this strategy. 

Suppose we want to estimate Ty with the Hajek estimator, 
                         
where the element selection probabilities, the , can be anything. 

The Hajek estimator is unbiased under the linear model, , and is almost randomization unbiased. 

When the are uncorrelated and each has variance , 
one can show that for a fixed (expected) sample size n, the (asymptotic) combined mse of  is minimized under Brewer selection; that is, when 

if all   
(otherwise k should be selected with certainty).
Although the  need only be known up to a constant for Brewer selection, even that is unusual.   

In establishment surveys it is often speculated that  for some  between 1/2 and 1 (note that xk must be known for all elements in U  to use this).

Getting the  right is unnecessary for randomization consistency or model unbiasedness. 

The Hajek estimator under stsrs is called the combined ratio estimator,

where  when k is in (the sample in stratum h ). 
 
It is randomization consistent and model unbiased under the simple linear model but not the group-ratio model.
Estimating the Model Variance of 

Observe that under the simple linear model:
                
Thus, the model bias of  is zero.   

If the are uncorrelated and each has variance ,  
then the model variance of  is

which can be estimated in a unbiased fashion if we knew an unbiased estimator for the .    

If we assume  for a known set of  (say  for some ),
then

is an unbiased estimator for 
Since assumptions about the  are very often dubious, a more robust approach to modeling variance estimation is advisable. 
Using a Robust Estimator for 

For k  S, define  
where  

This is an unbiased estimator for  when ,
and nearly (asymptotically) unbiased otherwise because

 .

Since  implies 

is an exactly unbiased as an estimator for the model variance of when . 

Moreover,  is almost unbiased as long as both n and N/n are large, in which case  can be approximated by   and   by  

.

N/n large means finite population correction is ignorably small 
often, but not always, the case with establishment surveys.  

Estimating the Randomization MSE
of the Hajek Estimator

A conventional nearly unbiased randomization mse estimator is

where   

This complex Horvitz-Thompson form simplifies under many commonly-used designs. 

For the combined ratio estimator under stsrs, , it translates into

For the special case of  under srs,   H = 1, and 

 also estimates the (large-sample) combined mse of .

We can estimate the combined mse of the Hajek estimator in general with
.

Here  is an element-specific model-based finite population correction (fpc) term.  

Its average values is   (a pooled, model-based fpc term), not the ad hoc 1 – n/N.
Systematic Sampling

It is impossible to estimate the randomization mse of a strategy employing systematic sampling from a purposefully-ordered list. 

If one assumes the selection mechanism is ignorable, however, one can estimate the combined mean squared error of this strategy. 

Usually when the ignorability assumption is violated, estimates of combined mse will be biased upward. 
                 Randomization-assisted Model-based
Variance Estimation

The weighted residual variance estimator for  is   

where  and 

It is a good estimator of randomization mse as vconv and a better estimator for model variance when all the  

Even better estimators for the model variance of the combined ratio estimator and the general Hajek estimator are

and
.

retains the large-sample randomization-based properties of , and is nearly model unbiased when .

 is also nearly model unbiased when , but has a small downward bias.  A slightly improved version replaces the 
 with    or, better yet,                                                                                     
                             Adjusting for Nonresponse

Let R be the respondent sample, and suppose the population can be separated into G groups such that the model 

holds with
.
The key assumption in prediction modeling for unit nonresponse is the ignorability of the response mechanism.   

It is not necessary for all the elements in a group to have the same probability of response.  That is the assumption of an alternative and complementary paradigm: quasi-random response modeling.  

Let                  
where   and   can be either  or . 
This combined-unbiased estimator generalizes the separate ratio estimator. 

 can be put in reweighted (or calibration-weighted) form as

where 

                                               for 

We can compute the model variance of  as

with 

There is also a randomization-based component of the combined mse when the  are themselves estimated.  

Surveys are usually designed to estimate totals for a number of variables, for example, not only total corn acres but total number of corn farms.  

Consequently, reweighting for unit nonresponse is often done by setting all the xk = 1, so that the same set of adjusted weights, {wk}, can be used for every survey variable.   

Item Nonresponse

Suppose there is item nonresponse for some y-variable closely related to an auxiliary x-variable known for all sampled elements. A missing y-value is often imputed with something in this form: 

where the akj can be anything from dj,  through 1/xj,  to
1 for a randomly chosen member of Rg and 0 for all others. 

In each case, we can, for variance-estimation purposes, rewrite the estimated total in the form of a reweighted estimator 
(there is a possibly different reweighted estimator for each survey variable) and compute 

        .

There are other approaches. 

Extensions

When the frame exhibits coverage errors (duplications or missing population elements), and true group population totals are known for some auxiliary variable x, then 
                                 
is a model unbiased estimator for Ty under the group-ratio model, assuming the coverage mechanism is ignorable.   serves as a model (and combined) variance estimator.

We can extend the framework to incorporate additional phases of sampling, with cluster sampling in the original phase(s).  

We can also incorporate a more complex set of benchmark variables than the estimated group totals for a scalar x.  

We can incorporate nonlinear prediction models for the yk.   
Some care must be taken, however, because  is itself a linear function of random variables under the prediction model. 

Concluding Remark

When all the sampling fractions (the nh/Nh) are small enough to ignore,  

 is a good estimator for the randomization mean squared error of estimator under stsrs and an even better estimator of its model variance.   

It can be shown that  is asymptotically closer to the stratified jackknife variance estimator than to the more conventional 

That is why I argue that the stratified jackknife should be viewed as a robust (to misspecification of the ) model-based variance estimator rather than a randomization-based variance estimator.   
References 

Brewer, K. R. W. (1963). Ratio estimation and finite populations: some results deductible
from the assumption of an underlying stochastic process.  Australian Journal of
Statistics 5, 93-105.
Brewer, K. R. W. (2002).  Combined survey and sampling inference: Weighing Basu’s
 elephants.  London: Arnold.  
Deville, J- C. and C- E. Särndal (1992).  Calibration estimators in survey sampling.  
Journal of the American Statistical Association 87, 376.
Isaki, C.T. and W. A. Fuller (1982).  Survey design under the regression super-population
model.   Journal of the American Statistical Association 77, 89-96.
Kott, P. S. (2004).  Randomization-assisted model-based survey sampling.  Journal of
Statistical Planning and Inference 48, 263-277.  
Kott, P. S. and Bailey, J. T. (2000).  The theory and practice of maximal Brewer selection. 
Proceedings of the Second International Conference on Establishment Surveys,
invited papers,  269-278.
Kott, P. S. and K. R. W. Brewer (2001). Estimating the model variance of a randomization-
consistent regression estimator.  Proceedings of the Section on Survey Research	
Methods, American Statistical Association, Washington DC.
Särndal, C- E, B. Swensson, and J. Wretman (1989).  The weighted residual technique for
estimating the variance of the general regression estimator of a finite population total.  Biometrika 76, 527-537. 
Valliant, R, Dorfman, A.H., and Royall, R.M. (2000). Finite population sampling and
inference: a prediction approach.  New York: Wiley.  

The full paper is available at my website:
http://www.nass.usda.gov/research/OD5.htm