Decomposition of Kullback-Leibler Risk with Applications to Exponential Families

• Activity Number: 585
• Type: Contributed
• Date/Time: Wednesday, August 4, 2010 : 2:00 PM to 3:50 PM
• Sponsor: IMS
• Author(s): Qiang Wu*+ and Paul Vos
• Companies: East Carolina University and East Carolina University
• Keywords: Kullback-Leibler Risk ; KL risk decomposition ; KL mean ; KL variance ; KL bias

Abstract

Since the Kullback-Leibler (KL) risk is defined without reference to parameterization, we require our estimators to be parameter invariant. For this reason, we define estimators to take values on the family of distributions so that estimates are probability distributions instead of parameter values. This construction allows a KL risk decomposition in a fashion parallel to the MSE decomposition for real valued parameter estimators. The KL decomposition consists of two nonnegative terms we call the KL variance and the squared KL bias. Each of these is defined using the KL mean which is a probability distribution. A dual version of the KL risk, as well as its corresponding decomposition, is also considered. Applications of the (dual) KL risks to distributions in the exponential family are given.