Calibration of Entropy of Continuous Distributions

• Activity Number: 444
• Type: Contributed
• Date/Time: Thursday, August 7, 2003 : 8:30 AM to 10:20 AM
• Sponsor: Section on Bayesian Stat. Sciences
• Author(s): Ali Dadpay*+ and Ehsan S. Soofi and Nader B. Ebrahimi
• Companies: University of Wisconsin, Milwaukee and University of Wisconsin and Northern Illinois University
• Keywords: concentration ; information ; uncertainty ; uniform distribution

Abstract

Entropy is used in many fields for measuring concentration, uncertainty, information, and as criteria for developing probability models and estimation. In the discrete case, entropy is nonnegative and a zero entropy indicates perfect concentration of the probability over a single point, the absence of uncertainty, and perfect information about the prediction of the outcome. However, in the continuous case, entropy takes values from negative to positive infinity. The entropy of uniform distribution over the unit interval is zero. This simple zero entropy distribution provides a useful calibration of the continuous entropy. Identifying zero entropy member(s) of a continuous family provides a calibration of entropy for the family in terms its parameters. Many well-known families of distributions have a unique zero entropy member, but for some families a zero entropy member does not exist and for some families there is no unique zero entropy member. We tabulate the unique zero entropy members of the well known families and provide contour plots for families with no unique zero entropy member. The idea is comparison of entropies across the parametric families.