Some Applıcatıons of Entropy-Based Statıstıcs ın Lınear Regressıon Analysıs

Statistical entropy is a measure of variation of a distribution especially when the random variable is qualitative. Entropy-based statistics are also used to measure the degree of association between qualitative variables. Two measures of divergence, namely, Kullback-Leibler divergence and Jeffreys’ divergence are closely related to loglikelihood function. Thus these two entropy-based measures can be used in hypothesis testing procedures as well. In this study, we discuss how relative entropy measures are applied in testing some hypotheses and how useful they would be in regression analysis especially in determining influential observations.