User Settings
Open AccessPreprint

ESTIMATION OF ENTROPIES AND DIVERGENCES Via Nearest Neighbors

Nikolai Leonenko,Luc Pronzato,Vippal Savani-2008-01-01
10

TL;DRAbstract

We extend the results in [L. F. Kozachenko, N. N. Leonenko: On; statistical estimation of entropy of random vector, Problems Inform. Transmission 23 (1987), 95–101; Translated from Problemy Peredachi Informatsii 23 (1987),
\n... Inverardi: A new class of random vector entropy estimators and its applications in testing statistical hypotheses, J. Nonparametr. Statist. 17 (2005), 277–297] and show how kth nearest-neighbor distances in a sample of N i.i.d. vectors distributed with the probability density f can be used to estimate consistently Rény and Tsallis entropies of the unknown f under minimal assumptions. The method is extended to the estimation of statistical distances between two distributions in the case when one i.i.d. sample from each is available.

Chat with Paper

AI Agents for this Paper

We extend the results in [L. F. Kozachenko, N. N. Leonenko: On; statistical estimation of entropy of random vector, Problems Inform. Transmission 23 (1987), 95–101; Translated from Problemy Peredachi Informatsii 23 (1987),
\n... Inverardi: A new class of random vector entropy estimators and its applications in testing statistical hypotheses, J. Nonparametr. Statist. 17 (2005), 277–297] and show how kth nearest-neighbor distances in a sample of N i.i.d. vectors distributed with the probability density f can be used to estimate consistently Rény and Tsallis entropies of the unknown f under minimal assumptions. The method is extended to the estimation of statistical distances between two distributions in the case when one i.i.d. sample from each is available.

Keywords

EstimatorMathematicsMultivariate random variableEntropy (arrow of time)Tsallis entropyRandom variablek-nearest neighbors algorithmStatistical physics

Chat

Click to start Chat