Density Estimation

14 Aug 2011 11:00

---

Yet Another Inadequate Placeholder, spun off from statistics.

Two topics of particular interest: estimating conditional densities, and estimating the densities of short subsequences from time series.

See also: density estimation on graphical models

Recommended:
• Luc Devorye and Gabor Lugosi, Combinatorial Methods in Density Estimation [comments]
• Peter Hall, Jeff Racine and Qi Li, "Cross-Validation and the Estimation of Conditional Probability Densities", Journal of the American Statistical Association 99 (2004): 1015--1026 [PDF]
• Bruce E. Hansen
  • "Nonparametric Conditional Density Estimation" [PDF preprint, 2004]
  • "Nonparametric Estimation of Smooth Conditional Distributions" [Preprint]
• Jeffrey S. Racine, "Nonparametric Econometrics: A Primer", Foundations and Trends in Econometrics 3 (2008): 1--88 [Good primer of nonparametric techniques for regression, density estimation and hypothesis testing; next to no economic content (except for examples). Presumes reasonable familiarity with parametric statistics. PDF reprint]
• Jeffrey S. Simonoff, Smoothing Methods in Statistics
• Larry Wasserman
  • All of Statistics
  • All of Nonparametric Statistics

Modesty forbids me to recommend:
• The lecture on density estimation (currently no. 28) in my data mining class notes

To read:
• Andrew R. Barron and Chyong-Hwa Sheu, "Approximation of Density Functions by Sequences of Exponential Families", Annals of Statistics 19 (1991): 1347--1369
• Alain Berlinet, Gérard Biau and Laurent Rouvière, "Optimal L1 Bandwidth selection for variable kernel density estimates", Statistics and Probability Letters 74 (2005): 116--128 ["[O]ne can improve performance of kernel density estimates by varying the bandwidth with the location and/or the sample data at hand. Our interest in this paper is in the data-based selection of a variable bandwidth... an automatic selection procedure inspired by the combinatorial tools developed in Devroye and Lugosi... the expected L1 error of the corresponding selected estimate is up to a given constant multiple of the best possible error plus an additive term which tends to zero under mild assumptions"]
• Z. I. Botev, J. F. Grotowski, and D. P. Kroese, "Kernel density estimation via diffusion", Annals of Statistics 38 (2010): 2916--2957
• Serge Cohen, Erwan Le Pennec, "Conditional Density Estimation by Penalized Likelihood Model Selection", arxiv:1103.2021
• Tilman M. Davies, Martin L. Hazelton, Jonathan. C Marshall, "sparr: Analyzing Spatial Relative Risk Using Fixed and Adaptive Kernel Density Estimation in R", Journal of Statistical Software 39:1 (2011)
• Sam Efromovich
  • "Distribution estimation for biased data", Journal of Statistical Planning and Inference 124 (2004): 1--43
  • "Dimension Reduction and Adaptation in Conditional Density Estimation", Journal of the American Statistical Association 105 (2010): 761--774
• Bradley Efron and Robert Tibshirani, "Using Specially Designed Exponential Families for Density Estimation", Annals of Statistics 24 (1996): 2431--2461
• Evarist Giné and Hailin Sang, "Uniform asymptotics for kernel density estimators with variable bandwidths", arxiv:1007.4350
• Evarist Giné and Richard Nickl
  • "Adaptive estimation of a distribution function and its density in sup-norm loss by wavelet and spline projections", Bernoulli 16 (2010): 1137--1163
  • "Uniform limit theorems for wavelet density estimators", arxiv:0805.1406 = Annals of Probability 37 (2009): 1605--1646
  • "Confidence bands in density estimation", Annals of Statistics 38 (2010): 1122--1170
• Han Liu, John Lafferty and Larry Wasserman, "Tree Density Estimation", arxiv:1001.1557
• Han Liu, Min Xu, Haijie Gu, Anupam Gupta, John Lafferty, Larry Wasserman, "Forest Density Estimation", Journal of Machine Learning Research 12 (2011): 907--951
• Brendan P. M. McCabe, Gael M. Martin, David Harris, "Efficient probabilistic forecasts for counts", Journal of the Royal Statistical Society B 73 (2011): 253--272
• Abdelkader Mokkadem, Mariane Pelletier, Yousri Slaoui, "The stochastic approximation method for the estimation of a multivariate probability density", arxiv:0807.2960
• Andrew B. Nobel, Gusztav Morvai, Sanjeev R. Kulkarni, "Density estimation from an individual numerical sequence", IEEE Transactions on Information Theory 44 (1998): 537--541, arxiv:0710.2500
• Andriy Norets, "Approximation of conditional densities by smooth mixtures of regressions", Annals of Statistics 38 (2010): 1733--1766, arxiv:1010.0581
• Michael Nussbaum, "Asymptotic Equivalence of Density Estimation and Gaussian White Noise", Annals of Statistics 24 (1996): 2399--2430
• Bruno Pelletier, "Kernel density estimation on Riemannian manifolds", Statistics and Probability Letters 73 (2005): 297--304
• Alessandro Rinaldo, Aarti Singh, Rebecca Nugent, Larry Wasserman, "Stability of Density-Based Clustering", arxiv:1011.2771
• Alessandro Rinaldo and Larry Wasserman, "Generalized Density Clustering", Annals of Statistics 38 (2010): 2678--2722, arxiv:0907.3454
• Olga Y. Savchuk, Jeffrey D. Hart, and Simon J. Sheather, "Indirect Cross-Validation for Density Estimation", Journal of the American Statistical Association 105 (2010): 415--423
• Yuefeng Wu, Subhashis Ghosal, "Kullback Leibler property of kernel mixture priors in Bayesian density estimation", Electronic Journal of Statistics 2 (2008): 298--331, arxiv:0710.2746