Statistics

28 Jul 2008 18:25

---

An application of probability, with intimate ties to machine learning, non-demonstrative inference and induction.

Since June 2005, I have been a (very, very junior) professor of statistics. This made me interested in how to teach it.

See also: Properties vs. principles in defining "good statistics"

Things I need to learn more about:

Dependent data

Statistical inference for stochastic processes, a.k.a. time-series analysis. Signal processing and filtering. Spatial statistics.

Model selection

Gets its own notebook.

Adapting statistical procedures to data without losing validity

Sequential inference, adaptive sampling.

Model discrimination

That is, designing experiments so as to discriminate between competing classes of model. Adaptation to data issues here, too.

Rates of convergence of estimators to true values

Empirical process theory. (Cf. some questions in ergodic theory).

Estimating distribution functions

And estimating entropies, or other functionals of distributions.

Non-parametric methods

Both those that are genuinely distribution-free, and those that would more accurately be mega-parametric (even infinitely-parametric) methods, such as neural networks

Resampling methods

Including distribution-free resampling methods, especially for dependent data

Sufficient statistics

Get their own notebook.

Decision theory

Conventional, and the sorts with some connection to how real decisions are made.

Graphical models

Monte Carlo and other simulation methods

"De-Bayesing"

Ways of taking Bayesian procedures and eliminating dependence on priors, either by replacing them by initial point-estimates, or by showing the prior doesn't matter, asymptotically or hopefully sooner. See: Frequentist consistency of Bayesian procedures.

Information Geometry

Partial identification of parametric statistical models

Causal Inference

Recommended, non-technical:
• Francis Galton, "Statistical Inquiries into the Efficacy of Prayer," Fortnightly Review 12 (1872): 125--135 [online]
• Larry Gonick and Woollcott Smith, The Cartoon Guide to Statistics
• Ian Hacking, The Taming of Chance [Putting chance to work in the 19th century]
• D. Huff, How to Lie with Statistics
• Theodore Porter, The Rise of Statistical Thinking, 1820--1900
• Constance Reid, Neyman from Life [Biography of Jerzy Neyman, one of the makers of modern statistical theory, and, I am happy to say, among the brighter lights of my alma mater. Reid does an excellent job of explaining Neyman's work in terms accessible to the general reader. There is a new edition, titled simply Neyman, but otherwise unchanged.]

Recommended, technical, big pictures:
• Richard A. Berk, Regression Analysis: A Constructive Critique [My comments]
• Leo Breiman, "Statistical Modeling: The Two Cultures", Statistical Science 16 (2001): 199--231 [very much including the discussion by others and the reply by Breiman. Thanks to Chris Wiggins for alerting me to this.]
• Harald Cramér, Mathematical Methods of Statistics [Review]
• Earman, Bayes or Bust? A Critical Account of Bayesian Confirmation Theory
• C. David Garson, Statnotes: An Online Textbook
• Peter Guttorp, Stochastic Modeling of Scientific Data [Good introduction to using dependent data]
• Tony Lin [Prof. Dr. Lin was working on his doctorate when I was an undergrad at Berkeley; we became friends at the I-House, if that is the word I want for someone who offered to keep my brain alive in a jigger-glass and subject it to random electrical shocks ("Jzzt! Jzzt!"). But despite his questionable tastes in acquaintances, he's a damn good statistician and a model teacher.]
  • Virtual Statistics 50 [Intro. statistics]
  • Virtual Statistics 154A [Intro. statistics with algebra and calculus]
• Deborah Mayo, Error and the Growth of Experimental Knowledge [Review: We Have Ways of Making You Talk, or, Long Live Peircism-Popperism-Neyman-Pearson Thought!]
• NIST, Electronic Handbook of Statistical Methods [Full text free online]
• E. J. G. Pitman, Some Basic Theory for Statistical Inference [Review: Intermediate Statistics from an Advanced Point of View]
• Jorma Rissanen, Stochastic Complexity in Statistical Inquiry [Review: Less Is More, or, Ecce data!]
• John R. Taylor, An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements [a.k.a. "the book with the train-wreck on the cover"]
• Edward R. Tufte
  • The Visual Display of Quantitative Information
  • Visual Explanations
• Larry Wasserman
  • All of Statistics
  • All of Nonparametric Statistics

Recommended, technical, close-ups:
• A. C. Atkinson and A. N. Donev, Optimum Experimental Design [Review]
• F. Bacchus, H. E. Kyburg and M. Thalos, "Against Conditionalization," Synthese 85 (1990): 475--506 [Why "Dutch book" arguments do not, in fact, mean that rational agents must be Bayesian reasoners]
• David Blackwell and M. A. Girshick, Theory of Games and Statistical Decisions
• Leo Breiman, "No Bayesians in Foxholes", IEEE Expert: Intelligent Systems and Their Applications 12 (1997): 21--24 [PDF reprint; comments by Andy Gelman]
• Hwan-sik Choi and Nicholas M. Kiefer, "Differential Geometry and Bias Correction in Nonnested Hypothesis Testing" [PDF preprint via Kiefer]
• J. Bradford DeLong and Kevin Lang, "Are All Economic Hypotheses False?", Journal of Political Economy 100 (1992): 1257--1272 [PDF preprint. The point is about abuses of hypothesis testing, not economic hypotheses as such. Note that the preprint, at least, systematically swaps the label of "type I" and "type II" errors.]
• Devorye and Lugosi, Combinatorial Methods in Density Estimation
• Bradley Efron, "Maximum Likelihood and Decision Theory", The Annals of Statistics 10 (1982): 340--356 [JSTOR]
• Andrew Gelman and Iain Pardoe, "Average predictive comparisons for models with nonlinearity, interactions, and variance components", Sociological Methodology forthcoming (2007) [PDF preprint, Gelman's comments]
• Christian Gouriéroux and Alain Monfort, Simulation-Based Econometric Methods [Review: By Indirection Find Direction Out]
• Mark S. Handcock and Martina Morris, Relative Distribution Methods in the Social Sciences [Review: Beyond Mean and Deviance]
• Gary King, A Solution to the Ecological Inference Problem: Reconstructing Individual Behavior from Aggregate Data [Review]
• Solomon W. Kullback, Information Theory and Statistics
• Michael Lavine and Mark J. Schervish, "Bayes Factors: What They Are and What They Are Not" [PS preprint]
• J. F. Lawless and Marc Fredette, "Frequentist prediction intervals and predictive distributions", Biometrika 92 (2005): 529--542 ["Frequentist predictive distributions are defined as confidence distributions .... A simple pivotal-based approach that produces prediction intervals and predictive distributions with well-calibrated frequentist probability interpretations is introduced, and efficient simulation methods for producing predictive distributions are considered. Properties related to an average Kullback-Leibler measure of goodness for predictive or estimated distributions are given."]
• Lucien Le Cam
  • "Neyman and Stochastic Models" [PDF. Some vignettes of Neyman putting together models, and his model-building process.]
  • "Maximum Likelihood; An Introduction" [PDF. Not really an introduction, but rather a collection of examples of where it just does not work, or at least doesn't work well.]
• Erich L. Lehmann, "On likelihood ratio tests", math.ST/0610835
• Bing Li, "A minimax approach to consistency and efficiency for estimating equations," Annals of Statistics 24 (1996): 1283--1297 [online version]
• Deborah G. Mayo and D. R. Cox, "Frequentist statistics as a theory of inductive inference", math.ST/0610846
• M. B. Nevel'son and R. Z. Has'minskii, Stochastic Approximation and Recursive Estimation
• David Pollard
  • "Asymptotics via Empirical Processes", Statistical Science 4 (1989): 341--354
  • Empirical Processes: Theory and Applications
• C. Scott and R. Nowak, "A Neyman-Pearson Approach to Statistical Learning", IEEE Transactions on Information Theory 51 (2005): 3806--3819
• Spyros Skouras, "Decisionmetrics: Towards a Decision-Based Approach to Econometrics," SFI Working Paper 2001-11-064 [Applies far outside econometrics. If what you really want to do is to minimize a known loss function, optimizing a conventional accuracy measure, e.g. least squares, can be highly counterproductive.]
• Aris Spanos
  • "The Curve-Fitting Problem, Akaike-type Model Selection, and the Error Statistical Approach" [Or: could your model selection tell you that Kepler is better than Ptolemy? Technical report, economics dept., Virginia Tech, 2006. PDF]
  • "Where do statistical models come from? Revisiting the problem of specification", math.ST/0610849
• UCB Statistics Technical Reports
• Sara van de Geer, Empirical Process Theory in M-Estimation [Finding non-asymptotic rates of convergence for common estimators]
• Quang H. Vuong, "Likelihood Ratio Tests for Model Selection and Non-Nested Hypotheses", Econometrica 57 (1989): 307--333
• Grace Wahba, Spline Models for Observational Data
• Michael E. Wall, Andreas Rechtsteiner and Luis M. Rocha, "Singular Value Decomposition and Principal Component Analysis," physics/0208101
• Achilleas Zapranis and Apostolos-Paul Refenes, Principles of Neural Model Identification, Selection and Adequacy, with Applications to Financial Econometrics

To read, textbooks, reviews, etc.:
• Hirotugu Akaike, Selected Papers
• Ole E. Barndorff-Nielsen and David R. Cox, Inference and Asymptotics
• Vic Barnett, Comparative Statistical Inference
• Bucklew, Large Deviation Techniques in Decision, Simulation, and Estimation
• Michael R. Chernick, Bootstrap Methods: A Practitioner's Guide
• Steve Fienberg, The Analysis of Cross-Classified Categorical Data
• Peter J. Huber, Robust Statistics
• Erich L. Lehmann, Elements of Large-Sample Theory
• National Institute of Standards and Technology, Engineering Statistics Handbook [All the sections I've looked at have been quite good.]
• Yudi Pawitan, In All Likelihood: Statistical Modeling and Inference Using Likelihood
• Aris Spanos
  • Statistical Foundations of Econometric Modeling
  • Probability Theory and Statistical Interference: Econometric Modeling with Observational Data
• Weisberg, Applied Linear Regression

To read, history and philosophy:
• Carolina Armenteros, "From Human Nature to Normal Humanity: Joseph de Maistre, Rousseau, and the Origins of Moral Statistics", Journal of the History of Ideas 68 (2007): 107--130 [Abstract, text links]
• Ian Hacking, "The Theory of Probable Inference: Neyman, Peirce and Braithwaite," in Science, Belief and Behavior: Essays in Honor of R. B. Braithwaite ed. D. H. Mellor
• Anders Hald, A History of Parametric Statistical Inference from Bernoulli to Fisher, 1713--1935 [Blurb]
• Trygve Haavelmo, "The Probability Approach in Econometrics", Econometrica 12 (1944, supplement): iii--115
• Kendall and Plackett (eds.), Studies in the History of Statistics and Probability
• Hugo A. Keuzenkamp, Probability, Econometrics and Truth: The Methodology of Econometrics [blurb]
• Kyburg, Uncertain Inference
• Mayo and Hollander (eds.), Accpetable Evidence: Science and Values in Risk Management
• Leland Gerson Neuberg, Conceptual Anomalies in Economics and Statistics: Lessons from the Social Experiment [blurb]
• Theodore Porter, Trust in Numbers
• Stephen M. Stigler
  • The History of Statistics: The Measure of Uncertainty before 1900
  • Statistics on the Table: The History of Statistical Concepts and Methods
• S. L. Zabell, Symmetry and Its Discontents: Essays on the History of Inductive Probability [blurb]

To read, research literature:
• Felix Abramovich, Yoav Benjamini, David L. Donoho and Iain M. Johnstone, "Adapting to Unknown Sparsity by controlling the False Discovery Rate", math.ST/0505374 [I don't really care about sparsity, but they promise novel relations between the FDR control and asymptotic minimaxity and complexity-penalized model selection.]
• Sophie Achard, "A quadratic measure dependence", math.ST/0609259 ["Asymptotic properties of a dimension-robust dependence measure are investigated. It is related to those used in independence tests, but is derivable, thus suitable for independent component analysis. An adjustable kernel allows to accelerate the convergence of the estimator without affecting the bias."]
• Shotaro Akaho, "A kernel method for canonical correlation analysis", cs.LG/0609071
• R. A. Bailey, Design of Comparative Experiments [Blurb]
• Ole E. Barndorff-Nielsen and David R. Cox, "Prediction and Asymptotics", Bernoulli 2 (1996): 319--340
• Ole E. Barndorff-Nielsen, David R. Cox and Claudia Klüppelberg (eds.), Complex Stochastic Systems
• Roger Barlow, "Asymmetric Errors", physics/0401042
• Alain Berlinet, Gérard Biau and Laurent Rouvière, "Optimal L1 Bandwidth selection for variable kernel density estimates", Statistics and Probability Letters 74 (2005): 116--128 ["[O]ne can improve performance of kernel density estimates by varying the bandwidth with the location and/or the sample data at hand. Our interest in this paper is in the data-based selection of a variable bandwidth... an automatic selection procedure inspired by the combinatorial tools developed in Devroye and Lugosi... the expected L1 error of the corresponding selected estimate is up to a given constant multiple of the best possible error plus an additive term which tends to zero under mild assumptions"]
• Patrice Bertail, Paul Doukhan and Philippe Soulier (eds.), Dependence in Probability and Statistics ["recent developments in ... probability and statistics for dependent data... from Markov chain theory and weak dependence with an emphasis on ... dynamical systems, to strong dependence in times series and random fields. ... section on statistical estimation problems and specific applications". Full blurb, contents]
• Rabi Bhattacharya and Vic Patrangenaru, "Large sample theory of intrinsic and extrinsic sample means on manifolds--II", math.ST/0507423 = Annals of Statistics 33 (2005): 1225--1259 [I need a notebook on statistics on manifolds, as opposed to statistical manifolds]
• G. Biau and L. Gyorfi, "On the Asymptotic Properties of a Nonparametric $-Test Statistic of Homogeneity", IEEE Transactions on Information Theory 51 (2005): 3965--3973
• David R. Bickel, "Selecting an optimal rejection region for multiple testing: A decision-theoretic alternative to FDR control, with an application to microarrays," math.PR/0212028
• David R. Bickel and Rudolf Fruehwirth, "On a Fast, Robust Estimator of the Mode: Comparisons to Other Robust Estimators with Applications", math.ST/0505419
• Peter J. Bickel, C. A. J. Klaassen, Y. Ritov and J. A. Wellner, Efficient and Adaptive Estimation for Semiparametric Models
• Peter J. Bickel and Y. Ritov, "Non-Parametric Estimators Which Can Be `Plugged-In' " UCB Stat. Tech. Rep. 602 [abstract, pdf]
• L. Birge, "A New Lower Bound for Multiple Hypothesis Testing", IEEE Transactions on Information Theory 51 (2005): 1611--1615
• Yu. I. Bogdanov, "Statistical Inverse Problem," physics/0211109 [A new density estimator]
• Borowiak, Model Discrimination for Nonlinear Regression Models
• Adrian W. Bowman and Adelchi Azzalini, Applied Smoothing Techniques for Data Analysis: The Kernel Approach with S-Plus Illustrations
• Thomas Brambor, William Roberts Clark and Matt Golder, "Understanding Interaction Models: Improving Empirical Analyses", Political Analysis 14 (2006): 63--82
• A. R. Brazzale, A. C. Davison and N. Reid, Applied Asymptotics: Case Studies in Small-Sample Statistics [blurb]
• Lawrence D. Brown and Mark G. Low, "Asymptotic Equivalence of Nonparametric Regression and White Noise", Annals of Statistics 24 (1996): 2384--2398 [JSTOR]
• Dizza Bursztyn and david M. Steinberg, "Comparison of designs for computer experiments", Journal of Statistical Planning and Inference 136 (2006): 1103--1119
• T. Tony Cai and Mark G. Low, "An adaptation theory for nonparametric confidence intervals", Annals of Statistics 32 (2004): 1805--1840 = math.ST/0503662
• Emmanuel Candes and Terence Tao, "Near Optimal Signal Recovery from Random Prjoections and Universal Encoding Strategies", math.CA/0410542
• Herve Cardot, Andre Mas and Pascal Sarda, "CLT in Functional Linear Regression Models", math.ST/0508073
• Djalil Chafai and Didier Concordet, "On the strong consistency of approximated M-estimators", math.ST/0507102 [Sounds cool...]
• In Hong Chang and Rahul Mukerjee, "Asymptotic results on the frequentist mean squared error of generalized Bayes point predictors", Statistics and Probability Letters 67 (2004): 65--71 [Note to self: file this one under "de-Bayesing".]
• Sandra Chapman, George Rowlands and Nicholas Watkins
  • "Extremum statistics: A framework for data analysis," cond-mat/0106015
  • "Extremum Statistics and Signatures of Long Range Correlations," cond-mat/0106015
  • "The relationship between extremum statistics and universal fluctuations," cond-mat/0007275
• Snigdhansu Chatterjee and Arup Bose, "Generalized bootstrap for estimating equations", math.ST/0504515 = Annals of Statistics 33 (2005): 414--436
• Fateh Chebana, "On the optimization of the weighted Bickel-Rosenblatt test", Statistics and Probability Letters 68 (2004): 333--345
• Xiaohong Chen, Markus Reiss, "On rate optimality for ill-posed inverse problems in econometrics", arxiv:0709.2003 [Non-parametric instrumental variables]
• Cheng Hsiao, Analysis of Panel Data [blurb]
• N. N. Chentsov, Statistical Decision Rules and Optimal Inference
• H. Chernoff, "A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the Sum of Observations," Annals of Mathematical Statistics 23 (1952): 493--507
• Arthur Cohen and Harold B. Sackrowitz, "Decision theory results for one-sided multiple comparison procedures", math.ST/0504505 = Annals of Statistics 33 (2005): 126--144
• Jerome Collet, "Estimating copula measure using ranks and subsampling: a simulation study", arxiv:0709.3860
• Daniel Commenges, Helene Jacqmin-Gadda, Cecile Proust, and Jeremie Guedj, "A Newton-Like Algorithm for Likelihood Maximization: The Robust-Variance Scoring Algorithm", math.ST/0610402
• J. Conrad, O. Botner, A. Hallgren and Carlos P. de los Heros, "Including Systematic Uncertainties in Confidence Interval Construction for Poisson Statistics," hep-ex/0202013
• Anirban DasGupta, Asymptotic Theory of Statistics and Probability [Blurb]
• Alexandre d'Aspremont, Onureena Banerjee, Laurent El Ghaoui, "First-order methods for sparse covariance selection", math.OC/0609812 ["Given a sample covariance matrix, we solve a maximum likelihood problem penalized by the number of nonzero coefficients in the inverse covariance matrix. Our objective is to find a sparse representation of the sample data and to highlight conditional independence relationships between the sample variables."]
• Herold Dehling (ed.), Empirical Process Techniques for Dependent Data
• Phoebus J. Dhrymes, Topics in Advanced Econometrics, vol. II: Linear and Nonlinear Simultaneous Equations
• David L. Donoho and Richard C. Liu, "The ``Automatic'' Robustness of Minimum Distance Functionals", Annals of Statistics 16 (1988): 552--586
• Mathias Drton and Seth Sullivant, "Algebraic statistical models", math.ST/0703609
• Sam Efromovich, "Distribution estimation for biased data", Journal of Statistical Planning and Inference 124 (2004): 1--43
• Bradley Efron and Robert Tibshirani, "Using Specially Designed Exponential Families for Density Estimation", Annals of Statistics 24 (1996): 2431--2461 [JSTOR]
• Jianqing Fan and Jian Zhang, "Sieve empirical likelihood ratio tests for nonparametric functions", Annals of Statistics 32 (2004): 1858--1907 = math.ST/0503667
• Julian J. Faraway, Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models
• Valerii V. Fedorov and Peter Hackl, Model-Oriented Design of Experiments
• Thomas S. Ferguson, A Course in Large Sample Theory
• Jean-David Fermanian and Bernard Salanié "A Nonparametric Simulated Maximum Likelihood Estimation Method", Econometric Theory 20 (2004): 701--734
• Ana K. Fermin and Carenne Ludena, "A Statistical view of Iterative Methods for Linear Inverse Problems", math.ST/0504064
• S. E. Fienberg, P. Hersh, A. Rinaldo and Y. Zhou, "Maximum Likelihood Estimation in Latent Class Models For Contingency Table Data", arxiv:0709.3535
• Magalie Fromont and Béatrice Laurent, "Adaptive goodness-of-fit tests in a density model", Annals of Statistics 34 (2006): 680--720 = math.ST/0607013
• Stephane Gaiffas, "Rates of convergence for pointwise curve estimation with a degenerate design", math.ST/0410354
• Seymour Geisser, Predictive Inference
• Andrew Gelman and Jennifer Hill, Data Analysis Using Regression and Multilevel/Hierarchical Models
• Andrew Gelman, Jennifer Hill and Masanao Yajima, "Why we (usually) don't have to worry about mutliple comparisons" [PDF preprint]
• Christopher R. Genovese and Larry Wasserman
  • "Confidence sets for nonparametric wavelet regression", math.ST/0505632 = Annals of Statistics 33 (2005): 698--729
  • "Adaptive Confidence Bands", math.ST/0701513
• Josep Ginebra, "On the Measure of the Information in a Statistical Experiment", Bayesian Analysis (2007): 167--212
• Jose M. Gonzalez-Barrios and Silvia Ruiz-Velasco, "Regression analysis and dependence", Metrica 61 (2005): 73--87
• Grassberger and Nadal (eds.), From Statistical Physics to Statistical Inference and Back
• Ulf Grenander, Abstract Inference
• Emmanuel Guerre and Pascal Lavergne, "Data-driven rate-optimal specification testing in regression models", math.ST/0505640 = Annals of Statistics 33 (2005): 840--870
• Laszlo Gyorfi et al., A Distribution-Free Theory of Nonparametric Regression
• Wolfgang Hárdle, Applied Nonparametric Regression [blurb]
• David A. Hensher et al. Applied Choice Analysis: A Primer ["Application of quantitative statistical methods to study choices made by individuals. This primer provides an introduction to the main techniques of choice analysis and also includes details on data collection and preparation, model estimation and interpretation and the design of choice experiments". Full blurb]
• Norbert Henze and Simos G. Meintanis, "Recent and classical tests for exponentiality: a partial review with comparisons", Metrica 61 (2005): 29--45
• Hettmansperger and McKean, Robust Nonparametric Statistical Methods
• Peter D. Hoff, "Marginal set likelihood for semiparametric copula estimation", math.ST/0610413
• Serkan Hosten, Amit Khetan and Bernd Sturmfels, "Solving the Likelihood Equations", math.ST/0408270
• Patrik O. Hoyer, "Non-negative sparse coding," cs.NE/0202009
• Tien-Chung Hu, Manuel Ordónez Cabrera and Andrei Volodin, "Asymptotic probability for the bootstrapped means deviations from the sample mean", Statistics and Probability Letters 74 (2005): 178--186 [The probability is supposedly obtained "without imposing any assumptions on joint distribution of the original sequence of random variables from which the bootstrap sample is withdrawn"]
• Hua Liang, "Comparison of curves based on a Cramér-von Mises statistic", Computational Statistics and Data Analysis 45 (2004): 805--812
• Stefano M. Iacus and Davide La Torre
  • "Approximating Distribution Functions by Iterated Function Systems," math.PR/0111152
  • "Nonparametric estimation of distribution and density functions in presence of missing data: an IFS approach," math.PR/0302016
• Sameer M. Jalnapurkar, "Learning a regression function via Tikhonov regularization", math.ST/0509420
• Ian H. Jermyn, "Invariant Bayesian estimation on manifolds", math.ST/0506296 = Annals of Statistics 33 (2005): 583--605
• D. F. Kerridge, "Inaccuracy and Inference", Journal of the Royal Statistical Society B 23 (1961): 184--194
• K. Kinoshita, "An unbinned goodness-of-fit test based on the random walk", physics/0312014
• B. Knuteson, H. Miettinen and L. Holmstrom, "alphaPDE: A New Multivariate Technique for Parameter Estimation," physics/0108002
• Roger Koenker, Quantile Regression [blurb]
• Michael R. Kosorok, Introduction to Empirical Processes and Semiparametric Inference [PDF preprint]
• Mikhail Kovtun, Igor Akushevich, Kenneth G. Manton and H. Dennis Tolley
  • "Linear Latent Structure Analysis: Mixture Distribution Models with Linear Constraints", math.PR/0507025
  • "A New Efficient Algorithm for Construction of LLS Models", math.PR/0507021
• Nicole Kraemer, Anne-Laure Boulesteix, Gerhard Tutz, "Penalized Partial Least Squares Based on B-Splines Transformations", math.ST/0608576
• Solomon Kullback, "Probability densities with given marginals," Annals of Mathematical Statistics 39 (1968): 1236--1243
• Masayuki Kumon and Akimichi Takemura, "On a simple strategy weakly forcing the strong law of large numbers in the bounded forecasting game", math.PR/0508190 ["In the framework of the game-theoretic probability of Shafer and Vovk (2001) it is of basic importance to construct an explicit strategy weakly forcing the strong law of large numbers (SLLN) in the bounded forecasting game. We present a simple finite-memory strategy based on the past average of Reality's moves, which weakly forces the strong law of large numbers with the convergence rate of $O(\sqrt{\log n/n})$.... We show that if Reality violates SLLN, then the exponential growth rate of Skeptic's capital process is explicitly described in terms of the Kullback divergence between the average of Reality's moves when she violates SLLN and the average when she observes SLLN."]
• Jon Lafferty and Larry Wasserman, "Rodeo: Sparse Nonparametric Regression in High Dimensions", math.ST/0506342 ["We present a method for simultaneously performing bandwidth selection and variable selection in nonparametric regression."]
• Mikhail Langovoy, "Data-driven goodness-of-fit tests", arxiv:0708.0169
• E. L. Lehmann and Joseph P. Romano, "Generalizations of the Familywise Error Rate", math.ST/0507420 = Annals of Statistics 33 (2005): 1138--1154
• Feng Liang, Sayan Mukherjee, Mike West, "The Use of Unlabeled Data in Predictive Modeling", arxiv:0710.4618 = Statistical Science 22 (2007): 189--205
• Oliver Linton and Zhijie Xiao, "A Nonparametric Regression Estimator That Adapts To Error Distribution of Unknown Form", Econometric Theory 23 (2007): 371--413
• Richard Lockhart and Federico O'Reilly, "A note on Moore's conjecture", Statistics and Probability Letters 74 (2005): 212--220 ["We establish the conjecture of Moore ... that the usual plug-in estimate of a distribution function and the Rao-Blackwell estimate of the distribution function are asymptotically equivalent for a wide class of exponential family distributions."]
• victor S. L'vov, Anna Pomyalov and Itamar Procaccia, "Outliers, Extreme Events and Multiscaling," nlin.CD/0009049
• Christian K. Machens, "Adaptive sampling by information maximization," physics/0112070
• Charles F. Manski, Identification for Prediction and Decision [Blurb]
• Laszlo Matyas, Generalized Method of Moments Estimation
• Robert Mariano, Til Schuermann and Melyvn J. Weeks (eds.), Simulation-Based Inference in Econometrics: Methods and Applications
• McCabe and Tremayne, Modern Asymptotic Theory
• Nicolai Meinshausen and John Rice, "Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses", math.ST/0501289
• Kevin Murphy and Brett Myors, Statistical Power Analysis: A Simple and General Model for Traditional and Modern Hypothesis Tests
• National Research Council
  • The Future of Statistical Software
  • Massive Data Sets
• Richard Nickl, "Donsker-type theorems for nonparametric maximum likelihood estimators", Probability Theory and Related Fields 138 (2007): 411--449
• Michael Nussbaum, "Asymptotic Equivalence of Density Estimation and Gaussian White Noise", Annals of Statistics 24 (1996): 2399--2430 [JSTOR]
• Adrian Pagan and Aman Ullah, Nonparametric Econometrics
• Bruno Pelletier, "Kernel density estimation on Riemannian manifolds", Statistics and Probability Letters 73 (2005): 297--304
• Donald A. Pierce and Dawn Peters, "Improving on exact tests by approximate conditioning", Biometrika 86 (1999): 265--277 [Via Andrew Gelman's blog; PDF reprint]
• Ramani S. Pilla, Catherine Loader and Cyrus Taylor, "A New Technique for Finding Needles in Haystacks: A Geometric Approach to Distinguishing Between a New Source and Random Fluctuations", physics/0505200
• Tomaz Podobnik and Tomi Zivko, "On Consistent and Calibrated Inference about the Parameters of Sampling Distributions", physics/0508017
• Thorsten Poeschel, Werner Ebeling, and Helge Rose, "Guessing probability distributions from small samples," cond-mat/0203467 = Journal of Statistical Physics 80 (1995): 1443
• Thorsten Poeschel and Jan A. Freund, "How to decide whether small samples comply with an equidistribution," cond-mat/0205225
• Odile Pons, "Bootstrap of means under stratified sampling", Electronic Journal of Statistics 1 (2007): 381--391 = arxiv:0709.3246
• Giovanni Punzi, "Ordering Algorithms and Confidence Intervals in the Presence of Nuisance Parameters", physics/0511202
• Sabbir Rahman and Mahbub Majumdar, "Maximum Entropy Multivariate Density Estimation: An exact goodness-of-fit approach", physics/0406023 [Looks like physicists rediscovering penalized maximum likelihood]
• Rajendran Raja, "Confidence Limits and their Robustness," physics/0207058
• Naren Ramakrishnan and Chris Bailey-Kellogg, "Sampling Strategies for Mining in Data-Scarce Domains," cs.CE/0204047
• C. Radhakrishna Rao, "Diversity: Its measurement, decomposition, apportionment and analysis", Sankhya: The Indian Journal of Statistics 44(A) (1982): 1--22 [Sankhya is not in JSTOR! Why is Sankhya not in JSTOR?!?!]
• J. C. W. Rayner and D. J. Best, Smooth Tests of Goodness of Fit
• R.-D. Reiss and M. Thomas, Statistical Analysis of Extreme Values: With Applications to Insurance, Finance, Hydrology and Other Fields
• Peter Riegler and Nestor Caticha, "Maxent Queries and Sequential Sampling," cond-mat/0010104
• Sylvain Rubenthaler, Tobias Ryden and Magnus Wiktorsson, "Fast simulated annealing in $\R^d$ and an application to maximum likelihood estimation", math.PR/0609353
• Cynthia Rudin, "Stability Analysis for Regularized Least Squares Regression", cs.LG/0502016
• Rustem and Howe, Algorithms for Worst-Case Design and Applications to Risk Management
• Ines Samengo, "Estimating probabilities from experimental frequencies," cond-mat/0201516
• Richard Samworth and Oliver Johnson, "The empirical process in Mallows distance, with application to goodness-of-fit tests", math.ST/0504424
• Thomas J. Santner, Brian J. Williams and William J. Note, Design and Analysis of Computer Experiments
• Mark Schervish, Theory of Statistics
• George A. F. Seber and C. J. Wild, Nonlinear Regression
• Sen, Sequential Nonparametrics
• Hichem Snoussi and Ali Mohammad-Djafari, "Penalized maximum likelihood for multivariate Gaussian mixture," physics/0111007
• James C. Spall, Introduction to Stochastic Search and Optimization [Book website]
• Aris Spanos, "Revisiting the Omitted Variables Argument: Substantive vs. Statistical Adequacy" [PDF preprint]
• Peter Stoica, Luzhou Xu and Jian Li, "A new type of parameter estimation algorithm for missing data problems", Statistics and Probability Letters 75 (2005): 219--229
• Ne-Zheng Sun, "Structure reduction and robust experimental design for distributed parameter identification", Inverse Problems 21 (2005): 739--758
• Thompson and Seber, Adaptive Sampling
• F. V. Tkachov, "Quasi-optimal observables: Attaining the quality of maximal likelihood in parameter estimation when only a MC event generator is available," physics/0108030
• Aad W. van der Vaart, Jon A. Wellner, "Empirical processes indexed by estimated functions", arxiv:0709.1013 ["We consider the convergence of empirical processes indexed by functions that depend on an estimated parameter $\eta$ and give several alternative conditions under which the ``estimated parameter'' $\eta_n$ can be replaced by its natural limit $\eta_0$ uniformly in some other indexing set $\Theta$"]
• Haonan Wang, J. S. Marron, "Object oriented data analysis: Sets of trees", arxiv:0711.3147 ["Object oriented data analysis is the statistical analysis of populations of complex objects"]
• Xiaogang Wang and James V. Zidek, "Selecting likelihood weights by cross-validation", math.ST/0505599 = Annals of Statistics 33 (2005): 463--500 ["The (relevance) weighted likelihood was introduced to formally embrace a variety of statistical procedures that trade bias for precision. Unlike its classical counterpart, the weighted likelihood combines all relevant information while inheriting many of its desirable features including good asymptotic properties. However, in order to be effective, the weights involved in its construction need to be judiciously chosen. Choosing those weights is the subject of this article in which we demonstrate the use of cross-validation. We prove the resulting weighted likelihood estimator (WLE) to be weakly consistent and asymptotically normal. An application to disease mapping data is demonstrated." Sounds interesting...]
• Holger Wendland, Scattered Data Approximation
• Halbert White
  • Asymptotic Theory for Econometricians [Useful source, it seems, for non-IID central limit theorems]
  • Estimation, Inference and Specification Analysis [On maximum-likelihood estimation and testing with mis-specified models, i.e., what happens when the actual distribution is nowhere in your class of admissible probability models. I've read about half of this.]
• Christopher K. I. Williams, "How to Pretend That Correlated Variables Are Independent by Using Difference Observations", Neural Computation 17 (2005): 1--6 ["In many areas of data modeling, observations at different locations (e.g., time frames or pixel locations) are augmented by differences of nearby observations.... These augmented observations are then often modeled as being independent. How can this make sense? We provide two interpretations, showing (1) that the likelihood of data generated from an autoregressive process can be computed in terms of 'independent' augmented observations and (2) that the augmented observations can be given a coherent treatment in terms of the products of experts model..."]
• Simon N. Wood, "Fast stable direct fitting and smoothness selection for Generalized Additive Models", arxiv:0709.3906
• Jeffrey M. Wooldridge, Econometric Analysis of Cross Section and Panel Data
• Hirokazu Yanagiharaa and Chihiro Ohmoto, "On distribution of AIC in linear regression models", Journal of Statistical Planning and Inference 133 (2005): 417--433