Statistics

08 Apr 2008 20:46

An application of probability, with intimate ties to machine learning, non-demonstrative inference and induction.

Since June 2005, I have been a (very, very junior) professor of statistics. This made me interested in how to teach it.

Dependent data: Statistical inference for stochastic processes, a.k.a. time-series analysis. Signal processing and filtering. Spatial statistics.
Model selection: Gets its own notebook.
Adapting statistical procedures to data without losing validity: Sequential inference, adaptive sampling.
Model discrimination: That is, designing experiments so as to discriminate between competing classes of model. Adaptation to data issues here, too.
Rates of convergence of estimators to true values: Empirical process theory. (Cf. some questions in ergodic theory).
Estimating distribution functions: And estimating entropies, or other functionals of distributions.
Non-parametric methods: Both those that are genuinely distribution-free, and those that would more accurately be mega-parametric (even infinitely-parametric) methods, such as neural networks
Resampling methods: Including distribution-free resampling methods, especially for dependent data
Sufficient statistics: Get their own notebook.
Decision theory: Conventional, and the sorts with some connection to how real decisions are made.
Graphical models
Monte Carlo and other simulation methods
"De-Bayesing": Ways of taking Bayesian procedures and eliminating dependence on priors, either by replacing them by initial point-estimates, or by showing the prior doesn't matter, in a sooner-than-asymptotic sense.
Information Geometry
Partial identification of parametric statistical models
Causal Inference

Francis Galton, "Statistical Inquiries into the Efficacy of Prayer," Fortnightly Review 12 (1872): 125--135 [online]
Larry Gonick and Woollcott Smith, The Cartoon Guide to Statistics
Ian Hacking, The Taming of Chance [Putting chance to work in the 19th century]
D. Huff, How to Lie with Statistics
Theodore Porter, The Rise of Statistical Thinking, 1820--1900
Constance Reid, Neyman from Life [Biography of Jerzy Neyman, one of the makers of modern statistical theory, and, I am happy to say, among the brighter lights of my alma mater. Reid does an excellent job of explaining Neyman's work in terms accessible to the general reader. There is a new edition, titled simply Neyman, but otherwise unchanged.]

Richard A. Berk, Regression Analysis: A Constructive Critique [My comments]
Leo Breiman, "Statistical Modeling: The Two Cultures", Statistical Science 16 (2001): 199--231 [very much including the discussion by others and the reply by Breiman. Thanks to Chris Wiggins for alerting me to this.]
Harald Cramér, Mathematical Methods of Statistics [Review]
Earman, Bayes or Bust? A Critical Account of Bayesian Confirmation Theory
C. David Garson, Statnotes: An Online Textbook
Peter Guttorp, Stochastic Modeling of Scientific Data [Good introduction to using dependent data]
Tony Lin [Prof. Dr. Lin was working on his doctorate when I was an undergrad at Berkeley; we became friends at the I-House, if that is the word I want for someone who offered to keep my brain alive in a jigger-glass and subject it to random electrical shocks ("Jzzt! Jzzt!"). But despite his questionable tastes in acquaintances, he's a damn good statistician and a model teacher.]
- Virtual Statistics 50 [Intro. statistics]
- Virtual Statistics 154A [Intro. statistics with algebra and calculus]
Deborah Mayo, Error and the Growth of Experimental Knowledge [Review: We Have Ways of Making You Talk, or, Long Live Peircism-Popperism-Neyman-Pearson Thought!]
NIST, Electronic Handbook of Statistical Methods [Full text free online]
E. J. G. Pitman, Some Basic Theory for Statistical Inference [Review: Intermediate Statistics from an Advanced Point of View]
Jorma Rissanen, Stochastic Complexity in Statistical Inquiry [Review: Less Is More, or, Ecce data!]
John R. Taylor, An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements [a.k.a. "the book with the train-wreck on the cover"]
Edward R. Tufte
- The Visual Display of Quantitative Information
- Visual Explanations
Larry Wasserman
- All of Statistics
- All of Nonparametric Statistics

A. C. Atkinson and A. N. Donev, Optimum Experimental Design [Review]
F. Bacchus, H. E. Kyburg and M. Thalos, "Against Conditionalization," Synthese 85 (1990): 475--506 [Why "Dutch book" arguments do not, in fact, mean that rational agents must be Bayesian reasoners]
David Blackwell and M. A. Girshick, Theory of Games and Statistical Decisions
Leo Breiman, "No Bayesians in Foxholes", IEEE Expert: Intelligent Systems and Their Applications 12 (1997): 21--24 [PDF reprint; comments by Andy Gelman]
Hwan-sik Choi and Nicholas M. Kiefer, "Differential Geometry and Bias Correction in Nonnested Hypothesis Testing" [PDF preprint via Kiefer]
J. Bradford DeLong and Kevin Lang, "Are All Economic Hypotheses False?", Journal of Political Economy 100 (1992): 1257--1272 [PDF preprint. The point is about abuses of hypothesis testing, not economic hypotheses as such. Note that the preprint, at least, systematically swaps the label of "type I" and "type II" errors.]
Devorye and Lugosi, Combinatorial Methods in Density Estimation
Bradley Efron, "Maximum Likelihood and Decision Theory", The Annals of Statistics 10 (1982): 340--356 [JSTOR]
Andrew Gelman and Iain Pardoe, "Average predictive comparisons for models with nonlinearity, interactions, and variance components", Sociological Methodology forthcoming (2007) [PDF preprint, Gelman's comments]
Christian Gouriéroux and Alain Monfort, Simulation-Based Econometric Methods [Review: By Indirection Find Direction Out]
Mark S. Handcock and Martina Morris, Relative Distribution Methods in the Social Sciences [Review: Beyond Mean and Deviance]
Gary King, A Solution to the Ecological Inference Problem: Reconstructing Individual Behavior from Aggregate Data [Review]
Solomon W. Kullback, Information Theory and Statistics
J. F. Lawless and Marc Fredette, "Frequentist prediction intervals and predictive distributions", Biometrika 92 (2005): 529--542 ["Frequentist predictive distributions are defined as confidence distributions .... A simple pivotal-based approach that produces prediction intervals and predictive distributions with well-calibrated frequentist probability interpretations is introduced, and efficient simulation methods for producing predictive distributions are considered. Properties related to an average Kullback-Leibler measure of goodness for predictive or estimated distributions are given."]
Erich L. Lehmann, "On likelihood ratio tests", math.ST/0610835
Bing Li, "A minimax approach to consistency and efficiency for estimating equations," Annals of Statistics 24 (1996): 1283--1297 [onliine version]
Deborah G. Mayo and D. R. Cox, "Frequentist statistics as a theory of inductive inference", math.ST/0610846
M. B. Nevel'son and R. Z. Has'minskii, Stochastic Approximation and Recursive Estimation
David Pollard, "Asymptotics via Empirical Processes", Statistical Science 4 (1989): 341--354
C. Scott and R. Nowak, "A Neyman-Pearson Approach to Statistical Learning", IEEE Transactions on Information Theory 51 (2005): 3806--3819
Spyros Skouras, "Decisionmetrics: Towards a Decision-Based Approach to Econometrics," SFI Working Paper 2001-11-064 [Applies far outside econometrics. If what you really want to do is to minimize a known loss function, optimizing a conventional accuracy measure, e.g. least squares, can be highly counterproductive.]
Aris Spanos
- "The Curve-Fitting Problem, Akaike-type Model Selection, and the Error Statistical Approach" [Or: could your model selection tell you that Kepler is better than Ptolemy? Technical report, economics dept., Virginia Tech, 2006. PDF]
- "Where do statistical models come from? Revisiting the problem of specification", math.ST/0610849
UCB Statistics Technical Reports
Sara van de Geer, Empirical Process Theory in M-Estimation [Finding non-asymptotic rates of convergence for common estimators]
Quang H. Vuong, "Likelihood Ratio Tests for Model Selection and Non-Nested Hypotheses", Econometrica 57 (1989): 307--333
Grace Wahba, Spline Models for Observational Data
Michael E. Wall, Andreas Rechtsteiner and Luis M. Rocha, "Singular Value Decomposition and Principal Component Analysis," physics/0208101
Achilleas Zapranis and Apostolos-Paul Refenes, Principles of Neural Model Identification, Selection and Adequacy, with Applications to Financial Econometrics

Hirotugu Akaike, Selected Papers
Ole E. Barndorff-Nielsen and David R. Cox, Inference and Asymptotics
Vic Barnett, Comparative Statistical Inference
Bucklew, Large Deviation Techniques in Decision, Simulation, and Estimation
Michael R. Chernick, Bootstrap Methods: A Practitioner's Guide
Steve Fienberg, The Analysis of Cross-Classified Categorical Data
Peter J. Huber, Robust Statistics
Erich L. Lehmann, Elements of Large-Sample Theory
National Institute of Standards and Technology, Engineering Statistics Handbook [All the sections I've looked at have been quite good.]
Yudi Pawitan, In All Likelihood: Statistical Modeling and Inference Using Likelihood
Aris Spanos
- Statistical Foundations of Econometric Modeling
- Probability Theory and Statistical Interference: Econometric Modeling with Observational Data
Weisberg, Applied Linear Regression

Carolina Armenteros, "From Human Nature to Normal Humanity: Joseph de Maistre, Rousseau, and the Origins of Moral Statistics", Journal of the History of Ideas 68 (2007): 107--130 [Abstract, text links]
Ian Hacking, "The Theory of Probable Inference: Neyman, Peirce and Braithwaite," in Science, Belief and Behavior: Essays in Honor of R. B. Braithwaite ed. D. H. Mellor
Anders Hald, A History of Parametric Statistical Inference from Bernoulli to Fisher, 1713--1935 [Blurb]
Kendall and Plackett (eds.), Studies in the History of Statistics and Probability
Hugo A. Keuzenkamp, Probability, Econometrics and Truth: The Methodology of Econometrics [blurb]
Kyburg, Uncertain Inference
Mayo and Hollander (eds.), Accpetable Evidence: Science and Values in Risk Management
Leland Gerson Neuberg, Conceptual Anomalies in Economics and Statistics: Lessons from the Social Experiment [blurb]
Theodore Porter, Trust in Numbers
Stephen M. Stigler
- The History of Statistics: The Measure of Uncertainty before 1900
- Statistics on the Table: The History of Statistical Concepts and Methods
S. L. Zabell, Symmetry and Its Discontents: Essays on the History of Inductive Probability [blurb]

Felix Abramovich, Yoav Benjamini, David L. Donoho and Iain M. Johnstone, "Adapting to Unknown Sparsity by controlling the False Discovery Rate", math.ST/0505374 [I don't really care about sparsity, but they promise novel relations between the FDR control and asymptotic minimaxity and complexity-penalized model selection.]
Sophie Achard, "A quadratic measure dependence", math.ST/0609259 ["Asymptotic properties of a dimension-robust dependence measure are investigated. It is related to those used in independence tests, but is derivable, thus suitable for independent component analysis. An adjustable kernel allows to accelerate the convergence of the estimator without affecting the bias."]
Shotaro Akaho, "A kernel method for canonical correlation analysis", cs.LG/0609071
Ole E. Barndorff-Nielsen and David R. Cox, "Prediction and Asymptotics", Bernoulli 2 (1996): 319--340
Ole E. Barndorff-Nielsen, David R. Cox and Claudia Klüppelberg (eds.), Complex Stochastic Systems
Roger Barlow, "Asymmetric Errors", physics/0401042
Alain Berlinet, Gérard Biau and Laurent Rouvière, "Optimal L1 Bandwidth selection for variable kernel density estimates", Statistics and Probability Letters 74 (2005): 116--128 ["[O]ne can improve performance of kernel density estimates by varying the bandwidth with the location and/or the sample data at hand. Our interest in this paper is in the data-based selection of a variable bandwidth... an automatic selection procedure inspired by the combinatorial tools developed in Devroye and Lugosi... the expected L1 error of the corresponding selected estimate is up to a given constant multiple of the best possible error plus an additive term which tends to zero under mild assumptions"]
Patrice Bertail, Paul Doukhan and Philippe Soulier (eds.), Dependence in Probability and Statistics ["recent developments in ... probability and statistics for dependent data... from Markov chain theory and weak dependence with an emphasis on ... dynamical systems, to strong dependence in times series and random fields. ... section on statistical estimation problems and specific applications". Full blurb, contents]
Rabi Bhattacharya and Vic Patrangenaru, "Large sample theory of intrinsic and extrinsic sample means on manifolds--II", math.ST/0507423 = Annals of Statistics 33 (2005): 1225--1259 [I need a notebook on statistics on manifolds, as opposed to statistical manifolds]
G. Biau and L. Gyorfi, "On the Asymptotic Properties of a Nonparametric $-Test Statistic of Homogeneity", IEEE Transactions on Information Theory 51 (2005): 3965--3973
David R. Bickel, "Selecting an optimal rejection region for multiple testing: A decision-theoretic alternative to FDR control, with an application to microarrays," math.PR/0212028
David R. Bickel and Rudolf Fruehwirth, "On a Fast, Robust Estimator of the Mode: Comparisons to Other Robust Estimators with Applications", math.ST/0505419
Peter J. Bickel, C. A. J. Klaassen, Y. Ritov and J. A. Wellner, Efficient and Adaptive Estimation for Semiparametric Models
Peter J. Bickel and Y. Ritov, "Non-Parametric Estimators Which Can Be `Plugged-In' " UCB Stat. Tech. Rep. 602 [abstract, pdf]
L. Birge, "A New Lower Bound for Multiple Hypothesis Testing", IEEE Transactions on Information Theory 51 (2005): 1611--1615
Yu. I. Bogdanov, "Statistical Inverse Problem," physics/0211109 [A new density estimator]
Borowiak, Model Discrimination for Nonlinear Regression Models
Thomas Brambor, William Roberts Clark and Matt Golder, "Understanding Interaction Models: Improving Empirical Analyses", Political Analysis 14 (2006): 63--82
A. R. Brazzale, A. C. Davison and N. Reid, Applied Asymptotics: Case Studies in Small-Sample Statistics [blurb]
Lawrence D. Brown and Mark G. Low, "Asymptotic Equivalence of Nonparametric Regression and White Noise", Annals of Statistics 24 (1996): 2384--2398 [JSTOR]
Dizza Bursztyn and david M. Steinberg, "Comparison of designs for computer experiments", Journal of Statistical Planning and Inference 136 (2006): 1103--1119
T. Tony Cai and Mark G. Low, "An adaptation theory for nonparametric confidence intervals", Annals of Statistics 32 (2004): 1805--1840 = math.ST/0503662
Emmanuel Candes and Terence Tao, "Near Optimal Signal Recovery from Random Prjoections and Universal Encoding Strategies", math.CA/0410542
Herve Cardot, Andre Mas and Pascal Sarda, "CLT in Functional Linear Regression Models", math.ST/0508073
Djalil Chafai and Didier Concordet, "On the strong consistency of approximated M-estimators", math.ST/0507102 [Sounds cool...]
In Hong Chang and Rahul Mukerjee, "Asymptotic results on the frequentist mean squared error of generalized Bayes point predictors", Statistics and Probability Letters 67 (2004): 65--71 [Note to self: file this one under "de-Bayesing".]
Sandra Chapman, George Rowlands and Nicholas Watkins
- "Extremum statistics: A framework for data analysis," cond-mat/0106015
- "Extremum Statistics and Signatures of Long Range Correlations," cond-mat/0106015
- "The relationship between extremum statistics and universal fluctuations," cond-mat/0007275
Snigdhansu Chatterjee and Arup Bose, "Generalized bootstrap for estimating equations", math.ST/0504515 = Annals of Statistics 33 (2005): 414--436
Fateh Chebana, "On the optimization of the weighted Bickel-Rosenblatt test", Statistics and Probability Letters 68 (2004): 333--345
Xiaohong Chen, Markus Reiss, "On rate optimality for ill-posed inverse problems in econometrics", arxiv:0709.2003 [Non-parametric instrumental variables]
Cheng Hsiao, Analysis of Panel Data [blurb]
N. N. Chentsov, Statistical Decision Rules and Optimal Inference
H. Chernoff, "A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the Sum of Observations," Annals of Mathematical Statistics 23 (1952): 493--507
Arthur Cohen and Harold B. Sackrowitz, "Decision theory results for one-sided multiple comparison procedures", math.ST/0504505 = Annals of Statistics 33 (2005): 126--144
Jerome Collet, "Estimating copula measure using ranks and subsampling: a simulation study", arxiv:0709.3860
Daniel Commenges, Helene Jacqmin-Gadda, Cecile Proust, and Jeremie Guedj, "A Newton-Like Algorithm for Likelihood Maximization: The Robust-Variance Scoring Algorithm", math.ST/0610402
J. Conrad, O. Botner, A. Hallgren and Carlos P. de los Heros, "Including Systematic Uncertainties in Confidence Interval Construction for Poisson Statistics," hep-ex/0202013
Alexandre d'Aspremont, Onureena Banerjee, Laurent El Ghaoui, "First-order methods for sparse covariance selection", math.OC/0609812 ["Given a sample covariance matrix, we solve a maximum likelihood problem penalized by the number of nonzero coefficients in the inverse covariance matrix. Our objective is to find a sparse representation of the sample data and to highlight conditional independence relationships between the sample variables."]
Herold Dehling (ed.), Empirical Process Techniques for Dependent Data
Phoebus J. Dhrymes, Topics in Advanced Econometrics, vol. II: Linear and Nonlinear Simultaneous Equations
David L. Donoho and Richard C. Liu, "The ``Automatic'' Robustness of Minimum Distance Functionals", Annals of Statistics 16 (1988): 552--586
Mathias Drton and Seth Sullivant, "Algebraic statistical models", math.ST/0703609
Sam Efromovich, "Distribution estimation for biased data", Journal of Statistical Planning and Inference 124 (2004): 1--43
Bradley Efron and Robert Tibshirani, "Using Specially Designed Exponential Families for Density Estimation", Annals of Statistics 24 (1996): 2431--2461 [JSTOR]
Jianqing Fan and Jian Zhang, "Sieve empirical likelihood ratio tests for nonparametric functions", Annals of Statistics 32 (2004): 1858--1907 = math.ST/0503667
Julian J. Faraway, Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models
Valerii V. Fedorov and Peter Hackl, Model-Oriented Design of Experiments
Thomas S. Ferguson, A Course in Large Sample Theory
Jean-David Fermanian and Bernard Salanié "A Nonparametric Simulated Maximum Likelihood Estimation Method", Econometric Theory 20 (2004): 701--734
Ana K. Fermin and Carenne Ludena, "A Statistical view of Iterative Methods for Linear Inverse Problems", math.ST/0504064
S. E. Fienberg, P. Hersh, A. Rinaldo and Y. Zhou, "Maximum Likelihood Estimation in Latent Class Models For Contingency Table Data", arxiv:0709.3535
Magalie Fromont and Béatrice Laurent, "Adaptive goodness-of-fit tests in a density model", Annals of Statistics 34 (2006): 680--720 = math.ST/0607013
Stephane Gaiffas, "Rates of convergence for pointwise curve estimation with a degenerate design", math.ST/0410354
Seymour Geisser, Predictive Inference
Andrew Gelman and Jennifer Hill, Data Analysis Using Regression and Multilevel/Hierarchical Models
Andrew Gelman, Jennifer Hill and Masanao Yajima, "Why we (usually) don't have to worry about mutliple comparisons" [PDF preprint]
Christopher R. Genovese and Larry Wasserman
- "Confidence sets for nonparametric wavelet regression", math.ST/0505632 = Annals of Statistics 33 (2005): 698--729
- "Adaptive Confidence Bands", math.ST/0701513
Josep Ginebra, "On the Measure of the Information in a Statistical Experiment", Bayesian Analysis (2007): 167--212
Jose M. Gonzalez-Barrios and Silvia Ruiz-Velasco, "Regression analysis and dependence", Metrica 61 (2005): 73--87
Grassberger and Nadal (eds.), From Statistical Physics to Statistical Inference and Back
Ulf Grenander, Abstract Inference
Emmanuel Guerre and Pascal Lavergne, "Data-driven rate-optimal specification testing in regression models", math.ST/0505640 = Annals of Statistics 33 (2005): 840--870
Laszlo Gyorfi et al., A Distribution-Free Theory of Nonparametric Regression
Wolfgang Hárdle, Applied Nonparametric Regression [blurb]
David A. Hensher et al. Applied Choice Analysis: A Primer ["Application of quantitative statistical methods to study choices made by individuals. This primer provides an introduction to the main techniques of choice analysis and also includes details on data collection and preparation, model estimation and interpretation and the design of choice experiments". Full blurb]
Norbert Henze and Simos G. Meintanis, "Recent and classical tests for exponentiality: a partial review with comparisons", Metrica 61 (2005): 29--45
Hettmansperger and McKean, Robust Nonparametric Statistical Methods
Peter D. Hoff, "Marginal set likelihood for semiparametric copula estimation", math.ST/0610413
Serkan Hosten, Amit Khetan and Bernd Sturmfels, "Solving the Likelihood Equations", math.ST/0408270
Patrik O. Hoyer, "Non-negative sparse coding," cs.NE/0202009
Tien-Chung Hu, Manuel Ordónez Cabrera and Andrei Volodin, "Asymptotic probability for the bootstrapped means deviations from the sample mean", Statistics and Probability Letters 74 (2005): 178--186 [The probability is supposedly obtained "without imposing any assumptions on joint distribution of the original sequence of random variables from which the bootstrap sample is withdrawn"]
Hua Liang, "Comparison of curves based on a Cramér-von Mises statistic", Computational Statistics and Data Analysis 45 (2004): 805--812
Stefano M. Iacus and Davide La Torre
- "Approximating Distribution Functions by Iterated Function Systems," math.PR/0111152
- "Nonparametric estimation of distribution and density functions in presence of missing data: an IFS approach," math.PR/0302016
Sameer M. Jalnapurkar, "Learning a regression function via Tikhonov regularization", math.ST/0509420
Ian H. Jermyn, "Invariant Bayesian estimation on manifolds", math.ST/0506296 = Annals of Statistics 33 (2005): 583--605
D. F. Kerridge, "Inaccuracy and Inference", Journal of the Royal Statistical Society B 23 (1961): 184--194
K. Kinoshita, "An unbinned goodness-of-fit test based on the random walk", physics/0312014
B. Knuteson, H. Miettinen and L. Holmstrom, "alphaPDE: A New Multivariate Technique for Parameter Estimation," physics/0108002
Roger Koenker, Quantile Regression [blurb]
Mikhail Kovtun, Igor Akushevich, Kenneth G. Manton and H. Dennis Tolley
- "Linear Latent Structure Analysis: Mixture Distribution Models with Linear Constraints", math.PR/0507025
- "A New Efficient Algorithm for Construction of LLS Models", math.PR/0507021
Nicole Kraemer, Anne-Laure Boulesteix, Gerhard Tutz, "Penalized Partial Least Squares Based on B-Splines Transformations", math.ST/0608576
Solomon Kullback, "Probability densities with given marginals," Annals of Mathematical Statistics 39 (1968): 1236--1243
Masayuki Kumon and Akimichi Takemura, "On a simple strategy weakly forcing the strong law of large numbers in the bounded forecasting game", math.PR/0508190 ["In the framework of the game-theoretic probability of Shafer and Vovk (2001) it is of basic importance to construct an explicit strategy weakly forcing the strong law of large numbers (SLLN) in the bounded forecasting game. We present a simple finite-memory strategy based on the past average of Reality's moves, which weakly forces the strong law of large numbers with the convergence rate of $O(\sqrt{\log n/n})$.... We show that if Reality violates SLLN, then the exponential growth rate of Skeptic's capital process is explicitly described in terms of the Kullback divergence between the average of Reality's moves when she violates SLLN and the average when she observes SLLN."]
Jon Lafferty and Larry Wasserman, "Rodeo: Sparse Nonparametric Regression in High Dimensions", math.ST/0506342 ["We present a method for simultaneously performing bandwidth selection and variable selection in nonparametric regression."]
Mikhail Langovoy, "Data-driven goodness-of-fit tests", arxiv:0708.0169
E. L. Lehmann and Joseph P. Romano, "Generalizations of the Familywise Error Rate", math.ST/0507420 = Annals of Statistics 33 (2005): 1138--1154
Feng Liang, Sayan Mukherjee, Mike West, "The Use of Unlabeled Data in Predictive Modeling", arxiv:0710.4618 = Statistical Science 22 (2007): 189--205
Oliver Linton and Zhijie Xiao, "A Nonparametric Regression Estimator That Adapts To Error Distribution of Unknown Form", Econometric Theory 23 (2007): 371--413
Richard Lockhart and Federico O'Reilly, "A note on Moore's conjecture", Statistics and Probability Letters 74 (2005): 212--220 ["We establish the conjecture of Moore ... that the usual plug-in estimate of a distribution function and the Rao-Blackwell estimate of the distribution function are asymptotically equivalent for a wide class of exponential family distributions."]
victor S. L'vov, Anna Pomyalov and Itamar Procaccia, "Outliers, Extreme Events and Multiscaling," nlin.CD/0009049
Christian K. Machens, "Adaptive sampling by information maximization," physics/0112070
Charles F. Manski, Identification for Prediction and Decision [Blurb]
Laszlo Matyas, Generalized Method of Moments Estimation
Robert Mariano, Til Schuermann and Melyvn J. Weeks (eds.), Simulation-Based Inference in Econometrics: Methods and Applications
McCabe and Tremayne, Modern Asymptotic Theory
Nicolai Meinshausen and John Rice, "Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses", math.ST/0501289
Kevin Murphy and Brett Myors, Statistical Power Analysis: A Simple and General Model for Traditional and Modern Hypothesis Tests
National Research Council
- The Future of Statistical Software
- Massive Data Sets
Richard Nickl, "Donsker-type theorems for nonparametric maximum likelihood estimators", Probability Theory and Related Fields 138 (2007): 411--449
Michael Nussbaum, "Asymptotic Equivalence of Density Estimation and Gaussian White Noise", Annals of Statistics 24 (1996): 2399--2430 [JSTOR]
Adrian Pagan and Aman Ullah, Nonparametric Econometrics
Bruno Pelletier, "Kernel density estimation on Riemannian manifolds", Statistics and Probability Letters 73 (2005): 297--304
Donald A. Pierce and Dawn Peters, "Improving on exact tests by approximate conditioning", Biometrika 86 (1999): 265--277 [Via Andrew Gelman's blog; PDF reprint]
Ramani S. Pilla, Catherine Loader and Cyrus Taylor, "A New Technique for Finding Needles in Haystacks: A Geometric Approach to Distinguishing Between a New Source and Random Fluctuations", physics/0505200
Tomaz Podobnik and Tomi Zivko, "On Consistent and Calibrated Inference about the Parameters of Sampling Distributions", physics/0508017
Thorsten Poeschel, Werner Ebeling, and Helge Rose, "Guessing probability distributions from small samples," cond-mat/0203467 = Journal of Statistical Physics 80 (1995): 1443
Thorsten Poeschel and Jan A. Freund, "How to decide whether small samples comply with an equidistribution," cond-mat/0205225
Odile Pons, "Bootstrap of means under stratified sampling", Electronic Journal of Statistics 1 (2007): 381--391 = arxiv:0709.3246
Giovanni Punzi, "Ordering Algorithms and Confidence Intervals in the Presence of Nuisance Parameters", physics/0511202
Sabbir Rahman and Mahbub Majumdar, "Maximum Entropy Multivariate Density Estimation: An exact goodness-of-fit approach", physics/0406023 [Looks like physicists rediscovering penalized maximum likelihood]
Rajendran Raja, "Confidence Limits and their Robustness," physics/0207058
Naren Ramakrishnan and Chris Bailey-Kellogg, "Sampling Strategies for Mining in Data-Scarce Domains," cs.CE/0204047
C. Radhakrishna Rao, "Diversity: Its measurement, decomposition, apportionment and analysis", Sankhya: The Indian Journal of Statistics 44(A) (1982): 1--22 [Sankhya is not in JSTOR! Why is Sankhya not in JSTOR?!?!]
J. C. W. Rayner and D. J. Best, Smooth Tests of Goodness of Fit
R.-D. Reiss and M. Thomas, Statistical Analysis of Extreme Values: With Applications to Insurance, Finance, Hydrology and Other Fields
Peter Riegler and Nestor Caticha, "Maxent Queries and Sequential Sampling," cond-mat/0010104
Sylvain Rubenthaler, Tobias Ryden and Magnus Wiktorsson, "Fast simulated annealing in $\R^d$ and an application to maximum likelihood estimation", math.PR/0609353
Cynthia Rudin, "Stability Analysis for Regularized Least Squares Regression", cs.LG/0502016
Rustem and Howe, Algorithms for Worst-Case Design and Applications to Risk Management
Ines Samengo, "Estimating probabilities from experimental frequencies," cond-mat/0201516
Richard Samworth and Oliver Johnson, "The empirical process in Mallows distance, with application to goodness-of-fit tests", math.ST/0504424
Thomas J. Santner, Brian J. Williams and William J. Note, Design and Analysis of Computer Experiments
George A. F. Seber and C. J. Wild, Nonlinear Regression
Sen, Sequential Nonparametrics
Hichem Snoussi and Ali Mohammad-Djafari, "Penalized maximum likelihood for multivariate Gaussian mixture," physics/0111007
James C. Spall, Introduction to Stochastic Search and Optimization [Book website]
Aris Spanos, "Revisiting the Omitted Variables Argument: Substantive vs. Statistical Adequacy" [PDF preprint]
Peter Stoica, Luzhou Xu and Jian Li, "A new type of parameter estimation algorithm for missing data problems", Statistics and Probability Letters 75 (2005): 219--229
Ne-Zheng Sun, "Structure reduction and robust experimental design for distributed parameter identification", Inverse Problems 21 (2005): 739--758
Thompson and Seber, Adaptive Sampling
F. V. Tkachov, "Quasi-optimal observables: Attaining the quality of maximal likelihood in parameter estimation when only a MC event generator is available," physics/0108030
Aad W. van der Vaart, Jon A. Wellner, "Empirical processes indexed by estimated functions", arxiv:0709.1013 ["We consider the convergence of empirical processes indexed by functions that depend on an estimated parameter $\eta$ and give several alternative conditions under which the ``estimated parameter'' $\eta_n$ can be replaced by its natural limit $\eta_0$ uniformly in some other indexing set $\Theta$"]
Stephen Walker, "New approaches to Bayesian consistency", Annals of Statistics 32 (2004): 2028--2043 = math.ST/0503672
Haonan Wang, J. S. Marron, "Object oriented data analysis: Sets of trees", arxiv:0711.3147 ["Object oriented data analysis is the statistical analysis of populations of complex objects"]
Xiaogang Wang and James V. Zidek, "Selecting likelihood weights by cross-validation", math.ST/0505599 = Annals of Statistics 33 (2005): 463--500 ["The (relevance) weighted likelihood was introduced to formally embrace a variety of statistical procedures that trade bias for precision. Unlike its classical counterpart, the weighted likelihood combines all relevant information while inheriting many of its desirable features including good asymptotic properties. However, in order to be effective, the weights involved in its construction need to be judiciously chosen. Choosing those weights is the subject of this article in which we demonstrate the use of cross-validation. We prove the resulting weighted likelihood estimator (WLE) to be weakly consistent and asymptotically normal. An application to disease mapping data is demonstrated." Sounds interesting...]
Holger Wendland, Scattered Data Approximation
Halbert White
- Asymptotic Theory for Econometricians [Useful source, it seems, for non-IID central limit theorems]
- Estimation, Inference and Specification Analysis [On maximum-likelihood estimation and testing with mis-specified models, i.e., what happens when the actual distribution is nowhere in your class of admissible probability models. I've read about half of this.]
Christopher K. I. Williams, "How to Pretend That Correlated Variables Are Independent by Using Difference Observations", Neural Computation 17 (2005): 1--6 ["In many areas of data modeling, observations at different locations (e.g., time frames or pixel locations) are augmented by differences of nearby observations.... These augmented observations are then often modeled as being independent. How can this make sense? We provide two interpretations, showing (1) that the likelihood of data generated from an autoregressive process can be computed in terms of 'independent' augmented observations and (2) that the augmented observations can be given a coherent treatment in terms of the products of experts model..."]
Simon N. Wood, "Fast stable direct fitting and smoothness selection for Generalized Additive Models", arxiv:0709.3906
Jeffrey M. Wooldridge, Econometric Analysis of Cross Section and Panel Data
Hirokazu Yanagiharaa and Chihiro Ohmoto, "On distribution of AIC in linear regression models", Journal of Statistical Planning and Inference 133 (2005): 417--433