Probability Theory

09 Nov 2009 16:01

---

One of my advisers in graduate school was a probability theorist, as was his adviser before him; I've not bothered to check, but I wouldn't be astonished if the chain went back to someone like Bernoulli. The fact that the chain could go back that far shows that mathematical probability is an old concept, almost as old as any other part of modern science; on the other hand, my adviser's adviser came just after the generation, between the wars, which made probability a respectable and rigorous branch of mathematics and removed countless obscurities from its applications, and the first serious use of statistical methods in the sciences came only about a hundred years before that. Now of course error analysis is the first thing my students learn when they enter the lab. (Well, almost the first thing, after "if you don't write it down, it never happened" and "Cosma can be bribed with chocolate.") I am conditioned to attack every problem as some kind of stochastic process; but a few generations back nobody had any but the vaguest idea what a stochastic process was.

Pet peeves: Physicists who do not distinguish between a random variable ("X = the roll of a die") and the value it takes ("x=5"). People who report numbers without error-bars or confidence-intervals. Bayesians.

Cf. math in general, stochastic processes, statistics, information theory, statistical mechanics, ergodic theory, machine learning, statistical inference and induction, dynamics; large deviations; empirical process theory

Recommended, big-picture:
• Patrick Billingsley, Probability and Measure
• Harald Cramér, Mathematical Methods of Statistics [Review]
• Feller, An Introduction to Probability Theory and Its Applications, vol. I [I've not finished vol. II yet...]
• Bert Fristedt and Lawrence Gray, A Modern Approach to Probability Theory [Extremely thorough measure-theoretic text; nice treatment of stochastic processes]
• Geoffrey Grimmett and David Stirzaker, Probability and Random Processes [Maybe the best contemporary textbook for those who do not need measure-theoretic probability]
• Ian Hacking
  • The Emergence of Probability [Where that strange two-faced notion came from, and why]
  • The Taming of Chance [Putting chance to work in the 19th century]
• Mark Kac
  • Engimas of Chance
  • Probability and Related Topics in Physical Science
  • Statistical Independence in Probability, Analysis and Number Theory
• Olav Kallenberg, Foundations of Modern Probability [My preferred textbook when teaching stochastic processes]
• Michel Loève, Probability Theory
• David Pollard, A User's Guide to Measure-Theoretic Probability
• R. F. Streater, "Classical and Quantum Probability," math-ph/0002049 ["There are few mathematical topics that are as badly taught to physicists as probability theory."]
• Aram Thomasian, The Structure of Probability Theory

Recommended, close-ups:
• Philippe Barbe, "An Elementary Approach to Extreme Values Theory", arxiv:0811.0753
• Clark Glymour, "Instrumental Probability", Monist 84 (2001): 284--300 [PDF reprint]
• Alexander E. Holroyd and Terry Soo, "A Non-Measurable Set from Coin-Flips", math.PR/0610705 [A cute construction to help students see the point of measure-theoretic probability]
• Jakob Rosenthal, "The Natural-Range Conception of Probability", phil-sci/4978 [Defends the thesis that "the probability of an event is the proportion of initial states that lead to this event in the space of all possible initial states, provided that this proportion is approximately the same in any not too small interval of the initial state space. This idea can also be expressed by saying that in the types of situations that give rise to probabilistic phenomena we may expect to find an initial state space such that any 'reasonable' density function over this space leads to the same probabilities for the possible outcomes."]

To read, historical:
• Lorraine Daston, Classical Probability in the Enlightenment [blurb]
• Gerd Gigerenzer, Zeno Switjtink, Theodore Porter, Lorraine Daston, John Beatty and Lorenz Krüger, The Empire of Chance: How Probability Changed Science and Everyday Life
• Kendall and Plackett (eds.), Studies in the History of Statistics and Probability
• Andrei Kolmogorov, Foundations of Probability Theory
• Glenn Shafer and Vladimir Vovk, "The Sources of Kolmogorov's Grundbegriffe", Statistical Science 21 (2006): 70--98 = math.ST/0606533
• Jan von Plato, Creating Modern Probability

To read, philosophical and foundational:
• Jill North, "Symmetry and Probability", phil-sci/2978
• Michael Strevens, Bigger than Chaos: Understanding Complexity through Probability

To read, pedagogical:
• Blom, Holst and Sandell, Problems and Snapshots from the World of Probability ["It is obvious that the authors have had fun in writing this book..."]
• Morris DeGroot, Probability and Statistics [Sainted founder of the CMU department of statistics, still fondly recalled as "Morrie" by the senior faculty, as in, "What would Morrie do?"]
• F. M. Dekking, C. Kraaikamp, H. P. Lopuhaä and L. E. Meester, A Modern Introduction to Probability and Statistics: Understanding How and Why [Blurb]
• Alvin W. Drake, Fundamentals of Applied Probability Theory [Gets rapturous reviews from students who've used it; this is a nearly infallible sign of quality in a textbook]
• Feller, An Introduction to Probability Theory and Its Applications vol. II
• Allan Gut, Probability: A Graduate Course [From the back: "'I know it's trivial, but I have forgotten why'. This is a slightly exaggerated characterization of the unfortunate attitude of many mathematicians toward the surrounding world. The point of departure of this book is the opposite. This textbook on the theory of probability is aimed at graduate students, with the ideology that rather than being a purely mathematical discipline, probability theory is an intimate companion of statistics." Blurb]
• Emmanuel Lesigne, Heads or Tails: An Introduction to Limit Theorems in Probability [blurb]
• Papoulis, Probability, Random Variables and Stochastic Processes
• Peter Olofsson, Probability, Statistics, and Stochastic Processes
• Sidney Resnick, A Probability Path
• A. Shiryaev, Probability Theory
• Stroock, Probability Theory: An Analytic View
• Paul Vitanyi, "Randomness," math.PR/0110086

To read, technical:
• Sergio Albeverio and Song Liang, "Asymptotic expansions for the Laplace approximations of sums of Banach space-valued random variables", Annals of Probability 33 (2005): 300--336 = math.PR/0503601
• David J. Aldous and Antar Bandyopadhyay, "A survey of max-type recursive distributional equations", math.PR/0401388 = Annals of Applied Probability 15 (2005): 1047--1110
• David Balding, Pablo A. Ferrari, Ricardo Fraiman and Mariela Sued, "Limit theorems for sequences of random trees", math.PR/0406280 [Abstract: " We consider a random tree and introduce a metric in the space of trees to define the "mean tree" as the tree minimizing the average distance to the random tree. When the resulting metric space is compact we show laws of large numbers and central limit theorems for sequence of independent identically distributed random trees. As application we propose tests to check if two samples of random trees have the same law." I wonder if the same technique could be applied to other kinds of random graphs, e.g., random scale-free networks?]
• Patrick Billinglsey, Convergence of Probability Measures
• Sourav Chatterjee, "A new method of normal approximation", arxiv:math/0611213
• Bernard Chazelle, The Discrepency Method: Randomness and Complexity
• Irene Crimaldi and Luca Pratelli, "Two inequalities for conditional expectations and convergence results for filters", Statistics and Probability Letters 74 (2005): 151--162
• Victor De La Pena and Evarist Gine, Decoupling: From Dependence to Independence
• Persi Diaconis and Svante Janson, "Graph limits and exchangeable random graphs", arxiv:0712.2749
• Peter Gács, "Uniform test of algorithmic randomness over a general space", Theoretical Computer Science 341 (2005): 91--137 ["The algorithmic theory of randomness is well developed when the underlying space is the set of finite or infinite sequences and the underlying probability distribution is the uniform distribution or a computable distribution. These restrictions seem artificial. Some progress has been made to extend the theory to arbitrary Bernoulli distributions (by Martin-Lof) and to arbitrary distributions (by Levin). We recall the main ideas and problems of Levin's theory, and report further progress in the same framework...."]
• Janos Galambos and Italo Simonelli, Bonferroni-type Inequalities with Applications
• J. A. Gonzalez, L. I. Reyes, J. J. Suarez, L. E. Guerrero, and G. Gutierrez, "A mechanism for randomness," nlin.CD/0202022 [Color me skeptical, from the abstract]
• Oliver Johnson and Andrew Barron, "Fisher Information inequalities and the Central Limit Theorem," math.PR/0111020
• Oliver Johnson and Richard Samworth, "Central Limit Theorem and convergence to stable laws in Mallows distance", math.PR/0406218
• Mark Kac, Selected Papers
• Olav Kallenberg, Probabilitic Symmetries and Invariance Principles ["This is the first comprehensive treatment of the three basic symmetries of probability theory - contractability, exchangeability, and rotatability - defined as invariance in distribution under contractions, permutations, and rotations." Blurb]
• National Research Council (USA), Probability and Algorithms [online]
• Victor H. de la Pena, Tze Leung Lai and Qi-Man Shan, Self-Normalized Processes: Limit Theory and Statistical Applications [blurb]
• Revesz, The Laws of Large Numbers
• R. Schweizer and A. Sklar, Probabilistic Metric Spaces
• Glenn Shafer and Vladimir Vovk, Probability and Finance: It's Only a Game! [Yet Another Foundation of Probability, this time from game-theory.]
• Christopher G. Small and D. L. McLeish, Hilbert Space Methods in Probability and Statistical Inference
• Jerzy Tyszkiewicz, Arthur Ramer and Achim Hoffmann, "The Temporal Calculus of Conditional Objects and Conditional Events," cs.AI/0110003
• Jerzy Tyszkiewicz, Achim Hoffmann and Arthur Ramer, "Embedding Conditional Event Algebras into Temporal Calculus of Conditionals," cs.AI/0110004