http://bactra.org/notebooks Cosma's Notebooks en http://bactra.org/notebooks/2005/11/09#large-deviations Large Deviations <P>The limit theorems of <a href="probability.html">probability</a> theory --- the weak and strong laws of large numbers, the central limit theorem, etc. --- basically say that averages taken over large samples converge on expectation values. (The strong law of large numbers asserts almost-sure convergence, the central limit theorem asserts a kind of convergence in distribution, etc.) These results say little or nothing about the <em>rate</em> of convergence, however, which is often important for many applications of probability theory, e.g., <a href="stat-mech.html">statistical mechanics</a>. One way to address this is the theory of large deviations. (I believe the terminology goes back to Varadhan in the 1970s, but that's just an impression, rather than research.) <P>Let me say things sloppily first, so the idea comes through, and then more precisely, so people who know the subject won't get too upset. Suppose <em>X</em> is a random variable with expected value <strong>E</strong><em>X</em>, and we consider <img align=absmiddle src="large-deviations_1.gif" alt="$ S_n \equiv \frac{1}{n}\sum_{i=1}^{n}{X_i} $ "> , the sample mean of <em>n</em> samples of <em>X</em>. <img align=absmiddle src="large-deviations_2.gif" alt="$ S_n $ "> "obeys a large deviations principle" if there is a non-negative function <em>r</em>, called the rate function, such that <img align=absmiddle src="large-deviations_3.gif" alt="\[ \mathrm{Pr}\left(\left| \mathbf{E}X - S_n \right| \geq \epsilon\right) \rightarrow e^{-nr(\epsilon)} \] "> . (The rate function has to obey some sensible but technical continuity conditions.) This is a <em>large</em> deviation result, because the difference between the empirical mean and the expectation is remaining constant as <em>n</em> grows --- there has to be a larger and large conspiracy, as it were, among the samples to keep deviating from the expectation in the same way. Now, one reason what I've stated isn't really enough to satisfy a mathematician is that the right-hand side converges on zero, so the functional form of the probability could be anything which also converges on zero and that'd be satisfied, but we want to pick out <em>exponential</em> convergence. The usual way is to look at the limiting growth rate of the probability. Also, we want the probability that the difference between the empirical mean and the expectation falls into any arbitrary set. So one usually sees the LDP asserted in some form like, for any reasonable set <em>A</em>, <img align=absmiddle src="large-deviations_4.gif" alt="\[ \lim_{n\rightarrow\infty}{-\frac{1}{n}\log{\mathrm{Pr}\left(\left| \mathbf{E}X - S_n \right| \in A\right)}} = \inf_{x\in A}{r(x)} \] "> (Actually, to be <em>completely</em> honest, I really shouldn't be assuming that there is a limit to those probabilities. Instead I should connect the lim inf of that expression to the infimum of the rate function over the interior of <em>A</em>, and the lim sup to the rate function over the closure of <em>A</em>.) <P>Similar large deviation principles can be stated for the empirical distribution, the empirical process, functionals of sample paths, etc., rather than just the empirical mean. There are tricks for relating LDPs on higher-level objects, like the empirical distribution over trajectories, to LDPs on lower-level objects, like empirical means. (These go under names like "the contraction principle".) <P>Since <a href="ergodic-theory.html">ergodic theory</a> extends the probabilistic limit laws to <a href="stochastic-processes.html">stochastic processes</a>, rather than just sequences of independent variables, it shouldn;t be surprising that large deviation principles also hold for some stochastic processes. I am particularly interested in LDPs for <a href="markov.html">Markov</a> processes, and their applications. There are also important connections to <a href="information-theory.html">information theory</a>, since in an awful lot of situations, the large deviations rate function is the Kullback-Leibler divergence, a.k.a. the relative entropy. <P>A related, but strictly speaking distinct, topic is the host of inequalities which give bounds on deviations of averages from expectations for various types of processes or sample functionals --- Bernstein, Chernoff, Hoeffding, etc. (For instance, Hoeffding's inequality gives deviation probabilities for averages of strictly bounded functionals.) Inequalities of this type are generally just upper bounds, but ones which apply even at small sample sizes, rather than just asymptotically, whereas the LDP is about asymptotic bounds from both directions. Since they've got very similar uses, however, I include them in here too. <ul>Recommended: <li>James Bucklew, <cite>Large Deviation Techniques in Decision, Simulation, and Estimation</cite> <li>Thomas Cover and Joy Thomas, <cite>Elements of Information Theory</cite> [Very nice chapter on large deviations for IID sequences] <li>Amir Dembo and Ofer Zeitouni, <cite>Large Deviations Techniques and Applications</cite> [Chapters 2, 4 and 5, and parts of chapter 6, are available in postscript format via <a href="http://www-stat.stanford.edu/~amir/">Prof. Dembo's</a> page for his <a href="http://www-stat.stanford.edu/~amir/large-deviations/">course on large deviations</a>] <li>Frank den Hollander, <cite>Large Deviations</cite> [Nice introductory text for people with an applied probability background. Short.] <li>Richard S. Ellis <ul> <li>"The Theory of Large Deviations: from Boltzmann's 1877 Calculation to Equilibrium Macrostates in 2D Turbulence", <cite>Physica D</cite> <strong>133</strong> (1999): 106--136 <li><cite>Entropy, Large Deviations, and Statistical Mechanics</cite> </ul> <li>M. I. Friedlin and A. D. Wentzell, <cite>Random Perturbations of Dynamical Systems</cite> <li>Hugo Touchette, "The Large Deviations Approach to Statistical Mechanics", <a href="http://arxiv.org/abs/0804.0327">arxiv:0804.0327</a> <li>S. R. S. Varadhan, "Large Deviations", <a href="http://dx.doi.org/10.1214/07-AOP348"><cite>Annals of Probability</cite> <strong>36</strong> (2008): 397--419</a> [<a href="http://math.nyu.edu/faculty/varadhan/wald.pdf">Copy via Prof. Varadhan</a>. Wald Lecture for 2005.] </ul> <ul>Recommended, more specialized: <li>R. R. Bahadur, <cite>Some Limit Theorems in Statistics</cite> [1971. The notation is now much more transparent, and the proofs of many basic theorems considerably simplified. But if there's a better source for statistical applications than this little book, I've yet to find it.] <li>Julien Barr&eacute;, Freddy Bouchet, <a href="http://perso.ens-lyon.fr/thierry.dauxois/">Thierry Dauxois</a> and Stefano Ruffo, "Large deviation techniques applied to systems with long-range interactions", <a href="http://arxiv.org/abs/cond-mat/0406358">cond-mat/0406358</a> = <a href="http://dx.doi.org/10.1007/s10955-005-3768-8"><cite>Journal of Statistics Physics</cite> <strong>119</strong> (2005): 677--713</a> <li>Michel Bena&iuml;n and J&ouml;rgen W. Weibull, "Deterministic Approximation of Stochastic Evolution in Games", <cite>Econometrica</cite> <strong>71</strong> (2003): 879--903 [<a href="http://www.jstor.org/pss/1555525">JSTOR</a>] <li>J.-R. Chazottes and D. Gabrielli, "Large deviations for empirical entropies of Gibbsian sources", <a href="http://arxiv.org/abs/math.PR/0406083">math.PR/0406083</a> = <a href="http://dx.doi.org/10.1088/0951-7715/18/6/007"><cite>Nonlinearity</cite> <strong>18</strong> (2005): 2545--2563</a> [This is a very cool result which shows that block entropies, and entropy rates estimated from those blocks, obey the large deviation principle even as one lets the length of the blocks grow with the amount of data, provided the block-length doesn't grow too quickly (only logarithmically). I wish I could write papers like this.] <li>W. De Roeck, Christian Maes and Karel Netocny, "H-Theorems from Autonomous Equations", <a href="http://arxiv.org/abs/cond-mat/0508089">cond-mat/0508089</a> [this basically derives the H-theorem of statistical mechanics as a large deviations result, assuming a certain reasonable Markovian form for the macroscopic dynamics. In fact, we have a separate argument that you <em>don't</em> have that Markovian form, you're just not trying hard enough; see <a href="http://arxiv.org/abs/cond-mat/0303625">here</a>] <li>Paul Dupuis, "Large Deviations Analysis of Some Recursive Algorithms with State-Dependent Noise", <a href="http://projecteuclid.org/euclid.aop/1176991581"><cite>Annals of Probability</cite> <strong>16</strong> (1988): 1509--1536</a> [Open access] <li>Gregory L. Eyink <ul> <li>"Action principle in nonequilbrium statistical dynamics," <citE>Physical Review E</cite> <strong>54</strong> (1996): 3419--3435 [Least action as a consequence of Markovian LDP] <li>"A Variational Formulation of Optimal Nonlinear Estimation," <a href="http://arxiv.org/abs/physics/0011049">physics/0011049</a> [Nice connections between optimal state estimation (assuming a known form for the underlying stochastic process), nonequilibrium statistical mechanics, and large deviations theory, leading to tractable-looking numerical schemes for estimation.] </ul> <li>Leonid (Aryeh) Kontorovich, "Obtaining Measure Concentration from Markov Contraction", <a href="http://arxiv.org/abs/0711.0987">arxiv:0711.0987</a> [I really don't know why Leo bothered to take my <a href="http://www.stat.cmu.edu/~cshalizi/754/">class</a>] <li>S. Orey and S. Peliken, "Large deviations principles for stationary processes", <cite>Annals of Probability</cite> <strong>16</strong> (1988): 1481--1496 </ul> <ul>To read: <li>Paul H. Algoet and Brian H. Marcus, "Large Deviation Theorems for Empirical Types of Markov Chains Constrained to Thin Sets," <cite>IEEE Trans. Info. Theory</cite> <strong>38</strong> (1992): 1276--1291 <li>Alexei Andreanov, Giulio Biroli, Jean-Philippe Bouchaud, and Alexandre Lef&egrave;vre, "Field theories and exact stochastic equations for interacting particle systems", <a href="http://dx.doi.org/10.1103/PhysRevE.74.030101"><cite>Physical Review E</cite> <strong>74</strong> (2006): 030101</a> = <a href="http://arxiv.org/abs/cond-mat/0602307">cond-mat/0602307</a> <li>Ellen Baake, Frank den Hollander and Natali Zint, "How T-Cells Use Large Deviations to Recognize Foreign Antigens", <a href="http://arxiv.org/abs/q-bio.SC/0605016">arxiv:q-bio.SC/0605016</a> <li>Matthias Birkner, Andreas Greven and Frank den Hollander, "Quenched large deviation principle for words in a letter sequence", <a href="http://arxiv.org/abs/0807.2611">arxiv:0807.2611</a> <li>Igor Bjelakovic, Jean-Dominique Deuschel, Tyll Krueger, Ruedi Seiler, Rainer Siegmund-Schultze and Arleta Szkola <ul> <li>"A quantum version of Sanov's theorem", <a href="http://arxiv.org/abs/quant-ph/0412157">quant-ph/0412157</a> = <a href="http://dx.doi.org/10.1007/s00220-005-1426-2"><cite>Communications in Mathematical Physics</cite> <strong>260</strong> (2005): 659--671</a> [Quantum large deviations!] <li>"Typical support and Sanov large deviations of correlated states", <a href="http://arxiv.org/abs/math.PR/0703772">math.PR/0703772</a> = <a href="http://dx.doi.org/10.1007/s00220-008-0440-6"><cite>Communications in Mathematical Physics</cite> <strong>279</strong> (2008): 559--584</a> </ul> <li>Amarjit Budhiraja, Paul Dupuis, Vasileios Maroulas, "Large deviations for infinite dimensional stochastic dynamical systems", <cite>Annals of Applied Probability</cite> <strong>36</strong> (2008): 1390--1420 = <a href="http://arxiv.org/abs/0808.3631">arxiv:0808.3631</a> <li>Patrick Cattiaux and Arnaud Guillin, "Deviation bounds for additive functionals of Markov process", <a href="http://arxiv.org/abs/math.PR/0603021">math.PR/0603021</a> [Non-asymptotic bounds for the probability that time averages deviate from expectations with respect to the invariant measure, when the process is stationary and ergodic and the invariant measure is reasonably regular.] <li>J.-R. Chazottes, P. Collet, C. Kuelske, and F. Redig, "Concentration inequalities for random fields via coupling", <a href="http://arxiv.org/abs/math.PR/0503483">math.PR/0503483</a> <li>Po-Ning Chen, "Generalization of Gartner-Ellis theorem", <a href="http://dx.doi.org/10.1109/18.887893"><cite>IEEE Transactions on Information Theory</cite> <strong>46</strong> (2000): 2752--2760</a> <li>Zhiyi Chi <ul> <li>"Large deviations for template matching between point processes", <a href="http://dx.doi.org/10%2E1214/105051604000000576"><cite>Annals of Applied Probability</cite> <strong>15</strong> (2005): 153--174</a> = <a href="http://arxiv.org/abs/math.PR/0503463">math.PR/0503463</a> <li>"On the asymptotic of likelihood ratios for self-normalized large deviations", <a href="http://arxiv.org/abs/0709.1506">arxiv:0709.1506</a> </ul> <li>Igor Chueshov and Annie Millet, "Stochastic 2D hydrodynamical type systems: Well posedness and large deviations", <a href="http://arxiv.org/abs/0807.1810">arxiv:0807.1810</a> <li>A. de Acosta, "A general nonconvex large deviation result II", <citE>Annals of Probability</cite> <strong>32</strong> (2004): 1873--1901 = <a href="http://arxiv.org/abs/math.PR/0410101">math.PR/0410101</a> <li>Zach Deitz and Sunder Sethuraman, "Large deviations for a class of nonhomgeneous Markov chains", <a href="http://arxiv.org/abs/math.PR/0404230">math.PR/0404230</a> <li>B. Derrida, "Non equilibrium steady states: fluctuations and large deviations of the density and of the current", <a href="http://arxiv.org/abs/cond-mat/0703762">cond-mat/0703762</a> <li>B. Derrida, Joel L. Lebowitz and Eugene R. Speer, "Exact Large Deviation Functional for the Density Profile in a Stationary Nonequilibrium Open System," <a href="http://arxiv.org/abs/cond-mat/0105110">cond-mat/0105110</a> <LI>Paul Dupuis and Richard S. Ellis, <cite>A Weak Convergence Approach to the Theory of Large Deviations</cite> [<a href="http://www.math.umass.edu/%7Ersellis/pdf-files/dupuis-book.pdf">PDF</a>] <li>Vlad Elgart and Alex Kamenev, "Rare Events Statistics in Reaction--Diffusion Systems", <a href="http://arxiv.org/abs/cond-mat/0404241">cond-mat/0404241</a> [i.e., large deviations] <li>Andreas Engel, Remi Monasson and Alexander K. Hartmann, "On Large Deviation Properties of Erdos-Renyi Random Graphs", <a href="http://dx.doi.org/10.1007/s10955-004-2268-6"><cite>Journal of Statistical Physics</cite> <strong>117</strong> (2004): 387--426</a> <li>Jin Feng and Thomas G. Kurtz, <cite>Large Deviations for Stochastic Processes</cite> [<a href="http://www.math.wisc.edu/~kurtz/feng/ldp.htm">Online</a>] <li>Cristian Giardina', Jorge Kurchan, Luca Peliti, "Direct evaluation of large-deviation functions", <a href="http://arxiv.org/abs/cond-mat/0511248">cond-mat/0511248</a> ["We introduce a numerical procedure to evaluate directly the probabilities of large deviations of physical quantities, such as current or density, that are local in time. The large-deviation functions are given in terms of the typical properties of a modified dynamics, and since they no longer involve rare events, can be evaluated efficiently and over a wider ranges of values."] <li>Nathael Gozlan and Christian L&eacute;onard, "A large deviation approach to some transportation cost inequalities", <a href="http://arxiv.org/abs/math.PR/0510601">math.PR/0510601</a> <li>Alice Guionnet, "Large deviations and stochastic calculus for large random matrices", <a href="http://dx.doi.org/10.1214/154957804100000033"><cite>Probability Surveys</cite> <strong>1</strong> (2004): 72--172</a> [Open access] <li>O. V. Gulinskii and <a href="http://www.eng.tau.ac.il/~liptser/">R. S. Liptser</a>, "Example of Large Deviations for Stationary Processes", <cite>Theory of Probability and Applications</cite> <strong>44</strong> (1999): 211--225 [<a href="http://www.eng.tau.ac.il/~liptser/papers/helix.pdf">PDF</a>] <li>Marian Grendar Jr. and Marian Grendar, "Chernoff's bound forms," <a href="http://arxiv.org/abs/math.PR/0306326">math.PR/0306326</a> <li>Te Sun Han <ul> <li>"Hypothesis Testing with the General Source", <a href="http://dx.doi.org/10.1109/18.887854"><cite>IEEE Transactions on Information Theory</cite> <strong>46</strong> (2000): 2415--2427</a> = <a href="http://arxiv.org/abs/math.PR/0004121">math.PR/0004121</a> ["The asymptotically optimal hypothesis testing problem with the general sources as the null and alternative hypotheses is studied.... Our fundamental philosophy in doing so is first to convert all of the hypothesis testing problems completely to the pertinent computation problems in the large deviation-probability theory. ... [This] enables us to establish quite compact general formulas of the optimal exponents of the second kind of error and correct testing probabbilities for the general sources including all nonstationary and/or nonergodic sources with arbitrary abstract alphabet (countable or uncountable). Such general formulas are presented from the information-spectrum point of view."] <li>"An information-spectrum approach to large deviation theorems", <a href="http://arxiv.org/abs/cs.IT/0606104">cs.IT/0606104</a> </ul> <li>Svante Janson, "Large deviations for sums of partly dependent random variables", <a href="http://dx.doi.org/10.1002/rsa.20008"><cite>Random Structures and Algorithms</cite> <strong>24</strong> (2004): 234--248</a> ["We use and extend a method by Hoeffding to obtain strong large deviation bounds for sums of dependent random variables with suitable dependency structure. The method is based on breaking up the sum into sums of independent variables. Applications are given to U-statistics, random strings and random graphs." Applied here only to Erdos-Renyi (IID) random graphs, but might be extendable to Markov random graphs...? <a href="http://www.math.uu.se/~svante/papers/sj150.pdf">PDF preprint</a>] <li>Vladislav Kargin, "A Large Deviation Inequality for Vector Functions on Finite Reversible Markov Chains", <a href="http://arxiv.org/abs/math.PR/0508538">math.PR/0508538</a> <li>Michael Keyl, "Quantum state estimation and large deviations", <a href="http://arxiv.org/abs/quant-ph/0412053">quant-ph/0412053</a> <li>F. Klebaner and <a href="http://www.eng.tau.ac.il/~liptser/">R. Liptser</a>, "Large Deviations for Past-Dependent Recursions", <a href="http://arxiv.org/abs/math.PR/0603407">math.PR/0603407</a> [Corrected version of <cite>Problems of Information Transmission</cite> <strong>32</strong> (1996): 23--34] <li>Leonid Kontorovich <ul> <li>"Measure Concentration of Hidden Markov Processes", <a href="http://arxiv.org/abs/math.PR/0608064">math.PR/0608064</a> <li>"Measure concentration of Markov Tree Processes", <a href="http://arxiv.org/abs/math.PR/0608511">math.PR/0608511</a> </ul> <li>Ioannis Kontoyiannis and S. P. Meyn <ul> <li>"Large deviations asymptotics and the spectral theory of multiplicatively regular Markov processes", <a href="http://arxiv.org/abs/math.PR/0509310">math.PR/0509310</a> = <cite>Electronic Journal of Probability</cite> <strong>10</strong> (2005): 61--123 <li>"Spectral Theory and Limit Theorems for Geometrically Ergodic Markov Processes", <a href="http://arxiv.org/abs/math.PR/0209200">math.PR/0209200</a> = <a href="http://dx.doi.org/10.1214/aoap/1042765670"><cite>Annals of Applied Probability</cite> <strong>13</strong> (2003): 304--362</a> </ul> <li>D. Lacoste, A. W. C. Lau and K. Mallick, "Fluctuation theorem and large deviation function for a solvable model of a molecular motor", <a href="http://dx.doi.org/10.1103/PhysRevE.78.011915"><cite>Physical Review E</cite> <strong>78</strong> (2008): 011915</a> <li>Vivien Lecomte, C&eacute;cile Appert-Rolland, and Fr&eacute;d&eacute;ric van Wijland <ul> <li>"Thermodynamic formalism for systems with Markov dynamics", <a href="http://arxiv.org/abs/cond-mat/0606211">cond-mat/0606211</a> <li>"Thermodynamic formalism and large deviation functions in continuous time Markov dynamics", <a href="http://arxiv.org/abs/cond-mat/0703435">cond-mat/0703435</a> </ul> <li>Vivien Lecomte and Julien Tailleur, "A numerical approach to large deviations in continuous time", <a href="http://dx.doi.org/10.1088/1742-5468/2007/03/P03004">Journal of Statistical Mechanics: Theory and Experiment</cite> <strong>2007</strong>: P03004</a> <li>Carlos A. Leon and Francois Perron, "Optimal Hoeffding bounds for discrete reversible Markov chains", <a href="http://arxiv.org/abs/math.PR/0405296">math.PR/0405296</a> <li><a href="http://www.eng.tau.ac.il/~liptser/">Robert Sh. Liptser</a> and Anatolii A. Pukhalskii, "Limit theorems on large deviations for semimartingales", <a href="http://arxiv.org/abs/math.PR/0510028">math.PR/0510028</a> [But published in a journal in 1992] <li>P. Major <ul> <li>"On a multivariate version of Bernstein's inequality", <a href="http://arxiv.org/abs/math.PR/0411287">math.PR/0411287</a> <li>"A multivariate generalization of Hoeffding's ineqality", <a href="http://arxiv.org/abs/math.PR/0411288">math.PR/0411288</a> </ul> <li>Satya N. Majumdar and Alan J. Bray, "Large-Deviation Functions for Nonlinear Functionals of a Gaussian Stationary Markov Process", <a href="http://arxiv.org/abs/cond-mat/0202138">cond-mat/0202138</a> = <cite>Physical Review E</cite> <strong>65</strong> (2002): 051112 <li>K. Marton, "Bounding <img align=absmiddle src="large-deviations_5.gif" alt="$ \bar{d} $ "> -distance by informational divergence: a method to prove measure concentration", <a href="http://dx.doi.org/10.1214/aop/1039639365"><cite>Annals of Probability</cite> <strong>24</strong> (1996): 857--866</a> [Thanks to Leo Kontorovich for the pointer] <li>David McAllester, "A Statistical Mechanics Approach to Large Devations Theorems" [E-print available via <a href="http://citeseer.ist.psu.edu/443261.html">CiteSeer</a> --- published?] <li>Abdelkader Mokkadem, Mariane Pelletier and Baba Thiam, "Large and moderate deviations principles for kernel estimators of the multivariate regression", <a href="http://arxiv.org/abs/math.ST/0703341">math.ST/0703341</a> <li>K. Netocny and F. Redig, "Large deviations for quantum spin systems", <a href="http://arxiv.org/abs/math-ph/0404018">math-ph/0404018</a> = <cite>Journal of Statistical Physics</cite> <strong>117</strong> (2004): 521--547 <li>Enzo Olivieri and Maria Eulalia Vares, <cite>Large Deviations and Metastability</cite> [<a href="http://www.cambridge.org/uk/catalogue/catalogue.asp?isbn=0521591635">Blurb</a>] <li>Y. Oono, "Large Deviation and Statistical Physics", <cite>Progress in Theoretical Physics</cite> <strong>99</strong> (1989): 165--205 [Should that be "Deviations"?] <li>Huyen Pham, "Some applications and methods of large deviations in finance and insurance",<a href="http://arxiv.org/abs/math.PR/0702473">math.PR/0702473</a> <li>Anatoly Puhalskii, <cite>Large Deviations and Idempotent Probability</cite> <li>Anatolii A. Puhalskii, "Stochastic processes in random graphs", <a href="http://arxiv.org/abs/math.PR/0402183">math.PR/0402183</a> [Large deviations for Erdos-Renyi graphs. Memo to self: how much work would it be to extend this to Markovian graphs?] <li>Hong Qian, "Relative Entropy: Free Energy Associated with Equilibrium Fluctuations and Nonequilibrium Deviations", <a href="http://arxiv.org/abs/math-ph/0007010">math-ph/0007010</a> = <a href="http://dx.doi.org/10%2E1103/PhysRevE%2E63%2E042103"><cite>Physical Review E</cite> <strong>63</strong> (2001): 042103</a> <li>Olivier Rivoire, "The cavity method for large deviations", <a href="http://arxiv.org/abs/cond-mat/0506164">cond-mat/0506164</a> = <a href="http://dx.doi.org/10.1088/1742-5468/2005/07/P07004"><cite>Journal of Statistical Mechanics: Theory and Experiment</cite> (2005): P07004</a> ["A method is introduced for studying large deviations in the context of statistical physics of disordered systems. The approach, based on an extension of the cavity method to atypical realizations of the quenched disorder, allows us to compute exponentially small probabilities (rate functions) over different classes of random graphs."] <li>David Ruelle, <cite>Thermodynamic Formalism</cite> <li>L. Saulis and V. A. Statulevicius, <cite>Limit Theorems for Large Deviations</cite> <li>Adam Shwartz, <cite>Large Deviations in Performance Modeling</cite> <li>Vincent Y. F. Tan, Animashree Anandkumar, Lang Tong and Alan S. Willsky, "A Large-Deviation Analysis of the Maximum-Likelihood Learning of Markov Tree Structures", <a href="http://arxiv.org/abs/0905.0940">arxiv:0905.0940</a> [Large deviations for Chow-Liu trees] <li>Yongqiang Tang, "A Hoeffding-Type Inequality for Ergodic Time Series", <a href="http://dx.doi.org/10.1007/s10959-007-0057-2"><cite>Journal of Theoretical Probability</cite> <strong>20</strong> (2007): 167--176</a> [<a href="http://www4.stat.ncsu.edu/~sghosal/papers/Tang.pdf">PDF preprint</a>] <li>Ted Theodosopoulos, "A Reversion of the Chernoff Bound", <a href="http://arxiv.org/abs/math.PR/0501360">math.PR/0501360</a> </ul> <ul>To write: <li>"Large Deviations in Exponential Families of Stochastic Automata/Hidden Markov Models" </ul>