Tsallis Statistics, Statistical Mechanics for Non-extensive Systems and Long-Range Interactions

A standard assumption of statistical mechanics is that quantities like energy are "extensive" variables, meaning that the total energy of the system is proportional to the system size; similarly the entropy is also supposed to be extensive. Generally, at least for the energy, this is justified by appealing to the short-range nature of the interactions which hold matter together, form chemical bonds, etc. But suppose one deals with long-range interactions, most prominently gravity; one can then find that energy is not extensive. This makes the life of the statistical mechanic much harder.

Constantino Tsallis is a physicist who came up with a supposed solution, based on the idea of maximum entropy. One popular way to derive the (canonical) equilibrium probability distribution in the following. One purports to know the average values of some quantities, such as the energy of the system, the number of molecules, the volume it occupies, etc. One then searches for the probability distribution which maximizes the entropy, subject to the constraint that it give the right average values for your supposed givens. Through the magic of Lagrange multipliers, the entropy-maximizing distribution can be shown to have the right, exponential, form, and the Lagrange multipliers which go along with your average-value constraints turn out to be the "intensive" variables paired with (or "conjugate to") the extensive ones whose means are constrained (energy <=> temperature, volume <=> pressure, molecular number <=> chemical potential, etc.). But, as I said, the entropy is an extensive quantity. What Tsallis proposed is to replace the usual (Gibbs) entropy with a new, non-extensive quantity, now commonly called the Tsallis entropy, and maximize that, subject to constraints. There is actually a whole infinite family of Tsallis entropies, indexed by a real-valued parameter q, which supposedly quantifies the degree of departure from extensivity (you get the usual entropy back again when q = 1). One can then grind through and show that many of the classical results of statistical mechanics can be translated into the new setting. What has really caused this framework to take off, however, is that while normal entropy-maximization gives you exponential, Boltzmann distributions, Tsallis statistics give you power-law, Pareto distributions, and everyone loves a power-law. (Strictly speaking, Tsallis distributions are type II generalized Pareto distributions, with power-law tails.) Today you'll find physicists applying Tsallis statistics to nearly anything with a heavy right tail.

I have to say I don't buy this at all. Leaving to one side my skepticism about the normal maximum entropy story, at least as it's usually told (e.g. by E. T. Jaynes), there are a number of features which make me deeply suspicious of Tsallis statistics.

It's simply not true that one maximizes the Tsallis entropy subject to constraints on the mean energy . Rather, to get things to work out, you have to fix the value of a "generalized mean" energy, . (This can be interpreted as replacing the usual average, an expectation take with respect to the actual probability distribution, by an expectation taken with respect to a new, "escort" probability distribution.) I have yet to encounter anyone who can explain why such generalized averages should be either physically or probabilistically natural; the usual answer I get is "OK, yes, it's weird, but it works, doesn't it?"
There is no information-theoretic justification for the Tsallis entropy, unlike the usual Gibbs entropy. The Tsallis form is, however, a kind of low-order truncation of the Rényi entropy, which does have information-theoretic interest. (The Tsallis form has been independently rediscovered many times in the literature, going back to the 1960s, usually starting from the Rényi entropy. A brief review of the "labyrinthic history of the entropies" can be found in one of Tsallis's papers, cond-mat/0010150.) Maximizing the Rényi entropy under mean-value constraints leads to different distributions than maximizing the Tsallis entropy.
I have pretty severe doubts about the backing story here, about long-range interactions leading to a non-extensive form for the entropy, particularly when, in derivations which begin with such a story, I often see people blithely factoring the probability that a system is in some global state into the product of the probabilities that its components are in various states, i.e., assuming independent sub-systems.
There are alternative, non-max-ent derivations of the usual statistical-mechanical distributions; such derivations do not seem forthcoming for Tsallis statistic. In particular, large deviations arguments, which essentially show how to get such distributions as emergent, probabilistic consequences of individual-level interactions, do not seem to ever lead to Tsallis statistics, even when one has the kind of long-range interactions which, supposedly, Tsallis statistics ought to handle.
There is no empirical evidence that Tsallis statistics correctly gives the microscopic energy distribution for any known system.
Zanette and Montemurro have shown that you can get any distribution you like out of the Tsallis recipe, simply by changing the function whose generalized average you take as your given. The usual power-law prescription only holds if you constrain either x or , but one of the more "successful" applications requires constraining the generalized mean of , with c and as adjustable constants! (In fairness, I should point out that if you're willing to impose sufficiently weird constraints, you can generate arbitrary distributions from the usual max. ent. procedure, too; this is one of the reasons why I don't put much faith in that procedure.)

I think the extraordinary success of what is, in the end, a slightly dodgy recipe for generating power-laws illustrates some important aspects, indeed unfortunate weaknesses, in the social and intellectual organization of "the sciences of complexity". But that rant will have to wait for my book on The Genealogy of Complexity, which, prudently, means waiting until I'm safely tenured.

I should also discuss the "superstatistics" approach here, which tries to generate non-Boltzmann statistics as mixtures of Boltzmann distributions, physically justified by appealing to fluctuating intensive variables, such as temperature. I will only remark that the superstatistics approach severes all connections between the use of these distributions and non-extensivity and long-range interactions; and that results in the statistical literature on getting generalized Pareto distributions from mixtures of exponentials go back to 1952 at least.

Finally, it has come to my attention that some people are citing this notebook as though it had some claim to authority. Fond though I am of my own opinions, this seems to me to be deeply wrong. The validity of Tsallis statistics, as a scientific theory, ought to be settled in the usual way, by means of the peer-reviewed scientific literature, subject to all its usual conventions and controls. It's obvious from the foregoing that I have pretty strong beliefs in how that debate ought to go, and (this may not be so clear) enough faith in the scientific community that I think, in the long run, it will go that way, but no one should confuse my opinion with a scientific finding. For myself, this page is a way to organize my own thoughts; for everyone else, it's either entertainment, or at best an opinionated collection of pointers to the genuine discussion.

Tsallis & co. maintain a pretty comprehensive and ever-growing bibliography on Tsallis statistics. This includes replies to many of the papers I list here.
Julien Barré, Freddy Bouchet, Thierry Dauxois and Stefano Ruffo, "Large deviation techniques applied to systems with long-range interactions", cond-mat/0406358 = Journal of Statistics Physics 119 (2005): 677--713 [What large deviation results for long-range interactions look like]
A. G. Bashkirov, "Comment on 'Stability of Tsallis entropy and instabilities of Rényi and normalized Tsallis entropies: A basis for q-exponential distributions'," Physical Review E 72 (2005): 028101 [There is also a reply by S. Abe, the author of the original article, which, predictably, I find unconvincing: Physical Review E 72 (2005): 028102.]
Christian Beck, "Superstatistics: Recent developments and applications", cond-mat/0502306
Freddy Bouchet and Thierry Dauxois, "Prediction of anomalous diffusion and algebraic relaxations for long-range interacting systems, using classical statistical mechanics", Physical Review E 72 (2005): 045103 = cond-mat/0407703
Freddy Bouchet, Thierry Dauxois, Stefano Ruffo, "Controversy about the applicability of Tsallis statistics to the HMF model", cond-mat/0605445 = Europhysics News 37 (2006): 9--10
Alice M. Crawford, Nicolas Mordant, Andy M. Reynolds, Eberhard Bodenschatz, "Comment on 'Dynamical Foundations of Nonextensive Statistical Mechanics'", physics/0212080
Thierry Dauxois, "Non-Gaussian distributions under scrutiny", Journal of Statistical Mechanics (2007) N08001
Peter Grassberger, "Temporal scaling at Feigenbaum points and non-extensive thermodynamics", cond-mat/0508110 = Physical Review Letters 95 (2005): 140601 [I can't resist quoting the abstract in full, if only because I enjoy Prof. Grassberger's no-quarter-asked-or-given tone: "We show that recent claims for the non-stationary behaviour of the logistic map at the Feigenbaum point based on non-extensive thermodynamics are either wrong or can be easily deduced from well-known properties of the Feigenbaum attractor. In particular, there is no generalized Pesin identity for this system, the existing 'proofs' being based on misconceptions about basic notions of ergodic theory. In deriving several ew scaling laws of the Feigenbaum attractor, thorough use is made of its detailed structure, but there is no obvious connection to non-extensive thermodynamics." One point made here (but passed over in the abstract) is that there are nearly as many estimates of the "right" value of the non-extensivity parameter q at the period-doubling accumulation point as there are papers on the system. This tends to reduce one's confidence that any of them is a physically meaningful parameter.]
H. J. Hilhorst and G. Schehr, "A note on q-Gaussians and non-Gaussians in statistical mechanics", Journal of Statistical Mechanics (2007) P06003 [Analytical results on the limiting distributions of certain sums of correlated random variables, supposed to follow "q-Gaussians", but not actually doing so. It strikes me as extraordinary that no one in this literature, on either side, pays any attention to actual results in probability theory about generalizations of the central limit theorem; one searches these bibliographies in vain for names like Lévy and Rosenblatt.]
Brian R. La Cour and William C. Schieve, "A Comment on the Tsallis Maximum Entropy Principle", cond-mat/0009216
B. H. Lavenda and J. Dunning-Davies, "Additive Entropies of degree-q and the Tsallis Entropy", physics/0310117
Michael Nauenberg, "Critique of q-entropy for thermal statistics", Physical Review E 67 (2003): 036114 [From the abstract: "[I]t is shown here that the joint entropy for systems having different values of q is not defined in this formalism, and consequently fundamental thermodynamic concepts such as temperature and heat exchange cannot be considered for such systems. Moreover, for q &neq; 1 the probability distribution for weakly interacting systems does not factor into the product of the probability distribution for the separate systems, leading to spurious correlations and other unphysical consequences, e.g., nonextensive energy, that have been ignored in various applications given in the literature." That the probabilities for sub-systems do not factor is, I think, especially devastating, because almost all of the work on the subject assumes that it does. See also comment by Tsallis, cond-mat/305091, and reply by Nauenberg, cond-mat/0305365, which I believe to be correct.]
Damién H. Zanette and Marcelo A. Montemurro
- "A note on non-therrmodynamical applications of non-extensive statistics", cond-mat/0305070 = Physics Letters A 324 (2004): 383--387 [An amusing and quite conclusive assault, culminating in a demonstration that you can use the non-extensive formalism to "derive" any probability distribution whatsoever.]
- "Thermal measurement of stationary nonequilibrium systems: A test for generalized thermostatistics", Physics Letters A 316 (2003): 184--189 = cond-mat/0212327 [And it doesn't even work for for thermodynamic systems.]

CRS, "Maximum Likelihood Estimation for q-Exponential (Tsallis) Distributions", math.ST/0701854 [If you have to use these things, you really should estimate their parameters this way, and not try to fit curves to the sample distribution.]

Andrea Antoniazzi, Francesco Califano, Duccio Fanelli, and Stefano Ruffo, "Exploring the Thermodynamic Limit of Hamiltonian Models: Convergence to the Vlasov Equation", Physical Review Letters 98 (2007): 150602
R. Bachelard, C. Chandre, D. Fanelli, X. Leoncini and S. Ruffo, "Abundance of Regular Orbits and Nonequilibrium Phase Transitions in the Thermodynamic Limit for Long-Range Systems", Physical Review Letters 101 (2008): 260603
Fulvio Baldovin and Enzo Orlandini
- "Hamiltonian Dynamics Reveals the Existence of Quasistationary States for Long-Range Systems in Contact with a Reservoir", Physical Review Letters 96 (2006): 240602 = cond-mat/0603383 ["We introduce a Hamiltonian dynamics for the description of long-range interacting systems in contact with a thermal bath (i.e., in the canonical ensemble). The dynamics confirms statistical mechanics equilibrium predictions for the Hamiltonian mean field model and the equilibrium ensemble equivalence. We find that long-lasting quasistationary states persist in the presence of the interaction with the environment. Our results indicate that quasistationary states are indeed reproducible in real physical experiments."]
- "Quasi-stationary states in long-range interacting systems are incomplete equilibrium states", cond-mat/0603659 = Physical Review Letters 97 (2006): 100601 ["Despite the presence of an anomalous single-particle velocity distribution, we find that ordinary Central Limit Theorem leads to the Boltzmann factor in Gibbs' $\Gamma$-space. We identify the non-equilibrium sub-manifold of $\Gamma$-space responsible for the anomalous behavior and show that by restricting the Boltzmann-Gibbs approach to such sub-manifold we obtain the statistical mechanics of the quasi-stationary states."]
Christian Beck, Ezechiel G. D. Cohen and Harry L. Swinney, "From time series to superstatistics", Physical Review E 72 (2005): 056133
Freddy Bouchet, "Stochastic process of equilibrium fluctuations of a system with long-range interactions", LinkPhysical Review E 70 (2004): 036113
F. Bouchet and J. Barré, "Classification of Phase Transitions and Ensemble Inequivalence, in Systems with Long Range Interactions", Journal of Statistical Physics 118 (2005): 1073--1105
Pierre-Henri Chavanis
- "Dynamics and thermodynamics of systems with long-range interactions: interpretation of the different functionals", arxiv:0904.2729
- "Statistical mechanics of geophysical turbulence: application to jovian flows and Jupiter's great red spot", Physica D 200 (2005): 257--272 [Listed here because this is (judging by the abstract) an instance of Chavanis's more general non-Tsallisite (non-Tsallisian?) approach to statistical mechanics with long-range interactions]
- "Generalized Fokker-Planck equations and effective thermodynamics", cond-mat/0504716 = Physica A 340 (2004): 57
- "Quasi-stationary states and incomplete violent relaxation in systems with long-range interactions", cond-mat/0509726
- "Lynden-Bell and Tsallis distributions for the HMF model", cond-mat/0604234
Pierre-Henri Chavanis, C. Rosier and C. Sire, "Thermodynamics of self-gravitating systems," cond-mat/0107345
Thierry Dauxois, Stefano Ruffo, Ennio Arimondo and Martin Wilkens (eds.), Dynamics and Thermodynamics of Systems With Long Range Interactions [Blurb]
V. Garcia-Morales, J. Pellicer, "Statistical mechanics and thermodynamics of complex systems", math-ph/0304013
Toshiyuki Gotoh, Robert H. Kraichnan, "Turbulence and Tsallis Statistics", nlin.CD/0305040
D. H. E. Gross, "Non-extensive Hamiltonian systems follow Boltzmann's principle not Tsallis statistics," cond-mat/0106496
Rudolf Hanel and Stefan Thurner, "On the Derivation of power-law distributions within standard statistical mechanics", cond-mat/0412016 = Physica A 351 (2005): 260--268
Petr Jizba and Toshihico Arimitsu, "The world according to Rényi: Thermodynamics of multifractal systems," cond-mat/0207707
Ramandeep S. Johal, Antoni Planes, and Eduard Vives, "Equivalence of nonadditive entropies and nonadditive energies in long range interacting systems under macroscopic equilibrium", cond-mat/0503329
T. Kodama, H.-T. Elze, C. E. Aguiar, T. Koide, "Dynamical Correlations as Origin of Nonextensive Entropy", cond-mat/0406732
Hiroko Koyama, Tetsuro Konishi, and Stefano Ruffo, "Clusters die hard: Time-correlated excitation in the Hamiltonian Mean Field model", nlin.CD/0606041 ["The Hamiltonian Mean Field (HMF) model has a low-energy phase where $N$ particles are trapped inside a cluster. ... each particle can be identified as a high-energy particle (HEP) or a low-energy particle (LEP), depending on whether its energy is above or below the separatrix energy. We then define the trapping ratio as the ratio of the number of LEP to the total number of particles and the ``fully-clustered'' and ``excited'' dynamical states as having either no HEP or at least one HEP. We analytically compute the phase-space average of the trapping ratio by using the Boltzmann-Gibbs stable stationary solution of the Vlasov equation associated with the $N \to \infty$ limit of the HMF model. The same quantity, obtained numerically as a time average, is shown to be in very good agreement with the analytical calculation. ... the distribution of the lifetime of the ``fully-clustered'' state obeys a power law. This means that clusters die hard, and that the excitation of a particle from the cluster is not a Poisson process and might be controlled by some type of collective motion with long memory. Such behavior should not be specific of the HMF model and appear also in systems where {\it itinerancy} among different ``quasi-stationary'' states has been observed. ... "]
Bernard H. Lavenda
- "Fundamental inconsistencies of 'superstatistics'", cond-mat/0408485
- "Information and coding discrimination of pseudo-additive entropies (PAE)", cond-mat/0403591
Massimo Marino, "Power-law distributions and equilibrium thermodynamics", cond-mat/0605644 [Makes the interesting claims that if you want a consistent thermodynamics with power-law distributions, then the entropy is uniquely determined to be the Rényi entropy, not the Tsallis entropy]
David Mukamel, "Statistical Mechanics of systems with long range interactions", arxiv:0811.3120
D. Mukamel, S. Ruffo and N. Schreiber, "Breaking of ergodicity and long relaxation times in systems with long-range interactions", cond-mat/0508604
Jan Naudts, "Parameter estimation in nonextensive thermostatistics", cond-mat/0509796
T. Padmanabhan
- "Statistical Mechanics of gravitating systems in static and cosmological backgrounds," astro-ph/0206131
- Structure Formation in the Universe
A. S. Parvana and T.S. Biró, "Extensive Rényi statistics from non-extensive entropy", Physics Letters A 340 (2005): 375--387
Daniel Pfenniger, "Virial statistical description of non-extensive hierarchical systems", cond-mat/0605665
Alessandro Pluchino, Vito Latora and Andrea Rapisarda, "Dynamics and Thermodynamics of a model with long-range interactions", cond-mat/0410213
S. M. Duarte Queirós and C. Tsallis, "Bridging a paradigmatic financial model and nonextensive entropy", Europhysics Letters 69 (2005): 893--899 [Approximation of ARCH model using Tsallis entropies. Thanks to Nick Watkins for bringing this to my attention.]
M. S. Reis, V. S. Amaral, R. S. Sarthour and I. S. Oliveira, "Experimental determination of the non-extensive entropic parameter $q$", cond-mat/0512208
T. M. Rocha Filho, A. Figueiredo, and M. A. Amato, "Entropy of Classical Systems with Long-Range Interactions", Physical Review Letters 95 (2005): 190601 [From the abstract: "We discuss the form of the entropy for classical Hamiltonian systems with long-range interaction using the Vlasov equation which describes the dynamics of a N particle [as N goes to infinity]. ... We show that the stationary states correspond to [extrema] of the Boltzmann-Gibbs entropy, and their stability is obtained from the condition that this extremum is a maximum. As a consequence, the entropy is a function of an infinite set of Lagrange multipliers that depend on the initial condition."]

Notebooks