Free energy and entropy were whirling through his brain
A few words about self-organization from the view-point of physics, in the course of which I'll say a few things about the physics of phase transitions and why they are relevant.
Agreement with intuition is nice. Unfortunately, intuition is falible (like everything else) and more importantly not even inter-subjective (so that your intuition and my intuition about what is self-organizing may not agree). Particularly disturbing is the fact that no one, at least in the West, seems to have had any intuitive notion of ``self-organization'' before the 17th century, when Descartes used it (the notion, if not the phrase) in his Monde. This wasn't published until the 18th century, and even in that era many of the most accute intellects show no sign of recognizing self-organization whatsoever. (Hume, in the Dialogues Concerning Natural Religion, comes very close to it, but ultimately dismisses such ideas as absurd. Perhaps his friend Adam Smith's ``invisible hand'' is first truly scientific, as opposed to merely polemical, use of the concept.) It has only held any sort of currency at all in the last two centuries, and the extent even of that may be gauged by its omission from the two-volume OED. Any idea so young is probably still changing a great deal. --- In sum, agreement with our intuition is nice, but this particular intuition is so weak that it's no more than nice.
Traditionally (since Boltzmann) identified by physicists with degree of (lack) of organization, see esp. Schrödinger, What Is Life?.
Are there advantages to using (one of the) Renyi entropies, or the Kullback information-gain?
I thought this way of looking at it was original, but it really comes from one of the early cyberneticists, W. Ross Ashby. The way he put it, in Design for a Brain, every deterministic system will select against some states and for others, thereby organizing itself into a stable configuration. I think this is wrong as he stated it (for instance, what about Hamiltonian systems?) but it has the feel of a useful notion, and some good people, like Stuart Kauffman, think well of it.
Whether entropy is increasing or decreasing depends entirely on the distribution of initial states. If this is wacked, the indicator could give very bad results. (Think about problems with pseudo-random number generators.)
Need an ensemble of systems, but often we want to say that some particular thing --- e.g. the beaker something just crawled out of --- is self-organizing.
Many of the more thoughtful biologists are unhappy with entropy as a measure of disorganization anyhow. (See, in particular, Medawar's ``Herbert Spencer and the Idea of General Evolution'' and Needham's ``Evolution and Thermodynamics,'' dividing through for the latter's Marxism.)
Technical notion: The free energy or thermodynamic potential has a singularity at the critical point.
There is an elaborate physical theory of phase transitions, which needs statistical mechanics in all its awful glory --- energy, Gibbs states, etc.
Technical: The temperature derivative of the free energy is discontinuous at a first-order transition = the phases have different entropies = heat of fusion, heat of vaporization, etc. In second-order transitions the first derivatives are continuous but the second need not be; in particular the second temperature derivative of free energy, the heat capacity, often isn't.
This is self-organization because the organization and loss of symmetry is affected by the dynamics of the system, not some external agency. No demons need apply.
Define an order parameter which is zero in the disordered phase, and not-zero in the ordered phase. Can be scalar (Ising model, binary alloys), vector (ferromagnet), tensor (nematics), complex (superconductivity). Theory proceeds to expand the free energy in (real, scalar) combinations of the order parameter.
If a first-order transition has an order parameter, as e.g. nematic-isotropic, the free energy is usually a cubic in it,
so that the order parameter is discontinuous at the critical point:
Second-order transitions usually have a quadratic and a quartic term but no cubic,
and so are continuous at the critical point,
Physically, correlations should go to zero as separation goes to infinity. In fact we like to assume that correlations are important only up to a characteristic length, the ``correlation length.'' (Or correlation time, of course.)
Increasing correlation lengths are nice indicators self-organization.
where l is the correlation length, a.k.a. correlation radius, and it goes like
Intuitively sensible: disordered phase ``wants'' to establish order, is ``nearly'' ordered. Ordered phase has lots of ``noise'' since it's near the transition. Once in the ordered phase, the correlation length-scale of the order parameter (not fluctuations in it) is very large but not infinite (if only because it's not in an infinite chunk of whatever!).
These serve as nuclei for the formation of the ordered phase. (E.g., ferromagnetism: if by chance a group of atoms align themselves, they'll tend to stay aligned, and align nearby atoms, and so spread. If they're cooled below the critical point before they are obliterated by fluctuations, this will ``lock-in.'')
What happens when growing regions of the ordered phase run into each other? It could be that they are incompatible, and one will grow at the expense of the others, to swallow everything; or they can be compatible, and when they run into each other will stop, upto fluctuations. This in fact happens in crystal growth, and in ferromagnetism; the regions which share a common direction of magnetization are called ``domains''
Aside: you make effective magnets by ensuring that different directions are not degenerate. The easiest way is to stick the ferromagnet in an external magnetic field, which will favor the growth of domains aligned with it. It's nice to know how to make self-organizers organize the way you want.
There are some interesting analogies to all this in economics, especially questions of how cities grow, where industries locate, how technologies are adopted, etc. See W. Brian Arthur, Increasing Returns and Path-Dependence in the Economy.
The idea of small fluctuations near the critical point selecting different ``modes'' has beennamed ``order through fluctuations'' by Ilya Prigogine, N. L.. He treats it with near-mystical reverence. (Rereading some particularly euphoric passages in Order out of Chaos, I think we can drop the ``near.'' When good phyisicists die, they come back as physicists; but the bad ones are re-born as philosophers.)
Also favors bland, simple patterns over more interesting ones. It will, for instance, report that bootstrap is more organized than spiral waves. Bite the bullet and say, ``It's not a perfect indicator, but do you have anything else that works?''; or bite the bullet and say ``Simple patterns really are more organized.'' I think there's a lot to be said for the latter.
What correlation length captures the difference between Kasparov and moving chessmen at random?
Avoids problem with periodicity, at least. (Size-scale = period)
Something like maze patterns or ``spaghetti'' can have several characteristic lengths (width of channels, length of straight section). What happens when these change in opposite directions?
We do have an intuitive sense for these things, but I think we tend to mix up highly organized with interesting organization.
May be too many diffrent sorts of organization for one indicator
Entropy and correlation lengths are (fairly) unambiguous, certainly objective and in many cases not-unreasonable. I don't know of any better ones.
