http://bactra.org/notebooks Cosma's Notebooks en http://bactra.org/notebooks/2009/04/10#stochastic-approximation <P>Logically, "stochastic approximation" could refer to a great range of things, but in practice it has become something of a technical term for procedures that approximate the solution of an equation, observed through noise, or which try to minimize a function, again observed through noise. The former --- root-finding --- is sometimes known as the Robbins-Monro problem, and the latter --- minimization --- as the Kiefer-Wolfowitz problem. (Tangentially, that Wolfowitz is the father of the Wolfowitz who helped drag us into the invasion of Iraq.) This turns out to be a subject with deep connections to the theory of on-line learning algorithms and recursive estimation, which is really how I became interested in it, but I also like it nowadays because it provides some very cute, yet powerful, probability examples... <P>See also: <a href="monte-carlo.html">Monte Carlo</a>; <a href="sequential-decisions.html">Sequential Decision-Making Under Uncertainty</a> <ul>Recommended: <li>Michel Bena&iuml;m, "Dynamics of stochastic approximation algorithms", <cite>S&eacute;minaire de probabilit&eacute;s (Strasbourg)</cite> <strong>33</strong> (1999): 1--68 [<a href="http://www.numdam.org/item?id=SPS_1999__33__1_0">Link to full text, bibliography, etc.</a>] <li>M. B. Nevel'son and R. Z. Hasminskii, <cite>Stochastic Approximation and Recursive Estimation</cite> [A truly excellent book; I don't suppose anyone has a copy they'd like to sell?] <li>Robin Pemantle, "A Survey of Random Processes with Reinforcement", <a href="http://arxiv.org/abs/math.PR/0610076">math.PR/0610076</a> <li>H. Robbins and S. Monro, "A Stochastic Approximation Method", <a href="http://projecteuclid.org/euclid.aoms/1177729586"><cite>Annals of Mathematical Statistics</cite> <strong>22</strong> (1951): 400--407</a> </ul> <ul>To read: <li>Arthur E. Albert and Leland A. Gardner, Jr., <cite>Stochastic Approximation and Nonlinear Regression</cite> <li>Vivek S. Borkar, <cite>Stochastic Approxmation: A Dynamical Systems Viewpoint</cite> <li>D. L. Burkholder, "On a Class of Stochastic Approximation Processes", <cite>Annals of Mathematical Statistics</cite> <strong>27</strong> (1956): 1044--1059 <li>Sebastien Gadat and Laurent Younnes, "A Stochastic Algorithm for Feature Selection in Pattern Recognition", <a href="http://jmlr.csail.mit.edu/papers/volume8/gadat07a/gadat07a.pdf"><cite>Journal of Machine Learning Research</cite> <strong>8</strong> (2007): 509--547</a> <li>H. J. Kushner and G. G. Yin, <cite>Stochastic Approximation Algorithms and Applications</cite> <li>Abdelkader Mokkadem and Mariane Pelletier <ul> <li>"Convergence rate and averaging of nonlinear two-time-scale stochastic approximation algorithms", <a href="http://arxiv.org/abs/math.PR/0610329">math.PR/0610329</a> = <cite>Annals of Applied Probability</cite> <strong>16</strong> (2006): 1671--1702 <li>"A companion for the Kiefer-Wolfowitz-Blum stochastic approximation algorithm", <a href="http://arxiv.org/abs/math.ST/0610487">math.ST/0610487</a> [Simultaneously estimating both the location and the magnitude of the maximum] </ul> <li>Abdelkader Mokkadem, Mariane Pelletier, Yousri Slaoui, "Revisiting R&eacute;v&eacute;sz's stochastic approximation method for the estimation of a regression function", <a href="http://arxiv.org/abs/0812.3973">arxiv:0812.3973</a> <li>Mariane Pelletier, "Weak Convergence Rates for Stochastic Approximation with Applications to Multiple Targets and Simulated Annealing", <a href="http://dx.doi.org/10.1214/aoap/1027961032"><cite>Annals of Applied Probability</cite> <strong>8</strong> (1998): 10--44</a> <li>D. Saad (ed.), <cite>Online Learning of Neural Networks</cite> </ul>