A couple of posts ago I wrote about an interview with Yann Le Cun. I subsequently found this interesting note in Smith’s Neural Networks for Statistical Modeling:

Backpropagation is an example of multiple invention. David Parker (1982,1985) and Yann LeCun (1986) working independently of each other and of the Rumelhart group, published similar discoveries. But none of these workers made the first discovery of backpropagation. That honor goes, belatedly, to Paul Werbos, whose 1974 Harvard Ph.D. thesis, Beyond Regression, contains the earliest exposition of the techniques involved (Werbos 1974).

Werbos’ 1974 discovery had gone unappreciated, but Rumelhart, Hinton, and Williams’ 1986 discovery did not. It kindled a firestorm of interest in Neural Networks.

I don’t know any details of Le Cun’s discovery in 1986, but I’m curious to look it up.  Note that it was published in the same year as the Rumelhart paper. Here’s the full reference:

Le Cun, Yann. 1986. Learning Processes in a Asymetric Threshold Network. In Disordered Systems and Biological Organization, ed. E. Bienenstock, F. Fogelman Soulie, and G. Weisbuch. Berlin: Springer.