江苏龙网

  1. 首页 > 生活 > >

BP神经网络与自动控制理论中‘反馈’的区别与相同之处是啥

引用一段 Wiki 上关于 Backpropagation 反向传播的历史 \u0026lt;https://en.wikipedia.org/wiki/Backpropagation\u0026gt;：

"The term backpropagation and its general use in neural networks was announced in Rumelhart, Hinton \u0026amp; Williams (1986a), then elaborated and popularized in Rumelhart, Hinton \u0026amp; Williams (1986b), but the technique was independently rediscovered many times, and had many predecessors dating to the 1960s.
The basics of continuous backpropagation were derived in the context of control theory by Henry J. Kelley in 1960, and by Arthur E. Bryson in 1961. They used principles of dynamic programming. In 1962, Stuart Dreyfus published a simpler derivation based only on the chain rule. Bryson and Ho described it as a multi-stage dynamic system optimization method in 1969. Backpropagation was derived by multiple researchers in the early 60\u0026#39;s and implemented to run on computers as early as 1970 by Seppo Linnainmaa. Examples of 1960s researchers include Arthur E. Bryson and Yu-Chi Ho in 1969. Paul Werbos was first in the US to propose that it could be used for neural nets after analyzing it in depth in his 1974 dissertation. While not applied to neural networks, in 1970 Linnainmaa published the general method for automatic differentiation (AD). Although very controversial, some scientists believe this was actually the first step toward developing a back-propagation algorithm. In 1973 Dreyfus adapts parameters of controllers in proportion to error gradients. In 1974 Werbos mentioned the possibility of applying this principle to artificial neural networks, and in 1982 he applied Linnainmaa\u0026#39;s AD method to non-linear functions.
In 1986 Rumelhart, Hinton and Williams showed experimentally that this method can generate useful internal representations of incoming data in hidden layers of neural networks. Yann LeCun, inventor of the Convolutional Neural Network architecture, proposed the modern form of the back-propagation learning algorithm for neural networks in his PhD thesis in 1987. But it is only much later, in 1993, that Wan was able to win an international pattern recognition contest through backpropagation.
During the 2000s it fell out of favour, but returned in the 2010s, benefitting from cheap, powerful GPU-based computing systems. This has been especially so in speech recognition, machine vision, natural language processing, and language structure learning research (in which it has been used to explain a variety of phenomena related to first and second language learning.)."

所引参考文献可见：https://en.wikipedia.org/wiki/Backpropagation

其中提到的 Ho 是指何毓琦先生（https://en.wikipedia.org/wiki/Yu-Chi_Ho；另可参考：中国在自动控制领域让你最引以为豪的成果是什么？）

---

另外推荐（上面也提到的） Werbos（https://en.wikipedia.org/wiki/Paul_Werbos；其个人网页：Welcome to the Werbos World）的一本书：

The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting


推荐阅读


上一篇：NextDay 究竟好在哪里

下一篇：刚入职程序员男友生日，送kindle还是dell显示器