3,078
edits
Changes
no edit summary
When α is high, the exponential function declines rapidly and puts all of the weight on the most recent experiences of the animal. When α is low, it declines slowly and averages together many observations, which is shown in Fig. 1.
[[File:Bush and Mosteller graph1.gif]]
Fig 1: “Weights determining the effects of previous rewards on current associative strength effectively decline as an exponential function of time”<ref>Schoenbaum G, Roesch MR, Stalnaker TA, Takahashi YK (2009) A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat Rev Neurosci 10:885–892.</ref>.
The Bush and Mosteller equation was critically important, because it was the first use of this kind of iterative error-based rule for reinforcement learning; additionally, it forms the basis of all modern approaches to this problem. This is a fact often obscured by what is known as the Rescorla–Wagner model of classical conditioning<ref>Rescorla RA, Wagner AR (1972) in Classical Conditioning II: Current Research and Theory, A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement, eds Black AH, Prokasy WF (Appleton Century Crofts, New York), pp 64–99.</ref>>. The Rescorla–Wagner model was an important extension of the Bush and Mosteller approach (23, 24) to the study of what happens to associative strength when two cues predict the same event. Their findings were so influential that the basic Bush and Mosteller rule is now often mistakenly attributed to Rescorla and Wagner by neurobiologists.
==References==