The role of successful and unsuccessful trials during motor learning

Reinforcement (‘good dog’, ‘bad dog’) is one of the main strategies to train animals and humans. We have used this strategy extensively in the lab to train songbirds to change their song and study the neural correlates of vocal and motor learning [1]. Reinforcement learning has been successfully modeled by mathematical theories to solve hard computational problems. Although theoretical frameworks relate motor variability and reward to learning outcome, it is not known how the brain uses information from successful and unsuccessful motor trials individually. Do neurobiological learning mechanisms rely more on successful motor trials (for example by memorizing them and learning to repeat them) or do they rely more on unsuccessful motor trials (for example by trying ‘the opposite’ or a new and contrasting approach)?
To answer this question we use electrical stimulation of specific brain areas. By electrically stimulating a specific brain area, it is possible to elicit small changes in the animal’s vocalization (Kao et al. 2005). By aversively reinforcing the stimulated or non-stimulated song (using playback of auditory white noise), we teach birds to avoid or reproduce changes elicited by the stimulation even during non-stimulated trials.

Available Project
While it is straight forward to analyze whether a bird reproduces a specific variation, it is less clear what it means to avoid it. We have open Semester and MSc projects for Computer Scientists, Physicists, or Electrical Engineers for developing song analysis methods to quantify small changes in a bird’s song. Song is a high-dimensional signal, and to quantify the manifold on which song variations reside is a non-trivial problem. Knowing how the brain uses individual trials to improve motor actions would provide many benefits: it could lead to improved training strategies for animals and humans, and it could result in better machine learning algorithms.

[1] Canopoli A, Herbst JA, Hahnloser RHR. A Higher Sensory Brain Region Is Involved in Reversing Reinforcement-Induced Vocal Changes in a Songbird. J Neurosci [Internet]. 2014 May 14 [cited 2014 May 14];34(20):7018–26. Available from: http://www.jneurosci.org/content/34/20/7018.short

Contact

Anja Zai, zaia (at) ini.ethz.ch

© 2017 Institut für Neuroinformatik