Chapter 2: The role of dopamine in the
maintenance of short-term memory in prefrontal cortex neurons: “input-driven”
versus “internally-driven” networks
2.1 Prefrontal
cortex in higher cognitive functions
The linkage
between working memory (WM), cognitive control and Prefrontal cortex (Pfc) has
been one of the most active topics of research in cognitive science (see
Gathercole, 1996; Logie, 1995; Logie & Gilhooly, 1998; Miyake & Shah,
1999; Richardson et al., 1996 for some reference textbook on the topic).
Historically, it was the work of Alan Baddeley (Baddeley & Della Sala,
1996; Baddeley, 1986; Baddeley & Hitch, 1974; Baddeley & Hitch, 1994)
which represented the first theoretical attempt to dissect the architecture of
WM, which resulted in the distinction between the two domain-specific buffer
systems – the phonological loop and the visuospatial scratchpad – whose
activity is coordinated by a central control structure termed the "central
executive". In this model, both storage and control processes are included
under the general process of WM. The basic intuition of Baddeley was then
adopten in a neuropsychological context by Shallice and colleagues. Shallice
has described the Pfc as a Supervisory Attentional System (SAS, Norman &
Shallice, 1986; Shallice, 1982; Shallice, 1988), a concept that is tightly
linked to Baddeley’s "central executive”. Baddeley himself has later
incorporated the SAS theory into his own model (Baddeley, 1986).
As an early
attempt to characterize the structure and functions of WM, Baddeley and
Shallice’s approach suffers from several problems. As a first, more general
remark, the "central executive" and the SAS do not have the necessary
articulation needed for a modern theory, which has to explain mechanistically
how the described properties naturally arise from the biological systems that
constitute their substrate. The homuncular connotation of the central executive
and the SAS are clearly a major obstacle to a full specification of the theory, thereby only postponing the
problem of how to characterize the system. Secondly, Pfc has been found to be
critically involved in short-term active maintenance of information (Fuster,
1989; Goldman-Rakic, 1987), which shows that the distinction of a merely
storage and executive components can be erroneous. Indeed, it has been
suggested that different Pfc subregions (principally ventral) may be involved
directly with active maintenance functions, whereas other Pfc regions
(principally dorsal) could instead be involved with the control aspects of WM
(D'Esposito et al., 1998; Owen, 1997; Smith & Jonides, 1999), although an
opposite account has been proposed by O’Reilly (O’Reilly et al., 2002).
At the time
this work is written, neuropsychological as well as neurophysiological studies
have provided strong evidence in favor of an involvement of Pfc in cognitive
control. In particular, Pfc has been shown to be involved in those tasks that
use a delayed-response paradigm. In these kind of tasks performance is linked
to the ability of the animal to maintain information over some delay in order
to release a response at a later point in time, often facing the presentation
of several distractors interleaved between the presentation of the to-be-stored
cue and the release of the action. The physiological signature of this process
has been known since Fuster and Alexander (Fuster and Alexander, 1971) and
Kubota (Kubota and Niki, 1971) demonstrated an involvement of Pfc more than
thirty years ago, and it has been confirmed during the last decades. Neurons in
Pfc exhibit stimulus-specific, sustained activation during performance of these
tasks, whereas lesions to Pfc impair their correct execution. These results are
corroborated by neuroimaging studies, where Pfc activity has been shown to
increase as memory load increases (Braver et al., 1997) and to be sustained
over the entire delay interval (Courtney et al., 1997).
Miller and
colleagues (Miller at al., 1996) have provided direct evidence for the ability
of Pfc to maintain information in the face of interference. While cue-related
information was held in both Pfc and Infero-temporal cortex (IT) after the
presentation of the stimulus, subsequent presentations of distractors
obliterated the activity in IT, while the pattern in Pfc was maintained until a
match (target presentation) occurred. Not surprisingly, Pfc damages have been
associated with increased distractibility and perseveration (Damasio, 1985).
Pfc is also a
target structure of DAergic innervation, and a linkage has been suggested
between an imbalanced DAergic system, a poor Pfc activation and some key
symptoms of neurological and psychiatric diseases (Goldman-Rakic, 1995). One of
the aims of the first part of this work is to clarify, with the assistance of a
computational model, the possible roles played by DA in maintaining STM
information in Pfc neurons. Two tasks with different levels of difficulty will
be simulated in order to test the ability of the model to perform a WM task in
which storage of information over time is required. The simulation will
demonstrate a biologically plausible neural network designed to model STM
sorage naturally match some of the biophysical properties of Pfc neurons
reported in the literature.
2.2 Dopamine and prefrontal cortex:
neurophysiology, behavioral data, and the contribution of modeling
The action of
DA in spatial WM has been investigated for many years. Both primates and
rodents studies have highlighted the role of dopaminergic receptors stimulation
in the maintenance of activity in Pfc neurons during a task that implies some
delay between the cue and the response. In particular, dopaminergic midbrain
neurons become active (Schultz et al., 1993) and DA levels in the Pfc increase
(Watanabe et al., 1997) during working memory performance. It has been shown
that blockade of the subtype D1 of DA receptors in PFC (Sawaguchi and
Goldman-Rakic, 1991, 1994; Seamans et al., 1998) and DA depletion in Pfc
(Brozoski et al., 1979; Simon et al., 1980) are both correlated with working
memory deficits. At the same time, DA agonists might enhance delay task
performance (Arnsten et al. 1994; Müller et al., 1998). In vivo and in
vitro by studies have shown the influence of the different subtype D1- and
D2-receptor-stimulation on Pfc activity (Bernardi et al., 1982; Sawaguchi et
al., 1990a,b; Williams and Goldman-Rakic, 1995).
The
relationship between DA receptor stimulation, WM and some neurological and
psychiatric deficits has been the focus of interest for researchers from
different fields in the past decade. In a review paper, Lidow, Williams and
Goldman-Rakic (Lidow et al., 1998) summarize the evidence supporting a precise
linkage between antipsychotics, D1 and D2 receptors, Pfc
performance and some major features of schizophrenia, one of the major
psychiatric disease that has been linked to an imbalanced DAergic transmission.
Evidence from both physiological and behavioral studies suggests that normal
cognitive performance is bounded within a limited D1 activation
(Figure 2.1).
Figure 2.1 I) The first panel shows dose-dependent effects of the selective D1
antagonist SCH 39166 on activity of a prefrontal neuron. Top: Control
recording showing significant delay period activity (for the preferred target
direction of 0°). Middle: SCH 39166 (25 nA) induces dramatic enhancement of
delay activity selectively, without any increase in the background activity of
the cell. Bottom: At a high dose (75 nA), SCH 39166 abolishes activity during
the delay period as well as other periods of the task. Note: C s cue
period 0.5s , D s delay period 3.0s , R s response period bin s
50 ms . Drug used at 10 mM, pH w x 3.5–4.0.II) The proposed model that
takes into account the relationship between glutamatergic input, DA, GABA
interneurons, and pyramidal cells. Se text for details III) This diagram
illustrates the response of what Goldman-Rakic calls a “memory field” in Pfc.
Pfc neurons are deeply modulated by the amount of D1 stimulation. In
the top two panels: low levels of DA (as might occur in Parkinson disease) and
the absence of glutamatergic input facilitation cause the lack of maintenance
of memory fields. (From Goldman-Rakic
et al., 2000).
Goldman-Rakic
and her colleagues have proposed a model describing the relationship between D1
receptors, pyramidal cells, GABAergic neurons and WM performance (Goldman-Rakic et al., 2000). They
suggest that, at optimal level of DA stimulation, glutamatergic neurons
activation is enhanced more than interneurons activation in a typical network
which incorporates a basic circuitry shown in Figure 2.1. At low levels of DA
the absence of modulation does not allow any activation to be stored in working
memory. At high D1 activation, D1 receptors in pyramidal
neurons will plateau while GABAergic interneurons will still be able to
increase their firing rate, causing an overall decrease in activation, and no
STM storage.
Despite decades
of studies on the nature of the action of DA in Pfc, the exact mechanism and,
more importantly, the behavioral and computational function of the innervation
of Pfc from the DAergic nucleus Ventral Tegmental Area (VTA) is still obscure.
Studies aimed at determining the excitatory or inhibitory influence of DA in
Pfc neurons have led to controversial results. Most of these controversies are
due to the fact that often the effects of DA are studied in a purely bottom-up
approach, without the help of a theoretical framework organizing the collection
of a virtually infinite amount of data at different levels of detail. An
example of a “pure” bottom-up approach is the work by Durstewitz and Seamans (Durstewitz
and Seamans, 2002), in which a detailed computational model is proposed that
takes into
Figure
2.2 Schematic summary proposed by Durstewitz and Seamans (2002) of D1
receptor dependent modulation of PFC pyramidal neurons. (a) D1 agonists
increase (solid lines) NMDA currents (yellow), IPSCs through an increase in
interneuron axonal excitability (blue), and the persistent Na current in deep
layer PFC pyramidal neurons. D1 agonists simultaneously decrease (dashed lines)
non-NMDA/AMPA EPSCs by reducing glutamate release probability (pink), N-type Ca
currents, and the slowly inactivating K current. (b) Group data showing
the mean change in the amplitude of pharmacologically isolated NMDA (yellow
triangles), GABA (blue squares) and AMPA (pink diamonds) currents evoked by an
extracellular stimulation electrode placed next to layer V of Pfc neurons
recorded using whole-cell patch-clamp techniques in brain slices of the
prelimbic region of the rat PFC.
account numerous biophysical end electrophysiological data collected
over the past years on the presumed role of DA in Pfc neurons functioning.
Figure 2.2 summarizes the neurobiological evidences
Figure
2.3. From Durstewitz and
Seamans (2002). Differential modulation of low and high activity states by dopamine.
(for details of the simulation, see Durstewitz and Seamans (2002). (a)
The network exhibits low spontaneous and high persistent activity states that
can be induced by brief stimulation. A simulation of D1-mediated
effects on synaptic conductances (right hand side) leads to a suppression of
low and an enhancement of high activity states. (b) Reduced phase
portrait of the model system showing the nullclines and flow field of the
firing rates of the pyramidal cells (blue) and interneurons (green) for the low
dopamine condition (see text). In addition, the change in pyramidal
cell-nullcline according to D1 receptor activation (red) is shown: the fixed point corresponding to the stable low
and high activity states are pushed further apart and their basins of
attraction become steeper (note that the flow of the pyramidal cells tends to
the right below the blue nullcline and to the left above their
nullcline—although this is not readily apparent from the figure as the
derivative of interneuron rates is much higher except for regions close to the
interneuron nullcline). Vectors were normalized to unity length to better
indicate the direction of flow close to the nullclines. The dotted line
indicates the approximate location of the stable manifold of the saddle node
that separates the basins of attraction.
that are incorporated in the model, consisting of 100 pyramidal neurons
and 37 inhibitory interneurons that exhibit realistic channel kinetics. Figure
2.3 shows the plot of the main results of their work, which show how DA can
modulate the “resistence” of a given Pfc neuron to shift between a resting
state and a “high memory” state, defined as the average firing rate of the Pfc
field. A simplified phase plane analysis of such a system shows how DA can
widen the distance between the two critical points that describe the two
states, thereby making more difficult to switch between low and high memory
state in presence of DA (Figure 2.3).
Although the paper is,
to my knowledge, the most sophisticated computational model in terms of
biophysical details simultaneously implemented in a single simulation, it lacks
an important aspect that would constitute a important test for its validity,
namely how does the model control behavior. There is in fact no attempt to simulate
any behavioral task, or any systematic simulation on how the model can preserve
the Pfc activity in the face of interference. Furthermore, there is no
discussion on what is actually the input of Pfc (spatial pattern, single
pulse?). Also not addressed is the issue of what happens to the input pattern
in Pfc once DA is delivered, as well as how to embed the simulated Pfc field
into a laminar and a more realistic model that incorporates the minimal neural
circuitry able to carry out a behavioral task which requires Pfc. It is in fact
the task of behavioral control the most challenging issue for a model of Pfc,
rather than reproducing the biophysical properties of their neurons.
On a
fundamentally different level of abstraction is the work of Braver and Cohen
(Braver and Cohen, 1999). Here the emphasis is placed on behavioural control,
and the aim of the study is to demonstrate the ability of a simple model that
incorporates interaction between DAergic system and Pfc to perform some simple
tasks known to involve WM load. Unlike Durstewitz and Seamans model, Braver and
Cohen discard most of the microscopic interactions between DA and Pfc neurons,
concentrating on some macroscopic characteristics that the model should express
in order to control behaviour.
The work
of Braver and Cohen focuses on the most prominent behavioural impairment in
schizophrenia, namely the loss of cognitive control. This involves impairment
in attention, WM and behavioural inhibition. As suggested by multiple
evidences, goal-related information is actively maintained in Pfc, area that is
believed to be responsible for a top-down influence for controlling behaviour.
The authors suggest a possible role for DA as a “gate” for Pfc, determining the
maintenance/update of information based on the phasic DAergic signal. Indeed,
the proposal of DA as exerting a general gating function with respect to distal
input is a recurring theme in the modelling literature, and is also emerging
with increasing insistence in the neurophysiological community (Williams and
Goldman-Rakic, 1995; Durstewitz and Seamans, 2002; O’Reilly et al, 2002; Dreher and Burnod, 2002). Despite this agreement, the exact
mechanisms of action of this gate are still under debate, with the main
friction point being the status of the gate, namely whether the gate is
normally closed or open, and therefore whether DA action is to open or close
the gate.
According
to Braver and Cohen, context (defined as prior test-relevant information
internally represented) can bias the selection of an appropriate behavioural
response. This is particularly true in complex situations where the
non-preponderant response must be selected against a preponderant one. The
computational framework of their work is based on the analogy between
fixed-point attractor networks and the function attributed to Pfc (online
maintenance of context information). Although these networks are able to store
information in STM by virtue of their recurrent connectivity, the fixed-point
(equilibrium) that is reached by this architecture is dependent on the incoming
input. A given pattern cannot be stored in STM for a given time if a new input
perturbs the activation of the network. Cohen and Braver propose that DA acts
by modulating this update/maintenance process, determining the degree at which
new information can modify the established pattern of Pfc activation. This
proposal is interesting since usually dynamics of many recurrent architectures
are studied in the absence of external input. In other words, the
suggested dynamic holds if the network is allowed to reverberate its activity
without any external source of activation perturbing its dynamics. Their
proposal is that DA has a key role in this process. The main idea behind the
first set of simulations carried out by Braver and Cohen is to test whether
information associated with gating signal produces updating of Pfc,
whereas new information not associated with DA does not produce updating
(Figure 2.4).
Figure 2.4. The network proposed by Braver and Cohen. See text
for details
(1)
The Pfc layer is a competitive recurrent network. Note that the two
interneurons are tonically active. The dopaminergic unit (Gating Unit)
can synapse on all pathways, but Braver and Cohen have found that the optimal
pattern of connectivity involves gating of bottom-up connections (from input to
Pfc) and gating of inhibitory – to – context units. The gating unit has a
multiplicative effect on the weights (w):
(2)
where
(3)
and a(t) is the activity of the gating unit at time t. By (2), if
the DA signal is less than or equal to 0, the gain is equal to 1. The
activation function of the units is governed by a temporal difference equation
(3):
where
(4)
dt = time step of
integration, g = gain, B
= bias, Zi(t) = gaussian noise, s = variance of the distribution.
The main questions points
investigated in their simulation are:
1.
The role of the connectivity pattern between DA unit
and the memory layer. This point has been investigated by varying the pattern
of connectivity.
2.
The nature of the DAergic signal (phasic vs. tonic).
Phasic activity was simulated by setting the activity in the DAergic unit to
its maximum value C for 2 time steps (and zero elsewhere), whereas tonic
activity was simulated by setting the activation of the DAergic unit to 50% of
its maximum value throughout the last 50 time steps of the delay period.
3.
The total strength of the gating signal influence was
examined in both phasic and tonic conditions.
The main
results are reported below.
Figure 2.5. The main simulation results.
The best
results (update only when DAergic unit is active, resistance to interference
otherwise) were obtained when DA modulated both interneurons and context
(pyramidal) cells in the memory layer (Pfc). The results are not surprising.
The pattern of connectivity is designed in a way that allows the context layer
to preserve the activation through strong reverberation unless input and
interneuron signals are enhanced in order to break this reverberatory loop.
A weakness of the model is the fact that it assumes that the incoming
input is somehow bounded. Furthermore the interneurons are tonically active and
their activation is not modifiable, suggesting that the circuit overall might
be not as robust as expected.
Despite these weaknesses, the model exhibits
an interesting property that shed some light in conflicting physiological
findings about the effect of DA on Pfc, where both excitatory and inhibitory
effects were observed. According to this model and the results, DA has
inhibitory effect unless it co-occurs with the bottom-up input.
In the second set of simulations Braver and
Cohen tested the hypothesis that an increase in tonic DA activity should
produce deficits in working memory by incorporating the gating module into an
existing computational model of the Continuous Performance Task (Braver, Cohen
and O’Reilly, 1996). The model and the task are shown in Figure 2.6.
Figure 2.6. Left: The second netword proposed by Braver and Cohen.
Right: the task. See text for details
In the
CPT the letters are presented in a sequence of cue-probe pairs. The letter X is
a target only if preceded by a designated cue (A), introducing a certain load
in working memory in order to perform the task correctly. The behavioural data
target of the model was taken from 16 non-medicated patients and 16 controls.
Context sensitivity (the ability to respond correctly to an X probe based on
its prior content, comparing AX and BX) and context cost (a degree of response
slowing on non-target trials due to the presence of an A cue) were measured in
both groups. The schizophrenic group showed a decrease in sensitivity to
context, as well as a reduction in context cost. The latter is actually an
“impairment”, since schizophrenic were actually faster than normals, showing
less interference in AY trials.
The simulations are not very enlightening. The authors
employed a supervised Backpropagation algorithm (Werbos, 1974; Rumelhart et
al., 1986) in order to train the weights. This poses a serious question on how
does the brain cope with a continuously varying environment that does not allow
the organism to “learn” every task. The DA impairment was simulated by adding a
noise component to the activation of the gating unit, with the result of
raising the mean value of gain for tonic gating and decreasing the mean value
of gain for phasic gating. The results show that the decreased phasic
activity causes a difficulty in updating new information, whereas the increased
tonic activation produces deficits in the maintenance of context. Despite
the overall good framework of the model, there is a general lack of
neurobiological constraints incorporated into the model, and the linkage
between known physiological and neuroanatomical data is generally lacking.
The
work by Dreher and Burnod (Dreher and Burnod, 2002) lies in an intermediate
position with respect to the previous studies in terms of neurobiological
details incorporated in the model and the behavioral database target of the
simulation. The network implemented by Dreher and Burnod is very simple and can
be defined as a “minimal architecture” for the task simulated: two neurons!
Dreher and Burnod have chosen to limit the study to a two-neuron system in
order to fully characterize the system dynamics and, with some simplifying
assumptions, to remain able to simulate a non trivial WM task. The task chosen
for the simulation is the delayed alternation task (Figure 2.7). In this task
the animal has to alternate
Figure 2.7. (a)
From Dreher and Burnod, 2002. Structure of the network model. Deep-layer
pyramidal Pfc neurons receive inputs from long-range cortical areas on superior
layers and recurrent excitatory inputs from neighboring cortical columns on
their basal dendrites. The mechanism that permits to switch OFF sustained
activities of PFC pyramidal neurons can either be attributed to intrinsic
neuronal properties, as the slowly inactivating potassium conductance, or to
recurrent inhibitory neurons. (b) Left: The activity of the network can
be switched ON and OFF by transient excitatory inputs x arriving on superficial
layers. Right: the conductance is recruited at an intermediary level during the
sustained activity of the neuron and increases rapidly with frequency y when a
new excitatory input arrives during the discharge, which can induce an
intrinsic inhibition, furnishing a mechanism to stop the discharge. This
mechanism allows the same excitatory input x to induce both ON/OFF and OFF/ON
transitions.
between two different responses
(eg., right/left button press) separated by a delay. This task is impaired in
rats and monkeys with Pfc lesions. Dreher and Burnod’s focus is on the subset
of Pfc neurons that show sustained activity during the delay between
right-sided and left-sided trials (note that in Pfc cells do not respond only
for delays, but also during movement execution or both).
VTA
lesions have been shown to cause deficits in this task. Infusion of D1 agonist
and antagonist in Pfc impairs performance (although this effect is dose and
task-dependent). Dreher and Burnod then comment on recent simulation works on
DA effects on Pfc and their consistency with new biological data.
Servan-Schreiber and Cohen (1990 ,1992) have proposed that DA increases the
gain (signal-to-noise ratio) of the sigmoidal activation function of their putative
Pfc neurons, facilitating both excitatory and inhibitory neurons. Dreher and
Burnod suggest that in-vivo data show an inhibitory influence of DA on Pfc.
Furthermore, this duality of effects can be also found in other target sites of
DA action, like the Nucleus Accumbens (NAc). In the NAc, nigrostriatal and
mesolimbic DA activity gates the input of sensory, motor, and incentive
motivational (e.g. reward) signals to the striatum (Horvitz, 2002). Striatal
reward signals, which can be
Figure 2.8. (a) Structure
of the delayed alternation task used by Dreher and Burnod. Successive
go-signals x (simultaneous apparition of the two circles) require to alternate
between two responses (right and left) separated by a delay of 5 s. (b)
Left: network activity and time course of the post-synaptic effect of DA
(temporary threshold s ) in the Pfc for successive transitory excitatory inputs
x. Right: when noise is added to the system, unexpected transitions can occur
before the next go-signal if the noise is higher than the temporary threshold.
Figure 2.9. (a) Activity of the network and time course of the
post-synaptic effect of DA (temporary threshold) in the PFC for two successive
transitory excitatory inputs x in the case in which the time constant of the
threshold is lower (left), adapted (middle) or higher (right) than the delay. (b)
Network performance for increasing level of D1 receptors stimulation. Noise is
distributed according to a Poisson process of mean 5 s.
originating in the orbitofrontal cortex and basolateral
amygdala, reach the NAc by known projections. A DA signal of salient
unexpected event occurrence can gate the effects of the glutamatergic
orbitofrontal reward input to the striatum just as it gates the throughput of
corticostriatal sensory and motor signals needed for normal response execution.
Processing of these incoming signals is enhanced when synaptic DA levels are
high, because DA enhances the synaptic
efficacy of strong concurrent glutamate inputs while reducing the efficacy of
weak glutamate inputs (Howitz,
2002). Dreher and Burnod’s model is built on the assumption
that DA reduces the impact of intervening stimuli on network activity trough D1
stimulation. Dreher and Burnod further comment that Cohen et al. (Cohen et
al. 1996) has proposed that the effect of DA is time independent. This time
independency is difficult to advocate, since VTA neurons show to respond at
precise time in behavior (unexpected stimuli, reward presentation, etc..). Braver et al. (Braver et al, 1999) has therefore proposed
that DA modulation facilitates incoming inputs at the precise time of their
presentation. The interesting comment by Dreher and Burnod is that DA is
unlikely to play this role, for different reasons. The first is that Dreher and
Burnod believe that DA has an inhibitory role on Pfc neurons. The second reason
is related to timing: when VTA neurons fire, the DA released by their terminals
in Pfc requires some time (100-150 msec for VTA to fire, plus some other time
for DA to act at the post-synaptic site – 200 msec observed in the striatum).
This time is greater that the time necessary for a stimulus to reach Pfc. Given
these assumptions, Dreher and Burnod propose that the role of DA transmission
is to momentarily isolate Pfc neurons, in particular their apical dendrites,
from the influence of distal input. This would be achieved by increasing a
threshold (the electrotonic distance between the dendrite and the proximal
dendritic regions). The novelty in Dreher and Burnod’s proposal is the fact
that they take into account both phasic and tonic DA effects.
The
main results of the simulation are shown in Figures 2.8 and 2.9. The network
shows an inverted U in performance with respect to the level of DA stimulation:
low or high levels of DA increase the error in the task. Overall, the
simulations are robust and the hypothesis of DA as restricting the apical
dendrites input in Pfc finds increasing consensus (Lewis and O’Donnell, 2000).
The main concern about Dreher and Burnod’s hypothesis is that the network used
is relatively small and simplified, and does not address how the interactions
within a more complex and realistic neural network can modify, and possibly
nullify, his results.
The
aim of this work is to investigate how this increased complexity can be studied
without the system and the results becoming too complicated and obscure.
2.3 A computational model of the delayed alternation task: role of dopamine
in short-term memory
The model
proposed in this section is a variation of a recurrent competitive field (RCF)
in which a non-specific input is added in order to mimic the DAergic input to
Pfc neurons. RCF have been introduced by Grossberg in the seventies in the
attempt to explain how an arbitrary activation pattern can be stored in STM in
a population of neurons that obey shunting membrane equations (Grossberg 1973).
Such systems have been characterized mathematically by later works (Cohen and
Grossberg, 1983) in an attempt to define the conditions, parameters and
architecture that allow a given activation pattern to be stored without major
distortions in STM. RCF are therefore particularly suitable for being employed
as a putative Pfc network able, given certain conditions, to store an arbitrary
STM pattern once the input is turned off.
Figure 2.10. Left: VTA innervated Pfc with its DAergic terminals. Right: A schematic of the basic
building-block of the model. A pyramidal neuron receives a) input from previous
cortical stages b) VTA DAergic innervation c) self-recurrent excitation and d)
Intracortical inhibition.
One
of the major issues in RCF is that their dynamics can be, up to a certain
extent, characterized and the stability of their pattern guaranteed only if the
system is considered in “isolation”. This means that a pattern that is stored
in STM would be “washed away” if no mechanism isolating the network from
non-recurrent, non-intrinsic input (i.e., distal afferents) would prevent the
external input to overcome the recurrent activation.
The
main brain circuitry target of the simulation is shown in Figure 2.10, whereas
the basic architecture of the model is shown in Figure 2.11. The network has 10
pyramidal (excitatory) neurons and 10 (inhibitory) interneurons. A pyramidal
node, simulating a Pfc neuron, receives
a) input from previous cortical stages b)
VTA DAergic innervation c)
self-recurrent excitation and d)
intracortical inhibition.
Figure 2.11. The model: every pyramidal neuron interact with the
neighboring pyramidal cells through a population of inhibitory
interneurons.
(5)
The equations describing the
dynamics of the excitatory (x) and
inhibitory (y) nodes are:
(6)
and
(7)
where
and where xi and
yi are the units activation (or STM), i = 1,2,…..,10, Ii
is the bottom-up input to the cell, 
is the self-excitatory input, and
are the recurrent excitatory and inhibitory inputs,
respectively, A and B and C
are the decay rate, the excitatory and the inhibitory saturation point,
respectively, and f(h) is the feedback function defined by Equation (7),
where h is the argument of the function and F is a constant. In Equation (5), 0 £ DA £ 1.. In all simulations, A = 1, B = 1, C = 0.2, F = 10. As shown in Equation (5), DA has a double effect on the RCF
excitatory neurons: from one side, it isolates apical dendrites from external
(distal) input trough the term (1-DA).
A high DAergic signal would “switch” Pfc mode from “externally-driven” to
“internally-driven” (recurrent mode). At the same time, DA is assumed to amplify self-recurrent connections trough the term
.
The
network described above has been tested in two tasks involving different
degrees of WM load. The first set of simulations, described in Figures 2.12
trough 2.18, involves a simple WM (sWM) task in which a stimulus, a spatial
pattern of activation instantiated in the RCF from the distal connections from
higher associative areas, is maintained until a new stimulus (for instance, an
unconditioned stimulus, or US) arrives in Pfc. In the sWM task the ISI is 1100
msec (where 1 msec = 1 network cycle), while both stimuli were on for 50 msec.
The network is simulated with variable DAergic innervation parameters. Each
figure contains 4 panels: the upper left shows the activation of the 10
pyramidal neurons, the upper right shows the activation of the DAergic node
(VTA), the lower left the input pattern and the lower right the inhibitory
interneurons activity.
Figures
from 2.19 to 2.23 show the network performance in a Delayed alternation Task
(DAT). This task, described previously (Figure 2.8) and employed in Dreher and
Burnod’s simulations, is slightly more complicated and demanding than the
previous one. At each moment in time, the subject (the monkey, or the system)
should alternate a left/right response in order to obtain a reward. A delay is
interposed by the execution of the movement and the consequent delivery of the
reward and the onset of the next cue that would trigger the next movement. This
implies that the subject has to temporarily store the previous movement in WM
throughout the delay in order to execute the correct alternation. The DAT
requires that the input trace, a 50 msec activation pattern, should precisely
alternate at each trial, and the activation pattern should be prevented from
being corrupted or decay trough the entire duration of the task. Furthermore,
the network has been tested in presence of noise during the delay interval. In
the delay interval, the network relies exclusively on its reverberatory
activity and the DAergic input in order to keep the pattern in WM. Noise
consisted in 50-msec-bursts of random input ranging from 0 to 0.4 (see figures
for details). Each simulation runs 12 seconds, with 5 alternations between
left/right response.
As
can be seen in the following simulations, the network can store information in
WM for a limited amount of time. This “design choice” is justified from the
assumption that important, salient information is paired with a DAergic
activity (e.g., a novel, arousing stimulus, a
Figure 2.12. sWM task. No DA. Each panel in the present and the following figures represent the
activation of the pyramilda neuron x over time (top-left, activation on
y axis, time on x axis), the DAergic activity (top-right, activation on y axis,
time on x axis), the input pattern (bottom-left, units on the x axis, time on
the y axis, activation on the z axis), and the activation of the inhibitory
interneuron y over time (bottom-right, activation on y axis, time on x
axis). Notice how the distal input from apical dendrites drives the network
activity. In the absence of DA, the information rapidly decays in STM.
Figure 2.13. sWM task. DA baseline. The distal input from apical
dendrites drives again the network activity, but this time an activity well
above baseline is preserved in STM. However, the pattern is lost after a few
msec.
Figure 2.14. sWM task. Phasic DA. The input pattern is preserved
for almost 500 msec in the presence of a strong DAergic spike timed with the
stimulus.
Figure 2.15. sWM task. Phasic and tonic DA. The presence of both
a phasic and a tonic component allows the pattern to be stored in STM for the
entire duration of a delay.
Figure 2.16. WM task. No tonic DA, noise. The 50-msec-noise
interferes with the STM pattern that, due to the absence of a tonic DA
baseline, decays.
Figure 2.17. sWM task. A hyperactive tonic DAergic input causes
the loss of the pattern. This effect is mainly due to the multiplicative effect
of DA on self-excitation and to the spatial limitation of the
on-center/off-surround architecture.
Figure 2.18. sWM task. A calibrated phasic-tonic Daergic
transmission prevents noise to disrupt the pattern.
Figure 2.19. The DAT task. The first two figures on the top show
the input for the DAT (left-right response), in which each bar represent the
strength of ths stimulation for a given input unit. The bottom panels show the
time course of the input pattern, in which the alternation of input is evident.
Figure 2.20. The DAT task. Optimal DAergic innervation allows a
correct alternation of left/right response. Notice that two units correctly
succeed in the competition for STM storage during alternating delays.
Figure 2.21. The DAT task. Tonic DAergic hyperactivity causes
interference in the pattern storage: in the final 200 msec before the response
is generated, one of the units increase its activation and matches the
“correct” units activation. This can potentially lead to an error.
Figure 2.21. The DAT task. Optimal DAergic input allows the
correct execution of the task despite noise. Notice that an error might occur
in the third trial.
Figure 2.22. The DAT task. Tonic DAergic hyperactivity disrupt
the performance in the noisy condition.
Figure 2.23. The DAT task. Tonic and Phasic DA. In a difficult, noisy task,
increasing the phasic level of DA can be beneficial to the target storage.
2.3 Dopamine, gating, timing and learning
In summary, the
simulations presented show the following interesting properties:
a) The DAergic input does not
necessarily have to be exactly timed with the input stimulus, as proposed by
Cohen and Braver. Indeed, a precise
coincidence of the DAergic spike with the external input would prevent the WM
of a given stimulus to be instated. This is a sharp discrepancy between the
present model and the one proposed by Cohen and Braver. In their model, the
DAergic spike should be exactly timed with the input stimulus in order to cause
an update in WM. This is unlikely to happen for the reason exposed in Dreher
and Burnod’s paper. In the current model, the DAergic input should follow the
pattern to be stored within a few msec, depending on the strength of the input
pattern and the DAergic spike. Furthermore, a DAergic spike causes a
long-lasting increase in DA level in Pfc that lasts from minutes to hours. This
result in not conflicting with the present model, while it is contradicting the
basic assumptions made by Braver and Cohen.
b) DAergic hyper and hypo-activity
cause different pattern of impairment in the model WM. DAergic hypoactivity
prevents the instantiation of the WM pattern. This would cause the RCF to be
driven by external input, and could be approximated to the increased
distractibility shown by patients with a Pfc lesion, in particular the fact
that hypofrontal patients are easily distractible by environmental events. On
the other side, DAergic hyperactivity causes a different sort of deficits. STM
is maintained and activation patterns are formed in WM. Depending on the input
pattern, the length of the delay, and the amont of DAergic hyperactivity, the
network can show interference of “inappropriate” patterns into WM, or the
expression of non-preponderant units activation due to the amplification of the
self-recurrency.
c) In order for the network to
alternate between rapidly changing input pattern and responses, a DAergic
“pause” was required at the time of the
Summarizing,
the present simulations constitute a promising approach for building an useful
model of WM and Pfc functioning. In particular, the level of abstractions of
the proposed architecture is optimal for achieving both a good level of
biological plausibility and for simulating higher-level tasks without loosing
track of the causal relationship between the components of the system. This is
a crucial step, because no neural model can be considered to be a useful one if
it does not capture the essence of what is the ultimate target of the nervous
system: adaptive control of behavior.
Finally, one of
the main weaknesses of the present work is that it does not address what is, in
my opinion, the most important aspect of Pfc-DA interactions, namely how does
the brain balances exactly the DAergic innervation of Pfc in order to fine-tune
the system and cope with complex behavioral tasks. The second part of this
thesis is an attept to address these issues.