close

Вход

Забыли?

вход по аккаунту

?

j.neuropsychologia.2018.08.016

код для вставкиСкачать
Author’s Accepted Manuscript
Semantic composition of sentences word by word:
MEG evidence for shared processing of conceptual
and logical elements
Linmin Zhang(张琳敏), Liina Pylkkänen
www.elsevier.com/locate/neuropsychologia
PII:
DOI:
Reference:
S0028-3932(18)30503-7
https://doi.org/10.1016/j.neuropsychologia.2018.08.016
NSY6889
To appear in: Neuropsychologia
Received date: 11 April 2018
Revised date: 17 August 2018
Accepted date: 18 August 2018
Cite this article as: Linmin Zhang(张琳敏) and Liina Pylkkänen, Semantic
composition of sentences word by word: MEG evidence for shared processing of
conceptual
and
logical
elements, Neuropsychologia,
https://doi.org/10.1016/j.neuropsychologia.2018.08.016
This is a PDF file of an unedited manuscript that has been accepted for
publication. As a service to our customers we are providing this early version of
the manuscript. The manuscript will undergo copyediting, typesetting, and
review of the resulting galley proof before it is published in its final citable form.
Please note that during the production process errors may be discovered which
could affect the content, and all legal disclaimers that apply to the journal pertain.
RESEARCH REPORT
TITLE PAGE
Title: Semantic composition of sentences word by word: MEG evidence for shared processing of
conceptual and logical elements
Author names and affiliation:
Linmin Zhang (张琳敏)1, 4 and Liina Pylkkänen1, 2, 3
1
Department of Linguistics, New York University, New York, NY, 10003
2
Department of Psychology, New York University, New York, NY, 10003
3
New York University Abu Dhabi Institute, Abu Dhabi, UAE
4
NYU-ECNU Institute of Brain & Cognitive Science, New York University Shanghai, Shanghai,
China
Corresponding author:
Linmin Zhang (张琳敏), linmin.zhang@nyu.edu
1555 Century Avenue, New York University Shanghai, Shanghai, China, 200122
Acknowledgements:
This research was funded by the National Science Foundation Grant BCS-1221723 (L.P.)
and grant G1001 from the NYUAD Institute, New York University Abu Dhabi (L.P.).
The first author acknowledges the support of the NYU-ECNU Institute of Brain and
Cognitive Science at NYU Shanghai during the stage of revision.
The authors declare no competing financial interests.
1 / 49
ABSTRACT
Human language is a mixture of many types of elements, some clearly conceptual, like dog and
run, and others more functional/logical, such as negation or quantificational elements (not, many,
all). While theories are emerging for the neurobiology of conceptual combination, the neural
mechanisms of integrating concepts with logical information remain largely unstudied. Do neural
correlates of concept composition also reflect the composition of concepts with logical elements?
In a previous MEG study, we have shown that in noun-noun compounds (e.g., tomato soup), the
conceptual specificity of the first word modulates left anterior temporal lobe (LATL) amplitudes
elicited on the second word, suggesting an effect of conceptual specificity, or informativeness,
on the process of conceptual combination. Here we tested how this pattern is affected by
negation, which has the ability to reverse informativeness relations: for example, while poodle is
conceptually more informative than dog, no dog negates more possibilities and is therefore more
informative than no poodle. We manipulated the informativeness of sentential subjects by fully
crossing conceptual specificity (poodle vs. dog) with the presence of negation (no vs. a) to create
positive and negative sentences (e.g., no/a-(green)-lizard-is-sleeping) and tested whether the
effect of conceptual specificity was reversed for the integration of negative as compared to
positive subjects. Exactly this pattern was observed in the LATL and surrounding frontotemporal cortex during the processing of the sentence-final verb, suggesting a shared mechanism
that tracks informativeness in integrating conceptual and logical elements in this network.
KEYWORDS: language comprehension, magnetoencephalography, semantic composition,
conceptual knowledge, sentential polarity
Graphical abstract:
2 / 49
1. INTRODUCTION
Human language is a rich combinatory system in which elements labelling concepts, such as cat
and sleep, combine with a wide array of logical or functional elements, such as all, not, or the.
How these two types of elements combine together is a fundamental issue in the neurobiology of
language. Even though many studies have characterized linguistic composition in general terms
(Lau et al. 2008, Pallier et al. 2011, Fedorenko et al. 2016) or by focusing on simple conceptual
composition in small phrases (Baron and Osherson 2011, Westerlund and Pylkkänen 2014, Price
et al. 2015, Zhang and Pylkkänen 2015, Ziegler and Pylkkänen 2016), it is still unknown whether
shared or distinct mechanisms support compositional processes between and beyond concepts.
Given that functional/logical elements are actually a heterogeneous group, as a starting point for
such research, the current study built on our extant understanding of conceptual composition and
investigated how one particular logical element, negation, gets composed with conceptual
knowledge.
Previous studies have identified the left anterior temporal lobe (LATL) as a combinatory
site that appears to specialize in the combination of concepts, as opposed to composition more
generally (Westerlund and Pylkkänen 2014, Zhang and Pylkkänen 2015, 2018). Core evidence
3 / 49
for this comes from manipulations of conceptual specificity, inspired by prior hemodynamic and
neuropsychological results on the effect of this variable (Hodges et al. 1992, Rogers and
Patterson 2007, Binney et al. 2010). For example, LATL amplitudes elicited by dish, as
measured millisecond by millisecond by MEG, are higher when dish is being integrated with a
modifier such as lamb, to yield lamb dish, than when it is being integrated with the more general
meat, yielding meat dish (Zhang and Pylkkänen 2015). Importantly, no reliable LATL
modulation was observed at the first word of the noun-noun phrase, i.e., at lamb vs. dish, despite
their contrasting conceptual specificity. These findings suggest that LATL compositional effects
track some type of informativeness in the dynamic process of integration.
Interestingly, negation, as a logical element, also contributes to informativeness by
reversing the specificity relation between conceptual items. For example, while poodle is a more
informative concept than dog, no dog negates more possibilities and is therefore more
informative than no poodle. We asked whether the LATL would be sensitive to informativeness
resulting from the composition of both conceptual and polarity information and when effects
reflecting this sensitivity would be elicited in the processing of positive and negative sentences.
For the first question, if the LATL is only sensitive to conceptual informativeness, then
when positive and negative subjects (e.g., a poodle, no dog) combine with a predicate, LATL
effects should only track the conceptual specificity of the noun in the subjects, and the
computation of polarity should be a separate process. Thus for positive and negative sentences,
we should only observe main effects of noun specificity in the LATL. Alternatively, if instead, a
mechanism sensitive to general informativeness is what modulates LATL activity, we should
observe a reversed pattern of LATL effects for negative as compared to positive sentences (i.e.,
4 / 49
interaction effects between sentential polarity and conceptual specificity), suggesting a shared
hub for the processing of conceptual and logical information (see Figure 1).
The second question is related to the temporal profile of negation processing. Prior
psycholinguistic research has reported extra processing costs for negative sentences in terms of
response time and/or error rate (Wason 1961, Cornish and Wason 1970, Clarke and Chase 1972,
Carpenter and Just 1975, Howard 1975, Oaksford and Stenning 1992, Evans et al. 1996, Reichle
et al. 2000, Mayo et al. 2004, Prado and Noveck 2006, Tettamanti et al. 2008, Tomasino et al.
2010). However, the measurements of response time and error rate reflect rather the strategy and
difficulty in evaluating the semantics of negative sentences under certain context, as opposed to
time course. The studies of Hagoort et al. (2004), Nieuwland and Kuperberg (2008), and Staab et
al. (2008) have also shown that the strategy and difficulty in this kind of semantic evaluation can
be modulated by our expectations and pragmatics. Thus, how the word-by-word composition of
negative sentences is performed and what its temporal profile looks like are still largely open
questions. Given that the processing of negative action-related sentences (e.g., I do (not) grasp
an apple) has an impact on motor-related brain activity (Tettamanti et al. 2008, Christensen
2009, Tomasino et al. 2010), recently, Papeo et al. (2016) measured motor excitability and
inhibition in processing sentences like I am not writing, revealing that negation processing takes
place immediately, not as a second stage following the processing of their positive counterparts,
and that by 250 ms after the onset of critical words indicating sentential polarity, effects related
to motor excitability can already be observed. Instead of using positive and negative predicates,
the current experiment embedded positive and negative nominal phrases in the subject position
of sentences, thus offering an opportunity to examine the temporal profile and compositional
mechanisms of negation processing in a novel way. Specifically, if the polarity of determiners
5 / 49
interacts with noun specificity in modulating LATL activity, the timing of these interaction
effects would tell us whether LATL activity reflects the result of calculating a general
informativeness immediately after the full presentation of positive and negative subjects, or
rather a dynamic integration that tracks the informativeness of to-be-integrated subjects during
the stage of combining subjects with predicates.
In sum, we manipulated the informativeness of positive and negative sentence subjects
and tested how integrating these subjects into subsequent intransitive verbs modulates brain
activity. Specifically, we varied three factors: (i) noun specificity (e.g., lizard vs. reptile), (ii)
determiner polarity (a vs. no) and (iii) whether the noun was modified by an adjective. All
sentences described ongoing events and fit into the frame: ‘determiner-(adjective)-nounauxiliary-verb’ (e.g., a (green) lizard is sleeping) (see Figure 2).
We used magnetoencephalography (MEG) to record participants’ brain activity
millisecond by millisecond for the entire sentence and analyzed source-localized MEG activity in
the entire left temporal and frontal lobes and the left temporo-parietal junction, allowing us to
investigate the role of several possibly relevant regions beyond the LATL.
2. MATERIALS & METHODS
2.1 Participants
22 right-handed, native English speakers participated in the study. All had normal or
corrected-to-normal vision and gave informed consent. One participant was excluded from MEG
data analyses due to excessive noise during the MEG recording. Another was excluded due to
low accuracy in the sentence-picture matching task, with an accuracy score that was more than 3
standard deviations below the group mean. Thus 20 participants were included in MEG data
6 / 49
analyses (11 females and 9 males; average age = 25 years, SD = 4 years). All data were collected
at the Neuroscience of Language Lab at New York University in New York, NY.
2.2 Experimental design and stimuli
Our experiment employed a 2 x 2 x 2 design (i.e., 8 conditions, see Figure 2) with
determiner type (a vs. no), noun specificity (specific vs. general), and adjectival modification
(+Adj vs. -Adj) as the three factors. These manipulations all occurred within the subject
determiner phrase (DP) in event-describing sentences (e.g., a/no (green) lizard/reptile is
sleeping). Inserting an adjectival modifier for half of the conditions allowed us to further adjust
conceptual specificity within the subject.
It seems that the insertion of an adjectival modifier might also allow us to test, within the
context of a full sentence, phrasal-level compositional effects that have been revealed previously
with two-word noun phrases (e.g., red boat, see Bemis and Pylkkänen 2011, 2013a, 2013b,
Westerlund and Pylkkänen 2014, Westerlund et al. 2015, Zhang and Pylkkänen 2015), but given
that in the current design, an adjective would always be fully predictive of an upcoming noun
whereas the determiner a/no could be followed by either a noun or an adjective, we were aware
that there might be spatiotemporal deviations for elicited effects of adjectival modification.
We used 30 different pairs of nouns (specific vs. general, e.g., lizard vs. reptile) to
construct stimuli for the 8 conditions. Each subject DP was then combined with two different
verbs: e.g., a/no (green) lizard/reptile is sleeping and a/no (green) lizard/reptile is eating. This
allowed us to increase power, and the use of two different verbs reduced the likelihood that
participants would predict the sentence’s conclusion by the time they read the noun. Similarly,
for the half of the trials that contained an adjectival modifier, we used a small set of adjectives
7 / 49
(10 adjectives) and combined each of them with 3 different noun pairs (e.g., green +
lizard/reptile/truck/vehicle/ ...) to prevent participants from associating a certain adjective with
any particular nouns. In total, there were 480 trials: 480 = 30 pairs of nouns * 8 conditions * 2
verbs per group.
We used WordNet Search 3.1 (http://word- net.princeton.edu/) (Miller et al. 1990, Miller
1995) to generate noun pairs like lizard vs. reptile, and then used Amazon Mechanical Turk
(AMT) to conduct norming tests, asking native speakers to judge which word in each pair is the
more specific one. We collected data from 100 participants on AMT. Data from 5 participants
were rejected due to low accuracy in answering filler items (noun pairs in which one word is
uncontroversially more specific than the other, e.g., apple vs. fruit). Table 1 shows the nouns
used in the current experiment and the results of the norming tests: i.e., the percentage of AMT
participants whose judgment was consistent with the specificity relation provided by WordNet.
General and specific nouns were matched for 7 lexical factors (see Table 2; all the data
are from the English Lexicon Project (ELP), Balota et al. 2007): (1) log frequency; (2) word
length; (3) number of phonemes; (4) number of syllables; (5) number of morphemes; (6) lexical
decision time; and (7) naming time. To control the naturalness and likelihood of all sentences,
phrases of the pattern ‘adjective + general/specific noun’ were matched for bigram frequency
(data of the Corpus of Contemporary American English (COCA), Davies 2008) and transition
probability from first word to second (calculated from the data of COCA) (see Table 2).
Similarly, bigram phrases of the pattern ‘general/specific noun + verb’ were also matched for
bigram frequency (data of COCA) and transition probability (calculated from the data of COCA)
(see Table 2).
8 / 49
In order to keep participants alert and to verify that they were indeed understanding the
presented sentences, we added a picture-matching task to the end of each trial. Each sentence
was followed by a picture and participants were instructed to judge whether the picture matched
the meaning of the sentence. For example, if the sentence no green lizard is sleeping was
followed by a picture showing only a blue sleeping lizard or a green lizard that was awake, the
correct answer should be ‘mismatch’. Thus, in order to give correct answers, participants needed
to pay attention to every word in a sentence, except for the auxiliary is. For half of the trials, the
picture meaning matched with the sentence meaning and for the other half it did not. The ‘match’
and ‘mismatch’ answers were evenly distributed among the 8 conditions. The picture stayed on
the screen until participants pressed a button to respond.
The trial structure is illustrated in Figure 3. Each trial started with a fixation ‘+’, and then
sentences were presented word by word. The fixation and each word remained on the screen for
300 ms, and there was a 300 ms interval between adjacent items.
Participants were asked to avoid blinking during sentence presentation. In order to give
them enough time to relax their eyes and to remind them not to blink during the word-by-word
presentation of sentences, we added a line of instruction before the fixation of each trial: ‘Press
when you’re ready. DO NOT BLINK!’ The instruction stayed on the screen until participants
pressed a button. Therefore, participants could self-pace the progression of the whole
experiment. The stimuli were presented by PsychToolBox software (Brainard 1997, Pelli 1997).
The target words were presented in lowercase letters; all of the characters were presented in
white 30-point Courier font on a grey background. The 480 trials were divided evenly into 16
blocks, with 3 to 4 trials for each condition in a block. The presentation of nouns (e.g., lizard and
reptile) was counterbalanced to avoid repetition in any block. The trials within each block and
9 / 49
block order were also randomized for each participant. Between blocks, participants could
choose to take a short rest or continue immediately. There were 20 additional trials for the
practice session. The purpose of the practice session was to familiarize participants with the
meaning judgment task.
2.3 Procedure
Prior to each MEG recording, we used a Polhemus Fastscan three-dimensional laser
digitizer to scan the participant’s head shape and locate the positions of five maker coils placed
across the forehead. The digitized head shape was later used to constrain source localization
during data processing by co-registering the position of the five coils with respect to the MEG
sensors.
Before the MEG recording, there was a practice session (containing 20 trials) outside the
magnetically shielded room. During this practice session, participants were given the same
instructions as in the MEG recording, and for each trial they got feedback after responses, so that
they could verify their comprehension of the task and the experiment process. The practice
session could be repeated if participants felt that they needed more practice to fully comprehend
the task.
During the MEG recording, participants lay in a dimly lit magnetically shielded room.
The positions of the marker coils were measured at the beginning and the end of the experiment.
MEG data were collected using a whole-head 157-channel axial gradiometer system (Kanazawa
Institute of Technology, Nonoichi, Japan), at a 1000 Hz sampling rate with a low-pass filter at
200 Hz and a notch filter at 60 Hz. Stimuli were projected onto a screen about 50 cm away from
participants’ eyes. During the recording, participants used the index and the middle finger of
10 / 49
their left hand to answer questions, but they got no feedback. The recording session lasted
approximately 65 minutes.
2.4 Data processing
Raw MEG data were first noise-reduced via the continuously adjusted least-squares
method (Adachi et al. 2001) in MEG Laboratory software 2.004A (Yokogawa Electric and Eagle
Technology Corporation, Tokyo, Japan), and then noise-reduced MEG data were processed and
analyzed with MNE-Python (Gramfort et al. 2013, 2014) and the Eelbrain package version
0.22.1 (http://pythonhosted.org/eelbrain/, Brodbeck 2016). MEG data were band-pass filtered
between 1 and 40 Hz with a hamming window (see https://mnetools.github.io/stable/generated/mne.io.Raw.html#mne.io.Raw.filter for details). The high-pass
filter was necessary for analyzing data acquired in an environment with high urban noise (in the
downtown NYC area). Then for the 4 conditions with no adjectival modifiers, MEG data were
epoched (i.e., segmented out) from 100 ms before to 2400 ms (= 600 ms/word * 4 words) after
the sentence onset (i.e., the onset of the determiner); while for the 4 conditions with adjectival
modifiers, MEG data were epoched from 100 ms before to 3000 ms (= 600 ms/word * 5 words)
after the sentence onset. Individual epochs were automatically rejected if any sensor value
exceeded 3000 fT at any time point. Epochs containing eye blinks were identified by
individually visualizing raw activity for each epoch: if an epoch showed sudden, stark increase in
amplitude, then the topography for that epoch was plotted. If the corresponding magnetic field
pattern showed the characteristic frontal distribution of a blink, with a sink and source over the
eyes, then that epoch was rejected. Moreover, if the raw activity for individual channels in an
epoch greatly deviated from neighboring channels, activity for those individual channels in that
11 / 49
epoch was removed, and field interpolation of MEG data was applied. Epochs were baseline
corrected with a pre-stimulus interval of -100 ms to the onset of the determiner.
The digitized head shape information and marker coils’ positions were used to constrain
the source localization by co-registering the 5 coils located around the face with respect to the
MEG sensors. 7 of the 20 participants have had structural magnetic resonance imaging
(structural MRI) scans (these structural MRI scans were acquired in a separate session at the
Center for Brain Imaging at New York University, with 3T Siemens Allegra scanner with T1weighted MPRAGE sequences), and we co-registered each one’s head shape with their structural
MRI scan to construct their cortical surface. For the remaining 13 participants, we co-registered
each one’s head shape with the standard FreeSurfer average brain fsaverage (CorTech and MGH/
HMS/MIT Athinoula A. Martinos Center for Biomedial Imaging) to construct their cortical
surface (since the head size of participants could be different from that of the average brain, this
co-registration involved both a shift and rotation and a scaling).
Cortically constrained minimum norm estimates (Hämäläinen and Ilmoniemi 1994) were
calculated with MNE (MGH/HMS/MIT Athinoula A. Martinos Center for Biomedial Imaging).
We followed a standard procedure, consisting of the following steps: (i) after constructing the
cortical surface of each participant, the boundary element model method was adopted to
calculate the forward solution (Mosher et al. 1999); (ii) then the inverse solution was computed
from the forward solution as well as the noise covariance matrix (in the current experiment, we
used the baseline part, i.e., the 100 ms pre-stimulus interval); (iii) the inverse solution was
applied to the evoked data to yield current estimates at each source for three orthogonal dipoles,
which represented the most likely distribution of neural activity, and we chose to retain the
length of the resulting current vector, i.e., the orientation of the current dipoles was not fixed
12 / 49
with relation to the cortical surface, and thus the brain activation was reported without reference
to orientation; (iv) the resulting minimum norm estimates of neural activity were noisenormalized at each spatial location, so as to reduce the location bias of the estimates (Dale et al.
2000), and the resulting time and location dependent values yielded dynamic statistical
parameter maps (dSPM), which provided information about the statistical reliability of the
estimated signal at each location with millisecond accuracy. dSPM values were the input for
spatiotemporal clustering tests. We performed the whole procedure with MNE-Python-based
Eelbrain package 0.22.1 (Brodbeck 2016). Source data were also down-sampled to one
observation every 5 ms, so as to achieve computational tractability in our spatiotemporal
permutation clustering tests (see Section 2.5.2).
2.5 Statistical analyses
2.5.1 Behavioral data
Behavioral data were analyzed with 2 x 2 x 2 repeated measures ANOVAs for both speed
and accuracy. As our hypotheses did not pertain to the behavioral data, these analyses simply
aimed to assess whether performance was roughly equal across conditions.
For accuracy, we first calculated the overall mean and standard deviation (SD) of correct
button press counts, and based on this, we excluded from further analyses the participant whose
overall accuracy was 3 SDs lower than the group mean. For the 20 participants whose data were
eventually included in the MEG analyses, the histogram of accuracies per condition per
participant was left-skewed, and there was a ‘ceiling’ effect at the upper end, suggesting that the
overall performance of participants was very good. Thus, accuracies were first rau-transformed
13 / 49
(Studebaker 1985, Studebaker et al. 1995, Sherbecoe and Studebaker 2004), and the 2 x 2 x 2
ANOVA was performed on the rau-transformed data.
Reaction times (RTs) were measured from the onset of picture presentation for each
correctly answered trial and for each participant.
2.5.2 Spatiotemporal clustering tests
To analyze the MEG data, we used spatiotemporal cluster-based permutation ANOVAs
and t-tests (Maris and Oostenveld 2007), testing for spatiotemporal clusters of brain activation
that showed statistically significant differences between our conditions, corrected for multiple
comparisons. Since at the determiner and the adjective, there was only one factor in play, i.e.,
polarity (or determiner type), simple t-tests were used for these two positions, but for all later
positions in the sentence, the full 2 x 2 x 2 ANOVA was used.
The procedure of spatiotemporal clustering tests. Cluster-based permutations were
used to address type I error (Maris and Oostenveld 2007). The specific steps of our
spatiotemporal cluster-based permutations were performed in Eelbrain 0.22.1 as follows: (i) for
each source at each time point (every 5 ms, see Section 2.4), an uncorrected t-test or ANOVA
was calculated; (ii) if a significant effect at a p-value of 0.05 (uncorrected) was observed in at
least 10 contiguous sources for at least 25 ms, these data points were treated as a cluster (also
called ‘exceedance mass’ in Nichols and Holmes 2002); (iii) within each cluster, all t- or Fvalues were summed up to form a test statistic for the cluster; (iv) steps (i) - (iii) were repeated
with 10,000 permutations of the dSPM data by randomly reassigning condition labels within
subjects, and test statistics were calculated from the clusters yielded by the permutations; (v) test
statistics from these 10,000 permutations formed the null distribution for the test statistics of the
14 / 49
clusters we were interested in (i.e., the clusters in step (ii)), and based on this null distribution,
each cluster of interest was assigned a corrected p-value (alpha < 0.05).
Area selected for cluster-based permutations. Though the LATL has been robustly
implicated in previous studies focusing on the processing of conceptual knowledge or
combinatorial stimuli (as opposed to unstructured word lists) (Mazoyer et al. 1993, Stowe et al.
1998, Humphries et al. 2001, 2005, 2006, Vandenberghe et al. 2002, Friederici and Kotz 2003,
Rogalsky and Hickok 2009, Bemis and Pylkkänen 2011, 2013a, 2013b, Brennan and Pylkkänen
2012), since our current study aimed to investigate the integrative mechanisms of composing
multiple kinds of information in sentence processing, we analyzed a much broader area in the
search for spatiotemporal clusters, covering other regions implicated for combinatory processing
of syntax or semantics: the ventromedial prefrontal cortex (vmPFC) (Pylkkänen and McElree
2007, Brennan and Pylkkänen 2008, 2010, Pylkkänen et al. 2009a, b), the left inferior frontal
gyrus (LIFG) (Hagoort et al. 2004, Grodzinsky and Santi 2008, Tesink et al. 2009, Pallier et al.
2011), the left posterior temporal lobe (LPTL) (Badre et al. 2005, Gitelman et al. 2005, Pallier et
al. 2011), and the angular gyrus (AG) (Binder et al. 1997, Binder and Desai 2011, Martin 2007,
Thompson et al. 2007, Lau et al. 2008, Pallier et al. 2011, Price et al. 2015, Boylan et al. 2015).
The area included in the search for spatiotemporal clusters is plotted in white on the cortical
surfaces in Figures 4 – 7. More specifically, in Desikan-Killiany Atlas (as implemented in
FreeSurfer, https://surfer.nmr.mgh.harvard.edu/fswiki/CorticalParcellation, Desikan et al. 2006),
it includes the frontal lobe (including insular cortex), the temporal lobe, the inferior parietal lobe
and supramarginal gyrus, and the cingulate cortex.
Time window selected for cluster-based permutations. Previous MEG studies using
two-word noun phrases as stimuli have found that phrasal-level compositional effects typically
15 / 49
peak around 200 – 250 ms after the onset of the second word (Bemis and Pylkkänen 2011,
2013a, 2013b, Westerlund and Pylkkänen 2014, Westerlund and Pylkkänen 2014, Zhang and
Pylkkänen 2015, 2018). Since the current study aimed to characterize the temporal progression
of specific semantic effects across a full sentence, and given that sentential context may alter
latencies observed in phrasal-level studies (see Brennan and Pylkkänen 2012, 2016), all the
spatiotemporal cluster-based permutation tests were performed on a time window from 0 ms to
600 ms after the onset of any given target word. If a test revealed significant clusters starting at
the onset of a word position, we re-performed the test with an extended time window: from 100
ms before to 600 ms after the onset of this position.
Permutation tests. We performed five tests (two-tailed t-tests at the determiner and
adjective positions, and three 2 x 2 x 2 ANOVAs at the noun, auxiliary and verb positions) to
investigate the temporal progression of processing conceptual knowledge and sentential polarity,
focusing especially on the verb position.
2.5.3 Post hoc single trial analysis
To further test whether our most important finding from the condition-level
spatiotemporal clustering tests (see Sections 2.5.2 and 3.2) – i.e., a significant cluster showing
interaction effects between sentential polarity and noun specificity during verb presentation
(Cluster 7 in Figure 7) – is also observable at the level of single trials as well as to address the
concern that lexical factors might have an influence, we designed a post hoc analysis of brain
activation at the level of single trials.
For each trial of each subject, dSPM values over the space and time within this
significant cluster were averaged and became the dependent variable in linear mixed-effects
16 / 49
models. It is worth noting that some basic methodological assumptions were different between
condition-level and trial-level analyses, thus at trial level, it is not necessary that it is those
spatiotemporal sources within the space and time of this significant cluster that actually
contributed to the elicited condition-level interaction effects. Thus, this post hoc single trial
analysis only aimed at providing a rough assessment of our most important finding: whether the
informativeness of subject DP predicted brain activation within the space and time of Cluster 7.
For simplicity, we divided the conditions into two groups: ‘DP of high informativeness’ and ‘DP
of low informativeness’. Given that in previous MEG studies with the paradigm ‘modifier + head
noun’, most robust effects of LATL activation were elicited by modifiers of high specificity
(Zhang and Pylkkänen 2015), we only labeled the most informative subject DPs – ‘[Pos, Spec,
+Adj]’ and ‘[Neg, Gen, -Adj]’ – as ‘DP of high informativeness’. We used the lme4 (Bates et al.
2015) and lmerTest (Kuznetsova et al. 2016) packages in R to fit a linear mixed-effects model
with a fixed effect of this DP informativeness, along with lexical factors of the noun (i.e., log
frequency, word length, number of morphemes, lexical decision reaction time, and naming
reaction time, all values taken from the English Lexical Project (Balota et al. 2007)) and the
bigram frequency and transition probability between the noun and the verb (these values were
calculated from COCA (Davies 2008)). We also included random intercepts for participants, and
word items (nouns and verbs) (Baayen et al. 2008). All continuous independent variables were
scaled in the R packages.
3. RESULTS
3.1 Behavioral data
17 / 49
The mean of overall accuracy for the 20 participants whose data were included in the
MEG analyses was 92% (SD = 4%). The 2 x 2 x 2 repeated measures ANOVA on rautransformed accuracies revealed a significant effect of determiner type (F(1,152) = 10.24; p =
0.0017). Positive sentences elicited somewhat higher accuracies (M = 93.75%, SD = 4.30%) than
negative sentences (M = 91.04%, SD = 6.34%). Responses to positive sentences were also faster
than those to negative sentences (F(1, 152) = 23.71; p < 0.0001). Mean accuracies and reaction
times per condition are summarized in Table 3. Since the measurements of reaction times and
accuracy are subjected to many task-related factors and thus cannot informatively reflect the
word-by-word compositional processing of sentential semantics by the brain, we only used
behavioral data to check whether participants paid attention to our stimuli.
3.2 Spatiotemporal clustering results
From the 2 permutation t-tests (at the determiner and adjective position) and the 3
permutation ANOVAs (at the noun, auxiliary, and verb position), we obtained 7 clusters that
were statistically significant or marginally significant (see Figures 4 – 7). Details of these
clusters are reported below.
Significant interaction effects between polarity and noun specificity during verb
presentation (see Cluster 7 in Figure 7). This cluster was elicited by a three-way ANOVA test at
270 – 400 ms (p = 0.22) after the onset of verb in left anterior temporal cortex as well as in
neighboring posterior and superior (LIFG) regions. This interaction was consistent with the
pattern predicted by the ‘shared processing’ hypothesis (see Figure 1): for positive sentences,
those containing specific nouns elicited more activity than those containing general nouns, while
for negative sentences, those containing general nouns elicited more activity.
18 / 49
Significant main effects of polarity during determiner and auxiliary presentation
(see Cluster 1 in Figure 4 and Cluster 4 in Figure 6). Cluster 1 was elicited by a two-tailed t-test
at 150 – 460 ms (p < 0.0001) after the onset of determiner, in almost the entire analyzed brain
area (including most parts of the left frontal and temporal lobes, the AG, and the left cingulate
cortex). Cluster 4 was elicited by a three-way ANOVA test with an extended time window: it
took place from 15 ms before to 140 ms after the onset of auxiliary (p = 0.047) in the left
temporal pole and the vmPFC. Both clusters exhibited increased activity for negative over
positive conditions.
Significant main effects of adjectival modification during noun presentation (see
Cluster 2 in Figure 5). This cluster was elicited by a three-way ANOVA test with an extended
time window: it took place from 95 ms before to 130 ms after the onset of noun (p = 0.0004) in
the LATL, the left mid-temporal cortex (LMTC), Broca’s area, and the vmPFC. It exhibited
increased activity for conditions containing adjectival modifiers.
Significant three-way interaction effects during auxiliary presentation (see Cluster 6
in Figure 6). This cluster was elicited at 80 – 230 ms (p = 0.013) after the onset of auxiliary, in
most of the temporal lobe and the LIFG. Apart from suggesting a critical time point for
integrating the subject meaning, the pattern of this three-way interaction was complicated and
not obviously interpretable.
Non-significant trends of noun specificity during noun and auxiliary presentation
(see Cluster 3 in Figure 5 and Cluster 5 in Figure 6). These two trends were elicited by three-way
ANOVA tests at 65 – 165 ms (p = 0.102) and 55 – 165 ms (p = 0.079) after the onset of noun
and auxiliary respectively. Within Cluster 3, increased amplitudes were elicited for specific over
general nouns in the left temporal pole and the vmPFC, however, within Cluster 5, more activity
19 / 49
was elicited for conditions with general nouns than for those with specific nouns in the left AG
and the left posterior temporal lobe.
No other significant clusters or non-significant trends were observed.
3.3 Results of single trial analysis
As shown in Table 4, in our linear mixed-effects model, none of the lexical factors of the
noun in a trial significantly predicted brain activation, and neither did the bigram frequency or
the transition probability between the noun and the verb in a trial. However, DP informativeness
was indeed a significant factor (p = 0.018): sentences containing a subject DP of high
informativeness elicited larger brain activation within the space and time of Cluster 7. Thus,
these results of single trial analysis were consistent with our main finding from the
spatiotemporal permutation ANOVA during verb presentation.
4. DISCUSSION
4.1 Interaction between polarity and conceptual knowledge
In this work, we investigated how the brain composes conceptual with logical information in
sentence processing, with a primary focus on the possible sensitivity of left anterior temporal
cortex to the impact of negation in reversing specificity relations.
We crossed negation with a manipulation of conceptual specificity inside the subject
position of sentences and found that when the subjects combined with their subsequently
presented verbs, a fronto-temporal network covering the left anterior temporal cortex and
adjacent regions showed an interaction between negation and the conceptual specificity of the
head noun. With positive determiners, conceptually more specific nouns elicited larger
20 / 49
amplitudes at the verb (a lizard > a reptile), but with negative determiners, the opposite pattern
was observed (no lizard < no reptile). These findings deepen our understanding of the interplay
between composition and information specificity, showing that in terms of incremental
computation of informativeness, the processing of negative and positive sentences roughly
follows the same temporal profile, and that left anterior temporal and surrounding cortices
represent specificity as calculated on the basis of both conceptual and logical elements.
While the sentence final interaction is consistent with prior effects of specificity in
minimal phrases (Westerlund and Pylkkänen, 2014; Zhang and Pylkkänen, 2015, 2018), the
spatiotemporal details of the current result diverged to some extent from previously reported
LATL effects on minimal two-word noun phrases. First, in terms of time course, the interaction
on the verb occurred at 270 – 400 ms after verb onset, while in previous studies, compositional
effects have been elicited at about 200 – 250 ms after the onset of the head noun. Likely, the
slightly later timing of the current interaction may relate to the increased complexity of stimuli.
At the verb, a full DP was composed with it. This is consistent with prior sentence-level data,
which have also shown that in a sentential context, LATL effects occur later than for minimal
phrases (Brennan and Pylkkänen 2012, 2016). Second, in terms of region, our interaction effects
implicated not only the LATL but also part of the LIFG/Broca’s area and the posterior temporal
lobe. The implication of the posterior temporal lobe in processing meanings of high-specificity
has also been reported in previous patient studies (see Mummery et al. 2000). The implication of
Broca’s area might be due to increased complexity of stimuli and increased working memory
demands (see Paulesu et al. 1993, Cohen et al. 1997, Rogalsky et al. 2008), since the semantic
computation at the end of the sentence requires integration of all previously encountered
information, and the task of integrating a subject bearing more specific meaning can be
21 / 49
computationally more complicated than integrating a single-word modifier in processing
minimal two-word noun phrases.
We might have also expected to see that the presence or absence of adjectival modifiers
would factor into the interaction between conceptual specificity and polarity as well. This was
not observed in our spatiotemporal permutation test at the verb position, but, to a certain extent,
reflected by our single trial analysis: higher activation was most robustly elicited by positive
sentences containing nouns of high specificity and adjectival modification as well as negative
sentences containing nouns of low specificity and no adjectival modification. In our design, it
seems likely that the contributions of the head noun and the adjective to the overall
informativeness of a subject DP are not linearly additive, and the head nouns might have been a
stronger modulator of specificity effects than the adjectives. Future research should delve deeper
into the possibly distinct contributions of adjectival and nominal meanings to the representations
of conceptual specificity at the sentence level.
Overall, our findings contribute to our understanding of the function of the LATL and
surrounding fronto-temporal cortex in semantic processing by showing that a shared mechanism
housed in this brain network underlies the processing of logical and conceptual meanings.
Essentially, this mechanism appears to track the overall informativeness of the expression being
composed. This type of LATL sensitivity to information encoded in non-lexical categories (i.e.,
encoded in logical operators or other types of functional words) was also recently reported by
Zhang and Pylkkänen (2018), who investigated LATL composition effects for noun phrases such
as same/different star, which in certain contexts may result in more specific meanings than noun
phrases with lexical adjectives such as green star. If meaning specificity is what matters for the
LATL even if the modifier is non-lexical, then same/different star should elicit at least as much
22 / 49
combinatory activity in the LATL as green star. This is exactly what was found. Thus together,
these findings suggest that the LATL and surrounding cortex may be a general combinatory
processor that is sensitive to informativeness in meaning composition, across both conceptual or
logical items.
4.2 Main effects of polarity
Our design employed word-by-word presentation of sentences that started with the word
indicating sentential polarity, and our results revealed two significant clusters showing main
effects of polarity. At the initial presentation of polarity information, with no prior context, the
entire analyzed cortical area was more activated by the negative determiner no than positive
determiner a for a sustained period (150 – 460 ms after determiner onset). No polarity effects
were elicited during either the adjective or noun. Then, from -15 to 140 ms in the auxiliary
position, polarity effects reappeared in the vmPFC and the left temporal pole, showing again the
pattern that negative conditions elicited more activity than positive conditions.
The lack of main effects of polarity during adjective or noun presentation seems
semantically reasonable. Notice that for a negative event-describing sentence, say no green
lizard is sleeping, the existence of green lizards is not negated, but the existence of sleeping
events with green lizards as sleepers is negated – there can be a green lizard which is eating or
playing. Thus it seems that negation did not affect the processing of conceptual content within
subject DPs, but its meaning was retrieved at a later stage when the semantics of a whole subject
DP was finalized before being integrated with the meaning of verbal predicate.
4.3 Effects of adjectival modification and noun specificity
23 / 49
Our design also included an adjectival modification factor, with half of the sentential subjects
containing an adjective and the other half not. This constituted a bridge to prior studies of
adjectival modification within two-word noun phrases, which have consistently shown LATL
compositional effects at about 200 – 250 ms after the onset of head noun. However, as we have
mentioned earlier (in Section 2.2), given that in the current design, an adjective was always fully
predictive of an upcoming noun whereas the determiner a/no could be followed by either a noun
or an adjective, we were aware that LATL compositional effects elicited with minimal two-word
noun phrases were likely not to be exactly replicated. Indeed, modification effects elicited in the
current design covered a broader area than the LATL, extending to left middle temporal, inferior
frontal and ventromedial prefrontal cortices. Further, the timing of these effects was substantially
earlier, starting at 95 ms before noun onset and lasting for over 225ms.
Since adjectives were fully predictive of an upcoming noun, the observed spatiotemporal
differences are likely to reflect anticipatory phrase construction and/or conceptual combination
between the adjective and noun. A similarly timed effect has been previously observed in an
EEG study on basic phrasal composition (Neufeld et al., 2016). However, firm conclusions on
this kind of anticipatory composition should wait for more evidence.
In our current experiment, we also elicited subtle trends of noun specificity, again
suggesting that access-related activity only subtly modulates LATL activity at most (see also
Westerlund and Pylkkänen 2014, Zhang and Pylkkänen 2015). However, this trend of noun
specificity was very early (65 ms after noun onset) and rather widespread, including both the left
temporal pole and the vmPFC. A possible reason for this might be anticipation and repetition
priming: each noun appeared 8 times in the whole experiment, and each adjective was used to
combine with 6 distinct nouns (3 pairs) only. It is likely that anticipation helped move the time
24 / 49
window of the trend earlier, and this kind of repetition would cause priming effects, making the
trend of noun specificity effects appear early, stronger, as well as implicating the vmPFC, given
that the vmPFC has been implicated in repetition effects in hemodynamic studies (Henson and
Rugg 2003) and animal studies (Rainer and Miller 2000).
Finally, in this experiment, the auxiliary is was perhaps the least interesting item in each
trial, carrying little meaning. Nevertheless, during auxiliary presentation, we found a trend of
noun specificity effects in the AG and in left posterior temporal cortex, and a significant
interaction among all three factors in the left temporal lobe and part of LIFG. Potentially, it is
during auxiliary presentation that the projection of the full sentential structure was initiated,
including the retrieval of negation and analysis of the relation between sentential polarity and
conceptual content. Without a detailed understanding of all these complex processes, it is rather
hard to disentangle these effects, but these patterns can nevertheless be used to inform future
investigations of the interaction of various interpretive processing in the course of sentence
comprehension.
5. CONCLUSION
Our findings offer an initial spatiotemporal characterization of how multiple semantic factors
interact in the course of sentence interpretation. We show that a set of fronto-temporal regions
responds more strongly during the integration of more informative linguistic material, with
informativeness being calculated both on the basis of concept denoting and logical words.
Importantly, this work offers a demonstration of how results from tightly controlled two-word
studies scale up to the next level, i.e., a full sentence, with the sentential context clearly affecting
both the timing and spatial extent of the relevant composition effects.
25 / 49
REFERENCES
Adachi, Y., M. Shimogawara, M. Higuchi, Y. Haruta, and M. Ochiai. (2001). Reduction of nonperiodic environmental magnetic noise in MEG measurement by continuously adjusted least
squares method. IEEE Transactions on Applied Superconductivity 11: 669–672.
Baayen, R.H., Davidson, D.J., Bates, D.M., 2008. Mixed-effects modeling with crossed random
effects for subjects and items. Journal of Memory and Language. 59 (4), 390–412.
Badre, David, Russell A. Poldrack, E. Juliana Paré-Blagoev, Rachel Z. Insler, and Anthony D.
Wagner. (2005). Dissociable controlled retrieval and generalized selection mechanisms in
ventrolateral prefrontal cortex. Neuron 47: 907–918.
Balota, David A., Melvin J. Yap, Michael J. Cortese, Keith A. Hutchison, Brett Kessler, Bjorn
Loftis, James H. Neely, Douglas L. Nelson, Greg B. Simpson, and Rebecca Treiman. (2007).
The English Lexicon Project. Behavior Research Methods 39 (3): 445–459.
Baron, Sean G., and Daniel Osherson. (2011). Evidence for conceptual combination in the left
anterior temporal lobe. NeuroImage 55: 1847–1852. Doi: 10.1016/j.neuroimage.2011.01.066.
ates, D.,
a chler, M., Bolker, B., Walker, S., 2015. Fitting linear mixed-effects models using
lme4. Journal of Statistical Software. 67 (1), 1–48.
Bemis, Douglas K., and Liina Pylkkänen. (2011). Simple composition: A
magnetoencephalography investigation into the comprehension of minimal linguistic phrases.
The Journal of Neuroscience 31 (8): 2801–2814.
Bemis, Douglas K., and Liina Pylkkänen. (2013a). Basic linguistic composition recruits the left
anterior temporal lobe and left angular gyrus during both listening and reading. Cerebral Cortex
23 (8): 1859–1973.
26 / 49
Bemis, Douglas K., and Liina Pylkkänen. (2013b). Flexible composition: MEG evidence for the
deployment of basic combinatorial linguistic mechanisms in response to task demands. PLoS
One 8 (9): e73949.
Binder, Jeffrey R., Julie A. Frost, Thomas A. Hammeke, Robert W. Cox, Stephen M. Rao, and
Thomas Prieto. (1997). Human brain language areas identified by functional magnetic resonance
imaging. The Journal of Neuroscience 17: 353–362.
Binder, Jeffrey R., and Rutvik H. Desai. (2011). The neurobiology of semantic memory. Trends
in Cognitive Sciences 15 (11): 527–536.
Binney, R.J., Embleton, K.V., Jefferies, E., Parker, G.J., Ralph, M.A., (2010). The ventral and
inferolateral aspects of the anterior temporal lobe are crucial in semantic memory: evidence from
a novel direct comparison of distortion-corrected fMRI, rTMS, and semantic dementia. Cereb.
Cortex 20, 2728–2738.
Boylan, Christine, John C. Trueswell, and Sharon L. Thompson-Schill. (2015). Compositionality
and the angular gyrus: A multi-voxel similarity analysis of the semantic composition of nouns
and verbs. Neuropsychologia 78: 130–141.
Brainard, David H. (1997). The Psychophysics Toolbox. Spatial Vision 10: 433–436.
Brennan, Jonathan, and Liina Pylkkänen. (2008). Processing events: Behavioral and neuromagnetic correlates of aspectual coercion. Brain and Language 106: 132–143.
Brennan, Jonathan, and Liina Pylkkänen. (2010). Processing psych verbs: Behavioural and MEG
measures of two different types of semantic complexity. Language and Cognitive Processes 25:
777–807.
Brennan, Jonathan, and Liina Pylkkänen. (2012). The time-course and spatial distribution of
brain activity associated with sentence processing. NeuroImage 60: 1139–1148.
27 / 49
Brennan, Jonathan and Liina Pylkkänen. (2016). MEG Evidence for Incremental Sentence
Composition in the Anterior Temporal Lobe. Cognitive Science. DOI: 10.1111/cogs.12445.
Brodbeck, Christian. (2016). Eelbrain 0.22.1 (Computer software).
http://pythonhosted.org/eelbrain (Also available at PYPI (https://pypi.org/project/eelbrain/0.22.1)
or GitHub (https://github.com/christianbrodbeck/Eelbrain/tree/r/0.22)).
Carpenter, P. A., and Marcel A. Just (1975). Sentence comprehension: A psycholinguistic
processing model of verification. Psychological Review, 82(1), 45–73.
Christensen, Ken Ramshøj. (2009). Negative and affirmative sentences increase activation in
different areas in the brain. Journal of Neurolinguistics 22: 1–17.
Clark, Herbert H., and William G. Chase (1972). On the process of comparing sentences against
pictures. Cognitive Psychology, 3(3), 472–517.
Cohen, Jonathan D., William M. Perlstein, Todd S. Braver, Leigh E. Nystrom, Douglas C. Noll,
John Jonides, Edward E. Smith, et al. (1997). Temporal dynamics of brain activation during a
working memory task. Nature 386: 604–608.
Cornish, Elizabeth, and P. C. Wason, (1970). The recall of affirmative and negative sentences in
an incidental learning task. The Quarterly Journal of Experimental Psychology, 22(2), 109– 114.
Dale, Anders M., Arthur K. Liu, Bruce R. Fischl, Randy L. Buckner, John W. Belliveau, Jeffrey
D. Lewine, and Eric Halgren. (2000). Dynamic statistical parametric mapping: combining fMRI
and MEG for high-resolution imaging of cortical activity. Neuron 26: 55–67.
Davies, Mark. (2008). The Corpus of Contemporary American English: 450 Million Words,
1990–2012.
28 / 49
Del Prato, Paul, and Liina Pylkkänen. (2014). MEG evidence for conceptual combination but not
numeral quantification in the left anterior temporal lobe during language production. Frontiers in
Psychology 5: Article 524. Doi: 10.3389/fpsyg.2014.00524.
Desikan, Rahul S., Florent Ségonne, Bruce Fischl, Brian T. Quinn, Bradford C. Dicker- son,
Deborah Blacker, Randy L. Buckner, Anders M. Dale, R. Paul Maguire, Bradley T. Hyman,
Marilyn S. Albert, and Ronald J. Killiany. (2006). An automated labeling system for subdividing
the human cerebral cortex on MRI scans into gyral based regions of interest. NeuroImage 31:
968–980.
Evans, Jonathan St.B.T., John Clibbens, and Benjamin Rood. (1996). The Role of Implicit and
Explicit Negation in Conditional Reasoning Bias. Journal of Memory and Language, 35(3), 392–
409.
Fedorenko, Evelina, Terri L. Scott, Peter Brunner, William G. Coon, Brianna Pritchett, Gerwin
Schalk, and Nancy Kanwisher. (2016). Neural correlate of the construction of sentence meaning.
Proceedings of the National Academy of Sciences 113 (41): E6256–E6262.
Friederici, Angela D., and Sonja A. Kotz. (2003). The brain basis of syntactic processes:
Functional imaging and lesion studies. NeuroImage 20: S8–S17.
Gitelman, Darren R., Anna C. Nobre, Sreepadma Sonty, Todd B. Parrish, and M-Marsel
Mesulam. (2005). Language network specializations: An analysis with parallel task designs and
functional magnetic resonance imaging. NeuroImage 26: 975–985.
Gramfort, Alexandre, Martin Luessi, Eric Larson, Denis A. Engemann, Daniel Strohmeier,
Christian Brodbeck, Roman Goj, Mainak Jas, Teon Brooks, Lauri Parkkonen, and Matti S.
Hämäläinen. (2013). MEG and EEG data analysis with MNE-Python. Frontiers in Neuroscience
7: Article: 267.
29 / 49
Gramfort, Alexandre, Martin Luessi, Eric Larson, Denis A. Engemann, Daniel Strohmeier,
Christian Brodbeck, Lauri Parkkonen, and Matti S. Hämäläinen. (2014). MNE software for
processing MEG and EEG data. NeuroImage 86: 446–460.
Grodzinsky, Yosef, and Andrea Santi. (2008). The battle for Broca’s region. Trends in cognitive
sciences 12: 474–480.
Hagoort, Peter, Lea Hald, Marcel Bastiaansen, and Karl Magnus Petersson. (2004). Integration
of word meaning and world knowledge in language comprehension. Science 304: 438–441.
Hämäläinen, Matti S., and Risto J. Ilmoniemi. (1994). Interpreting magnetic fields of the brain:
minimum norm estimates. Medical & Biological engineering & Computing 32: 35–42.
Henson, R. N. A., and M. D. Rugg. (2003). Neural response suppression, haemodynamic
repetition effects, and behavioural priming. Neuropsychologia 41: 263–270.
Hodges, J.R., Patterson, K., Oxbury, S., Funnell, E., (1992). Semantic dementia. Progressive
fluent aphasia with temporal lobe atrophy. Brain 115, 1783–1806.
Howard, James H. (1975). The attentional demands of negation in a memory-scanning task.
Memory & Cognition 3(3), 319–324.
Humphries, Colin, Kimberley Willard, Bradley Buchsbaum, and Gregory Hickok. (2001). Role
of anterior temporal cortex in auditory sentence comprehension: an fMRI study. NeuroReport 12
(8): 1749–1752.
Humphries, Colin, Tracy Love, David Swinney, and Gregory Hickok. (2005). Response of
anterior temporal cortex to syntactic and prosodic manipulations during sentence processing.
Human Brain Mapping 26: 128–138.
30 / 49
Humphries, Colin, Jeffrey R. Binder, David Medler, and Einat Liebenthal. (2006). Syntactic and
semantic modulation of neural activity during auditory sentence comprehension. Journal of
Cognitive Neuroscience 18 (4): 665–679.
Kuznetsova, A., Brockhoff, P.B., Christensen, R.H.B., 2016. Package lmerTest.
Lau, Ellen F., Colin Phillips, and David Poeppel. (2008). A cortical network for semantics:
(de)constructing the N400. Nature Reviews Neuroscience 9: 920–933.
Maris, Eric, and Robert Oostenveld. (2007). Nonparametric statistical testing of EEG- and MEGdata. Journal of Neuroscience Methods 164: 177–190.
Martin, Alex. (2007). The representation of object concepts in the brain. Annual Review of
Psychology 58: 25–45.
Mayo, Ruth, Yaacov Schul, and Eugene Burnstein. (2004). “I am not guilty” vs “I am innocent”:
Successful negation may depend on the schema used for its encoding1. Journal of Experimental
Social Psychology, 40(4), 433–449.
Mazoyer, Bernard M., Nathalie Tzourio, Victor Frak, Andre Syrota, Nina Murayama, Olivier
Levrier, Georges Salamon, Stanislas Dehaene, Laurent Cohen, and Jacques Mehler. (1993). The
cortical representation of speech. Journal of Cognitive Neuroscience 5 (4): 467–479.
Miller, George A. (1995). WordNet: a lexical database for English. Communications of the ACM
38 (11): 39–41.
Miller, George A., Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine J.
Miller. (1990). Introduction to WordNet: An on-line lexical database. International Journal of
Lexicography 3 (4): 235–244.
Mosher, John C., Richard M. Leahy, and Paul S. Lewis. (1999). EEG and MEG: forward
solutions for inverse methods. IEEE Transactions on Biomedical Engineering 46 (3): 245–259.
31 / 49
Mummery, C. J., Patterson, K., Price, C. J., Ashburner, J., Frackowiak, R. S. J. and Hodges, J.
R., (2000). A voxel-based morphometry study of semantic dementia: relationship between
temporal lobe atrophy and semantic memory. Annals of neurology, 47(1), pp.36–45.
Neufeld, Chris, Stephanie E. Kramer, Natalia N. Lapinskaya, Christopher C. Heffner, Anton
Malko, and Ellen F. Lau. (2016). The electrophysiology of basic phrase building. PloS One
11(10): e0158446.
Nichols, Thomas E, and Andrew P. Holmes. (2002). Nonparametric permutation tests for
functional neuroimaging: A primer with examples. Human brain mapping 15: 1–25.
Nieuwland, Mante. S., and Gina R. Kuperberg. (2008). When the truth isn’t too hard to handle:
An event-related potential study on the pragmatics of negation. Psychological Science 19 (12):
1213–1218.
Oaksford, Mike, and Keith Stenning. (1992). Reasoning with conditionals containing negated
constituents. Journal of Experimental Psychology: Learning, Memory and Cognition, 18(4),
835–854.
Pallier, Christophe, Anne-Dominique Devauchelle, and Stanislas Dehaene. (2011). Cortical
representation of the constituent structure of sentences. Proceedings of the National Academy of
Sciences 108 (6): 2522–2527
Papeo, Liuba, Jean-Remy Hochmann, and Lorella Battelli. (2016). The Default Computation of
Negated Meanings. Journal of Cognitive Neuroscience (early access): 1–7. Doi:
110.1162/jocn_a_01016.
Paulesu, Eraldo, Christopher D. Frith, and Richard S. J. Frackowiak. (1993). The neural
correlates of the verbal component of working memory. Nature 362: 342–345.
32 / 49
Pelli, Denis G. (1997). The VideoToolbox software for visual psychophysics: Transforming
numbers into movies. Spatial Vision 10 (4): 437 –442.
Prado, Jérôme, and Ira A. Noveck. (2006). How reaction time measures elucidate the matching
bias and the way negations are processed. Thinking & Reasoning, 12(3), 309–328.
Price, Amy R., Michael F. Bonner, Jonathan E. Peelle, and Murray Grossman. (2015).
Converging evidence for the neuroanatomic basis of combinatorial semantics in the angular
gyrus. The Journal of Neuroscience 35: 3276–3284.
Pylkkänen, Liina, and Brian McElree. (2007). An MEG study of silent meaning. Journal of
Cognitive Neuroscience 19: 1905–1921.
Pylkkänen, Liina, Andrea E. Martin, Brian McElree, and Andrew Smart. (2009a). The anterior
midline field: Coercion or decision making? Brain and Language 108: 184–190.
Pylkkänen, Liina, Bridget Oliveri, and Andrew J. Smart. (2009b). Semantics vs. world
knowledge in prefrontal cortex. Language and cognitive processes 24: 1313–1314.
Rainer, Gregor, and Earl K. Miller. (2000). Effects of visual experience on the representation of
objects in the prefrontal cortex. Neuron 27: 179–189.
Reichle, Erik D., Patricia A. Carpenter, and Marcel A. Just. (2000). The neural bases of strategy
and skill in sentence-picture verification. Cognitive Psychology, 40(4), 261–95.
Rogalsky, Corianne, and Gregory Hickok. (2009). Selective attention to semantic and syn- tactic
features modulates sentence processing networks in anterior temporal cortex. Cerebral Cortex
19: 786–796.
Rogalsky, Corianne, William Matchin, and Gregory Hickok. (2008). Broca’s area, sentence
comprehension, and working memory: An fMRI study. Frontiers in human neuroscience 2: 14.
33 / 49
Rogers, T.T., Patterson, K., (2007). Object categorization: Reversals and explanations of the
basic-level advantage. J. Exp. Psychol. Gen. 136, 451–469.
Sherbecoe, Robert L., and Gerald A. Studebaker. (2004). Supplementary formulas and tables for
calculating and interconverting speech recognition scores in transformed arcsine units.
International Journal of Audiology 43(8): 442–448.
Staab, Jenny, Thomas P. Urbach, and Marta Kutas. (2009). Negation processing in context is not
(always) delayed. Center for Research in Language Technical Reports 20 (3): 3–34.
Stowe, Laurie A., Cees A. J. Broere, Anne M. J. Paans, Albertus A. Wijers, Gijsbertus Mulder,
Wim Vaalburg, and Frans Zwarts. (1998). Localizing components of a complex task: Sentence
processing and working memory. NeuroReport 9: 2995–2999.
Studebaker, Gerald A. (1985). A ‘rationalized’ arcsine transform. Journal of Speech, Language,
and Hearing Research 28(3): 455–462.
Studebaker, Gerald A., D. Michael McDaniel, and Robert L. Sherbecoe. (1995). Evaluating
relative speech recognition performance using the proficiency factor and rationalized arcsine
differences. Journal of the American Academy of Audiology 6(2):173–182.
Tesink, Cathelijne M. J. Y., J. K. Buitelaar, Karl Magnus Petersson, R. J. Van der Gaag, C. C.
Kan, I. Tendolkar, and Peter Hagoort. (2009). Neural correlates of pragmatic language
comprehension in autism spectrum disorders. Brain 132.7: 1941–1952.
Tettamanti, Marco, Rosa Manenti, Pasquale A. Della Rosa, Andrea Falini, Daniela Perani,
Stefano F. Cappa, and Andrea Moro. (2008). Negation in the brain: Modulating action
representations. NeuroImage 43: 358–367.
34 / 49
Thompson, Cynthia K., Borna Bonakdarpour, Stephen C. Fix, Henrike K. Blumenfeld, Todd B.
Parrish, Darren R. Gitelman, and M-Marsel Mesulam. (2007). Neural correlates of verb
argument structure processing. Journal of Cognitive Neuroscience 19: 1753–1767.
Tomasino, Barbara, Peter H. Weiss, and Gereon R. Fink. (2010). To move or not to move:
Imperatives modulate action-related verb processing in the motor system. Neuroscience 169:
246–258.
Vandenberghe, Rik, Anna C. Nobre, and C. J. Price. (2002). The response of left temporal cortex
to sentences. Journal of Cognitive Neuroscience 14 (4): 550–560.
Wason, P. C. (1961). Response to affirmative and negative binary statements. British Journal of
Psychology, 52, 133–142.
Westerlund, Masha, and Liina Pylkkänen. (2014). The role of the left anterior temporal lobe in
semantic composition vs. semantic memory. Neuropsychologia 57: 59–70.
Westerlund, Masha, Itamar Kastner, Meera Al Kaabi, and Liina Pylkkänen. (2015). The LATL
as locus of composition: MEG evidence from English and Arabic. Brain and Language 141:
124–134.
Zhang, Linmin, and Liina Pylkkänen. (2015). The interplay of composition and concept
specificity in the left anterior temporal lobe: An MEG study. NeuroImage 111: 228–240.
Zhang, Linmin, and Liina Pylkkänen. (2018). Composing lexical vs. functional adjectives:
Evidence for uniformity in the left temporal lobe. Psychonomic Bulletin & Review. doi:
https://doi.org/10.3758/s13423-018-1469-y.
Ziegler, Jayden, and Liina Pylkkänen. (2016). Scalar adjectives and the temporal unfolding of
semantic composition: An MEG investigation. Neuropsychologia 89: 161–171.
35 / 49
LEGENDS
Table 1: Stimuli: the nouns used in this study. The percentages indicate the results of the
norming test performed on Amazon Mechanical Turk: the proportion of participants (N = 95)
whose judgments conformed to the specificity relation of a word pair as defined in WordNet
Search 3.1 (http:// wordnet.princeton.edu/).
Table 2: Statistics of the stimuli. General and specific nouns were matched for log frequency,
length, number of phonemes, syllables, and morphemes, and lexical decision and naming times
(data from the English Lexicon Project). Phrases of the patterns ‘adjective + general/specific
noun’ or ‘general/specific noun + verb’ were matched for bigram frequency and transition
probability (data from COCA).
Table 3: Behavioral data: Means and SDs of accuracy (%) and reaction times (second) for each
of the 8 conditions.
Table 4: Results of single trial analysis.
Figure 1: The predicted patterns of LATL effects in processing positive and negative sentences.
Above shows the prediction of the ‘conceptual-only’ hypothesis: the same pattern holds for
positive and negative sentences, and those containing specific nouns should elicit more brain
activity than those containing general nouns. Below shows the prediction of the ‘sharedprocessing’ hypothesis: positive and negative sentences elicit opposite patterns.
Figure 2: Experimental design. We varied determiner type (a vs. no), noun specificity, and the
presence or absence of an adjectival modifier to create sentence subjects and combine these
subjects with verbs in progressive tense to create sentences describing ongoing events.
Figure 3: Trial structure.
36 / 49
Figure 4: The results of spatiotemporal permutation t-test during determiner presentation.
Cluster 1 (p < 0.0001) exhibits significantly increased brain activation for negative determiner no
over positive determiner a from 150 to 460 ms after determiner onset in most parts of the left
frontal and temporal lobes, the AG, and the left cingulate cortex.
Figure 5: The results of 3-way spatiotemporal permutation ANOVA during noun presentation.
On the left, Cluster 2 shows significant main effects of adjectival modification (p = 0.0004):
nouns with adjectival modification elicited higher brain activity from 95 ms before to 130 ms
after noun onset in the LATL, the left mid-temporal cortex, roca’s area, and the vmPFC.
On the right, Cluster 3 shows a non-significant trend of noun specificity (p = 0.102): nouns of
high-specificity elicited higher brain activity from 65 to 165 ms after noun onset in the left
temporal pole and the vmPFC.
Figure 6: The results of 3-way spatiotemporal permutation ANOVA during auxiliary
presentation.
On the left, Cluster 4 shows significant main effects of polarity (p = 0.047): negative sentences
elicited higher brain activity from 15 ms before to 140 ms after auxiliary onset in the left
temporal pole and the vmPFC.
In the middle, Cluster 5 shows a trend of noun specificity (p = 0.079): sentences with nouns of
low-specificity elicited higher brain activity from 55 to 165 ms after auxiliary onset in the left
AG and the left posterior temporal lobe.
On the right: Cluster 6 shows significant 3-way interaction effects (p = 0.013) from 80 to 230
after auxiliary onset in most of the temporal lobe and the LIFG.
Figure 7: The results of 3-way spatiotemporal permutation ANOVA during verb presentation.
37 / 49
Cluster 7 shows significant interaction effects between sentential polarity and noun specificity (p
= 0.022): sentences with higher DP informativeness (i.e., positive sentences with nouns of highspecificity, and negative sentences with nouns of low-specificity) elicited increased brain
activation from 270 to 400 ms after verb onset in left anterior temporal cortex and neighboring
posterior and superior (LIFG) regions.
Colors should be used for all figures in print.
Table 1: Stimuli: the nouns used in this study.
Nouns: General / Specific
%
Nouns: General / Specific
%
beast / wolf
100%
primate / monkey
97%
vessel / submarine
100%
canine / poodle
100%
kid / toddler
97%
servant / maid
99%
boat / canoe
98%
garment / shirt
100%
vehicle / truck
100%
leader / president
100%
scholar / biologist
100%
technician / programmer
93%
ball / marble
99%
creature / dragon
100%
machine / clock
100%
bag / backpack
99%
container / bottle
99%
reptile / lizard
100%
musician / drummer
99%
performer / singer
99%
scientist / physicist
97%
bird / robin
100%
38 / 49
Nouns: General / Specific
%
Nouns: General / Specific
%
device / phone
99%
fish / salmon
100%
flower / rose
100%
mammal / horse
100%
plane / helicopter
100%
fowl / chicken
96%
plant / lily
100%
toy / balloon
99%
Table 2: Statistics of the stimuli.
Num
General nouns
mean (SD)
Specific nouns
mean (SD)
p-value of
two-tailed ttests
ELP log frequency
30
8.99 (1.37)
8.84 (1.29)
0.6493
length
30
6.1 (1.9)
6.4 (1.8)
0.5368
Num. of phonemes
30
4.6 (1.7)
5.2 (1.8)
0.1893
Num. of syllables
30
1.8 (0.7)
2.0 (0.85)
0.1971
Num. of morphemes
30
1.3 (0.5)
1.4 (0.6)
0.4836
Lexical decision RT
30
654.5 (69.8)
653.3 (68.6)
0.9451
Naming RT
30
633.3 (50.4)
634.8 (68.9)
0.9247
39 / 49
Num
General nouns
mean (SD)
Specific nouns
mean (SD)
p-value of
two-tailed ttests
Adj.-Noun bigram
frequency
30
29 (85.5)
52.4 (104.9)
0.3467
Adj.-Noun transition
probability
30
0.0004106
(0.0007355)
0.0006841
(0.001234)
0.3014
Noun-Verb bigram
frequency
60
36.9 (106.0)
19 (56.5)
0.2503
Noun-Verb transition
probability
60
0.0008742
(0.001351)
0.001148
(0.001401)
0.2774
Table 3: Behavioral data: Means and SDs of accuracy (%) and reaction times (second).
Specific nouns
Positive
General nouns
+ Adj
- Adj
+ Adj
- Adj
95.17% (3.82%)
1.353 (0.410)
93.5% (3.78%)
1.359 (0.368)
92.5% (5.31%)
1.332 (0.360)
93.83% (3.98%)
1.316 (0.375)
40 / 49
Specific nouns
Negative
General nouns
+ Adj
- Adj
+ Adj
- Adj
90.08% (5.91%)
1.728 (0.469)
91.92% (6.03%)
1.612 (0.453)
89.92% (7.08%)
1.694 (0.482)
92.25% (6.43%)
1.596 (0.392)
Table 4: Results of single trial analysis.
Intercept
DP
Informativeness
(Low)
Log frequency
of the noun
Length
of the noun
Num. of
morphemes of
the noun
Lexical
decision RT of
the noun
Naming RT of
the noun
Noun-verb
bigram
frequency
Noun-verb
transition
probability
* < 0.05
*** < 0.001
Estimate
1.196
Std. Err.
0.0219
df
24.60
t
54.614
p
< 0.001 ***
-0.02272
0.009608
6497
-2.365
0.0181 *
0.006222
0.006263
131.5
0.993
0.3223
-0.003060
0.007057
533.6
-0.434
0.6647
0.001858
0.006413
260.9
0.290
0.7723
0.005564
0.007247
188.9
0.768
0.4436
0.004111
0.005704
694.3
0.721
0.4714
-0.005614
0.005376
505.7
-1.044
0.2969
-0.001100
0.005400
227.9
-0.204
0.8388
41 / 49
Figure 1: The predicted patterns of LATL effects in processing positive and negative sentences.
42 / 49
Figure 2: Experimental design.
Figure 3: Trial structure.
43 / 49
44 / 49
Figure 4: The results of spatiotemporal permutation t-test during determiner presentation.
45 / 49
Figure 5: The results of 3-way spatiotemporal permutation ANOVA during noun presentation.
46 / 49
Figure 6: The results of 3-way spatiotemporal permutation ANOVA during auxiliary
presentation.
47 / 49
Figure 7: The results of 3- way spatiotemporal permutation ANOVA during verb presentation.
48 / 49
Highlights:

Are conceptual specificity and negation parallel information sources for the brain?

To address this, we studied the composition of a/no dog/poodle with a verb.

Positive subjects containing nouns of high-specificity elicited larger LATL activity.

Negative subjects containing nouns of low-specificity elicited larger LATL activity.

The processing of logical and conceptual information share a mechanism in the LATL.
49 / 49
Документ
Категория
Без категории
Просмотров
4
Размер файла
2 976 Кб
Теги
016, neuropsychological, 2018
1/--страниц
Пожаловаться на содержимое документа