Using event-related potentials to measure phrase boundary perception in English

Most adults understand the speech of other people with little apparent effort. This is an amazing feat given the complexity of the speech signal, which contains a continuous stream of words that are seldom separated by perceptible gaps. To make the meaning of a word-stream clearer to a listener, the speaker groups the words into separate meaningful phrases [1]. In English, the end of each phrase - the phrase boundary - is marked by acoustic cues, such as lengthening the final syllable [2], rise or fall in the fundamental frequency of the speech [3],[4], and by inserting a pause after the final syllable [5].

Phrase boundaries aid speech comprehension in at least two ways. First, phrase boundaries promote semantic processing. The sentence “Rob or Sandra and Mary will come” has different meanings depending on whether a phrase boundary is placed after “Rob” or after “Sandra”. Second, phrase boundaries can define grammatical boundaries in speech. For example, in the sentence “The little girl over there, is the one who likes snails”, a careful speaker of the language will insert a phrase boundary between the words “there” and “is”. By dividing the sentence into two meaningful chunks - a noun phrase and a verb phrase – the speaker clarifies the syntax of the language [6]-[8].

Many behavioural studies have investigated the importance of phrase boundaries for the comprehension of language (for a detailed review, see [9]). Most of these studies have focused on how phrase boundaries resolve syntactic and semantic ambiguities, and hence have presented listeners with atypical sentences that contain violations in syntactic or semantic structure. Unfortunately, these studies cannot tell us how phrase boundaries are processed in typical sentences that do not have structural violations [10].

Fortunately, a handful of event-related potential (ERP) studies have investigated the processing of phrase boundaries in normal sentences. ERPs reflect the average pattern of electrical activity generated by large groups of brain cells in response to a particular sensory or cognitive event [11]. This electrical activity can be detected by small metal sensors which are placed on the scalp. Since ERPs are a direct measure of the electrical activity in the brain, they reflect processing in almost real time. Thus, ERPs can be used to measure the temporal processing of phrase boundaries in normal sentences.

Steinhauer, Alter and Friederici [12] were the first to report an ERP component that could be elicited by phrase boundaries in normal sentences. They presented listeners with German sentences with either one (“Peter promises Anna to work # and to clean the office”) or two (“Peter promises # to support Anna # and to clean the office”) phrase boundaries (# indicates phrase boundary). In 20% of trials, the listeners were asked a yes/no question at the end of the sentence. The “active” ERP (i.e., the ERP measured while listeners actively attended the stimuli) showed a positive shift in the electrical activity around 500 ms after the phrase boundary, which was largest at central and parietal scalp sites. Since the ERP component showed a positive shift in electrical activity at the closure of the phrase, the component was called closure positive shift (CPS).

Steinhauer and Friederici [13] investigated whether the CPS was related to acoustic or linguistic cues at phrase boundaries. They presented listeners with normal and delexicalised sentences (i.e., low-pass filtered sentences with no linguistic cues) that had one or two phrase boundaries. For normal sentences, listeners were asked a yes/no question about the meaning of the sentence. The delexicalised sentences were followed by visual presentation of a normal sentence, and listeners were instructed to recreate the melody of the delexicalised sentence while reading the normal sentence. A CPS was seen at phrase boundaries in the normal sentences, and at the first phrase boundary in the delexicalised sentences. From this, the authors concluded that the CPS is related to the acoustic cues that mark phrase boundaries, rather than linguistic cues. However, the figures in this study showed that the CPS in the delexicalised sentences was different in shape and amplitude to the CPS elicited by the normal sentences. Also, no CPS was seen at a second phrase boundary for delexicalised sentences. This suggests that the CPS is not the result of acoustic cues alone. Instead, both acoustic and linguistic cues elicit the CPS at phrase boundaries in normal sentences.

Pannekamp et al. [10] furthered this work by presenting adults with four types of sentences with either one or two phrase boundaries: normal sentences, jabberwocky sentences (i.e., content words were replaced by meaningless words that lacked semantic information), pseudo sentences (i.e., content words and function words were replaced by meaningless words that lacked syntactic and semantic information) and hummed sentences (i.e., the intonation contour of the sentences were hummed and hence had no syntactic, semantic or phonemic content). After each sentence, listeners were asked to decide whether a probe word was a part of the sentence. A CPS was present in all the conditions. However, the topography of the CPS (i.e., the pattern of electrical activity across the scalp) differed between sentences. For normal sentences, a CPS was seen at both central and lateral sites. As the linguistic cues were reduced (i.e., syntactic, semantic and phonemic information removed), the CPS was lateralised to anterior locations in the right hemisphere. This again suggests that both acoustic and linguistic cues elicit the CPS in normal sentences.

The degree to which acoustic and linguistic cues contribute to the CPS has yet to be established. To date, the majority of CPS studies support the idea that acoustic prosodic cues are primarily responsible for the generation of CPS (though cf. [14]), while linguistic cues modulate its amplitude and scalp topography [15]. However, most of these studies have been done in German [10],[12],[13],[16]-[21], with a handful in Swedish [22], Korean [23], Dutch [24],[25], Mandarin [26], and Japanese [27]. This is problematic for understanding the CPS in English because phrase boundarires are marked by different acoustic cues in different languages. For example, phrase boundaries are marked by changes in duration and intensity in German [28] but by changes in duration, pitch and pause in English [2],[3],[5]. This raises the question of whether acoustic cues for phrase boundaries elicit a CPS in normal English sentences.

The role of acoustic cues in the generation of CPS in English has been addressed in part by Itzhak et al. [29]. They investigated acoustic phrase boundaries and transitivity bias on ERPs in English speakers. Transitivity bias is the probability that a verb will be followed by an object (e.g. “He lifted the bag” versus “She slept”). They presented listeners with sentences that either had a high transitively biased verb or a low transitively biased verb. In half the sentences, the verb was followed an acoustic phrase boundary (e.g. While Billy was playing # the game seemed simple). Participants were asked to make an acceptability judgement after each sentence. The authors found a front-central CPS between 150–300 ms for the sentences with a phrase boundary, and a CPS-like positivity in sentences without a phrase boundary when the verb was highly transitively biased. They concluded that the high transitivity of the verb was sufficient to drive the brain systems to impose a boundary after the verb despite the absence of any acoustic phrase boundary.

The presence of a CPS in the absence of acoustic cues in Itzhak et al. [29] suggests that acoustic cues for phrase boundaries are not a major generator of the CPS. However, the strength of this suggestion is limited by the fact that this study did not measure the CPS to acoustic cues alone. To determine if acoustic cues elicit a CPS in English, we need to measure the CPS in normal sentences in a way that minimises the processing of linguistic cues (i.e., syntax and semantics). This might be done with passive ERPs, which are ERPs measured while a subject’s attention is diverted away from the stimulus of interest (e.g., to a computer game or movie). Thus, the aim of this study was to use passive ERPs to test, for the first time, if acoustic cues for phrase boundaries elicit a CPS in normal English sentences in adults.

We outline below the results for the whole sentence ERPs and the second phrase ERPs separately. Only main effects and interactions involving the factor condition are reported here since they directly related to the aim of the study (i.e., effect of phrase boundaries on ERP).

Whole sentence ERPs

Grand averaged ERPs to early phrase boundary and late phrase boundary sentences at the electrode Cz are shown in Figure 2. The ERPs from all the electrodes used for analysis are shown in Figure 3.

The ERPs to whole sentences with early and late phrase boundaries showed a positive shift starting immediately after the onset of a phrase boundary which lasted around 600 ms. This observation was supported by the statistical analysis. The three-way ANOVA at the midline electrodes revealed a main effect of condition for all three time windows (0–200 ms: F(1,23) = 4.64, p = 0.04, partial ŋ² = 0.17; 200–400 ms: F(1,23) = 18.74, p = 0.001, partial ŋ² = 0.45; 400–600 ms: F(1,23) = 12.76, p = 0.002, partial ŋ² = 0.36). In all time windows, ERPs were significantly more positive for the phrased condition compared to unphrased condition (0–200 ms: phrased M = −2.36, SE = 0.32, unphrased M = −2.90, SE = 0.28; 200–400 ms: phrased M = −2.00, SE = 0.30, unphrased M = −2.98, SE = 0.30; 400–600 ms: phrased M = −2.11, SE = 0.33, unphrased M = −2.95, SE = 0.31). Therefore, the phrase boundaries generated a statistically significant CPS at central electrodes for all time windows.

The four-way ANOVA at lateral electrodes showed a main effect of condition at 200–400 ms (F(1,23) = 19.32, p = 0.001, partial ŋ² = 0.46) and 400–600 ms (F(1,23) = 15.40, p = 0.001, partial ŋ² = 0.40), where the ERPs for phrased condition showed a significantly larger positivity as compared to unphrased condition (200–400 ms: phrased M = −1.19, SE = 0.25, unphrased M = −1.94, SE = 0.25; 400–600 ms: phrased M = −1.28, SE = 0.29, unphrased M = −1.93, SE = 0.24). There was a significant condition × hemisphere interaction between 0–200 ms F(1,23) = 4.25, p = 0.050, partial ŋ² = 0.16. To disentangle the interaction, follow-up one-way ANOVAs were computed for each hemisphere separately with the factor condition. These ANOVAs showed the main effect of condition only at right hemisphere F(1,23) = 4.73, p = 0.04, partial ŋ² = 0.17, where the ERPS were more positive for the phrased condition (phrased M = −1.71, SE = 0.31, unphrased M = −2.17, SE = 0.31).

There was also a significant condition x location interaction between 200–400 ms F(1,23) = 4.62, p = 0.04, partial ŋ² = 0.17. Follow-up one-way ANOVAs conducted at each location showed a main effect of condition at both anterior (F(1,23) = 18.37, p = 0.001, partial ŋ² = 0.44) and posterior locations (F(1,23) = 13.36, p = 0.001, partial ŋ² = 0.37). At both locations, the phrased condition generated a larger positivity compared to the unphrased condition (anterior: phrased M = −1.39, SE = 0.28, unphrased M = −2.32, SE = 0.30; posterior: phrased M = −0.99, SE = 0.24, unphrased M = −1.57, SE = 0.23).

In sum, the analysis on lateral electrodes showed a CPS over right hemisphere between 0–200 ms and at all locations between 200–400 ms and 400–600 ms time intervals. There were no main effects or interactions for boundary type at either midline or lateral electrodes indicating that CPS did not depend on the position of the boundary.

Second phrase ERPs

The ERPs time locked to the onset of the second phrase are shown in Figure 4.

Figure 4 shows that the positivity at the phrase boundary started before the onset of the second phrase and continued after the second phrase onset. This effect was confirmed by the ANOVAs computed for midline and lateral electrodes.

The three-way ANOVA of midline electrodes showed a main effect of condition between for −250-0 ms F(1,23) = 6.89, p = 0.015, partial ŋ² = 0.23; and 0–250 ms F(1,23) = 18.65, p = 0.001, partial ŋ² = 0.45. Phrased stimuli elicited larger positivity compared to unphrased stimuli at both time windows (−250-0 ms: phrased M = −2.21, SE = 0.25; unphrased M = −3.20, SE = 0.34; 0–250 ms: phrased M = −1.16, SE = 0.26; unphrased M = −2.95, SE = 0.33).

The four-way ANOVA at lateral electrodes also showed a main effect of condition for −250-0 ms F(1,23) = 4.41, p = 0.047, partial ŋ² = 0.16; and 0–250 ms F(1,23) = 14.66, p = 0.001, partial ŋ² = 0.39. Similar to the findings at medial electrodes, the phrased condition generated larger positivity at lateral electrodes (−250-0 ms: phrased M = −1.47, SE = 0.21; unphrased M = −2.12, SE = 0.26; 0–250 ms: phrased M = −0.56, SE = 0.22; unphrased M = −1.88, SE = 0.25). Therefore the positivity at the phrase boundaries started before the start of the second phrase and later merged with the N1-P2 responses to the second phrase.

The aim of this study was to use passive ERPs to test if acoustic cues for phrase boundaries in normal English sentences elicit a CPS. We presented 24 English-speaking adults with two types of sentences: sentences with an early phrase boundary, and sentences with a late phrase boundary. We found that both early and late phrase boundaries elicited a significantly more positive ERP from 0 to 600 ms after the onset of the phrase boundary at midline and right hemisphere electrodes, and from 200 to 600 ms at left hemisphere electrodes, than the ERP to the same sentence portion without a phrase boundary.

The positivity seen at phrase boundaries might indicate either an ERP effect for phrase boundary processing (CPS), an enhanced N1-P2 response to the second phrase onset (speech onset following a pause is expected to generate a larger N1-P2 response) or a combination of the two. In order to disentangle the CPS from the N1-P2 response, a second analysis was conducted between −250 ms and 250 ms with respect to the onset of the second phrase [17],[18]. This time window allowed us to differentiate the ERP effects that started prior to the onset of the second phrase (−250 ms to 0 ms) where the acoustic cues of phrase boundary are available (pre-boundary lengthening, pitch change) from the ERPs effects that started after the onset of the second phrase (0 to 250 ms) where the N1-P2 response to the second phrase onset is overlapped with the ERP response to phrase boundary (if any). This analysis revealed that the positivity at the phrase boundary started before the second phrase onset and later merged with the N1-P2 response. Therefore, the positive shift observed after the onset of the phrase boundary in the whole sentence analysis does not appear to solely reflect an enhanced N1-P2 response for the onset of speech after a pause. Rather, it contains an additional ERP component possibly reflecting the processing of phrase boundaries in unattended speech. Hence, we call the positivity between 0 to 600 ms in the whole sentence analysis the “passive CPS”.

This is the first study to investigate and show the presence of a passive CPS to unattended normal sentences in adult English speakers. The results confirm that a CPS can be elicited under passive listening conditions, and suggest that acoustic cues that mark phrase boundaries do indeed elicit a CPS in normal English sentences. These findings are similar to previous studies [10],[12],[13] which demonstrated that the CPS is elicited by phrase boundaries in normal sentences in languages other than English. However, our results differ from these previous studies in three interesting ways. First, the CPS in the current study was present immediately after the onset of the phrase boundary (significant positivity in 0–200 ms window at midline and right hemisphere electrodes). In previous studies, the CPS occurred around 500 ms after the phrase boundary [10],[12]. Second, in the current study, the CPS was around 2 μV in size. Only one previous study has reported the amplitude of the CPS (in μV) previously, which was 4 μV [19].

There are two obvious methodological differences that could explain these slight differences in CPS latency and amplitude between studies. First, the previous studies analysed the ERPs in 500-ms time windows, while we analysed the ERPs in smaller 200-ms time windows. Our smaller time window should have allowed a finer analysis of the temporal processing of phrase boundaries, which could reveal different results. Secondly, the current study used a passive auditory paradigm, while the previous studies used an “active” ERP paradigm that asked listeners to actively comprehend the presented sentences. Overt sentence comprehension may trigger additional linguistic processing (i.e., phonemic, syntactic and semantic processing) which may extend up to 600 ms [35]. These additional processes might affect the latency and amplitude of CPS. Hence, the delayed and larger CPS at phrase boundaries in attended sentences in previous studies might reflect both linguistic and acoustic processing, which we wanted to avoid in this study by the passive ERPs. Thus, the CPS in the current study may more closely reflect the acoustic processing of phrase boundaries in speech than linguistic processing of phrase boundaries in speech.

An alternative account for the early passive CPS might be the changes in F0 contour (i.e., pitch) at the phrase boundary. Roll et al. [36],[37] found that an increase in pitch of the first word in a phrase generated a relative large P2 response (around 200 ms) in Swedish sentences. In the present study, ERPs were measured to the offset of the final word in a phrase. However the increase in F0 signalling the phrase boundary is present approximately 146 ms before the offset of the final word. It is possible that the positivity observed in the 0–200 ms time window in this study could have contributions from a larger P2 generated by changes in pitch in the last word of the phrase. The similarity between the ERPs to pitch changes in the first word of a phase [36],[37] and pitch changes in the final word of the previous phrase (present study) require further investigation.

The third difference between the results of the present study and previous CPS studies is the topography of the CPS. Centro-parietal distribution for the active CPS is previously reported for German sentences [10],[12]. Using English sentences, Itzhak et al. [29] reported CPS with fronto-central distribution. Central and right hemisphere distribution of CPS is reported by Pannekamp et al. [10] in the conditions that used jabberwocky sentences, pseudo sentences and hummed sentences. In the present study, we found the passive CPS with central and right hemisphere distribution in the 0–200 ms time window and a broad distribution in the 200–600 ms time windows. Broad distribution of CPS is also reported in active paradigms [15],[16]. This suggests that the topography of the CPS is dependent on the stimuli and task. For attended normal sentences, CPS may be triggered by a combination of acoustic and linguistic cues (phonemic, syntactic, and semantic information) in the speech signal. When linguistic cues are reduced, the CPS is broadly distributed, with greater activity in the right hemisphere than left hemisphere.

Interestingly, the location of the phrase boundary did not have an effect on the CPS. Both early and later phrase boundaries elicited CPS with similar magnitude and scalp distribution. This contrasts with the findings of Holzgrefe et al. [16], who found the CPS was seen only for late phrase boundary in German sentences. Differences in stimuli might explain the different outcomes between the studies. The early phrase boundary condition in Holzgrefe et al. [16] inserted a phrase boundary after the first word in the sentence. They attributed the absence of CPS in this “early” phrase boundary condition to two factors: (1) absence of previous prosodic information to process acoustic cues as phrase boundary cues, and (2) an absence of chunking the first word into a larger prosodic unit in the sentences. In the present study the early phrase boundary occurred after three or four words in the sentence (~1 second after the sentence onset). Therefore there was enough prior prosodic information to process the acoustic cues as phrase boundary cues, as well as enough words before the phrase boundary to chunk them together. Hence, a CPS was seen for both early and late phrase boundaries.

In conclusion, the present study shows that a passive CPS is elicited by acoustic cues that mark phrase boundaries normal English sentences. The timing, amplitude, and topography of the passive CPS are different to the active CPS measured in previous studies. These results show that acoustic cues at the phrase boundary are sufficient to elicit the CPS, and suggest that different processes might be involved in the generation of the CPS in active and passive conditions. The passive CPS that we observed appears to primarily reflect the processing of the acoustic cues in normal English sentences, whereas the active CPS might be reflecting acoustic and linguistic processing.