Comparing serial and parallel language input: Shared neural representations with distinct encoding strengths

Abstract

Language can be experienced in highly serial form, as in speech, or in fully parallel form, as in text, yet how our brains adapt to differences in seriality is not understood. Here, we held modality constant (visual) and examined whether processing mechanisms underlying grammaticality detection are similar under Rapid Serial Visual Presentation (RSVP) and Rapid Parallel Visual Presentation (RPVP). To identify shared neural representations of grammaticality, we trained a classifier on magnetoencephalography (MEG) responses to grammatical and ungrammatical three-word phrases in one presentation mode and tested its generalization to the other. Behaviorally, accuracy was higher and reaction times faster for grammatical vs. ungrammatical phrases across presentation modes. Neurally, cross-presentation decoding revealed shared representations of grammaticality at word 2 across train-test directions--albeit with different latencies and durations--and an early shared representation at word 3 when trained on RPVP. We take the bidirectional representations at word 2 to capture distinct but presentation-invariant combinatorial processes and the asymmetry at word 3 as reflecting stronger global structure encoding in RPVP.