Decoding human brain activities based on linguistic representations
has been actively studied in recent years. However,
most previous studies exclusively focus on word-level representations,
and little is learned about decoding whole sentences
from brain activation patterns. This work is our effort
to mend the gap. In this paper, we build decoders to associate
brain activities with sentence stimulus via distributed
representations, the currently dominant sentence representation
approach in natural language processing (NLP).We carry
out a systematic evaluation, covering both widely-used baselines
and state-of-the-art sentence representation models. We
demonstrate how well different types of sentence representations
decode the brain activation patterns and give empirical
explanations of the performance difference. Moreover, to
explore how sentences are neurally represented in the brain,
we further compare the sentence representation’s correspondence
to different brain areas associated with high-level cognitive
functions. We find the supervised structured representation
models most accurately probe the language atlas of human
brain. To the best of our knowledge, this work is the first
comprehensive evaluation of distributed sentence representations
for brain decoding. We hope this work can contribute
to decoding brain activities with NLP representation models,
and understanding how linguistic items are neurally represented.
会议录出版者
Association for the Advancement of Artificial Intelligence
1.CAS Center for Excellence in Brain Science and Intelligence Technology, Beijing, China 2.University of Chinese Academy of Sciences, Beijing, China 3.National Laboratory of Pattern Recognition, CASIA, Beijing, China
推荐引用方式 GB/T 7714
jingyuan sun,shaonan wang,jiajun zhang,et al. Towards Sentence-Level Brain Decoding with Distributed Representations[C]. 见:. usa. 2019.2.
修改评论