Akpan, E. | Search bioRxiv

bioRxiv2020

Representational Connectivity Analysis: Identifying Networks of Shared Changes in Representational Strength through Jackknife Resampling

The structure of information in the brain is crucial to cognitive function. The representational space of a brain region can be identified through Representational Similarity Analysis (RSA) applied to functional magnetic resonance imaging (fMRI) data. In its classic form, RSA collapses the time-series of each condition, eliminating fluctuations in similarity over time. We propose a method for identifying representational connectivity (RC) networks, which share fluctuations in representational strength, in an analogous manner to functional connectivity (FC), which tracks fluctuations in BOLD signal, and informational connectivity, which tracks fluctuations in pattern discriminability. We utilize jackknife resampling, a statistical technique in which observations are removed in turn to determine their influence. We applied the jackknife technique to an existing fMRI dataset collected as participants viewed videos of animals (Nastase et al., 2017). We used ventral temporal cortex (VT) as a seed region, and compared the resulting network to a second-order RSA, in which brain regions representational spaces are compared, and to the network identified through FC. The novel representational connectivity analysis identified a network comprising regions associated with lower-level visual processing, spatial cognition, perceptual-motor integration, and visual attention, indicating that these regions shared fluctuations in representational similarity strength with VT. RC, second-order RSA and FC identified areas unique to each method, indicating that analyzing shared fluctuations in the strength of representational similarity reveals previously undetectable networks of regions. The RC analysis thus offers a new way to understand representational similarity at the network level.

neuroscience↗

bioRxiv2019

Image memorability is predicted by activity across different stages of convolutional neural networks and the human ventral stream

The features of an image can be represented at multiple levels - from its low-level visual properties to high-level meaning. What drives some images to be memorable while others are forgettable? We address this question across two behavioral experiments. In the first, different layers of a convolutional neural network (CNN), which represent progressively higher levels of features, were used to select the images that would be shown to 100 participants through a form of prospective assignment. Here, the discriminability/similarity of an image with others, according to different CNN layers dictated the images presented to different groups, who made a simple indoor vs. outdoor judgment for each scene. We find that participants remember more scene images that were selected based on their low-level discriminability or high-level similarity. A second experiment replicated these results in an independent sample of fifty participants, with a different order of post-encoding tasks. Together, these experiments provide evidence that both discriminability and similarity, at different visual levels, predict image memorability.

neuroscience↗