Search bioRxiv⌕ Search

bioRxiv · 10.1101/2023.04.05.535734

Second-order Citations in Altmetrics: A Case Study Analyzing the Audiences of COVID-19 Research in the News and on Social Media

Abstract

The potential to capture the societal impact of research has been a driving motivation for the use and development of altmetrics. Yet, to date, altmetrics have largely failed to deliver on this potential because the primary audience who cites research on social media has been shown to be academics themselves. In response, our study investigates an extension of traditional altmetric approaches that goes beyond capturing direct mentions of research on social media. Using research articles from the first months of the COVID-19 pandemic as a case study, we demonstrate the value of measuring second-order citations, or social media mentions of news coverage of research. We find that a sample of these citations, published by just five media outlets, were shared and engaged with on social media twice as much as the research articles themselves. Moreover, first-order and second-order citations circulated among Twitter accounts and Facebook accounts that were largely distinct from each other. The differences in audiences and engagement patterns found in this case study highlight the importance of news coverage as a public source of science information and provide strong evidence that investigating these second-order citations can be an effective way of observing non-academic audiences that engage with research content.

Source connections

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Alperin, J. P., Fleerackers, A., Riedlinger, M., Haustein, S.. 2023-04-06. Second-order Citations in Altmetrics: A Case Study Analyzing the Audiences of COVID-19 Research in the News and on Social Media. https://doi.org/10.1101/2023.04.05.535734

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗

Seeing isn't believing? Mixed effects of a perspective-getting intervention to improve mentoring relationships for science doctoral students

Science doctoral students can experience negative interactions with faculty mentors and internalize these experiences, potentially leading to self-blame and undermining their research self-efficacy. Helping students perceive these interactions adaptively may protect their research self-efficacy and maintain functional mentoring relationships. We conducted a pre-registered, longitudinal field experiment of a novel perspective-getting intervention combined with attribution retraining to help students avoid self-blame and preserve research self-efficacy. Science doctoral students (n = 155) were randomly assigned to read about mentor perspectives on negative interactions (i.e., Perspective-getting Condition) or about mentoring with no mentor perspective (i.e., control condition). Contrary to our hypotheses, we found no main effects of the intervention on students self-blame or research self-efficacy. However, for students with lower pre-intervention mentorship relationship satisfaction, the intervention preserved research self-efficacy six months later. This study provides evidence that perspective getting may be protective for students who are most in need of relationship intervention. Educational Relevance and Implications StatementEffective mentoring relationships are fundamental for promoting the success of doctoral students in science, yet not all mentoring relationships are high quality. This study assessed the effectiveness of a brief perspective-getting intervention (where students are given the perspective of what it is like to be a research mentor) that aimed to protect science doctoral students from blaming themselves for negative interactions with faculty mentors and maintain their research self-efficacy. Results showed that on average across all students, the intervention did not affect students self-blame for negative interactions or their research self-efficacy. However, the intervention did help students with less satisfying mentoring relationships maintain their self-efficacy. Thus, perspective-getting shows some promise for protecting science doctoral students from harm that can be caused by negative interactions with faculty mentors.

scientific communication and education↗

Benefits of combining individual and small group assessments as an instructional strategy

Assessment is an essential curricular component despite being often seen only as a performance metric resource. However, its full potential can be harnessed if it is used as a formative instrument. The present study evaluated the benefits of combining individual and group assessments. Students perceptions of this type of strategy, assessed through a Likert scale questionnaire and semi-structured interviews, showed that students acknowledge the benefits of this procedure. We then carried out a dual assessment in an active learning environment. Students were given individual written tests and completed the same test immediately after, but now in a group. The average group scores were higher than the average individual scores, even for students who scored the highest within a group. Put together, these results indicate combining individual and group assessments can be an effective teaching tool in addition to simply measuring student performance.

scientific communication and education↗