Search bioRxiv⌕ Search
Home/Archive/bioRxiv

bioRxiv · 10.1101/2020.07.28.225953

Selecting Student-Authored Questions for Summative Assessments

Huang, A.·Hancock, D.·Clemson, M.·Yeo, G.·Harney, D. J.·Denny, P.·Denyer, G.
View original source ↗Export metadata ↓

Abstract

Production of high-quality multiple-choice questions (MCQs) for both formative and summative assessment is a time-consuming task requiring great skill, creativity, and insight. The transition to online examinations, with the concomitant exposure of previously tried-and-tested MCQs, exacerbates the challenges of question production, and highlights the need for innovative solutions. Several groups have shown that it is practical to leverage the student cohort to produce a very large number of syllabus-aligned MCQs for study banks. Although student-generated questions are well suited for formative feedback and practice activities, they are generally not thought to be suitable for high-stakes assessments. In this study, we aimed to demonstrate that training can be provided to students in a scalable fashion to generate questions of similar quality to those produced by experts, and that identification of suitable questions can be achieved with minimal academic review and editing. Biochemistry and Molecular Biology students were assigned a series of activities designed to coach them in the art of writing and critiquing MCQs. This training resulted in the production of over one thousand MCQs that were then gauged for potential by either expert academic judgement, or via a data-driven approach in which the questions were trialled objectively in a low-stakes test. Questions selected by either method were then deployed in a high-stakes in-semester assessment alongside questions from two academically authored sources: textbook-derived MCQs, and past paper questions. A total of 120 MCQs from these four sources were deployed in assessments attempted by over 600 students. Each question was subjected to rigorous performance analysis, including the calculation of standard metrics from classical test theory and more sophisticated Item Response Theory (IRT) measures. The results showed that MCQs authored by students and selected at low cost performed as well as questions authored by academics, illustrating the potential of this strategy for the efficient creation of large numbers of high quality MCQs for summative assessment.

Source connections

Explore related subjects

scientific communication and educationPreprint

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Huang, A., Hancock, D., Clemson, M., Yeo, G., Harney, D. J., Denny, P., Denyer, G.. 2020-07-29. Selecting Student-Authored Questions for Summative Assessments. https://doi.org/10.1101/2020.07.28.225953

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related preprints

bioRxiv2026

Evaluating Large Language Models as Tools to Navigate Researchers in Rapidly Evolving Research Landscapes: A Case Study in Cancer Drug Response Prediction

Large Language Models (LLMs) have emerged as promising tools for assisting researchers in automating and accelerating the synthesis of literature reviews. However, their reliability is a significant concern due to issues like factual inaccuracies and hallucinations. The key question is whether LLMs can reliably provide comprehensive, up-to-date overviews and analyses. This study evaluates the performance of three leading LLMs (OpenAI's ChatGPT, Google's Gemini, and DeepSeek) on the complex task of generating a comprehensive survey paper on deep learning for cancer Drug Response Prediction (DRP). By testing both standard and Deep Research (DR) / Deep Think (DT) modes of LLMs with prompts of varying detail, this paper assesses key academic dimensions, including reference management, content quality, and analytical depth. Key findings reveal that while DR modes of LLMs significantly improve reliability by eliminating hallucinations, performance variations exist across models and prompts. A trade-off between reference quantity and integration quality was observed, and even the best-performing models lacked the analytical depth of human experts, often requiring extensive human supervision. The study concludes that LLMs currently serve as powerful assistive tools but still cannot replace the critical validation and synthesis provided by human researchers. Choosing the best LLM to use depends on the task in hand, while several strategies can be implemented to improve the produced output.

scientific communication and education↗
bioRxiv2020

Cooperative efforts on developing vaccines and therapies for COVID-19

Health organizations have always sought partnership to join competencies in innovation, even with fierce competition in this sector. In this pandemic moment it is relevant to observe how organizations behave to seek quick and safe answers. The present research analyzes how the cooperation networks were set off considering the clinical trials on therapies and vaccines that were developed specifically to treat or prevent COVID-19. Social Network Analysis technique was used to build cooperation networks and apply metrics that characterize these connections. There was an evaluation of statistics of Strength of cooperation and Unilateral dependence of cooperation that identify the cooperation strength between two organizations, and the dependence of this relations. A total of 415 clinical trial were identified, of which 42% are in cooperation. From organizations that have partnership, firms are the first, followed by universities. We extracted the main categories that concentrate 74% of partnerships in the trials of antibody, and vaccine. Several organizations cooperate in multiple categories of trials, evidencing the efforts to focus on different strategies to treat the disease. We found high strength of cooperation and an assimetryc dependency between partners, which can be assigned to specialized models of partnership and it occurs in competitive enviroments like this pandemic moment. Cooperation were not limited to geographical proximity and the advent of Chinese players can represent a new change in the biotechnological development axis. Finally, the challenge of finding therapeutic or immunological solutions for COVID-19 demonstrates a clear composition of cooperation groups that complement their skills to manage organizational strategies to beat the pandemic. In this new paradigm, there can be partnerships not only in clinical trial but also in pre-competitive technologies development. This experience is expected to change the way of organizations define their R&D strategies and start to adopt more a collaborative innovation model.

scientific communication and education↗
bioRxiv2020

A Mental Health Paradox: Mental health was both a motivator and barrier to physical activity during the COVID-19 pandemic

The COVID-19 pandemic has impacted the mental health, physical activity, and sedentary behavior of citizens worldwide. Using an online survey with 1669 respondents, we sought to understand why and how by querying about perceived barriers and motivators to physical activity that changed because of the pandemic, and how those changes impacted mental health. Consistent with prior reports, our respondents were less physically active (aerobic activity, -11%, p <0.05; strength-based activity, -30%, p<0.01) and more sedentary (+11%, p<0.01) during the pandemic as compared to 6-months before. The pandemic also increased psychological stress (+22%, p <0.01) and brought on moderate symptoms of anxiety and depression. Respondents whose mental health deteriorated the most were also the ones who were least active (depression r = -.21, p<0.01; anxiety r = -.12, p<0.01). The majority of respondents were unmotivated to exercise because they were too anxious (+8%, p <0.01), lacked social support (+6%, p =<0.01), or had limited access to equipment (+23%, p <0.01) or space (+41%, p <0.01). The respondents who were able to stay active reported feeling less motivated by physical health outcomes such as weight loss (-7%, p<0.01) or strength (-14%, p<0.01) and instead more motivated by mental health outcomes such as anxiety relief (+14%, p <0.01). Coupled with previous work demonstrating a direct relationship between mental health and physical activity, these results highlight the potential protective effect of physical activity on mental health and point to the need for psychological support to overcome perceived barriers so that people can continue to be physically active during stressful times like the pandemic.

scientific communication and education↗