Biology subjects

Shackelford, G. E.

Publications and source records attributed to Shackelford, G. E..

4 recordsLinked to original sources

bioRxiv2020

Dynamic meta-analysis: a method of using global evidence for local decision making

Meta-analysis is often used to make generalizations across all available evidence at the global scale. But how can these global generalizations be used for evidence-based decision making at the local scale, if only the local evidence is perceived to be relevant to a local decision? We show how an interactive method of meta-analysis -- dynamic meta-analysis -- can be used to assess the local relevance of global evidence. We developed Metadataset (www.metadataset.com) as an example of dynamic meta-analysis. Using Metadataset, we show how evidence can be filtered and weighted, and results can be recalculated, using dynamic methods of subgroup analysis, meta-regression, and recalibration. With an example from agroecology, we show how dynamic meta-analysis could lead to different conclusions for different subsets of the global evidence. Dynamic meta-analysis could also lead to a rebalancing of power and responsibility in evidence synthesis, since evidence users would be able to make decisions that are typically made by systematic reviewers -- decisions about which studies to include (e.g., critical appraisal) and how to handle missing or poorly reported data (e.g., sensitivity analysis). We suggest that dynamic meta-analysis could be scaled up and used for subject-wide evidence synthesis in several scientific disciplines (e.g., agroecology and conservation biology). However, the metadata that are used to filter and weight the evidence would need to be standardized within disciplines.

ecology↗

bioRxiv2020

Poor availability of context-specific evidence hampers decision-making in conservation

Evidence-based conservation relies on robust and relevant evidence. Practitioners often prefer locally relevant studies whose results are more likely to be transferable to the context of planned conservation interventions. To quantify the availability of relevant evidence for amphibian and bird conservation we reviewed Conservation Evidence, a database of quantitative tests of conservation interventions. Studies were geographically clustered and found at extremely low densities - fewer than one study was present within a 2,000 km radius of a given location. The availability of relevant evidence was extremely low when we restricted studies to those studying biomes or taxonomic orders containing high percentages of threatened species, compared to the most frequently studied biomes and taxonomic orders. Further constraining the evidence by study design showed that only 17-20% of amphibian and bird studies used robust designs. Our results highlight the paucity of evidence on the effectiveness of conservation interventions, and the disparity in evidence for local contexts that are frequently studied and those where conservation needs are greatest. Addressing the serious global shortfall in context-specific evidence requires a step change in the frequency of testing conservation interventions, greater use of robust study designs and standardized metrics, and methodological advances to analyze patchy evidence bases.

ecology↗

bioRxiv2019

The challenge of heterogeneous evidence in conservation

Conservation efforts to tackle the current biodiversity crisis need to be as efficient and effective as possible. To inform decision-makers of the most effective conservation actions, it is important to identify biases and gaps in the conservation literature to prioritize future evidence generation. We assessed the state of this global literature base using the Conservation Evidence database, a comprehensive collection of quantitative tests of conservation actions (interventions) from the published literature. For amphibians and birds, we investigated the nature of Conservation Evidence spatially and taxonomically, as well as by biome, effectiveness metrics, and study design. Studies were heavily concentrated in Western Europe and North America for birds and particularly amphibians. Studies that used the most robust study designs - Before-After Control-Impact and Randomized Controlled Trials - were also the most geographically restricted. Furthermore, there was no relationship between the number of studies in each 1x1 degree grid cell and the number of species, threatened species or data-deficient species. Taxonomic biases and gaps were apparent for amphibians and birds - some orders were absent from the evidence base and others were poorly represented relative to the proportion of threatened species they contained. Temperate forest and grassland biomes were highly represented, which reinforced observed geographic biases. Various metrics were used to evaluate the effectiveness of a given conservation action, potentially making studies less directly comparable and evidence synthesis more difficult. We also found that the least robust study designs were the most commonly used; studies using robust designs were scarce. Future research should prioritize testing conservation actions on threatened species outside of Western Europe and North America. Standardizing metrics and improving the robustness of study designs used to test conservation actions would also improve the quality of the evidence base for synthesis and decision-making.

ecology↗

bioRxiv2019

Simple study designs in ecology produce inaccurate estimates of biodiversity responses

O_LIEcologists use a wide range of study designs to estimate the impact of interventions or threats but there are no quantitative comparisons of their accuracy. For example, while it is accepted that simpler designs, such as After (sampling sites post-impact without a control), Before-After (BA) and Control-Impact (CI), are less robust than Randomised Controlled Trials (RCT) and Before-After Control-Impact (BACI) designs, it is not known how much less accurate they are.\nC_LIO_LIWe simulate a step-change response of a population to an environmental impact using empirically-derived estimates of the major parameters. We use five ecological study designs to estimate the effect of this impact and evaluate each one by determining the percentage of simulations in which they accurately estimate the direction and magnitude of the environmental impact. We also simulate different numbers of replicates and assess several accuracy thresholds.\nC_LIO_LIWe demonstrate that BACI designs could be 1.1-1.5 times more accurate than RCTs, 2.9-4.1 times more accurate than BA, 3.8-5.6 times more accurate than CI, and 6.8-10.8 times more accurate than After designs, when estimating to within {+/-}30% of the true effect (depending on the sample size). We also found that increasing sample size substantially increases the accuracy of BACI designs but only increases the precision of simpler designs around a biased estimate; only by using more robust designs can accuracy increase. Modestly increasing replication of both control and impact sites also increased the accuracy of BACI designs more than substantially increasing replicates in just one of these groups.\nC_LIO_LIWe argue that investment into using more robust designs in ecology, where possible, is extremely worthwhile given the inaccuracy of simpler designs, even when using large sample sizes. Based on our results we propose a weighting system that quantitatively ranks the accuracy of studies based on their study design and the number of replicates used. We hope these accuracy weights enable researchers to better account for study design in evidence synthesis when assessing the reliability of a range of studies using a variety of designs.\nC_LI

ecology↗