Estimation of discovery rate in multiple-comparison tests at low sampling numbers
The traditional statistical methods are very powerful if the sampling number in an experiment meets at least the minimum power analysis requirement. However, at lower sampling numbers, they tend to dramatically underestimate the discoveries leading to a large percentage of type II errors. Sometimes, the minimum sampling number is not reached, particularly in preliminary data in large-scale, simultaneously multiple-comparison experiments, where such a required number is much larger than that needed for one-comparison experiments. This work presents a simple approach for searching for discoveries in multiple-comparison experiments with low sampling numbers, based on the fact that the obtained p-values should be uniformly random when no discovery is present. While the proposed approach is less powerful than traditional methods at large sampling numbers, it maintains its efficiency at low sampling numbers. Thus, it is important for preliminary data analysis, for which traditional methods might not exhibit any statistically significant discoveries due to the low available sampling numbers. This approach helps to design further experiments with sufficient power for the desired significance level.