The big consequences of small biases: : A simulation of peer review

• Autores: Theofore E. Day
• Localización: Research Policy, ISSN-e 1873-7625, Vol. 44, Nº. 6, 2015, págs. 1266-1270
• Idioma: inglés
• Texto completo no disponible (Saber más ...)
• Resumen

  • Objective To determine the effect of reviewer bias on grant application funding rates between a “preferred class” (PC) and a “non-preferred class” (NPC) of principal investigator.

    Methods A discrete event simulation (DES) of grant review was developed which mimics the production, review, and funding determination of grants. Grants were defined to have an intrinsic quality. Three reviewers then score each grant, and assign it a value. Zero (control), one, or all reviewers may exhibit biases of varying severity against NPC investigators.

    Results When total review bias exceeds 1.9% of grant score, statistically significant variation in scores between PC and NPC investigators is discernable in a pool of 2000 grant applications. When total review bias exceeds 2.8% of total grant score, statistically significant discrepancies in funding rates between PC and NPC investigators are detectable in a simulation of grant review.

    Conclusions Review bias affects funding rates even when total review bias is less than half the amplitude of normal variation in an individual reviewer’s score. Addressing reviewer bias will improve equity among investigators and may improve the overall quality of funded grant applications.