8 theses for a lot element in research funding
Quality assurance is and remains absolutely indispensable for science. However, the limitations of peer review are becoming increasingly evident. New approaches are therefore urgently needed – such as the partially randomized selection of projects by lot.
Since 2017, the Volkswagen Foundation has been testing a novel procedure for selecting projects for funding in the initiative 'Experiment!': Following initial quality assurance by a jury, it selects a share of the funded projects by lot. What effect does this random element have on the selection of projects?
The following theses are based on practical experience and initial results from the accompanying research.
In the case of highly competitive procedures, the lottery relieves the burden on reviewers faced with the problem of differentiating quality among a large number of equally high-ranking proposals.
Decisions by lot are free of any bias and of any influences caused by group dynamics.
The lot is blind to quality. To work properly, therefore, the process requires an initial quality assurance.
In the event where a reviewer panel does not cover all topics equally well, such a procedure ensures fairness among eligible applications.
Regarding the outcome, diversity is enhanced by drawing lots. Often, procedures based on consensus tend to favor established topics and conventional methods ("mainstreaming").
Accompanying research shows that the partially randomized selection, i.e. including a lottery, encourages the submission of risk-taking research proposals.
The introduction of a randomized element is broadly welcomed by the scientific community, including reviewers, and by an increasing number of funding organizations.
Selection by lot has to be regarded as a useful supplement – but not an alternative – to peer review and cannot replace scientific discourse.
What is your opinion on lottery procedures in research funding? Join the discussion on Twitter: #PeerReviewLottery.
Background: the funding initiative 'Experiment!'
With its funding initiative "Experiment!", the Volkswagen Foundation supports scientists who want to test a radically new and risky research idea. Failure of the concept and unexpected findings are acceptable results.
Facts and figures
The initiative was started in November 2012, funded projects are selected via a partially randomized procedure since 2017.
The initiative addresses researchers in science and engineering as well as in the life sciences. The funding is limited to 18 months and up to 120,000 Euro. The projects selected for funding cover a wide range of subject areas, with participants from research institutions across Germany.
The accompanying research project
The lottery as a new selection element and the entire selection process are being evaluated. In our interview "High level of acceptance for project selection by lot", Dr. Dagmar Simon and Dr. Martina Röbbecke (Evaconsult) report on initial results from their accompanying research project.
