During 2024, we ran a call for proposals for cooperative AI research with four submission deadlines distributed over the year. We funded a total of three proposals with a total budget of approximately GBP 660,000, and have pending revision requests for two additional projects (for approximately an additional GBP 170,000). Further details about our past grantmaking can be found here. We believe that these projects will be important for driving the development of the field of cooperative AI.
For 2025, we are making some changes to our grant program with the aim of further increasing the amount and impact of funded proposals, and addressing a number of specific challenges that have been highlighted in feedback from applicants and reviewers.
In 2024, we deliberately formulated the scope to be as inclusive as possible within the boundaries of what we consider to be cooperative AI research, but for the coming year, the scope is much more precisely defined. A number of specific research areas are listed, and applicants must select one or more of these areas that fit their proposal. The purpose of this change is to communicate our priorities more clearly and make it easier for applicants to assess how well their ideas align with our mission.
In 2024, all applicants were invited to submit full proposals. After review, a decision was communicated, in most cases accompanied with some level of feedback.
The purpose of this change is to focus more of our resources on the most promising proposals, while also reducing the time spent by applicants and reviewers on unsuccessful applications. In the first stage, we will accept short pre-proposals. In the second stage, the top proposals will then receive reviewer feedback during the process (instead of at the end once the decision is made), and applicants will be given the opportunity to respond to feedback and if they so choose revise their proposals and have them reassessed. We believe that this has the potential to significantly increase the number of accepted proposals without lowering the bar for acceptance.
We are establishing a new “early-career track” where postdoctoral and similar applicants can apply for grants up to GBP 100,000 for projects of up to 12 months that are primarily carried out by one individual. Such proposals will be assessed on the basis of furthering the career of a promising researcher, in addition to the merits and expected impact of the specific research project.
Our overall ambition with the changes to our grant program is to be able to fund more proposals and increase the impact of the program. You can find the full information about the current grant program and scope here.
In the first year of the grant program, we experienced several challenges that needed to be addressed in order to increase the impact and cost-effectiveness of our grantmaking:
A large majority of the proposals that we received in 2024 were out of scope. Even among the proposals that were in scope, a large fraction were not properly aligned with our grantmaking priorities. This points to a failure in communication on our part of what cooperative AI is and what kind of work we see as high priority.
In 2024, we funded three proposals with a total budget of approximately GBP 660,000, and we have pending revision requests for two additional projects (for approximately an additional GBP 170,000). This is less than 50% of what had been budgeted for grantmaking in 2024. One of the successful proposals had gone through a revision, and the other two were accepted with conditions based on reviewer feedback.
This points to a failure to solicit as many high-quality proposals as we would have capacity to fund, which in a very direct way decreases our potential impact through grantmaking. It is notable that several proposals that were seen as having promising elements, and that were rejected with encouragement and feedback, have not been re-submitted.
The decision-making process initially relied on internal technical staff to assess which proposals should proceed to external review, and on advisors serving on the grantmaking committee to make the final recommendations to our trustees. The rationale for using advisors, who are external senior researchers in the field, for the grantmaking committee was to make use of their additional high-level expertise and reduce the bias that may come from concentrating decision-making power in the hands of very few people.
During the year, additional reviewers had to be brought in as contractors for the internal review stage, as we lacked the capacity to handle the volume of applications. Due to the occasional unavailability of our advisors, we also sometimes brought in other senior, external researchers to serve on the grantmaking committee. From the third round, based on trustee feedback, we added an extra committee review stage to increase oversight of internal reviews. This gave the committee members the option to overrule the decisions made by reviewers about which proposals should proceed to external review.
From the perspective of ensuring good decision-making, there were issues with this process where the decision-making power lay primarily with people external to CAIF. Understandably, our grantmaking was not their top professional priority, and so it was not always easy to ensure a consistent, high level of engagement. As the same people were not involved each round it was also hard to create good feedback loops for learning, and there was a lack of direct accountability for the decisions that were made.
Despite limited outreach and the very limited number of proposals being funded, the grant process required a lot of resources in terms of senior technical staff time and the support of highly skilled external researchers/advisors, who in most cases volunteered their time. This has had a significant influence on our capacity to invest in other activities.
One factor that contributed to the high resource consumption was that for proposals where there was any doubt regarding whether they should stay in the process, our policy has been to err on the side of progressing them to the next stage. This applied to both screening and initial review decisions, and led to significant reviewer effort being spent on proposals with a low chance of acceptance.
Another factor was that we prioritised providing detailed rejection feedback for a large number of proposals. This both increased the demands on reviewers, as they needed to provide detailed comments also for proposals that were clearly not going to be funded, and on senior staff, as feedback needed to be carefully and precisely communicated. We also noted that any rejection feedback, even when not formulated to be especially encouraging, was often taken as encouragement by applicants and led to resubmissions also for proposals that were very unlikely to be funded in any version. At the same time, it appeared that feedback needed to be quite detailed and extensive to be useful for applicants.
Finally the two grantmaking committee stages turned out to require a lot of time in terms of securing committee members as well as preparation for and processing of the meetings themselves. Each of the two committee stages also slowed the process down by approximately one week, as material needed to be shared with committee members some days in advance.
During the first half of 2025 we expect to still be very limited by the capacity of our technical staff. Even so, the ambition with the changes we are now implementing is to increase the impact of the grant programme, not by adding resources but by re-prioritisation aimed at improving our cost-effectiveness.
In order to address the challenges with communicating our scope and priorities, the scope of the grant call will be made more specific with a number of research areas from which an applicant must choose. For each area, we provide specific information on what we are looking for. We also highlight to areas that are particularly high priority for us: Understanding and Evaluating Cooperation-Relevant Capabilities and Understanding and Evaluating Cooperation-Relevant Propensities. Our aim is to help applicants understand our scope and priorities better and to enable them to submit stronger proposals in the first iteration.
We have several other ideas for how to improve the communication of our scope and priorities, such as through webinars, talks on external events and conferences, blog posts clarifying our views on relevant topics and individual feedback on ideas over email, calls or at conferences. We do not expect that there will be capacity for significantly scaling up our activities in any of these areas in the first half of 2025, but there is potential to do so in the second half of 2025.
Considering how not a single proposal so far has been funded without conditions or revisions, while at the same time there are a lot more proposals containing promising ideas or components, we believe that a more interactive process will lead to more grants being made while maintaining a high bar.
We will therefore split the submission of proposals into two phases. First, applicants will be asked to submit a short pre-proposal. Second, the top applicants from the first phase will be invited to submit a full proposal. These applicants will receive reviewer feedback during the process (instead of at the end once the decision is made), and be given the opportunity to respond to feedback, to revise their proposals, and to have them reassessed.
This will make each grant round slightly more prolonged for the top proposals, but has the potential to significantly increase the number of accepted proposals without lowering the bar for acceptance. We also hope that it will reduce the amount of effort spent by researchers on unsuccessful proposals. To make this interactive process possible, we will reduce the number of grant rounds from four to two rounds annually.
We will further establish an explicit “early-career track” where postdoctoral and similar applicants can apply for grants up to GBP 100,000 for projects up to 12 months that are primarily carried out by one individual. Such proposals will be assessed on the basis of furthering the career of a promising researcher, which is slightly different from an assessment focused only on the merits and expected impact of the specific research project.
The role of advisors and external reviewers in the process will be adjusted to be more focused on providing expertise, advice and oversight, while decision-making and final recommendations are moved to CAIF staff who are heavily involved in the process and have context, feedback loops and a higher degree of accountability on the decisions made. While the same level of expertise will still be consulted, this will improve our ability to align decisions with our overall strategy, and ensure consistency over time.
Advisors will review the shortlisting selection as well as the recommendations for final decisions and have the option to give their independent recommendations, alongside comments on the process and other proposals.
We believe that these changes address the main challenges that we have seen related to the decision-making process, while keeping the benefits of involving advisors to draw on their expertise and ensure a high-quality process. As the changes also somewhat reduce the amount of work required by the committee members, we expect that this will make it easier to secure senior committee members with an in-depth understanding of cooperative AI. We also think this will make it easier for our trustees to govern the process and grantmaking priorities.
To streamline the process, we plan to make more decisive evaluations earlier, allowing us to focus on the most promising ideas. This means increasing our tolerance for the risk of mistakenly rejecting a proposal too early in the process.
As we have found it challenging to find enough good proposals to fund, it might seem counterintuitive to increase our tolerance for mistaken rejections. However, it is important to keep in mind the following two considerations:
It is of course still regrettable if we reject a promising proposal, but if what we want to optimise for is impact, it is more important to focus on increasing the number of “true positives” – the amount of impactful research that we make happen – than on reducing the number of “false negatives” – the potentially promising funding opportunities that we miss. As the field grows and the number of in-scope applications we receive increases, it is unrealistic to aim to keep the number of false negatives to zero.
Further, we plan to reduce the resources spent on individual rejection feedback dramatically, so that we only give feedback to a small number of applicants. To the extent that we have capacity for feedback, we will prioritise feedback for relatively strong proposals in high priority areas, as that could lead to more fundable proposals in the short term, as well as feedback for particularly promising early-career researchers, as that could be important for the field in the longer term.
Our judgement is that if we invest this freed up staff and reviewer time wisely in improved feedback for selected proposals and a greater quantity and quality of external outreach, this will lead to more high-quality proposals being funded (both in the short and the long term) than our current resource allocation.
December 18, 2024
