Planning Grants to Create Artificial Intelligence (AI)-Ready Test Beds
Funding Agency:
- National Science Foundation
The U.S. National Science Foundation's (NSF) Directorates for Computer and Information Science and Engineering (CISE) and Technology, Innovation and Partnerships (TIP) are seeking new approaches to develop and evaluate novel artificial intelligence (AI) methods in real-world settings. Too often new AI systems are deployed before the interactions with and impacts on users can be fully evaluated or understood. Often, when AI is evaluated, it is with an inadequate number of samples that do not scale or generalize beyond a limited number of use cases.
This Dear Colleague Letter (DCL) seeks to address these limitations by encouraging the community to pursue Planning Grants designed to develop AI-Ready Test Beds that can be used by researchers to test novel AI methods in potential real-world application scenarios. Because of the current limits in scalability and scope, the AI community is encouraged to expand existing test beds and infrastructure to make them AI-ready and appropriate for use in evaluating the impact and effect of AI tools and systems on users. To do this, proposers are encouraged to create teams of researchers who have expertise in AI as well as domain experts and staff managing existing test beds. Existing test beds and infrastructure can be found in a range of settings, including, but not limited to: NSF-funded centers (e.g., Engineering Research Centers) and facilities, infrastructure funded by other federal agencies (e.g., Department of Energy, National Institute of Standards and Technology, National Oceanic and Atmospheric Administration, Department of Homeland Security, Department of Transportation, etc. or at https://www.nitrd.gov/apps/ai-rd-testbed-inventory/), state agencies (e.g., https://testbed.cityofnewyork.us/) and industry. These Planning Grants will support costs associated with the formation of teams, the planning of the desired or envisioned AI-Ready infrastructure, the collection of preliminary pilot data and creation of governance and management plans for scalable AI-Ready Test Beds.
AI-Ready Test Beds are envisioned as comprehensive services that will provide infrastructure to support researchers to bring innovative applications of AI to bear in high-impact settings. The AI-Ready Test Beds will include technical staff and domain experts to guide and enable new researchers to easily connect to the application domains and provide human and technology interfaces that embed with front-line service providers and domain experts. Planning Grant proposers should consider that the targeted AI-Ready Test Beds could include both co-located and remote research teams and that the AI-Ready Test Beds should support multiple and varied use cases, facilitating inter-project collaboration and nexus-building as appropriate. Planning Grant proposers should envision AI-Ready Test Beds that add AI components to test beds that are already suited to use case experimentation, adopter demonstrations, longer baseline validation testing, etc.
While building AI systems with static data is well-understood, building them or evolving them based on continuously changing or dynamic data, including real-time streaming data and data with significant spatio-temporal characteristics that are difficult to predict, is a significant issue. Hence, AI-Ready Test Beds should embrace a "living lab" model that aims to provide researchers with the ability to study societal impacts including harms to individuals and communities; assess various risks related to security, safety, privacy and fairness, and help stakeholders effectively deploy AI systems following well-established AI risk management frameworks; and experiment with and develop various red-teaming techniques as well as evaluate their effectiveness. AI-Ready Test Beds are envisioned to provide the infrastructure to lower the costs and reduce the communication barriers that researchers currently encounter upon entering an application domain, while allowing real-world tests of innovative AI at scale and in real time.
Funding is available for Planning Grants in Fiscal Year (FY) 2025 to provide support to teams so that they may later submit a full proposal for an AI-Ready Test Bed.
The budget for a planning proposal may be up to $100,000 per year, with total funding requested of up to $200,000 for up to 24 months.
5:00 p.m., submitting organization's local time, on November 13, 2024.
For further information, please contact: AI-Ready_testbeds@nsf.gov.