NIST's U.S. AI Safety Institute to Collaborate with Anthropic and OpenAI on AI Research

The U.S. AI Safety Institute, part of the National Institute of Standards and Technology (NIST), is partnering with AI companies Anthropic and OpenAI to collaborate on AI safety research, testing, and evaluation.

Under the Memoranda of Understanding, the U.S. AI Safety Institute will gain access to new AI models from both companies before and after their public release. This collaboration aims to assess the capabilities and risks of these models and develop methods to mitigate potential safety concerns.

"Safety is essential to fueling breakthrough technological innovation," said Elizabeth Kelly, director of the U.S. AI Safety Institute, in a statement. "With these agreements in place, we look forward to beginning our technical collaborations with Anthropic and OpenAI to advance the science of AI safety."

"These agreements are just the start," she added, "but they are an important milestone as we work to help responsibly steward the future of AI."

The U.S. AI Safety Institute also intends to work closely with its partners at the U.K. AI Safety Institute to offer feedback to Anthropic and OpenAI on potential safety enhancements to their models.

"Safe, trustworthy AI is crucial for the technology's positive impact," said Anthropic co-founder and head of policy Jack Clark, in a statement. "Our collaboration with the U.S. AI Safety Institute leverages their wide expertise to rigorously test our models before widespread deployment."

The agreements come at a time of increasing regulatory scrutiny over the safe and ethical use of AI technologies. California legislators are also poised to vote on a bill regulating AI development and deployment.

The initiative builds on NIST’s longstanding legacy in advancing measurement science and standards, with the aim of fostering the safe, secure, and trustworthy development and use of AI, as outlined in the Biden-Harris administration’s Executive Order on AI.

"We believe the institute has a critical role to play in defining U.S. leadership in responsibly developing artificial intelligence," said OpenAI chief strategy officer Jason Kwon, "and hope that our work together offers a framework that the rest of the world can build on."

About the Author

John K. Waters is the editor in chief of a number of Converge360.com sites, with a focus on high-end development, AI and future tech. He's been writing about cutting-edge technologies and culture of Silicon Valley for more than two decades, and he's written more than a dozen books. He also co-scripted the documentary film Silicon Valley: A 100 Year Renaissance, which aired on PBS.  He can be reached at [email protected].

Featured

  • Report Explores Teacher and Administrator Attitudes on K–12 AI Adoption

    K–12 administration software provider Frontline Education recently released a new research brief regarding the use of AI adoption in schools, according to a news release. “Insights into K–12 AI Adoption: Educator Perspectives and Pathways Forward” was developed from the results of the Frontline Research and Learning Institute’s annual survey of district leaders.

  • PowerBuddy for Data

    PowerSchool Releases AI-Powered Tools for Students, Admins

    PowerSchool recently announced the general availability of two new AI-powered education tools, one for students and one for education data managers.

  • abstract pattern of interlocking circuits, hexagons, and neural network shapes

    Anthropic Offers Cautious Support for New California AI Regulation Legislation

    Anthropic has announced its support for an amended version of the "Safe and Secure Innovation for Frontier Artificial Intelligence Models Act," California’s Senate Bill 1047 (SB 1047), because of revisions to the bill the company helped to influence — but not without some reservations.

  • person signing a bill at a desk with a faint glow around the document. A tablet and laptop are subtly visible in the background, with soft colors and minimal digital elements

    California Governor Signs Off on AI Content Safeguard Laws

    California Governor Gavin Newsom has officially signed a series of landmark artificial intelligence bills into law, signaling the state’s latest efforts to regulate the burgeoning technology, particularly in response to the misuse of sexually explicit deepfakes. The legislation is aimed at mitigating the risks posed by AI-generated content, as concerns grow over the technology's potential to manipulate images, videos, and voices in ways that could cause significant harm.