We’re releasing our AI red teaming app under an OSS license

Thanks to the generous support of Google.org’s Scientific Ecosystem team, I am thrilled to announce that our proposal to release our AI red teaming app under an open source software (OSS) license has been fully funded! Over the next year, we will be working with Google.org, a few strategic service providers, and you, our greater community, to realize this vision.

Back in July 2025, we defined a (code) forking strategy that would allow us to both offer a version of our AI red teaming app under an OSS license and improve our managed instance. A key goal for both versions is to bring down engagement costs so that more organizations and people can engage in human-centered AI red teaming, which is critical to implementing and deploying generative AI for good solutions at scale.

Three Main Phases

Our work will commence this month, December 2025, and will result in the initial OSS release of the AI red teaming app during the lat quarter of 2026. The work will roll out in three main phases:

Phase 1: Front-End Improvements

As we detailed in our concept note, the original purpose of the AI red teaming app was to simplify the process by which any individual could red team an AI model or system, in order to grow the field of evaluators. With time, we’ve realized that the highest demand to use the app is from organizations that wish to red team their specific AI model or system. Therefore, we need to rewrite the app in a way that makes it easier, cheaper, and quicker to test a wide range of AI models and systems.

This gives us an opportunity to conduct user research with our end users to modify our taxonomy of terms, improve our primary paths and core workflows, make the app experience more intuitive, and decouple the front-end and back-end so that only some back-end interactions require a fully supported front-end. In August, we refreshed our organizational branding and in September, we relaunched our website with a new design system. In phase 1 of our OSS release, we’ll update the visual design and UI of our managed instance to match our new website and branding. We’ll also create a simplified version of our visual design and UI to release under an open content license as part of the OSS release.

Phase 2: Refactor / Rewrite the Back-End

Decoupling the front-end and back-ends of our AI red teaming app gives us several options to refactor and rewrite the back-end of our code base. Python has continued to emerge as the language of LLMs. Based on the needs expressed in the phase 1 user research, we will select the most common LLMs / data streaming formats to support.

We’ll also create flexibility for potential users of our OSS AI red teaming app to modify the code base for other LLM and AI system data streaming formats. To that aim, we plan to create a public product roadmap on a community-dedicated GitHub repo to solicit feedback and advice about what features or services to include, and in what format. Generative AI continues to evolve at a rapid pace, and it’s therefore critical we keep the lines of communication open with our potential OSS users.

Phase 3: The OSS Release

In parallel to the end of phase 2, we will prepare for the actual OSS release. This entails the final selection of our OSS license, setting up our governance structure, and implementing our community contribution strategy. Ahead of the release, we will create robust documentation to help software developers wishing to fork and deploy the OSS code. During phase 3, we will also apply to have the OSS AI red teaming app included in the Digital Public Goods Alliance registry. Within 12 months of the OSS release, it’s our goal for the tool to have been adopted by at least three INGOs, and for us to have an initial report out on user adoption and its benefits.

Stay tuned for more updates on our blog and on our OSS page, including ways to get involved. Thanks again to Google.org for their support in this worthy initiative! We can’t wait to get started!