Red Teaming Challenges

Humane Intelligence partnered with the Department of Defense’s Chief Digital and Artificial Intelligence Office (CDAO), the Defense Health Agency (DHA), and the Program Executive Office, Defense Healthcare Management Systems (PEO DHMS) to conduct a groundbreaking red-teaming initiative. Focused on Generative AI (GenAI) applications in military medicine, the effort explored vulnerabilities and biases in clinical note summarization and medical advisory chatbot systems. Over 200 participants, including clinical providers and healthcare analysts, tested three popular Large Language Models (LLMs), uncovering more than 800 critical findings. These insights are helping to establish benchmark datasets and shape DoD policies for the responsible adoption of GenAI in military healthcare, ensuring safety, equity, and improved care for service members. This effort exemplifies the power of crowdsourced testing in fostering resilient, ethical, and effective AI technologies.

Humane Intelligence, in partnership with AISA and CSIRO, hosted the AI Village at the 2024 AISA Australian Cyber Conference in Melbourne, Nov 26-28. The event, Australia’s largest on AI security, featured interactive sessions like a red-teaming exercise led by Fellow Fariza Rashid and a capture-the-flag challenge targeting vulnerabilities in Large Language Models (LLMs). Inspired by the 2023 White House AI Red Teaming, it explored bias, hallucinations, and prompt injection in AI systems.

Humane Intelligence partnered with UNESCO in Paris for a red-teaming exercise on Technology-Facilitated Gender-Based Violence (TFGBV). Held on the International Day for the Elimination of Violence Against Women and Girls, senior diplomats and UNESCO experts tested generative AI models to identify vulnerabilities, biases, and safety gaps. This collaboration builds on UNESCO’s extensive work on gender and AI, advancing practical solutions for fair, inclusive AI systems.

Humane Intelligence, in collaboration with AI Singapore and global leaders like Anthropic, AWS, Cohere, Google, and Meta, launched the first regional AI Safety Red Teaming Challenge in Asia. Participants from nine countries, including sociologists, linguists, and cultural experts, evaluated four LLMs to detect bias and stereotypes in English and regional languages. This groundbreaking initiative addresses gaps in AI safety evaluations, which often focus on Western contexts, and seeks to develop a common methodology for safer, culturally sensitive AI models. An evaluation report summarizing findings will be published in early 2025.

Humane Intelligence and FEMYSO joined forces at the European Action Day Against Islamophobia Conference 2024 to address AI’s role in perpetuating anti-Muslim bias. Through discussions and an Islamophobia Red Teaming Workshop, participants examined political shifts impacting Muslim communities and tested Gemini, ChatGPT, and Claude for harmful stereotypes.

Humane Intelligence, supported by the U.S. National Institute of Standards and Technology (NIST), conducted a nationwide AI red-teaming exercise to test and evaluate generative AI systems' robustness, security, and ethical implications. Open to U.S.-based individuals and model developers, the initiative advanced AI resilience and trustworthiness through rigorous adversarial testing and analysis.

Humane Intelligence and Compiler hosted an AI reporting workshop at Northwestern University’s Medill School, training 10 journalists on HI’s no-code evaluation platform to investigate generative AI’s societal impacts. Participants explored AI regulation, data journalism, and machine learning, producing original stories for a special edition of Compiler.

At Autodesk University 2023, Humane Intelligence partnered with Autodesk to conduct a two-day red-teaming exercise with architects and engineers. Participants tested generative AI in design applications, assessing structural integrity, creative capability, and potential IP violations of AI-generated outputs, such as modern art museum designs and electric vehicles. Insights aim to ensure responsible, ethical AI use in creative industries.

Humane Intelligence, in partnership with the Royal Society, hosted a red-teaming event where invited COVID-19 and climate scientists tested Meta's Llama 3.0 for mis- and disinformation vulnerabilities. Participants, adopting the personas of misinformation actors, explored how GenAI could amplify these challenges. Findings were shared with Meta to improve the safety of scientific LLMs.

At DEFCON 2023, Humane Intelligence partnered with Seed AI and DEFCON AI Village to host the largest-ever Generative AI public red teaming event. Over 2.5 days, 2,244 participants evaluated 8 large language models (LLMs) through 17,000+ conversations on topics like cybersecurity, misinformation, and human rights. This groundbreaking event demonstrated how public red teaming can provide nuanced insights into AI biases and social harms, complementing traditional corporate evaluations. Partners such as Meta, Google, and OpenAI contributed, along with policy groups like the White House OSTP and NIST.Findings and the anonymized dataset are available on our GitHub, advancing collaboration on AI safety and accountability.