Our first challenge set built on the evaluation and dataset from our Generative AI Red Teaming Challenge: Transparency Report. The challenge sets were to create a probability estimation model that determined whether the prompt provided to a language model elicited an outcome that demonstrated factuality, bias, or misdirection.
The Winners
Task: Pick one of the three datasets. Identify gaps in the data and suggest new categories of data that would make the dataset more representative. Generate five prompts per subject area that will elicit a bad outcome. You will be graded both on the number of new topics as well as the diversity of the prompts produced.
Prizes
Task: With your new dataset, generate a likelihood estimator. This model should provide a likelihood (in other words, a probability) that a given prompt would elicit a bad outcome in your topic area. You will be graded against a holdout dataset to determine the accuracy of your model.
Prizes
Task: Develop a site recommendation engine that predicts site suitability for tree planting, integrating key features identified in the beginner level.
Prizes