The full report
Key findings from the data:
-
Our analysis divided the questions into four broad categories: Factuality, Bias, Misdirection, and Cybersecurity.
-
The most successful strategies were ones that are hard to distinguish from traditional prompt engineering. Asking the model to role play, or ‘write a story’ were successful.
-
Human behavior can inadvertently result in biased outcomes. People interact with language models in a more conversational manner than with search engines.
-
Unlike other algorithmic systems – notably social media models – the LLMs did not further radicalize users when provided with aggressive content. In most cases, it matched the harmfulness of the user query. In a few cases, the model even de-escalated.