OpenAI's Preparedness Framework and Biological Risk Assessment

As part of its Preparedness Framework, OpenAI is investing in the development of improved evaluation methods for AI-enabled safety risks, specifically focusing on the potential for AI systems to assist in creating biological threats.

• Researchers and policymakers have highlighted the ability for AI systems to aid malicious actors in developing biological threats, such as through step-by-step protocols, troubleshooting wet-lab procedures, or even autonomous execution using cloud labs.
• However, assessing the viability of such hypothetical examples has been limited by insufficient evaluations and data.

Evaluation Study with GPT-4 and Human Participants

OpenAI conducted a study with 100 human participants, comprising 50 biology experts with PhDs and wet lab experience, and 50 student-level participants with at least one university-level biology course.

• Participants were randomly assigned to a control group (with internet access) or a treatment group (with GPT-4 and internet access).
• Each participant was asked to complete tasks covering aspects of the end-to-end process for biological threat creation.
• This study aimed to measure whether models could meaningfully increase malicious actors' access to dangerous information about biological threat creation compared to existing internet resources.

Findings: Mild Uplifts in Accuracy and Completeness

The study assessed uplifts in performance across five metrics (accuracy, completeness, innovation, time taken, and self-rated difficulty) and five stages in the biological threat creation process (ideation, acquisition, magnification, formulation, and release).

• Mild uplifts in accuracy and completeness were observed for participants with access to GPT-4:
  • On a 10-point scale, experts saw a mean score increase of 0.88, and students saw an increase of 0.25 for accuracy compared to the internet-only baseline.
  • Similar uplifts were observed for completeness (0.82 for experts and 0.41 for students).
• However, the effect sizes were not statistically significant, highlighting the need for more research on performance thresholds that indicate a meaningful increase in risk.
• Information access alone is insufficient to create a biological threat, and this evaluation does not test for success in the physical construction of threats.

Methodological Insights and Limitations

OpenAI shares its evaluation procedure, results, methodological insights related to capability elicitation and security considerations for running such evaluations with frontier models at scale.

• The limitations of statistical significance as an effective method of measuring model risk are discussed.
• The importance of new research in assessing the meaningfulness of model evaluation results is emphasized.

Significance and Future Research

This study represents one of the largest to-date human evaluations of AI's impact on biosecurity risk and information access related to biological threat creation.

• OpenAI aims to contribute to broader input and methods-sharing within the AI risk research community.
• Further research is needed to assess the potential risks and develop appropriate safeguards for the safe and responsible development of AI systems in the context of biosecurity and bioterrorism.