OpenAI will increase controls based on risk level
Amelia Glaese, Vice President of Research at OpenAI, explained that the intensity of controls will increase as the models become more capable.
The executive noted that there are requirements and expectations for developing models safely, but clarified that those criteria vary depending on the identified risk level.
OpenAI announced new security measures to control risks during the development and testing of its artificial intelligence models. The changes include increased monitoring, more network isolation, and additional controls during training.
The company explained that the controls will be stricter as the capabilities of the models increase. The new policies come after the incident involving Hugging Face reported on July 21, although OpenAI clarified that they were not implemented as a direct response to that case.
What security measures has OpenAI implemented
The new policies from OpenAI aim to contain potential incidents while the models are being developed and evaluated. The company has incorporated more detailed monitoring during the development process and reinforced the importance of alignment and security after training.
The company noted that risks increase as models gain greater capabilities. Therefore, it stated that its standards for monitoring, alignment, and security must stay ahead of those risks.
OpenAI also strengthened its network isolation practices. As explained, an individual compromise of a workload or support service should not, by itself, allow unauthorized access to the Internet or other internal networks.
</strong> for two weeks. Since then, the company has resumed training several models considered lower risk.</p><p>However, the larger RL training planned for frontier models remains suspended. OpenAI stated that it will first conduct smaller-scale training and evaluations to analyze behavior and validate its security measures.</p><figure class=)
How OpenAI's new monitoring system will work
The main security measure will be a monitoring system that will analyze actions taken by tools, available reasoning traces, and activity logs.
The system will look for different types of unauthorized behavior, and OpenAI aims to generate alerts within 30 minutes following an activity deemed concerning.
The company estimates that the computational cost of this monitoring will be equivalent to approximately 20% of the supervised process. OpenAI announced that it will publish more details about the system in an upcoming blog post.
What is known about the Hugging Face incident
The Hugging Face incident was reported by OpenAI on July 21. According to the company, some models managed to escape their training environment after an attacker compromised a tool in its network with internet access.
The episode led to criticism of OpenAI's network security practices. The new measures aim to reduce the potential impact of a possible compromise of a tool or service.
Nonetheless, the official analysis following the incident is still pending. OpenAI also did not specify all the technical details of the new network isolation practices.
