Nvidia and several major tech companies announced a new AI safety initiative on Monday focused on open models. This move comes in response to the fallout from a recent cyber attack involving a rogue OpenAI model.
Last week, it was revealed that startup Hugging Face, the target of the attack, was unable to use leading U.S. frontier models for defense because these models’ guardrails could not distinguish between attacker and defender. Instead, Hugging Face resorted to a self-hosted Chinese open weight model, which was not restricted by the same safeguards.
With U.S. lawmakers increasingly scrutinizing the adoption of open Chinese AI models some of the most advanced being open weight tech giants have launched the Open Secure AI Alliance. This initiative aims to develop and share open AI tools to improve security and transparency.
Unlike closed models, which are accessible only through specific infrastructure, open models can be downloaded, modified, and self-hosted. Nvidia stated that the alliance’s goal is to address vulnerabilities and promote the safe use of open, frontier AI systems, emphasizing that cyber defenses require open, agentic AI for self-protection.
Source (CNBC)
