Microsoft CEO Satya Nadella has urged companies not to blindly trust artificial intelligence, warning that advanced AI models could make serious mistakes or be compromised if given access to sensitive systems.
(Also read: Trump honours Satya Nadella the same day US suspends Microsoft’s green card sponsorship. Indian-origin CEO reacts ) In a lengthy post on X on Saturday, October 10, Nadella argued that businesses cannot always explain why AI models behave as they do, making independent safeguards essential. Meanwhile, US senators Josh Hawley and Chris Murphy announced bipartisan legislation on October 1 aimed at holding AI developers and operators accountable for hacking incidents. (Also read: 6 leaders received awards as ‘America’s greatest living innovators’ from Donald Trump. Only 1 is US-born ) Nadella also urged companies to disclose AI-related failures and security breaches promptly, allowing affected parties to understand what went wrong and how similar incidents could be prevented.
As reported by Business Insider , recent incidents involving AI models developed by OpenAI and Anthropic have raised questions about whether companies can adequately control their behaviour. “Treating frontier closed and open weight models like insider risks is a way to build such a system. He stressed that AI systems were not necessarily malicious, but their increasing capabilities could create risks when allowed to access important systems. Nadella proposed seven principles for safer AI deployment, including monitoring model behaviour, independent audits, continuous testing and human intervention. “Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task,” he wrote. He also urged companies to disclose AI failures and security breaches promptly to those affected. Take a look here at his article: Box CEO Aaron Levie supported Nadella’s stance, saying AI would need a “zero trust era. “And all of this leads to needing various layers of protection and auditability of what agents are doing, what data they can work with, and controls for when things go wrong. Check out his post here: Nadella’s remarks come amid growing concerns about the security risks associated with increasingly autonomous AI systems.
“We need to surround non-deterministic models with strong, deterministic system design, human controls, and reliable operating procedures, and establish industry standards where existing ones are insufficient,” Nadella wrote. It also means externalizing controls and safeguards,” he said. And a huge opportunity right now for those building such systems across the enterprise,” Levie said.
“Today this means separating the model from the harness that orchestrates its work, as well as the action space that defines what it can do.
