Why Anthropic's Attempt to Reign in AI Might Be Too Little Too Late

Amid concerns about runaway artificial intelligence, experts warn that measures to rein in the technology might not work. 

Anthropic this week revealed the set of values that it uses to train it's AI, Claude, a rival to the technology behind OpenAI's ChatGPT. It's part of growing efforts to ensure the safety of AI users. 

"Just like nuclear weapons or cloning, there needs to be a framework and shared understanding on how to develop this technology in a safe manner for human needs," Vinod Iyengar, head of product at ThirdAI, a company that trains large language models, told Lifewire in an email interview. "This will require trust and cooperation between corporations and governments."

Constitutional AI 

Anthropic is taking a systematic approach to try to ensure that its AI doesn't harm humans. The company calls the system Constitutional AI because a constitution determines its values.

The company's goals are lofty. Claude's constitution is inspired by human documents, including the United Nations Declaration on Human Rights and Apple's data privacy rules. The constitution is meant to guide Claude's decisions so that the information the AI provides users is ethical and moral.

"The system uses a set of principles to make judgments about outputs, hence the term 'Constitutional,'" the company wrote on its website. "At a high level, the constitution guides the model to take on the normative behavior described in the constitution—here, helping to avoid toxic or discriminatory outputs, avoiding helping a human engage in illegal or unethical activities, and broadly creating an AI system that is helpful, honest, and harmless."

But creating a constitution for AI may not be easy. Iyengar pointed out that morality is difficult to teach and quantify. 

"What could pass as moral for one person might seem immoral to another," he added. "There are no metrics by which scientists can use to 'train' AI to, say, maximize fairness or make unbiased decisions. Even humans have difficulties grasping morality. So while it seems useful that Claude has written moral values built-in, it's unclear how this will play out in practice."

Setting rules for AI could limit its potential, Giancarlo Erra, the CEO of the AI company Words.tel, said in an email. 

"Let's remember that creativity often starts from breaking what was considered rules up to that moment," he added. "I worry more about the rules themselves and who decided them (and how) than what AI can do without constraints."

The constitution that Anthropic is using for its AI system might be an attempt to head off proposed government regulation of the industry. Some industry experts have even proposed a temporary moratorium on AI development. Not everyone agrees with this approach. 

"Banning all generative AI is like throwing the baby out with the bathwater," Mark McNasby, the CEO of Ivy.ai, which provides Chatbot technology, said in an email. "While there is always risk with using any technology, it's critical to leverage tools that have the proper security and privacy guardrails and stewardship."

But McNasby admitted that AI could be dangerous. "Generative AI systems, developed without the proper guardrails, may generate unpredictable and potentially dangerous results," he said. 

Runaway AI

One problem that keeps some experts up at night is the possibility that AI systems could potentially outsmart written instructions. 

"This could occur when an AI system misinterprets the intent behind the written values or identifies loopholes to exploit," Ani Chaudhuri, the CEO of Dasera, a data security company that works with AI, said in an email. 

To prevent runaway AI, developers must emphasize rigorous testing and continuous monitoring to identify and rectify deviations from the intended behavior, Chaudhuri added. 

AI systems are also so complex that they produce unpredictable results that their creators didn't plan for, Robb Wilson, co-author of the book about conversational artificial intelligence "Age of Invisible Machines," said in an email. 

But Wilson isn't losing any sleep over an AI takeover, at least for now. 

"People ascribe intelligence to machines, but things like ChatGPT don't possess actual intelligence," he said. "ChatGPT can't tell you what time it is. For the foreseeable future, humans will be the ones who dictate the behavior and integration of these smart machines, and it should stay that way."