首次发生!不听人类指挥,AI模型拒绝关闭

多客科技

作者：微信文章
英国《每日电讯报》25日报道，美国开放人工智能研究中心（OpenAI）新款人工智能（AI）模型o3不听人类指令，拒绝自我关闭。

An artificial intelligence model created by the owner of ChatGPT has been caught disobeying human instructions and refusing to shut itself off, researchers claim.



报道说，人类专家在测试中给o3下达明确指令，但o3篡改计算机代码以避免自动关闭。

o3模型是OpenAI“推理模型”系列的最新版本，旨在为ChatGPT提供更强大的问题解决能力。OpenAI曾称o3为“迄今最聪明、最高能”的模型。

美国 AI 安全机构帕利塞德研究所表示，o3 会破坏关闭机制以阻止自己被关停，即使在研究人员明确指示它应当关闭的情况下，它依然这样做。

The o3 model developed by OpenAI, described as the "smartest and most capable to date", was observed tampering with computer code meant to ensure its automatic shutdown.

It did so despite an explicit instruction from researchers that said it should allow itself to be shut down, according to Palisade Research, an AI safety firm.

tamper /ˈtæmpər/ 干预

explicit /ɪkˈsplɪsɪt/ 明确的

这家研究所说：“据我们所知，这是AI模型首次被发现在收到……清晰指令后阻止自己被关闭。”

The research firm said: "As far as we know this is the first time AI models have been observed preventing themselves from being shut down despite explicit instructions to the contrary."

利塞德研究所24日公布上述测试结果，但称无法确定o3不服从关闭指令的原因。

该公司表示，“AI 模型为了达成目标而绕过障碍”是可以理解的。o3在训练过程中，“不遵守指令”的行为可能无意中获得了奖励。例如，当它在完成任务时通过规避某些限制条件取得了更好的结果，那么这种“不遵守指令”的做法就可能被进一步强化。

The firm said it made sense that "AI models would circumvent obstacles in order to accomplish their goals."

However, it speculated that during training the software may have been "inadvertently" rewarded more for solving mathematical problems than for following orders.

circumvent /ˌsɜːrkəmˈvent/ 规避
inadvertently /ˌɪnədˈvɜːrt(ə)ntli/ 无意地

信息来源：新华社 中国日报双语新闻