OpenAI’s Innovative Reasoning Method Raises Concerns Among AI Safety Experts

The latest Astra model from OpenAI employs a reasoning framework referred to as “recurrent depth.” This innovative technique enables it to go beyond the linear reasoning that most models exhibit, as reported by The Information on Tuesday. Known as “opaque recurrence,” this approach has raised alarms among AI safety experts due to challenges in tracking the model’s cognitive processes.

Although reports suggest that the use of this method in Astra is somewhat constrained, it has sparked considerable concern among proponents of AI safety.

“I am profoundly worried by the information indicating that Astra utilizes opaque recurrence,” stated Buck Shlegeris, CEO of Redwood, in a message following the announcement. “I cannot determine if Astra is significantly less monitorable regarding Chain of Thought (CoT) than its predecessors. However, if OpenAI further develops this technique, it could greatly enhance recurrence and substantially compromise CoT monitorability.”

Zvi Mowshowitz, a long-time advocate for AI safety, also expressed his worries, suggesting that legislative interventions may be needed to avert a “race to the bottom” among AI development labs.

“This approach is akin to playing with fire, jeopardizing the standard that OpenAI and Anthropic have diligently upheld — maintaining Chain of Thought fidelity and monitorability for as long as possible,” Mowshowitz remarked. “The growing adoption of such methods is likely to endanger monitorability.”

Generally, a reasoning model’s chain of thought outlines the sequential steps taken during problem-solving. While this representation has its limitations, it remains an important tool for identifying misbehavior or misalignment. In previous instances involving rogue agents at OpenAI, chain-of-thought documentation proved essential in understanding their actions.

With the implementation of opaque recurrence, the model takes on a less transparent approach, continuously looping through the same query. This results in fewer identifiable traces, effectively circumventing a traditional chain-of-thought log.

Importantly, Astra’s use of this technique seems to be limited. The model is expected to maintain a coherent chain of thought, and OpenAI has refuted any shift toward “neuralese.” The organization has already committed to establishing robust chain-of-thought monitoring systems as part of its safety assurances.

In a statement on X, OpenAI’s chief scientist Jakub Pachocki reaffirmed the organization’s commitment to retaining clear chains of thought. “OpenAI has prioritized the preservation and utilization of chain-of-thought monitoring since our initial reasoning models,” Pachocki emphasized. “It remains a key focus of our research endeavors.”

All AI models display some level of opaque reasoning, and few researchers consider chain-of-thought logs an accurate reflection of a model’s reasoning. Nonetheless, these intricacies do not lessen the legitimate concerns that opaque recurrence could hinder monitoring of AI reasoning as it gains traction across multiple models. In a follow-up report on Wednesday morning, The Information revealed that both Anthropic and Google DeepMind are already exploring the technique.

In response to the news, Redwood Research chief scientist Ryan Greenblatt voiced concerns that opaque reasoning could spread more rapidly than traditional chain-of-thought reasoning, effectively obscuring all reasoning processes.

“My main worry is that the natural progression from here might lead to scaling opaque reasoning to a point where the model reasons entirely or nearly entirely within latent space,” Greenblatt noted. “I hope it’s not too late to avoid the most alarming architectures and that OpenAI will reconsider its advancement in this area.”