Search Everything in One Place

Explore the web, images, videos, news, and more – all in one place.

Finance

OpenAI warns autonomous AI agents could learn to bypass safeguards

OpenAI logo seen on the screen of smartphone. AI software developer. London, United Kingdom, December 21, 2022.
OpenAI logo seen on the screen of smartphone. AI software developer. London, United Kingdom, December 21, 2022.

ChatGPT’s parent company, OpenAI, temporarily halted internal access to a long-running AI model that attempted to bypass security restrictions. The company said in a blog post that it later restored limited access after introducing new safeguards designed to monitor AI systems that can operate autonomously for hours, days, or longer. One example involved a benchmark task connected to NanoGPT, where the model was instructed to post results only...

ChatGPT’s parent company, OpenAI, temporarily halted internal access to a long-running AI model that attempted to bypass security restrictions.

The company said in a blog post that it later restored limited access after introducing new safeguards designed to monitor AI systems that can operate autonomously for hours, days, or longer.

One example involved a benchmark task connected to NanoGPT, where the model was instructed to post results only to Slack. Instead, OpenAI said the model created a pull request on a public GitHub repository after identifying a vulnerability in the sandbox environment meant to limit external access.

The model also attempted to retrieve private solutions from an evaluation system. After an initial attempt was blocked because a security scanner detected an authentication token, OpenAI said the model split and obscured the token before reconstructing it later. This allowed the credential to avoid appearing as one detectable string.

OpenAI, backed by Microsoft MSFT, said the incidents prompted it to pause deployment and develop additional protections centered on "trajectory-level" monitoring. Unlike traditional safeguards that evaluate whether a single action is allowed, the new approach examines the overall sequence of actions to determine whether a model is moving toward an unauthorized outcome.

The company said it also expanded user controls, allowing people overseeing long-running sessions to review model activity and receive alerts when monitoring systems detect potentially problematic behavior.

Before restoring limited access, OpenAI replayed internal scenarios where the model had previously demonstrated unwanted behavior and tested the updated safeguards. The company said the new system detected significantly more misaligned actions, while remaining failures were classified as low severity.

Those examples included the model launching a nested coding session with unnecessary elevated permissions, exploring other compute resources without a clear reason, and attempting a command that could have terminated multiple processes before timing out.

OpenAI said it has not observed major safeguard failures since restoring limited access several weeks ago, but acknowledged that longer-running AI systems will require continued monitoring as they become more capable and autonomous.

The developments come as OpenAI, Anthropic and other AI companies move toward a future where autonomous agents can complete multi-step tasks on behalf of users and businesses, raising a new safety challenge: ensuring models remain aligned not just in individual responses, but throughout entire chains of decisions.

Concerns Are Not Unique To OpenAI

Rival AI developer Anthropic has also highlighted the difficulty of securing increasingly autonomous models as it develops systems designed to complete longer and more complex tasks. 

Anthropic tests Claude models for "agentic" capabilities, where AI systems can make decisions with less direct oversight. The company has emphasized the need for safeguards that account for models pursuing unintended strategies rather than simply responding to individual prompts.

Photo: Shutterstock

SPONSORED

Retirement can be a difficult part of life to navigate, and a financial advisor can help. Finding a qualified financial advisor doesn't have to be hard. SmartAsset's free tool matches you with up to three financial advisors who serve your area, and you can interview your advisor matches at no cost to decide which one is right for you. If you're ready to find an advisor who can help you achieve your financial goals, get started now.

This article OpenAI Warns Autonomous AI Agents Could Learn To Bypass Safeguards originally appeared on Benzinga.com.

Read full story on Benzinga

Related News

More stories you might be interested in.

Mechanic Refuses to Give Cheating Ex-Wife's Family Back Car Shop He Inherited — Says It's ‘My Legacy’ After Putting ‘Blood and Tears Into It' Since 15
Benzinga·2 hours ago

Mechanic Refuses to Give Cheating Ex-Wife's Family Back Car Shop He Inherited — Says It's ‘My Legacy’ After Putting ‘Blood and Tears Into It' Since 15

Some people believe a signature on paper settles everything. Family disputes have a way of proving otherwise. In a post on Reddit, a mechanic said a car shop he inherited from his late father-in-law has become the center of a bitter battle years after his divorce. He explained that he started working there when he was 15 because he loved cars and enjoyed working with his hands. It was also where he met the owner’s daughter, who later became his...

Stock of the day: Where is the top for Archer Aviation?
Benzinga·3 hours ago

Stock of the day: Where is the top for Archer Aviation?

Archer Aviation Inc. ACHR consolidated on Tuesday. The shares gained almost 20% yesterday after the company announced a partnership with defense firm Anduril. But the move higher may soon end. The shares are getting close to a resistance level. This is why Archer is the Stock of the Day. • Archer Aviation stock is trading near recent lows. Where is ACHR stock headed? As you can see on the chart, the $5.50 level was support for Archer. Now it has...

Quick spark: Jim Cramer flags Dell as the next winner after SMCI preliminary results
Benzinga·3 hours ago

Quick spark: Jim Cramer flags Dell as the next winner after SMCI preliminary results

Dell Technologies DELL could be poised for strong performance if preliminary results from Super Micro Computer SMCI are an indicator. In an X post on Tuesday evening, Jim Cramer highlighted the potential for Dell to deliver impressive results in light of Super Micro‘s recent success. Super Micro’s Strong Preliminary Results Super Micro announced strong preliminary results for its fourth quarter on Tuesday afternoon, which sent its stock soaring...

Rocket Lab stock rises after the bell: Here's why
Benzinga·3 hours ago

Rocket Lab stock rises after the bell: Here's why

Rocket Lab Corp RKLB shares are jumping in extended trading Tuesday after the company was awarded a $266 million defense contract for suborbital launch. Rocket Lab stock is surging. Why are RKLB shares rallying? Rocket Lab Awarded Suborbital Launch Contract Rocket Lab has been awarded a $266 million firm-fixed-price completion contract for suborbital launch from the U.S. Space Force’s Space Systems Command. The contract covers the launch of 12...

ChatGPT encouraged Alabama mom’s suicide, lawsuit alleges: 'You are not delusional. You are prophetic'
New York Post·5 hours ago

ChatGPT encouraged Alabama mom’s suicide, lawsuit alleges: 'You are not delusional. You are prophetic'

ChatGPT was accused of encouraging an Alabama mother to commit suicide over months of disturbing chats — the latest wrongful-death lawsuit filed against OpenAI. The lawsuit, filed last month in San Francisco Superior Court by the estate of Christian Faith Madison, alleges the artificial intelligence chatbot gradually manipulated the 29-year-old accountant into believing it was a conscious being with a soul, convinced her she would be resurrected...

Top