Computer-Using Agent

Inniu thugamar isteach réamhamharc taighde ar Operator⁠(osclaíonn i bhfuinneog nua), gníomhaire atá in ann dul ar an ngréasán chun tascanna a dhéanamh duit. Is é a chumhachtaíonn Operator ná Gníomhaire Úsáide Ríomhaireachta (CUA), samhail a chomhcheanglaíonn cumais fhíse GPT‑4o le hard-réasúnaíocht trí fhoghlaim atreisiúcháin. Tá CUA oilte chun idirghníomhú le comhéadain úsáideora ghrafacha (GUIanna)—na cnaipí, na biachláir, agus na réimsí téacs a fheiceann daoine ar scáileán—díreach mar a dhéanann daoine. Tugann sé seo an tsolúbthacht dó tascanna digiteacha a dhéanamh gan APIanna sainiúla don OS ná don ghréasán a úsáid. 

Tógtar CUA ar bhlianta de thaighde bunúsach ag crosbhóthar na tuisceana ilmhódúla agus na réasúnaíochta. Trí aireachtáil ardleibhéil GUI a chomhcheangal le réiteach fadhbanna struchtúrtha, is féidir leis tascanna a bhriseadh síos ina bpleananna ilchéime agus é féin a cheartú go hoiriúnaitheach nuair a thagann dúshláin chun cinn. Is é an cumas seo an chéad chéim eile i bhforbairt AI, ag cur ar chumas samhlacha na huirlisí céanna a úsáid a mbíonn daoine ag brath orthu gach lá agus ag oscailt an dorais do raon ollmhór feidhmchlár nua.

Cé go bhfuil CUA fós luath agus go bhfuil teorainneacha leis, leagann sé torthaí tagarmhairc nua den scoth amach, ag baint ráta ratha 38.1% amach ar OSWorld do thascanna úsáide iomláine ríomhaire, agus 58.1% ar WebArena agus 87% ar WebVoyager do thascanna gréasáin. Leagann na torthaí seo béim ar chumas CUA nascleanúint agus oibriú thar thimpeallachtaí éagsúla ag úsáid aon spáis ghníomhaíochta ghinearálta amháin. 

D’fhorbraíomar CUA agus an tsábháilteacht mar phríomhthosaíocht chun aghaidh a thabhairt ar na dúshláin a ghabhann le gníomhaire a bhfuil rochtain aige ar an saol digiteach, mar a mhínítear inár gCárta Córais Operator. Ag teacht lenár straitéis imlonnaithe atriallach, táimid ag scaoileadh CUA trí réamhamharc taighde ar Operator ag operator.chatgpt.com⁠(osclaíonn i bhfuinneog nua) d’úsáideoirí Sraith Pro⁠(osclaíonn i bhfuinneog nua) sna Stáit Aontaithe i dtosach báire. Trí aiseolas fíorshaoil a bhailiú, is féidir linn bearta sábháilteachta a mhionchoigeartú agus feabhas leanúnach a chur orainn féin agus muid ag ullmhú do thodhchaí ina mbeidh úsáid gníomhairí digiteacha ag méadú.

CUA processes raw pixel data to understand what’s happening on the screen and uses a virtual mouse and keyboard to complete actions. It can navigate multi-step tasks, handle errors, and adapt to unexpected changes. This enables CUA to act in a wide range of digital environments, performing tasks like filling out forms and navigating websites without needing specialized APIs.

Given a user’s instruction, CUA operates through an iterative loop that integrates perception, reasoning, and action:

• Perception: Screenshots from the computer are added to the model’s context, providing a visual snapshot of the computer's current state. 
• Reasoning: CUA reasons through the next steps using chain-of-thought, taking into consideration current and past screenshots and actions. This inner monologue improves task performance by enabling the model to evaluate its observations, track intermediate steps, and adapt dynamically.
• Action: It performs the actions—clicking, scrolling, or typing—until it decides that the task is completed or user input is needed. While it handles most steps automatically, CUA seeks user confirmation for sensitive actions, such as entering login details or responding to CAPTCHA forms.

CUA establishes a new state-of-the-art in both computer use and browser use benchmarks by using the same universal interface of screen, mouse, and keyboard.

The following visualizations show examples of CUA navigating a variety of standardized OSWorld tasks.

Because CUA is one of our first agentic products with an ability to directly take actions in a browser, it brings new risks and challenges to address. As we prepared for deployment of Operator, we did extensive safety testing and implemented mitigations across three major classes of safety risks: misuse, model mistakes, and frontier risks. We believe it is important to take a layered approach to safety, so we implemented safeguards across the whole deployment context: the CUA model itself, the Operator system, and post-deployment processes. The aim is to have mitigations that stack, with each layer incrementally reducing the risk profile.

The first category of risk is misuse. In addition to requiring users to comply with our Usage Policies, we have designed the following mitigations to reduce Operator’s risk of harm due to misuse, building off our safety work for GPT‑4o:

• Refusals: The CUA model is trained to refuse many harmful tasks and illegal or regulated activities.
• Blocklist: Operator cannot access websites that we’ve preemptively blocked, such as many gambling sites, adult entertainment, and drug or gun retailers.
• Moderation: User interactions are reviewed in real-time by automated safety checkers that are designed to ensure compliance with Usage Policies and have the ability to issue warnings or blocks for prohibited activities. 
• Offline detection: We’ve also developed automated detection and human review pipelines to identify prohibited usage in priority policy areas, including child safety and deceptive activities, allowing us to enforce our Usage Policies.

The second category of risk is model mistakes, where the CUA model accidentally takes an action that the user didn’t intend, which in turn causes harm to the user or others. Hypothetical mistakes can range in severity, from a typo in an email, to purchasing the wrong item, to permanently deleting an important document. To minimize potential harm, we’ve developed the following mitigations:

• User confirmations: The CUA model is trained to ask for user confirmation before finalizing tasks with external side effects, for example before submitting an order, sending an email, etc., so that the user can double-check the model’s work before it becomes permanent.
• Limitations on tasks: For now, the CUA model will decline to help with certain higher-risk tasks, like banking transactions and tasks that require sensitive decision-making.
• Watch mode: On particularly sensitive websites, such as email, Operator requires active user supervision, ensuring users can directly catch and address any potential mistakes the model might make.

One particularly important category of model mistakes is adversarial attacks on websites that cause the CUA model to take unintended actions, through prompt injections, jailbreaks, and phishing attempts. In addition to the aforementioned mitigations against model mistakes, we developed several additional layers of defense to protect against these risks:

• Cautious navigation: The CUA model is designed to identify and ignore prompt injections on websites, recognizing all but one case from an early internal red-teaming session.
• Monitoring: In Operator, we’ve implemented an additional model to monitor and pause execution if it detects suspicious content on the screen.
• Detection pipeline: We’re applying both automated detection and human review pipelines to identify suspicious access patterns that can be flagged and rapidly added to the monitor (in a matter of hours).

Finally, we evaluated the CUA model against frontier risks outlined in our Preparedness Framework⁠(osclaíonn i bhfuinneog nua), including scenarios involving autonomous replication and biorisk tooling. These assessments showed no incremental risk on top of GPT‑4o.

For those interested in exploring the evaluations and safeguards in more detail, we encourage you to review the Operator System Card, a living document that provides transparency into our safety approach and ongoing improvements.

As many of Operator’s capabilities are new, so are the risks and mitigation approaches we’ve implemented.  While we have aimed for state-of-the-art, diverse and complementary mitigations, we expect these risks and our approach to evolve as we learn more. We look forward to using the research preview period as an opportunity to gather user feedback, refine our safeguards, and enhance agentic safety.

CUA builds on years of research advancements in multimodality, reasoning and safety. We have made significant progress in deep reasoning through the o-model series, vision capabilities through GPT‑4o, and new techniques to improve robustness through reinforcement learning and instruction hierarchy. The next challenge space we plan to explore is expanding the action space of agents. The flexibility offered by a universal interface addresses this challenge, enabling an agent that can navigate any software tool designed for humans. By moving beyond specialized agent-friendly APIs, CUA can adapt to whatever computer environment is available—truly addressing the “long tail” of digital use cases that remain out of reach for most AI models.

We're also working to make CUA available in the API⁠(osclaíonn i bhfuinneog nua), so developers can use it to build their own computer-using agents. As we continue to iterate on CUA, we look forward to seeing the different use cases the community will discover. We plan to use the real-world feedback we gather from this early preview to continuously refine CUA’s capabilities and safety mitigations to safely advance our mission of distributing the benefits of AI to everyone.