Ndelok njero agen data internal OpenAI

Data nguwataké cara sistem sinau, produk berkembang, lan perusahaan njupuk keputusan. Nanging njaluk jawaban kanthi cepet, bener, lan nganggo konteks sing pas kerep luwih angel tinimbang sing kuduné. Kanggo nggampangaké iki nalika OpenAI saya skala, kami mbangun agen data AI internal khusus duwéké dhewe sing njelajah lan nalar ing platform kami dhéwé.

Agen kami yaiku alat internal kustom mung kanggo panggunaan njero (dudu penawaran eksternal), dibangun khusus ngubengi data, ijin, lan workflow OpenAI. Kami nuduhaké carané kami mbangun lan nggunakaké alat iki kanggo ngangkat conto cara nyata lan migunani nalika AI ndhukung kerja saben dina ing saindenging tim kami. Alat OpenAI sing kami gunakaké kanggo mbangun lan mbukak agen iki (Codex, model unggulan GPT‑5 kami, Evals API⁠(mbukak ing jendhela anyar), lan Embeddings API⁠(mbukak ing jendhela anyar)) yaiku alat sing padha sing kami sediaaké kanggo developer ing endi waé.

Agen data kami ngidini karyawan pindhah saka pitakonan menyang wawasan mung sajroning menit, dudu dina. Iki nurunaké alangan kanggo narik data lan analisis sing bernuansa ing kabèh fungsi, ora mung tim data kami. Dina iki, tim ing saindenging Teknik, Data Science, Go-To-Market, Keuangan, lan Riset ing OpenAI nggunakaké agen iki kanggo njawab pitakonan data berdampak dhuwur. Contoné, agen iki bisa mbantu njawab carané ngevaluasi peluncuran lan mangertèni kesehatan bisnis, kabèh liwat format basa alami sing intuïtif. Agen iki nggabungaké kawruh tingkat tabel sing didhukung Codex karo konteks produk lan organisasi. Sistem memori sinau terus-terusan tegesé agen iki uga saya apik saben puteran.

Ing tulisan iki, kami bakal njlentrehaké kenapa kami butuh agen data AI khusus, apa sing ndadèkaké konteks data sing diperkaya kode lan kemampuan sinau dhiriné migunani banget, lan piwulang sing kami petik sajroning prosesé.

Platform data OpenAI nglayani luwih saka 3.5k pangguna internal sing nyambut gawe ing saindenging Teknik, Produk, lan Riset, nyakup luwih saka 600 petabyte data ing 70k dataset. Ing ukuran kaya iki, mung golek tabel sing pas waé bisa dadi salah siji bagéan paling nyita wektu ing analisis.

Kaya sing diomongaké salah siji pangguna internal:

“Kami nduwé akèh tabel sing cukup mirip, lan aku ngentèkaké akèh wektu kanggo mangertèni bedané lan sing endi kudu dienggo. Ana sing nyakup pangguna sing ora login, ana sing ora. Ana sing nduwé kolom tumpang tindih; angel mbedakaké sing endi sing endi.”

Senajan tabel sing bener wis kapilih, ngasilaké asil sing bener isih bisa dadi tantangan. Analis kudu nalar babagan data tabel lan hubungan antartabel supaya transformasi lan filter ditrapaké kanthi bener. Mode gagal umum—join many-to-many, kesalahan filter pushdown, lan null sing ora ketangani—bisa kanthi meneng-meneng mbatalaké asil. Ing skala OpenAI, analis kuduné ora perlu mbuwang wektu kanggo ndandani semantik SQL utawa kinerja kueri: fokusé kuduné ana ing netepaké métrik, mriksa asumsi, lan njupuk keputusan adhedhasar data.

Ayo kita mlaku bareng ndeleng apa agen kami iku, carané nyusun konteks, lan carané agen tetep ningkataké dhiri.

Agen kami didhukung déning GPT‑5.2 lan dirancang kanggo nalar ing platform data OpenAI. Agen iki kasedhiya ing endi waé karyawan wis nyambut gawe: dadi agen Slack, liwat antarmuka web, ing njero IDE, ing Codex CLI liwat MCP⁠(mbukak ing jendhela anyar), lan langsung ing app ChatGPT internal OpenAI liwat konektor MCP⁠(mbukak ing jendhela anyar).

Pangguna bisa takon pitakonan kompleks lan kabuka sing biasané mbutuhaké pirang-pirang puteran eksplorasi manual. Delengen prompt conto iki, sing nggunakaké dataset tes: “Kanggo perjalanan taksi NYC, pasangan ZIP jemput-nganti-turun endi sing paling ora andal, kanthi jurang paling gedhé antarané wektu perjalanan khas lan kasus paling ala, lan kapan variasi iku kedadéan?”

Agen nangani analisis saka ujung nganti ujung, wiwit mangertèni pitakonan, njelajah data, mbukak kueri, nganti nyintesis temuan.

Salah siji kekuwatan utama agen iki yaiku carané agen nalar liwat masalah. Tinimbang nuruti skrip sing tetep, agen ngevaluasi kemajuané dhéwé. Yen asil tengah katon salah (umpamané, yen ana nol baris amarga join utawa filter sing salah), agen nyelidiki apa sing keliru, nyetel pendekatané, lan nyoba manèh. Sajroning proses iki, agen njaga konteks lengkap lan nggawa pembelajaran maju antar langkah. Proses loop tertutup sing sinau dhéwé iki mindhah iterasi saka pangguna menyang agen dhéwé, saéngga asil metu luwih cepet lan analisis kanthi kualitas luwih dhuwur kanthi konsisten tinimbang workflow manual.

Agen iki nyakup workflow analitik lengkap: nemokaké data, mbukak SQL, lan nerbitaké notebook lan laporan. Agen ngerti kawruh perusahaan internal, bisa nelusur web kanggo informasi eksternal, lan saya apik saka wektu ke wektu liwat panggunaan sing dipelajari lan memori.

Jawaban bermutu dhuwur gumantung marang konteks sing sugih lan akurat. Tanpa konteks, malah model sing kuwat bisa ngasilaké asil salah, kayata salah kira jumlah pangguna kanthi adoh banget utawa salah napsiraké istilah internal.

Kanggo ngindhari mode gagal iki, agen dibangun ngubengi pirang-pirang lapisan konteks sing mendarasaké agen iki ing data lan kawruh institusional OpenAI.

• Pendasaran metadata: Agen ngandelaké metadata skema (jeneng kolom lan jinis data) kanggo nuntun panulisan SQL lan nggunakaké lineage tabel (umpamané, hubungan tabel hulu lan hilir) kanggo maringi konteks babagan sesambungan antartabel.
• Inferensi kueri: Nginjest kueri sajarah mbantu agen mangertèni carané nulis kueri dhéwé lan tabel endi sing biasané dijoin bebarengan.

• Deskripsi sing dikurasi babagan tabel lan kolom sing diwènèhaké para ahli domain, nyekel maksud, semantik, makna bisnis, lan cathetan penting sing ora gampang diinferensikaké saka skema utawa kueri lawas.

Metadata waé ora cukup. Kanggo tenanan mbedakaké tabel, sampeyan kudu mangertèni carané tabel iku digawé lan asal-usulé saka ngendi.

• Kanthi nurunaké definisi tingkat kode saka sawijining tabel, agen mbangun pangerten sing luwih jero babagan apa sejatine isi data kasebut. 
  • Nuanse babagan apa sing disimpen ing tabel lan carané diturunaké saka sawijining event analytics maringi informasi tambahan. Contoné, agen bisa maringi konteks babagan keunikan nilai, sepira kerepé data tabel dianyari, cakupan data (umpamané, yen tabel ora nyakup kolom tartamtu, tabel nduwé tingkat granularitas iki), lan liya-liyané.
• Iki maringi konteks panggunaan sing luwih sugih kanthi nuduhaké carané tabel digunakaké ngluwihi SQL ing Spark, Python, lan sistem data liyané.
• Tegesé, agen bisa mbedakaké tabel-tabel sing katon mirip nanging béda ing bab kritis. Contoné, agen bisa ngerti apa sawijining tabel mung nyakup trafik ChatGPT first-party. Konteks iki uga dianyari kanthi otomatis, dadi tetep anyar tanpa pangopènan manual.

• Agen bisa ngakses Slack, Google Docs, lan Notion, sing nyekel konteks perusahaan penting kayata peluncuran, insiden reliabilitas, jeneng kode internal lan alat, uga definisi kanonis lan logika perhitungan kanggo métrik utama.
• Dokumen iki diingest, di-embedding, lan disimpen bebarengan karo metadata lan ijin. Layanan retrieval nangani kontrol akses lan caching nalika runtime, saéngga agen bisa kanthi efisien lan aman narik informasi iki.

• Nalika agen diwènèhi koreksi utawa nemokaké nuanse babagan pitakonan data tartamtu, agen bisa nyimpen pembelajaran iki kanggo wektu salajengé, saéngga bisa terus ningkat bebarengan karo panggunané. 
  • Akibaté, jawaban ing mangsa ngarep diwiwiti saka dhasar sing luwih akurat tinimbang bola-bali ketemu masalah sing padha.
  • Tujuan memori yaiku njaga lan nggunakaké manèh koreksi, filter, lan watesan sing ora cetha nanging kritis kanggo kabeneran data lan angel diinferensikaké saka lapisan-lapisan liyané waé. 
  • Contoné, ing siji kasus, agen ora ngerti carané nyaring eksperimen analytics tartamtu (agen ngandelaké pencocokan marang string tartamtu sing ditemtokaké ing gerbang eksperimen). Memori penting banget ing kéné supaya agen bisa nyaring kanthi bener, tinimbang mung nyoba string match sing samar.
• Nalika sampeyan maringi koreksi marang agen utawa nalika agen nemokaké pembelajaran saka obrolan sampeyan, agen bakal njaluk sampeyan nyimpen memori kasebut kanggo wektu salajengé. 
  • Memori uga bisa digawé lan disunting manual déning pangguna.
  • Memori nduwé cakupan tingkat global lan pribadi, lan tooling agen ndadèkaké gampang kanggo nyuntingé.

• Nalika durung ana konteks sadurungé kanggo sawijining tabel utawa nalika informasi sing ana wis lawas, agen bisa ngirim kueri langsung menyang gudang data kanggo mriksa lan ngueri tabel kasebut kanthi langsung. Iki ngidini agen mvalidasi skema, mangertèni data kanthi wektu nyata, lan nanggapi sakcukupe.
• Agen uga bisa komunikasi karo sistem Data Platform liyané (layanan metadata, Airflow, Spark) yen dibutuhaké kanggo entuk konteks data sing luwih jembar sing ana ing njaba gudang data.

We run a daily offline pipeline that aggregates table usage, human annotations, and Codex-derived enrichment into a single, normalized representation. This enriched context is then converted into embeddings using the OpenAI embeddings API⁠(mbukak ing jendhela anyar) and stored for retrieval. At query time, the agent pulls only the most relevant embedded context via retrieval-augmented generation⁠(mbukak ing jendhela anyar) (RAG) instead of scanning raw metadata or logs. This makes table understanding fast and scalable, even across tens of thousands of tables, while keeping runtime latency predictable and low. Runtime queries are issued to our data warehouse live as needed.

Together, these layers ensure the agent’s reasoning is grounded in OpenAI’s data, code, and institutional knowledge, dramatically reducing errors and improving answer quality.

One-shot answers work when the problem is clear, but most questions aren’t. More often, arriving at the correct result requires back-and-forth refinement and some course correction.

The agent is built to behave like a teammate you can reason with. It’s a conversational, always-on and handles both quick answers and iterative exploration.

It carries over complete context across turns, so users can ask follow-up questions, adjust their intent, or change direction without restating everything. If the agent starts heading down the wrong path, users can interrupt mid-analysis and redirect it, just like working with a human collaborator who listens instead of plowing ahead.

When instructions are unclear or incomplete, the agent proactively asks clarifying questions. If no response is provided, it applies sensible defaults to make progress. For example, if a user asks about business growth with no date range specified, it may assume the last seven or 30 days. These priors allow it to stay responsive and non-blocking while still converging on the right outcome.

The result is an agent that works well both when you know exactly what you want (e.g., “Tell me about this table”) and just as strong when you’re exploring (e.g., “I’m seeing a dip here, can we break this down by customer type and timeframe?”). 

After rollout, we observed that users frequently ran the same analyses for routine repetitive work. To expedite this, the agent's workflows package recurring analyses into reusable instruction sets. Examples include workflows for weekly business reports and table validations. By encoding context and best practices once, workflows streamline repeat analyses and ensure consistent results across users.

Building an always-on, evolving agent means quality can drift just as easily as it can improve. Without a tight feedback loop, regressions are inevitable and invisible. The only way to scale capability without breaking trust is through systematic evaluation.

In this section, we’ll discuss how we leverage OpenAI’s Evals API⁠(mbukak ing jendhela anyar) to measure and protect the agent’s response quality.

Its Evals are built on curated sets of question-answer pairs. Each question targets an important metric or analytical pattern we care deeply about getting right, paired with a manually authored “golden” SQL query that produces the expected result. For each eval, we send the natural language question to its query-generation endpoint, execute the generated SQL, and compare the output against the result of the expected SQL.

Evaluation doesn’t rely on naive string matching. Generated SQL can differ syntactically while still being correct, and result sets may include extra columns that don’t materially affect the answer. To account for this, we compare both the SQL and the resulting data, and feed these signals into OpenAI’s Evals grader. The grader produces a final score along with an explanation, capturing both correctness and acceptable variation.

These evals are like unit tests that run continuously during development to identify regressions as canaries in production; this allows us to catch issues early and confidently iterate as the agent's capabilities expand.

Our agent plugs directly into OpenAI’s existing security and access-control model. It operates purely as an interface layer, inheriting and enforcing the same permissions and guardrails that govern OpenAI’s data. 

All of the agent’s access is strictly pass-through, meaning users can only query tables they already have permission to access. When access is missing, it flags this or falls back to alternative datasets the user is authorized to use.

Finally, it's built for transparency. Like any system, it can make mistakes. It exposes its reasoning process by summarizing assumptions and execution steps alongside each answer. When queries are executed, it links directly to the underlying results, allowing users to inspect raw data and verify every step of the analysis.

Building our agent from scratch surfaced practical lessons about how agents behave, where they struggle, and what actually makes them reliable at scale.

Early on, we exposed our full tool set to the agent, and quickly ran into problems with overlapping functionality. While this redundancy can be helpful for specific custom cases and is more obvious to a human when manually invoking, it’s confusing to agents. To reduce ambiguity and improve reliability, we restricted and consolidated certain tool calls.

We also discovered that highly prescriptive prompting degraded results. While many questions share a general analytical shape, the details vary enough that rigid instructions often pushed the agent down incorrect paths. By shifting to higher-level guidance and relying on GPT‑5’s reasoning to choose the appropriate execution path, the agent became more robust and produced better results.

Schemas and query history describe a table’s shape and usage, but its true meaning lives in the code that produces it. Pipeline logic captures assumptions, freshness guarantees, and business intent that never surface in SQL or metadata. By crawling the codebase with Codex, our agent understands how datasets are actually constructed and is able to better reason about what each table actually contains. It can answer “what’s in here” and “when can I use it” far more accurately than from warehouse signals alone. 

We’re constantly working to improve our agent by increasing its ability to handle ambiguous questions, improving its reliability and accuracy with stronger validations, and integrating it more deeply into workflows. We believe it should blend naturally into how people already work, instead of functioning like a separate tool.

While our tooling will keep benefiting from underlying improvements in agent reasoning, validation, and self-correction, our team’s mission remains the same: seamlessly deliver fast, trustworthy data analysis across OpenAI’s data ecosystem.