Ndani ya wakala wa data wa ndani wa OpenAI

Data huwezesha jinsi mifumo inavyojifunza, bidhaa zinavyobadilika, na jinsi makampuni yanavyofanya maamuzi. Lakini kupata majibu haraka, kwa usahihi, na katika muktadha unaofaa mara nyingi ni vigumu zaidi kuliko inavyopaswa kuwa. Ili kurahisisha hili kadri OpenAI inavyopanuka, tuliunda wakala wetu maalum wa data wa AI wa ndani anayechunguza na kutoa hoja kuhusu jukwaa letu wenyewe.

Wakala wetu ni zana maalum ya ndani pekee (si toleo la nje), iliyoundwa mahsusi kwa data, ruhusa, na taratibu za kazi ya OpenAI. Tunaonyesha jinsi tulivyounda na kuitumia ili kusaidia kuibua mifano ya njia halisi na zenye matokeo ambazo AI inaweza kusaidia kazi za kila siku katika timu zetu. Zana za OpenAI tulizotumia kuunda na kuendesha (Codex, muundo mkuu wa GPT‑5, Evals API⁠(fungua katika dirisha jipya), na Embeddings API⁠(fungua katika dirisha jipya)) ni zana zilezile tunazozifanya zipatikane kwa wasanidi programu kila mahali.

Wakala wetu wa data huwawezesha wafanyakazi kwenda kutoka swali hadi maarifa kwa dakika chache, si siku. Hii inapunguza kizingiti cha kuvuta data na uchanganuzi wa kina katika kazi zote, si tu na timu yetu ya data. Leo, timu za Uhandisi, Sayansi ya Data, Mkakati wa Kuingia Sokoni, Fedha, na Utafiti katika OpenAI zinategemea wakala kujibu maswali ya data yenye matokeo kubwa. Kwa mfano, inaweza kusaidia kujibu jinsi ya kutathmini uzinduzi na kuelewa hali ya biashara, yote kupitia muundo angavu wa lugha ya asili. Wakala huchanganya maarifa yanayoendeshwa na Codex katika kiwango cha jedwali pamoja na muktadha wa bidhaa na shirika. Mfumo wake wa kumbukumbu unaojifunza kila wakati unamaanisha pia unaboreka kwa kila hatua.

Katika chapisho hili, tutachambua kwa nini tulihitaji wakala maalum wa data wa AI, ni nini kinachofanya muktadha wake wa data ulioboreshwa kwa msimbo na uwezo wa kujifunza wenyewe kuwa wa manufaa sana, na masomo tuliyojifunza njiani.

Jukwaa la data la OpenAI linahudumia zaidi ya watumiaji wa ndani 3.5k wanaofanya kazi katika Uhandisi, Bidhaa, na Utafiti, ikijumuisha zaidi ya petabaiti 600 za data katika seti za data 70k. Kwa ukubwa huo, kupata tu jedwali sahihi kunaweza kuwa mojawapo ya sehemu zinazochukua muda mwingi zaidi katika kufanya uchambuzi.

Kama mtumiaji mmoja wa ndani alivyosema:

“Tuna majedwali mengi yanayofanana kwa kiasi fulani, na ninatumia muda mwingi sana kujaribu kufahamu zinatofautianaje na ni ipi ya kutumia. Baadhi zinajumuisha watumiaji ambao waliondoka kwenye akaunti, baadhi hazijumuishi. "Baadhi zina sehemu zinazofanana; ni vigumu kutambua ni kipi ni kipi.”

Hata ukiwa umechagua majedwali sahihi, kutoa matokeo sahihi kunaweza kuwa changamoto. Wachambuzi lazima watafakari kuhusu data ya jedwali na uhusiano wa majedwali ili kuhakikisha mabadiliko na vichujio vinatumika kwa usahihi. Njia za kawaida za kushindwa—mahusiano ya majedwali mengi, makosa ya kushinikiza chujio, na batili zisizoshughulikiwa—zinaweza kubatilisha matokeo kimya kimya. Kwa kiwango cha OpenAI, wachambuzi hawapaswi kulazimika kupoteza muda katika kutatua hitilafu za semantiki za SQL au utendaji wa maswali: lengo lao linapaswa kuwa kufafanua vipimo, kuthibitisha dhana, na kufanya maamuzi yanayotokana na data.

Hebu tuangalie wakala wetu ni nani, jinsi anavyopanga muktadha, na jinsi anavyoendelea kujiboresha.

Wakala wetu anaendeshwa na GPT‑5.2 na ameundwa kufanya uchanganuzi kwenye jukwaa la data la OpenAI. Anapatikana popote ambapo wafanyakazi tayari hufanya kazi: kama wakala wa Slack, kupitia kiolesura cha wavuti, ndani ya IDE, katika Codex CLI kupitia MCP⁠(fungua katika dirisha jipya), na moja kwa moja katika programu ya ndani ya ChatGPT ya OpenAI kupitia kiunganishi cha MCP⁠(fungua katika dirisha jipya).

Watumiaji wanaweza kuuliza maswali magumu, yasiyo na majibu ya moja kwa moja ambayo kwa kawaida yangehitaji raundi nyingi za uchunguzi wako mwenyewe. Chukua mfano huu wa dokezo, unaotumia fungudata la majaribio: “Kwa safari za teksi za NYC, ni jozi zipi za ZIP za kuchukua-hadi-kushukisha ambazo si za kuaminika zaidi, zilizo na pengo kubwa zaidi kati ya nyakati za kawaida na za hali mbaya zaidi za safari, na ni lini utofauti huo hutokea?”

Wakala hushughulikia uchambuzi kutoka mwanzo hadi mwisho, kuanzia kuelewa swali hadi kuchunguza data, kuendesha maswali, na kuunganisha matokeo.

Mojawapo ya nguvu kuu za wakala ni jinsi anavyotatua matatizo. Badala ya kufuata maandishi yaliyowekwa, wakala hutathmini maendeleo yake binafsi. Ikiwa matokeo ya kati yanaonekana kuwa si sahihi (mfano, ikiwa yana safu mlalo sifuri kutokana na kuunganisha au kuchuja kusio sahihi), wakala huchunguza kilichokwenda vibaya, hurekebisha mbinu yake, na kujaribu tena. Katika mchakato huu wote, inahifadhi muktadha kamili, na kuendeleza mafunzo kati ya hatua. Mchakato huu wa mfumo otomatiki, unaojifunza wenyewe huhamisha urudiaji kutoka kwa mtumiaji hadi kwa wakala mwenyewe, na kuwezesha matokeo ya haraka na uchanganuzi thabiti wa ubora wa juu kuliko taratibu za kazi unazofanya mwenyewe.

Wakala anashughulikia utaratibu mzima wa kazi wa uchanganuzi: kugundua data, kuendesha SQL, na kuchapisha daftari na ripoti. Anaelewa maarifa ya ndani ya kampuni, anaweza kutafuta taarifa za nje kwenye wavuti, na anaboreka kadri muda unavyopita kupitia matumizi na kumbukumbu aliyojifunza.

Majibu ya ubora wa juu yanategemea muktadha mzuri na sahihi. Bila muktadha, hata miundo imara inaweza kutoa matokeo yasiyo sahihi, kama vile kukadiria vibaya sana idadi ya watumiaji au kutafsiri vibaya istilahi za ndani.

Ili kuepuka hali hizi za kushindwa, wakala ameundwa kulingana na tabaka nyingi za muktadha zinazoziweka katika data ya OpenAI na maarifa ya taasisi.

• Uegemezaji wa metadata: Wakala hutegemea metadata ya kielezo (majina ya safu wima na aina za data) kuongoza uandishi wa SQL na hutumia ufuatiliaji wa jedwali (mfano, mahusiano ya jedwali ya juu na ya chini) kutoa muktadha kuhusu jinsi majedwali tofauti yanavyohusiana.
• Uchambuzi wa maswali: Kuingiza maswali ya kihistoria humsaidia wakala kuelewa jinsi ya kuandika maswali yake mwenyewe na ni majedwali yapi kwa kawaida huunganishwa pamoja.

• Maelezo yaliyokusanywa ya majedwali na safu wima yanayotolewa na wataalamu wa eneo husika, yakieleza nia, semantiki, maana ya biashara, na tahadhari zinazojulikana ambazo si rahisi kubainika kutoka kwa kielezo au maswali ya awali.

Metadata pekee haitoshi. Ili kutofautisha majedwali kwa kweli, unahitaji kuelewa jinsi yalivyoundwa na yanatoka wapi.

• Kwa kupata ufafanuzi wa kiwango cha msimbo wa jedwali, wakala hujenga uelewa wa kina zaidi wa kile ambacho data inajumuisha. 
  • Utofautishaji wa kile kilichohifadhiwa kwenye jedwali na jinsi kinavyotokana na tukio la uchanganuzi hutoa maelezo ya ziada. Kwa mfano, inaweza kutoa muktadha kuhusu upekee wa thamani, ni mara ngapi data ya jedwali inasasishwa, upeo wa data (kwa mfano, ikiwa jedwali halijumuishi sehemu fulani, lina kiwango hiki cha kina), na kadhalika.
• Hii inatoa muktadha ulioboreshwa wa matumizi kwa kuonyesha jinsi jedwali linavyotumika zaidi ya SQL katika Spark, Python, na mifumo mingine ya data.
• Hii inamaanisha kwamba wakala anaweza kutofautisha kati ya majedwali yanayofanana kwa sura lakini yanatofautiana kwa njia muhimu. Kwa mfano, inaweza kueleza ikiwa jedwali linajumuisha trafiki ya ChatGPT ya mhusika wa kwanza pekee. Muktadha huu pia husasishwa kiotomatiki, hivyo unabaki kuwa wa kisasa bila matengenezo ya mikono.

• Wakala anaweza kufikia Slack, Hati za Google, na Dhana, ambazo zinanasa muktadha muhimu wa kampuni kama vile uzinduzi, matukio ya kuaminika, majina ya msimbo na zana za ndani, na ufafanuzi wa kisheria na mantiki ya hesabu kwa vipimo muhimu.
• Hati hizi huingizwa, kupachikwa, na kuhifadhiwa pamoja na metadata na ruhusa. Huduma ya kurejesha data hushughulikia udhibiti wa ufikiaji na uhifadhi wa data wakati wa utekelezaji, na kumwezesha wakala kuingiza taarifa hii kwa ufanisi na kwa usalama.

• Wakati wakala anapopewa masahihisho au kugundua mambo madogo madogo kuhusu maswali fulani ya data, anaweza kuhifadhi mafunzo haya kwa ajili ya wakati ujao, na kuiruhusu kuboreka kila mara na watumiaji wake. 
  • Kwa hivyo, majibu ya baadaye huanza kutoka kwenye msingi sahihi zaidi badala ya kukutana mara kwa mara na masuala yale yale.
  • Lengo la kumbukumbu ni kuhifadhi na kutumia tena masahihisho yasiyo dhahiri, vichujio, na vikwazo ambavyo ni muhimu kwa usahihi wa data lakini ni vigumu kubaini kutoka kwa tabaka nyingine pekee. 
  • Kwa mfano, katika kisa kimoja, wakala hakujua jinsi ya kuchuja kwa ajili ya jaribio fulani la uchanganuzi (alitegemea kulinganisha dhidi ya mfuatano mahususi wa herufi uliobainishwa katika lango la jaribio). Kumbukumbu ilikuwa muhimu sana hapa ili kuhakikisha kwamba ingeweza kuchuja kwa usahihi, badala ya kujaribu kulinganisha tungo .
• Unapompa wakala marekebisho au anapopata jambo la kujifunza kutoka kwenye mazungumzo yako, itakuhimiza kuhifadhi kumbukumbu hiyo kwa ajili ya wakati mwingine. 
  • Kumbukumbu pia zinaweza kuundwa na kuhaririwa na watumiaji wenyewe.
  • Kumbukumbu zimepangwa katika ngazi ya kimataifa na ya kibinafsi, na zana za wakala hufanya iwe rahisi kuzihariri.

• Wakati hakuna muktadha wa awali uliopo kwa jedwali au wakati taarifa zilizopo zimepitwa na wakati, wakala anaweza kutoa maswali ya moja kwa moja kwa hifadhi ya data ili kukagua na kuuliza jedwali moja kwa moja. Hii inaruhusu kuthibitisha vielezo, kuelewa data kwa wakati halisi, na kujibu ipasavyo.
• Wakala pia anaweza kuzungumza na mifumo mingine ya Jukwaa la Data (huduma ya metadata, Airflow, Spark) inapohitajika ili kupata muktadha mpana wa data uliopo nje ya hifadhi.

We run a daily offline pipeline that aggregates table usage, human annotations, and Codex-derived enrichment into a single, normalized representation. This enriched context is then converted into embeddings using the OpenAI embeddings API⁠(fungua katika dirisha jipya) and stored for retrieval. At query time, the agent pulls only the most relevant embedded context via retrieval-augmented generation⁠(fungua katika dirisha jipya) (RAG) instead of scanning raw metadata or logs. This makes table understanding fast and scalable, even across tens of thousands of tables, while keeping runtime latency predictable and low. Runtime queries are issued to our data warehouse live as needed.

Together, these layers ensure the agent’s reasoning is grounded in OpenAI’s data, code, and institutional knowledge, dramatically reducing errors and improving answer quality.

One-shot answers work when the problem is clear, but most questions aren’t. More often, arriving at the correct result requires back-and-forth refinement and some course correction.

The agent is built to behave like a teammate you can reason with. It’s a conversational, always-on and handles both quick answers and iterative exploration.

It carries over complete context across turns, so users can ask follow-up questions, adjust their intent, or change direction without restating everything. If the agent starts heading down the wrong path, users can interrupt mid-analysis and redirect it, just like working with a human collaborator who listens instead of plowing ahead.

When instructions are unclear or incomplete, the agent proactively asks clarifying questions. If no response is provided, it applies sensible defaults to make progress. For example, if a user asks about business growth with no date range specified, it may assume the last seven or 30 days. These priors allow it to stay responsive and non-blocking while still converging on the right outcome.

The result is an agent that works well both when you know exactly what you want (e.g., “Tell me about this table”) and just as strong when you’re exploring (e.g., “I’m seeing a dip here, can we break this down by customer type and timeframe?”). 

After rollout, we observed that users frequently ran the same analyses for routine repetitive work. To expedite this, the agent's workflows package recurring analyses into reusable instruction sets. Examples include workflows for weekly business reports and table validations. By encoding context and best practices once, workflows streamline repeat analyses and ensure consistent results across users.

Building an always-on, evolving agent means quality can drift just as easily as it can improve. Without a tight feedback loop, regressions are inevitable and invisible. The only way to scale capability without breaking trust is through systematic evaluation.

In this section, we’ll discuss how we leverage OpenAI’s Evals API⁠(fungua katika dirisha jipya) to measure and protect the agent’s response quality.

Its Evals are built on curated sets of question-answer pairs. Each question targets an important metric or analytical pattern we care deeply about getting right, paired with a manually authored “golden” SQL query that produces the expected result. For each eval, we send the natural language question to its query-generation endpoint, execute the generated SQL, and compare the output against the result of the expected SQL.

Evaluation doesn’t rely on naive string matching. Generated SQL can differ syntactically while still being correct, and result sets may include extra columns that don’t materially affect the answer. To account for this, we compare both the SQL and the resulting data, and feed these signals into OpenAI’s Evals grader. The grader produces a final score along with an explanation, capturing both correctness and acceptable variation.

These evals are like unit tests that run continuously during development to identify regressions as canaries in production; this allows us to catch issues early and confidently iterate as the agent's capabilities expand.

Our agent plugs directly into OpenAI’s existing security and access-control model. It operates purely as an interface layer, inheriting and enforcing the same permissions and guardrails that govern OpenAI’s data. 

All of the agent’s access is strictly pass-through, meaning users can only query tables they already have permission to access. When access is missing, it flags this or falls back to alternative datasets the user is authorized to use.

Finally, it's built for transparency. Like any system, it can make mistakes. It exposes its reasoning process by summarizing assumptions and execution steps alongside each answer. When queries are executed, it links directly to the underlying results, allowing users to inspect raw data and verify every step of the analysis.

Building our agent from scratch surfaced practical lessons about how agents behave, where they struggle, and what actually makes them reliable at scale.

Early on, we exposed our full tool set to the agent, and quickly ran into problems with overlapping functionality. While this redundancy can be helpful for specific custom cases and is more obvious to a human when manually invoking, it’s confusing to agents. To reduce ambiguity and improve reliability, we restricted and consolidated certain tool calls.

We also discovered that highly prescriptive prompting degraded results. While many questions share a general analytical shape, the details vary enough that rigid instructions often pushed the agent down incorrect paths. By shifting to higher-level guidance and relying on GPT‑5’s reasoning to choose the appropriate execution path, the agent became more robust and produced better results.

Schemas and query history describe a table’s shape and usage, but its true meaning lives in the code that produces it. Pipeline logic captures assumptions, freshness guarantees, and business intent that never surface in SQL or metadata. By crawling the codebase with Codex, our agent understands how datasets are actually constructed and is able to better reason about what each table actually contains. It can answer “what’s in here” and “when can I use it” far more accurately than from warehouse signals alone. 

We’re constantly working to improve our agent by increasing its ability to handle ambiguous questions, improving its reliability and accuracy with stronger validations, and integrating it more deeply into workflows. We believe it should blend naturally into how people already work, instead of functioning like a separate tool.

While our tooling will keep benefiting from underlying improvements in agent reasoning, validation, and self-correction, our team’s mission remains the same: seamlessly deliver fast, trustworthy data analysis across OpenAI’s data ecosystem.