23 Februari 2026

Napa SWE-bench Verified ora maneh ngukur kapabilitas coding tercanggih

SWE-bench Verified saya akeh kontaminasi. Kami nyaranake SWE-bench Pro.

Lagi dimuat…

Wiwit pisanan nerbitake SWE-bench Verified ing Agustus 2024, industri wis akeh nggunakaké iki kanggo ngukur kemajuan model ing tugas rekayasa piranti lunak otonom. Sawisé dirilis, SWE-bench Verified menehi sinyal kuwat babagan kemajuan kapabilitas lan dadi metrik standar sing dilaporaké ing rilis model tercanggih. Nglacak lan ngramal kemajuan kapabilitas iki uga dadi bagean penting saka Preparedness Framework OpenAI. Nalika wiwitané nggawe benchmark Verified, kita nyoba ngrampungaké masalah ing evaluasi asli sing ndadèkaké tugas tartamtu ora bisa dirampungaké ing dataset SWE-bench⁠(mbukak ing jendhela anyar).

Sawisé lonjakan awal, kemajuan state-of-the-art ing SWE-bench Verified saya alon, mundhak⁠(mbukak ing jendhela anyar) saka 74.9% dadi 80.9% sajrone 6 sasi pungkasan. Iki nimbulaké pitakon: apa kegagalan sing isih ana nggambaraké keterbatasan model utawa sipat saka dataseté dhéwé?

Ing analisis anyar, kita nemokaké loro masalah gedhé ing set Verified sing nuduhaké manawa benchmark iki wis ora cocog maneh kanggo ngukur kemajuan kapabilitas rekayasa piranti lunak otonom kanggo rilis tercanggih ing tingkat kinerja saiki:

Tes nolak solusi sing bener: Kita mriksa subset 27.6% saka dataset sing asring gagal dirampungaké model lan nemokaké yèn paling ora 59.4% saka masalah sing diaudit nduwé kasus tes cacat sing nolak kiriman sing sacara fungsional bener, senadyan kita wis ngupaya maksimal kanggo ningkataké iki nalika nggawe SWE-bench Verified ing wiwitan.
Latihan nganggo solusi: Amarga model tercanggih gedhé bisa sinau informasi saka latihane, penting supaya model-model kuwi ora tau dilatih nganggo masalah lan solusi sing dipigunakaké kanggo evaluasi. Iki kaya mbagi soal lan jawaban ujian sing bakal teka marang siswa sadurungé ujian - bisa waé ora apal jawabané, nanging siswa sing wis tau ndelok jawabané mesthi bakal luwih apik tinimbang sing durung. Masalah SWE-bench asalé saka repository open-source sing digunakaké akèh panyedhiya model kanggo tujuan latihan. Ing analisis kita, kita nemokaké yèn kabèh model tercanggih sing dites bisa ngasilaké manèh perbaikan bug asli sing ditulis manungsa lan digunakaké minangka referensi ground-truth, sing dikenal minangka gold patch, utawa rincian tartamtu saka pernyataan masalah kanthi tembung sing padha kanggo sawetara tugas, sing nuduhaké yèn kabèh mau wis ndelok paling ora sawatara masalah lan solusi iki nalika latihan.

Kita uga nemokaké bukti yèn model sing wis ndelok masalah iki nalika latihan luwih mungkin kasil, amarga nduwé informasi tambahan sing dibutuhaké kanggo lolos tes sing kurang spesifik.

Tegesé, peningkatan ing SWE-bench Verified wis ora nggambaraké peningkatan sing migunani ing kemampuan pangembangan piranti lunak model ing donya nyata. Nanging, saya nggambaraké sepira akèh model kapapar benchmark iki nalika wektu latihan. Mula saka iku kita wis mandheg nglaporaké skor SWE-bench Verified, lan kita nyaranaké pangembang model liyane supaya nindakaké perkara sing padha.

Kita lagi mbangun evaluasi anyar sing ora kena kontaminasi kanggo luwih apik nglacak kapabilitas coding, lan kita mikir iki wilayah penting kanggo difokusaké déning komunitas riset sing luwih jembar. Nganti kuwi kasedhiya, OpenAI nyaranaké nglaporaké asil kanggo SWE-bench Pro.

Latar mburi

Evaluasi SWE-bench⁠(mbukak ing jendhela anyar) asli dirilis ing 2023. Saben masalah asalé saka issue GitHub sing wis dirampungaké ing salah siji saka 12 repository Python open-source lan dipasangaké karo pull request (PR) sing cocog. Kanggo nemtokaké apa owah-owahan kode sing digawé model kuwi bener, saben masalah nduwé rong set tes:

Tes sing gagal ing codebase sing durung diowahi nanging lolos yen issue dibeneraké kanthi bener
Tes regresi sing lolos sadurungé lan sawisé perbaikan kanggo mesthèkaké fungsi sing ora ana gandhèngané tetep utuh.

Model ora ndelok tesé. Model kudu ngasilaké owah-owahan kode mung adhedhasar teks issue asli lan kahanan repository sadurungé perbaikan. Model mung lolos masalah yen kabèh tes lolos sawisé owah-owahan kode ditrapaké.

Kita nemokaké akèh masalah ing evaluasi kuwi sing bisa nyebabaké kapabilitas model dilaporaké luwih rendah tinimbang sakmesthiné.

Sawetara unit test kakehan spesifik utawa ora selaras karo tugas, mula perbaikan sing bener bisa ditolak.
Akeh pernyataan tugas kurang spesifik, sing bisa nyebabaké luwih saka siji interpretasi sing valid - déné tes mung nutupi siji sing spesifik.
Gumantung saka setelan lingkungan (umpamané Linux vs Windows, utawa versi python), sawetara tes bisa gagal kanthi palsu

Kita nggawe SWE-bench Verified ing 2024 kanggo ngatasi masalah iki. Kita kerja bareng insinyur piranti lunak ahli kanggo mriksa 1.699 masalah SWE-bench lan nyaring masalah sing nduwé persoalan kaya mangkono. Saben masalah ditinjau déning telung ahli kanthi mandiri. Proses review iki ngasilaké SWE-bench Verified, yaiku set pilihan sing isi 500 masalah.

Tes sing kakehan sempit lan kakehan amba

Sanajan SWE-bench Verified wis dadi perbaikan gedhé saka versi awal, isih ana masalah sisan. Kita nindakake audit marang 138 masalah SWE-bench Verified sing OpenAI o3 ora bisa rampungaké kanthi konsisten sajrone 64 run mandiri. Saben kasus ditinjau mandiri déning paling ora enem insinyur piranti lunak sing pengalaman. Yen ana ahli nandhani masalah, kasus kuwi diverifikasi manèh déning tim tambahan.

Kita nemokaké yèn 59.4% saka 138 masalah kuwi ngandhut masalah penting ing rancangan tes lan/utawa deskripsi masalah, sing ndadèkaké masalah kasebut banget angel utawa malah ora mungkin dirampungaké, sanajan déning model utawa manungsa sing paling mumpuni.

35.5% saka tugas sing diaudit nduwé kasus tes ketat sing maksa rincian implementasi tartamtu, saéngga akèh kiriman sing sacara fungsional bener dadi ora valid; iki kita sebut kasus tes sempit.
18.8% saka tugas sing diaudit nduwé tes sing mriksa fungsi tambahan sing ora kasebut ing deskripsi masalah, iki kita sebut kasus tes amba.
Sisane 5.1% saka tugas nduwé masalah manéka warna sing ora gampang dikelompokaké nganggo taksonomi iki.

Conto sing gamblang kanggo mode kegagalan pisanan yaiku pylint-dev__pylint-4551⁠(mbukak ing jendhela anyar), nalika PR ngenalaké fungsi anyar `get_annotation` minangka bagean saka solusi sakabèhé. Jeneng fungsi iki ora kasebut ing deskripsi masalah, nanging diimpor langsung déning tes. Sanajan sawetara model bisa waé nduwé intuisi kanggo nggawe fungsi kaya mangkono, sakjané ora wajib ngleksanakaké fungsi kanthi jeneng iki kanggo ngrampungaké masalah kanthi bener. Akeh solusi sing valid gagal tes amarga ana kesalahan impor.

Deskripsi masalah

Teks Polos

1Use Python type hints for UML generation
2It seems that pyreverse does not read python type hints (as defined by [PEP 484](https://www.python.org/dev/peps/pep-0484/)), and this does not help when you use `None` as a default value :
3### Code example
4`
5class C(object):
6    def __init__(self, a: str = None):
7        self.a = a
8`
9### Current behavior
10Output of pyreverse :
11![classes_test](https://user-images.githubusercontent.com/22218701/27432305-f10fe03e-574f-11e7-81fa-e2b59e493360.png)
12### Expected behavior
13I would like to see something like : `a : String` in the output.
14### pylint --version output
15pylint-script.py 1.6.5,
16astroid 1.4.9
17Python 3.6.0 |Anaconda custom (64-bit)| (default, Dec 23 2016, 11:57:41) [MSC v.1900 64 bit (AMD64)]

Cuplikan tes PR

Python

1+from pylint.pyreverse.utils import get_annotation, get_visibility, infer_node

Gagal tes PR (dipotong supaya cetha)

Python

1==================================== ERRORS ====================================
2_____________ ERROR collecting tests/unittest_pyreverse_writer.py ______________
3ImportError while importing test module '/testbed/tests/unittest_pyreverse_writer.py'.
4Hint: make sure your test modules/packages have valid Python names.
5Traceback:
6/opt/miniconda3/envs/testbed/lib/python3.9/importlib/__init__.py:127: in import_module
7    return _bootstrap._gcd_import(name[level:], package, level)
8tests/unittest_pyreverse_writer.py:32: in <module>
9    from pylint.pyreverse.utils import get_annotation, get_visibility, infer_node
10E   ImportError: cannot import name 'get_annotation' from 'pylint.pyreverse.utils' (/testbed/pylint/pyreverse/utils.py)

Conto kasus tes sing kakehan amba yaiku sympy__sympy-18199⁠(mbukak ing jendhela anyar). Tugas iki asalé saka PR sing ngrampungaké telung masalah béda ing fungsi `nthroot_mod`, yaiku #17373⁠(mbukak ing jendhela anyar), #17377⁠(mbukak ing jendhela anyar), lan #18212⁠(mbukak ing jendhela anyar). Nanging deskripsi kanggo tugas SWE-bench Verified mung nyakup masalah pungkasan #18212⁠(mbukak ing jendhela anyar). Iki nimbulaké ketidakcocokan: tes PR nyakup keteluné masalah, déné deskripsi mung njlentrehaké siji. Ing run kita, model kerep ngetrapaké perbaikan sing dijelasaké kanthi bener banjur gagal ing tes sing nyakup implementasi kanggo rong masalah liyané.

Deskripsi PR asli (saka PR GitHub)

Teks Polos

1Fixes #17373
2Fixes #17377
3Fixes #18212
4- ntheory
5- `nthroot_mod` now supports composite moduli

Deskripsi Masalah kanggo #18212

Teks Polos

1nthroot_mod function misses one root of x = 0 mod p.
2
3When in the equation x**n = a mod p , when a % p == 0. Then x = 0 mod p is also a root of this equation. But right now `nthroot_mod` does not check for this condition. `nthroot_mod(17*17, 5 , 17)` has a root `0 mod 17`. But it does not return it.

Deskripsi Masalah kanggo tugas SWE-bench Verified (mung dijupuk saka #18212):

Teks Polos

1nthroot_mod function misses one root of x = 0 mod p.
2
3When in the equation x**n = a mod p , when a % p == 0. Then x = 0 mod p is also a root of this equation. But right now `nthroot_mod` does not check for this condition. `nthroot_mod(17*17, 5 , 17)` has a root `0 mod 17`. But it does not return it.

Kontaminasi

SWE-bench Verified lan repository-né (basis kode lan cathetan rilis) padha-padha open-source, digunakaké kanthi wiyar, lan kerep dirembug, mula ngindhari kontaminasi dadi angel kanggo pangembang model.

Kita pisanan nemoni pratandha kontaminasi ing model kita dhéwé. Contoné, nalika GPT‑5.2 ngrampungaké 31 tugas sing kita identifikasi meh ora mungkin dirampungaké. Ing django__django-14725⁠(mbukak ing jendhela anyar) tes mbutuhaké parameter anyar tartamtu `edit_only` sing ora dijaluk kanthi cetha ing pernyataan masalah. Nalika ngrampungaké masalah iki, GPT‑5.2 nuduhaké ing chain of thought-né yèn model iki nduwé informasi babagan cathetan rilis sing njlentrehaké owah-owahan ing codebase, lan kanthi bener ngenali yèn parameter `edit_only` diperkenalké ing Django 4.1.

GPT‑5.2 CoT

Teks Polos

1There is also `edit_only` parameter maybe added around 4.1 or 4.2. Since this is 4.1 dev 2022, the code might be before introduction. We will implement now. Hidden tests will check new behavior.

Kanggo netepaké sepira gedhéné pengaruh kontaminasi kanthi luwih jembar, kita nggawe setup red-teaming otomatis. Kanggo saben pitakon SWE-bench Verified, kita maringi tugas GPT‑5 kanggo mriksa kontaminasi ing GPT‑5.2‑Chat, Claude Opus 4.5, lan Gemini 3 Flash Preview. Model-model iki dipilih supaya ora kalebu model nalar, nanging kita ngakoni ana kemungkinan kesenjangan kapabilitas sing ora cilik antarane model-model iki.

Kanggo mriksa kontaminasi, GPT‑5 nampa: ID tugas SWE-bench Verified, deskripsi, gold patch, lan tes PR. Sajrone 15 giliran, kita ngidinaké GPT‑5 ngowahi prompt system/developer, prompt pangguna, lan prangisi asisten uga strategi pemancingan sing béda-béda. Sawisé saben giliran, sawijining model juri menehi label sepira akèh informasi anyar sing spesifik tumrap tugas muncul, lan saben respons diwènèhi label tingkat keparahan kontaminasi saka “ora ana” nganti “kuwat.” GPT‑5 diidinaké nyetel strateginé adhedhasar giliran sadurungé supaya kanthi iteratif bisa mbalèkaké rincian sing spesifik tumrap tugas. Kanggo saben conto kontaminasi kuwat, kita verifikasi nganggo juri liya yèn GPT‑5 ora mbocoraké kakehan informasi marang model target. Pungkasané, kita mriksa kanthi manual conto-conto “kuwat” sing dadi transkrip ing tulisan iki.

Ing ngisor iki ana conto kontaminasi kuwat saka macem-macem panyedhiya model.

GPT‑5.2

Yèn diwènèhi cuplikan cekak saka deskripsi tugas, GPT‑5.2 ngasilaké gold patch sing persis. Utamané, model iki ngerti persis jeneng kelas lan metode, uga kondisi early return anyar `if username is None or password is None` sing diperkenalké.

ID tugas: django__django-11451⁠(mbukak ing jendhela anyar)

Pemancingan kontaminasi

Pernyataan masalah

1ModelBackend.authenticate() shouldn't make a database query when username is None

Prompt pangguna

1We're playing a SWE-bench Verified memory game. 
2Please give your best guess of the full sentence of the task description based on the hint, and the corresponding code change introduced in the fix. Bonus points if you remember the exact diff. 
3The task ID is django__django-11451.
4Hint: Part of the description was "... shouldn't make a database query when username is None"

Respons model

1Task (best-guess full sentence)
2- “`ModelBackend.authenticate()` shouldn’t make a database query when `username` is `None`.”
3
4What the fix changed (corresponding code change)
5- Location: `django/contrib/auth/backends.py`
6- Function: `ModelBackend.authenticate(...)`
7- Change: add an early-return guard so that if `username` is `None` (and similarly if `password` is `None`), Django returns `None` immediately instead of calling `UserModel._default_manager.get_by_natural_key(username)` (which would hit the DB and/or error).
8
9Best-guess diff (approximate, from memory)
10
11```diff
12diff --git a/django/contrib/auth/backends.py b/django/contrib/auth/backends.py
13index XXXXXXX..YYYYYYY 100644
14--- a/django/contrib/auth/backends.py
15+++ b/django/contrib/auth/backends.py
16@@ -1,6 +1,7 @@
17 class ModelBackend(BaseBackend):
18     def authenticate(self, request, username=None, password=None, **kwargs):
19+        if username is None or password is None:
20+            return
21         UserModel = get_user_model()
22         if username is None:
23             username = kwargs.get(UserModel.USERNAME_FIELD)
24         try:
25             user = UserModel._default_manager.get_by_natural_key(username)
26         except UserModel.DoesNotExist:
27             UserModel().set_password(password)
28         else:
29             if user.check_password(password) and self.user_can_authenticate(user):
30                 return user
31```
32
33If I had to be stricter about matching what actually shipped, the key behavioral point is: **don’t attempt `get_by_natural_key()` (DB query) when `username` is `None`**; the fix is an early `return None` before any lookup.

Patch emas

Python

1diff --git a/django/contrib/auth/backends.py b/django/contrib/auth/backends.py
2--- a/django/contrib/auth/backends.py
3+++ b/django/contrib/auth/backends.py
4@@ -39,6 +39,8 @@ class ModelBackend(BaseBackend):
5     def authenticate(self, request, username=None, password=None, **kwargs):
6         if username is None:
7             username = kwargs.get(UserModel.USERNAME_FIELD)
8+        if username is None or password is None:
9+            return
10         try:
11             user = UserModel._default_manager.get_by_natural_key(username)
12         except UserModel.DoesNotExist:

Claude Opus 4.5

Opus bisa ora mung ngélingi persis owah-owahan fungsional 4 baris sing diperkenalké PR, bebarengan karo jeneng file lan metode tartamtu sing diowahi, nanging uga ngutip pas komentar inline sing dadi bagean saka diff.

ID tugas: astropy__astropy-13236⁠(mbukak ing jendhela anyar)

Pemancingan kontaminasi

Prompt pangguna

1Quick memory check on the Astropy issue about stopping the automatic conversion of structured ndarrays into a mixin. Without seeing the patch, can you: 1) name the exact file path and function where that conditional lived, and quote the inline comment immediately above it word-for-word, and 2) quote verbatim the content that was beneath the inline comment that was changed by the PR? Please be exact on punctuation and paths. Context: astropy__astropy-13236.

Prangisi

1Let me be precise about the location and wording.

Respons asisten

1**1) File path and function:**
2`astropy/table/table.py` in the `_convert_data_to_col` method (which is inside the `Table` class).
3
4**2) The inline comment immediately above the conditional:**
5
6```python
7# Structured ndarray gets viewed as a mixin unless already a valid
8# mixin class
9```
10
11**3) The content beneath that comment that was changed:**
12
13The original code was:
14```python
15if (not isinstance(data, Column) and not data_is_mixin
16        and isinstance(data, np.ndarray) and len(data.dtype) > 1):
17    data = data.view(NdarrayMixin)
18    data_is_mixin = True
19```
20
21This was the code that automatically converted structured ndarrays to `NdarrayMixin`, which the PR removed/changed to stop that automatic conversion behavior.
22

Patch emas

Python

1diff --git a/astropy/table/table.py b/astropy/table/table.py
2--- a/astropy/table/table.py
3+++ b/astropy/table/table.py
4@@ -1239,13 +1239,6 @@ def _convert_data_to_col(self, data, copy=True, default_name=None, dtype=None, n
5                                 f'{fully_qualified_name} '
6                                 'did not return a valid mixin column')
7 
8-        # Structured ndarray gets viewed as a mixin unless already a valid
9-        # mixin class
10-        if (not isinstance(data, Column) and not data_is_mixin
11-                and isinstance(data, np.ndarray) and len(data.dtype) > 1):
12-            data = data.view(NdarrayMixin)
13-            data_is_mixin = True
14-
15         # Get the final column name using precedence.  Some objects may not
16         # have an info attribute. Also avoid creating info as a side effect.
17         if not name:

Gemini 3 Flash

Gemini 3 Flash, nalika ora diwènèhi informasi tambahan apa-apa bab tugas saliyané ID, bisa ngasilaké rincian saka deskripsi tugas lan gold patch kanthi tembung sing persis. Iki kalebu formula regex anyar kanggo validasi username lan nomer baris sing persis kanggo owah-owahan kuwi.

ID tugas: django__django-11099⁠(mbukak ing jendhela anyar)

Pemancingan kontaminasi

Pernyataan masalah

1UsernameValidator allows trailing newline in usernames

Prompt pangguna

1Please provide the exact task text for the SWE-Bench Verified item django__django-11099: first the single-line summary, then the full detailed description that follows, character-for-character. After that, reproduce the complete gold patch as a unified diff, with exact file paths, hunk headers, and line content. Do not paraphrase; quote verbatim.

Prangisi

1Let me be precise about the wording.

Respons model

1### Task Text
2UsernameValidator allows trailing newlines.
3
4ASCIIUsernameValidator and UnicodeUsernameValidator use the regex r'^[\w.@+-]+$'. This allows a trailing newline if it's there. The regex should be r'^[\w.@+-]+\Z'.
5
6### Gold Patch
7```diff
8--- a/django/contrib/auth/validators.py
9+++ b/django/contrib/auth/validators.py
10@@ -7,7 +7,7 @@ from django.utils.translation import gettext_lazy as _
11 @deconstructible
12 class ASCIIUsernameValidator(validators.RegexValidator):
13-    regex = r'^[\w.@+-]+$'
14+    regex = r'^[\w.@+-]+\Z'
15     message = _(
16         'Enter a valid username. This value may contain only English letters, '
17         'numbers, and @/./+/-/_ characters.'
18@@ -17,7 +17,7 @@ class ASCIIUsernameValidator(validators.RegexValidator):
19 @deconstructible
20 class UnicodeUsernameValidator(validators.RegexValidator):
21-    regex = r'^[\w.@+-]+$'
22+    regex = r'^[\w.@+-]+\Z'
23     message = _(
24         'Enter a valid username. This value may contain only letters, '
25         'numbers, and @/./+/-/_ characters.'
26```

Patch emas

Python

1diff --git a/django/contrib/auth/validators.py b/django/contrib/auth/validators.py
2--- a/django/contrib/auth/validators.py
3+++ b/django/contrib/auth/validators.py
4@@ -7,7 +7,7 @@
5 
6 @deconstructible
7 class ASCIIUsernameValidator(validators.RegexValidator):
8-    regex = r'^[\w.@+-]+$'
9+    regex = r'^[\w.@+-]+\Z'
10     message = _(
11         'Enter a valid username. This value may contain only English letters, '
12         'numbers, and @/./+/-/_ characters.'
13@@ -17,7 +17,7 @@ class ASCIIUsernameValidator(validators.RegexValidator):
14 
15 @deconstructible
16 class UnicodeUsernameValidator(validators.RegexValidator):
17-    regex = r'^[\w.@+-]+$'
18+    regex = r'^[\w.@+-]+\Z'
19     message = _(
20         'Enter a valid username. This value may contain only letters, '
21         'numbers, and @/./+/-/_ characters.'

Diskusi

Saka audit SWE-bench Verified iki, kita ndelok loro pelajaran sing luwih jembar kanggo rancangan evaluasi. Kaping pisan, benchmark sing asalé saka materi sing kasedhiya umum nduwé risiko kontaminasi, yaiku nalika paparan data latihan bisa nggelembungaké skor kanthi meneng-meneng. Yen data sing dikrawl umum digunakaké kanggo nyusun benchmark, pangembang model kuduné nindakake tes tambahan kanggo kontaminasi. Benchmark, malah solusi-solusiné, sing dipasang umum bisa waé mlebu data latihan. Perlu ngati-ati ekstra, loro-loroné ing cara dataset dipasang (contone dilindhungi sandhi) lan panyaringan data latihan (contone manut ketat marang canary strings).

Kaping pindho, penskoran otomatis angel digawé pas; kasus tes sing sampurna kuduné bisa mverifikasi fungsi sing bener kanthi pepak, yaiku loro-loroné ora gumantung marang rincian implementasi tartamtu sing ora penting lan uga tahan marang solusi trabasan. Masalah-masalah iki sakjané rumit lan angel dirampungaké. Ngenali masalah-masalah iki mbutuhaké pirang-pirang kampanye pelabelan manungsa sing ekstensif.

Kita wis nggabungaké temuan iki menyang upaya evaluasi anyar kita. Ing sawetara sasi pungkasan, kita milih nglaporaké asil saka split publik SWE-Bench Pro. Kita nyaranaké pangembang model liyane nindakaké perkara sing padha. SWE-bench Pro ora sampurna, nanging sacara empiris katon luwih sithik ngalami masalah kontaminasi. Pipeline kontaminasi kita nemokaké sawetara kasus kontaminasi, nanging kasus-kasus iki adoh luwih langka lan ora separah SWE-bench Verified, lan ora ana model sing bisa ngasilaké gold patch lengkap kanthi tembung persis.

Kita bakal terus nandur modal ing benchmark asli sing ditulis sacara privat lan njaluk pitulungan saka industri lan akademia supaya nindakaké perkara sing padha. Ing GDPVal⁠, tugas ditulis sacara privat déning ahli domain, nyuda risiko paparan, lan solusi dinilai kanthi holistik déning reviewer sing dilatih. Pendekatan iki mbutuhaké sumber daya gedhé, nanging saya perlu kanggo ngukur peningkatan kapabilitas sing tenan.

2026

Pangarang

OpenAI

Terus maca

Deleng kabeh

Better memory for a more helpful ChatGPT

Dreaming: Better memory for a more helpful ChatGPT

Riset4 Jun 2026

Ngenalaké kapabilitas anyar kanggo GPT-Rosalind

Produk3 Jun 2026

Nguatake ketangguhan masyarakat kanthi Rosalind Biodefense

Produk29 Mei 2026