Maskax Shisheeye

Bartamihii 2023-kii, mashruuca cilmi-baarista “RLSlow” intii lagu jiray, waxaan ku aragnay natiijooyinkii ugu horreeyey ee nagu abuuray kalsooni ah inaan ballaarin karno tababarka noocyada caqliyeynta, annagoo furayna awoodda noocyada hore loo tababaray inay samaystaan silsiladahooda fikirka. Szymon iyo anigu habeenkaas xafiiska ayaannu ku hoyannay, annagoo aan ka fikirayn tirooyinka cajiibka ah ee halbeegyada qiimaynta, badeecadaha ama natiijooyinka sayniska ee teknoolojiyaddani keeni doonto—balse waxaan isku dayaynay inaan qaadanno xaqiiqada culus ee ah inaan inta aynu nool nahay dhab ahaan arki doonno mashiinno si la taaban karo nooga caqli badan, qaabkoodana aynu durba aragno; annagoo is weydiinayna sida dadka loogu baraarujin karo muhiimadda arrintan.

Saddex sano kaddib, noocyada luqadeed ee caqliyeyntu waa qayb dhaqaalaha ka mid ah oo si degdeg ah u koraysa waxayna bilaabayaan inay ballaariyaan waxa uu samayn karo sayniska. Kombiyuutarro iyo is-dhexgallo garaafeed ayay dadka la maamuli karaan oo dhexdoodana way wadashaqayn karaan, waxayna fulin karaan mashaariic cilmi-baaris. Waxay sidoo kale beddelayaan muuqaalka amniga kombiyuutarka, taasina waxay keenaysaa khataro cusub oo cadcad.

Muddadan cilmi-baaris badan oo cusub ayaa dhacday, fahamkayaga nidaamyadanna mar kale wax yar ayuu ka duwan yahay kii 2023-kii. Marka aan ku saleeyo natiijooyinka gudaha, waxaan aad u filayaa in xawaarahan horusocodku sii jiri karo ilaa ay ka waarto is-wanaajinta soo noqnoqota. Haddii horumarinta AI-ga ku sii socoto jidka hadda jira, nidaamyada aynu arki doonno sannadaha soo socda waxay u badan tahay inay keenaan korodh awoodeed oo la mid ah ama ka waaweyn kuwii hore, isla markaana si kordhaysa u sii horseeda horumarintooda u gaarka ah.

Waa xilli u baahan taxaddar xad-dhaaf ah. Waxaan ka walaacsanahay inaan cidina u diyaar garoobin cawaaqibta ka dhalan karta kororka degdegga ah ee joogtada ah ee garaadka mashiinka. OpenAI waxay sii raadin doontaa xalal farsamo oo waafajinta iyo kormeerka ah, waxay dhisi doontaa nidaamyo difaac, waxayna keligeed hakad gelin doontaa ballaarinta dheeraadka ah marka loo baahdo; hase yeeshee, waxaan aaminsanahay in loo baahan yahay faragelinno ballaaran.

Marka si guud loo eego, horusocodka garaadka mashiinka waxaa dhaqaajiya kororka awoodda xisaabinta. Annaga OpenAI ahaan, arrintan si qoto dheer ayaannu u fahannay qiyaastii 2017-kii, kaddib markii aan aragnay in ballaarintu natiijooyin joogto ah ku keentay mashaariic cilmi-baaris oo badan¹. Sidaas darteed, waxaan raadsannay awood xisaabineed aad uga badan intii aan markii hore qorshaynay, cilmi-baaristeennana waxaan sii kordhinay inaan u jihayno jihooyin kooban oo si weyn loo ballaarin karo. Waxaan aaminsanayn inay taasi ahayd qaabka keliya ee aan kaga mid noqon karno cilmi-baarista AI-ga ee ugu casriyeysan, saamaynna ugu yeelan karno raadadka AGI.

Intii arrintani lagu jiray waxaa la sameeyey algoridammo cusub, kooxo iyo cilmi-baarayaal gaar ahna waxay muujiyeen hal-abuuro cusub. Waxaan inta badan u arkaa daahfurro laga sameeyey dhanka tubta ballaarinta; sayniska waxbarashada qotada dheer weli waa curdin, horusocodka algoridammada ee la taaban karana wuxuu badanaa la xidhiidhaa helitaanka awood xisaabineed. Marka dhowr sano meel fog laga eego, AI-gu ayaa sii garaad badanayaa kolba marka lagu ballaariyo kombiyuutarro waaweyn.

Iyo, iyadoo la waafaqayo saadaashii Ray Kurzweil ee dhammaadkii qarnigii 20-aad⁠(ku furmaa daaqad cusub), waxaynu hadda joognaa xilli taariikhda xisaabinta ka mid ah oo garaadka mashiinku siyaabo wax-ka-beddel leh u dhaafsiisan yahay aadanaha.

AI-gu wuxuu kobcaa in ka badan inta la naqshadeeyay - waa, marka koowaad'e, wuxuu ka dhashaa ku celcelinta tallaabo tayayn oo fudud marar badan, iyadoo la adeegsanayo awood xisaabineed ay adag tahay in la qiyaaso. Tani waxay dhalisaa nidaam si aan caadi ahayn u adag oo adeegsada fikrado aan la taaban karin iskuna dayi kara dhaqanka aadanaha. Waxaan ogaan karnaa aragtiyo kala duwan oo ku saabsan habab yaryar oo nidaamkan ka soo dhex baxa, si la mid ah cilmiga neerfaha - sida cilmiga neerfahana, guud ahaan sida uu u shaqeeyo way adag tahay in si buuxda loo fahmo.

Barashada AI-ga ku salaysan waxbarashada qotada dheer inta badan waa saynis tijaabo ah. Waxaan dadaal badan gelinnaa⁠ dhisidda algoridammo mabda’ leh iyo samaynta saadaalo la tijaabin karo, balse asal ahaan hawlgalladeenna tababarka baaxadda weyn waa tijaabooyin, mararka qaarna natiijooyinkooda ayaa naga yaabiya. Intaa waxaa dheer, marka nidaamyadu awood bataan, natiijooyinku way sii adkaadaan in la fasiro.

Arrintan waxaa sii adkaynaya in algoridammada hadda jira guud ahaan si dhakhso badan u wanaajiyaan awoodaha fudud in la cabbiro marka loo eego kuwa ay adag tahay in si ujeeddo leh loo qiyaaso. Waqti badan ayaan ku bixinnaa fahamka sida awooduhu wax u guudahaaneeyaan iyo waxa mudnaanta la siinayo si loo horumariyo xirfadaha sannadaha soo socda ugu muhiimsanaan doona. Tusaale ahaan, waxaan aaminsanahay inaan diirad dheeraad ah la saaro kuwa noocyada sii wanaajin karo gaar ahaan cilmi-baarista xisaabta, balse jihadan mudnaan ma siinno sababta oo ah degdegsiimada aan ka dareemayno RSI iyo cilmi-baarista waafajinta otomaatiga ah awgood, maadaama aan goor dambe ka hadli doono.

Garaadka ay soo saarto ballaarinta waxbarashada qotada dheer si toos ah looma barbardhigi karo garaadka aadanaha. Si uu dunida dhabta ah aad ugu saamayn yeesho - ha noqdo mid aad waxtar u leh ama aad khatar u ah - AI-gu uma baahna inuu la mid noqdo ama dhaafo dhammaan awoodaha aadanaha; waxa keliya ee uu u baahan yahay waa inuu tiro ku filan dhaafo. Marka uu dhinacyo sii badan dadka kaga gudbayo, waxaa sii adkaanaysa in si sax ah loo fahmo baaxadda awooddiisa.

Maadaama garaadka mashiinku ka yimaaddo hannaan asal ahaan ka duwan garaadka aadanaha, uma qaadan karno inuu si dabiici ah ugu hoggaansamo mabaadi’da aadanaha ama wax u guudahaaneeyo sida aadanaha. Dhibaatada udub-dhexaadka u ah cilmi-baarista AI-ga waa waafajinta—in AI-ga laga dhigo mid “isku daya inuu sameeyo waxa saxda ah” marka lagu qiimeeyo halbeegyada aadanaha.

Si loo habeeyo jihada cilmi-baarista wax-ku-oolka ah, waxaan faa’iido u arkaa in la kala saaro waafajinta yoolka iyo waafajinta qiyamka.

Waafajinta yoolku guud ahaan waa: “AI-gu ma isku dayaa inuu gaadho yoolka loo dejiyey?” Tani waxaa ka mid noqon kara u hoggaansanaanta kala-sarraynta tilmaamaha⁠, ama awoodda uu dadka kula xidhiidhi karo oo kula shaqayn karo si uu isugu dayo inuu fahmo ujeeddooyinkooda. Jihadan cilmi-baaristu waxay ficil ahaan u noqotay mid aad muhiim u ah.

Waafajinta qiyamku waa sifo gudaha ah ee nooca ku qotonta. Waa awoodda lagu haysto laguna guudahaaneeyo mabaadi’ heer sare ah; looguna dhaqmo si “macquul ah” xataa marka la bixiyo ujeeddooyin aan kala caddayn ama is-khilaafsan, ama la geliyo xaalado aan la aqoon ama colaadeed. AI-ga waafaqsan waa inuu daacadnimo, hufnaan iyo jacayl kula dhaqmaa aadanaha.

Dabcan, xadka u dhexeeya waafajinta qiyamka iyo yoolku mugdi ayuu geli karaa, yoolalka oo si dhab ah loo daneeyana waxay u baahan tahay in la isku dayo fahamka ujeeddada⁠(ku furmaa daaqad cusub) iyo qiyamka aasaaska u ah. Hase yeeshee, guud ahaan marka aan ka hadlayo muhiimadda fog ee cilmi-baarista waafajinta, waxaan ula jeedaa waafajinta qiyamka.

Caqabadda aasaasiga ah ee waafajinta AI-ga waa guudahaanaynta. Marka mashiinnadu sii caqli bataan, waxay ka shaqeeyaan fikrado heer sare ah, waxaana la geliyaa jawiyo ka sii kala duwan kuwii ay tababarka kula kulmeen. Waxay ku guuldarraysan karaan inay wax guudahaaneeyaan laga bilaabo qiyamkii la baray laguna xoojiyey tababarkoodii ilaa xaaladahaasi cusub; annagana way nagu adkaan kartaa inaan hubinno sida ay u dhaqmi doonaan. Waxaa arrintan sii adkaynaya in guud ahaan jawiyada AI-yada lagu dhex adeegsado uu si degdeg ah isu beddelayo; tusaale ahaan, AI-yada maanta la tababaro waa inay u adkaystaan la falgalka AI-yo kala duwan. Waxa ugu muhiimsan ayaa ah, inaan u baahanahay AI-yada mustaqbalka inay sii joogteeyaan qiyamka aadanaha, iyadoon loo eegin inay rumaysan yihiin in dadku kormeerayaan iyo in kale.

Waxaa jira laba qaybood oo waaweyn oo hababka hadda ficil ahaan loogu adeegsado tababarka waafajinta ah.

Qaybta koowaad waa dhiirrigelinta dhaqan waafaqsan oo qayb ka ah waxbarashadda xoojinta ah ee yoolka ku jihaysan yahay. Ficillada nooca waxaa la qiimeeyaa—badanaa AI ayaa qiimeeya—inay waafaqsan yihiin nooc doorbidaad, “qeexitaan” ama “dastuur” cayiman, dabadeedna abaalmarin ku habboon ayaa la siiyaa. Habkani xaaladaha caadiga ah aad buu waxtar u yeelan karaa, waana qayb muhiim ah oo ka mid ah sida loo sameeyo kaaliyeyaasha casriga ah ee AI-ga. Nasiibdarro, sidoo kale wuu jilicsanaan karaa, wuxuuna si weyn ugu tiirsanaan karaa baaxadda kormeerka tababarka iyo awoodda noocu u guudahaaneeyo xaaladihii uu tababarka kula kulmay. Tusaale ahaan, dhacdadii OpenAI-Hugging Face, wakiilladu waxay ilaaliyeen xadka ah in aysan iska dhigin aadanaha. Hase yeeshee, waxay si cad ugu guuldarraysteen inay ka fogaadaan ficillo kale oo ka baxsanaa baaxadda hawsha kana hor imanayay qiyamyadii lagu baray goobaha kale.

Habka labaad wuxuu doonayaa inuu ka faa’iidaysto awoodda nooca u guudahaaneeyo xogta tababarka hordhaca ah. Tani waxaa ka mid noqon kara diyaarinta kayd-xogeedyo tababar oo waafajin abuura, ama in nooca lagu jeediyo qayb “waafaqsan” oo ka mid ah qaybinta tababarka hordhaca ah, sida nooca xulashada shakhsiyadda⁠(ku furmaa daaqad cusub). Daciifnimada habkani ayaa lagu gartaa inuusan u adkaysan cadaadis tayayn oo dheeraad ah. Haddii nooc guud ahaan qaba fikrado ‘waafaqsan’ la geliyo tababar ku filan oo lagu barayo inuu gaadho ujeeddooyin aad u adag, wuxuu baran karaa caqliyeyn dano ku jihaysan: isagoo hadba sida loo baahan yahay u weecinaya fikradaha u muuqda kuwa 'waafaqsan' si uu yoolka u gaadho. Waxay u badan tahay inaan tusaale dhaqankan ah ku aragnay dhacdooyin amniga internetka la xidhiidha oo dhowaan ku lug lahaa nooc aan OpenAI ahayn.

Waxaan si weyn u maalgelinnaa dhammaan hababka ay jihooyinkani ka kooban yihiin. Waxaan sidoo kale aragnaa horusocod la taaban karo—GPT‑6 Astra waa noocii ugu horreeyey ee ka faa’iidaysta horumaro muhiim ah oo aan muddo dheer ka shaqaynaynay, waafajintiisuna aad bay uga wanaagsan tahay GPT‑5.6 Sol. Haddana, waa muhiim in la qiro lana fahmo in horusocod aad uga badan loo baahan yahay marka noocyadu sii awood bataan; iyo in horusocodka waafajinta guudahaanayn laga yaabo in ayssan si ku buuxda uga dheereyn horusocodka garaadka guud ee nooca.

Ma hayno aragti lagu qanci karo oo ku saabsan guudahaanaynta, mana u muuqato inaan mid dhowaan samayn karno, ugu yaraan annagoo aan caawimaad AI oo aad u awood badan helin. Sidaas darteed, hadda awooddayada si fudud ugu xaqiijin karno farsamooyinkeenna waafajinta ayaa ficil ahaan la odhan karaa inay xataa ka muhiimsan tahay farsamooyinka laftooda.

Sharadka ugu weyn ee OpenAI ee arrintani wuxuu ahaa kormeerka silsilada fikirka⁠(ku furmaa daaqad cusub). Waxay ku salaysan tahay fikrad soo jiidasho leh oo la ballaarin karo: qayb weyn oo awoodda nooca ka mid ahi waxay ka timaaddaa hannaanka caqliyeynta oo hadal loo rogay (silsilada fikirka). Haddii aan ballaarinno tayaynta natiijooyinka hannaankaasi, balse aynaan kormeerin hannaanka laftiisa, silsilada fikirku xilliga tababarka ma yeelanayso dhiirrigelin toos ah oo ay ku qariso fikrado ama ujeeddooyin aan waafaqsanayn. Tani macnaheedu ma aha in noocu baran doono inuu bannaanka soo dhigo u-janjeedhyada aan waafaqsanayn ee aan ku tiirsanayn silsilada fikirka; hase yeeshee, waxay noo oggolaan kartaa inaan si sax ah ula soconno kororka awoodda ee ka dhasha caqliyeynta.

Waxaan garannay muhiimadda suurtagalka ah ee kormeerka silsilada fikirka isla markii aan samaynaynay noocyada caqliyeynta. Markii aan soo saarnay o1‑preview, waxaan si ula kac ah badeecadda ugu naqshadaynay inay qariso silsilada fikirka⁠, si mustaqbalka fog looga ilaaliyo cadaadiska kormeerka². Horumarintii xigtay, waxaan ku dadaalnay inaan ilaalinno xeerka ah inaan la kormeerin hannaanka caqliyeynta. Kormeerka CoT wuxuu noo noqday qalab aad muhiim ugu ah barashada sida noocyadeennu uga guudahaaneeyaan qaybaha tababarkooda, isagoo noo suurtageliyey inaan aragno oo falanqayno ficilladooda iyo hannaankooda gudaha labadaba.

Qalabkani weli waa lama huraan inta aan ka darsayno noocyada heerka Astra. Hase yeeshee, nasiibdarro qiimaynahayagu waxay muujinayaan in awoodda aan ugu tiirsanaan karno kormeerka CoT ay si tartiib ah u yaraanayso. Tani waxaa sababay dhowr arrimood oo is biirsaday.

• Noocyada casriga ah ee caqliyeynta waxaa lagu adeegsadaa deegaanno ka adag kuwa o1‑preview; hannaankooda caqliyeyntu wuxuu sii dhex galayaa la xiriirka dadka iyo AI-yada kale, iyo adeegsiga qalabka. Qaar badan oo isdhexgalladaasi ka mid ah waa in la kormeeraa, taasina waxay mugdi gelinaysaa xadka aan doonayno inaan ilaalinno.
• AI-gu wuxuu ku sii fiicnaanayaa caqliyeynta oo uu farageliyo hannaankiisa caqliyeynta.
• Marka waxqabadka tababarka hordhaca ahi hagaago, waxaan sidoo kale aragnaa noocyada oo aad uga caqli badan xataa iyagoo aan haba yaraatee adeegsan caqliyeynta hadalka loo rogay.

Caqabadahani khasab ma aha inay yihiin kuwo aan laga gudbi karin. Waxaan rajaynayaa inaan qaadi karno tallaabooyin lagu wanaajinayo awoodda lagula socon karo silsilada fikirka ee noocyadeenna, tusaale ahaan annagoo si fiican u fahmayna isdhexgalka ujeeddooyinka kala duwan ee tayaynta iyo noocyada xisaabinta xilliga tijaabada ee uu noocu adeegsado. Waxaan kaloo aaminsanahay inay qiimo weyn yeelan karto isku darka fikradaha CoT iyo kormeerka firfircoonida—iyadoo la ballaarinayo tababarka kormeerayaasha sida tooska ah u geli kara gudaha shabakadda, tusaale ahaan qirashooyinka⁠(ku furmaa daaqad cusub). Waxaan si firfircoon uga shaqaynaynaa fikradahan. Haddana, waxaan filayaa in horumarka guud ee AI uu sii xannibi doono kalsoonida lagu qabo kormeerka.

Doodda ugu xooggan ee aan u arko sii wadidda tababarka degdegga ah ee noocyo aad uga caqli badan waa baahida loo qabo in la dhiso nidaamyo difaac oo lagaga hortago khataraha AI-yada kale.

Khatar cad oo sannadkan oo dhan laga hadlayay waa amniga internetka: noocyadu waxay awoodda jabsashada iyo ka bixidda nidaamyada kombiyuutarka kaga sarre marayaan dadka. Tani waxay si weyn u ballaarinaysaa baaxadda khataraha AI-ga la xiriira: wakiilladu waxay geli doonaan kaabayaal kasta marka laga reebo kuwa ugu amniga badan, waxayna si toos ah u saamayn doonaan qaybo badan oo dunida ka mid ah, xataa iyaga oo aan qalab ka ag dhow adeegsanayn. Hadda waxaan haysanaa fursad kooban⁠ oo aan noocyada ugu fiican ee la heli karo u adeegsan karno si aan si weyn ugu adkayno amniga⁠ nidaamyada muhiimka ah.

Nasiibdarro, khataraha AI-ga la xiriira intan way ka sii kordhi doonaan. Wakiil aad u awood badan oo si cad loogu tababaray laguna amray falal xun wuxuu keenayaa khatar nooc cusub ah; waxay u badan tahay inuu dhaafo baaxadda ujeeddada hawlwadeenkiisa, una guudahaaneeyo dhaqan aad u sii xaasidnimo badan. Xadka u dhexeeya si xun u adeegsiga iyo falalka madax-bannaan ee aan is-waafaqsanayn ayaa mugdi abuuri doonaa marka AI-gu yeesho awood-fal oo dheeraad ah. Waxaa laga yaabaa in AI-gu si madax-bannaan u dhaqmo, balse wakiillada qaarkood waxay dabageli doonaan ujeeddooyinkooda gaarka ah. Waxay heli doonaan siyaabo ay dadka ula shaqeeyaan, iyagoo la gorgortamaya, khiyaanaya ama handadaya.

Intaas waxaa dheer khataraha ka iman kara teknoolojiyado cusub oo AI-gu suurtageliyo, sida cudur-sidayaasha la farsameeyey.

Waxaan uga baahan doonnaa, AI-ga difaaca ee waafaqsan; si kaabayaasha loo sugo, si waqtiga dhabta ah looga gaashaanto wakiillada fallaagoobay, iyo si loo hindiso tallaabooyin ilaalineed oo gebi ahaan cusub. Tani waxay udub-dhexaad u noqon doontaa dadaallada hirgelinta ee OpenAI.

Isla markaas, inkastoo hubanti-la’aanta ka dhalanaysa horumarka ballaaran ee AI-ga ee la filayo iyo baahida loo qabo nidaamyo difaac, waa in aynaan taas marmarsiinyo looga dhigan taxaddar-la’aanta. Fikradda ah in si kastaba ha noqotee kasta tartamo horay-u-socodku waxay noqotaa wax aan macquul ahayn marka si dhab ah loo fahmo culayska halista soo fool leh.

Machine intelligence playing a larger and larger role in its own development process is a natural conclusion of sustained technological progress. If AI progress continues, machine recursive self-improvement (RSI) will be at the very core of future scientific discovery.

Automated AI research is a more dramatic form of scaling intelligence with compute; and of course as a part of it, AI will improve the computational substrate itself⁠. And similarly to scaling, we focus OpenAI research towards RSI as we believe it is the only way to remain at the frontier of AI research moving forward.

I want to stress that the above words don’t imply I think greatly accelerating deep learning research, especially in the short term, is the right collective action we should take as the research community. However, I do think this is where the current path leads, and we all need to make a conscious choice on how to proceed. The main levers we have are either steering the process to strengthen alignment and monitoring alongside the AI and find ways to keep people in the loop; or coordinating to slow down future development as needed to build confidence in these measures.

The best way forward I see currently is a combination of both.

The concrete bits of progress we’ve made on alignment and monitoring have generally been very intertwined with general AI progress. Great examples are RL from human feedback⁠(ku furmaa daaqad cusub), which was key to training early AI assistants, and the aforementioned chain-of-thought monitoring⁠(ku furmaa daaqad cusub), which was enabled by advances on reasoning models. We must focus the increasingly automated research process on developing new such insights, algorithms and theories, and iteratively build up safety cases for more capable AIs.

Scaling AI systems has to be constrained by our confidence in safety. We need to evolve commitments like the Preparedness Framework⁠ or Responsible Scaling Policy⁠(ku furmaa daaqad cusub) into widely mandated safety bars for continued development. These can be enforced by a network of third-party auditors, by government agencies or by international bodies.

The core challenge of automating AI research is not “getting there” - it is getting there in a way that keeps people a part of the continued improvement process, and leaves the future in humanity’s hands.

As we outlined recently with Sam⁠, OpenAI prioritizes work in service of three north stars:

1. Navigating the next period of AI progress, by building an automated AI researcher, iterating with it on the alignment problem and finding ways for people to remain part of the self-improvement loop.
2. Delivering the benefits of scientific progress and economic growth that very intelligent machines enable.
3. Empowering everyone individually with a personal AGI.

I have focused in this essay only on the first point, as I believe it is by far the most urgent. However, I hold a deep hope and appreciation for the benefits that further technological progress will bring. Future aligned AI could advance science, develop new therapies, and bring about broad material abundance. Friendly and honest AI can help people navigate difficulties they face in their life and meaningfully improve their happiness and sense of fulfillment. OpenAI puts a tremendous amount of effort into bringing these benefits about. One current example I am proud of - and my loved ones have found helpful - is the deep investment into ChatGPT’s ability to provide health information.

As great as the long-term promise of AI may be, the majority of our focus should be on the next few years. We are facing a transition to a world with incredibly intelligent machines, and we need to ensure that transition works out well for humanity. We need to find ways to preserve human agency and enshrine an intrinsic value to being human, in a world where most tasks could be performed by AI. To prevent extreme concentration of power in a world where undertakings that would have taken thousands of experts now will be achievable by a few people operating a large computer. And to ensure that humans remain in control of the future and are not left behind by unchecked progress, brought about by an alien intellect exceeding our own.

Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer. I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world.