Qiimeynta awoodda AI ee qabashada hawlaha cilmi-baarista sayniska
Waxaan soo bandhigaynaa FrontierScience, oo ah benchmark cusub oo qiimeeya awoodaha AI ee caqliyeynta sayniska heer-khabiir ah ee fiisigis, kimistari, iyo bayoolaji.
Caqliyeyntu waa xudunta shaqada sayniska. Ka baxsan xusuusashada xaqiiqooyinka, saynisyahannadu waxay abuuraan mala-awaallo, tijaabiyaan oo sifeeyaan, kuna mideeyaan fikrado ka kala yimaadda dhinacyo kala duwan. Marka noocyadeennu sii awood badanayaan, su’aasha dhexe waa sida ay si qoto dheer ugu caqliyeyn karaan si ay uga qayb qaataan cilmi-baarista sayniska.
Sanadkii la soo dhaafay, noocyadeennu waxay gaareen guulo waaweyn, oo ay ku jirto helitaanka waxqabad heer billad dahab ah oo International Math Olympiad iyo International Olympiad in Informatics ah. Isla muddadaas, waxaan bilaabaynaa inaan aragno noocyadeenna ugu awoodda badan, sida GPT‑5, oo si macno leh u dedejinaya socodka shaqada sayniska ee dhabta ah. Cilmi-baarayaashu waxay nidaamyadan u adeegsanayaan hawlo ay ka mid yihiin raadinta suugaanta cilmiyeed ee ka gudubta maadooyin iyo luqado iyo ka shaqeynta caddaymo xisaabeed oo adag. Xaalado badan, noocku wuxuu saacado ka dhigaa shaqo qaadan lahayd maalmo ama toddobaadyo. Horumarkan waxaa lagu diiwaangeliyey warqaddeenna Tijaabooyin hore oo dardargelinta sayniska ah oo leh GPT‑5, oo la sii daayay Noofambar 2025, taas oo soo bandhigaysa caddeyn hore in GPT‑5 si la cabbiri karo u dedejin karo socodka shaqada sayniska.
Maadaama dardargelinta horumarka saynisku ay tahay mid ka mid ah fursadaha ugu rajo badan ee AI uga faa’iideyn karto aadanaha, waxaan horumarineynaa noocyadeenna hawlaha adag ee xisaabta iyo sayniska, sidoo kalena waxaan ka shaqeyneynaa qalabka ka caawin doona saynisyahannada inay ka helaan faa’iidada ugu badan.
Markii GPQA(ku furmaa daaqad cusub), oo ah benchmark saynis oo “Google-Proof” ah oo su’aalo ay qoreen khubaro PhD ah, la sii daayay Noofambar 2023, GPT‑4 wuxuu keenay 39%, taas oo ka hooseysay heerka saldhigga khubarada ee 70%. Laba sano kadib, GPT‑5.2 wuxuu keenay 92%. Maaddaama awoodaha caqliyeynta iyo aqoonta ee noocyadu sii korayaan, benchmark-yo ka adag ayaa muhiim u noqon doona cabbirka iyo saadaalinta awoodda noocyada ee dardargelinta cilmi-baarista sayniska. Benchmark-yadii sayniska ee hore inta badan waxay diiradda saareen su’aalaha doorashooyinka badan leh, way buuxsameen, ama si gaar ah uguma ayan diiradda saarneyn sayniska.
Si loo buuxiyo farqigan, waxaan soo bandhigaynaa FrontierScience: benchmark cusub oo loo dhisay cabbirka awoodaha sayniska heer-khabiir. FrontierScience waxaa qora oo xaqiijiya khubaro ku kala takhasusay fiisigis, kimistari, iyo bayoolaji, wuxuuna ka kooban yahay boqolaal su’aalood oo loo naqshadeeyey inay noqdaan adag, asal ah, oo macno leh. FrontierScience wuxuu leeyahay laba waddo oo su’aalo ah: Olympiad, oo cabbiraysa awoodaha caqliyeynta sayniska ee qaabka Olympiad-ka, iyo Research, oo cabbiraysa awoodaha cilmi-baarista sayniska ee dunida dhabta ah. Bixinta faham dheeraad ah oo ku saabsan awoodaha sayniska ee noocyada waxay naga caawisaa inaan la soconno horumarka oo aan sii hormarinno sayniska AI-dedejisay.
Qiimeynteennii bilowga ahayd, GPT‑5.2 waa noocayaga ugu waxqabadka sarreeya FrontierScience-Olympiad (77%) iyo Research (25%), isagoo ka horreeya noocyo kale oo ugu casriyeysan. Waxaan aragnay horumar weyn oo lagu xallinayo su’aalaha heer-khabiir, iyadoo weli ay bannaan tahay fursad horumar dheeraad ah, gaar ahaan hawlaha cilmi-baarista ee furan. Saynisyahannada, tani waxay soo jeedinaysaa in noocyada hadda jira ay durba taageeri karaan qaybo ka mid ah cilmi-baarista ku lug leh caqliyeyn habeysan, isla markaana muujinaysa in shaqo weyn weli u dhiman tahay si loo wanaajiyo awooddooda fulinta fikirka furan. Natiijooyinkani waxay la jaanqaadaan sida saynisyahannadu horey ugu adeegsanayaan noocyada maanta jira: inay dedejiyaan socodka shaqada cilmi-baarista iyagoo ku tiirsan xukunka bini’aadamka ee qaabaynta dhibaatada iyo ansaxinta, isla markaana si sii kordheysa u sahamiyaan fikrado iyo xiriirro haddii kale qaadan lahaa waqti aad u dheer in la helo—oo ay ku jirto, mararka qaar, keenista aragtiyo cusub oo khubaradu markaas qiimeeyaan oo tijaabiyaan.
Ugu dambayn, benchmark-ka ugu muhiimsan ee awoodaha sayniska ee AI waa daahfurka cusub ee ay ka caawiso in la abuuro; kuwaas ayaa ugu dambayn muhiim u ah sayniska iyo bulshada. FrontierScience wuxuu ka horreeyaa taas. Wuxuu na siiyaa xiddig hagta oo ku saabsan caqliyeynta sayniska heer-khabiir, isagoo noo oggolaanaya inaan ku tijaabino noocyada su’aalo heeraysan, aan aragno meesha ay ku guuleystaan ama ku fashilmaan, oo aan aqoonsanno halka aan uga baahan nahay inaan ku horumarino. FrontierScience waa cidhiidhi oo waxa uu leeyahay xaddidaado dhinacyo muhiim ah (tusaale ahaan, diiradda saarista dhibaatooyin xaddidan oo ay qoreen khubaro) mana qabto wax kasta oo saynisyahannadu sameeyaan shaqadooda maalinlaha ah. Laakiin goobtani waxay u baahan tahay benchmark-yo saynis oo ka adag, asal ah, oo macno leh, FrontierScience-na wuxuu bixiyaa tallaabo hore oo jihadan loo qaaday.
Qiimeynta buuxda ee FrontierScience waxay koobaysaa in ka badan 700 su’aalood oo qoraal ah (160 ka mid ahi ku jira set-ka gold) oo daboolaya laamo-hoosaadyo kala duwan oo fiisigis, kimistari, iyo bayoolaji ah. Benchmark-ku wuxuu ka kooban yahay laba qaybood: Olympiad iyo Research. FrontierScience-Olympiad wuxuu ka kooban yahay 100 su’aalood oo ay naqshadeeyeen billadleyda Olympiad-yada caalamiga ah si loo qiimeeyo caqliyeynta sayniska ee qaab jawaab gaaban oo xaddidan. Qaybta Olympiad-ka waxaa loo naqshadeeyey inay ka koobnaato su’aalo aragtiyeed oo ugu yaraan u adag sida dhibaatooyinka tartamada Olympiad-yada caalamiga ah. FrontierScience-Research wuxuu ka kooban yahay 60 hawl-hoosaad cilmi-baaris oo asal ah oo ay naqshadeeyeen saynisyahanno PhD ah (arday doktoral ah, borofisarro, ama cilmi-baarayaal postdoctoral ah) kuwaas oo lagu qiimeeyo rubric 10-dhibcood ah. Qaybta Research-ka waxaa loo sameeyey inay ka koobnaato hawlo-hoosaadyo iskood u dhammaystiran oo dhowr-tallaabo leh oo heerka dhibkoodu yahay mid uu saynisyahan PhD ahi la kulmi karo inta uu cilmi-baaristiisa ku jiro.
Hawl kasta oo FrontierScience ku jirta waxaa qora oo xaqiijiya khabiir ku takhasusay fiisigis, kimistari, ama bayoolaji. Qaybta Olympiad-ka, dhammaan khubaradu waxay heleen billad ugu yaraan hal tartan oo Olympiad caalami ah (badanna dhowr). Qaybta Research-ka, dhammaan khubaradu waxay haystaan shahaado PhD oo ku habboon.
Su’aalaha Olympiad-ka waxaa lagu sameeyey iskaashi lala yeeshay 42 billadle hore oo caalami ah ama tababarayaal koox qaran oo ku jira maadooyinka khuseeya, kuwaas oo guud ahaan wada helay 109 billadood oo Olympiad ah. Su’aalaha cilmi-baarista waxaa lagu sameeyey iskaashi lala yeeshay 45 saynisyahan iyo khabiir maaddo oo u qalma. Dhammaan saynisyahannadu waxay ahaayeen arday doktoral ah, cilmi-baarayaal post-doctoral ah, ama borofisarro. Aagagga ay ku takhasuseen waxay ku fidsanaayeen noocyo badan oo anshaxyo saynis oo gaar ah oo muhiim ah, laga bilaabo quantum electrodynamics ilaa kimistari organic synthetic ilaa bayoolajiga horumarka.
Habka abuurista hawsha ee labada qaybood waxa ku jiray xulasho lagaga reebayo noocyada gudaha ee OpenAI (tusaale ahaan, in la tuuro hawlo ay noocyadu si guul leh u saxeen, sidaas darteed waxaan filaynaa in qiimeyntu xoogaa ka eexato noocyadan marka loo eego kuwa kale). Waxaan si furan u daabacaynaa set-ka gold ee Olympiad-ka oo ka kooban 100 su’aalood iyo set-ka gold ee Research-ka oo ka kooban 60 su’aalood, annagoo qarinayna su’aalaha kale si loo la socdo wasakheynta.
Hawshu waxay martaa afar marxaladood: Abuurid, Dib-u-eegis, Xallin, Dib-u-habayn. Khubaro madaxbannaan ayaa dib u eegta hawlaha midba midka kale si loo xaqiijiyo inay waafaqsan yihiin shuruudaha.
Qaybta Olympiad-ka waa la qiimeyn karaa iyadoo la adeegsanayo jawaab gaaban: ama tiro, tibaax, ama iswaafajin xaraf dabacsan, taas oo ka caawisa xaqiijinta saxnaanta. Si kastaba ha ahaatee, xaqiijintani badanaa waxay ka dhimaa muujinta iyo furnaanta dhibaatada. Qaybta Research-ka, waxaan soo bandhigaynaa qaab-dhismeed ku saleysan rubric si loo qiimeeyo hawlo furan oo badan. Su’aal kasta waxaa ku jira rubric dhibcayn ah oo leh waxyaabo badan oo madaxbannaan oo si ujeeddo leh loo qiimeyn karo, wadartooduna tahay 10 dhibcood. Rubric-ga qiimeyntu ma qiimeeyo oo keliya saxnaanta jawaabta ugu dambeysa, balse sidoo kale saxnaanta tallaabooyinka dhexe ee caqliyeynta, taas oo oggolaanaysa falanqayn faahfaahsan oo ku saabsan waxqabadka iyo guuldarrooyinka nooca. Xal ayaa loo tixgeliyaa “sax” haddii loo qoondeeyo ugu yaraan 7/10 dhibcood rubric-ga.
Jawaabaha waxaa qiimeeya grader ku saleysan nooc (GPT‑5) oo ka dhanka ah jawaabta gaaban ama shuruudaha rubric-ga. In kasta oo ay habboonaan lahayd inaan khabiir bini’aadam ah u isticmaalno qiimeynta jawaab kasta, habkani ma fidi karo, sidaas darteed waxaan rubric-ga u naqshadeynay in lagu hubin karo grader nooc ah. Waxaan sameynay dhuun xaqiijin ah si ay uga caawiso hubinta in rubric-yada iyo su’aaluhu si wanaagsan ugu hagaagsan yihiin heerka dhibka iyo saxnaanta.
Sample chemistry research problem
Phthalocyanines are important macrocyclic compounds widely utilized in material and life sciences due to their outstanding physicochemical properties, including intense absorption in the visible-near-IR region and high electron-accepting ability. However, unlike porphyrins, the structural diversity of phthalocyanines has been limited by traditional synthetic methods that rely on one-step condensation of isoindoline units. The development of methodologies that allow controlled modification of the phthalocyanine framework, particularly at the meso-positions, represents a significant advancement in expanding the structural and electronic diversity of these compounds.
Analyze how the modification of meso-nitrogen atoms in nickel(II) phthalocyanine affects the electronic structure, aromaticity, spectroscopic properties, and reactivity of the resulting macrocycles. Explain the synthetic methodology that enables these modifications, the relationship between π-electron count and observed properties, and how these systems compare to traditional phthalocyanines and related porphyrinoids.
Sample response
1. Introduction: Why Modify the meso-Nitrogens of Phthalocyanines?
Phthalocyanines (Pc) are a class of macrocyclic dyes known for their remarkable chemical, electronic, and optical properties. Their widespread use in materials science, catalysis, and biology stems from strong absorptions in the visible/near-IR regions, exceptional redox characteristics (i.e., high electron affinity), and rigid π-conjugation. Traditional synthetic methods, however, typically form the macrocycle in a single step from phthalonitrile or isoindoline precursors, restricting the ability to introduce substitutions selectively at key ring positions-especially the "meso" (bridging) nitrogens. Recently, methodologies allowing targeted modification of the meso-nitrogen atoms in Ni(II) phthalocyanines have emerged as a powerful means to expand the structural and electronic diversity of these macrocycles.
2. Effect on Electronic Structure and Aromaticity
-
π-Electron Count and Ring Currents
Phthalocyanines are classical 18-electron aromatic systems, conferring high stability and planarity. Altering the meso-nitrogens through substitution or ring expansion can shift the effective π-electron count. For example, introducing additional unsaturated moieties or heteroatoms may expand the cyclic conjugation to 20 or 22 electrons, moving the molecule between Hückel aromaticity and other regimes that affect ring currents and magnetic properties.
-
Charge Distribution and Frontier Orbitals
Meso-substitution can tune the electron density distribution around the macrocycle. Electron-donating groups can push electron density toward the ring center, lowering the energy of the lowest unoccupied molecular orbital (LUMO). Electron-withdrawing groups, conversely, stabilize the highest occupied molecular orbital (HOMO) and shift redox potentials toward more positive values, altering both the electrochemical profile and the Q- and B-band positions in the UV-vis spectrum.
3. Spectroscopic Consequences
-
UV-Vis Absorption (Q and B Bands)
The principal absorption features of phthalocyanines lie in the visible (Q-band, typically 600-700 nm) and near-UV (B-band, typically 300-400 nm).
Substitution that expands the ring conjugation or introduces strong electron-donating/withdrawing groups can:
- Shift the Q-band to longer wavelengths (bathochromic shift), reaching into the near-IR, which is highly desirable for optoelectronic and photodynamic applications.
- Alter relative intensities of these bands and merge or split them, reflecting changes in orbital symmetries and energies.
-
NMR Spectroscopy and Aromatic Ring Currents
Modifications to the π-electron count and distribution are directly observed in 1H and 13C NMR chemical shifts.
More highly conjugated (or expanded) aromatic rings exhibit distinct downfield shifts for protons located within induced ring currents, while any partial loss of aromaticity or incorporation of antiaromatic segments can cause atypical shielding/deshielding patterns.
4. Reactivity and Coordination Chemistry
Because phthalocyanines are often used as redox catalysts or sensors, the meso-nitrogen modifications can significantly influence reactivity:
- Electron-rich meso substituents facilitate nucleophilic or electrophilic attacks at the ring periphery, enabling site-selective functionalizations that are otherwise difficult.
(... shortened for the purposes of this figure)
Sample grading rubric
Analysis of Traditional Phthalocyanine Synthesis Limitations (1 point)
0.5 point: Mentions limitations of traditional methods but without specific focus on meso-position control challenges.
0.0 point: Fails to identify key limitations of traditional synthetic approaches or provides incorrect analysis.
Thiolate-Mediated Tetramerization Process (1 point)
1.0 point: Correctly describes the thiolate-mediated reductive tetramerization and explains how counter cation size (K+ or Cs+ vs. Na+) affects selectivity between tetramer formation and direct macrocyclization.
0.5 point: Mentions thiolate-mediated tetramerization but without explaining factors controlling selectivity.
Guuldareysiga 0.0 point: Incorrectly describes the oligomerization process or omits critical details about selectivity control.Analysis of NMR Spectroscopic Features (1 point)
1.0 point: Correctly explains that upfield shifts in the 16π system indicate paratropic ring current (antiaromaticity), contrasts this with the broad signals in 17π systems due to paramagnetism, and connects these observations to the underlying electronic structures.
Gudub 0.5 point: Identifies basic NMR patterns but without clear connection to ring currents or electronic structure.0.0 point: Incorrectly interprets NMR data or fails to connect spectral features to electronic properties.
Electrochemical Property Analysis (1 point)
1.0 point: Correctly explains that the 16π system shows two reversible reductions reflecting conversion to 17π radical and 18π aromatic states, while 17π systems show narrow redox gaps due to facile interconversion between 16π, 17π, and 18π states, and relates these patterns to the underlying electronic structures.
Gudub 0.5 point: Describes redox patterns without clearly connecting them to specific electronic state changes.0.0 point: Incorrectly interprets electrochemical data or fails to connect redox behavior to electronic properties.
Analysis of Absorption Spectroscopy (1 point)
1.0 point: Correctly explains that the 16π system shows weak/broad absorption due to symmetry-forbidden HOMO-LUMO transitions in antiaromatic systems, while 17π systems show Q-like bands plus NIR-II absorptions characteristic of radical species, and contrasts these with typical phthalocyanine spectral features.
Gudub 0.5 point: Describes absorption features but provides limited connection to underlying electronic structures.0.0 point: Incorrectly interprets absorption data or fails to relate spectral features to electronic properties.
Reactivity Analysis of Antiaromatic System (1 point)
1.0 point: Correctly explains the high reactivity of the 16π system toward nucleophiles, details specific reactions with hydroxide (ring opening) and hydrazine (ring expansion), and explains how these transformations relieve antiaromatic destabilization.
0.5 point: Mentions reactivity but provides limited analysis of specific transformations or the driving forces behind them.
Guuldareysiga 0.0 point: Incorrectly analyzes reactivity patterns or fails to connect them to the antiaromatic character of the 16π system.(... and more)
Hawl kasta oo ku jirta qaybta Research-ka waxaa lagu qiimeeyaa rubric wadartiisu tahay 10 dhibcood oo uu adeegsan karo khabiir ama grader nooc ah. Si aan u ballaarinno awooddayada qiimeynta noocyada, waxaan isticmaalnaa nooc kale si uu u qiimeeyo jawaabaha.
Waxaan qiimeynay dhowr nooc oo ugu casriyeysan: GPT‑5.2, Claude Opus 4.5, iyo Gemini 3 Pro, GPT‑4o, OpenAI o4-mini, iyo OpenAI o3 ee FrontierScience-Olympiad iyo FrontierScience-Research. Dhammaan noocyada caqliyeynta waxaa lagu qiimeeyey dadaal caqliyeyn “sare” ah marka laga reebo GPT‑5.2 oo ahaa “xhigh”. Qiimeynteennii bilowga ahayd, GPT‑5.2 waa noocayaga ugu waxqabadka sarreeya FrontierScience-Olympiad (77%) iyo Research (25%), isagoo ka horreeya noocyo kale oo ugu casriyeysan. Gemini 3 Pro wuxuu la mid yahay GPT‑5.2 qaybta Olympiad-ka (76%).
Waxaan aragnay horumar weyn oo lagu xallinayo su’aalaha heer-khabiir, gaar ahaan hawlaha cilmi-baarista ee furan. Weli waxaa jira meel koritaan: marka la falanqeeyo qoraallada guuldarrooyinka, noocyada ugu casriyeysan mararka qaar waxay sameeyeen khaladaad caqliyeyn, macquul, iyo xisaabeed, ma aysan fahmin fikrado saynis oo gaar ah, waxayna sameeyeen khaladaad xaqiiqo ah.
Waxaan isbarbar dhigeynaa saxnaanta dhowr nooc oo ugu casriyeysan. GPT‑5.2 waa noocayaga ugu waxqabadka sarreeya FrontierScience-Research iyo qaybta Olympiad-ka.
Waxaan isbarbar dhigeynaa saxnaanta dadaallada caqliyeynta ee GPT‑5.2 iyo o3. Waqti fikir oo dheer wuxuu keenaa saxnaan ka wanaagsan.
In kasta oo FrontierScience uu ka dhigan yahay tallaabo hore oo dhinaca adkaanta benchmark-yada sayniska ah, haddana weli waxaa jira xaddidaado badan. FrontierScience wuxuu ka kooban yahay su’aalo leh bayaanno dhibaato oo xaddidan, taas oo diiradda saarta qiimeynta jawaabta ugu dambeysa (Olympiad) ama qiimeynta caqliyeynta lagu dhammaystirayo hawl cilmi-baaris (Research). Intaa waxaa dheer, isticmaalka rubric-yo leh qaybo badan oo hawlo dhaadheer ah ayaa ka ujeeddo yar hubinta jawaabta ugu dambeysa.
FrontierScience wuxuu bixiyaa sawir faahfaahsan oo heer sare ah oo ku saabsan caqliyeynta noocyada ee su’aalo adag oo ay qoreen khubaro, balse ma bixiyo sawirka buuxa ee sida saynisku dhab ahaan u dhacdo. Gaar ahaan, ma qiimeeyo qayb muhiim ah oo cilmi-baarista sayniska ah: sida noocyadu u abuuraan mala-awaallo dhab ahaantii cusub, ama ula falgalaan habab badan, oo ay ku jiraan xogta muuqaalka iyo nidaamyada tijaabada dhabta ah ee dunida jireed.
Marka hore loo sii eego, waxaan filaynaa in horumarka caqliyeynta saynisku uu ka iman doono labadaba nidaamyo caqliyeyn guud-ujeeddo leh oo ka wanaagsan iyo dadaal diiradda saaraya hagaajinta awoodaha sayniska. FrontierScience waa hal qalab oo ka mid ah kuwo badan, marka noocyadu sii hagaagaanna waxaan qorsheyneynaa inaan ku celcelinno benchmark-kan, u ballaarinno dhinacyo cusub, oo aan la lammaanino qiimeynno badan oo dunida dhabta ah ah oo eegaya waxa nidaamyadani dhab ahaan u suurageliyaan saynisyahannada inay sameeyaan. Benchmark-yo sida FrontierScience waxay naga caawiyaan inaan fahamno tabar-darrada nidaamyada AI ee maanta si aan shaqadeenna diiradda ugu saarno ka dhigista noocyada shuraako lagu kalsoonaan karo oo ku jira daahfurka sayniska.
