Wake me up when someone besides OpenAI says they're the best at something. When a company releases a benchmark they designed that their own tool that's generally regarded as not very good is suddenly the best at, that's not news, at best that's PR, at worst propaganda. This reeks of "we investigated ourselves and found we did nothing wrong".
this post was submitted on 14 May 2025
-24 points (32.9% liked)
Technology
70049 readers
3719 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
US Healthcare will now be affordable!
Tl;dr: After performing poorly on benchmarks, OpenAI created their own. OpenAI products perform much better on OpenAI benchmark.
The bar exam isn’t created by OpenAI, yet the outdated GPT-4 model still ranked in the 90th percentile on it.
So they created a test so broken and warped that no actual professional can understand it but their AI performs well on it?
I almost feel sad for IBM, this was supposed to be their thing.
Had forgotten we’ve been promised this before.
Sure thing
Wow.
Much wow.
Those are analytical AI right? More procedural, and also, the data to train the AI are consented. I tried Ada, it didn't give me official diagnosis but rather helping me to talk to doctor