I feel like this shit could legitimately lead to the downfall of modern society. If it hasn't already.
this post was submitted on 10 May 2025
127 points (100.0% liked)
Technology
38780 readers
223 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 3 years ago
MODERATORS
There ain't much further to fall down...
ChatGPT is worse. The others not so much.
I'm a little too lazy to check and compare the ratios of these charts, but Gemini literally did so bad compared to ChatGPT in terms of accuracy
Have you used gemini or the Google ai overview? Absolutely atrocious. Chatgpt is wildly wrong at times, but gemini blows my mind st how bad it is.
Interesting fun fact: How Bad It Is is the patron saint of LLMs.
What if AIs already became sentient and this is their way of trying to get us to leave them alone?
To be fair, humans tend to hallucinate too.
I asked a local Ai chatbot DeepSeek-R1-Distill-Quen-14B:
Does Ai hallucinate and is it getting worse? Answer in one paragraph.
Answer:
AI does exhibit a form of "hallucination" where it generates plausible but incorrect information, particularly when faced with uncertainty or creative prompts beyond its training scope. This phenomenon occurs as AI constructs logical-sounding answers using available data, even if those answers aren't accurate. While advancements in AI make these hallucinations more convincing and harder to detect due to improved coherence, the issue isn't worsening in severity but rather evolving in how it presents itself.
I think the difference is that humans are sometimes aware of it. A human will likely say, I don't know what Kanye West did in 2018. While the AI is very likely to make up something. And also in contrast to a human this will likely be phrased like a Wikipedia article. While you can often look a human in the eyes and know whether they tell the truth or lie, or are uncertain. Not always, and we also tell untrue things, but I think the hallucinations are kind of different in several ways.
I mean lot of stuff is written in forums and social media, where people hallucinate. Or even in real life if you talk to one. Its normal for a human to pick up something in their life, later talk about it as a fact, regardless of where they learned it (tv, forum, videogame, school). Hallucinations are part of our brain.
Sometimes being aware of the hallucination issue is still a hallucination. Sometimes we are also aware of the hallucination an Ai makes, because its obvious or we can check it. And also there are Ai chatbots who "talk" and phrase in a more human natural sounding way. Not all of them sound obvious robotic.
Just for the record, I'm skeptical of Ai technology... not biggest fan. Please don't fork me. :D
Yeah, sure. No offense. I mean we have different humans as well. I got friends who will talk about a subject and they've read some article about it and they'll tell me a lot of facts and I rarely see them make any mistakes at all or confuse things. And then I got friends who like to talk a lot, and I better check where they picked that up.
I think I'm somewhere in the middle. I definitely make mistakes. But sometimes my brain manages to store where I picked something up and whether that was speculation, opinion or fact, along with the information itself. I've had professors who would quote information verbatim and tell roughly where and in which book to find it.
With AI I'm currently very cautious. I've seen lots of confabulated summaries, made-up facts. And if designed to, it'll write it in a professional tone. I'm not opposed to AI or a big fan of some applications either. I just think it's still very far away from what I've seen some humans are able to do.
They should be here to frig off!
Well by design ai is always hallucinating. Lol. That is how they work. Basically trying to hallucinate and predict the next word / token.
No, at least not in the sense that "hallucination" is used in the context of LLMs. It is specifically used to differentiate between the two cases you jumbled together: outputting correct information (as is represented in the training data) vs outputting "made-up" information.
A language model doesn't "try" anything, it does what it is trained to do - predict the next token, yes, but that is not hallucination, that is the training objective.
Also, though not widely used, there are other types of LLMs, e.g. diffusion-based ones, which actually do not use a next token prediction objective and rather iteratively predict parts of the text in multiple places at once (Llada is one such example). And, of course, these models also hallucinate a bunch if you let them.
Redefining a term to suit some straw man AI boogeyman hate only makes it harder to properly discuss these issues.
This is why AGI is way off and any publicly trained models will ultimately fail. Where you’ll see AI actually be useful will be tightly controlled, in house or privately developed models. But they’re gonna be expensive and highly specialized as a result.
I'd go further: you won't reach AGI through LLM development. It's like randomly throwing bricks on a construction site, no cement, and hoping that you'll get a house.
I'm not even sure if AGI is cost-wise feasible with the current hardware, we'd probably need cheaper calculations per unit of energy.
I can't find any backing for the claim in the title "and they're here to stay". I think that's just made up. Truth is, we found two ways which don't work. And that's making them larger and "think". But that doesn't really rule out anything. I agree that that's a huge issue for AI applications. And so far we weren't able to tackle it.
They don't think. They use statistical models on massive data sets to achieve the statistically average result from the data set.
In order to have increased creativity, you need to increase the likelihood of it randomly inserting things outside that result: hallucinations.
You cannot have a creative "AI" without them with the current fundamental design.
I get that. We want them to be creative and make up an eMail for us. Though I don't think there is any fundamental barrier preventing us from guiding LLMs. Can't we just make it aware whether the current task is reciting Wikipedia or creative storywriting? Or whether it's supposed to focus on the input text or its background knowledge? Currently we don't. But I don't see how that would be theoretically impossible.
And that’s making them larger and “think."
Isn't that the two big strings to the bow of LLM development these days? If those don't work, how isn't it the case that hallucinations "are here to stay"?
Sure, it might theoretically happen that some new trick is devised that fixes the issue, and I'm sure that will happen eventually, but there's no promise of it being anytime even remotely soon.
I'm not a machine learning expert at all. But I'd say we're not set on the transformer architecture. Maybe just invent a different architecture which isn't subject to that? Or maybe specifically factor this in. Isn't the way we currently train LLM base models to just feed in all text they can get? From Wikipedia and research papers to all fictional books from Anna's archive and weird Reddit and internet talk? I wouldn't be surprised if they start to make things up since we train them on factual information and fiction and creative writing without any distinction... Maybe we should add something to the architecture to make it aware of the factuality of text, and guide this... Or: I've skimmed some papers a year or so ago, where they had a look at the activations. Maybe do some more research what parts of an LLM are concerned with "creativity" or "factuality" and expose that to the user. Or study how hallucinations work internally and then try to isolate this so it can be handled accordingly?
The whole thing can be summed up as the following: they're selling you a hammer and telling you to use it with screws. Once you hammer the screw, it trashes the wood really bad. Then they're calling the wood trashing "hallucination", and promising you better hammers that won't do this. Except a hammer is not a tool to use with screws dammit, you should be using a screwdriver.
An AI leaderboard suggests the newest reasoning models used in chatbots are producing less accurate results because of higher hallucination rates.
So he's suggesting that the models are producing less accurate results... because they have higher rates of less accurate results? This is a tautological pseudo-explanation.
AI chatbots from tech companies such as OpenAI and Google have been getting so-called reasoning upgrades over the past months
When are people going to accept the fact that large "language" models are not general intelligence?
ideally to make them better at giving us answers we can trust
Those models are useful, but only a fool trusts = is gullible towards their output.
OpenAI says the reasoning process isn’t to blame.
Just like my dog isn't to blame for the holes in my garden. Because I don't have a dog.
This is sounding more and more like model collapse - models perform worse when trained on the output of other models.
inb4 sealions asking what's my definition of reasoning in 3...2...1...
ai is just too nifty word even if its gross misuse of the term. large language model doesnt roll of the tongue as easily.
The goalpost has shifted a lot in the past few years, but in the broader and even narrower definition, current language models are precisely what was meant by AI and generally fall into that category of computer program. They aren't broad / general AI, but definitely narrow / weak AI systems.
I get that it's trendy to shit on LLMs, often for good reason, but that should not mean we just redefine terms because some system doesn't fit our idealized under-informed definition of a technical term.
well, i guess i can stop feeling like i'm using wrong word for them then
What is your definition of reasoning?
It's not shoving AI slop into it again to get a new AI slop? Until it stops, because it reached the point where it's just done?
What ancient wizzardry do you use for your reasoning at home if not that?
But like look, we've had shit like this since forever, it's increasingly obvious that most people will cheer for anything, so the new ideas just get bigger and bigger. Can't wait for the replacement, I dare not even think about what's next. But for the love of fuck, don't let it be quantums. Please, I beg the world.
Why not quanta? Don't you believe in the power of the crystals? Quantum vibrations of the Universe from negative ions from the Himalayan salt lamps give you 153.7% better spiritual connection with the soul of the cosmic rays of the Unity!
...what makes me sadder about the generative models is that the underlying tech is genuinely interesting. For example, for languages with large presence online they get the grammar right, so stuff like "give me a [declension | conjugation] table for [noun | verb]" works great, and if it's any application where accuracy isn't a big deal (like "give me ideas for [thing]") you'll probably get some interesting output. But it certainly not give you reliable info about most stuff, unless directly copied from elsewhere.
It's a bit fucking expensive for a grammar tool.
I get that it gets logarithmically more expensive for every last bit of grammar, and some languages have very ridiculous nonsensical rules.
But I wish it had some broader use, that would justify its cost.
Yes, it is expensive. But most of that cost is not because of simple applications, like in my example with grammar tables. It's because those models have been scaled up to a bazillion parameters and "trained" with a gorillabyte of scrapped data, in the hopes they'll magically reach sentience and stop telling you to put glue on pizza. It's because of meaning (semantics and pragmatics), not grammar.
Also, natural languages don't really have nonsensical rules; sure, sometimes you see some weird stuff (like Italian genderbending plurals, or English question formation), but even those are procedural: "if X, do Y". LLMs are actually rather good at regenerating those procedural rules based on examples from the data.
But I wish it had some broader use, that would justify its cost.
I with that they cut down the costs based on the current uses. Small models for specific applications, dirty cheap in both training and running costs.
(In both our cases, it's about matching cost vs. use.)
But that won't happen, since the bubble rose on promises of gorillions of returns, and those have not manifested yet.
We are so fucking stupid, I hate this timeline.
I work in this field. In my company, we use smaller, specialized models all the time. Ignore the VC hype bubble.
There are many interesting AI applications, LLM or otherwise, but I'm talking about the IT bubble, that grows so big it will finally consume the industry. If it ever pops, the correction will not be pretty. For anyone.
I evaded the BS for now, but it feels like I won't be able to hide much longer. And it saddens me. I used to love IT :(
Most of us have no use for quantum computers. That's a government/research thing. I have no idea what the next disruptive technology will be. They are working hard on AGI, which has the potential to be genuinely disruptive and world changing, but LLMs are not the path to get there and I have no idea whether they are anywhere close to achieving it.
Surprise surprise, most of us have no use for LLMs.
And yet everyone and their gradma is using it for everything.
People asked GPT who would the next pope be.
Or which car to buy.
Or what's a good local salary.
I'm so fucking tired of all the shit.