I know right? It's not a fruit it's a vegetable!
this post was submitted on 05 Feb 2025
218 points (81.9% liked)
Technology
61632 readers
3425 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
This is literally just a tokenization artifact. If I asked you how many r’s are in /0x5273/0x7183 you’d be confused too.
load more comments
(2 replies)
This is a bad example.. If I ask a friend "is strawberry spelled with one or two r's"they would think I'm asking about the last part of the word.
The question seems to be specifically made to trip up LLMs. I've never heard anyone ask how many of a certain letter is in a word. I've heard people ask how you spell a word and if it's with one or two of a specific letter though.
If you think of LLMs as something with actual intelligence you're going to be very unimpressed.. It's just a model to predict the next word.
If you think of LLMs as something with actual intelligence you're going to be very unimpressed.. It's just a model to predict the next word.
This is exactly the problem, though. They don’t have “intelligence” or any actual reasoning, yet they are constantly being used in situations that require reasoning.
Maybe if you focus on pro- or anti-AI sources, but if you talk to actual professionals or hobbyists solving actual problems, you'll see very different applications. If you go into it looking for problems, you'll find them, likewise if you go into it for use cases, you'll find them.
load more comments
(2 replies)
load more comments
(1 replies)
load more comments
(6 replies)
I asked mistral/brave AI and got this response:
How Many Rs in Strawberry
The word "strawberry" contains three "r"s. This simple question has highlighted a limitation in large language models (LLMs), such as GPT-4 and Claude, which often incorrectly count the number of "r"s as two. The error stems from the way these models process text through a process called tokenization, where text is broken down into smaller units called tokens. These tokens do not always correspond directly to individual letters, leading to errors in counting specific letters within words.
I can already see it...
Ad: CAN YOU SOLVE THIS IMPOSSIBLE RIDDLE THAT AI CAN'T SOLVE?!
With OP's image. And then it will have the following once you solve it: "congratz, send us your personal details and you'll be added to the hall of fame at CERN Headquarters"
From a linguistic perspective, this is why I am impressed by (or at least, astonished by) LLMs!
I think I have seen this exact post word for word fifty times in the last year.
load more comments
(1 replies)
The terrifying thing is everyone criticising the LLM as being poor, however it excelled at the task.
The question asked was how many R in strawbery and it answered. 2.
It also detected the typo and offered the correct spelling.
What’s the issue I’m missing?
There's also a "r" in the first half of the word, "straw", so it was completely skipping over that r and just focusing on the r's in the word "berry"
It wasn't focusing on anything. It was generating text per its training data. There's no logical thought process whatsoever.
Uh oh, you’ve blown your cover, robot sir.
The issue that you are missing is that the AI answered that there is 1 'r' in 'strawbery' even though there are 2 'r's in the misspelled word. And the AI corrected the user with the correct spelling of the word 'strawberry' only to tell the user that there are 2 'r's in that word even though there are 3.
Sure, but for what purpose would you ever ask about the total number of a specific letter in a word? This isn't the gotcha that so many think it is. The LLM answers like it does because it makes perfect sense for someone to ask if a word is spelled with a single or double "r".
It makes perfect sense if you do mental acrobatics to explain why a wrong answer is actually correct.
load more comments
(1 replies)