this post was submitted on 27 May 2025
2062 points (99.4% liked)

Programmer Humor

23794 readers
2812 users here now

Welcome to Programmer Humor!

This is a place where you can post jokes, memes, humor, etc. related to programming!

For sharing awful code theres also Programming Horror.

Rules

founded 2 years ago
MODERATORS
 
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 2 days ago (1 children)

That’s unreal. No, you cannot come up with your own scientific test to determine a language model’s capacity for understanding. You don’t even have access to the “thinking” side of the LLM.

[–] [email protected] 1 points 2 days ago* (last edited 1 day ago)

You can devise a task it couldn't have seen in the training data, I mean.

You don’t even have access to the “thinking” side of the LLM.

Obviously, that goes for the natural intelligences too, so it's not really a fair thing to require.