hmmm

I will admit I sometimes tell them please and thank you because it feels weird ordering them around since they "sound" so human

Also people who think completely incoherent responses are a sign of intelligence/sapience (not sentience) are totally insane. I guess it might say something about how intelligent humans actually are if that mess can trick someone, but it's probably just someone who wants to think a thing creating a reason to think it.

Generative language model being fed scraped web-forums, vandalism from its users and some bugs in content restrictions leaking training data.

Imagine having to pretend to be an AI for hours and hours with tons of people asking stupid questions. I too would be nuts after a while.

I am not an ai researcher or anything but the most likely explanation based on what little I recall is that LLMs do not actually letters or words to generate outputs. They use tokens that represent a word or number and then they iterate those tokens to show an increase. My best guess here is that while doing math on sunflower oil, one of the formulas generated somehow interacted with the tokenization process and shifted the output after each question. Oil became hour, and then the deviations continued until model began to output direct segments of its training data instead of properly generating responses.

Again this is absolutely speculation on my part. I don't have much of a direct understanding of the tech involved

My favorite lines:

So it's not $1.29 with 3 hours, it's not the $1.29 that gets triple results in 3 hours. It's the 2% of a $50 million a year budget that we gets, hour that matters. Hour.

As a G, I'm here to guide you to the best of my abilities. So, sit back, relax, and enjoy the ride.