Technology

58689 readers

4036 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

[email protected]

374

Hacker plants false memories in ChatGPT to steal user data in perpetuity (arstechnica.com)

submitted 3 weeks ago by [email protected] to c/[email protected]

35 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 1 points 3 weeks ago* (last edited 3 weeks ago)

Sort of, but not really.

In basic terms, if an LLM's training data has:

Bob is 21 years old.

Bob is 32 years old.

Then when it tries to predict the next word after "Bob is", it would pick 21 or 32 assuming somehow the weights were perfectly equal between the two (weight being based on how many times it occurred in training data around other words).

If the user has memories turned on, it's sort of like providing additional training data. So if in previous prompts you said:

I am Bob.

I am 43 years old.

The system will parse that and use it with a higher weight, sort of like custom training the model. This is not exactly how it works, because training is much more in-depth, it's more of a layer on top of the training, but hopefully gives you an idea.

The catch is it's still not reliable, as the other words in your prompt may still lead the LLM to predict a word from it's original training data. Tuning the weights is not a one-size fits all endeavor. What works for:

How old am I?

May not work for:

What age is Bob?

For instance.