this post was submitted on 28 Feb 2024
474 points (97.4% liked)

Technology

59137 readers
2312 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
top 50 comments
sorted by: hot top controversial new old
[–] [email protected] 29 points 8 months ago* (last edited 8 months ago)

For a rough approach, imagine a parrot taught by another parrot, which was in turn taught by another parrot which was taught by a human.

Sure, some things might survive as somewhat understandable vaguelly human sounding sentences, but overall it's still going to be pretty bad a few parrots down the chain.

[–] [email protected] 12 points 8 months ago (1 children)

Its funny how something like this get posted every few days and people keep falling for it like its somehow going to end AI. The people that make these models are acutely aware of how to avoid model collapse.

It's totally fine for AI models to train on AI generated content that is of high enough quality. Part of the research to train models is building data sets with a text description matching the content, and filtering out content that is not organic enough (or even specifically including it as a 'bad' example for the AI to avoid). AI can produce material indistinguishable from human work, and it produces material that wasn't originally in the training data. There's no reason that can't be good training data itself.

[–] [email protected] 5 points 8 months ago

Especially since they can just pay someone to sit down and sift through it, or re-use the old training data that they already have from before it all blew up.

[–] [email protected] 40 points 8 months ago* (last edited 3 months ago)

Most people here don’t understand what this is saying.

We’ve had “pure” human generated data, verifiably so since LLMs and ImageGen didn’t exist. Any bot generated data was easily filterable due to lack of sophistication.

ChatGPT and SD3 enter the chat, generate nearly indistinguishable data from humans, but with a few errors here and there. These errors while few, are spectacular and make no sense to the training data.

2 years later, the internet is saturated with generated content. The old datasets are like gold now, since none of the new data is verifiably human.

This matters when you’ve played with local machine learning and understand how these machines “think”. If you feed an AI generated set to an AI as training data, it learns the mistakes as well as the data. Every generation it’s like mutations form until eventually it just produces garbage.

Training models on generated sets slowly by surely fail without a human touch. Scale this concept to the net fractionally. When 50% of your dataset is machine generated, 50% of your new model trained on it will begin to deteriorate. Do this long enough and that 50% becomes 60 to 70 and beyond.

Human creativity and thought have yet to be replicated. These models have no human ability to be discerning or sleep to recover errors. They simply learn imperfectly and generate new less perfect data in a digestible form.

[–] [email protected] 5 points 8 months ago

It would be hilarious if we entered the deep fried Marquaud era of ai where responses degenerate into rehashed responses that just get progressively more jumbled and unintelligible as the models cannibalise each other's generated content

[–] [email protected] 4 points 8 months ago* (last edited 8 months ago)

Like a billion hours of YouTube videos out there I am not seeing the issue plus the entire library of Congress

[–] [email protected] 3 points 8 months ago (3 children)

Wasn't there a paper not long time ago that it was possible to generate data with AI as a training set for AI? I was surprised (and the math is to much for me to check out my self) but that seems to solve that problem.

[–] [email protected] 1 points 8 months ago (1 children)

Sorta. This "model collapse" thing is basically an urban legend at this point.

The kernel of truth is this: A model learns stuff. When you use that model to generate training data, it will not output all it has learned. The second generation model will not know as much as the first. If you repeat this process a couple times, you are left with nothing. It's hard to see how this could become a problem in the real world.

Incest is a good analogy, if you know what the problem with inbreeding is: You lose genetic diversity. Still, breeders use this to get to desired traits and so does nature (genetic bottleneck, founder effect).

[–] [email protected] 2 points 8 months ago (1 children)

Training data for models in general was a big problem when I studied systems biology. Interesting that we finding works around, since it sounded rather fundamental to me. I found your metaphor rather helpful, thanks.

[–] [email protected] 3 points 8 months ago

I wouldn't say we've really found a workaround. AI companies hire lots of people to parse and clean data. That can work for things like pose estimation, which are largely a once and done thing. But for things that are constantly evolving, language/art/videos, it may not be a viable long term strategy.

[–] [email protected] 3 points 8 months ago (1 children)

As far as I know, that is mainly used where a better, bigger model generates training data for a more efficient smaller model to bring it a bit closer to its level.

Were there any cases of an already state of the art model using this method to improve itself?

[–] [email protected] 1 points 8 months ago* (last edited 8 months ago)

I will search for the paper.

EDIT: can't find it, dang.

[–] [email protected] 4 points 8 months ago (1 children)

Microsoft's Phi model was largely trained on synthetic data derived from GPT-4.

[–] [email protected] 1 points 8 months ago* (last edited 8 months ago)

I'm to lazy to search for the paper, not sure it was Microsoft, but with my rather basic knowledge of modeling (studied system biology) - it seemed rather crazy and impossible, so I remembered it.

[–] [email protected] 21 points 8 months ago (1 children)

The "solutions" to model collapse - essentially retraining on the original data set - suggests LLMs plateau or deteriorate. Especially without a way to separate out good and bad quality data (or ad they euohemistically try and say human vs AI data).

Were increasingly seeing the limitations and flaws with LLMs. "Hallucinations" or better described as serious errors, model collapse and complete collapse suggest the current approach to LLMs is probably not going to lead to some gone of general AI. We have models we don't really understand that have fundamental flaws and limitations.

Unsurprising that they probably can't live up to the hype.

[–] [email protected] -4 points 8 months ago

Even if it will plateau, same was said with moorrs law, which held up way longer than expected. There are so many ways to improve this. Open source community is getting to the point where you can actually run decent models on normal private hardware (talking about 70-120b model)

[–] [email protected] 10 points 8 months ago (4 children)

If the AI generated content is labeled, or has context, or has comments or descriptions created by people, then wouldn't it just be the same as synthetic training data? Which is shown to still be very useful for training.

[–] [email protected] 1 points 8 months ago

Sorta. This "model collapse" thing is basically an urban legend at this point.

The kernel of truth is this: A model learns stuff. When you use that model to generate training data, it will not output all it has learned. The second generation model will not know as much as the first. If you repeat this process a couple times, you are left with nothing. It's hard to see how this could become a problem in the real world.

Incest is a good analogy, if you know what the problem with inbreeding is: You lose genetic diversity. Still, breeders use this to get to desired traits and so does nature (genetic bottleneck, founder effect).

[–] [email protected] 7 points 8 months ago

Exactly what percentage of AI data in the wild is labeled?

Close to zero I'd say.

[–] [email protected] 8 points 8 months ago

Most AI-generated data in the wild won't have labels because there's no incentive to label it, and in a lot of cases there are incentives to not label it.

[–] [email protected] 8 points 8 months ago

Yes it's still useful and it's basically how we made our last couple of jumps. An AI training on AI generated data being graded by another AI. We've hit diminishing returns though.

load more comments
view more: next ›