457
China announces plan to label all AI-generated content with watermarks and metadata.
(english.news.cn)
National gpu registry!
this post was submitted on 17 Mar 2025
457 points (98.7% liked)
Technology
66831 readers
5007 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
I think there was a similar idea in the USA with the COPIED Act, but I haven't heard about it since.
As an exception to most regulations that we hear about from China, this approach actually seems well considered - something that might benefit people and work.
Similar regulations should be considered by other countries. Labeling generated content at the source, hopefully without the metadata being too extensive (this is where China might go off the handle) would help avoid at least two things:
- casual deception
- training AI with material generated by another AI, leading to degradation of ability to generate realistic content
USA announces plan to ban the buds of the cannabis plant
They plan to ban hating on the supreme leader.
China is long ahead with that so maybe there is hope.
About as enforceable as banning bitcoin.
Stable Diffusion has the option to include an invisible watermark. I saw this in the settings when I was running it locally. It does something like adds a pattern that is easy to detect with machines but impossible to see. The idea was that you could check an image for it before putting it into training sets. Because I never needed to lie about things I generated I left it on.
Something that we've needed for too long. Good on China :)
China, oh you Remembering something about go green and bla bla, but continue to create coal plants.
The Chinese government has been caught using AI for propaganda and claiming to be real. So I don't see it happening within the Chinese government etc.
It makes more sense to mark authentic content but sure.
Me: "hé remove the small text which is at the bottom right in this picture"
AI: "Done, here is the picture cleaned of the text"
Lol. So everything and anything can just be AI generated fakenews.
Will be interesting to see how they actually plan on controlling this. It seems unenforceable to me as long as people can generate images locally.
That's what they want. When people doing it locally, they can discredit anything as AI generated. The point isn't about enforability, but can it be a tool to control narative.
Edit: it doesn't matter if people actually generating locally, but if people can possibly doing it. As long as it is plausible, the argument stands and the loop completes.
It's not like this wasn't always the issue.
Anything and everything can be labelled as misinformation.
True. It is just another avenue to label things.
Having some AIs that do this and some not will only muddy the waters of what’s believable. We’ll get gullible people seeing the ridiculous and thinking “Well there’s no watermark so it MUST be true.”
Sorry but the problem right now is much simpler. Gullibility doesn't require some logical premise. "It sounds right so it MUST be true" is where the thought process ends.
And the lack of label just reinforced the confirmation bias.
This is a bad idea. It creates a stigma and bias against innocent Artificial beings. This is the equivalent of forcing a human to wear a collar. TM watermark
Forgot the /s I assume
But I put in the water mark!
Imma be honest with ya, did not notice it at all lol now I see what you did there and no /s needed of course
That's something that was really needed.
Would it be more effective to have something where cameras digitally sign the photos? Then, it also makes photos more attributable, which sounds like China's thing.
Sort of. A camera with internet connectivity could automatically "notarize" photos. The signing authority would vouch that the photo (or other file) hasn't been altered since the moment of signing. It wouldn't be evidence that the photo was not manipulated before that moment.
That could make, EG, photos of a traffic accident good evidence in court. If there wasn't time to for manipulation, then the photos must be real. It wouldn't work for photos that could have been taken at any time.
You could upload a hash to the blockchain of a cryptocurrency for the same purpose. The integrity of the cryptocurrency would then vouch that the photo was unaltered since the upload. But that's not cost-effective. You could even upload the hash to Reddit, since it's not believable that they would manipulate timestamps to help some random guy somewhere in the world commit fraud.
This is the one area where blockchain could have been useful instead of greater-fool money schemes. A system where people can verify provenance of images or videos pertaining to matters of importance such as news stories. All reputable journalism already attributes their photos anyways. Cryptographic signing is just taking it to a logical conclusion. But of course the scary word 'china' is involved here therefore we must only contrarian post.
That's actually already a thing: https://www.theregister.com/2022/08/15/sony_launches_forgeryproof_incamera_digital/
That's a different thing. C2PA is proving a photo is came from a real camera, with all the editing trails. All in a cryptographic manner. This in the topic is trying to prove what not real is not real, by self claiming. You can add the watermark, remove it, add another watermark of another AI, or whatever you want. You can just forge it outright because I didn't see cryptographic proof like a digital sign is required.
Btw, the C2PA data can be stripped if you know how, just like any watermarks and digital signatures.
Stripping C2PA simply removes the reliability part, which is fine if you don't need it. It is something that is effective when present and not when it isn't.
It's never effective. At best, you could make the argument that a certain person lacks the wherewithal to have manipulated a signature, or gotten someone else to do it. One has to hope that the marketing BS does not convince courts to assign undue weight to forged evidence.
No, I don't want my photos digitally signed and tracked, and I'm sure no whistleblower wants that either.
Of course not. Why would they? I don’t want that either. But we are considering the actions of an authoritarian system.
Individual privacy isn’t relevant in such a country. However, it’s an interesting choice that they implement it this way.
Apart from the privacy issues, I guess the challenge would be how you preserve the signature through ordinary editing. You could embed the unedited, signed photo into the edited one, but you'd need new formats and it would make the files huge. Or maybe you could deposit the original to some public and unalterable storage using something like a blockchain, but it would bring large storage and processing requirements. Or you could have the editing software apply a digital signature to track the provenance of an edit, but then anyone could make a signed edit and it wouldn't prove anything about the veracity of the photo's content.
Hm, that’s true there’s no way to distinguish between editing software and photos that have been completely generated. It only helps if you want to preserve and modified photos. And of course, I’m making assumptions here that China doesn’t care very much about privacy.
This is a smart and ethical way to include AI into everyday use, though I hope the watermarks are not easily removed.
I'm going to develop a new AI designed to remove watermarks from AI generated content. I'm still looking for investors if you're interested! You could get in on the ground floor!
I've got a system that removes the watermark and adds two or three bonus fingers, free of charge! Silicon Valley VC is gonna be all over this.
It will be relatively easy to strip that stuff off. It might help a little bit with internet searches or whatever, but anyone spreading deepfakes will probably not be stopped by that. Still better than nothing, I guess.
it will be relatively easy to strip off
How so? If it's anything like llm text based "water marks" the watermark is an integral part of the output. For an llm it's about downrating certain words in the output, I'm guessing for photos you could do the same with certain colors, so if this variation of teal shows up more than this variation then it's made by ai.
I guess the difference with images is that since you're not doing the "guess the next word" aspect and feeding the output from the previous step into the next one, you can't generate the red green list from the previous output.
Having an unreliable verification method is worse than nothing.
Think a layer deeper how can it misused to control naratives.
You read some wild allegation, no AI marks (they required to be visible), so must written by someone? Right? What if someone, even the government jumps out as said someone use an illiegal AI to generate the text? The questioning of the matter will suddently from verifying if the allegation decribed happened, to if it itself is real. The public sentiment will likely overwhelmed by "Is this fakenews?" or "Is the allegation true?" Compound that with trusted entities, discrediting anything become easier.
Give you a real example. Before Covid spread globally there was a Chinese whistleblower, worked in the hospital and get infected. He posted a video online about how bad it was, and quickly got taken down by the government. What if it happened today with the regulation in full force? Government can claim it is AI generated. The whistleblower doesn't exist. Nor the content is real. 3 days later, they arrested a guy, claiming he spread fakenews using AI. They already have a very efficient way to control naratives, and this piece of garbage just give them an express way.
You though that only a China thing? No, every entities including governments are watching, especially the self-claimed friend of Putin and Xi, and the absolute free speech lover. Don't think it is too far to reach you yet.
It's still a good thing. The alternative is people posting AI content as though it is real content, which is a worldwide problem destroying entire industries. All AI content should by law have to be clearly labeled.
Then what AI generated slop without label are to the plain eyes? That label just encourge the laziness of the brain as an "easy filter." Those slop without label just evelated itself to be somewhat real, becuase the label exist exploiting the laziness.
Before you said some AI slop are clearly identifiable, you can't rule out everyone can, and every piece are that identifiable. And for those images that looks a little unrealistic, just decrease the resolution to very grainy and hide those details. That will work 9 out of 10. You can't rule out that 0.1% content that pass sanity check can't do 99.9% damage.
After all, human are emotional creatures, and sansationism is real. The urge of share something emotional is why misinformation and disinformation are so common these days. People will overlook details when the urge hits.
Somethimes, labeling can do more harm than good. It just give a false sense.
Just because something is theoretically circumventable doesn't mean we shouldn't make it as hard as possible to circumvent it.
The reason why misinformation is so common these days is because of concerted effort by fascists to obtain control over media companies. Once they are in power and have significant influence within those companies they can poison them, turning them into massive misinformation engines churning out content at a pace even faster than we ever believed possible. This problem has existed since the rise of mass media especially in the 19th century. But social media presents far faster and more direct throughlines to spreading misinformation to the masses.
And those masses do not care if something is labeled as AI or not. They will believe it one way or the other. This still doesn't change that it is necessary to directly label AI generated content as such. What is and isn't made by a human is extremely important. We cannot equate algorithms with people, and it's necessary to make that distinguishment as clearly as possible.
The problem is you can't make a digital label that hard to circumvent. Much like a signature, you sign something you want to prove it is genuinely from you, but you won't sign something that's not from you while not signing things that are, especially in digital format. Digital signature can just be stripped out of the data. Watermarks on images can now patched with the help of inpainting models. Disclaimers in text can just be deleted. The default shouldn't be "This thing doesn't have an AI label so it would be written by human." The label itself it a slippery slope that helps misinformation spread faster and aid building alternate facts. Adding a label won't help people identify contents generated with ML models, but let them defer the identification to that mere label because it said so, or didn't.
Misinformation didn't spread fast simply because fascists obtained controls on medias. Just look at how China, Russia, and Iran launch misinformation campaigns. They didn't have to control those media, but some seed accounts that make sensational title that attracts people in more powerful position and recognition to spread it out. For more info on misinformation and disinformation, I recommend you watch Ryan McBeth's video on YT.
Yes, we need a way to identify what is and what not generated by ML models, but that should not be done by labeling ML contents.
I'm curious what you would suggest to aid identifying generated content if not clear labeling. Sure its circumventable but again its more than what already exists. It provides legal precedence for repercussions to companies trying to pass off AI generated content as human created.
Please allow me to have a little bit of time deep thoughts and organize myself. It might take a while, but I will give you a response.