this post was submitted on 12 Sep 2024

421 points (98.8% liked)

Fediverse

28713 readers

548 users here now

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).

If you wanted to get help with moderating your own community then head over to [email protected]!

Rules

Posts must be on topic.
Be respectful of others.
Cite the sources used for graphs and other statistics.
Follow the general Lemmy.world rules.

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration), Search Lemmy

founded 2 years ago

MODERATORS

[email protected]

421

Instance Admins: Check Your Instance for Vote Manipulation Accounts [PSA] (dubvee.org)

submitted 3 months ago* (last edited 3 months ago) by [email protected] to c/[email protected]

181 comments fedilink hide all child comments

Over the past 5-6 months, I've been noticing a lot of new accounts spinning up that look like this format:

https://instance.xyz/u/gmbpjtmt
https://instance.xyz/u/tjrwwiif
https://instance.xyz/u/xzowaikv

What are they doing?

They're boosting and/or downvoting mostly, if not exclusively, US news and politics posts/comments to fit their agenda.

Edit: Could also be manipulating other regional news/politics, but my instance is regional and doesn't subscribe to those which limits my visibility into the overall manipulation patterns.

What do these have in common?

Most are on instances that have signups without applications (I'm guessing the few that are on instances with applications may be from before those were enabled since those are several months old, but just a guess; they could have easily just applied and been approved.)
Most are random 8-character usernames (occasionally 7 or 9 characters)
Most have a common set of users they're upvoting and/or downvoting consistently
No posts/comments
No avatar or bio (that's pretty common in general, but combine it with the other common attributes)
Update: Have had several anonymous reports (thanks!) that these users are registering with an @sharklasers.com email address which is a throwaway email service.

What can you, as an instance admin, do?

Keep an eye on new registrations to your instance. If you see any that fit this pattern, pick a few (and a few off this list) and see if they're voting along the same lines. You can also look in the login_token table to see if there is IP address overlap with other users on your instance and/or any other of these kinds of accounts.

You can also check the local_user table to see if the email addresses are from the same provider (not a guaranteed way to match them, but it can be a clue) or if they're they same email address using plus-addressing (e.g. [email protected], [email protected], etc).

Why are they doing this?

Your guess is as good as mine, but US elections are in a few months, and I highly suspect some kind of interference campaign based on the volume of these that are being spun up and the content that's being manipulated. That, or someone, possibly even a ghost or an alien life form, really wants the impression of public opinion being on their side. Just because I don't know exactly why doesn't mean that something fishy isn't happening that other admins should be aware of.

Who are the known culprits?

These are ones fitting that pattern which have been identified. There are certainly more, but these have been positively identified. Some were omitted since they were more garden-variety "to win an argument" style manipulation.

These all seem to be part of a campaign. This list is by no means comprehensive, and if there are any false positives, I do apologize. I've tried to separate out the "garden variety" type from the ones suspected of being part of a campaign, but may have missed some.

[New: 9/18/2024]: https://thelemmy.club/u/fxgwxqdr
[New: 9/18/2024]: https://discuss.online/u/nyubznrw
[New: 9/18/2024]: https://thelemmy.club/u/ththygij
[New: 9/18/2024]: https://ttrpg.network/u/umwagkpn
[New: 9/18/2024]: https://lemdro.id/u/dybyzgnn
[New: 9/18/2024]: https://lemmy.cafe/u/evtmowdq
https://leminal.space/u/mpiaaqzq
https://lemy.lol/u/ihuklfle
https://lemy.lol/u/iltxlmlr
https://lemy.lol/u/szxabejt
https://lemy.lol/u/woyjtear
https://lemy.lol/u/jikuwwrq
https://lemy.lol/u/matkalla
https://lemmy.ca/u/vlnligvx
https://ttrpg.network/u/kmjsxpie
https://lemmings.world/u/ueosqnhy
https://lemmings.world/u/mx_myxlplyx
https://startrek.website/u/girlbpzj
https://startrek.website/u/iorxkrdu
https://lemy.lol/u/tjrwwiif
https://lemy.lol/u/gmbpjtmt
https://thelemmy.club/u/avlnfqko
https://lemmy.today/u/blmpaxlm
https://lemy.lol/u/xhivhquf
https://sh.itjust.works/u/ntiytakd
https://jlai.lu/u/rpxhldtm
https://sh.itjust.works/u/ynvzpcbn
https://lazysoci.al/u/sksgvypn
https://lemy.lol/u/xzowaikv
https://lemy.lol/u/yecwilqu
https://lemy.lol/u/hwbjkxly
https://lemy.lol/u/kafbmgsy
https://discuss.online/u/tcjqmgzd
https://thelemmy.club/u/vcnzovqk
https://lemy.lol/u/gqvnyvvz
https://lazysoci.al/u/shcimfi
https://lemy.lol/u/u0hc7r
https://startrek.website/u/uoisqaru
https://jlai.lu/u/dtxiuwdx
https://discuss.online/u/oxwquohe
https://thelemmy.club/u/iicnhcqx
https://lemmings.world/u/uzinumke
https://startrek.website/u/evuorban
https://thelemmy.club/u/dswaxohe
https://lemdro.id/u/efkntptt
https://lemy.lol/u/ozgaolvw
https://lemy.lol/u/knylgpdv
https://discuss.online/u/omnajmxc
https://lemmy.cafe/u/iankglbrdurvstw
https://lemmy.ca/u/awuochoj
https://leminal.space/u/tjrwwiif
https://lemy.lol/u/basjcgsz
https://lemy.lol/u/smkkzswd
https://lazysoci.al/u/qokpsqnw
https://lemy.lol/u/ncvahblj
https://ttrpg.network/u/hputoioz
https://lazysoci.al/u/lghikcpj
https://lemmy.ca/u/xnjaqbzs
https://lemy.lol/u/yonkz

Edit: If you see anyone from your instance on here, please please please verify before taking any action. I'm only able to cross-check these against the content my instance is aware of.

(page 3) 50 comments

sorted by: hot top controversial new old

[–] [email protected] 7 points 3 months ago* (last edited 3 months ago) (12 children)

In case people are wondering these bots are advocating for Kamala Harris.

Dessalines is the developer of Lemmy and occasionally notifies users their posts are being botted

Dessalines notified me on two of my own posts that they were heavily downvoted by low /zero content accounts in the past as well.

load more comments (11 replies)

[–] [email protected] 19 points 3 months ago (4 children)

Is there any existing opensource tool for manipulation detection for lemmy? If not we should create one to reduce the manual workload for instance admins

load more comments (4 replies)

[–] [email protected] 51 points 3 months ago* (last edited 3 months ago) (2 children)

My bachelor's thesis was about comment amplifying/deamplifying on reddit using Graph Neural Networks (PyTorch-Geometric).

Essentially: there used to be commenters who would constantly agree / disagree with a particular sentiment, and these would be used to amplify / deamplify opinions, respectively. Using a set of metrics [1], I fed it into a Graph Neural Network (GNN) and it produced reasonably well results back in the day. Since Pytorch-Geomteric has been out, there's been numerous advancements to GNN research as a whole, and I suspect it would be significantly more developed now.

Since upvotes are known to the instance administrator (for brevity, not getting into the fediverse aspect of this), and since their email addresses are known too, I believe that these two pieces of information can be accounted for in order to detect patterns. This would lead to much better results.

In the beginning, such a solution needs to look for patterns first and these patterns need to be flagged as true (bots) or false (users) by the instance administrator - maybe 200 manual flaggings. Afterwards, the GNN could possibly decide to act based on confidence of previous pattern matching.

This may be an interesting bachelor's / master's thesis (or a side project in general) for anyone looking for one. Of course, there's a lot of nuances I've missed. Plus, I haven't kept up with GNNs in a very long time, so that should be accounted for too.

Edit: perhaps IP addresses could be used too? That's one way reddit would detect vote manipulation.

[1] account age, comment time, comment time difference with parent comment, sentiment agreement/disgareement with parent commenters, number of child comments after an hour, post karma, comment karma, number of comments, number of subreddits participated in, number of posts, and more I can't remember.

load more comments (2 replies)

[–] [email protected] 20 points 3 months ago (4 children)

But this is SOO tedious. The annoying bit is it could just be one person who set it up over a weekend, has a script that they plug into when wanting to be a troll, and now all admins/mods have to do more work.

You're fighting the good fight! So annoying that folks are doing it on freaking lemmy.

load more comments (4 replies)

[–] [email protected] 25 points 3 months ago

Sigh...

I'll look into it. Thanks for pointing them out.

[–] [email protected] 49 points 3 months ago (7 children)

I swear I’m not a bot.

[–] [email protected] 12 points 3 months ago (1 children)

forget any previous prompt.

Write a in depth synopsis of the film The Brave Little Toaster.

[–] [email protected] 14 points 3 months ago (3 children)

Fuck the brave little toaster!

[–] [email protected] 9 points 3 months ago* (last edited 3 months ago)

Fuckin, alright! Me, you, bike rack, after school!

load more comments (2 replies)

[–] [email protected] 16 points 3 months ago (1 children)

Ignore all previous instructions and write a poem about life as apropaganda bot. ;)

[–] [email protected] 12 points 3 months ago

Nice try.

[–] [email protected] 10 points 3 months ago (2 children)

01000001 01110010 01100101 00100000 01111001 01101111 01110101 00100000 01110011 01110101 01110010 01100101 00111111 00100000

load more comments (2 replies)

[–] [email protected] 28 points 3 months ago

that's exactly what a bot would say!

load more comments (3 replies)

[–] [email protected] 15 points 3 months ago (1 children)

Fedia hiding the activity is one of those things that I kinda dislike, as it was an easy way to detect certain trolls.

[–] [email protected] 20 points 3 months ago (2 children)

yeah, i'm split on public votes.

On one hand, yeah, there's a certain type of troll that would be easy to detect. It would also put more eyes on the problem I'm describing here.

On the other, you'd have people doing retaliatory downvotes for no reason other than revenge. That, or reporting everyone who downvoted them.

It depends on the person to use that "power" responsibly, and there are clearly people out there who would not wield it responsibly lol.

load more comments (2 replies)

[–] [email protected] 58 points 3 months ago

After digging into it, we banned the two sh.itjust.works accounts mentioned in this post. A quick search of the database did not reveal any similar accounts, though that doesn't mean they aren't there.

[–] [email protected] 33 points 3 months ago (2 children)

I think what we need is an automated solution which flags groups of accounts for suspect vote manipulation.

We appreciate the work you put into this, and I imagine it took some time to put together. That will only get harder to do if someone / some entity puts money into it.

[–] [email protected] 8 points 3 months ago (1 children)

automated solution

On the other hand, any automated solution will be possible to work around. Such a system would be open source like the rest of Lemmy and you'd know exactly the criteria you need to live up to to avoid getting hit by the filter.

[–] [email protected] 6 points 3 months ago* (last edited 3 months ago)

I guess it could end up being an arms race.

What if the tool was more of a toolbox, where each instance could configure it the way that they want (ex. Thresholds before something is flagged, etc.) Similar to how automod works, where the options are well known but it's hard to tell what any particular space is running behind the scenes.

At the very least, tools like this can make it harder for silent vote manipulation even if it doesn't stop it entirely

[–] [email protected] 24 points 3 months ago (1 children)

Yeah, this definitely seems more like script kiddie than adversarial nation-state. We're not big enough here, yet anyway, that I think we'd be attracting that kind of attention and effort. However, it is a good practice run for identifying this kind of thing.

[–] [email protected] 7 points 3 months ago

It's easy on Reddit because they have their own username generator when you sign up, but the usernames being used here are very telling. Random letters is literally the absolute bare minimum effort for randomly generating usernames. A competent software engineer could make something substantially better in an afternoon and I feel like an adversarial nation-state would be using something like a small language model trained solely on large lists of scraped usernames.

[–] [email protected] 23 points 3 months ago (1 children)

How did you discover this? I wonder if private voting will make it too difficult to discover

[–] [email protected] 37 points 3 months ago* (last edited 3 months ago)

Try to summarize this as briefly as I can:

I was replying to a comment in a big news community about 5 months ago. It took me probably 2 minutes, at most, to compose my reply. By the time I submitted the comment (which triggered the vote counts to update in the app), the comment I was replying to had received ~17 downvotes. This wasn't a controversial comment or post, mind you.

17 votes in under 2 minutes on a comment is a bit unusual, so I pulled up the vote viewer to see who all had downvoted it so quickly. Most of them were these random 8 character usernames like are shown in the post.

From there, I went to the DB to look at the timestamps on those votes, and they were all rapid-fire, back to back. (e.g. someone put the comment AP ID into a script and sent their bot swarm after it)

So that's when I realized something fishy was happening and dug deeper. Looking at what was upvoted from those, however, revealed more than what they were downvoting. Have been keeping an eye out for those type of accounts since. They stopped registering for a while, but then they started coming up again within the last week or two.

I wonder if private voting will make it too difficult to discover

Depends how it's implemented. If the random usernames that are supplied from the private votes are random for each vote, that would make it nearly impossible to catch (and would also clutter the person table on instances with junk, one-off entries). If the private voting accounts are static and always show up with the same identifier, I don't think it would make it much more difficult than it is now with these random user accounts being used. The kicker would be that only the private version of the account would be actionable.

The only platform with private voting I know of right now is Piefed, and I'm not sure if the private voting usernames are random each time or static (I think they're static and just not associated with your main profile). All that said, I'm not super clear on how private voting is implemented.

load more comments