Technology

Besides finding better ways to positively recognize bots, we also need to invent ways that make it quicker to realize "false alarm, this user is actually legit".

For example, users should have an option to pin posts and comments to their profile, and I suggest to provide at least two different 'tabs' for this in the public profile: One tab just for the usual "posts and comments you would like the world to see", but another tab for "some recent, complex interactions between you and other (established) users that in your eyes prove quite well you're not a bot". The purpose is simply to save others, worried that you could be a bot, some time of going through your posts in search of signs for humanity. Yes, this can be gamed to some degree (what can't?). However, at a technical level, the feature is little more than a copy of the "pin" feature that would be very nice to have anyways, so we can get an appreciable improvement in our ability to tell users from bots for very little programming effort.

The future is probably going to be:

• Whitelist federation models with invite-only instances.
• Anubis and similar software being a base requirement for operation.
• Getting creative with stuff like picture-based logic puzzles as a type of captcha.
• A retreat into less publicly visible spaces like chat apps that you can only get access to through networking.

believe that i've seen versions of this attack where the spammer makes a few innocuous non-spam (but not contribution either) posts before going full LLM bot

A more critical weakness is that these accounts only posted obvious spam; they made no effort to build up a plausible persona. Generating plausible human posts is more difficult, but broadly feasible with current LLM technology.

i will have to screenshot next time and see what people think

obviously they could be hijacked accounts and i might just be extremely judgey of your average twitter flight account

I mean... I've been using forums since '93 and lies were its past, too.

At least those liars were human beings that actually showed some creativity with their made up bullshit.

I was going to say "but someone could post a picture with the date and their username" and then realized that can be spoofed. And then I thought what about some kind of video? Wait that could probably be spoofed too.

Maybe you could have it so only real people could get in via word of mouth? But eventually I'm sure someone with bad intentions could enter

I wonder if a personalized reputation system based on your votes of other people's comments, and influenced by votes from folks who have earned enough upvotes from you, could be developed without turning your feed into an echo chamber like Facebook.

Sort of like PageRank, but for fediverse users instead of web pages, and with each user keeping (and seeing) their own rankings of everyone else.

These attacks do not have to be reliable to be successful. They only need to work often enough to be cost-effective, and the cost of LLM text generation is cheap and falling. Their sophistication will rise. Link-spam will be augmented by personal posts, images, video, and more subtle, influencer-style recommendations—“Oh my god, you guys, this new electro plug is incredible.” Networks of bots will positively interact with one another, throwing up chaff for moderators. I would not at all be surprised for LLM spambots to contest moderation decisions via email.

I don’t know how to run a community forum in this future. I do not have the time or emotional energy to screen out regular attacks by Large Language Models, with the knowledge that making the wrong decision costs a real human being their connection to a niche community.

Ouch. I'd never want to tell someone 'Denied. I think you're a bot.' -- but I really hate the number of bots already out there. I was fine with the occasional bots that would provide a wiki-link and even the ones who would reply to movie quotes with their own quotes. Those were obvious and you could easily opt to ignore/hide their accounts. As the article states, the particular bot here was also easy to spot once they got in the door, but the initial contact could easily have been human and we can expect bots to continuously seem human as AI improves.

Bots are already driving policy decisions in government by promoting/demoting particular posts and writing their own comments that can redirect conversations. They make it look like there is broad consensus for the views they're paid to promote, and at least some people will take that as a sign that the view is a valid option (ad populum).

Sometimes it feels like the internet is a crowd of bots all shouting at one another and stifling the humans trying to get a word in. The tricky part is that I WANT actual unpaid humans to tell me what they actually: like/hate/do/avoid. I WANT to hear actual stories from real humans. I don't want to find out the 'Am I the A-hole?' story getting everyone so worked up was an 'AI-hole' experiment in manipulating emotions.

I wish I could offer some means to successfully determine human vs. generated content, but the only solutions I've come up with require revealing real-world identities to sites, and that feels as awful as having bots. Otherwise, I imagine that identifying bots will be an ever escalating war akin to Search Engine Optimization wars.

Definitely something I've observed even here. Luckily we get few applications and there is a report button, but I share the author's frustration and the author's jaded view of a limited timeline on services such as ours being tenable. Eventually it will be trivially easy to flood this place with slop.