simontherockjohnson

While still a valid callout, these “new chips made out of non-silicon are BETTER in EVERY WAY!” experiments happen about once every 6 months.

China has been on the ball to curb publish or perish pop science bullshit for about 5 years now. They banned incentive and performance structures in universities that rely on publishing rankings.

https://www.mpiwg-berlin.mpg.de/observations/1/end-publish-or-perish-chinas-new-policy-research-evaluation

Despite this reform, China has maintained it's leadership in paper quality and quantity and is now leading in high-level SME's

https://www.scmp.com/news/china/science/article/3295011/china-surpasses-us-tally-top-scientists-first-time-report

It's more likely this is real than it was 5 years ago.

From my other comment about o1 and o3/o4 potential issues:

The other big difference between o1 and o3 and o4 that may explain the higher rate of hallucinations is that the o1’s reasoning is not user accessible, and it’s purposefully trained to not have safe guards on reasoning. Where o3 and o4 have public reasoning and reasoning safeguards. I think safeguards may be a significant source of hallucination because they change prompt intent, encoding and output. So on a non-o1 model that safeguard process is happening twice per turn once for reasoning and once for output, then being accumulated into the input. On an o1 model that's happening once per turn only for output and then being accumulated.

Another point of anecdata is that I've read that vibe coders say that non-reasoning models lead to better results for coding tasks because they are faster and they tend to hallucinate less because they don't pollute with automated CoT. I've seen people recommend Deepseek V3 03/2025 release (with deep think turned off) over R1 for that reason.

my money is on the higher hallucination rate being a result of the data being polluted with synthetic information. I think its model collapse

But that is effectively what happening with RLMs and refeed. LLMs have statistical weights between model and inputs. For example RAG models will add higher weights to the text retrieved from your source documents. RLM reasoning is a fully automated CoT prompting technique. You don't provide the chain, you don't ask the LLM to create the chain, it just does it all the time for everything. Meaning the inputs becomes more polluted with generated text which reinforces the existing biases in the model.

For example if we take the em dash issue, the idea is that LLMs already generate more em dashes than exist in human written text. Let's say turn 1 you get an output with em dashes. On Turn 2 this is fed back into the machine which reinforces that over indexing on em dashes in your prompt. This means turn 2's output is going to potentially have more em dashes, because the input on turn 2 contained output from turn 1 that had more em dashes than normal. Your input over time end up accumulating the model's biases through the history. The shorter your inputs on each turn and the longer the conversation the faster the conversation input converges on being mostly LLM generated text.

When you do this with an RLM, you have even more output being added to the input automatically with a CoT prompt. Meaning that any model biases accumulate in the input even faster.

Another reason I suspect the CoT refeed vs training data pollution is that GPT-4.5 which is the latest (Feb 2025) non-reasoning model seems to have a lower hallucination rate on SimpleQA than o1. If the training data were the issue we'd see rates closer to o3/o4.

The other big difference between o1 and o3 and o4 that may explain the higher rate of hallucinations is that the o1's reasoning is not user accessible, and it's purposefully trained to not have safe guards on reasoning. Where o3 and o4 have public reasoning and reasoning safeguards. I think safeguards may be a significant source of hallucination because they change prompt intent, encoding and output. So on a non-o1 model that safeguard process is happening twice per turn once for reasoning and once for output, then being accumulated into the next turn's input. On an o1 model that's happening once per turn only for output and then being accumulated.

AI alchemists discovered that the statistics machine will be in a better ball park if you give it multiple examples and clarifications as part of your asks. This is called Chain of Thought prompting. Example:

Then the AI Alchemists said, hey we can automate this by having the model eat more of it's own shit. So a reasoning model will ask it self "What does the user want when they say < Your prompt>?" This will generate text that it adds to your query, to generate the final answer. All models with "chat memory" effectively eat their own shit, the tech works by reprocessing the whole chat history (sometimes there's a cache) every time you reply. Reasoning models because of the emulation of chain of thought eat more of their own shit than non-reasoning models do.

Some reasoning models are worse than others because some refeed the entire history of the reasoning, and others only refeed the current prompt's reasoning.

Essentially it's a form of compound error.

Are the people too insignificant to care about or so powerful they cost the election, which is it??

If only there was a name for this concept of painting your enemies as simultaneously too weak and too strong and a political ideology associated with it.

Uhh.....

The literal axiomatic "good guys" are magic space cops and the main story is about how they cannot police their own nor can they police their political structure so the entire galaxy falls into a fascist regime that can only be saved by yep you guessed it the same "good guys" who fucked up the first time, joined by a sins of the father trope character.

The Jedi Order collects devshirme (blood tax) because it quite literally rips away force sensitive children from their families. It's quite literally one of the Order's primary functions. Qui Gon Jinn was going to let Anakin rot in slavery until he realized he was force sensitive. Furthermore it tells these children to forget their feelings for their family as part of their training. it's literally Janissary shit.

Furthermore most of the media treat this like Anakin is totally wrong for coping the way that he does because his literal choices are between slavery under a happy merchant style character and slavery under a child cop training program, but he's the magical chosen boy so he should be thankful that he never gets to see his mother again until he see's her violent death. What a piece of shit, loving his mom, having feelings for his family, doesn't he know that there are more important things like institutions that are axiomatically good and their extremist religious beliefs???

Jedi's esp prior to Disney were religious zealots. They literally had their own terms for apostasy that were differentiated from followers of the "bad" religion, e.g. Dark Jedi were apostates and Sith were the evil religion guys. The Jedi Order does not recognize anything outside of it's orthodoxy as "good".

Oh and the whole Anakin thing literally happened because their prophecy which was defined as "bringing balance to the force" was always interpreted as wiping out the Sith. It was essentially heresy that "balance" could possibly mean the ruination of the order because the Sith believed loosely in Rule of 2 and there was quite literally battalions of Jedi.

Canonically the question of "why would anyone trust the Jedi as negotiators since they have the power of mind control?" Is literally answered by the fact that they do not police capital with no further thought on why that's a good answer logistically speaking or morally speaking. And the whole neutrality thing is canonically a lie as well both in the trilogies and in Legends. Jedi are consistently not impartial.

A large chunk of the story is centered on Tatooine a planet with outright slavery, our "good guys" regardless of their level of institutional power at any given point in the plot have always just shrugged their shoulders at this. By the way when Starwars guys defend this they always say something like the Jedi coudl never stand up to the Hutts because of you name it, "jusidiction", "alliances", and "strength". Which is funny because the guy who controls the Five Families after the fall of the Republic is literally Darth Maul. So yeah our "good guys" can't end literal slavery for no good reason.

Like Lucas can say what he wants, and I believe that he believes it and I believe that he took inspiration from all the leftist political things he says he took inspiration from. But, he's libbed up as hell, and the story line is basically exhalted good guys vs exhalted bad guys duke it out in space with everyone else dodging lasers and orbital cannons. The lore of the Jedi actually makes them fairly awful fascists in their own right.

S2 blows S1 out of the sky.

Just saw it. Season 2 keeps getting better and better.

AI slop is decreasing literacy (in the broad sense, including things like media literacy) at a faster rate than it was decreasing before.

This is the cornerstone of your argument and there is no real proof of:

1. Literacy falling faster after November 30th 2022 than before that date (correlation)
2. Literacy falling because of LLMs (causation)

This is exactly where you're making a jump that I cannot make. My argument is that I can believe in LLMs being responsible for exacerbating upstream effects, but I cannot accept that LLMs are even in the running compared to the elephant in the room: austerity

We're arguing a well known documented effect vs a novel unstudied effect. There is more evidence that austerity is a fuzzy vibes based inverse of the printing press than there that LLMs are.

Yeah, and the printing press allowed weirdos like Martin Luther to spread ideas faster.

Except it's not the same thing. It's different in economic function, social function, and historical context.

You can effectively own an LLM. It's incredibly cheap all things considered. You can tell an LLM to constantly generate a stream of bullshit from here into infinity your only limit is cost of compute which is cheap.

Martin Luther did not own a printing press. Martin Luther's success is attributable to speculative nature of publishing and his ability at being at being charismatic through the written word. Yes without the printing press Martin Luther would be an unknown weirdo that may have been rediscovered at some point. However the content itself was what was driving the economic demand for proliferation, not the printing press itself.

The printing press revolutionized the creation and distribution of media. LLMs have done no such thing, the various media we have is already cheap, easy to distribute and widely proliferated.

Slop is much more comparable to straight up forgery and propaganda factories because it exists despite market demand. Propaganda is always a cost center, and forgery is a profit center because the lie enables leverage. However these are not things that people typically want to the same degree as something like Martin Luther's ideas.

Martin Luther actually convinced people of something. AI Slop isn't reaching the same levels of demand as Luther. AI Slop isn't even reaching the same historic levels of demand as Artisanal Hand Crafted Slop like Marvel Movies or NYT Op-Eds. AI Slop undercuts its own value proposition!

We were already drowning in bullshit prior to the proliferation of LLMs. Grandma believed foreigners were making no-go zones in inner cities when that bullshit was fed to her manually by unscrupulous news editors. LLMs don't change the consumption of the information market directly. They only change the upstream circumstances that lead to consumption of slop, like poor education -- which has already been failing.

Don't get me wrong it's a perfect storm, but putting the blame mostly on ChatGPT is tech doomerism, humans were doing this to each other in industrial quantities before we outsourced it to a machine that could be owned and operated by anyone. We've been ignoring and exacerbating the issues that lead to this problem for at least a decade. 1 in 5 adults was functionally illiterate in 2014. We've known since the 90's that whole language approach was leading to more illiteracy and worse educational outcomes among students. We failed so long ago that we cannot even agree on what was the primary failure.

Martin Luther on the other hand is actually credited with an increase in literacy.

We've been destroying the institutions and mechanisms that give the general population the cognitive tools to fight back against LLM bullshit for a very long time, and the statistics on education among the general populace in the US bear that out.

I manage software production and I have similar problems regarding programmers and code quality. My red line is, I'll explain the whys and hows but at the end of the day I don't give a shit what you think because you don't have to be responsible for the software and I do -- so we're gonna make it easier on me. You can lead a horse to water and all....