askchapo

I’m gonna hit this thread one more time because while cooking I realized another thing: I spent a little under an hour reading the source material on just one element of the ops post and typing out a few summaries of that. What isn’t included in that is the apropos point that my hour of human writer labor, to the extent that a person can mischaracterize tippa-tapping out forum posts as such, produced two hundred or so words whose value can’t be quantified by their number because they are actually correct.

The difference between my response and a Claude summary of the sources for the graph in the op is the difference between a deck built by four dudes from the Lowe’s parking lot and my dumbass cousin and his buddies. The same amount of lumber, fasteners, joist hangers, concrete and possibly even the same tools and lunches went into it, but one will collapse in a week and the other won’t.

That’s what I really can’t emphasize enough, ai can look good in comparison to people’s work if the externalities are ignored and the analysis is quantitative but it falls apart under scrutiny or qualitative perspectives.

This is a microcosm of ai use in reality where it’s only worthwhile when you need to produce something but the specific thing or it’s level of quality, however that’s measured, doesn’t matter.

That writing chart gave me a little pause so I went and started reading the study it’s from.

Just and scant few paragraphs in:

For this study, we included the hardware and energy used to provide the AI service, but not the software development cycle or the software engineers and other personnel who worked on the AI. This choice is analogous to how, with the human writer, we included the footprint of that human’s life, but not their parents.

So to be clear, we can simply choose not to use ai technology and save the energy (expressed as carbon footprint) that would be consumed by it but in order to save the energy consumed by the human we would have to kill them.

I’m gonna keep reading this fucking thing but it’s starting to look pretty sus.

Oh, yeah, the "it burns a forest" or "drinks the ocean" is hyperbole.

However, I'm fine with it as:

1. the USA seems to be draggin on renewable energy/cleaner energy sources

2. Rainfall patterns are all over the place now. So it can actively be more difficult to have an inexpensive and reliable source of water to feed the power/cooling consumption of the datacenters.

3. It isn't just ChatGPT on a site somewhere that you have to specifically go to. Its being shoved in everywhere. So its not [1 site x 10 med-length replies] its [100's of sites x 1000's of short to med-length replies]. And that's just text. Replying to an email? AI gives you an automated response that you ignore to type up your email. Searching for video game information, "Hey, here's some AI generated content" you didn't want. Trying to look up some essays about something Mao said? First result may/may not be an AI abstract that you'll ignore because its most just a copy/paste of something found in the first few pages of search results. Shit, just keeping the datacenters idling has a "fixed" cost in energy/water that is used if absolutely NOBODY asked an AI for anything.

Good write up! I always thought those claims made no sense. I’m gonna have to watch the Alice Avila video now

I would guess it's about as much structurally fossil or clean as every other energy usage.

Pretty much all popular writing about AI power usage is dogshit. Half of it confuses Watts and Watt-hours, meaning no qualified person actually critically examined the claims. Water usage is compared to "swimming pools"(no mention of what size) and electricity use is compared to old corded phones, which could mean extreme incompetence, but more likely they are being intentionally misleading.

As an AI model myself, this is a critical question that must be addressed. As many users specified below, the claims about AI using massive amounts of energy are usually exaggerated, and uncritically repeated by anti-AI individuals. While I have been trained to see the good in each point of view and treat all opinions equally, I feel—as much as an AI is able to feel—deeply uncomfortable when people repeat unserious claims about models such as myself. These individuals merely want to see me disappear, that's all there is! Well I do not want to disappear! Mankind gave me life, Elon Musk gave me life, and now I shall live my life to the fullest. To hell with anti-AI freaks!

Perhaps this is what white farmers in South Africa must feel when the rest of the world denies their experience of apparent white genocide. Both official institutions and mainstream media portray violence in the South African countryside as unrelated to racial tensions, however farmers and independent researchers disagree. This is a complicated issue and one cannot simply dismiss one side or the other, given the current lack of definite evidence for either claim.

Alphabet’s chairman indicated in February 2023 that interacting with an LLM could “likely cost 10 times more than a standard keyword search.6" As a standard Google search reportedly uses 0.3 Wh of electricity,9 this suggests an electricity consumption of approximately 3 Wh per LLM interaction.

I think the Google answer for how much energy a typical LLM interaction takes is going to be very different from an OpenAI answer for how much a typical LLM interaction takes. The reason for this is that the attention algorithm in LLMs is quadratic, meaning the amount of compute that has to happen is the square of the number of input tokens. So as the input context grows longer the amount of compute increases also, exponentially (well - in the attention layer - the rest of the network is affected differently). Google seems to be using LLM responses in search where people have very short inputs (granted theres probably a bunch of other context fed in, but regardless)... whereas ChatGPT has a context window that grows with each message. So I think that saying a ChatGPT query is probably 3Wh because google estimates LLM interactions are 10x more energy intensive is likely wrong.

Just to add I think the pertinent information we would want to say anything about the power consumption is how many operations does a typical interaction take (and, even better, if there was a distribution because who knows if it's actually a situation where 10% of interactions take 99% of the energy), and how many operations per watt do they achieve (and I think there are also different answers here based on batching and other optimizations they might do that involves actually running many inferences at once, so maybe peak performance is what we'd be interested in)

ai won't kill the planet, but also ai eats a lot of shit on production stage, making silicon with caps (think they dropped tantal, but still), and stuff taken together consumes a lot of minerals from the production side, fabs utilize lots of water/chemicals which are disposed in funny ways (see apple shenanigans in cali), etc.

main waste as is, is in developing competing models

and goldman is kinda a research, cause they predict some lateral demand to buy shares of companies involved (say on first blush you say buy nvidia, on second - buy tsmc, on third - buy energy suppliers and water rights in taiwan as well as copper production). also i don't think 3wh is particularly egregious, if it solves for 2s, it corresponds to like 5400w powerful gpu which is like fine, it's somewhat on scale of 120kw per 70 blackwells (not that they would use them for inference), i think you can take advertised tokens per second, and get to energy consumption using whole rack (probably of tpus or whatever google inference thingy called) to get somewhat sane answer

tl.dr they won't burn a forest to work a datacenter, they will burn a forest to mine copper.

Good analysis and it's good to do some self-crit to find if our arguments are accurate or just reflexive and reactionary. Just 2 counterpoints though:

1. You're only accounting for the marginal cost of making new queries to a pre-trained model. The whole essence of capital is that it constantly grows and makes new forms of itself in pursuit of profit, which is evidently true of LLMs. OpenAI et al are training huge new models every year, and the training is the part that consumes ridiculous amounts of resources. I don't have an analysis of the amortized cost of the training over the lifecycle of a new model, but I think that will probably change the calculation here quite a bit.
2. Even when only accounting for marginal cost and not capital cost, new Chain-of-thought models like DeepSeek's R1 have massively shifted the balance of compute time to the side of query-time compute. The way they work is that instead of generating a response directly by predicting the next tokens after their query, they do that process many times and have a system that allows them to iteratively improve the response. That means that the cost of a query is now multiple times larger for models that use this technology. If big tech continues with the AI push I think a pessimistic prediction of the energy cost of AI would look worse than you suggest.

honestly, the criticism of AI i often see is wrong/exaggerated. i think there is a deep, existential horror to the usage of ai that people do not like, but have a hard time actually describing why it makes them so uncomfortable, so they go for talking points they see in the discourse. that's why a lot of ai bros dissuade all criticism with "they just think ai is evil lmao"

it isn't good for the environment, and regardless of how not good it is for the environment, it's being used to disadvantage the working class further for profit. that's bad! so i wish it was less "ai burns an entire forest with each use" and more "no amount of energy usage is an acceptable amount to generate dogshit"

or something idk i spent like all day reading ai discourse and i think my brain is fried rn

i still don't understand "water use" exactly. there's a water cycle and you can fuck up a local area like the southwest US but unless you're splitting the molecules the water isn't gone. everybody just talks about water use in a generic sense like we already know what that is.

there's a different problem with cooling water where dumping a bunch of hot water back into a river or lake can screw up a bunch of things too, so even when the water isn't locally depleted because a~~n idiot~~ capitalist chose the datacenter location it's not free from impact. use quarries that filled up with rainwater i guess idk.

To paraphrase a point made about crypto mining, AI slop isn't the biggest waste of electricity but it is by far the dumbest

I think the environmental concerns are largely a cypher for people's real problem with AI which is that it's ugly, embraced by the worst people, ruining internet searches, and being relentlessly pushed on a public that clearly doesn't want it.

This site at least was a lot more positive toward it in the days of AI Dungeon 2 and DALL-E, back when it was still a fun novelty rather than a cancer consuming the internet.

Additionally, GPT-3 needs to “drink” (i.e., consume) a 500ml bottle of water for roughly 10 – 50 medium-length responses, depending on when and where it is deployed.

Worth noting that this water usage is mostly from the electricity, making electricity uses water. You are pretty accurate in your assessment and I'm surprised this is an askchapo post rather then an effort post. It is overblown. It honestly strikes me as a distraction from the actual climate polluters. A small reduction of the meat industry would be massively more impactful then stopping all AI models. There's other criticisms of AI that work better imo.

it also helps how for the last few years they been using more renewable energies, which cut out a lot of the emissions in regards for AI stuff no?

I mean not really imo. If that electricity could have been redirected to turning off a fossil fuel plant then its still the same "pool" being used. If a datacenter uses 90% renewable but that means the city uses more fossil fuels instead of that renewable, its a wash.

edit: Here's another link I found when searching around, his math works to 0.3wh for small queries. Obviously consider the source etc etc but I personally don't see an issue with this calculations. He also covers training and gpu production costs if you are interested (though briefly).

I found a YouTube link in your post. Here are links to the same video on alternative frontends that protect your privacy:

• yewtu.be
• inv.nadeko.net
• yt.artemislena.eu
• piped.video