Technology

job wasn’t that important

I keep telling you that changing out the battery in the smoke alarm isn't worth the effort and you keep telling me that the house is currently on fire, we need to get out of here immediately, and I just roll my eyes because you're only proving my point.

Define "works"?

If you're a CEO, cutting all your talent, enshittifying your product, and pocketing the difference in new, lower costs vs standard profits might be considered as "working".

An inherent flaw in transformer architecture (what all LLMs use under the hood) is the quadratic memory cost to context. The model needs 4 times as much memory to remember its last 1000 output tokens as it needed to remember the last 500. When coding anything complex, the amount of code one has to consider quickly grows beyond these limits. At least, if you want it to work.

This is a fundamental flaw with transformer - based LLMs, an inherent limit on the complexity of task they can 'understand'. It isn't feasible to just keep throwing memory at the problem, a fundamental change in the underlying model structure is required. This is a subject of intense research, but nothing has emerged yet.

Transformers themselves were old hat and well studied long before these models broke into the mainstream with DallE and ChatGPT.

It's based on the last few years of messaging. They've consistently said AI will do X, Y, and Z, and it ends up doing each of those so poorly that you need pretty much the same staff to babysit the AI. I think it's actually a net-negative in terms of productivity for technical work because you end up having to go over the output extremely carefully to make sure its correct, whereas you'd have some level of trust with a human employee.

AI certainly has a place in a technical workflow, but it's nowhere close to replacing human workers, at least not right now. It'll keep eating at the fringes for the next 5 years minimum, if not indefinitely, and I think the net result will be making human workers more productive, not replacing human workers. And the more productive we are per person, the more valuable that person is, and the more work gets generated.

It's tons easier to repkace CEOs, HR, managers and so on than coders. Coders needs to be creative, an HR or manager not so much. Are they leaving three months from now you think?

I'll start worrying when they are all gone.

We are 18 months into AI replacing me in 6 months. I mean… the CEO of OpenAI as well as many researchers have already said LLMs have mostly reached their limit. They are “generalizers” and if you ask them to do anything new they hallucinate quite frequently. Trying to get AI to replace developers when it hasn’t even replaced other menial office jobs is like saying “we taught AI to drive, it will replace all F1 drivers in 6 months”.