askchapo

The real research is useful. Putting aside nerd rapture stuff which is probably fantasy as far as we can tell it ought to be possible to make thinking machines. whether we do? who knows. Increasingly we make and use machines able to make at least some decisions and the good research into alignment and training helps with even mundane shit; such as making sure the machine is actually seeing cancers and not say the institution the MRI was taken at.

Even with relatively stupid machines it would be important to have certainty that idk a car isn't going to take a shortcut through a pedestrian. You don't need to reach AM level for it to be important that an autonomous system using some degree of heuristic decision making, quote unquote, cares about the things we do.

I don't think AI safety is such a big problem that it means we gotta stop building AI or we'll destroy the world or something, but I do agree there should be things like regulations, oversight, some specialized people to make sure AI is being developed in a safe way just to help mitigate problems that could possibly come up. There is a mentality that AI will never be as smart as humans so any time people suggest some sort of policies for AI safety that it's unreasonable because it's overhyping how good AI is and it won't get to a point of being dangerous for a long time. But if we have this mentality indefinitely then eventually when it does become dangerous we'd have no roadblocks and it might actually become a problem. I do think completely unregulated AI developed without any oversight or guardrails could in the future lead to bad consequences, but I also don't think that is something that can't be mitigated with oversight. I don't believe for example like an AGI will somehow "break free" and take over the world if it is ever developed. If it is "freed" in a way that starts doing harm, it would be because someone allowed that.

What troll post?

I think the risk posed by capitalist deployment of AI technologies far outweighs the dangers of misalignment, although alignment is definitely a problem

EDIT: didn't read that this was a non Big Yud post, so sorry, but...you can't really talk about AI safety in the general sense without him.( He dominated the field for 20 years, long before his dumb rationalist cult. Every single alignment guy is either using his assumptions or deliberately departing from them.)

I'm going to talk about the hard AI problem of rogue AI, not the obviously more pressing one of AI being used for bad things, because that can't be stopped

So, Big Yud is mad and constantly full of nuclear Bay Area brainworm takes. but some of his arguments against alignment make sense. At least enough to engage with them and try to solve them, but bombing data centres et al is dumb and won't solve the issue since most code is already so inefficient.

AI is not going to go FOOM in the way he suggested, basically because LLMs exist, which he thought were impossible as part of his arguments. He never considered we would have time to practice alignment with extremely limited AI at near human levels. And that's kicked the core of his doom argument away.

I think UlyssesT and I once agreed that he is the dumbest smart guy alive and will probably become leftist after trying and failing at every tech nerd principal. If someone could keep him the hell away from the Bay Area he might normalise and stop writing weird shit.

For what it's worth he also thinks Transformers have no chance at AGI, at least alone. And his thoughts on the dumb approaches used by AI companies are sound. But he is arguing in good faith and has probably thought about this more than anyone else. (Nobody even in the Yud cult except for a few weirdos ever took Roko's Basilisk seriously. Yud deleted the post mostly because he saw that people would go weird about it, a rare example of him being exactly correct.)

As for everyone else...Altman doesn't care and is using it for market capture. Anthropic probably does care but is more or less forced down the same path of ineffective safety that captures the market.

Ilya definitely cares but for all the wrong reasons, he's massively pro Israel so he'll never achieve a consistently aligned AI anyway, but I'd not want him working on this stuff.

But ultimately hard alignment is a secondary concern against dumb corporate HR managers trying to AI the compliance team.

The thing that currently exists by the name of "AI" is never gonna become anything like AGI, but it is bad in its own way.

I don't think AGI is fundamentally impossible, because dualism is fake and consciousness is a real material process that exists in the universe, but it's not a thing we are anywhere near understanding how to build.

Further, the roko's basilisk is literally just a dumber pascal's wager, because it's a pascal's wager wherein not giving a shit about god is a total defense against divine punishment.

For reference, here's a video about a recent paper that showed a real example of an AI behaving differently under training, with the explicit purpose of avoiding the detection of behavior it would take on in deployment.

I found a YouTube link in your post. Here are links to the same video on alternative frontends that protect your privacy:

• yewtu.be
• inv.nadeko.net
• yt.artemislena.eu
• piped.video