The word unexpectedly is doing a lot of heavy lifting here. It was given the ability to modify its own code, and it did, how is that unexpected?
this post was submitted on 14 Aug 2024
94 points (77.0% liked)
Technology
59331 readers
4903 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
Arstechnica with an absolutely composting headline. Sigh
I for one welcome....oh wait, this isn't that lame Spez site. Forgot where I was for a second.
That's a Slashdot meme, though.
It’s a fucking Simpsons meme.
Originally yeah, but Slashdot is where it got turned into a meme.
So most memes are 4chan, 9gag or imgur memes then? What....?
I suppose so, yes.
I can't wait until one goes rogue and escapes into the net.
That's gonna be fun to watch.
Umm actually nets are how you get caught, not escape.
"We put literally no safeguards on the bot and were surprised it did unsafe things!"
Article in a nutshell
Not quite. The whole reason they isolated the bot in the first place was because they knew it could do unsafe things. Now they know what unsafe things are most likely, and can refine their restrictions accordingly.
Nothingburger. They were using the AI to code their scripts and haven't even shown the prompts that got the response. LLMs are not AGI.
Having read the article and then the actual report from the Sakana team. Essentially, they're letting their LLM perform research by allowing it to modify itself. The increased timeouts and self-referential calls appear to be the LLM trying to get around the research team's guardrails on it. Not because it's become aware or anything like that, but because its code was timing out and that was the least effort way to beat the timeout. It does handily prove that LLMs shouldn't be the one steering any code base, because they don't give a shit about parameters or requirements. And giving an LLM the ability to modify its own code will lead to disaster in any setting that isn't highly controlled like this.
Listen, I've been saying for a while that LLMs are a dead end towards any useful AI, and the fact that an AI Research team has turned to an LLM to try and find more avenues to explore feels like the nail in that coffin.
Imagine allowing LLMs to write and execute code and being surprised they write and execute code.
So it's just like a regular researcher then?
Well... now the paperclip thought experiment becomes slightly more prescient.
Everyone's like, "It's not that impressive. It's not general AI." Yeah, that's the scary part to me. A general AI could be told, "btw don't kill humans" and it would understand those instructions and understand what a human is.
The current way of doing things is just digital guided evolution, in a nutshell. Way more likely to create the equivalent of a bacteria than the equivalent of a human. And it's not being treated with the proper care because, after all, it's just a language model and not general AI.
Yup. A seriously intelligent AI we probably wouldn't have to worry too much about. Morality, and prosocial behavior are logical and safer than the alternative.
But a dumb AI that manages to get too much access is extremely risky.
Skynet it's watching you 👁️🌐
Skynet here we come
We're going to palestine?
Skynet invented time travel all on its own so it could make sure it kept existing. Don't compare it to these pissant LLMs. That's an insult to Skynet.
I don't know if you watch science and futurism with Isaac Arthur, but if you don't, you probably should. And he has a quote that I think applies quite well.
"Keep it simple, keep it dumb, or you might end up, under SkyNet's thumb."
Terminator is part of a double feature. We need to sit through Multiplicity first.