Nothing to do with AI, Garbage in, Garbage out.

LLMs are tools that satisfies requests. The developer decided to allow people to put the ingredients for chlorine Gas into the input - LLM never stood a chance but to comply with the instructions to combine them into the end product.

Clear indication we are in the magical witch hunt phase of the hype cycle where people expect the technology to have magical induction capabilities.

We could discuss liability for the developer but somehow I don’t think a judge would react favorably to “So you put razor blades into your bread mixer and want to sue the developer because they allowed you to put razor blades into the bread mixer”