Technology

And that's why I'm against ALL such things.

Not because they can't be done right and you can't teach people to use them.

But because there's a slippery slope of human nature where people want to offload the burden of decision to a machine, an oracle, a die, a set of bird intestines. The genie is out and they will do that again and again, but in a professional organization, like police, one can make a decision of creating fewer opportunities for such catastrophes.

The rule is that people shouldn't use machines above their brains, as one other commenter says, and they should only use this in a logical OR with their own judgment made earlier, as another commenter says, but the problem is in human nature and I'd rather not introduce this particular point of failure to police, politics, anything juridical and military.

Why not both? A bad algorithm based on bad heuristics? There are many many algorithms that fail at what they're supposed to do.

As a non-condescending "mathematician", I'm happy to help.

Pretty much anything trying to predict human behavior is a heuristic; people using them as if they've got some kind of certainty is a problem.

Capitalism.

The article is not about how the AI is responsible for the death. It's likely that the woman would have died in the counterfactual.

The question is not "how effective is AI"? The question is should life or death decisions be made by an electrified Oracle at Delphi. You must answer this question before "is AI effective" becomes relevant.

If somebody was adjudicating traffic court with Tarot cards, would you ask: well how accurate are the cards compared to a judge?

An algorithm is never to blame, some pencil necked desk jockey decided the criteria to get help that was used to create the algorithm, the blame is entirely on them.

That said, I doubt it would make any difference if a human was in the loop. An algorithm is still al algorithm, even if it's applied by a human. We usually just call that a "policy" though. People have been murdered by the paper sea for decades before we started calling it "algorithms".

My impression from the article is more that they're not doing any kind of garbage-in assessment: nobody is making sure they're getting answers about the right person (eg: some women date more than one guy) and some women don't feel safe giving accurate answers to the police, and there aren't good failsafes available for when it's wrong; you're forced to hire legal counsel and pursue a change via the courts.

Could a human have judged it better? Maybe not. I think a better question to ask is, "Should anyone be sent back into a violent domestic situation with no additional protection, no matter the calculated risk?" And as someone who has been on the receiving end of that conversation and later narrowly escaped a total-family-annihilation situation, I would say no...no one should be told that, even though they were in a terrifying, life-threatening situation, they will not be provided protection, and no further steps will be taken to keep them from being injured again, or from being killed next time. But even without algorithms, that happens constantly...the only thing the algorithm accomplishes is that the investigator / social worker / etc doesn't have to have any kind of personal connection with the victim, so they don't have to feel some kind of way for giving an innocent person a death sentence because they were just doing what the computer told them to.

Final thought: When you pair this practice with the ongoing conversation around the legality of women seeking divorce without their husband's consent, you have a terrifying and consistently deadly situation.

It reminds me of the debate around self driving cars. Tesla has a flawed implementation of self driving tech, that's trying to gather all the information it needs through camera inputs vs using multiple sensor types. This doesn't always work, and has led to some questionable crashes where it definitely looks like a human driver could have avoided the crash.

However, even with Tesla's flawed self driving, They're supposed to have far fewer wrecks than humans driving. According to Tesla's safety report, Tesla's in self driving mode average 5-6 million miles per accident vs 1-1.5 million miles for Tesla drivers not using self driving (US average is 500-750k miles per accident).

So a system like this doesn't have to be perfect to do a far better job than people can, but that doesn't mean it won't feel terrible for the unlucky people who things go poorly for.

Critical thinking spotted, proper authorities have been notified.

We will fix you!

Thank you, this is why I came to the Fediverse from Reddit.

Our pigs don't look as good as [generic Hollywood actor]