the dubious estimates nH Predict spits out seem to be a feature, not a bug
This is the major problem with algorithms, one of the issue being that they will produce a lot of false positives even if there are best intentions.
But another major problem is that you can influence the outcome by altering the parameters as the article also says. We have been observing similar issues in health and social policy in many countries over the last years, and the results have always been devastating. And research suggests that biases may increases dramatically in the future if we continue to use these algorithms the way we do it now.
The title of the article on arstechnica comes from the following quote, a few paragraphs in
Though few patients appeal coverage denials generally, when UnitedHealth members appeal denials based on nH Predict estimates—through internal appeals processes or through the federal Administrative Law Judge proceedings—over 90 percent of the denials are reversed, the lawsuit claims. This makes it obvious that the algorithm is wrongly denying coverage, it argues.
While they are correct that error rate applies to the number of misclassified cases (denied when it should not have been), it’s only 90 percent of the denials which are appealed which are overturned. As stated in the quote above, few patients appeal their coverage denials, so it is possible the error rate is much lower as presumably the denials which are not appealed would not be overturned at the same rate.
