Six cases, six PreTrial Risk Assessments, six decisions with your name on them. Here is how your calls lined up against the tool, and what Docket 2214-D was built to show.
The science
Madeleine Clare Elish named this pattern the moral crumple zone in 2019: in a human-machine system, legal and moral responsibility gets displaced onto the nearest human operator, the one person with a name and a signature, even when that operator had limited real control over the outcome. The human absorbs the failure the way a car's crumple zone absorbs impact: by design, so the frame behind it does not have to.
Ben Green reviewed dozens of policies requiring human oversight of government algorithms in 2022 and found that the oversight largely fails to catch bad recommendations in practice, while doing real work to legitimize the systems it supervises. A human signed off, so the process looks accountable, whether or not that human actually second-guessed the machine.
The vise you just felt in Docket 2214-D, follow the tool and own its errors, or deviate and own the deviation statistics, is what "we'll just put a human in the loop" actually delegates to that human. The tool is never named in either headline. The signature under review is always the judge's.
Real pretrial risk instruments carry documented bias problems that go well beyond what a six-case fictional demo can show; treat this simulation as a lesson in accountability structure, not a benchmark of any actual tool.