Tuesday, September 1, 2015

Bad Science--The Meta-Study on Racial Bias and Shootings

I saw a couple news stories today reporting on a University of Illinois "meta-study"--a review of other studies performed on the topic--that it claims show a racial bias as to defensive shootings of whites versus blacks. (CBS and the Pacific Standard). The inference from both articles is that police are quicker to shoot black suspects than they are to shoot white suspects. The CBS article merely suggests the inference, using the term "people" to refer to those making the shoot/no shoot decision. The Pacific Standard is explicit as to the inference, beginning:
The recent, well-publicized shootings of unarmed black men by police officers have raised many troubling questions. Perhaps the most fundamental: whether these tragedies reflect a larger societal bias that sees men of color as threatening figures, which could in turn prompt even well-trained police officers to adopt a shoot-first-ask-questions-later mindset.
There are several problems with the meta-study that a non-biased "journalist" should have easily found and pointed out. First, although the authors of the meta-study claim to have reviewed 42 prior studies on the topic, there is no reference to the studies that they reviewed. However, I would not be surprised if the majority of the studies were, in fact, a series of reports from research over a period of years at the University of Chicago that found that participants (students at the University of Chicago) were quicker to push a button marked "shoot" when flashed a picture of a black man, versus one with a white man. Of course, even that research does not actually show that people are quicker to shoot blacks--if anything, it only shows the proclivities of those students that attend the University of Chicago. More importantly, it doesn't test whether police officers are quicker to pull the trigger or, rather, push the button.

However, in subsequent study, the University of Chicago performed a follow up study at the behest of the Denver Police Department. While the study found that both police and citizens had a faster reaction time when responding to armed black suspects, the error rate (i.e., mistakenly pushing the button for an unarmed suspect versus an armed suspect, or vise versa) showed no such bias on the part of the Denver and national police officers--i.e., the officers were no more likely to shoot a black suspect than a white suspect.

Last year, Lois James, David Klinger and Bryan Vila--researchers at Washington State University (Spokane)-- studied for bias using a more realistic scenario, which was published in the Journal of Experimental Criminology. Instead of flashing pictures--like a video game--as in the University of Chicago study, where the shoot/no-shoot decision was merely based on the presence of a weapon, the UW researchers used a full-sized HD video screen and a replica of a firearm to more realistically mimic lethal force scenarios. Using citizens (not police) as test subjects, the report found that participants were actually quicker to shoot white suspects. (PDF of the study). James, et al., had made a preliminary study the prior year that apparently found that police were slower to shoot black suspects, and less likely to make mistakes when shooting black suspects. (James, L., Vila, B. & Daratha, K. (2013) Influence of Suspect Race and Ethnicity on Decisions to Shoot in High Fidelity Deadly Force Judgment and Decision-Making Simulations. Journal of Experimental Criminology, 9(2), 189–212), that had found that various people--police, military, and untrained civilians--were quicker to shoot white suspects.

Second, there is no information on how (or whether) the University of Illinois researchers weighted the prior studies. So we don't know if they simply added up the number of studies showing bias versus those that did not to come to their conclusions, or if they gave greater weight to certain studies over others. If there was no weighting, then it shows that the University of Illinois authors made no attempt to discriminate between studies of good and poor quality, or adjust for changes in bias over time: garbage in, garbage out.

No comments:

Post a Comment

New Bombs & Bants (Streamed 8/10/2022)

  VIDEO: " Bombs and Bants Live! Ep 47 " (52 min.) We needed a break from the doom and gloom, so in this week's episode we tal...