Why our Police Department Conducted Data-Driven and Experimental Trials

The benefits of practitioner-led research trials are immeasurable. In particular, homegrown practitioner-led trials allow for quick adaptability and adjustment. Trials should emphasize putting research into practice because there are clear benefits when those closest to a problem are empowered to create solutions (Hawken 2018). For example, companies such as Amazon and Google test thousands of proposed innovations every year. Jeff Bezos of Amazon describes the success at Amazon as being “…a function of how many experiments we do per year, per month, per week, per day….” (Hawken 2018). “Failing forward” is critical to learning but failing in an unconventional way can be perceived as problematic. 

The Vallejo, California Police Department (VPD) recently demonstrated how police-run trials can help determine if an intervention, policy, or decision had the desired results. However, conducting experimental tests in policing comes with challenges; data that suggest a policy or initiative was a success or failure may be challenged, or perceived to be ambiguous or easily explained away by something else. Further, front-line officers often question the purpose of spending resources on analyzing data as they run from call to call, are short staffed, or are subjected to forced overtime.

 In the face of this strain, we have a generation of young police officers motivated to make a difference; they understand the importance of emerging technology, data, and research for efficiency. Our 3-5-year cops, mostly millennials, are more apt to embrace technology, data, and analysis. They have grown up with smartphones, the internet, and some are inclined to know the basics of coding, algorithms, and analytics. Further, today, there is a growing group of professionals such as members of the American Society of Evidence-Based Policing (ASEBP) eager to find practical, smart, and intuitive data collection portals to provide a means to think differently. ASEBP and its members are dedicated to using science, data, and research to inform decisions, policies, and practices to improve policing.

In this vein, members of the VPD sought out to test the effectiveness of theft deterrent strategies to combat thefts from autos and automatic license plate readers to combat auto thefts.

Data-Driven Intervention: HideitLockitTakeit Campaign

In 2017 VPD launched a crime prevention campaign called HideitLockitTakeit that aimed to reduce auto burglaries in a high-density shopping center during the holiday season. Data analysis allowed us to be more laser-focused in our approach to combating crime by providing information on where and when theft was occurring; it enabled us to be strategic, rather than to conduct enforcement blindly or ineffectively (Kaste 2018). This idea is supported by research, which suggests that as much as half of all crime in any jurisdiction occurs in just 3-5% of locations (Sherman et al. 1989).

We tested a multitude of interventions focused on a small geographic area in the shopping center where theft from autos was occurring, while analyzing and assessing data with our crime analyst. We also utilized a partnership with BetaGov – a research organization led by Angela Hawken that supports practitioner-led research (Potts 2018). The BetaGov statistician conducted correlation analysis, which demonstrated that deterrence methods such as increased officer/citizen contacts were correlated with fewer auto burglaries (p<0.05). The combined deterrence methods corresponded with a 40% percent drop for auto burglaries in 2017 from the same period in 2016 (Potts 2018). This was notable, as incidents of theft from autos were up 25% leading up to the interventions compared to the previous year.

BetaGov also helped us disentangle the various interventions to show which ones had a potential impact. “When implementing multiple interventions simultaneously, there is always a challenge of determining causality. Many police researchers and evidence-based policing purists may prefer the police implement one intervention at a time, to isolate the causal effect better. However, in the real world of policing – with real victims and a constant focus on reducing crime and costs – it makes sense to implement multiple interventions simultaneously” (Potts 2018).

Randomized Control Trial: ALPR Technology

Analyzing intervention effectiveness using randomized controlled trials (RCTs) is considered to be the gold-standard in research, as these experiments have high internal validity and can help isolate the causal effect of an intervention by comparing an intervention group to a control group. (Hawken 2017; Potts 2018). Using this scientific process, VPD assessed whether automatic license plate reader (ALPR) technology was useful in its jurisdiction and working as intended. Those randomly assigned to the intervention group of ALPR vehicles had their alerts “activated” (ALPR alert function on), while the control group had their alerts “deactivated” (ALPR alert function off for the officer but with the ability to retrieve the data later) (Potts 2018).

Obtaining data for our ALPR RCT proved cumbersome due to a Windows-based system that was limited by garbage-in, garbage-out data by front-end users. For example, it was not always immediately clear if the stolen auto recovery was from a call for service (dispatched) or if another officer without ALPR was responsible for the initial spot. Further, all too often police departments utilize outdated records management systems (RMS) that are far from intuitive. For example, it should not require us months to competently learn an RMS system, and we should not have to muddle through numerous fields to get accurate data. Because of these limitations, correctly assessing outcomes required us to read almost every report to ensure the accuracy of the data.

Nonetheless, after painstakingly looking at all the data outcomes, we discovered the police cars equipped with ALPR technology showed a 140% greater ability to detect stolen vehicles. However, further analysis showed this particular technology also identified many more lost or stolen plates–many of which were duplicates and may have desensitized officers to legitimate hits. We also discovered that 35-37% of all hits resulted in a misread, which we would not have known without the trial (Potts, 2018). 

Surprisingly, a survey of our officers showed that only 1 out of 37 surveyed would not participate in further trials. Nonetheless, RCTs are not always feasible in law enforcement or even in medicine, as there are concerns with parts of an experiment being withheld treatment in an attempt at isolating the intervention. Not everything can or should be an RCT, and some external findings may or may not be generalizable to another jurisdiction or agency (Alpert and Cordner 2018).

Conclusion

As we advance into 2020, obtaining the most accurate contextual data while being able to analyze it, will be at the forefront of our ability to effectively police — accomplished through robust social network analysis and evidence-based policing approaches. These approaches will be even more evident on January 1, 2021, when the Federal Bureau of Investigation (FBI) withdraws the Uniform Crime Reporting (UCR) Program’s Summary Reporting System (SRS). After then, the FBI will only collect crime statistics through the National Incident-Based Reporting System (NIBRS). We should understand why this is a significant move and prepare for it. NIBRS will allow us to look at victim and offender relationships in a more detailed way, and harm focused crime will be addressed more appropriately, and this where robust and efficient RMS software will be critical.

Finally, there’s no cure-all fix to improve the challenges of crime reduction. Policing is based mostly on “culture, politics, law, agency-specific values, and public opinion,” but the hope is that by continually analyzing and assessing data with effective systems we can better understand the impact of our responses; we can review and use the best available evidence to challenge and strategically inform our long-term decisions, policies, and practices to make us more effective in improving public safety (Alpert and Cordner 2018; Potts 2018). We still have a long way to go to embrace data in policing thoroughly. But, if Billy Beane of the Oakland A’s can embrace data and analysis in professional baseball in the face of much resistance, then we in policing shouldn’t be deterred by the field’s overemphasis on experiences and tradition over science (Worden & McLean 2018).

References 

Alpert, G and Cordner, G (2018) “Striking a Balance – Research, Science, and Policing,” Research in Brief, The Police Chief (August, 2018): 14-15

BetaGov (2018). About BetaGov. Retrieved from http://betagov.org/html/about-betagov.html (accessed 14 August 2018)

Hawken, Angela (2018). “Evidence-Based Policing – The Importance of Research and Evidence. NU Research for the Real World – June 2018. Retrieved from https://nij.gov/multimedia/Pages/video-rfrw-evidence-based-policing-transcript.aspx

Kaste. M (2018). How Data Analysis is Driving Policing. NPR. Retrieved from https://www.npr.org/2018/06/25/622715984/how-data-analysis-is-driving-policing

Potts, J. (2018). How a California Agency’s Theft Deterrent Strategies Led to a 40% Decline in Auto Burglaries. Police One. Retrieved from https://www.policeone.com/investigations/articles/478243006-How-a-Calif-agencys-theft deterrent-strategies-led-to-a-40-percent-decline-in-auto-burglaries/

Potts, J. (2018) “Assessing the Effectiveness of Automatic License Plate Readers,” Research in Brief, The Police Chief (March 2018): 14–15.

Sherman, L. W., Gartin, P. R., & Buerger, M. E. (1989). Hot spots of predatory crime: Routine activities and the criminology of place. Criminology, 27(1), 27-56 

Worden, R. & McLean, S. (2018). Leveraging CompStat to Include Community Measures in Police Performance Management. Publisher: Vera Institute of Justice. DOI: 10.1111/j.1745-9133.2003.tb00006.x/pdf