By: Chris Kamin, Chief Business Development Officer, equivant Supervision + Pretrial
In recent years, the COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) instrument has become a focal point of debate, particularly concerning its role in the criminal justice system. Many misconceptions surround its purpose, functionality, and implications, often leading to a skewed public perception. In this blog, we’ll explore some of the most common inaccuracies about the COMPAS tool and provide clarity on its intended use.
Understanding the Difference Between Risk and Classification Instruments
One of the most critical misunderstandings about the COMPAS instrument is its categorization as a classification tool rather than a risk assessment tool. This distinction is crucial because it fundamentally changes how the instrument should be interpreted and used.
Risk Instruments vs. Classification Instruments:
Risk instruments, such as COMPAS, are designed to estimate the likelihood of a future event—in this case, the probability of recidivism (the tendency of a convicted criminal to reoffend). They do not classify individuals as “recidivist” or “non-recidivist” because these are not static categories that can be verified at the time of assessment. In contrast, classification instruments aim to identify and categorize existing conditions, such as distinguishing between different patterns or characteristics that can be immediately verified.
To illustrate this, consider the difference between a weather forecast and a handwriting recognition system. A weather forecast (risk instrument) might predict an 80% chance of rain but cannot guarantee that rain will occur. Similarly, COMPAS might estimate a high risk of recidivism without claiming that recidivism will certainly happen. On the other hand, a handwriting recognition system (classification instrument) identifies a particular letter based on its features, and its accuracy can be immediately checked and corrected.
Given this, calibration fairness—the accuracy with which predicted probabilities match actual outcomes—is the most appropriate measure of fairness for risk instruments like COMPAS.
The Nature and Purpose of the COMPAS Instrument
Another common misconception is that COMPAS functions as an Artificial Intelligence (AI) system that adjusts its algorithms based on new data. In reality, COMPAS is an Actuarial Risk Assessment Instrument (ARAI), which means it relies on statistical methods to assess risk based on existing data. It does not “learn” or change over time, and it performs a straightforward calculation to assist justice professionals in decision-making.
What COMPAS Measures:
The COMPAS tool assesses the probability of both general and violent recidivism. It also identifies criminogenic needs (factors that contribute to criminal behavior) and stabilization factors, aligning with the Risk-Need-Responsivity (RNR) model to help tailor case management and treatment strategies to individual offenders.
Importantly, COMPAS is a post-conviction tool—it is used after an individual has been convicted to aid in decisions related to parole, probation, and rehabilitation, rather than to determine sentencing length or severity.
Addressing Claims of Racial Bias
One of the most contentious issues surrounding COMPAS has been allegations of racial bias, particularly stemming from a 2016 ProPublica study. However, these claims have been extensively scrutinized and challenged by independent researchers.
Independent Analyses:
Studies conducted by researchers from California State University, Bakersfield, the Crime and Justice Institute at Community Resources for Justice, and the Administrative Office of the U.S. Courts Probation and Pretrial Services Office found significant flaws in ProPublica’s analysis. They concluded that the report was based on “faulty statistics and data analysis” and did not prove that COMPAS itself was racially biased.
Additionally, the Royal Statistical Society, a leading authority in the field of statistics, criticized the reasoning in the ProPublica study, emphasizing that differences in recidivism rates between racial groups are more likely due to systemic issues in the broader justice system, rather than the COMPAS algorithm itself.
To further clarify, COMPAS does not incorporate race as a variable in its risk assessment process, and it does not attempt to determine or influence sentencing severity. Its role is supportive, providing justice professionals with an analytical tool to help guide decisions based on a combination of interview responses, criminal history, and other documented data.
Addressing the Creation of COMPAS-R
COMPAS-R Core was not created to replace COMPAS Core and, in fact, equivant has many satisfied customers using both instruments, simply based on preferences and what works for their agencies.
Both the standard COMPAS Core and COMPAS-R Core are designed to assess risk and needs of persons recently removed from the community or currently in the community.
It is important to note that the COMPAS-R Core is based on the standard COMPAS Core, which was developed empirically with a focus on predictors known to affect recidivism. Like the standard COMPAS Core, it includes dynamic risk factors and assesses persons on a variety of well-validated risk and need factors designed to aid in correctional intervention to decrease the likelihood of reoffending.
The COMPAS-R Core consists of one risk scale, the Summative General Recidivism Risk Scale (Summative GRRS), and fifteen descriptive scales that measure criminal history, dynamic criminogenic needs, and stabilization factors. It is fully configurable, allowing agencies to create assessments that are composed of scales relevant to their particular requirements.
In addition to creating this document detailing the differences and similarities, we created a three-part blog series to answer the most commonly-asked questions regarding these two instruments:
- Why was the COMPAS-R Core Created and How Does it Differ from the Standard COMPAS Core? https://equivant-supervision.com/why-was-the-compas-r-core-created-and-how-does-it-differ-from-the-standard-compas-core/
- What New Configurations are Available in the COMPAS-R Core? https://equivant-supervision.com/what-new-configurations-are-available-in-the-compas-r-core/
- How Do the Scales in the COMPAS-R Core Differ from those in the Standard COMPAS Core?https://equivant-supervision.com/how-do-the-scales-in-the-compas-r-core-differ-from-those-in-the-standard-compas-core/
The debate over COMPAS and its role in the criminal justice system is complex, but it’s essential to separate fact from fiction. COMPAS is a risk assessment tool designed to provide probabilistic estimates, not definitive classifications, and it operates within a framework that aims to assist rather than dictate justice-related decisions. Misunderstandings about its function and purpose can lead to misguided criticisms, which is why it’s important to approach such discussions with a clear understanding of what COMPAS does—and does not—do.
References
- Flores, A. W., Lowenkamp, C. T., & Bechtel, K. (2016, July). False positives, false negatives, and false analyses: A rejoinder. Federal Probation Journal. https://www.uscourts.gov/federal-probation-journal/2016/09/false-positives-false-negatives-and-false-analyses-rejoinder
- Royal Statistical Society Section on Statistics and the Law. (2018, November 8). Algorithms in the justice system: Some statistical issues.
