Validation. You’ve heard about it over the years. You know it’s a best practice. But can you explain it to your agency’s stakeholders?
Many criminal justice practitioners, even those of us who have undergone a validation study, are still fuzzy on the details. And while many of the details can be safely left up to the researchers and scientists, as practitioners, we still need to be conversant enough on validation to answer questions from our stakeholders.
“There is a distinction between risk validation and needs validation,” said Becky Kelderhouse, previous General Manager of Northpointe. “We hear about ‘risk and needs validation’ quite a bit these days, and much of the chatter indicates that validation is one process done with one set of data at a certain point in time. The expected result? Getting a stamp of ‘Validated!’ and a confirmation that the ‘risk and needs assessment’ has been successfully validated. This is not correct: Each assessment is different, and each validation process is different.”
There is no such thing as “risk and needs validation.” Let’s take a closer look at both kinds of assessment validation:
Risk validation tests the output of the risk assessment to see how often it was accurate. For example, if the tool predicted a low risk level for an offender (low likelihood of re-offense), did that individual commit another offense within a set period of time? If they did re-offend, the prediction wasn’t accurate for that person. The process of risk validation determines whether the assessment tool predicted properly an acceptable proportion of times. Predictive models are never 100% accurate, and in criminal justice, the accepted accuracy level for risk assessments is 70%.
Needs validation, on the other hand, tests whether the tool itself measures what it is intended to measure. For example, when you’re using a scale (set of questions) for antisocial personality, is it asking the right questions to accurately test for antisocial personality? In a needs validation, researchers examine whether the assessment covers the proper domains based on criminological evidence, whether it meets the criteria established by academic researchers, and if independent peer reviews exist. Because the needs measured are criminogenic, a slight, positive correlation with recidivism may be found, but this should not be the measure by which validity is determined.
Validation is not a one-and-done exercise – it’s an ongoing process that needs to be repeated periodically over time to ensure the assessment’s validity endures.
Assessment validation is a best practice. Or, as most criminal justice researchers would argue, “best practice” isn’t strong enough – validation is critical. Unless your assessments are validated, you don’t have documented evidence to show how the tool is working in your agency.
Need more than the basics to properly educate your agency and your stakeholders on validation? Talk to us. We’re happy to walk you through the validation process from start to finish, and we can give you the tools to advocate for the validation studies your agency needs.
