Skip to content

Public Safety Assessment

PSA Research

The PSA and its use in local jurisdictions are regularly evaluated and validated. Independent and transparent research is essential for contributing to the field’s understanding of pretrial practices and ensuring that the PSA accurately predicts pretrial outcomes.

About the PSA How the PSA Works

Research Types

PSA research falls into two major categories: validation studies and implementation studies. Some studies involve both types of research.

Validation Studies

These studies use either historical or post-implementation data from a community to test how the PSA performs in predicting pretrial outcomes. They also examine how accurately the PSA predicts pretrial outcomes across race and gender subgroups. The consistency of results across multiple PSA validation research studies suggests that pretrial practitioners can have confidence that the PSA, when combined with professional judgment, can contribute to improved pretrial decision making.

Implementation Studies

These studies examine how local jurisdictions make decisions to use the PSA, how it is implemented, and how its use in decision-making may influence pretrial outcomes in a jurisdiction. With rare exceptions, most studies find that using the PSA combined with other pretrial improvements is associated with higher rates of pretrial release without negative impacts on court appearance rates or new arrests. The research to date also suggests that implementation of the PSA is not associated with increased racial disparities.

Other Research

Additional studies of the PSA are neither implementation nor validation studies, but address topics related to the assessment, such as judicial decision-making and scoring rules.

This summary reviews the results of several PSA validation studies and presents the findings of recent implementation studies.

Download PDf

This summary reviews the research literature on the accuracy of pretrial actuarial assessment tools and explores how these tools may enhance pretrial outcomes.

Download PDF

Validation Studies

Authors: DeMichele, M., Tueller, S., Inkpen, C., Dawes, D., and Lattimore, P.
January 2023

  • RTI conducted a validation and bias study of the PSA in Fulton County, Georgia using a validation sample of over 20,000 individuals between January 1, 2017, and December 31, 2018. The study sample was limited to people who were booked into jail and subsequently released before case disposition. The sample excluded those who were released prior to booking or who posted bond prior to booking.
  • One-third of the people booked in Fulton County are there for a violent offense, 31 percent are booked for a property offense, 20 percent for a drug offense, about 10 percent for a public order offense, and 10 percent for other unspecified offenses.
  • Fulton County had a 60 percent pretrial release rate and detained 13,965 people for their entire pretrial period.
  • The validation found that people with lower PSA scores were somewhat more likely to be released than higher-scoring people. However, the differences in average scores are not large, suggesting that many people with low scores were detained, and many people with high scores were released. For instance, nearly 40 percent of the detained individuals scored 1-2 on the failure to appear (FTA) scale, and about 10 percent of the released individuals scored 5-6 on the FTA scale. There is a similar pattern across the other scales, with many individuals who scored less than 3 on the new criminal arrest (NCA) and new violent criminal arrest (NVCA) scales having been detained until case disposition in Fulton County. 
  • PSA “base rates” are the percentages of all people in the sample population who were released before trial and had an FTA, NCA, or NVCA during the study period. In Fulton County, the base rates were 16 percent (FTA), 24 percent (NCA), and 7 percent (NVCA).  The base rates vary across population groups: males and non-white people have significantly higher FTA, NCA, and NVCA rates compared to females and white people).
  • On average, individuals in Fulton County scored about 3 on the FTA and NCA) scales and 2 on the NVCA scale, with nearly 60 percent of the sample scoring between 1-2 on the FTA and NCA scales and nearly 80 percent scoring 1-2 on the NVCA scale.
  • The results demonstrate that the PSA meets standards of predictive validity for criminal legal system instruments. For the three scales, we found that the Area Under the Curve (AUC) values are in the fair (FTA, AUC = 0.62) and good (NVCA, AUC = 0.65 and NCA, AUC = 0.65) ranges. The results show that for each point increase in the FTA, NCA, and NVCA score there is a 34 percent, 51 percent, and 63 percent increase in the odds of those outcomes occurring, respectively. Higher scores are related to a significantly greater likelihood that someone will miss court or be rearrested for any crime or a violent crime during their pretrial release.
  • The results show that in Fulton County, the PSA is associated with pretrial outcomes.  The researchers did not find evidence that the PSA exacerbates predictive bias related to race and sex and did not find evidence that people of color are being scored higher than their actual outcome rates. Proper use of assessments requires ongoing research to ensure that the PSA remains valid for Fulton County.
Download PDF

Authors: DeMichele, M., Tueller, S., Burtch, E., Dawes, D., and Lattimore, P.
January 2023

  • RTI conducted a validation and bias study of the PSA in Pierce County, Washington. The data for this study include 6,437 bookings in Pierce County’s Superior Court from January 1, 2017, through December 31, 2018. These data were used to conduct an historical validation by scoring all people booked into the Pierce County jail between 2017 and 2018.
  • The most serious charge associated with a booking was classified as property (39 percent), violent (37 percent), drug (16 percent), or public order (9 percent). Slightly over one-third (36 percent) of the booked sample were detained, resulting in a 64 percent release rate.
  • Overall, people with lower PSA scores were more likely to be released and those with higher scores were more likely to be detained.
  • Most people have low PSA scores in Pierce County, with substantial majorities scoring between 1 and 3 on each of the scales:  failure to appear (FTA) (59 percent), new criminal arrest (NCA) (61 percent), and new violent criminal arrest (NVCA) (more than 90 percent). Relatedly, few people score high, with about 23 percent (FTA), 16 percent (NCA), and less than 2 percent (NVCA) scoring 5 or 6.
  • PSA base rates are the percentages of all people in the sample population who were released before trial and had an FTA, NCA, or NVCA during the study period. The overall base rate is 32.5 percent (FTA), 29.3 percent (NCA), and 8.3 percent (NVCA). The results demonstrate that the PSA meets standards of predictive validity for criminal legal system instruments. The AUCs (“area under the curve”) for NCA (0.61) and NVCA (0.66) are within the typical range for pretrial assessments. The FTA AUC score is a bit lower but remains statistically significant and above the 0.50 threshold. Future validations in Pierce County are recommended to make sure the predictive validity for FTAs improves if the PSA is implemented.
  • There are no significant differences in the predictive validity of the PSA across race and sex.
  • The results show that for each point increase in the FTA, NCA, and NVCA score there is a 22 percent, 31 percent, and 56 percent increase in the probability of those outcomes, respectively. Higher scores are related to a significantly greater likelihood that someone will miss court or be rearrested during their pretrial release.
Download PDF

Authors: D. James Greiner, Matthew Stubenberg, Ryan Halen
February 2021

  • Researchers conducted a validation study of the PSA in Kane County, Illinois, using over 13,000 felony and misdemeanor cases between February 2016 and October 2019. The study sample was limited to people who were booked into the jail and subsequently released before case disposition. The sample excluded those who were released prior to booking or who posted bond prior to booking.
  • Findings provided moderate evidence that the PSA is a valid predictor of pretrial outcomes overall. Lower scores on the failure to appear (FTA), new criminal arrest (NCA), and new violent criminal arrest (NVCA) scales were generally associated with increased rates of pretrial success. There was one exception: there were no statistically significant differences in NCA rates between scores 4–5 and scores 5–6, suggesting that the NCA scale is not discriminating as well at the higher end of the risk spectrum.
  • In terms of predictive accuracy as measured by AUC (“area under the curve”) and balanced accuracy metrics, the study found that the PSA’s validity falls within what is considered fair (for NCA and NVCA) to good (for FTA) in the criminal justice field. However, the research could not count all new arrest outcomes due to the unavailability of statewide criminal records for the sample, potentially limiting the accuracy of the findings for NCA and NVCA.
  • Aside from overall validity measures described above, the authors also examined the “uniform validity” of the PSA, or whether the magnitude of the differences in failure rates between each paired score comparison differed significantly. But for both NCA and FTA, there are differing patterns when you look at the lower score transitions compared to the higher score transitions (smaller differences on the low end of the FTA scale, compared to the higher end; smaller differences at the higher end of the NCA scale compared to the lower end.)
  • With respect to the validity of the PSA across race and gender groups, results were mixed. Accuracy metrics showed modest but statistically significant differences between Black and white people as well as between males and females on all three scales. But different metrics contradicted one another, with some showing better validity for Black people and others showing better validity for white people. The same was true of gender. As a result, the authors concluded that they had no evidence that the PSA was equitably invalid, although they also could not confirm that it was equitably valid in Kane County.
Download PDF

Authors: D. James Greiner, Matthew Stubenberg, Ryan Halen
November 2020

  • Researchers validated the PSA in Harris County (Houston), Texas, using over 60,000 felony and misdemeanor cases between July 2017 and December 2019. 
  • Findings show that the PSA is a valid predictor of pretrial success. Specifically, people with lower scores on the new criminal arrest (NCA) and failure to appear (FTA) scales were three to five times more likely to succeed pretrial compared to those with higher scaled scores. Moreover, the new violent criminal arrest (NVCA) flag was associated with a threefold increase in the probability of a new violent arrest, though overall rates of NCVA were relatively low for all people (3.6 percent for those without the flag and 10.2 percent for those with the flag).
  • In terms of predictive accuracy, the PSA’s predictive accuracy falls within what is considered fair (FTA, NCVA) to good (NCA) in the criminal justice field.
  • A secondary accuracy metric confirmed these findings. In this secondary analysis, of the three PSA scales, the NCA scale had the strongest predictive accuracy.
  • For NCA and NVCA, the PSA predicts equally well for Black and white people, and also for men and women. For FTA, researchers found differences in prediction rates for Black and white people but concluded they were either substantively small or inconsistent in direction.
Download PDF

Authors: D. James Greiner, Matthew Stubenberg, Ryan Halen
November 2020

  • Researchers analyzed the validity of the PSA using a wide range of methods on over 6,000 felony and misdemeanor cases in McLean County, central Illinois, between 2016 and 2020.
  • One traditional criminal justice validation technique (called “area under the curve,” or AUC) showed moderate to strong evidence that two of three PSA scales – failure to appear (FTA) and new criminal arrest (NCA) – are valid overall (meaning, higher scores have higher failure rates and PSA factors correlate with outcomes). This same technique showed weak evidence of the overall validity of the third PSA scale, new violent criminal arrest (NVCA).
  • With other techniques, the classifying accuracy of the PSA was moderate for FTA and NCA and weak for NVCA. However, the research could not count all failures due to the absence of statewide criminal history for assessing outcomes. 
  • There was some evidence that changes between scores on the FTA scale are not uniform. Failure rates increase little between scores 1 and 2 (low ends of the scale) and increase much more between scores 3 and 4 and between scores 4 and 5 (high ends of the scale.) In other words, there are larger differences in failure rates between high scores than between low scores. In contrast, changes on the NCA scale were uniform.
  • Results are mixed across equitable validity tests with the authors concluding “weak evidence exists” that the PSA classifications are the same for Black and white people, and also for men and women.
Download PDF

Authors: Matthew DeMichele, Peter Baumgartner, Michael Wenger, Kelle Barrick, Megan Comfort, and Shilpi Misra
April 2018

  • The PSA predicts well across all three outcomes—failure to appear (FTA), new criminal activity (NCA), and new violent criminal activity (NVCA); its predictive abilities fall within what is considered good in the criminal justice field.
  • For NCA and NVCA, the PSA predicts equally well for black and white individuals.
  • For FTA, the PSA predicts differently for black and white individuals in that it assigns black individuals lower risk scores than white individuals who fail to appear at the same rate.
Learn More

Implementation Studies

Authors: Glenn A. Grant, J.A.D.
2019

  • This report summarizes the performance of New Jersey’s Criminal Justice Reform (CJR), which was implemented in 2017. It analyzes the outcomes of people arrested in 2018 and examines the jail population as of 2019. Despite more people being released pretrial in 2018 than in 2017, the rate of pretrial failure was nearly identical to the rate observed in 2017: 13.8 percent of people released in 2018 were rearrested for an indictable offense and less than 1 percent were rearrested for the most serious or firearms-related crimes. Nearly 90 percent of people continued to appear in court.
  • The report confirms the predictive ability of the PSA. Only 9 percent of people who received a New Criminal Arrest (NCA) score of “1” were charged with a new offense while on pretrial release, while 53 percent of people with an NCA score of “6” were charged with a new offense while on pretrial release.
  • The overall number of inmates in custody continued to decline under CJR. There were 15,006 inmates in custody in 2012. The population dropped to 8,482 in 2018 and 7,937 in 2019—a 6.4 percent decrease between 2018 and 2019.  In addition, the crimes with which people in jail are charged are more serious, with two-thirds of those detained pretrial in 2019 charged with first- or second-degree offenses.
  • As compared to other racial demographics, Black people continue to be disproportionately arrested and still comprise 55 percent of the state’s jail population. The analysis in the report shows the data related to fairness and equity, and New Jersey acknowledges that all stakeholders must work together to address disparities.
  • In 2019, over 80 percent of people arrested on a warrant (custodial arrest) were released pretrial. Nearly 10 percent were released on recognizance.
Download PDF

Authors: Cindy Redcross, Brittany Henderson with Luke Miratix, Erin Valentine
March 2019

  • The PSA and related policy changes were associated with a decrease in the use of financial conditions of release and an increase in individuals released on their own recognizance.
  • The greatest impact on pretrial release conditions occurred at the initial magistrate hearing, before the PSA was completed; researchers suggest the PSA sparked a change in culture that contributed to this effect.
  • Following implementation of the PSA, Mecklenburg County released more people pretrial but did not see an increase in missed court appointments or new criminal charges.
Learn More

Authors: Chloe Anderson, Cindy Redcross, Erin Valentine with Luke Miratrix
November 2019

  • New Jersey’s Criminal Justice Reform (CJR) (which refers to a series of pretrial improvements, including the use of the Public Safety Assessment) nearly eliminated the use of financial conditions of pretrial release.
  • CJR was associated with a significant reduction in the total number of misdemeanor arrests in the year following implementation.
  • CJR appears to have led to an increase in the use of summonses (citations) in lieu of warrants (custodial arrests).
  • After CJR’s implementation, the amount of time a person spends in jail in the month following arrest was significantly reduced.
Download PDF

Author: Glenn A. Grant, J.A.D.
2018

  • New Jersey implemented the PSA and other criminal justice improvements on Jan. 1, 2017. There were 6,000 fewer people incarcerated on Oct. 3, 2019, than on the same day in 2012.
  • More than 70 percent of people charged with a crime were released with a summons pending the disposition of their cases without first being placed in jail.
  • Although more people were released pretrial in New Jersey after the changes, the rates of new criminal arrests and court appearance for individuals released pretrial remained virtually the same.

 

Download PDF

Author: Claire M. B. Brooker
November 2017

  • This preliminary study compared 250 cases during a six-month period after pretrial improvements were implemented with 250 cases during a six-month period before pretrial improvement implementation.
  • Judges decreased their use of secured financial conditions of release, relying far more heavily on unsecured conditions in the post-implementation time period.
  • While the release rates for white individuals did not differ significantly between the pre- and post-implementation time periods, the rates increased significantly for Latino/Hispanic and other groups (from 49 percent to 75 percent and 41 percent to 65 percent, respectively).
  • The pretrial release rate increased from 53 percent to 73 percent with no statistically significant difference observed in public safety and court appearance outcomes.
Download PDF

Validation and Implementation Studies

Authors: Christopher Lowenkamp, Matthew DeMichele, Lauren Klein Warren
January 2021

  • In January 2015, Lucas County (Toledo), Ohio, implemented the PSA along with other pretrial improvements. This study compares pretrial outcomes in the three years prior to PSA implementation to those observed in the four years post-implementation, and it revalidates the PSA, including testing for predictive bias.
  • This study included a replication of a 2016 published report and subsequent unpublished analyses that included pre- and post-implementation analyses and validations. The researchers were unable to replicate the original results.
  • Overall, jail bookings decreased substantially. Post-implementation, Lucas County reduced the number of people booked into jail by an average of almost 1,600 bookings per year.
  • The study found a significant decrease in jail bookings on lower-level charges during the post-implementation period and an increase in bookings on more serious charges.
  • The release rate was lowest immediately following implementation (70 percent) and increased gradually, with an average release rate above 80 percent during the final months of the study.
  • Pretrial success rates increased in the post-implementation period. The failure to appear (FTA) rate decreased from 30 percent to 24 percent, the new criminal arrest (NCA) rate decreased from 20 percent to 15 percent, and the new violent criminal arrest (NVCA) rate decreased from 6 percent to 4 percent.
  • Validation results revealed that the PSA scales were statistically associated with FTA, NCA, and NVCA outcomes in Lucas County. In terms of predictive accuracy, the PSA’s predictive accuracy falls within what is considered fair (FTA, NVCA) to good (NCA) in the criminal justice field.
  • For NCA and NVCA, the PSA predicts equally well for Black and white people, and equally well for men and women.
  • A well-calibrated assessment tool is one in which different subgroups produce equivalent outcomes at each score of an instrument. Predictive bias testing revealed some calibration issues with the FTA scale, specifically with respect to race and gender groups. Black people had higher predicted FTA rates than white people at the lower end of the scaled scores, and they had lower rates of FTA at the highest end of the scaled scores. Women were shown to have a slower increase in the rate of FTA as scores increased when compared to men.
  • Finally, the researchers examined PSA error rates—false positive and false negative rates–across gender and race groups. False positives occur when people are classified at the higher end of an assessment instrument but do not fail pretrial, while false negatives occur when people are placed at the lower end of the spectrum but do fail pretrial. In Lucas County, false positives were more common among Black people when compared to non-Black people, while false negatives were more common among women relative to men.
Learn More

Authors:  Elise Ferguson, M.A., Helen De La Cerda, B.A., Paul Guerin, Ph.D.
November 2019

  • The researchers studied pretrial outcomes (failure to appear, new criminal arrest, and new violent criminal arrest) associated with 6,000 felony cases disposed of between July 2017 and March 2019.
  • Overall, the PSA performed as expected, with increases on each scale corresponding to higher rates of pretrial failure.
  • Consistent with previous studies in other jurisdictions, the study found that a large majority of people assessed with the PSA are successful: 82 percent of people released pretrial returned to court as required, and 83 percent of people released pretrial were not arrested while on pretrial release.
Download PDF

Other Research

Authors: DeMichele, M., Inkpen, C., Silver, I., and Walker, J.
November 2023

  • Researchers studied the effects of using different time frames for the criminal history items on the PSA’s New Criminal Arrest (NCA) scale, which is currently scored using a person’s lifetime criminal convictions and sentences.
  • Using jail admission data from a large jurisdiction, this study varied the time frame – from 1 to 10 years before the current arrest – for the criminal history items on the NCA scale and examined scoring differences and the predictive validity of the tool by race. The results showed reductions in the proportion of Black people scored as high risk and no loss in validity using a shorter criminal history review.
  • Authors conclude that the study “demonstrates that shorter recall periods for criminal history items on pretrial instruments could potentially provide a direct and practical strategy to mitigate the effects of mass incarceration while maintaining predictive validity.”
Learn More

Authors: James Austin, Wendy Naro-Ware
April 2020

  • Researchers studied the impact of pretrial improvements, including the use of the PSA and fewer secured financial release conditions, that began in Cook County (Chicago) in 2017 by comparing pretrial outcomes for people charged with felonies during a 15-month period before improvements were implemented to a 15-month period after implementation.
  • The pretrial release rate increased from 72 percent to 81 percent, contributing to approximately 1,700 fewer people in jail on a daily basis.
  • The overall arrest-free rate remained the same at approximately 83 percent, and the violent arrest-free rate remained the same at approximately 98 percent.
  • The number of people transferred in one year for a sentence to state prison after a lengthy jail stay was reduced from 9,800 to 6,600.
Download PDF

Authors: Matthew DeMichele, Peter Baumgartner, Kelle Barrick, Megan Comfort, Samuel Scaggs, and Shilpi Misra
April 2018

  • Seventy-nine percent of judges surveyed report that the PSA “always” or “often” informs their decision making.
  • Sixty-one percent of judges, prosecutors, public defenders, and pretrial officers surveyed report that they “often” agree with the recommendation in the decision-making framework.
Learn More

Authors: Matthew DeMichele, Megan Comfort, Shilpi Misra, Kelle Barrick & Peter Baumgartner
April 2018

  • Judges perceive a tension when reconciling the actuarial aspect of the PSA and their experience-based inclination to learn about people’s lives as a way of assessing risk through determinations of culpability and blameworthiness.
  • Authors recommend the creation of researcher–judge feedback loops along with increased transparency of model development.
Learn More

Author: Megan T. Stevenson
August 2017

  • A 2011 law, making use of a pretrial assessment mandatory in Kentucky, led to an increase in non-financial release for people assessed as more likely to succeed, but a lower rate of release for those assessed as more likely to fail, resulting in only a small overall increase in pretrial release.
  • These changes eroded over time, however, and within a couple of years, the pretrial release rate was lower than it was before the new law.
  • Pretrial assessment had no effect on racial disparities once differing regional trends were accounted for.
  • The author calls for additional research and experimentation to help pretrial assessment produce larger benefits.

 

Learn More

More on the PSA

About the PSA

Learn how the PSA can enhance pretrial outcomes when implemented with fidelity.

Learn More

How It Works

The PSA factors and algorithm are publicly available here.

Learn More

Implementing the PSA

Discover what it takes to implement the PSA effectively and responsibly.

Learn More

PSA Guides

The guides provide all the information needed to implement the PSA.

Learn More

PSA Map

Explore which states and local jurisdictions use the PSA.

Learn More