The MPG’s Program Rating Tool, which was previously used by the expert peer review panel to determine a program’s rating of effectiveness, has recently undergone some revisions. The rating instrument is now in line with the Program Evidence Rating Instrument used by the Office of Justice Programs’ new Web site, www.CrimeSolutions.gov.

The new instrument continues to rate the same four dimensions of program effectiveness: the conceptual framework, the evaluation design, outcome evidence, and program fidelity. However, the individual elements in each dimension have been modified. In addition, the scoring procedures have changed such that expert reviewers will no longer assess the entirety of a program’s evidence base to determine a final rating of effectiveness. Rather, reviewers score the most rigorous evaluation research available that will make up the program’s evidence base. No more than three studies will be included and scored as part of the evidence base. A program’s final rating will be determined by the overall strength of the studies included in the evidence base.

Many of the juvenile justice–related programs included in the MPG will also appear in the CrimeSolutions.gov database, but the final ratings used by the two sites differ. (The MPG currently rates programs as exemplary, effective, or promising. CrimeSolutions.gov rates programs as effective, promising, or no effects.)

Programs that were accepted for inclusion in the MPG before January 2011 will undergo a re-review to ensure conformity with the criteria and standards set by the new instrument. The following outlines how MPG’s existing database will be reassessed:

• MPG programs will be examined to ensure that they meet the new screening criteria. This involves ensuring that all evaluations and studies included in a program’s evidence base are either experimental or quasi-experimental designs with a comparison condition. Evaluations that are simple one-group pretest/posttest designs will no longer qualify or be scored.
  
• New studies will be identified and reviewed to determine whether they should be included in the evidence base used to rate a particular MPG program. (This process is being completed first for the MPG’s “exemplary”-rated programs. The MPG’s “effective” and “promising” programs will be reassessed at a later date.)
  
• Using the CrimeSolutions.gov Scoring Instrument, the studies will be reviewed and rerated, if necessary.
  
• New evaluation methodology and outcomes sections are being written to reflect any new studies included in the evidence base.