HCZ – Promise Academy Charter Middle School (OJJDP Model Program Guide)

Office of Justice Programs, Innovation - Partnerships – Safer Neighborhoods

Office of Juvenile Justice and Delinquency Prevention (OJJDP) Serving Children, Families and Communities

PREVENTION

IMMEDIATE SANCTIONS

INTERMEDIATE SANCTIONS

RESIDENTIAL

REENTRY

MPG HOME SEARCH HOME SEARCH ENTIRE PROGRAM DATABASE SEARCH USING RISK & PROTECTIVE FACTORS PROGRAM TYPES RATING SYSTEM REFERENCES NOMINATE PROGRAMS RELATED LINKS RESOURCES DMC-REDUCTION BEST PRACTICES DATABASE DSO BEST PRACTICES DATABASE

HCZ – Promise Academy Charter Middle School

Intervention:

The Harlem Children’s Zone (HCZ) is a 97-block area in the Harlem neighborhood of New York that combines “no excuses” charter schools with communitywide initiatives to address some of the main problems that underprivileged children face every day, such as inadequate schools, high-crime neighborhoods, and health issues. One important component of the HCZ Project is the Promise Academy Charter Middle School, which aims to provide students in grades 6–8 with a well-rounded, high-quality education.

The Promise Academy middle school serves predominately low-income, minority students in Harlem who are usually 2 or 3 years behind grade level. While the community programs offered by HCZ are available to anyone living near the area, the charter middle school is not open to all students. Because of the limited amount of space available, admission to the Promise Academy is only offered to students whose numbers are randomly selected during the middle school lottery. The school began in 2004 with 100 sixth graders, and by 2009, there were 700 middle school students in the system.

The Promise Academy combines structural reforms with wraparound services to provide students with a comprehensive college preparatory program. The Promise Academy provides an extended school day and year, coordinated afterschool tutoring, and additional classes on Saturdays for anyone who needs extra help with homework. In the classroom, strong focus is placed on mastering basic skills, especially literacy and mathematics skills. Students have more than 2 hours of literacy instruction and more than 90 minutes of mathematics instruction every day. Students also explore coursework in the arts and sciences.

In addition, many students spend 10 or more hours a day in school. An average school day runs from 8 a.m. to 4 p.m., which is a 20 percent increase over a typical school day. Students are also given the opportunity to participate in afterschool programs, such as photography or music activities, that run from 4 p.m. to 6 p.m. The academic year is 210 days of school, which includes a mandatory 25-day summer program. Only 180 days of school are required by law.

The Promise Academy also runs a health clinic in the middle school building that provides students with free medical, dental, and mental-health services. Students are screened upon entering the school, and they receive regular check-ups through the Children’s Health Fund. Students also receive incentives for achievements (such as money or trips), nutritious and freshly made meals, bus fare, and other intangible benefits, such as support from committed staff members.

Evaluation Methodology:

Study 1
Dobbie and Fryer (2010) examined the effects of being offered admission or attending the Promise Academy charter schools in the Harlem Children’s Zone on academic outcomes. In New York State, oversubscribed charter schools are required by law to allocate enrollment offers through a random lottery system. The study took advantage of this system by comparing the average outcomes of students who “won” the lottery and were offered admission to the Promise Academy to the average outcomes of students who “lost” the lottery and were not offered admission.

The sample included students from the middle school lotteries from 2005 through 2006 (data was not available from the 2004 middle school lottery). The treatment group included 169 lottery winners in grades 6–8. The lottery winners were 51 percent male, and 84 percent black and 14.5 percent Hispanic. Seventy-two percent qualified for free lunch. The control group included 289 lottery losers in grade 6–8. The group was 43.3 percent male, and 82.6 percent black and 15.4 percent Hispanic. Sixty-five percent qualified for free lunch. There were no statistically significant differences between the groups.

The study used an intent-to-treat (ITT) analysis, which meant the treatment group included all students who were offered admission to Promise Academy, even if they did not attend. The ITT analysis therefore captured the effect of being offered admission, not of actually attending. The study also included treatment-on-treated (TOT) analysis, which looked at the average effect of enrollment in the Promise Academy on those students who won the lottery and chose to attend.

Data was collected from files at HCZ as well as administrative data on student demographics and academic outcomes from the New York City Department of Education. Academic outcomes in math and English Language Arts (ELA) were measured by the New York State tests that are conducted in the winters of third through eighth grade. The math portion of the exam includes questions on number sense and operations, algebra, geometry, measurement, and statistics. The section on ELA assesses students on information and understanding, literary response and expression, and critical analysis and evaluation.

Note: The researchers also examined the outcomes of the elementary school students. However, it was not possible to estimate the treatment effects using the ITT or TOT method because the elementary school lotteries had never been significantly oversubscribed. As a result, nearly all lottery losers were eventually offered admission into the Promise Academy. The study ultimately used two different statistical approaches to analyze the outcomes of elementary school students, but the results should be viewed with caution due to the methodological limitations.

Evaluation Outcome:

Study 1
The results of the study by Dobbie and Fryer (2010) showed that enrollment in the Promise Academy Charter Middle School run by the Harlem Children’s Zone has the potential to eliminate racial gaps in both math and ELA test scores between white and African American middle school students in New York City, N.Y.

Math Scores
The academic data showed that in fourth and fifth grade, prior to entering the Promise Academy, lottery winners, lottery losers, and the average black student in NYC had virtually the same math test scores, which were roughly 0.75 standard deviations behind the average white student in the city. The intent-to-treat analysis (ITT) showed that, compared to the control group, lottery winners had a modest but significant increase in math test scores in sixth grade, followed by a more substantial increase in seventh grade and an even larger increase in eighth grade. Lottery winners were 11.9 percent more likely to be performing on grade level in math in the sixth grade, 16.3 percent more likely in the seventh grade, and 27 percent more likely in eighth grade.

The treatment-on-treated (TOT) analysis, which looked at the effects of actually attending the charter middle school, showed a similar pattern. There was a convergence in test scores between students who attended the Promise Academy middle school and the average white student in NYC. In other words, Promise Academy students had nearly caught up to the average white student in New York City public schools in math test scores after 3 years of schooling. The TOT analysis found that lottery winners who enrolled in the charter middle school scored 0.338 standard deviations higher in sixth grade math, 0.371 higher in seventh grade math, and 0.745 higher in eighth grade math than the control group. The results were statistically significant in all 3 years.

ELA Scores
As with math scores, lottery winners, losers, and the average African American student in NYC had similar ELA scores, which were roughly 0.65 standard deviations behind the average white student in the city. ITT analysis showed there were no significant differences in ELA scores between the lottery winners and losers until eighth grade. In the eighth grade, Promise Academy students had significantly higher scores than the control group, although the treatment effect was not as strong on ELA scores as it was on math scores (lottery winners scored 0.196 standard deviations higher in eighth grade). The TOT analysis found that scores for students enrolled in Promise Academy had 0.279 standard deviations higher in eighth grade ELA compared to the control group, but there was no effect on sixth- and seventh-grade ELA scores.

Absences
In addition, compared to the control group, lottery winners had fewer absences in the first 180 days of school in every grade. Lottery winners were 2.85 days less absent in sixth grade, 2.31 days less absent in seventh grade, and 3.9 days less absent in eighth grade. There were no significant differences between lottery winners and losers in matriculation.

Elementary School Results
The researchers also analyzed academic results for students in the elementary school portion of the Promise Academy. Overall, Promise Academy students in elementary school significantly improved their math and ELA scores, and reported significantly fewer absences than the control group. The results suggest that the effects of attending the charter elementary school could potentially eliminate any racial gaps between white and black students in NYC. However, the limitations of this analysis make it impossible to draw any definitive conclusions about the treatment effect, and the results should be interpreted with caution. (For more information on the analysis limitations, please see the note in the “Evaluation Methodology” section.)

Other Information:

Additional Information:The Promise Academy Charter Middle School is part of the HCZ Project that provides a continuum of services focusing on the developmental needs of children in Harlem. In addition to the middle school programs, the Project includes elementary school programs that are available to students in grades K–5. However, the evaluation research examining the effectiveness of the elementary school programs has not yet reached the rigor of the research looking at the middle school programs.

Costs:In the 2008–09 school year, the Harlem Children’s Zone in the Harlem neighborhood of New York, N.Y., spent approximately $19,272 per pupil at the Promise Academy. This included in-school costs and afterschool programs. This can be compared to the median school district in New York State in 2006 that spent approximately $16,171 per pupil (Dobbie and Fryer 2010).

References:

Dobbie, Will, and Roland G. Fryer Jr. 2010. Are High-Quality Schools Enough to Increase Achievement Among the Poor? Evidence from the Harlem Children’s Zone. Cambridge, Mass.: The Kennedy School, Harvard University. http://www.economics.harvard.edu/faculty/fryer/files/HCZ_Nov_2010.pdf

Harlem Children’s Zone. 2009. “Home Page.” Accessed April 25, 2011. http://www.hcz.org/home/

Harlem Children’s Zone. N/d. Whatever It Takes: A White Paper on the Harlem Children’s Zone. http://www.hcz.org/images/stories/HCZ%20White%20Paper.pdf

Hoxby, Caroline M., and Sonali Murarka. 2009. Charter Schools in New York City: Who Enrolls and How They Affect Their Students’ Achievement. Cambridge, Mass.: National Bureau of Economic Research Working Paper. http://www.nber.org/papers/w14852

Hoxby, Caroline M., Jenny Lee Kang, and Sonali Murarka. 2009. Technical Report: How New York City Charter Schools Affect Achievement. Cambridge, Mass.: National Bureau of Economic Research. http://www.nber.org/~schools/charterschoolseval/how_nyc_charter_schools_affect_achievement_technical_report_2009.pdf

Top Tier Evidence Initiative. 2010. “Evidence Summary for the Promise Academy Charter Middle School in Harlem Children’s Zone.” http://evidencebasedprograms.org/wordpress/wp-content/uploads//PromiseAcademy.pdf

Program Specification:

Current Rating:

Effective

Expected Date of Re-Review:

Program Type:

Academic Skills Enhancement

Classroom Curricula

Mentoring

School/Classroom Environment

Truancy Prevention

Ethnicity:

African American

Hispanic or Latino (of any race)

Gender:

Both

Age:

11 - 14

Problem Behaviors:

Academic Problems

Risk & Protective Factors:

Risk

Community

Community crime / High crime neighborhood

Community instability

Economic deprivation / Poverty / Residence in a disadvantaged neighborhood

Feeling unsafe in the neighborhood

Low community attachment

Neighborhood youth in trouble

Social and physical disorder / Disorganized neighborhood

Individual

Antisocial behavior and alienation / Delinquent beliefs / General delinquency involvement / Drug dealing

Life stressors

Poor refusal skills

Victimization and exposure to violence

School

Inadequate school climate / Poorly organized and functioning schools / Negative labeling by teachers

Low academic achievement

Low academic aspirations

Low parent college expectations for child

Negative attitude toward school / Low bonding / Low school attachment / Commitment to school

Truancy / Frequent absences

Protective

Community

Presence and involvement of caring, supportive adults in the community

Prosocial opportunities for participation / Availability of neighborhood resources

Family

Good relationship with parents / Bonding or attachment to family

High family expectations

Individual

Healthy / Conventional beliefs and clear standards

High individual expectations

Perception of social support from adults and peers

Positive / Resilient temperament

Positive expectations / Optimism for the future

Social competencies and problem solving skills

School

Above average academic achievment / Reading and math skills

High expectations of students

High quality schools / Clear standards and rules

Opportunities for prosocial school involvement

Presence and involvement of caring, supportive adults in school

Rewards for prosocial school involvement

Strong school motivation / Positive attitude toward school

Student bonding (attachment to teachers, belief, commitment)

Additional Information:

Status:

Program is in operation at this time.

Performance Measures:

Suggested OJJDP Performance Measures for the Program Types(s):

Delinquency Prevention
Academic Skills Enhancement
Logic Model: PDF
Performance Matrix:PDF
School Programs
Academic Skills Enhancement
Logic Model: PDF
Performance Matrix:PDF
Delinquency Prevention
Classroom Curricula
Logic Model: PDF
Performance Matrix:PDF
School Programs
Classroom Curricula
Logic Model: PDF
Performance Matrix:PDF
Mentoring
Mentoring
Logic Model: PDF
Performance Matrix:PDF
School Programs
Truancy Prevention
Logic Model: PDF
Performance Matrix:PDF
School Programs
School/Classroom Environment
Logic Model: PDF
Performance Matrix:PDF

Contact Information:

Program Developer:
Harlem Children’s Zone
35 East 125th Street
New York, NY 10035
Phone: 212.360.3255
Fax: 212.289.0661
Email: Click Here
Website: Click Here