NMSA Research Summary

Definition
Most of the research conducted on middle schools focuses on one of the six programmatic components of a successful middle school for young adolescents. For example, a multitude of studies exist on the effects of interdisciplinary teaming. Additionally, there is a significant body of research on advisory programs, student grouping, and developmentally appropriate approaches to teaching, learning, and assessment. In order to answer questions related to the middle school concept and its effects on student achievement and socio-emotional development, middle grades practitioners, researchers, and policymakers must move beyond this focus on individual components and look at research that addresses the reform as an integrated model, including the impact on student learning and achievement (Anfara & Lipka, 2003).

For the purpose of this research summary, student achievement is defined as academic achievement as measured by standardized test scores (e.g., state assessments, ITBS, CTBS, NAEP, NELS). To be included in this summary, the described studies met the following criteria: (1) research used large-scale study samples, so as to generalize the study results to the larger population; (2) research methods were scientifically based (valid and reliable) and replicable; and (3) studies that examined the effect of middle grades components (e.g., teaming, advisory, climate) on student outcomes, including student achievement (i.e., standardized test scores).

Research Summary
Lee and Smith (1993) conducted one of the first studies to use a large-scale sample to address the link between the implementation of middle school components and student achievement. Their study examined the effects of school restructuring on achievement and engagement of middle grades students. The study found that the following elements needed to be present in a middle school for it to be considered restructured in a way that was faithful to the middle school concept: reduced or eliminated departmental structure, heterogeneously grouped instruction, and team teaching. Academic achievement was defined as a composite score combining reading and math. Engagement was defined by measuring two variables: (1) the involvement of students in their academic work (e.g., homework, class work, preparation for and participation in class, and the like), and (2) the incidence of at-risk behaviors (i.e., the lower the incidence of at-risk behaviors the more engaged a student is). The results of this study can be divided into four categories: (1) student outcomes, (2) student backgrounds, (3) school demographics, and (4) school restructuring.

Lee and Smith (1993) found that elements of restructuring were positively associated with academic achievement and engagement. Specifically, there were modest increases in academic achievement (e.g., reading and mathematics), increases in student engagement (e.g., student completing homework and being prepared for class), and greater equity of student outcomes.

In 1997, the results of an Illinois middle school study examining the impact of school reform on student achievement was published in Phi Delta Kappan (Felner, Jackson, Kasak, Mulhall, Brand, & Flowers, 1997). Specifically, the study evaluated the effect of the Turning Points recommendations on student academic achievement, socio-emotional development, and behavior. Data were collected from 31 Illinois middle schools over a two year period from 1990 to 1992. Three levels of structural/organizational implementation were determined for each school based on the following characteristics: (1) levels of interdisciplinary teaming combined with high common planning time; (2) team size; (3) presence and frequency of advisory periods; and (4) levels of instruction, decision making, and teacher norms consistent with educational practices. Schools were categorized into one of three categories: low, partial, or high implementation.

The results of this study can be summarized in two categories: student achievement and other student outcomes. Using student achievement scores (reading, language, and math) from the Illinois state assessment, Felner and associates (1997) found that students in highly implemented schools outperformed students in partial and low implemented schools in all subject areas. Using teacher ratings of student behavior (aggression, anxiety, learning-related problems), they found that students in highly implemented schools had lower levels of behavior problems. In addition, students in more highly implemented schools reported lower levels of worry and fear and higher levels of self-esteem.

In 1999, a Chicago Consortium study examined the relationships of student social support and academic press to gains in student achievement in 304 Chicago schools (Lee, Smith, Perry, & Smylie, 1999). Survey data were collected from teachers and students and achievement data were obtained from sixth and eighth grade students in 1997. Social support was defined as an average score from four composite measures on the student survey. Each composite measure describes support from one of four sources: teachers, parents, peers, and the student's community (e.g., people in the neighborhood can be trusted). Academic press was derived from teachers' reports about their focus on academic achievement and student reports about being challenged by teachers to reach high levels. Student achievement data consisted of student scores on reading and math portions of 1997 Iowa Test of Basic Skills.

The results of this study can be summarized into three categories—social support, academic press, and combined effects. First, Lee and associates (1999) found that the amount of social support is strongly related to one-year gains in both reading and math. In addition, schools with high social support had average reading gains of 1.42 grade equivalents and average math gains of 1.67 grade equivalents. Second, the amount of academic press is strongly related to one-year gains in reading and math and schools with high academic press had average reading gains of 1.37 grade equivalents and average math gains of 1.64 grade equivalents. Last, and most significant, Lee and associates examined the combined effects by grouping schools into one of three categories (low, medium, or high). They found that students in schools identified as having both high social support and academic press reported the greatest gains in reading (1.82 grade equivalents) and math (2.39 grade equivalents).

The Center for Prevention Research and Development (CPRD) at the University of Illinois conducted several studies examining the impact of middle school components on student achievement using Self-Study data. The Self-Study is composed of a set of quantitative surveys completed by students, parents, administrators, and parents. Self-Study research data were collected from hundreds of schools in Arkansas, Louisiana, Michigan, and Mississippi between 1994 and 2003.

CPRD examined several middle school components including impact of teaming combined with common planning time, team size, length of time teaming, teacher certification, student latchkey status, and levels of structural/organizational implementation. CPRD research suggests that the implementation of middle school reform elements positively impacts student learning and achievement. Specific findings include

• Achievement scores are higher for students in schools that are teaming with high common planning time (Mertens & Flowers, 2006; Mertens, Flowers, & Mulhall, 1998).
• Team size and length of time teaming also affect student achievement scores (Flowers, Mertens, & Mulhall, 1999).
• Teachers with middle grades certification engage more frequently in "best practices," which impacts achievement (Mertens, Flowers, & Mulhall, 2002).
• Students home alone after school for three days or more report lower levels of self-esteem and academic efficacy and higher levels of behavior problems (Mertens, Flowers, & Mulhall, 2003).

In addition to the aforementioned research, several other studies warrant mention. Backes, Ralston, and Ingwalson (1999) examined the impact of middle school practices on student achievement in six schools in North Dakota. They found that achievement scores were generally higher in the schools implementing the Turning Points recommendations. Lee and Smith (2000) examined the impact of school size on student achievement and found that students in small schools (fewer than 400 students) performed better on standardized achievement tests and teachers reported a more positive attitude about responsibility for student learning. Sweetland and Hoy (2000) studied the relationship between school characteristics and educational outcomes and found that teacher empowerment (decision making) was linked to student achievement (reading and math). Last, McLaughlin and Drori (2000) conducted a study of school-level correlates of academic achievement in 20 states that combined teacher data from the National Center for Education Statistics' Schools and Staffing Survey (SASS) and student achievement data from state assessment and the National Assessment of Education Progress (NAEP). Using multivariate methods, they found that smaller class sizes had a significant impact on student achievement. In addition, they found relatively strong correlations between positive school climate and student achievement.

The results of these middle grades studies are promising. They provide middle grades practitioners, scholars, advocates, and policymakers with a firm foundation that links the middle school concept to improved student academic and socio-emotional development. These studies also provide a point of departure for the design and conduct of future research. Future research on student achievement, based on the recommendations contained in Research and Resources in Support of This We Believe (Anfara, Andrews, Hough, Mertens, Mizelle, & White, 2003), should include

• More large-scale, longitudinal studies.
• Studies combining quantitative and qualitative methodologies.
• Studies that examine more than one reform recommendation, practice, or design element.
• More studies that replicate previous methods and designs.

References

Anfara, V. A., Jr., Andrews, P. G., Hough, D. L., Mertens, S. B., Mizelle, N. B., & White, G. P. (2003). Research and resources in support of This We Believe. Westerville, OH: National Middle School Association.

Anfara, V. A., Jr., & Lipka, R. P. (2003). Relating the middle school concept to student achievement. Middle School Journal, 35(1), 24-32.

Backes, J., Ralston, A., & Ingwalson, G. (1999). Middle level reform: The impact on student achievement. Research in Middle Level Education Quarterly, 22(3), 43-57.

Felner, R. D., Jackson, A. W., Kasak, D., Mulhall, P., Brand, S., & Flowers, N. (1997). The impact of school reform for the middle years: Longitudinal study of a network engaged in Turning Points-based comprehensive school transformation. Phi Delta Kappan, 78(7), 528-532, 541-550.

Flowers, N., Mertens, S. B., & Mulhall, P. (1999). The impact of teaming: Five research-based outcomes of teaming. Middle School Journal, 31(2), 57-60.

Lee, V., & Smith, J. (1993). Effects of school restructuring on the achievement and engagement of middle-grades students. Sociology of Education, 66(3), 164-187.

Lee, V., & Smith, J. (2000). School size in Chicago elementary schools: Effects on teachers' attitudes and students' achievement. American Educational Research Journal, 37(1), 3-31.

Lee, V., Smith, J., Perry, T. E., & Smylie, M. A. (1999). Social support, academic press, and student achievement: A view from the middle grades in Chicago. Chicago: Consortium on Chicago School Research, University of Chicago.

McLaughlin, D., & Drori, G. (2000). School-level correlates of academic achievement: Student assessment scores in SASS public schools. (NCES 2000-303). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.

Mertens, S. B., & Flowers, N. (2006). Middle Start's impact on comprehensive middle school reform. Middle Grades Research Journal, 1(1), 1-26.

Mertens, S. B., Flowers, N., & Mulhall, P. (1998). The Middle Start Initiative, phase I: A longitudinal analysis of Michigan middle-level schools. Champaign, IL: Center for Prevention Research and Development, University of Illinois.

Mertens, S. B., Flowers, N., & Mulhall, P. (2002). The relationship between middle-grades teacher certification and teaching practices. In V. A. Anfara, Jr., & S. L. Stacki (Eds.), Middle school curriculum, instruction, and assessment (pp. 119-138). Greenwich, CT: Information Age Publishing.

Mertens, S. B., Flowers, N., & Mulhall, P. (2003). Should middle grades students be left alone after school? Middle School Journal, 34(5), 57-61.

Sweetland, S. R., & Hoy, W. K. (2000). School characteristics and educational outcomes: Toward an organization model of student achievement in middle schools. Educational Administration Quarterly, 36(5), 703-729.

Annotated References

Lee, V., & Smith, J. (2000). School size in Chicago elementary schools: Effects on teachers' attitudes and students' achievement. American Educational Research Journal, 37(1), 3-31.

This study explored whether teachers and students are influenced by the size of the inner-city elementary school to which they belong. Focusing on teachers' attitudes about their responsibility for student learning and students' one-year gains in mathematics achievement scores, Lee and Smith used data from almost 5,000 teachers and 23,000 sixth and eighth grade students in 264 K-8 Chicago schools. The data were collected through 1997 surveys and annual standardized tests. Hierarchical linear modeling (HLM) was employed to estimate school effects. On both outcomes, small schools (enrolling fewer than 400 students) are favored compared with medium-sized or larger schools. In small schools, teachers have a more positive attitude about their responsibility for students' learning and students learn more. Even after taking size into account, learning is also higher in schools with higher levels of collective responsibility. Thus, they concluded that school size influences student achievement directly and indirectly, through its effect on teachers' attitudes.

McLaughlin, D., & Drori, G. (2000). School-level correlates of academic achievement: Student assessment scores in SASS public schools. (NCES 2000-303). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.

This study from the National Center for Education Statistics combined two large-scale, national data sets—the Schools and Staffing Survey (SASS) and the National Assessment of Educational Progress (NAEP). The primary aim of the study was to demonstrate the potential value of linking SASS (process/context) data and NAEP (student achievement) data. The methodological approach was twofold. First, the researchers matched the 1993-1994 SASS data with state reading and mathematics NAEP data for public schools in 20 states. Second, by combining these data sources, they identified school-level correlates of student achievement in a broad sample of U.S. public schools. The study investigated the relationships in over 1,100 public elementary schools, 496 middle schools, and 595 high schools. The major finding was that average student achievement in a school is related to student background factors (e.g., poverty, race), school organizational features (e.g., school and class size), professional characteristics, and school climate.

Sweetland, S. R., & Hoy, W. K. (2000). School characteristics and educational outcomes: Toward an organization model of student achievement in middle schools. Educational Administration Quarterly, 36(5), 703-729.

In this study, empowerment is defined and measured in terms of teachers' power to control critical decisions about teaching and learning conditions. This research first considers the relationship between school climate and teacher empowerment, and then the relationship between teacher empowerment and school effectiveness, which includes measures of mathematics and reading achievement in 86 middle schools. The results of this study support the pivotal importance of teacher empowerment in the effectiveness of schools. Finally, a theoretical model is proposed to explain the linkages between organizational characteristics and student achievement.

Recommended Resources

Brown, K. M., Roney, K., & Anfara, V. A., Jr. (2003). Organizational health directly influence student performance at the middle level. Middle School Journal, 34(5), 5-15.

Cotton, K. (1982). Effects of interdisciplinary team teaching, research synthesis. Portland, OR: Northwest Regional Educational Laboratory. (ERIC Document Reproduction Service No. ED230533).

Hough, D., & Sills-Briegel, T. (1997). Student achievement and middle level programs, policies, and practices in rural America: The case of community-based versus consolidated organizations. Journal of Research in Rural Education, 13(1), 64-70.

Hoy, W. K., & Hannum, J. W. (1997). Middle school climate: An empirical assessment of organizational health and student achievement. Educational Administration Quarterly, 33(3), 290-311.

Roney, K., Anfara, V. A., Jr., & Brown, K. M. (2002, April). Revealing what's in the black box: The middle school movement and high student achievement. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.

Russell, J. F. (1997). Relationships between the implementation of middle-level program concepts and student achievement. Journal of Curriculum and Supervision, 12(2), 169-185.

Authors

Steven B. Mertens is a senior research scientist at the Center for Prevention Research and Development (CPRD) at the University of Illinois. He is currently a member of NMSA's Research Advisory Board, a council member in AERA's Middle Level Education Research SIG, and a member of the National Forum to Accelerate Middle-Grades Reform.

Vincent A. Anfara, Jr., is associate professor of educational administration and supervision at The University of Tennessee. He is currently the chair of NMSA's Research Advisory Board and is the column editor for What Research Says in Middle School Journal.

Citation

Mertens, S. B., & Anfara, V. A., Jr. (2006). Research summary: Student achievement and the middle school concept. Retrieved [date] from http://www.nmsa.org/ResearchSummaries/StudentAchievement/tabid/276/Default.aspx