A Look at those Accountability Levels

Here Worcester School Committee makes some interesting points about schools designated Level 4 according to the Massachusetts Department of Elementary and Secondary Education. She writes:

"So, when you line up all of the schools in the state in rank order, and draw a line at the bottom 1/5, that's your Level 3's.

But what makes a Level 4?

Beyond their having to be in Level 3 status, the only other restriction the state law makes on Level 4 and 5 schools is that they can make up no more than 4% of the schools in the state at any one time.

So which schools fall from 3 to 4?

The state, in their explanation of the classification of Level 3 schools says this:

The state’s lowest achieving, least improving Level 3 schools are candidates for classification into Level 4 at the discretion of the Commissioner of Elementary and Secondary Education. A Level 4 school may be classified into Level 5 by the Commissioner on behalf of the Board of Elementary and Secondary Education if it fails to improve; or if district conditions make it unlikely that the school will make significant improvement without a Level 5 designation.

(emphasis added) 
"...at the discretion of the Commissioner of Elementary and Secondary Education."

Interestingly, although no more than 4% of schools statewide can be categorized as Level 4 at any given time, Massachusetts' Level 4 schools are not necessarily all in the bottom 4% of schools. Let's look at Boston as an example. Below are the schools that fall in the bottom 5% of schools statewide. 

School	Percentile	Accountability & Assistance Level
Mattahunt	1	Level 4
Mildred Avenue K-8	1	Level 3
Henry Grew	2	Level 3
John Winthrop	2	Level 4
Higginson/Lewis K-8	2	Level 3
Madison Park High	2	Level 3
Ellis Mendell	3	Level 3
John P. Holland	3	Level 5
John W. McCormack	3	Level 3
Paul Dever	4	Level 5
Greenwood Leadership Academy	4	Level 4
William Ellery Channing	4	Level 4
King K-8	4	Level 3
Washington Irving	4	Level 3
Jeremiah Burke High	4	Level 4
Dorchester Academy	4	Level 3
Michael J. Perkins	5	Level 3
Gardner Pilot Academy	5	Level 3
Young Achievers	5	Level 3
Timilty Middle	5	Level 3
East Boston High	5	Level 3

Here you can see that Boston has two Level 5 schools, Holland and Dever, that are in the 3rd and 4th percentiles respectively while Mildred Avenue K-8 school is at the 1st percentile and is a Level 3 school. Further Dearborn Middle School, which was not included on the list, is at the 10th percentile and is a Level 4 school. In 2012, Lynn Classical was a Level 3 school despite being at the 28th percentile due to its graduation rate among some of its subgroups. The John Avery Parker School in New Bedford, a school at the 15th percentile, was named Level 5 last year even though it met all of its PPI targets for both the 'High Needs' and 'All Students' groups and had a Student Growth Percentile (SGP) that was above above target. The four schools designated Level 5 by the Commissioner were at the 3rd, 4th, 8th and 15th percentiles. Thus, a school's accountability level might not necessarily mean what we think it means.

In looking at this information more in depth, Novick's conclusion does seem apropos:

"An enormous degree of importance is placed on these standings, however, and the most critical ones, the ones that do the most damage to a school's image, the ones that cause the most angst in and about districts are the ones that are not based simply on a calculation (however flawed those might be). They're chosen.

So when we're making assumptions and drawing conclusions about what the levels mean, let's keep that in mind."