The Transparency Files: Standardized Testing

This is our third year of publishing the “Grade Equivalent Scores” for the Iowa Test of Basic Skills or ITBS – the standardized test we take annually at the Martin J. Gottlieb Day School.  We did not have comparison data the first year we published results.  I also blogged that year about our overall philosophy regarding the proper context for standardized testing.  Last year we able for the first time to graph out comparison data and provided an analysis of our findings.

There was, and continues to be, some confusion about the proper understanding of what a “grade equivalent score” is and, more importantly, is not.  I am happy to refer you to a thorough explanation, but if you want the quick summary:

Grade-equivalent scores attempt to show at what grade level and month your child is functioning.  However, grade-equivalent scores are not able to show this.  Let me use an example to illustrate this.  In reading comprehension, your son scored a 7.3 grade equivalent.  The seven represents the grade level while the 3 represents the month.  7.3 would represent the seventh grade, third month, which is December.  The reason it is the third month is because September is zero, October is one, etc.  It is not true though that your son is functioning at the seventh grade level since he was never tested on seventh grade material.  He was only tested on fifth grade material.  That’s why the grade-equivalent scores should not be used to decide at what grade level a students is functioning.

This year, I blogged about our firm belief that “reflection leads to achievement”.  In that post, I attempt to peg our pioneering work in 21st century learning to academic success. All the blogging, reflecting, technology, project-based learning, flipped classrooms, etc., we not only do because we believe it leads to the best outcomes…we do it because it does lead to the best outcomes.

We do not believe that standardized test scores represent the only, nor surely the best, evidence for academic success.  Our goal continues to be providing each student with a “floor, but no ceiling” representing each student’s maximum success.  Our best outcome is still producing students who become lifelong learners.

But I also don’t want to undersell the objective evidence that shows that the work we are doing here does in fact lead to tangible success!

Our graduates the last three years have successfully placed into the high school programs of their choice.  Each one had a different ceiling – they are all different – but working with them, their families and their teachers, we successfully transitioned them all to the schools and programs they qualified for.

And for three years running, despite all the qualifications and caveats, our ITBS scores continue to demonstrate excellence.  Excellence within the grades and between them. And let’s be clear, this academic excellence comes with an inclusive admissions process.

That’s the headline…let’s look more closely at the story.

First up is “Language”.

MJGDS ITBS 2013 - Language

 

Remember…in order to track a class you have to compare 2012 to 2013.  For example, in 2012, the Language Grade Equivalent of Average for Grade Two was 3.4.  In 2013, those kids are now in Grade Three and scored 4.9.  That class “grew” 1.5 from last year to this. (Also, the scale stops at 13…it is the highest score available.)

The positive, of course, is that each grade is functioning at an extremely high level!  There are dips up and down, but when both the averages and the diversity level is high, it hard to find much to point to.  One data point to explore is that although most classes grew at least one grade equivalency, Grades One and Two did not.  They still have high averages, but this is worth looking at further.  This is also the benefit of not teaching to the test…it can sometimes uncover gaps in curriculum or teaching.

Let’s move onto “Reading”.

MJGDS ITBS 2013 - Reading

 

Here the news is all positive!  Each class grew at least one grade equivalency (technically Grade One grew .9) and all are functioning at very high levels.  One possible influence that we will be looking to track is the growing impact of the Daily Five, which we began piloting in Grades 4 & 5, but also had influence on other grades.

Let’s take a look at “Math”.

MJGDS ITBS 2013 - Math

Again, the overwhelming news is positive.  This marks the second year we are using Singapore Math, the first year for a few of our teachers, and the first year of departmentalization in Grades Four & Five.  The only trend worth noting is the relatively flat growth in the youngest grades.  The grade averages, even in those grades, are appropriately high and the class averages still show growth.  It is the rate of growth we will need to explore.  It could be that our students are covering more than the curriculum in Kindergarten – where the curriculum is the most sparse – and it takes until Grades Two or Three before the curriculum jumps.  [NOTE: It takes a lot of courage for teachers to work under this level of transparency.]  We have noted in the past that the curriculum tends to start out slow and build…and if that is the case, we may need to think about how to supplement in Kindergarten and Grade One.

 

To sum up, despite our focus on individual growth, our average growth continues to significantly outpace national percentiles and grade equivalency scores.  Does “reflection lead to achievement” at MJGDS?  Does being a 21st century learning pioneer translate into high academic success?

Three years in a row may not be conclusive, but it carries weight.

As with last year, all receiving teachers will have prior years’ data and be charged with making the next year even better.  They have been up to the task these last three years and we look forward to more learning, more growth and more excellence in the year to come.

 

 

 

 

The Transparency Files: Annual Parent Survey

After making transparent the results of my own evaluation by both myself and my faculty, it is time to turn to our other annual survey: the Annual Parent Survey.  [For comparison sake, you are welcome to explore last year’s reflection.]

I will try to highlight any trends I see over the years as well as indicate anything of import in this year’s survey.

The first trend is that each year far fewer parents (22 vs. 39 vs. 64) filled out surveys!

Possible explanations?

Families could be thrilled with what’s going on.  Families could be resigned that the results are not taken seriously enough to invest the time in.  There could be a certain amount of apathy.  Or perhaps we are not marketing the surveys enough.

Regardless, the lower the rate, the possibility of strong validity is reduced.  But in the spirit of hoping to learn from whatever there is to learn…let’s move on to the results.

 

Parent Survey Graph #1

 

This is more even than last year, but still skewed a bit towards the younger grades, as expected in school surveys.  Let’s look at the BIG PICTURE:

ChartExport

 

The score is still promising.  On a  scale of 1-10, our average score wound up being an 8.0 Last year we scored an 8.3.  The year prior we scored 7.7.  It is a fairly stable score – especially considering the sample – but still leaves us some room to grow.  Let’s dig deeper.

ChartExport

 

When it comes to communication, we continue to have high marks.  Each one has gone up (except the first, which is virtually the same)!  The biggest jumps come in providing opportunities for parents to be involved in student learning (which was the lowest one the prior year).  Hopefully this indicates growth on our part from what we learned. Another big jump was in parent-teacher conferences, which I would like to attribute to our piloting Student-Led Conferences.  I am pleased that the biggest jump came in front office attitude, and I have shared that pleasure with those people.  The place to keep working appears to be ongoing communication about children’s academic status, which could be a concern about our current system in the Middle School or lack of a system in the Lower School.

ChartExport

 

I know it is a little crowded, so let me break down some of the highlights.

Let me unpack the non-subject specific areas first:

  • Most important increase!  The education offered at our school is high quality: 8.4 vs. 8.28 (last year) vs. 7.72 (the year before last year)
  • EVERY category in this section is up – most importantly questions that deal with “teaching in different styles”, “teaching to different levels” and “satisfied…(with) individual attention”.  All these categories blend with “21st century learning”.

General Studies:

  • The big picture is great!  The OVERALL rating has gone to 8.3 from last year’s 7.97 from the prior year’s 7.19. That is a trend in the right direction.
  • Based on that it makes sense that all General Studies scores are up from last year, with one exception.  The impact of Singapore Math continues to positively impact as does, we hope, the influence of the Daily Five on Language Arts.
  • But there is still room to grow.  Science took a drop – and that is despite dedicating time and an instructor in the Lower School for the first time.  It is impossible to know because of how the survey is done whether or not to attribute the drop to dissatisfaction in Grades K-2 (where there is time, but no teacher), Grades 3-5 (where there is time and a new position), or Middle School (which is the same).  Regardless, this is something very important for us to explore.  There is no “21st century learning” without high-quality Science instruction.

Jewish Studies, Resources and Extracurricular Activities:

  • The marks in these areas are, again, all slightly down from last year.  Not dramatically so, but still noticeable.  With renewed emphasis on language immersion, we really do believe our program is stronger, but for whatever reason it has not translated to parent satisfaction (at least from the sample).  It is possible that we have set a higher bar and the distance from it is now more noticeable.  We have pledged to post more video of our students engaged in authentic Hebrew speech and this will hopefully contribute to higher marks in the years to come.
  • Our weakest area has consistently been “After School Activities”.  It is hard to know if this is a referendum on the quality of what we do offer or on the quantity of what we do not offer.  We have added more club sports and a Chess Club on top of an Enhanced Kindergarten Program and partnership with the JCA.  We might wish to do some parent surveys to unpack this one more fully.

So there you have it for 2012-2013!

Thanks to all the parents who took the time and care to fill out surveys.  In addition to the multiple choice questions, there were opportunities for open-ended responses.  They added an additional layer of depth; one which is difficult to summarize for a post like this.  But please know that all comments will be shared with those they concern as we use this data to make enhancements and improvements headed into next year.  By the by, we are very pleased with how well satisfied our parents are with how the school is going…but be assured, just like with everything else, we expect to see growth and progress in a school where there is “a floor, but no ceiling”.