Tag: standardized testing

NOT Preparing for the CAT4 – How OJCS Thinks About Standardized Testing

From November 5th – 7th, students at the Ottawa Jewish Community School in Grades 3 – 8 will be writing the Fourth Edition of the Canadian Achievement Tests (CAT4).  The purpose of this test is to inform instruction and programming for the 2019-202o school year, and to measure our students’ growth over time.  

  • If this is the first time you are visiting this topic on my blog, I encourage you to read my post on our philosophy of standardized test-taking.
  • If you are curious about how we share the results of our standardized test-taking (and what those results have been), I encourage you to read that post as well.

What’s new for 2019-2020?

We have gone from offering the exam in Grades 3, 6, and 8 to Grades 3 – 8 in order to ensure that the data is actionable on all four levels – that of the individual student (is there something to note about how Jonny did in Mathematics from last year to this year?), individual classes (is there something to note about how Grade 5 scored in Spelling compared to when they were in Grade 4?), grades (is there something to note about how Grade 3 performed in Reading  this year when compared to how Grade 3 did last year?), and the school as a whole (how does OJCS do in Vocabulary across the board?).  Without testing the same students in the same subjects at the same time of year on an annual basis, we would not be able to notice, track or respond to meaningful patterns.

Reminder:

Standardized tests in schools that do not explicitly teach to the test nor use curriculum specifically created to succeed on the tests – like ours – are very valuable snapshots.  Allow me to be overly didactic and emphasize each word: They are valuable – they are; they really do mean something.  And they are snapshots – they are not the entire picture, not by a long shot, of either the child or the school.  Only when contextualized in this way can we avoid the unnecessary anxiety that often bubbles up when results roll in.

Last year it took about six weeks to get results back, analyzed and shared out – to parents with individual results and to community with school metrics.  We hope to be in that window of time again and look forward to making full use of them to help each student and teacher continue to grow and improve.  We look forward to fruitful conversations.  And we welcome questions and feedback through whatever channels they come…

The Transparency Files: CAT*4 Results

In the lull between parent-teacher conferences, I spent my time reading and analyzing the results of this year’s CAT*4 testing.  [I strongly encourage you to reread (or read for the first time) my philosophy on test-taking and how we planned on both sharing the tests with parents and utilizing the data in our decision-making.]  We are in the process of providing our teachers with the data they need to better understand their students and to identify which test results fully resemble their children well enough to simply pass on and which results require contextualization in private conversation.

In terms of sharing out the results publicly, which I will happily do, there are a few things worth pointing out:

  • Although we do have prior years, they are not “apples to apples” enough to plot as comparison data.  This is mostly because of our decision to change our testing window and partially because we don’t have enough grades taking the test often enough.  (I have data on spring tests from two and three years ago for grades 3 & 6.)  If that changes, part of this annual analysis will consist of tracking the grades over time to see if…
    • The same grade scores as well or better each year.
    • The same class grows at least a year’s worth of growth.
  • The other issue is in the proper understanding of what a “grade equivalent score” really is.

Grade-equivalent scores attempt to show at what grade level and month your child is functioning.  However, grade-equivalent scores are not able to show this.  Let me use an example to illustrate this.  In reading comprehension, your son in Grade 5 scored a 7.3 grade equivalent on his Grade 5 test. The 7 represents the grade level while the 3 represents the month.  7.3 would represent the seventh grade, third month, which is December.  The reason it is the third month is because September is zero, October is one, etc.  It is not true though that your son is functioning at the seventh grade level since he was never tested on seventh grade material.  He was only tested on fifth grade material.  He performed like a seventh grader on fifth grade material.  That’s why the grade-equivalent scores should not be used to decide at what grade level a student is functioning.

One final caveat about why share out grade and class averages at all when so much of our focus is on personalized learning and individual growth…

Here, my thinking has been influenced by the work I was doing prior to coming to Ottawa, in my role as Executive Director of the Schechter Day School Network and then part of the transition team which helped create Prizmah.  I cannot tell you how many conversations I have had with colleagues about the different challenges Jewish day schools often have from their secular private school and high-achieving public (and/or gifted programs and in the States and/or magnet and/or charter) school neighbors.  The biggest difference comes down to a philosophy of admissions.  [Please note that although a primary audience for my blog are OJCS parents, other folk read as well, so I am including references to forms of public education that are commonly found in the States.]

Most Jewish day schools attempt to cast the widest net possible, believing it is our mission to provide a Jewish day school education to all who may wish one.  We do not, often, restrict admission to a subset of the population who score X on an admissions test and we do not, often, adjust birthday cutoffs or recommend grade repeating to maximize academic achievement. However, schools who we are most often compared to in terms of academic achievement often do one or both.  If you then factor in whether or not you exempt special needs students from the testing and whether or not you explicitly teach to the test, you may have quite an uneven playing field to say the least.

To reframe and reset the discussion:

Jewish day schools have an inclusive admissions policy, but are expected to compete equally with elite private and high-achieving public (and gifted and, in the States, magnet and charter and suburban public) schools who have exclusive admissions policies or homogeneous populations.

So, in light of all of that – if a Jewish day school with an inclusive admissions policy, a non-exempted special needs population, and a commitment to “not teach to the test” – if that kind of school could demonstrate that it was achieving secular academic excellence on par with elite schools; well to me that would be news worth sharing.

So with all those caveats in mind, in the spirit of full transparency, and with the attitude that all data is valuable data, allow me to present this year’s results:

The bottom line of this graphic is that each grade in the Ottawa Jewish Community School scored, with a few exceptions, at a mean grade equivalent a full year higher than their current grade.  There are a few (Grade 3 Writing, Grade 3 Spelling, Grade 6 Writing, Grade 6 Spelling and Grade 6 Computation) that are closer to their current grade.  [Part of our ongoing analysis and annual comparison would be to learn more about our current spelling and writing outcomes.  Part of our deeper investigation is whether there is a way to layer on standardized French and possibly Hebrew tests to learn more about those important outcomes.]  There are a lot of grades/topics whose averages are significantly higher than that, but let the boldface sink in for a bit.

Too much time dedicated to Jewish Studies?  Nope – a high-quality Jewish Studies program enhances secular academics.  Too much time dedicated to Skyping or blogging?  Nope – an innovative learning paradigm not only positively impacts student motivation, but leads to higher student achievement.

I can sense the tone of triumphalism in my writing and, although I am extremely proud of our students and teachers for their achievements, I do not wish to sound boastful.  But with the state of Jewish day school education being what it is, when there is good news to share…share it one must!  I firmly believe that Jewish day schools with dual-curricula (and in our case tri-curricula!) and innovative pedagogy and philosophy produce unmatched excellence in secular academics.  Here in our school, we will have to prove it year after year, subject after subject, and student after student in order to live up to our mutually high expectations, but what an exciting challenge it shall be coming to school each day to tackle!

This Is (Not) A Test: OJCS (Doesn’t) Prep For CAT-4

From October 29th-31st, students at the Ottawa Jewish Community School in Grades 3,  6 and 8 will be writing the Fourth Edition of the Canadian Achievement Tests (CAT- 4).  The purpose of this test is to inform instruction and programming for the 2018-2019 school year, and to measure our students’ achievement growth over time.

Seems pretty non-controversial, eh?

These days, however, the topic of “standardized testing” has become a hot topic.  So with our testing window ready to open next week, this feels like a good time to step back and clarify why we take this test and how we intend to use and share the results.  But first, two things that are new this year:

  • We moved our test window from the spring to the fall to align ourselves with other private schools in our community.  This will be helpful for comparison data.  (This is also why we didn’t take them last year.)
  • We have expanded the number of grades taking the test.  We have not yet decided whether that number will expand again in future years.

What exactly is the value of standardized testing and how do we use the information it yields?

It sounds like such a simple question…

My starting point on this issue, like many others, is that all data is good data.  There cannot possibly be any harm in knowing all that there is to know.  It is merely a question of how to best use that data to achieve the fundamental task at hand – to lovingly move a child to reach his or her maximum potential.  [North Star Alert!  “We have a floor, but not a ceiling.”]  To the degree that the data is useful for accomplishing this goal is the degree to which the data is useful at all.

Standardized tests in schools that do not explicitly teach to the test nor use curriculum specifically created to succeed on the tests – like ours – are very valuable snapshots.  Allow me to be overly didactic and emphasize each word: They are valuable – they are; they really do mean something.  And they are snapshots – they are not the entire picture, not by a long shot, of either the child or the school.  Only when contextualized in this way can we avoid the unnecessary anxiety that often bubbles up when results roll in.

Like any snapshot, the standardized test ought to resemble its object. The teacher and the parent should see the results and say to themselves, “Yup, that’s him.”  It is my experience that this is the case more often than not.  Occasionally, however, the snapshot is less clear.  Every now and again, the teacher and/or the parent – who have been in healthy and frequent communication all the year long – both look at the snapshot and say to themselves, “Who is this kid?”

When that happens and when there is plenty of other rich data – report cards, prior years’ tests, portfolios, assessments, etc. and/or teacher’s notes from the testing which reveal anxiety, sleepiness, etc. – it is okay to decide that someone put their thumb on the camera that day (or that part of the test) and discard the snapshot altogether.

Okay, you might say, but besides either telling us what we already know or deciding that it isn’t telling us anything meaningful, what can we learn?

Good question!

Here is what I expect to learn from standardized testing in our school over time if our benchmarks and standards are in alignment with the test we have chosen to take:

Individual Students:

Do we see any trends worth noting?  If the overall scores go statistically significantly down in each area test after test that would definitely be an indication that something is amiss (especially if it correlates to grades).  If a specific section goes statistically significantly down test after test, that would be an important sign to pay attention to as well.  Is there a dramatic and unexpected change in any section or overall in this year’s test?

The answers to all of the above would require conversation with teachers, references to prior tests and a thorough investigation of the rest of the data to determine if we have, indeed, discovered something worth knowing and acting upon.

This is why we will be scheduling individual meetings with parents in our school to personally discuss and unpack any test result that comes back with statistically significant changes (either positive or negative) from prior years’ testing or from current assessments.

Additionally, the results themselves are not exactly customer friendly.  There are a lot of numbers and statistics to digest, “stanines” and “percentiles” and whatnot.  It is not easy to read and interpret the results without someone who understands them guiding you.  As the educators, we feel it is our responsibility to be those guides.

Individual Classes:

Needless to say, if an entire class’ scores took a dramatic turn from one test to the next it would be worth paying attention to – especially if history keeps repeating.  To be clear, I do not mean the CLASS AVERAGE.  I do not particularly care how the “class” performs on a standardized test qua “class”.  [Yes, I said “qua” – sometimes I cannot help myself.]  What I mean is, should it be the case that each year in a particular class each student‘s scores go up or down in a statistically significant way – that would be meaningful to know. Because the only metric we concern ourselves with is an individual student’s growth over time – not how s/he compares with the “class”.

That’s what it means to cast a wide net (admissions) while having floors, but no ceilings (education).

School:

If we were to discover that as a school we consistently perform excellently or poorly in any number of subjects, it would present an opportunity to examine our benchmarks, our pedagogy, and our choice in curriculum.  If, for example, as a Lower School we do not score well in Spelling historically, it would force us to consider whether or not we have established the right benchmarks for Spelling, whether or not we teach Spelling appropriately, and/or whether or not we are using the right Spelling curriculum.

Or…if we think that utilizing an innovative learning paradigm is best for teaching and learning then we should, in time, be able to provide evidence from testing that in fact it is.  (It is!)

We eagerly anticipate the results to come and to making full use of them to help each student and teacher continue to grow and improve. We look forward to fruitful conversations.

That’s what it means to be a learning organization.