Tag: transparency

This is it…the last…the very last…

…this version of the Annual Parent Survey will be put to bed.  Faithful readers will recall that last year’s was going to be the last, but October 7th and the relocation/renovation rendered that a bit of a bridge too far.

So, yes, this is it.  This Annual Parent Survey has served me and my school(s) well these last 15 years, but the people have spoken – er, rather, I guess, the people have not spoken, or at least have not been willing/interested in completing this survey in this format and so we will finally bid it adieu…

We have found ourselves in this fascinating cycle where each year the enrollment goes up and the participation rate in the Annual Parent Survey goes down.  This year, although 47 individual surveys were turned in, only 36 individual surveys provided data on the main sections.  That means this survey only represents 19% of the students in our school.   It simply defeats the purpose of gathering feedback in service of making decisions that impact students if 1/5 of students are sharing that feedback.  Whether we move to a third-party vendor, a new format for surveys, focus groups or some combination therein, we will cast a different and a wider net to ensure we truly capture the feedback we need – and your children deserve – to aim closer to our North Stars; to be the best version of ourselves we can.

But that’s the future…for now, one last time, let’s thank and lean into the parents who did participate and try to make meaning of what they are telling us.  [If you would like to see a full comparison with last year, you can reread those results, or have them open so you can toggle back and forth.  In this post, I will try to capture the highlights and identify what trends seem worth paying attention to.]

Not surprising to be clustered so low…it does make questions about “high school readiness” less helpful with such little representation from Grades 7 & 8.

Without knowing how representative this fifth of students is, this year’s data set is lighter on the “no’s”.  Of course the “no’s” are always complicated to unpack because we have no way of knowing who of the “no’s” represent graduation or relocations, as opposed to choosing to attrit prior to Grade 8.  However, what continues to be true is that the overwhelming majority of families – regardless of their feedback – stay with us year-after-year.  This continues to say a lot about them and a lot about us.

Let’s look at the BIG PICTURE:

The first chart gives you the breakdown by category; the second chart gives you the weighted average satisfaction score (out of 10).  I will remind you that for this and all categories, I look at the range between 7-9 as the healthy band, obviously wanting scores to be closer to 9 than to 7, and looking for scores to go up each year.  In terms of “overall satisfaction”, we have now gone from 7.13 to 7.20 to 8.17 to 7.91 to 8.0 to 7.44 to 7.53.  Although it is a tick up from last year, the difference is statistically insignificant.

Overall, this seems to be a good news story, but let’s dig deeper…

Almost every one of these numbers are up from last year!  And the only number that is below the “acceptable range” is trending upwards…

Again everyone one of these is higher than last year!  And, again, the only one that is below the “acceptable range” is also trending upwards…

So far, same trend.  Every number is up and the OVERALL number and the Science number are as about as high we’ve ever had.

Here we hit our first trouble spot.  The less-than-great news is that these numbers, at least for those families who filled it out, are all (still) below the acceptable range.  The not-as-bad news is that numbers are fairly flat.  “French reading” is down pretty significantly and “French PE” in its second year took a dip as well, so there continues to be meaningful work ahead.

Overall these numbers are mostly flat with a few small dips.  Again, anything in those “high sixes” are targets for improvement.  [I’m looking at you Tefillah which I take the most personally since I teach it!]

Work to be done!  Although we think the transition we made in Art and the addition we made to Music (not yet represented in the survey) has brought significant improvements to our program.  There are variables here that are not entirely within our control, but this entire section is worth our thinking more deeply about and identifying a few changes for next year.

These scores are mostly down as well.  Pairing this with comments, we know we have work to do when it comes to the transition to semesters, the way we weave in progress reports, how we approach goal-setting, etc.  We believe we have a clear path forward and fully expect to see these numbers grow next year.  The one score we want to better understand is how parents view “provides regular opportunities for parents to be involved in student learning”.  We do this, or we think we do, so part of what has to be sussed out is whether we are providing the right or preferred opportunities.

These numbers are almost all higher than the prior year.  The two that are below the acceptable range (relevant learning for parents) and (student code of conduct) are both up, if not yet where we prefer them.  Considering how much energy we put into new behavior systems this year, I would liked to see that number jump higher, but we will keep working to improve.

Last data point [Remember this question was scaled 1-5.]:

Our score remains consistent from 4.44 to 4.34 to 4.34 to 4.14 to 3.92.  This one actually doesn’t jive with the rest of the results which almost universally had higher ratings than the prior year.  Could be that this data point, which is supposed to rate Net Promoter Scoring (for those who are familiar) is not well understood.  Either way, the trend line is concerning…at least for the minority of families who are represented in this year’s results.

So there you have it for 2023-2024!

Thanks to all the parents who took the time and care to fill out surveys!  In addition to the multiple choice questions, there were opportunities for open-ended responses and a couple of experimental sections.  Your written responses added an additional layer of depth; one which is difficult to summarize for a post like this.  Please know that all comments will be shared with those they concern.  (This includes a full set of unedited and unredacted results which goes to the Head Support and Evaluation Committee of our Board of Trustees.)

As I said at the beginning, without meaningful data we don’t know how high to put the “floor” we stand upon to reach towards our North Stars.  We will change our model next year to ensure we get better data from more families.  That way, we can make sure that without a ceiling, we aim to reach higher each year…

It is that time of year again…but, perhaps, it may not continue to be “that time of year” – at least in this format – into the future…

As our enrollment continues to increase and our participation in the Annual Parent Survey continues to flatline – our survey has gone from covering 81 students to 84 students to 54 students to 58 students to 52 students and this year, the majority of questions were only answered by families covering 45 students.  That barely captures one quarter of our student population and it is only fair to ask, at this point, whether or not this continues to be the best way to solicit meaningful and actionable data.  Because that is the reason we do this – to learn how to continue to grow as a school.  Without three-quarters of the students accounted for, the data can only be so meaningful.  Without casting any aspersions about who is, or who is not, more likely to fill these surveys out, the odds of this 1/4 mapping onto the opinions of the remaining 3/4 seems long at best.  As the survey is per student, not per family, it runs the risk of being even less representative than that.  (In the service of anonymity, we have no way of knowing how many families the survey actually represents.)

Our goal of 50% seems more and more unrealistic each year.

So, after this year, we are going to have a think.  We could incentivize families to fill this out, as some parents have suggested.  We could consider moving to a model where we do focus groups some years and surveys other years.  Perhaps if we hired a third-party vendor to issue the survey, analyze the data and share the report, people would be more comfortable believing it is as anonymous as it is (it is!), or have more faith in an unbiased deliverable.  Either way, I think it is time to acknowledge that this methodology is no longer serving its intended purpose and the goals of receiving feedback and sharing it transparently likely require a new approach.

But that’s for next year…for now, instead of worrying about the motivations for why families did or didn’t fill out surveys, let’s thank the parents who did participate and try to make meaning of what they are telling us.  [If you would like to see a full comparison with last year, you can reread those results, or have them open so you can toggle back and forth.  In this post, I will try to capture the highlights and identify what trends seem worth paying attention to.]

For a third straight year, we have more spread than normal.  It is more typical to have a big cluster in the youngest grades with diminishing returns as you get older.  Again this year, we have a healthy (if low) distribution across most of our grades.

Without knowing how representative this quarter of students is, this year’s data set is heavier on the “no’s”.  Of course the “no’s” are always complicated to unpack because we have no way of knowing who of the “no’s” represent graduation or relocations, as opposed to choosing to attrit prior to Grade 8.  [If a higher percentage of the small number who attrit are represented in these results than in prior years, it would provide added context for the results.]  However, what continues to be true is that the overwhelming majority of families – regardless of their feedback – stay with us year-after-year.  This continues to say a lot about them and a lot about us.

Let’s look at the BIG PICTURE:

The first chart gives you the breakdown by category; the second chart gives you the weighted average satisfaction score (out of 10).  I will remind you that for this and all categories, I look at the range between 7-9 as the healthy band, obviously wanting scores to be closer to 9 than to 7, and looking for scores to go up each year.  In terms of “overall satisfaction”, we have now gone from 7.13 to 7.20 to 8.17 to 7.91 to 8.0 to 7.44.  Although it is a tick down from last year, the difference is statistically insignificant.

The one thing that jumps out here that will continue throughout all the data is that the “standard deviation” is much higher than in prior years.  Meaning, we have higher concentrations on both ends than normal – which combined with the low engagement probably explains the most about the data above and below.  This 1/4 of the student population could capture more families with the strongest feelings than in a typical year.

However, overall, this continues to be a good news story, but let’s dig deeper…

• The overall theme, which I have been suggesting above and will carry forward is that almost every single category is slightly down.  That is not a trajectory that pleases us, even with all the possible caveats and contexts.  We like numbers to go up; we don’t like when numbers go down.  However, because they are consistent all across the board, it is a bit challenging to identify one issue, program or idea to work on.  I’ll have more to say below.
• It remains true, however, that even with this disappointing trend line, all data points (here) round into the acceptable range.  So we don’t like the way the arrow is facing, but this constitutes a challenge, not a crisis.

• I won’t repeat the same comment each time – each score being slightly down, but mostly in the acceptable range.
• The one thing that jumps out from this, which we know, is that the school has been trying to provide the same level of quality in terms of “resource” with the same staffing structure spread across +40 students.  Meaning, during my six years at OJCS, we have over 40 more students in our school and the same number of personnel allocated to providing resource support – whether that is direct services to students, coaching and support for teachers in making accommodations, or even the customer service of reporting back and forth with parents.  We keep trying to do more with less, and if this data yields anything actionable it is this – even leaving aside potential differences in philosophy; we are not yet adequately staffed to deliver on our promise to parents.

Action to follow.  Watch this space.

• Now this data set is revealing.  This is the first section that does not indicate a meaningful dip.  Despite the concerns above, this same set of parents has given us very high marks for General Studies.
• With a full year in the Makerspace, with having done meaningful PD on the “Science of Reading”, and an overall return to business as usual, it is nice to find all our scores in the acceptable range (especially in a “down year”) in the category that most parents would deem paramount.

• I would like these numbers to be higher, of course, but they are fairly in line with prior years.  One would have hoped that the added emphasis this year – the DELF, the investment in new curricular materials, etc., – would have yielded higher results, but the anonymity makes it hard to know how many students represented in this survey are in Core French, Extended French, etc.  I do know how hard our French Faculty work, so bon travail to the French Department as we continue to raise the bar each year.
• I am very pleased with a baseline 6.79 for the first year of French PE – that rounds into the acceptable range with lots of room to grow.

• We are pleased to see all our Jewish Studies metrics continue to hold strong for another year.   Considering, that we have transitioned away from a “Head of Jewish Studies” model (for now), this is especially encouraging.  Kol ha’kavod to the Jewish Studies Department!
• Last year I said that, “I am taking the slight dip in “Tefillah” as a personal challenge!  It is my favourite subject to teach (students) and to coach (teachers) and I am going to make it my mission to push prayer past 7.0.”  Mission accomplished!
• Last year, I said that, “I am also going to – assuming a return to normal – encourage our community’s rabbis to resume a greater role in Jewish life at OJCS.”  Well, it went up…but we could and will do more.

• Great job Mr. Ebbs for keeping our Art Program moving in the right direction!
• Coming out of COVID, it is nice to see that both Extracurricular and Athletics have ticked up a notch!
• Hot Lunch and After School Programming have ticked down a bit – these are areas where we work with partners and we’ll be carrying these results to them in order to see where we can improve for next year.

• These are mostly wonderful scores, all just about the same and well into the healthy ranges.  We know that we have our teachers and Ellie to thank for a lot of those high scores!
• We will be making meaningful changes next year to our academic calendar – which will include when and how we schedule Parent-Teacher Conferences.  We are excited to share this with you soon and think that will have a positive impact on how parents receive feedback on their child(ren)’s academic progress.  Stay tuned!

• I have already shared my thoughts on my own job performance in my prior “Transparency Files” post.  I will simply state here my numbers, like all the above are largely the same, with the same across-the-board dip.  The one data point that I will be reflecting on is my “responsive/accessibility” – I’d like to see this trend upwards in the year to come.
• The one metric that I am disappointed to see take a dip down after three straight positive years is the last one, which essentially serves as a proxy for school-wide behavior management.  Four years ago we scored a 6.69 and I stated that, “we are working on launching a new, school-wide behavior management system next year based on the “7 Habits” and anchored in our “North Stars”.  I will be surprised if this score doesn’t go up next year.”  Well, three years ago it came in at 7.65, two years it climbed up to 8.19, and it remained high at 7.85 last year.  6.73 puts at back at square one – even if it rounds into the acceptable range, and even with a small sample size.  Parents at OJCS can expect to see significant attention being paid to overall behavior management in 2023-2024.

Last data point [Remember this question was scaled 1-5.]:

Our score remains consistent from 4.44 to 4.34 to 4.34 to 4.14.  Considering the overall results, this is a fairly positive data point, even if the trend line is not what we would prefer.

So there you have it for 2022-2023!

Thanks to all the parents who took the time and care to fill out surveys!  In addition to the multiple choice questions, there were opportunities for open-ended responses and a couple of experimental sections.  Your written responses added an additional layer of depth; one which is difficult to summarize for a post like this.  Please know that all comments will be shared with those they concern.  (This includes a full set of unedited and unredacted results which goes to the Head Support and Evaluation Committee of our Board of Trustees.)

As I said at the beginning, without meaningful data we don’t know how high to put the “floor” we stand upon to reach towards our North Stars.  We will likely look to change our feedback loop to ensure we hear more from more families.  That way, we can make sure that without a ceiling, we aim to reach higher each year…

Looking out my office window at the sunny skies [when I first wrote this on Tuesday afternoon!], is both a reminder of the first stirrings of normalcy and what we hope next year and beyond will bring… as atypical as this third year of COVID has been, we do find comfort in familiar habits and experiences.  And so if it is May, it must be time to share the results of this year’s Annual Parent Survey.  If you would like to see a full comparison with last year, you can reread those results or have them open so you can toggle back and forth.  In this post, I will try to capture the highlights and identify what trends seem worth paying attention to.

The first thing to name, which does not come as a tremendous surprise considering the times we are living through, is that we continue to have a less-than representation.   In fact, it seems that the more we grow, the percentage of students represented by the survey decreases.  Our enrollment has grown each year that I have been here, but our survey has gone from covering 81 students to 84 students to 54 students to 58 students to 52 students.  That means that this year’s survey represents barely more than a quarter of our student population!  As the survey is per student, not per family, it runs the risk of being even less representative than that.  (In the service of anonymity, we have no way of knowing how many families the survey actually represents.)  Our goal of 50% seems more and more unrealistic each year.  [If you have feedback on what might incentivize greater participation, please drop it in the comments or email it to us directly.]

I simply no longer know if or how to draw meaningful conclusions about participation rates.  Whereas it is common wisdom that folks with concerns are usually more likely to fill out these surveys, there is no common wisdom when it comes to pandemic times.  So for what we hope is one final year, instead of worrying about the motivations for why families did or didn’t fill out surveys, let’s celebrate the parents who did participate and try to make meaning of what they are telling us.

For the second consecutive year, we have more spread than normal.  It is more typical to have a big cluster in the youngest grades with diminishing returns as you get older.  Again this year, we have a healthy distribution across most of our grades.

Without knowing how representative this quarter of students is, this data for sure lines up with what is true – that we have, again, had a fast and successful re-registration.  The percentage who replied “yes” is up and the “noes” are always complicated to unpack because we have no way of knowing who of the “noes” represent graduations or relocations, as opposed to choosing to attrit prior to Grade 8).  What continues to be true is that the overwhelming majority of families – regardless of their feedback – stay with us year-after-year.  This continues to say a lot about them and a lot about us.

Let’s look at the BIG PICTURE:

The first chart gives you the weighted average satisfaction score (out of 10); the second chart gives you the breakdown by category.  I will remind you that for this and all categories, I look at the range between 7-9 as the healthy band, obviously wanting scores to be closer to 9 than to 7, and looking for scores to go up each year.  In terms of “overall satisfaction”, we have now gone from 7.13 to 7.20 to 8.17 to 7.91 to 8.0.  Although it is just a tick up from last year, the difference is statistically insignificant.  This is just the second time that no families graded the school a 1, 2 or 3.  Of course, we always want to see numbers continue to go up, but based on how we survey it is hard to get much higher.

This continues to be a good news story, but let’s dig deeper…

• The topline number – probably the most important – like our overall satisfaction is slightly up 7.91 to 7.93 and a very positive outcome.
• I am very pleased to see that every single category is up from last year’s all-time highs and that each score is well within the healthy range!
• I am thrilled to see that relationships with faculty again comes in with the highest score (8.65) in this block, especially when you factor in all the challenges the of pandemic have created.  Kudos to our teachers!
• Our lowest score (again) is again in “Homework” but it does continues to climb from 6.56 to 6.91 to 7.0 to 7.31.  Progress has been steady, and we are seeing steady improvement in the full implementation of our new Homework Philosophy.
• I am thrilled to see such a high score (8.41) for “creative and critical thinking skills”…that is very much #TheOJCSWay.

• After having seen steady growth on the topline number, which again is so critical to our school, it is a bit disappointing to see a drop.  We have gone from 6.61 to 6.97 to 7.58 to 7.15.  It remains well within the healthy range, but we will be looking to get back on the upward track next year.
• The metrics for Spec Ed are a bit of mixed bag with the communication score holding steady, but the satisfaction score for those who have IEPs dipping just a bit.  The numbers remain strong and of all the things to suffer during the pandemic, it is not surprising to see it impact our most vulnerable students.  Kudos to Sharon Reichstein, our Director of Special Needs Education, and her new team for all their work this year!

• Thrilled to see that our topline number continues to remain (essentially) at 8!
• Very happy to see that every metric in General Studies is well into the healthy band and each one is essentially unchanged.
  • Math: 7.09 to 7.60 to 7.67 to 7.15.
  • Science: 7.09 to 7.72 to 7.61 to 8.37.
  • Social Studies: 7.41 to 7.96 to 7.95 to 7.86.
  • Reading: 6.93 to 8.0 to 7.85 to 8.29.
  • Writing: 6.51 to 7.07 t0 7.41 to 7.95
• The biggest movement this year, which I am thrilled to see is “Science”, where we have invested precious bandwidth in Hackathons and Innovation Day and reopening the OJCS Makerspace, and in “Reading” where we have, not coincidentally, a number of teachers focusing on their professional growth.  This is a clear example over time where parent voice, aligned with teacher and student voice, leads to meaningful action.  (Fill out those surveys y’all!  We really do pay attention.)
• I am also very pleased to see “Writing”, like “Reading” continuing on a strong upward trajectory over the last four years.

• I am pleased to report that despite another year of COVID functioning and the continued integration, that the quarter of students represented in this survey are reporting steady numbers for French outcomes.
• We would like to believe that the result of our TACLEF consultancy is continuing to pay dividends and that our recent announcement of expanding our French program to incorporate French-language physical education will help these numbers continue to tick up in the years ahead.  Bon travail to the French Department!

• We are again thrilled to see all our Jewish Studies metrics continue to hold strong for another year.  We are especially pleased to see the OVERALL metric essentially hold steady from 7.29 to 8.08 to 7.91 to 7.90.  Considering, that we again went forward without filling the “Dean of Judaics” position and all the additional COVID-related challenges, this is especially encouraging.  Kol ha’kavod to the Jewish Studies Department!
• I am taking the slight dip in “Tefillah” as a personal challenge!  It is my favourite subject to teach (students) and to coach (teachers) and I am going to make it my mission to push prayer past 7.0.
• I am also going to – assuming a return to normal – encourage our community’s rabbis to resume a greater role in Jewish life at OJCS.

• Considering the circumstances, it is both surprising and positive that with all the protocols that were in place that we’d see growth in both Art and PE – both of which have reentered the healthy band.  We know that even with a rigorous, trilingual curriculum, that we need to continue to offer the kinds of high-quality PE/Drama/Art experiences that make a well-rounded education.  We said last year to “look for these numbers to go back up next year”.  Mission accomplished!
• It is worth noting that even though none of our extracurriculars, athletics, hot lunch, etc., programming has reached pre-COVID numbers (understandably) they are all up from last year…with lots of room to grow.

From this year’s experimental section, we yield these two data points (and two sets of meaningful commentary).  Compared to last year, there is a larger cluster in “very satisfied” and “extremely important” – which is likely not a coincidence.  As we cannot predict the future, even with wholehearted hope of a return to year-round, in-person learning next year, our ability to navigate situations like these last few years with minimal disruption and maximal academic progress – not to mention the continuance of meaningful Jewish experiences – will likely continue to be powerful value-adds for OJCS in the years ahead.

• These are mostly wonderful scores, all just about the same and well into the healthy ranges.  We know that we have Ellie to thank for a lot of those high scores!
• I am sadly saying again this year, that, “[a]fter having to take a COVID pause, I will be interested to see what the impact of ‘Student-Led Conferences’ will be on the ‘parent-teacher conferences’ metric once finally launched.

• I have already shared my thoughts on my own job performance in my prior “Transparency Files” post.  I will simply state here my pleasure in seeing my numbers holding strong, with the weakest one – providing learning opportunities for parents and caregivers with some health post-COVID room to grow.
• The one metric that I am very pleased to see holding strong is the last one, which essentially serves as a proxy for school-wide behavior management.  Three years ago we scored a 6.69 and I stated that, “we are working on launching a new, school-wide behavior management system next year based on the “7 Habits” and anchored in our “North Stars”.  I will be surprised if this score doesn’t go up next year.”  Well, two years ago it came in at 7.65, last year it climbed up to 8.19, and it remains high at 7.85 this year.

Last data point:

Remember this question was scaled 1-5.   Our score remains consistent from 4.44 to 4.34 to 4.34 (again).  I have said that I truthfully don’t know how much more there reasonably is to grow here, but we’ll keep doing our best to find out!

So there you have it for 2021-2022!

Thanks to all the parents who took the time and care to fill out surveys!  In addition to the multiple choice questions, there were opportunities for open-ended responses and a couple of experimental sections.  Your written responses added an additional layer of depth; one which is difficult to summarize for a post like this.  Please know that all comments will be shared with those they concern.  (This includes a full set of unedited and unredacted results which goes to the Head Support and Evaluation Committee of our Board of Trustees.)  As you can see, we really do use this data to make enhancements and improvements each year.

We very much wish to continue into next year, this year’s trend in maintaining and increasing positive outcomes and satisfaction.  To mix school metaphors, each year simply becomes the higher “floor” we stand upon to reach towards our North Stars.  With no ceiling, we aim to reach a little closer each time.