Health Department documents obtained by Cosmos through FoI shed light on the effectiveness of COVID school measures, while also raising questions.
After two years in and out of lockdowns and 35 weeks of remote learning, Victorian schools began 2022 with some of the most stringent COVID-19 measures in the country.
Mandatory masks for Grade 3 and above, twice-weekly RAT screening, mandatory vaccines for teachers, air purifiers and operational measures – were announced by Premier Daniel Andrews and then Education Minister James Merlino on 23 January 2022 and designed to keep “education open and safe”.
Experts say compliance with COVID-19 measures depends on public trust and engagement, which they say can be supported by governments sharing information and evidence about the effectiveness of those measures, as well as risks and downsides.
But how effective were Victoria’s school measures? Given their extent, any data from the state would have been critical to the ongoing management of COVID-19 for children. The health department not only collected comprehensive data, but analysed the results. They never released their findings.
Using Freedom of Information laws, Cosmos has obtained documents which shed light on whether those efforts paid off. Cosmos believes it’s important to make the raw data available, so we’ve posted it below.
Masks
Victoria was the only jurisdiction to mandate masks for primary school children at the start of the 2022 school year. Masks were required indoors for all students Grade 3 and above. In other states, masks were encouraged or recommended for children under 12, but not required.
“Victoria has always been the state with the strictest public health rules around COVID, and I don’t think the educational rules are any exception,” says Professor Julie Leask, from the Susan Wakil School of Nursing and Midwifery at the University of Sydney, specialising in vaccination and the prevention of infectious diseases.
An ‘Evidence Review’ of mask effectiveness dated August 2022 by Victoria’s Health Department, released under Freedom of Information laws, states the removal of masks in secondary schools increased the risk of COVID-19 in those students by approximately 23%.
The 23% figure is based on departmental modelling tracking the divergence in cases after the mask mandate was removed for high school students from 25 February, but remained in place for primary school children in Grades 3–6.
The blue line is ages 8-11. The red line is ages 13-17.
The modelling is detailed in emails dated 8 and 14 April 2022 from the Department’s Director – Modelling and Analytics, also released under FoI. It says bias could be introduced “if primary school students ‘play’ closer together than secondary school students […] the model may over- or underestimate the effects of masks”.
Epidemiologist Professor Catherine Bennett describes the analysis as comparing “apples with lemons”.
“Bizarrely, they’ve put two lines on the same graph, where one is an age cohort that includes five years of age [secondary children aged 13 – 17], and the other is an age cohort that includes people of four different years of age [primary children aged 8 – 11],” the Chair of Epidemiology at Deakin University says. Not every calendar age has the same population, but the 8–11 age bracket would have had fewer children.
Cosmos asked the Department of Health why it chose to compare those two groups and whether it had updated its mask analysis since, but did not receive a response.
Bennett says the analysis appears to lack sophisticated understanding about case reporting, the timing of waves and other factors likely to influence case rates over time.
Even so, she questions why the department did not share the information with the public at the time, which would result in “both explaining to people why you’re asking them to do something, and giving them the respect and the knowledge that that you’ve gained internally to say, we’re backing up what we’re doing, this is why we’re doing it,” she says.
At the beginning of Term 2, 2022 Victoria removed its remaining mask mandate for Grades 3 – 6. At the beginning of Term 3, the Department of Education, Independent Schools Victoria and the Catholic Education Commission sent a letter asking children to return to wearing masks until the end of winter. The letter didn’t give any data, but pointed to the Health Department’s strong recommendation that face masks be worn indoors.
A survey of 7,889 parents and guardians conducted from 19–24 July 2022 and released under FoI reveals that 74% of parents did not ask their child to wear a mask at school. More than half of that group cited reasons including: masks interfered with learning (65%), were uncomfortable (61%), my child doesn’t want to wear one (55%), and masks don’t help in stopping the spread of COVID (52%).
Bennett says sharing data on the effectiveness of and compliance with public health measures can form part of an important feedback loop with the general public, helping to keep them engaged. In this case, “a lot of kids’ education, and parents’ sanity, and teachers’ anxiety was at stake here.”
So, were mandatory masks worth it?
Leask is ambivalent. “Twenty-three percent is not to be sneezed at, but it’s not 100%. And a lot of transmission is still happening despite the mask mandate, as we see in the early phase … I don’t see any sort of systematic assessment of the educational impacts on children, for example. A teacher wearing a mask, can they be heard? The loss of facial expression is a problem for younger children …
“Those sorts of downsides don’t appear to be systematically measured from the data.”
RAT Screening
Twice-weekly COVID-19 rapid antigen tests (RATs) were strongly recommended for primary and secondary school students in Victoria and New South Wales from the start of Term 1, 2022. While the NSW Government dropped the requirement from 28 February, Victoria continued its surveillance program and provision of free RATs into Term 2.
The FoI documents reveal the Victorian Department of Health collected data on the uptake, effectiveness and acceptability of RATs, including weekly COVID-19 cases in schools and surveys of staff and parent attitudes.
A report dated 28 March 2022 shows the Victorian Government distributed more than 13 million RATs across primary, secondary and specialist schools and early learning facilities to support the surveillance program.
According to a document titled ‘DH data. Case Notifications & Trends’ dated 9 May 2022 “School-aged children remained a cohort with a higher case rate than the adult population in term 1”.
The document says considerations include: differing vaccination rates, the removal of face masks and “case detection relative to the general population in children may be elevated during school term time with the return to school and the RAT screening program”.
Case rate (per 10,000 population) by age group. Graph shows daily diagnoses for four age groups. The red line is age 0 – 4, the green line is age 5 – 11, the blue line is age 12 – 18, the purple line is age 19+
Cosmos asked the Department of Health how significant the RAT screening was in children’s elevated share of COVID-19 cases, but did not receive a response.
The department’s media team did send a link to an April 2022 press release estimating the RAT screening program prevented 113,500 COVID-19 infections. The number is not referenced in the FoI documents and no further information is provided on the assumptions or modelling behind the figure.
Bennett says infection rates are influenced by a range of factors, such as background rates in the community, the vaccine rollout, timing of waves and surveillance testing. She says it’s likely the elevated case rates reflect the impact of the twice-weekly RAT screening program for children, which didn’t apply to the broader population.
Bennett doesn’t think the data shows any “major effects of kids going back to school with or without masks”, adding that serosurveys show little difference in infection rates for school aged children in Victoria compared to other states.
Health department surveys of parents, teachers and principals, also released under FoI, show a majority of primary and secondary school families complied with the twice-weekly RAT testing requirement. However, compliance declined over time.
Read more:
- Only marginal benefit for wearing masks during big COVID surges, finds study
- Clean air in schools
- How do Rapid Antigen Tests work, and are we using them all wrong?
- COVID-19 led to largest learning disruption in history
Were the measures worth it?
When it comes to COVID-19, 2022 can sometimes feel a bit like ancient history.
But experts say there is value in analysing the effectiveness of measures, particularly in a state with such a stringent approach.
Leask says a multitude of institutions and individuals could benefit from information on the effectiveness of public health measures. “Transparency promotes trust. So, it’s good for government,” she says.
“Trust is the bedrock on which we get good public cooperation with public health measures, because they’re well justified and well supported by evidence.”
Bennett agrees.
“What’s interesting here is that this work was going on, but we didn’t know about it,” she says.
She says the documents released under FoI contain the sort of data and analysis needed to evaluate the effectiveness of COVID-19 measures, an important part of evidence-based policy.
“But it was not public. And it’s not as complete as it could be,” she says.
For instance, Bennett would like to see further analysis, on the impact of vaccination on infection rates drawing on data from positive COVID-19 case reports.
Leask also has concerns about the data.
She says the approach by the Health Department appears to be lacking any systematic assessment of the educational impacts on children or the downsides of those school-based interventions.
“Whenever there is a policy implemented in schools, which has such a wide, broad impact on children, [governments] need to be very careful and purposeful about how they consider the risks of that policy along with its benefits.”
Leask says it can often be easier to quantify the benefits of restrictive measures, but much harder to measure the social and educational risks felt further down the track, like consequences for mental health or the educational progress of children.
The question remains: did Victoria’s strict measures lead to a better result?
“You can’t say overall, that the combined effort – which was more demanding on students, parents and schools than other states – was actually an effective intervention,” says Bennett. “That’s what you’ve got to ask. That’s how you learn.”