The way forward for LC2021

rainbowtrout · 09-02-2021 12:15AM

deiseindublin wrote: »

It didn't apply to individuals a_z , it was schools that were ranked as such , or so we were told.

I'm not convinced of that. Seeing a student in my school get downgraded in a subject 87 down to 79, and the three students lower than her in the H2 category were untouched. It means ranking didn't count for anything. If downgrading was applied to the school then all of the grades would shift and the lowest one on 81-82 would get the boot into the H3s.

There were posts on here talking about students being moved from H1 to H3 and all the H2s inbetween left untouched.

History Queen · 09-02-2021 12:28AM

Posted this in the other thread by accident, meant to post here.

https://twitter.com/GaryGannonTD/status/1358741533323890689?s=19

am_zarathustra · 09-02-2021 12:40AM

This exactly, even from that 205 page report theire is a lot of over explained statistics saying very little but again, no code and no clear indication of weighting. Same as you rainbowtrout, we had kids leap frogged. The way an algorithm works should be logically. They didn't even test it properly to ensure outliers gave the correct results, otherwise they would have picked up the CSPE issue. It's programming 101. I don't honestly believe anyone in the Dept knows what was done

Maybe it's nothing, maybe there are reasonable reasons I can't see but if they published it no one would need to wonder

Treppen · 10-02-2021 12:16PM

deiseindublin wrote: »

It didn't apply to individuals a_z , it was schools that were ranked as such , or so we were told.

No, junior cert results were taken into account. I put up a link last year (in October I think) from a report to the minister on what options/models were available and the advised one was where the JC results would be used. Take 3 Core + 2 of the best others.

I can't find the report but here's a mention from the Irish Times

https://www.irishtimes.com/news/education/leaving-cert-results-all-you-need-to-know-about-calculated-grades-1.4345625

As part of the “standardisation” process – in which grades are adjusted up or down for consistency – it also took into account the likely performance of your class based on how students fared in the Junior Cert, along with the historic pattern of Leaving Cert results nationally.

Treppen · 10-02-2021 12:26PM

horay.. found it..

These are the technical reports published after the grading (7th September) they are not the algorithms.

https://www.gov.ie/en/publication/2ed9b-leaving-certificate-2020-calculated-grades-technical-reports/

86709_aacaf08c-a32b-4e74-817b-5a3ba1e2a7e8.pdf

From Section 5:

"With the recent decision to remove the school historical data from consideration and to have
a greatly diminished role for the national standards, the information sources remaining
available for use are the school estimates and rank orders, and the prior attainment and
related variables of the 2020 Leaving Certificate students from when they sat their Junior
Cycle."

In the appendix E what they went with seems to be a composite of Irish, English, Maths + 2 other strongest subjects in the JC....Pg. 91

". To address this, prior performance could be combined into a single composite score.
However, the advice of the psychometricians is that the greatest gain would be achieved by using a
subset of the JC subjects in the model. These would be Irish, English and mathematics (taken by the
very great majority of candidates) and the best two of each candidate’s other subjects. The advice is
that a single composite will be less valuable than a relatively small number of predictors, such as the
set suggested. Taking more subjects would only add very slightly to the predictive power of
candidates’ prior performance and would begin to reintroduce the problem of missing data as more
(less often taken) subjects were added."

It would also appear that they tried to push certain school class groups grades towards the national average too, in order to get their bell curve in line.

Wombatman · 10-02-2021 12:27PM

Treppen wrote: »

No, junior cert results were taken into account. I put up a link last year (in October I think) from a report to the minister on what options/models were available and the advised one was where the JC results would be used. Take 3 Core + 2 of the best others.

I can't find the report but here's a mention from the Irish Times

https://www.irishtimes.com/news/education/leaving-cert-results-all-you-need-to-know-about-calculated-grades-1.4345625

I think the point is it wasn't done on a pupil by pupil level. It took the class as a whole and adjusted on the basis of their past performance.

Treppen · 10-02-2021 12:34PM

am_zarathustra wrote: »

This exactly, even from that 205 page report theire is a lot of over explained statistics saying very little but again, no code and no clear indication of weighting. Same as you rainbowtrout, we had kids leap frogged. The way an algorithm works should be logically. They didn't even test it properly to ensure outliers gave the correct results, otherwise they would have picked up the CSPE issue. It's programming 101. I don't honestly believe anyone in the Dept knows what was done

Maybe it's nothing, maybe there are reasonable reasons I can't see but if they published it no one would need to wonder

I know it's terrible but when I found out they included CSPE I thought it was hillarious. I knew of 2 CSPE teachers, one who kept on beating the drum for CSPE saying it was the most important subject ever and demanded from their Students that they get top marks in the JC, and the other .... not so much (he said some students didn't bother turning up for the exam or failed on purpose and wrote 'funny answers').

Also the fact that a previous union leader (who was head of the CSPE subject association) pushed for CSPE to be kept alive for as long as possible.

So when students ask you in 1st year "Will this go on our permanent record?"... the answer is always "oh you better believe it".

Treppen · 10-02-2021 12:39PM

Wombatman wrote: »

I think the point is it wasn't done on a pupil by pupil level. It took the class as a whole and adjusted on the basis of their past performance.

I'm not so sure though, I had assumed they did use each student's individual JC results as a composite which influenced their individual results in the LC ... as well as all the other criteria. I'll have a read again , maybe it's purposefully ambiguous and we're getting different things from it.

rainbowtrout · 10-02-2021 12:44PM

Treppen wrote: »

No, junior cert results were taken into account. I put up a link last year (in October I think) from a report to the minister on what options/models were available and the advised one was where the JC results would be used. Take 3 Core + 2 of the best others.

I can't find the report but here's a mention from the Irish Times

https://www.irishtimes.com/news/education/leaving-cert-results-all-you-need-to-know-about-calculated-grades-1.4345625

Yes, wasn't there an issue where it took their worst JC results into account instead of their best ones or something along those lines? Or it counted CSPE when it shouldn't have? I can't remember the exact detail.

rainbowtrout · 10-02-2021 12:46PM

Wombatman wrote: »

I think the point is it wasn't done on a pupil by pupil level. It took the class as a whole and adjusted on the basis of their past performance.

But if it did that, then a student who was the highest score in a range of H2 grades shouldn't be the one that's downgraded to a H3, it should be the lowest scoring H2 as all grades are shifted downwards.

It also wouldn't explain the St. Killian's debacle as I'd assume their JC German grades are pretty much in line with their LC German grades.

Treppen · 10-02-2021 12:51PM

rainbowtrout wrote: »

Yes, wasn't there an issue where it took their worst JC results into account instead of their best ones or something along those lines? Or it counted CSPE when it shouldn't have? I can't remember the exact detail.

I think it was both issues. Worst result + CSPE result.

IIRC they said the worst result scenario only affected a very tiny cohort (as, I assume, they would have been fail grades ,,, which in turn would only affect students who mightn't being doing so hot in the LC anyway,,, but been bumped up some other way).

deiseindublin · 10-02-2021 12:56PM

Wonder when we will actually hear for sure on LC2021? Can't see anything for midterm, I'd say we'll be lucky to have something solid for Friday 19th, just before schooling restarts after midterm.

Wombatman · 10-02-2021 02:32PM

rainbowtrout wrote: »

But if it did that, then a student who was the highest score in a range of H2 grades shouldn't be the one that's downgraded to a H3, it should be the lowest scoring H2 as all grades are shifted downwards.

It also wouldn't explain the St. Killian's debacle as I'd assume their JC German grades are pretty much in line with their LC German grades.

The likes of St. Kilian's are an unfortunate outlier. Grind schools usually do well across a range of subjects whereas St. Killian's excelled at a single subject, German.

Their results would mimic a teacher awarding ridiculously high grades and the component of the model designed to deal with these spikes unfortunately flattened a legitimate spike in the case of St. Kilian's.

Smacruairi · 10-02-2021 03:28PM

Wombatman wrote: »

The likes of St. Kilian's are an unfortunate outlier. Grind schools usually do well across a range of subjects whereas St. Killian's excelled at a single subject, German.

Their results would mimic a teacher awarding ridiculously high grades and the component of the model designed to deal with these spikes unfortunately flattened a legitimate spike in the case of St. Kilian's.

Kilians would do extremely well most years. Their whole student population should have had an above average spike across the board. The fact that the algorithm couldn't account for that means it really isn't up to scratch, sure was it 5errors in the code they spotted?

Wombatman · 10-02-2021 04:50PM

Smacruairi wrote: »

Kilians would do extremely well most years. Their whole student population should have had an above average spike across the board. The fact that the algorithm couldn't account for that means it really isn't up to scratch, sure was it 5errors in the code they spotted?

The model would have been as accurate as you are going to get for a generalisation I expect. It's a pity it wasn't implemented correctly. Good chance Kilians fell foul of the main error, including the two weakest with the three core instead of the two strongest. Hopefully their grades were corrected when they were run back through the patched software.

Student’s individual Junior Cycle results were not used to determine their Calculated Grades
Your Junior Cycle exam results, or whether you sat the Junior Cycle examinations at all, were not used to determine your own specific Calculated Grades. Instead, the overall, or aggregate, Junior Cycle results for class groupings were used, as this aggregate data provides a good means of determining the performance of a class group in the Leaving Certificate.

https://www.gov.ie/en/publication/ad9de-leaving-certificate-2020-your-questions-answered-september-2020/

On 30 September, the Department of Education and Skills announced that two errors had been identified in the Leaving Certificate 2020 Calculated Grades process.

The Department then contracted Educational Testing Services (ETS), to conduct a review of the code for the system of Calculated Grades. In their review, ETS raised two issued, including a third error which they had identified regarding the code.

All three errors concerned how the coding for the system of Calculated Grades drew in and handled Junior Cycle data.

The first error was in a single line of code programmed by the Department’s external consultant, Polymetrika International Inc. That error affected the way in which students’ Junior Cycle results were included in the national standardisation process.

It was intended that students’ Junior Cycle results in Irish, English and Maths would be included in the data used by the national standardisation process, together with their best two other subjects. These data were to be considered at an aggregated class level.

The error meant that students’ results in Irish, English and Maths were put together with their weakest two other subjects for determining the overall performance of their class.

The second error related to the inclusion of candidates’ Junior Cycle results in the subject Civic, Social and Political Education (CSPE) as part of the data used in the national standardisation process. This was incorrect, as in line with the design of the Calculated Grades process, this subject should not have been included in the data used by the model.

The third error, identified by ETS, relates to where students did not sit all three core subjects (Irish, English and Maths) at Junior Cycle level. In those cases, the system was meant to use the average national Junior Cycle score of the group of students who took their Leaving Certificate in 2017, 2018, 2019 and 2020. Instead, it chose the student’s next best subject for inclusion in the group computation.

These three errors have now been corrected and the full set of student data has been re-run in the corrected model.

Mardy Bum · 10-02-2021 05:09PM

Smacruairi wrote: »

Kilians would do extremely well most years. Their whole student population should have had an above average spike across the board. The fact that the algorithm couldn't account for that means it really isn't up to scratch, sure was it 5errors in the code they spotted?

It did seem to take account of this though. The issue was that other schools which typically don't do as well received more points which drove up the points for certain courses which led some to miss out. It came out in the High Court that Blackrock did better in 2020 overall when compared with 2019. The student involved still missed out on their course. Certain schools over inflated their grades and were repaid because of the use of a national averages rather than historical school specific data.

This then comes back to the teacher as professional or advocate in the system. You could not be both but plenty were the latter.

History Queen · 10-02-2021 08:52PM

TUI hoping for LC and special class return week of Feb22nd (subject to public health advice)
https://www.tui.ie/news/tui-members-to-facilitate-re-opening-of-special-classes-in-post-primary-schools-%e2%80%93-union-also-indicates-willingness-to-facilitate-return-of-leaving-cert-students.14037.html

km79 · 10-02-2021 09:27PM

History Queen wrote: »

TUI hoping for LC and special class return week of Feb22nd (subject to public health advice)
https://www.tui.ie/news/tui-members-to-facilitate-re-opening-of-special-classes-in-post-primary-schools-%e2%80%93-union-also-indicates-willingness-to-facilitate-return-of-leaving-cert-students.14037.html

As part of their plan the s and s scheme is suspended ?

Bobtheman · 10-02-2021 09:43PM

According to news LC will be back week of Feb 22nd subject to Asti giving the green light.
Nothing about childcare issue if primary schools not open?

History Queen · 10-02-2021 09:47PM

km79 wrote: »

As part of their plan the s and s scheme is suspended ?

I assume to prevent extra staff having to travel to school? Not sure how it'll work if someone who is meant to have LC is sick?

km79 · 10-02-2021 09:48PM

History Queen wrote: »

I assume to prevent extra staff having to travel to school? Not sure how it'll work if someone who is meant to have LC is sick?

Exactly

History Queen · 10-02-2021 09:51PM

km79 wrote: »

Exactly

Strange that it isn't set out in the guidance. Speaking to two LC students I know in two different schools (not my students) they are delighted they're going back but annoyed an announcement about exams still hasn't been made. I was saying to them that it looksgood re going back but wait for official announcement.

Bobtheman · 10-02-2021 09:58PM

So how do people see this working out?

History Queen · 10-02-2021 10:05PM

ASTI onboard also for phased return anyway. Brilliant. Hopefully whatever issues need to be ironed out are sorted relatively quickly.

https://twitter.com/Jessjcasey/status/1359608811116445697?s=19

Treppen · 10-02-2021 10:07PM

Bobtheman wrote: »

So how do people see this working out?

Don't know if this Feb return is real or just more kiteflying.
As usual no other details forthcoming.

PoolDude · 10-02-2021 10:10PM

Bobtheman wrote: »

So how do people see this working out?

TUI may know public health advice will be to stay off longer so now can say they were prepared to go back - bargaining chip for something else or am I too cynical? It’s just that all indicators were they wouldn’t advise schools reopening that soon and in the LC instance you have the age group with the highest transmission rates (LC) going in to meet the age group with the highest infection rate (people in the 35 - 45 age group - last Saturday’s stats was 59% from that age group).

I’d be happy if they do go back if it’s safe to do so.

Edit - I see ASTI saying plans are inadequate beyond special classes and they will first monitor that in the “weeks ahead” but will negotiate with the department

km79 · 10-02-2021 10:15PM

PoolDude wrote: »

TUI may know public health advice will be to stay off longer so now can say they were prepared to go back - bargaining chip for something else or am I too cynical? It’s just that all indicators were they wouldn’t advise schools reopening that soon and in the LC instance you have the age group with the highest transmission rates (LC) going in to meet the age group with the highest infection rate (people in the 35 - 45 age group - last Saturday’s stats was 59% from that age group).

I’d be happy if they do go back if it’s safe to do so.

Edit - I see ASTI saying plans are inadequate beyond special classes and they will first monitor that in the “weeks ahead” but will negotiate with the department

Well the whole plan falls down on the whole no s and s thing for a start !

Bobtheman · 10-02-2021 10:19PM

It has to be flexible. Childcare is still an issue.
If you have primary kids and they are not open (primary schools) when sixth years are back...

km79 · 10-02-2021 10:26PM

Bobtheman wrote: »

It has to be flexible. Childcare is still an issue.
If you have primary kids and they are not open (primary schools) when sixth years are back...

How have other essential workers managed ?
This will hold no water with many tbh
We have oft argued ourselves against the use of schools as childcare ......

Bobtheman · 10-02-2021 10:40PM

km79 wrote: »

How have other essential workers managed ?
This will hold no water with many tbh
We have oft argued ourselves against the use of schools as childcare ......

Good point

The way forward for LC2021

Comments