Coding Horror

Deleted User wrote: »

Begs the question was there any testing at all. Seems trivial

An even better question is how did 6,500 students grades only get affected. Code can only do things you tell it to do. It does not make mistakes people do.

iLikeWaffles wrote: »

An even better question is how did 6,500 students grades only get affected. Code can only do things you tell it to do. It does not make mistakes people do.

Most likely down to the impact of the standardization shift not resulting in an overall grade change.

tobsey

Wombatman wrote: »

Most likely down to the impact of the standardization shift not resulting in an overall grade change.

If they’ve determined that already they’ve known about this bug for a while.

tobsey wrote: »

If they’ve determined that already they’ve known about this bug for a while.

Yep and we're talking months here probably, Realistically 1 month not 1 week. 1 week is bad enough as there might have been time for some to get a course they wanted. How does it actually look now; teachers being advised to destroy students records of how they calculated their grades, that is.

Just 1 of the many farcical things this current government has been the cause of. Absolute disgrace, not only has it thrown students back 1 year to get the education they desire for the good of their country, students coming up 1 year and or 2-3 years from now will have limited spaces available for what they want to study. Also it doesn't bode well for future plans of calculating leaving cert grades and tossing the written exam altogether.

tobsey wrote: »

If they’ve determined that already they’ve known about this bug for a while.

Not necessarily. Standardization will only cause a grade shift for students with marks on the outer edges. This error only impacted on a small part of the overall standardization formula. Many students may not have tripped over to a new grade because of it's influence.

Once the error was identified and corrected, all they needed to do was run the data through again and compare the result sets to see where grades differed.

zephyro

iLikeWaffles wrote: »

Would be way more than 1 line of code!

It would have had to find the minimum grade and the second second minimum grade. These functions are separate lines of code! Further more the minimum grade would likely be stored somewhere removed from the array of grades in order to find the second minimum.

Reasonable and likely speculation would be that the company responsible were given the wrong instructions from the DoE. As in: combine the 2 minimum grades instead of the 2 maximum non-core subjects!

tbf most likely is that one function returns a sorted array of non-core grades (correctly), then the famous "one line of code" simply takes 2 from the wrong end! easy enough mistake to make when coding but even debugging should catch it never mind the most basic of test plans, makes you wonder what other creepy-crawlies are lurking in there! :eek:

zephyro wrote: »

tbf most likely is that one function returns a sorted array of non-core grades (correctly), then the famous "one line of code" simply takes 2 from the wrong end! easy enough mistake to make when coding but even debugging should catch it never mind the most basic of test plans, makes you wonder what other creepy-crawlies are lurking in there! :eek:

There would need to be an assignment of the 2 variables so even if it is a sorted array these things are separate lines of code i.e.

grade1 = marks[array.length()];
grade2 = marks[array.length() - 1];

as opposed to

grade1 = marks[0];
grade2 = marks[1];

depending on what way it is sorted of course

I'm sure anyone here could write has written code as an assignment for college that would be able to get grades correctly, FFS even in upskill courses paid for by the Government. Why it was outsourced for stupid money in the first place is another question that makes matters worse!

zephyro

iLikeWaffles wrote: »

There would need to be an assignment of the 2 variables so even if it is a sorted array these things are separate lines of code i.e.

grade1 = marks[array.length()];
grade2 = marks[array.length() - 1];

language and variable-dependent of course but these 2 lines would fit the story:

grades = sum(marks[2:])
vs.
grades = sum(marks[:2])

(excuse the magic number!!

)

zephyro wrote: »

language and variable-dependent of course but these 2 lines would fit the story:

grades = sum(marks[2:])
vs.
grades = sum(marks[:2])

(excuse the magic number!! )

Now that would be horrendous!!

Seth Brundle

Deleted User wrote: »

Begs the question was there any testing at all. Seems trivial

Of course there was.
It's just that it was all done in the production environment

GarIT

Seth Brundle wrote: »

Of course there was.
It's just that it was all done in the production environment

Production environment? Are you mad. The first time the code ran using the debugger in Eclipse they sent the output to whoever was expecting it.

GreeBo

iLikeWaffles wrote: »

Would be way more than 1 line of code!

Maybe it was written in Perl? :pac:

[Deleted User]

GreeBo wrote: »

Maybe it was written in Perl? :pac:

Polymetrika appears to be a one man show who uses .Net and Excel.

If you ignore the invalid certificate:
https://polymetrika.com/Downloads/IATA
https://polymetrika.com/Downloads/TestLoom
https://polymetrika.com/Downloads/Excel

GreeBo

In their defence, I doubt anyone was able to provide them with test input and expected output.

Very hard to write tests in that scenario.

GreeBo wrote: »

In their defence, I doubt anyone was able to provide them with test input and expected output.

Very hard to write tests in that scenario.

On the contrary this is a very easy case for testing as it is numbers based solution and isn't complicated by the vagaries of user behavior.

The calculations could have absolutely been done on paper to validate for a sample control set of individual students.

GarIT

Wombatman wrote: »

On the contrary this is a very easy case for testing as it is numbers based solution and isn't complicated by the vagaries of user behavior.

The calculations could have absolutely been done on paper to validate for a sample control set of individual students.

Maybe they were, do it manually for 10 students and there's a 90% chance they all come out correct.

I doub't the analytics on how a teacher scored this year vs their usual students etc could be done manually with any ease.

...The system was meant to draw on the core subjects of Irish, English and Maths and their two strongest non-core subjects, instead, it combined the weakest non-core subjects....In the course of the review by Department, staff found a further error in the code, where the Civic Social Public Education module was meant to be discarded and wasn't, as a result approximately 6,500 students received at least one result one grade lower than should've been, that is being rectified.

That sounds like two bugs but no explanation why this only effected a small group (relatively) of students. On the face of it, it either should have effected a larger group. Either should have been spotted both in the spec and in the testing.

From experience. Often technical information passes through a number of people before it gets to the press. Often these people are not technical and often reword and put their own spin on an issue (they have to have skin in game and add value in their mind) without realizing they are changing the information fundamentally. They also might be obfuscating deliberately.

I've seen this happen before. We said a piece of work was outstanding, and there would be a cost associated with it. That went out as the work had been done, so there would no further cost associated with it.

It is a quote from the Minister of Education talking to the press. So are you saying that she is wrong about the whole thing?

GarIT

iLikeWaffles wrote: »

It is a quote from the Minister of Education talking to the press. So are you saying that she is wrong about the whole thing?

She doesn't have any technology related experience or qualifications, she can easy say something that's not 100% accurate.

Plus, the dev would have told their manager, who would have passed the info on to a business person who passed it on to the minister's staff who wrote the statment.

iLikeWaffles wrote: »

It is a quote from the Minister of Education talking to the press. So are you saying that she is wrong about the whole thing?

I never said she was wrong ; )

Slutmonkey57b

iLikeWaffles wrote: »

It is a quote from the Minister of Education talking to the press. So are you saying that she is wrong about the whole thing?

A politician would never lie.

ShatterProof

Slutmonkey57b wrote: »

A politician would never lie.

100% of the time

The politics of it don't interest me.

I'm just curious about it from a development point of view, and how they deal with situation.

Besides which, there are no bugs, just undocumented features.

beauf wrote: »

I never said she was wrong ; )

It was a question not a statement!

GreeBo

Wombatman wrote: »

On the contrary this is a very easy case for testing as it is numbers based solution and isn't complicated by the vagaries of user behavior.

The calculations could have absolutely been done on paper to validate for a sample control set of individual students.

It's only numbers based if the requirements provide all the input scenarios and all the expected results.

The problem here is that the system itself is the thing providing expected results so only a code review could find the issue.

https://www.rte.ie/news/education/2020/1003/1169207-leaving-cert-calculated-grades/

Must be riddled with bugs.......

The third error discovered in this latest review had to do with the incorrect treatment of Junior Certificate results belonging to students who were missing one of the three core subjects of Irish, English, and Maths. The company contracted to carry out this latest review Educational Training Systems corrected this newly discovered mistake during the review process.

ETS also found a fourth "issue" which "does not exactly match what is described". However the Department was reluctant to call this an error. It said it was an anomaly which had not impacted on students' results.

Its the nature of software development to have bugs. Nothing remarkable about that.

https://www.gov.ie/en/press-release/bc698-statement-from-the-minister-for-education-norma-foley-td-and-the-department-of-education-and-skills-regarding-leaving-certificate-2020-calculated-grades-process/

beauf wrote: »

Its the nature of software development to have bugs. Nothing remarkable about that.

It's the nature of software developers to find bugs.

Treppen

I heard that they are going to publish the code!

Does anyone know how the JC results exactly impacted on the LC results. All I could find is this.

The model was designed to take the Junior Cycle results of a group of students and use that group’s results to inform the calculation of their Leaving Certificate Results. It bears re-stating; the system did not take the results of a single student’s Junior Cycle exams and apply it to that specific student’s Leaving Certificate. Rather, the system looked at performance at an aggregate class level in the Junior Cycle examinations, and applied that in calculating students’ Leaving Certificate Results.