“These are the leaders of tomorrow; they are meant to be people managers,” the applicant said.
Loading
But their capabilities have been thrown into doubt after Turnitin software found that some had used AI for more than 50 per cent of their applications.
Candidates were warned multiple times, including by text message, that their applications would be screened for AI.
While 50 per cent was the benchmark FRNSW set for elimination from the selection process, the actual rate of AI use in the applications could be much higher.
In its advice to candidates, FRNSW said that if Turnitin identifies that 50 per cent of a document is probably written by an AI tool, it could contain as much as 65 per cent AI writing.
“This is the first time FRNSW has used an AI integrity checker in recruitment,” Genders wrote in an earlier email, announcing the introduction of the AI screening software.
“Following extensive research and consultation with educational institutions and organisations experienced in this technology, Turnitin was selected for its proven reliability and accuracy compared with other systems assessed.”
But the FRNSW controversy is not the first time Turnitin has faced questions about its reliability.
In March, students at Australian Catholic University claimed it had wrongly accused them of using artificial intelligence. Curtin University announced in September that it was disabling the Turnitin tool.
Advance Publications acquired the US-based multinational for $US1.75 billion ($2.6 billion) in 2019. Turnitin did not respond to requests for comment.
Loading
On its website, Turnitin states that its writing detection model may not always be accurate: “it may misidentify human-written, AI-generated, and AI-generated and AI-paraphrased text, so it should not be used as the sole basis for adverse action”.
Genders told candidates he understood the process may have been “disappointing and frustrating”.
“While this approach was well-intentioned and supported by consultation with the [Fire Brigade Employees Union], the trial has highlighted inconsistencies in how AI usage was detected and interpreted,” he said.
A spokesperson said FRNSW was satisfied with the outcomes of the recruitment process.
“FRNSW is committed to maintaining a fair, transparent and merit-based selection process for all applicants in accordance with established recruitment guidelines and policies,” they said.
The Morning Edition newsletter is our guide to the day’s most important and interesting stories, analysis and insights. Sign up here.
