An algorithm that was being tested as a recruitment tool by online giant Amazon was sexist and had to be scrapped, according to a Reuters report. The artificial intelligence system was trained on data submitted by applicants over a year period, much of which came from men, it claimed.
Reuters was told by members of the team working on it that the system effectively taught itself that male candidates were preferable. Reuters spoke to five members of the team who developed the machine learning tool innone of whom wanted to be publicly named. They told Reuters that the system was intended to review job applications and give candidates a score ranging from one to five stars.
Byit was clear that the system was not rating candidates in a gender-neutral way because it was built on data accumulated from CVs submitted to the firm mostly from males, Reuters claimed. The system started to penalise CVs which included the word "women".
The program was edited to make it neutral to the term but it became clear that the system could not be relied upon, Reuters was told. The project was abandoned, although Reuters said that it was used for a period by recruiters who looked at the recommendations generated by the tool but never relied solely on it.
It is not the first time doubts have been raised about how reliable algorithms trained on potentially biased data will be. An experiment at the Massachusetts Institute of Technology, which trained an AI on images and videos of murder and deathfound it interpreted neutral inkblots in a negative way.
And in May last year, a report claimed that an AI-generated computer program used by a US court was biased against black people, flagging them as twice as likely to reoffend as white people. Predictive policing algorithms were spotted to be similarly biased, because the crime data they were trained on showed more arrests or police stops for black people. IBM launches bias detector for AI. Meet Norman, the psychopathic AI. Amazon has not responded to the claims.
According to Amazon, its current global workforce is split in favour of males. Related Topics. Artificial intelligence Amazon Sexism Employment.
More on this story. Published 19 September Published 2 June