Amazon (AMZN) was utilizing a secret AI for recruiting purposes. However, after discovering that its AI didn't like hiring women, Amazon had to pull the plug from the AI tool.
The company has a history of using AI tools since 2014 to filter out the top talent from job applicants. Considering how Amazon is so keen on automation, it makes sense to believe that it also wanted to automate its hiring process. From making pricing decisions to decisions inside the warehouses, Amazon relies a lot on automation.
Considering how useful the AI recruiting tool is, everyone would have wanted something like this for their company. But in 2015, the company found that the system was not accurately rating the candidates for technical jobs. Furthermore, the AI tool was not acting in a gender-neutral way (giving less preference to women).
To train the AI tool, Amazon fed 10 years' worth of applicants' data to the computer. Since most of the resumes were from women, the AI formed a preference for male candidates based on the training data.
Considering how the training data was biased from the start, it made sense that the AI tool preferred male candidates.
The AI recruiting tool filtered out any resumes that included words related to women. Furthermore, the tool also penalized the resumes in which two specific women colleges were mentioned.
To correct the mistakes in the AI tool, the company made several attempts to make it more neutral. However, there was no way to know that the AI tool would not find a way to discriminate against candidates based on gender again.
By the end of last year, Amazon finally decided to disband the team working o the AI recruiting tool. According to the inside sources, the recruiters at Amazon did rely on n the recommendations for making new hires - However, the rankings from the AI tool were never the sole criteria.