Automation has been essential to many companies’ e-commerce model and has been using computer programs to review prospective applications since 2014. Recently, the company attempted to implement a new hiring tool using AI that would rank candidates like search results from a search engine, i.e.., the best 5 candidates from a batch of 100.

It was later identified that one company’s algorithm was not ranking their candidates in gender neutral way. The recruitment process using AIs were systematically ruling out applicants who attended all-women’s colleges, or even resumes that contained the word “women’s”. The system was essentially teaching itself that male candidates were preferable to their female counterparts.

The company later disbanded the project.  

However, this has not stopped recruiters from looking at the recommendations generated from the system in conjunction with conventional evaluations of potential candidates. 

The issue of fairness and bias in machine learning is becoming an increasingly important question in society. AI systems are essentially perpetuating historical patterns of bias due for a variety of reasons; incomplete or skewed training datasets where certain demographics are missing from datasets, and bias induced labels used for training AI systems.