Artificial Intelligence and Job Interviews: Is the AI Tech Ready?

Artificial Intelligence Job Interviews

• Traditionally, for job interviews candidates sit with an HR manager, chat, and answer questions, either in person or on Zoom. With AI job interviews, there is no manager to interact with. The candidate simply receives pre-recorded questions on their computer monitor or phone , and they record themselves answering those prompts. Those answers are then assessed by artificial intelligence.
  • The AI makes assessments based on any number of factors, including word choice, language fluency, voice intonation, even facial expressions, and those scores are then sent to hiring departments.
• There are many positives with the AI job interview process, for both the employers and the job candidates:
  • In AI interviews, this structured interview process means every candidate is asked the same questions, eliminating some degree of potential bias from the process;
  • AI interviews save labor costs for the HR department because it is faster, and HR professionals can either watch the recordings or have artificial intelligence score the interviews.
  • From a scheduling standpoint, it’s more convenient for candidates because it allows them to record their responses at a time when it is most convenient for them.
  • Some candidates appreciate that AI interviews are conducted more as a presentation of themselves, rather than as a conversation, because it takes away a lot of the anxiety around talking to another person. 

Research into the Claims

• Companies claim the AI interview process is not only efficient, but that it can find the strongest candidates, and now, researchers are trying to assess how that works and how true this claim is.
• Ms. Schellmann and her team focused their research on companies that use AI hiring processes that have a certain market share. They looked at:
  • How the AI job interviews worked at these companies;
  • How the AI interview processes are marketed; and
  • Whether or not AI can, in fact, find the best candidates.
• After nearly five years of research on this topics, Ms. Schellmann is currently in the process of writing a book on this subject, taking an even deeper dive into AI testing, and her book is expected to hit shelves in early 2024.
• Ms. Schellmann took part in some of the testing herself, to see what other candidates go through when going through artificial intelligence.

Takeaways from AI Testing

• Ms. Schellmann and her team discovered through their testing that the artificial intelligence systems they looked at had flaws.
  • One system was supposed to be able to detect the candidate’s proficiency as an English speaker. As a test, Ms. Schellmann, who is fluent in German, intentionally answered all the interview questions in German. The AI dubbed her fluent in English – even though she’s not spoken a work of English during the interview!
  • Another tool tested promised to assess the candidates’ personality, based on the Big 5 Personaility Traits, but delivered different results every time she took it, indicating that the tool was neither reliable or stable in its assessment.
• These results are concerning because these systems are already being used, companies are displaying their use, with confidence.
  • These systems are being used on real people, for high stakes hiring decisions.

The Ethics of AI Interviews

• There are many questions surrounding the use of artificial intelligence during the hiring process.
  • Experts say there should be an audit done on the tool to test its fairness and functionality.
  • Fairness
    • In some of the AI games, some people with disabilities were not able to complete the challenges because of their motor abilities, and some of the games had to relevancy to their competency to complete the job. This poses a fairness question for those people, especially because they do not know if they need to ask for an accommodation if they do not know what the questions may be.
    • Fairness also comes into play when thinking about if women and men play AI games in the same way, if people have an accent or a speech impairment, and that needs to be taken into account. The priority should be thinking about if artificial intelligence hiring practices will work for them.
  • Functionality
    • U.S. law dictates that these AI hiring tools don’t need to work, they just need to be fair for everyone, but while fairness is important, it  doesn’t necessarily yield results - on paper, hiring with a random number generator would be considered “fair:, but would not necessarily yield the best people for the job.
    • Companies, and job applicants, want the AI interview process to work, and have it be fair to candidates and their skills.Not getting hired because of flawed technology would be very frustrating for potential candidates.
• The EOC recently came out with a statement about the use of artificial intelligence in hiring, and the rules currently in place still apply, but those rules are not necessarily picked up on the scope of video interviews and remote hiring practices.
  • Previously, it was easier to demonstrate a case of unfair hiring practices, but now, it is more difficult to prove if AI is being used in the hiring process, how candidates are classified, and if candidates are being wronged. This is more challenging for candidates to prove biases or unfairness in court.
  • The EOC statement provided technical advice to employers, saying they acknowledge AI hiring practices, and reminding them that hiring and employment laws still apply.
• Companies need to make sure that the technology works before using it in their hiring practices.
  • While smaller companies probably don’t have data scientists, it is the companies responsibility to test the AI technology, rather than relying on a larger company or a third party vendor.

The AI Employer Experience

• AI is a technology that feels like it would help employers, and it can be more efficient for them.
  • Technology can eliminate human biases about experience, education, etc. however, it is important to make sure the technology works.
  • Using AI can save companies time and money, and provide a very structured based process.
• AI can be an effective interviewer through structure, but it is still unclear if it can accurately identify strong job candidates.
  • Researchers and companies need to continue testing the efficacy of AI, and if it is predictive of employer success. Most HR departments are not set up for this testing, which would take many years and is very cost intensive. 
  • Most companies do not have enough data to test their employees' success regularly, nevermind the efficacy of AI technology, and it is challenging to determine employee success depending on the career. 
• Using AI technology in the hiring process can bring value to an organization if they are used in the right way.
  • The technology might not only be used in an AI job interview, but also looking at peoples resume, at what drives candidates, skills, and an overall holistic approach to thinking about how successful candidates could be on the job.
  • Companies turn to AI to be more cost effective, but some candidates really do not like virtual or AI job interviews, and that can be a turnoff for them working at the company.
• Companies need to be more transparent and do more testing regarding their use of artificial intelligence. 
  • Companies need to look at what goes into these algorithms, the training data, and understand the results of the algorithm.
  • Vigorous testing and vigorous transparency are the best ways to see if an algorithm works before being really put into practice company wide. 
  • There are new laws, such as in NYC, about remote hiring practices. The GDPR is also looking into storing hiring data, access, and what companies can do with the results. 

Hilke Schellmann Background

• M.S., Investigative Reporting, Broadcast, Multimedia from Columbia University - Graduate School of Journalism
• Magister (Combined B.A./M.A.), Cultural Theory, American Studies and Political Science from Humboldt-Universität zu Berlin
• Graduate School of Arts & Sciences, American Studies, Performance Studies from New York University
• Ms. Schellmann began her career working on documentaries, including work as an Assistant Editor for the American Undercover Series for HBO, and serving as Coordinating producer for a National Geographic special about the Berlin Wall
• She served as the Staff Reporter/Producer for Story House Productions
• Her freelance work includes work as a Radio Producer for WNYC, as a Videojournalist for The New York Times, as a Producer for German Public Television, ARD and ZDF, and as a Showrunner/Executive Producer and Reporter for The Wall Street Journal
• She worked as a Multimedia Reporter/Producer for The Wall Street Journal
• She was a Producer at Vice Media, LLC
• She served at the Director of Video Journalism/Adjunct Professor for Columbia University - Graduate School of Journalism
• Ms. Schellmann has won numerous awards and recognition for her work as Reporter/Producer with H2H Films
• Additionally, she currently serves as Assistant Professor of Journalism with New York University

Links

• In Machines We Trust
• https://www.technologyreview.com/author/hilke-schellmann/ 
• https://www.lexology.com/library/detail.aspx?g=1a9f4917-134e-4e4b-96ac-6569e7c3d7d5 
• https://www.eeoc.gov/newsroom/eeoc-launches-initiative-artificial-intelligence-and-algorithmic-fairness 
• https://journalism.nyu.edu/graduate/programs/magazine/faculty/ 

Contact

• Hilke Schellmann on LinkedIn
• Hilke Schellmann email: hilke.schellmann@gmail.com