Abstract: A processing device is to: identify, using digital interview data of interviewees captured during interviews, a subset of the interviewees that have a disability; label a first group of the interviewees as disabled and a second group of the interviewees as not disabled with reference to the disability; identify features from the digital interview data for the first group that correlate with the disability; formulate a digital fingerprint of the features that identifies how the first group differs from the second group with reference to the disability; map the digital fingerprint of the features onto a dataset of an interviewee belonging to the second group of the interviewees, to generate a mapped dataset; and determine, from the mapped dataset, effects of the digital fingerprint on a job performance score for the interviewee.

Abstract: A computing system includes a processing device to execute instructions to: store a trained modeling function that embodies a correlation between historical ratings and historical evaluation results of recorded responses that were responsive to prompts presented to a set of persons; receive a request to initiate a digital evaluation process for a second person; receive recorded responses of the second person that are responsive to the prompts; generate, by applying the trained modeling function to the recorded responses of the second person, a reviewing sequence by which to view the recorded responses of the second person. The reviewing sequence orders the recorded responses from a most predictive prompt to a least predictive prompt to reduce a time required to efficiently determine an evaluation result. The system further transmits data that causes display of a list of the recorded responses of the second person in the reviewing sequence.

Abstract: A method for detecting bias in an evaluation process is provided. The method includes operations of receiving evaluation data from a candidate evaluation system. The evaluation data is provided by a set of evaluators based on digital interview data collected from evaluation candidates. The operations of the method further include extracting indicators of characteristics of the evaluation candidates from the digital interview data, classifying the evaluation candidates based on the indicators extracted from the digital interview data, and determining whether the evaluation data indicates a bias of one or more evaluators with respect to a classification of the evaluation candidates.

Abstract: Described are methods and systems for interview competency and question validation and analysis to improve the effectiveness of evaluation campaigns. In one method, processing logic of an interview design program, hosted by a digital evaluation platform, receives a request from a first device to create a digital interview for a position. The interview design program sends a list of competencies associated with the position and receives a selection of a set of desired competencies. The interview design program determines a list of questions that differentiate candidates within the desired competencies, ranks the list of questions by importance, and sends the list of questions and the ranking information to the first device and receives a selection of a set of desired questions. The interview design program creates the digital interview with the set of desired questions. The digital evaluation platform presents the digital interview to a candidate on a second device.

Abstract: Technology for training a predictive model is described. A processing device collects digital interview data including features identified from candidate interviews. A model training tool selects a data set of the digital interview data. The data set includes a predicted performance outcome and an actual performance outcome for each of a plurality of candidates. The model training tool determines an error metric for each of the plurality of candidates. The error metric includes a relationship between the predicted performance outcome and the actual performance outcome for each candidate. The model training tool determines a number of candidates whose digital interview data includes a feature corresponding to a protected class. The model training tool normalizes an effect of each candidate on the error metric based on the corresponding protected class and applies the normalized error metric to reduce bias in the predictive model with respect to the protected class.

Abstract: A method for facilitating the evaluation of candidates in an evaluation process is provided. The method includes operations of selecting a first data set including ratings of recorded responses of a set of evaluated persons and of selecting a second data set including additional recorded responses of a person under evaluation. The recorded responses are responsive to multiple prompts presented to each of the set evaluated persons in a recording sequence. The first data set further includes an evaluation result of each evaluated person in the evaluation. The additional recorded responses are responsive to the plurality of prompts and ordered in a first order corresponding to a recording sequence. The method includes analyzing the first data set to calculate a reviewing sequence and presenting the additional recorded responses in a second order corresponding to the reviewing sequence. Associated systems and other methods are also provided.

Abstract: Described herein are methods and systems for interview question or prompt recommendation and analysis to improve the quality and efficacy of subsequent evaluation campaigns by combining data sets are described herein. In one method, processing logic selects a first prompt from a first data set of a first candidate evaluation campaign and a second prompt from a second data set of a second candidate evaluation campaign. The processing logic determines whether a degree of similarity between the first prompt and the second prompt exceeds a threshold and combines data from the first data set with data from the second data set to create a combined data set associated with the first prompt and with the second prompt based on the determination.

Abstract: A method for facilitating the evaluation of candidates in an evaluation process is provided. The method includes operations of selecting a first data set including ratings of recorded responses of a set of evaluated persons and of selecting a second data set including additional recorded responses of a person under evaluation. The recorded responses are responsive to multiple prompts presented to each of the set evaluated persons in a recording sequence. The first data set further includes an evaluation result of each evaluated person in the evaluation. The additional recorded responses are responsive to the plurality of prompts and ordered in a first order corresponding to a recording sequence. The method includes analyzing the first data set to calculate a reviewing sequence and presenting the additional recorded responses in a second order corresponding to the reviewing sequence. Associated systems and other methods are also provided.

Abstract: A method for detecting bias in an evaluation process is provided. The method includes operations of receiving evaluation data from a candidate evaluation system. The evaluation data is provided by a set of evaluators based on digital interview data collected from evaluation candidates. The operations of the method further include extracting indicators of characteristics of the evaluation candidates from the digital interview data, classifying the evaluation candidates based on the indicators extracted from the digital interview data, and determining whether the evaluation data indicates a bias of one or more evaluators with respect to a classification of the evaluation candidates.

Abstract: Described herein are methods and systems for interview question or prompt recommendation and analysis to improve the quality and efficacy of subsequent evaluation campaigns by combining data sets are described herein. In one method, processing logic selects a first prompt from a first data set of a first candidate evaluation campaign and a second prompt from a second data set of a second candidate evaluation campaign. The processing logic determines whether a degree of similarity between the first prompt and the second prompt exceeds a threshold and combines data from the first data set with data from the second data set to create a combined data set associated with the first prompt and with the second prompt based on the determination.

Abstract: Methods and systems for model-driven candidate sorting for evaluating digital evaluations are described. In one embodiment, a sorting tool selects a data set of digital evaluation data for sorting. The data set includes candidate for evaluation candidates. The sorting tool analyzes the candidate data for the respective evaluation candidate to identify digital evaluation cues and applies the digital evaluation cues to a prediction model to predict an achievement index for the respective evaluation candidate. The list of evaluation candidates is sorted according the predicted achievement indices and the sorted list is presented to the reviewer in a user interface.

Abstract: Interview scheduling technologies are described. In one method, the interview-scheduling tool presents an interface to receive candidate availability information from a user for possible interview schedules for an interview candidate. The user interface permits the user to define an interview schedule framework that specifies desired criteria for a desired interview schedule for the interview candidate. The desired criteria of the interview schedule framework specify one or more interview sessions. The interview-scheduling tool receives the candidate availability information and the desired criteria and defines a data structure representing the interview schedule framework, wherein the data structure comprises a set of constraint satisfaction problems (CSPs). The interview-scheduling tool presents the interview schedule results that fit the desired interview schedule for selection by the user.

Abstract: A method for facilitating the evaluation of candidates in an evaluation process is provided. The method includes operations of selecting a first data set including ratings of recorded responses of a set of evaluated persons and of selecting a second data set including additional recorded responses of a person under evaluation. The recorded responses are responsive to multiple prompts presented to each of the set evaluated persons in a recording sequence. The first data set further includes an evaluation result of each evaluated person in the evaluation. The additional recorded responses are responsive to the plurality of prompts and ordered in a first order corresponding to a recording sequence. The method includes analyzing the first data set to calculate a reviewing sequence and presenting the additional recorded responses in a second order corresponding to the reviewing sequence. Associated systems and other methods are also provided.

Abstract: Methods and systems for model-driven candidate sorting for evaluating digital interviews are described. In one embodiment, a model-driven candidate-sorting tool selects a data set of digital interview data for sorting. The data set includes candidate for interviewing candidates (also referred to herein as interviewees). The model-driven candidate-sorting tool analyzes the candidate data for the respective interviewing candidate to identify digital interviewing cues and applies the digital interview cues to a prediction model to predict an achievement index for the respective interviewing candidate. This is performed without reviewer input at the model-driven candidate-sorting tool. The list of interview candidates is sorted according the predicted achievement indices and the sorted list is presented to the reviewer in a user interface.

Abstract: Methods and systems for model-driven candidate sorting based on audio cues for evaluating digital interviews are described. In one embodiment, a processing device converts audio data of a candidate in a digital interview into voice-to-text data. An audio cue generator executing on the processing device, generates audio cues of the digital interview based on the voice-to-text data. The audio cues are applied to a prediction model to predict an achievement index for the candidate based on the audio cues. The processing device displays the candidate in a list of candidates based on the achievement index. The list of candidates is sorted according to the candidates' achievement index.

Abstract: Interview scheduling technologies are described. In one method, the interview-scheduling tool presents an interface to receive candidate availability information from a user for possible interview schedules for an interview candidate. The user interface permits the user to define an interview schedule framework that specifies desired criteria for a desired interview schedule for the interview candidate. The desired criteria of the interview schedule framework specify one or more interview sessions. The interview-scheduling tool receives the candidate availability information and the desired criteria and defines a data structure representing the interview schedule framework, wherein the data structure comprises a set of constraint satisfaction problems (CSPs). The interview-scheduling tool presents the interview schedule results that fit the desired interview schedule for selection by the user.

Abstract: Interview scheduling technologies are described. In one method of scheduling an interview with a candidate, the method receives scheduling information including constraint and parameter information for scheduling the interview with the candidate and receives candidate information including candidate availability. The method determines, by a computer system, at least one optimized interview schedule matching the constraint information, the parameter information, and the candidate availability, the at least one optimized interview schedule including at least one interview slot and at least one interviewer associated with the at least one interview slot. The method provides the at least one optimized interview schedule to a user.

Abstract: Methods and systems for model-driven candidate sorting based on audio cues for evaluating digital interviews are described. In one embodiment, an audio cue generator identifies utterances in audio data of a digital interview. The utterances each include a group of one or more words spoken by a candidate in the digital interview. The audio cue generator generates audio cues of the digital interview based on the identified utterances. The audio cues are applies to a prediction model to predict an achievement index for the candidate based on the audio cues. The candidate is displayed in a list of candidates based on the achievement index. The list of candidates is sorted according to the candidates' achievement index.

Abstract: A user interface for evaluating interview candidates. A user interface, including a first interview card is displayed. The first interview card includes information about a first interviewee and the first interviewee's response to a first interview question. User input indicative of a direction is received. The user input is either indicative of a direction in a first plane or indicative of a direction in a second plane. When the user input is in a direction in the first plane, a second interview card is displayed including information about a second interviewee and the second interviewee's response to the first question. When the user input is in a direction in the second plane, a third interview card is displayed including information about the first interviewee and the first interviewee's response to a second question.

Abstract: Selecting interview questions to be posed to an interviewee. A method includes receiving user input. The user input selects an interview evaluation system from among a number of different interview evaluation systems. Each interview evaluation system in the number of different interview evaluation systems includes functionality for at least one of scoring interview interactions or rating interviewees. At least two or more of the interview evaluation systems from among the number of different interview evaluation systems include different functionality for scoring interview interactions or rating interviewees. Based on the user input selecting an interview evaluation system, a set of interview queries from among a plurality of pre-defined interview queries are automatically selected. The selected set of interview queries are provided for use in an interview of an interviewee.