Designing a Diversity-Enabling Talent Assessment Process

May 25, 2021 Sarah Kenehan

To build a sustainable DE&I program, we believe that you need a foundation of fair, objective and bias-free assessment processes – be they for new hires into the organization or to support internal mobility across the firm.

From the outset, we take a holistic view while working with clients on their DE&I strategy and actions. We look at current processes and procedures and, when available, work with the client to review the data they may have recorded from applications, drop-out rates, progress and final hire. We look at candidate-attraction activity, desired skills and behaviors (both for now and for the future) and how best to eliminate bias and build an objective process.

Take, for example, our work with a worldwide asset management firm.

Committed to reducing bias

The firm is committed to ensuring objectivity in its selection processes. Any new assessment process needs to minimize the potential for unconscious bias and eliminate any adverse impact that is caused by using a particular assessment procedure.

Aon designed talent assessments for the initial screening of the firm’s school leaver, apprenticeship and graduate program applicants, measuring the fit between the candidate and the firm’s core values, culture and expectations.

It made use of a contemporary, designed-for-mobile, gamified assessment, a bespoke job simulation activity that captured what it was like to work for the firm and a short behavioral assessment focused on the firm’s core values. This meant that those candidates who were successful in the assessments were also those aligned with the firm’s needs.

Those scoring as a match with the requirements went on to record and submit a video interview, providing responses to questions concerning the firm’s values. Shortlisted candidates were then invited to attend an assessment center and the best performers were then invited to attend the final interview stage before offers were made.

Using AI as well as objective assessment

As well as deploying robust objective assessment to help eliminate human bias in the initial application stages, the firm also wanted to exploit technical advances in video interviewing and the use of artificial intelligence (AI) to score these. It knew that, in order to be fair, the AI rating needed to be close to the ratings provide by a group of human raters. This became the focus of a study in its own right.

For the first cohort of candidates, video interviews were rated by both trained hiring managers and scored by the expert-trained AI. Ratings were then compared. This analysis highlighted the questions which showed closest alignment and were to be used going forward as the initial screen of the video interviews. Using AI in this way helped to remove individual human bias, provided a more objective way in which to gauge interviewee performance and saved hiring manager resources.

Analyzing adverse impact data

The firm needed to check on the assessment’s robustness. Did the assessments used in the redesigned process have a negative impact on any specific group of candidates?

The answer could only be obtained by digging deeper into the results for each candidate – in an anonymous fashion – and analyzing that data. For many in-house HR teams, this data analysis can be time-consuming and a challenge. For Aon, it is straightforward.
 
With over 5,000 people registering for the role within the firm’s applicant tracking system and the subsequent application and assessment completion figures at each stage, there was sufficient data for a meaningful analysis of assessment performance.

The results

Detailed analysis showed no adverse impact on either gender or ethnicity.

Analysis also highlighted that the pass rates and performance scores were similar for both the majority and minority subgroups. While fewer females than males completed the assessments, females scored slightly better, although not significantly better. There was no significant difference in the scoring of assessments in different countries and regions.

Not only did the new selection process hold true to the firm’s commitment to hire a workforce objectively and without bias, but candidates engaged with and enjoyed the experience. When measured through the Candidate Net Promoter Score, it achieved 37 and an average rating of 8.2/10.  

In addition, the hiring managers were positive on the effectiveness of the new early career recruitment process. In particular, the video-interviewing stage was rated by 85% of hiring managers as average or above average at highlighting those candidates who were a good fit for the firm.

Recruiters too were positive, saving significant time thanks to the AI-supported interview.  

However, there is no room for complacency. The firm continues to track and monitor the data it receives and knows that adjustments and refinements may be needed. To effectively implement the DE&I strategy requires insights to be gained and actions to be taken on a continual basis.

Contact us if you would like to learn more about how to redesign your assessment processes to ensure fairness and engagement and encourage a more diverse cohort.

About the Author

Sarah Kenehan

Sarah is an Organisational Psychology Consultant and currently part of the commercial team at Aon's Assessment Solutions. She is responsible for bringing a psychological perspective to our high level propositions.

More Content by Sarah Kenehan
Previous Article
Unlocking the Potential for Female Mobility in the Workforce
Unlocking the Potential for Female Mobility in the Workforce

How a large banking organization was able to identify those women ready for leadership positions and how th...

NEXT FEATURE
The Journey Towards a More Diverse Leadership Pipeline
The Journey Towards a More Diverse Leadership Pipeline

There is plenty of debate about what is – and who has – potential. Surely, the majority of the workforce ha...