Encouraging Diversity Through Fairness in Assessment

November 16, 2018 Aon's Assessment Solutions

Diversity within a workforce can bring innovation and competitive advantage. Appreciating and recognizing differences can deliver business benefit. And yet, how does an organization ensure it is building and drawing from a wider talent pool, being fair to applicants and offering equal opportunities for employment when trying to recruit a diverse workforce or bring together diverse teams?

A good selection process starts with assessing those qualities of candidates that are related directly to job performance — and excluding the criteria that are not related to the job from any decision making. This creates a fair assessment process for all applicants in selection decisions.

This white paper, written by Dr. Katharina Lochner and Dr. Achim Preuss, outlines how fair testing can be beneficial for organizational effectiveness by promoting diversity.

Did You Know?

Assessment Barometer survey results relating to diversity in the workplace show that:

  • Creating a diverse workforce is an issue among more mature markets. Employers want to show they are making objective, unbiased selection decisions.
  • Major market organizations are looking to expand the diversity of their workforce and are interested in sourcing talent in new ways, more so than those in the intermediate and development markets.

The Benefits of Fair Testing

The composition of the workforce will change dramatically during the coming years.

On the one hand, with the expected increase in the mobility of employees between countries as technology advances and enables a more global workforce, it’s likely that there will be an increasing number of international applicants.

On the other hand, we face an aging society (demographic change), and this mean companies will have to become more flexible with respect to whom they recruit – in terms of age, gender, education, nationality and ability, but also in terms of beliefs. Their applicants will, without doubt, become more diverse.

Benefits of Creating a Diverse Workforce

  • The Right Thing to Do
  • Greater Innovation
  • Fair and Brand-Supporting
  • Make More Informed Decisions

Valuing Diversity

Diversity itself brings value to an organization. According to Kandola and Fullerton, "Valuing diversity means valuing the differences between people and the ways in which those differences can contribute to a richer, more creative and more productive business environment." (1998)

Rethinking how we view diversity means seeing it in terms of what individuals can bring to the organization. "Diversity should be understood as the varied perspectives and approaches to work that members of different groups bring," write Thomas and Ely. (1996) Thus, a diverse team can comprise members from different ethnic or cultural backgrounds, different ages, different genders and different abilities. But the concept goes even further. It also refers to different educational and social backgrounds or different experiences and beliefs that an applicant might have. A workforce with varied backgrounds and lived experiences can Offer an employer the opportunity to create a richer; more vibrant company culture.

Fair testing helps increase diversity within a company. This leads to increased organizational effectiveness, because often those with a diverse workforce are better able to adapt quickly and successfully to changes in the market. There is a wider range of approaches for problem solving and enhanced creativity and innovation. Companies with diverse workforces experience increased productivity and have access to a wider base of potential customers. (Cox, 1991)

"Valuing diversity means valuing the differences between people and the ways in which those differences can contribute to a richer, more creative and more productive business environment."

The Notion of Fair Testing

The notion of fair testing has been pursued for many years. In 1949, Raymond B. Cattell created his Culture Fair Intelligence Test (CFI T), which intended to assess intelligence independently of the test-taker's language or culture. Since then there have been attempts to make tests fairer, and test fairness has become an important quality criterion for applicant assessment tools.

So what is fair testing?

A test is fair if it does not systematically disadvantage certain groups of people.

And what does "not systematically disadvantage certain groups" mean? Does it mean that all applicants achieve the same score? Not quite.

The aim of a test or questionnaire is to differentiate between different persons — but it can't be discriminatory. Lilienfeld, Lynn, Ruscio & Beyerstein (2010) illustrate this: When doctors look at their records of the average weight of their male and female patients, they notice that men are heavier on average than women. Of course, this doesn't mean that the scales used are biased, but that there's a difference in weight between men and women. It also doesn't mean that a man is always heavier than a woman!

When companies develop tools for employment assessments, there are two general approaches to test fairness:

  • Fairness in predicting job performance from a given test score, and
  • Fairness in selection when job performance is given. (Hartigan and Wigdor, 1989)

Companies that engage in fair testing assess only job-relevant characteristics because it is only these aspects that can predict job performance. A fair test predicts job performance equally well for all subgroups. Therefore, test fairness is closely linked to a test's purpose. Test results are unfair when the test is used for purposes it was originally not intended and validated for. (Bortz & Döring, 2006)

How Does Fair Testing Influence Recruiting?

  1. What does the law say?

When companies launch a recruitment campaign, they need to make sure that they are not disadvantaging potential applicants because of their race, ethnicity, sex, religion or world view, disability, age or sexual orientation, because such bias violates the law. If a company is sued by an applicant because of adverse impact and is unable to prove that it did not disadvantage an applicant because of one of these areas, it will be fined. But not disadvantaging certain groups of applicants has another benefit for the company. It enlarges the pool of potential applicants.

  1. How does fairness affect a company's reputation?

When applicants feel treated in an unfair manner, they spread the bad news. This not only harms the company's reputation but can make the recruiting of talent even more difficult: There will be fewer and fewer applicants. A company with a strong reputation for fairness will attract more talent and will be able to choose its employees from a larger pool.

  1.  How can the criteria be assessed to ensure fairness?

The best candidate for a job cannot be identified and selected unless only job-related criteria are used. As soon as other criteria are considered, which is the case when the testing process is not fair, it's likely that the candidate appointed will not fit the job as well as another candidate might have fitted, and performance can suffer as a result.

  1. How can fairness help build a diverse workforce?

Diverse applicants give a company the opportunity to build up a more diverse workforce. And diversity can offer a competitive advantage, improving a company's performance and adaptability to changing markets.

Fair testing will help companies find the best candidate for a vacancy and will prevent them from exerting adverse impact due to unfair, biased, discriminatory or unlawful procedures. Furthermore, if candidates feel treated in a fair manner, they will share a positive image of the company.

Test Bias, Adverse Impact and Fairness

Test bias and adverse impact are concepts that can cause confusion when talking about fairness. Understanding the difference between the terms can help employers better create fair recruitment practices.

Bias

Test bias, also called "predictive bias" or "differential prediction," occurs when job performance is predicted differently for different groups.

To put it another way, test bias happens when there are two people from two different groups with the same trait, but they score differently on the test or its items, or when two people who receive the same score on a test are in fact different with respect to the trait measured. For example, a test that overpredicts job performance for a certain group would predict the performance to be better than it actually is. If the test indicates the reverse, that the group will perform less well, this is called underprediction.

Tests can be biased against different groups, including ethnic or racial groups, socio- economic groups, or men or women.

Adverse impact

Adverse impact is defined as a "substantially different rate of selection in hiring, promotion, or other employment decision which works to the disadvantage of members of a race, sex, or ethnic group." (Uniform Guidelines on Employee Selection Procedures, 1978)

Adverse impact is not a property of a test, but rather it results from the actual employment decisions made on the basis of tests or other selection procedures.

When is a selection rate "substantially different" so that it's considered to have adverse impact? Generally speaking, this is the case when the selection rate for any race, sex or ethnic group is less than 80 percent of the rate for the group with the highest rate (the so-called Four-Fifths Rule). There may be exceptions, and the rule could indicate adverse impact even when it may not exist.

Bias, adverse impact and fairness: What employers need to know

Adverse impact can result from unfair, biased, discriminatory or unlawful procedures, but also from true differences between two groups on a relevant, job-related characteristic. So, when there are in fact differences between groups on job-related characteristics, even a fair selection procedure will create adverse impact. However, in many cases, the adverse impact is created by using selection criteria that are not related to job performance. In these cases, fair testing can significantly contribute to the reduction of adverse impact.

Fairness and bias are not the same.

  • Bias is a psychometric concept, referring to the fact that there are systematic differences in the meaning of test scores associated with group membership. As such, it's objective.
  • Fairness, on the other hand, is more a social concept. There are different attitudes toward what is considered fair (What is the measure? Equality? Performance? Needs?), and according to these ideas, the same outcome can be considered fair or not.

For employers, creating a process that's fair and that accurately assesses applicants based on job-related characteristics is essential for ensuring unbiased hiring practices.

So, what are the qualities of a hiring process that's more likely to be fair? Here are three key constructs to help achieve this goal:

  • Procedural justice. How is an outcome or decision achieved?
  • Distributive justice. How are the outcomes distributed?
  • Interactional justice. How do the people involved feel they have been treated?

We can see the relevance of these questions to employment testing.

  • Procedural justice encourages us to consider the testing process by which outcomes or decisions are attained, and the factors that improve perceived fairness such as consistency, accuracy, ethics, and lack of bias. (Leventhal, 1980)
  • Distributive justice encourages us to consider the testing process from the perspective of whether outcomes are likely to be seen as equally distributed across stakeholders, including different applicant groups as well as the employer. (Adams, 1965)
  • Interactional justice encourages us to consider how explanations and communications are framed for decision making and the outcomes attained by the stakeholders in a process. Colquitt (2001) has shown that positive perceptions of interpersonal justice are more likely when someone is treated with respect (interpersonal justice) and when the explanations given are adequate (informational justice).

Why Fair Testing Enhances Diversity

Diversity is beneficial for performance. This is why it makes sense to recruit diverse employees. Here are a couple of examples that illustrate the value of fair testing for employers and how screening tools have a big impact on whether an employer is able to recruit the kind of diverse workforce it needs.

Example 1

A local authority, city or municipality wants to recruit firefighters.

Being a firefighter is physically demanding — for example, trees felled by a storm may have to be removed, and heavy gear has to be lifted and carried. Therefore, when defining the job requirements the recruiters decide that the ideal firefighter, among other characteristics, has to be strong. They define that "being strong" means being able to lift 80 kg over one's head. me job is advertised, and it receives many applications. The recruiters decide to invite only male applicants to the assessment center and turn all female applicants down because, according to statistical data, men are on average stronger than women.

This procedure is unfair because not all men are stronger than all women. During the selection process, it's not the characteristic of “being strong” that's being assessed; it's the characteristic “being male,” which is certainly not job- relevant. Furthermore, being strong' ' does not necessarily mean "being able to lift 80 kg," There are techniques for moving heavy gear and other objects; therefore, it's not necessary that applicants are able to lift 80 kg over their heads.

Also, the applicants were not assessed with respect to the other job requirements. Firefighters, according to the O*NET job description, need to have a number of skills and abilities, among them active listening and critical thinking, as well as multi -limb coordination and static strength. In a selection process that assesses only these qualities, people of both sexes, of different cultural backgrounds and different world views, will have the chance to be offered the job.

Example 2

A company employing technical apprentices usually receives many applications.

The company has found that to be successful, the apprentices need to have good analytical skills. Because high-school students with good grades in math and physics generally have good analytical skills, only applicants with good and very good grades in these two subjects are Invited to the assessment center.

This is another example of an unfair selection process.

First, it's not the characteristic itself (analytical abilities) that is being assessed, but another criterion (school grades) that's related to, but not identical with, the job-relevant criterion. Second, only one aspect is considered, not all job-relevant characteristics. Other aspects — such as mechanical-technical understanding, practical experience or social skills — are not considered

Thus, a fair selection process is one in which only job-relevant criteria and multiple criteria are used for shortlisting candidates.

Employers that limit their assessments to job-related characteristics will have the most success recruiting a dynamic, diverse workforce.

It's important that test scores constitute only the minimum requirements that applicants should fulfill. This means that companies don't simply select those who score best on the test and offer them the job. The reason for this is a statistical one: Correlations, the measures used when predicting job performance from a test result, can only be high if there is a wide range of test scores. If the sample is range-restricted, meaning that only the top scorers are taken into consideration, the predictive power of the test approaches zero.

Practically, this means that a slightly better test result of one applicant does not mean that he or she is significantly more likely to do well on the job than another applicant who scores slightly lower. Therefore, in a fair selection process, only those definitely not fitting the selection criteria will be opted out, while all those who meet the minimum requirements will stay in the process.

The 5 Factors of Fair Testing

Kunnan (2004) suggests a framework that "views fairness in terms of the whole system of a testing practice, not just the test itself." His framework included:

  • Accuracy: A test has to be representative of items, tasks or topics (content validity) as well as of the construct or underlying trait it measures (construct validity). It also needs to predict the criterion it's meant to predict (criterion validity), and needs to be reliable.
  • Equality: The test must not systematically disadvantage certain groups. The content of the test must not be offensive toward certain groups in terms of language or content and must not penalize certain groups of test-takers because of their background. Furthermore, differences between different groups of test-takers must be examined and considered when setting standards.
  • Accessibility: Groups of test-takers must not be disadvantaged in their access to the test. This comprises the opportunity to prepare for the test and familiarize with the procedure and equipment, to get access to the location the test is administered in and to financially afford the test. Furthermore, accommodations for test-takers with challenges (e.g. dyslexia, ADHD, visual and motoric handicaps) must be made.
  • Practicability. For supervised tests, physical conditions — for example, light and temperature at the testing location — must be appropriate. Furthermore, test- taking conditions must be uniform and secure for all. Unsupervised tests are to be designed in such a way that differences — for example, in the quality of the computer display or the software installed on the computer — don't affect the results.
  • Reversibility. The test results must have an effect on instructional practices, but there must also be the opportunity for test-takers to proceed against detrimental social effects of the test — for example, by rescoring of the test or by legal remedies.

Maximizing the Benefits of Diversity

Cultivating a company culture that values diversity helps employers maximize the productivity and innovation of their workforce. It also allows them to engage in fair and ethical hiring practices that strengthen their reputation and company values. Here are some strategies for encouraging diversity and for maximizing the benefits of a diverse workforce.

Organizational culture:

  • Create a culture that's open and appreciative of differences. This will encourage employees to contribute their strengths to achieve the company's goals.
  • Have high expectations of all your employees and encourage them to develop their unique strengths.
  • Stimulate personal development; adapt the job profiles to those doing the job as far as possible.
  • Make employees feel valued the way they are.

Mission:

  • Clearly articulate the company's mission and make sure it is widely understood.
  • Encourage a mission-centered approach to the company's work and allow employees to help contribute to sense of mission at the organization.

Structures:

  • Establish egalitarian, nonbureaucratic structures.
  • Encourage leaders to see that there are different perspectives and approaches to work.

To learn more about creating a diverse workforce, contact us:

APME.assess@aon.com

Europe.assess@aon.com

NA.assess@aon.com

About the Author

Aon creates smart measurement solutions with valid and innovative online assessment products. Aon is globally the preferred partner for organisations who demand the best.

Follow on Linkedin Visit Website More Content by Aon's Assessment Solutions
Previous Flipbook
Digital Readiness - A Question of Survival
Digital Readiness - A Question of Survival

How to manage a digital transformation process to secure your organization’s future

NEXT FEATURE
Artificial Intelligence in Assessment
Artificial Intelligence in Assessment

This paper explains the role of AI in assessment, highlighting how it can be used today and how it might be...