Does Context Make Tests More Socially Acceptable?

May 18, 2016 Katharina Lochner

tests and face validity

Acceptance of Tests and their Perceived Face Validity

When candidates complete tests and questionnaires during a selection process they often wonder what these instruments have to do with the job they are applying for. Sometimes even recruiting companies do. This is particularly the case for abstract-logical reasoning tests. Therefore very often contextualised tests are used. These are tests that are set up e.g. in a business context so that it is more obvious what they have to do with the job, while the ability assessed is still logical reasoning. However, the question is whether context really makes a difference for social acceptance of tests.

This is what Miriam Meurer and Martin Kersting from the University of Gießen, Germany, looked at in a study. They used our test scales verbal (consumer compact), a test measuring the ability to draw logical conclusions from complex verbal information. The task is to evaluate statements based on the information given on a company specializing in manufacturing consumer goods. Thus the test reflects a business context. For the study, the researchers designed a test that was equivalent in terms of format and display, but the content was fantasy/science fiction – “Planet Zorax”, the description of a fictitious planet.

In the study, candidates were told to imagine they were taking two tests in a high stakes situation and completed both tests, the one with the business context (scales verbal (consumer compact)) and the one with the fantasy/science fiction context (Planet Zorax). After each test they completed a questionnaire assessing social acceptance, AKZEPT! -L (Kersting, 2008). It assesses four dimensions of social acceptance: appropriateness, controllability, quality of measurement, and face validity. Additionally candidates completed questionnaires assessing personality and test anxiety.

What the researchers found was:

  • Overall, there was NO difference between the two tests in social acceptance. However, participants considered scales verbal (consumer compact), the test with the business context, to be more face valid than Planet Zorax, the fantasy/science fiction test.
  • Neither objective nor perceived performance had an impact on social acceptance, so candidates who performed poorly or thought they had performed poorly accepted the tests just as much as candidates who performed well.
  • When comparing candidates who are high vs. low on test anxiety it turned out that there was NO difference in their ratings on the dimensions appropriateness, controllability, and face validity. Only quality of measurement was rated lower by those who were high on test anxiety than those who were low. Overall again there was no difference in social acceptance between the two groups. The same effects could be seen when comparing generally anxious candidates to generally not anxious ones.
  • There was NO difference in performance on scales verbal (consumer compact) between candidates interested in economics and candidates who weren’t. Similarly, there was no difference in performance on Planet Zorax between candidates who were interested in fantasy or science fiction and candidates who weren’t. This can be seen as proof that the tests require no previous knowledge and that they assess cognitive ability only.

Overall this means that context does not have an impact on performance and that the two contextualised tests require no previous knowledge. They assess cognitive ability. Moreover, context does not impact overall social acceptance, but it does have an impact on face validity. This is important particularly when recruiting in areas where there are few applicants. Anything that positively impacts their perception of the selection process will prevent them from dropping out. Thus from a standpoint of candidate experience it does make sense to use contextualised instruments.

References:

Kersting, M. (2008). Zur Akzeptanz von Intelligenz- und Leistungstests. Report Psychologie, 33, 420-433.

About the Author

Katharina Lochner

Dr Katharina Lochner is the former research director for the cut-e Group which was acquired by Aon in 2017. Katharina is now a researcher and lecturer at the University of Applied Sciences Europe in Iserlohn, Germany. In her role at cut-e, she applied the research in organizational and work psychology to real-world assessment practice. She has a strong expertise in the construction and evaluation of online psychometric tools.

Follow on Linkedin Visit Website More Content by Katharina Lochner
Previous Article
How a Smile Changes How You See the World
How a Smile Changes How You See the World

NEXT FEATURE
Collaborating Remotely – Because of Or In Spite of Communication Technology?
Collaborating Remotely – Because of Or In Spite of Communication Technology?

Subscribe to our talentNews

Subscribe