ROI of assessment: in practice

February 21, 2018 Davide Cannata

ROI of assessment

How to work out the ROI of assessment

In our recent blog post we looked at how the Taylor-Russell approach helps us to predict those most likely to succeed in a job role and to understand the ROI of assessment. In this article we look at a worked example to see how this approach may be applied in practice.

Case study: The bartenders

An international hostel chain – let’s call it Sleepnomore – is starting a recruitment campaign  to hire 100 new bartenders in their new hostels in Cuba and the Caribbean.

The job itself is relatively simple – and most open-minded young people who speak good English and Spanish could be suited to the job. The company has calculated that around 80% of its candidate pool could be a good bartender after a 2-week training period. However, they’ve had bad experiences in the past with unreliable people that have been found drunk at work or stealing cash. Sleepnomore has calculated that on average, each of those problematic hires has cost the company around €3,000. Sleepnomore has received 500 CVs for this position and has been in contact with an HR consultancy that has proposed three different packages for assessing people, with different costs and validity.

The question is, which package, if any, should Sleepnomore choose?

Procedure
To solve the problem, we need to use the Taylor-Russell tables three times (once for each package) to see how many ‘good’ employees will be selected.  We will calculate the false positive (F+) rate we expect using each method. This will be equal to 1-the number we find in the Taylor-Russell table.

1. Base Rate: This is .8 for all the packages
2. Benchmark: This is equal to applicants selected/total applicants = 100/500, so .2 (for all the packages)
3. Validity: Different per each package (.25,.45,.65)

No package bought
F+(P0-No package bought) = 1-.08=.2
The validity in this case is equal to zero. Therefore, the number of poor performers selected will consist of the same percentage as in the general population, equal to 1-.8 (satisfactory employees) = .2

Package 1 bought
F+(P1)
= 1-.89= .11
The base rate of suitable employees is .8. Therefore, we should have a look at the 9th table in the Taylor and Russell article. The selection ratio is .20 (1 out of 5 candidates is hired) and the predictive validity is .25. We can then use these 2 values in the table to find the extent of the true positives: .89. The false positive will be the rest of the selected employees = 1-.89 = .11 (11%)

Package 2 bought
F+(P2)
= 1-.95=.05

Everything here is the same except that the predictive validity is .45. Therefore, the correct match is .95 and this represents the proportion of true positives. False positives will be 1-.95, = .05

Package 3 bought
F+(P3)=
1-.98=.02

Again, predictive validity is the only thing to change and it goes up to .65. The new match is .98, and therefore false negatives will be 1-.98= .02

Now we need to calculate the expected cost of each of these options. The expected cost (EC) is: Cost of selection + Cost per bad hires*number of bad hires.

No package: TC(P0) = 0 + 3,000 * 20 = €60,000

Package 1: TC(P1) = 10,000 + 3,000 * 11 = €43,000

Package 2: TC(P2) = 20,000 + 3,000 * 5 = €35,000

Package 3: TC(P3) = 30,000 + 3,000 * 2 = €36,000

 

The result
The best package on offer from the HR Consultancy is the second (Online integrity test + structured interview).

The total cost of hires with this procedure is expected to be €35,000. Sleepnomore could potentially save around €25,000 of the €60,000 lost if it had not used any selection procedure.

 

References:
Taylor, H. C. & Russell, J. T. (1939). The relationship of validity coefficients to the practical effectiveness of tests in selection: discussion and tables. Journal of Applied Psychology, 23(5), 565.

About the Author

Davide Cannata is the head of people analytics within Aon's Assessment Solutions. He is a work and organisational psychologist with advanced skills in statistics and data analysis, as well as project management accreditation. Davide has expertise and experience in the design and application of psychometric tests and assessment centers for organizations across the world. His experience spans a range of sectors including retail, automotive, aviation, telecommunication, financial services, consultancy and pharmaceuticals. Aon's Assessment Solutions undertake 30 million assessments each year in 90 countries and 40 languages.

Follow on Linkedin Visit Website More Content by Davide Cannata
Previous Article
How to ask for – and give – feedback on a creative project
How to ask for – and give – feedback on a creative project

Explores how best to ask for and give feedback on creative projects.Suggests how to ask the right questions...

NEXT FEATURE
Stress: does our view of stress influence how we cope with it?
Stress: does our view of stress influence how we cope with it?

How does how we view stress impact the way in which we deal with stress? Explores how our stress mind set a...

Subscribe to our talentNews

Subscribe