How to work out the ROI of assessment
In our recent blog post we looked at how the Taylor-Russell approach helps us to predict those most likely to succeed in a job role and to understand the ROI of assessment. In this article we look at a worked example to see how this approach may be applied in practice.
Case study: The bartenders
An international hostel chain – let’s call it Sleepnomore – is starting a recruitment campaign to hire 100 new bartenders in their new hostels in Cuba and the Caribbean.
The job itself is relatively simple – and most open-minded young people who speak good English and Spanish could be suited to the job. The company has calculated that around 80% of its candidate pool could be a good bartender after a 2-week training period. However, they’ve had bad experiences in the past with unreliable people that have been found drunk at work or stealing cash. Sleepnomore has calculated that on average, each of those problematic hires has cost the company around €3,000. Sleepnomore has received 500 CVs for this position and has been in contact with an HR consultancy that has proposed three different packages for assessing people, with different costs and validity.
The question is, which package, if any, should Sleepnomore choose?
To solve the problem, we need to use the Taylor-Russell tables three times (once for each package) to see how many ‘good’ employees will be selected. We will calculate the false positive (F+) rate we expect using each method. This will be equal to 1-the number we find in the Taylor-Russell table.
1. Base Rate: This is .8 for all the packages
2. Benchmark: This is equal to applicants selected/total applicants = 100/500, so .2 (for all the packages)
3. Validity: Different per each package (.25,.45,.65)
No package bought
F+(P0-No package bought) = 1-.08=.2
The validity in this case is equal to zero. Therefore, the number of poor performers selected will consist of the same percentage as in the general population, equal to 1-.8 (satisfactory employees) = .2
Package 1 bought
F+(P1) = 1-.89= .11
The base rate of suitable employees is .8. Therefore, we should have a look at the 9th table in the Taylor and Russell article. The selection ratio is .20 (1 out of 5 candidates is hired) and the predictive validity is .25. We can then use these 2 values in the table to find the extent of the true positives: .89. The false positive will be the rest of the selected employees = 1-.89 = .11 (11%)
Package 2 bought
F+(P2) = 1-.95=.05
Everything here is the same except that the predictive validity is .45. Therefore, the correct match is .95 and this represents the proportion of true positives. False positives will be 1-.95, = .05
Package 3 bought
Again, predictive validity is the only thing to change and it goes up to .65. The new match is .98, and therefore false negatives will be 1-.98= .02
Now we need to calculate the expected cost of each of these options. The expected cost (EC) is: Cost of selection + Cost per bad hires*number of bad hires.
No package: TC(P0) = 0 + 3,000 * 20 = €60,000
Package 1: TC(P1) = 10,000 + 3,000 * 11 = €43,000
Package 2: TC(P2) = 20,000 + 3,000 * 5 = €35,000
Package 3: TC(P3) = 30,000 + 3,000 * 2 = €36,000
The best package on offer from the HR Consultancy is the second (Online integrity test + structured interview).
The total cost of hires with this procedure is expected to be €35,000. Sleepnomore could potentially save around €25,000 of the €60,000 lost if it had not used any selection procedure.
Taylor, H. C. & Russell, J. T. (1939). The relationship of validity coefficients to the practical effectiveness of tests in selection: discussion and tables. Journal of Applied Psychology, 23(5), 565.
About the AuthorFollow on Linkedin Visit Website More Content by Davide Cannata