Eye in the Sky (aka EITS) is an Online Remote Proctoring solution built by us (Aon Assessment Solutions, India). Online Remote Proctoring is a way to proctor a candidate from a remote place by a system or a user.
Everyone is moving towards online virtual systems and assessment is no exception. Assessment is also going through many changes - pen and paper system being replaced by online tests, and a proctor or supervisor in a class room being replaced by Online Remote Proctoring system like Eye In The Sky (EITS).
While many corporates have already replaced their hiring assessments with online assessments, educational institutes are still doubtful about moving to online assessments for their class tests and semester exams. COVID-19 situation has left little options for educational institutes, but they are not shying away from raising concerns about online assessments, especially regarding Online Remote Proctoring Solutions.
Some of the concerns are valid from a candidate's point of view who are taking online assessment for jobs or exams. However, few of these concerns are ignored by the remote proctoring solutions because these solutions are built keeping proctors' need in mind, not the candidates’. These systems are designed in a way to doubt every move of a candidate but it only adds to the anxiety that the candidate is going through under stress of the tests.
Hence, I feel that there are a few principles to keep in mind while building a remote proctoring solution.
Focus on Candidates
Assessments are to measure skills of the candidates and candidates are the ones being monitored. Any kind of monitoring system makes humans nervous, and it is natural for candidates to be nervous knowing that they are being monitored. So,
- Avoid displaying unwarranted messages or pop-ups
- Keep the frequency of such messages to minimum.
- Gather all the messages and display them after a while at once rather than disturbing candidates too often.
Trust your Candidates
Most candidates take assessment to pass the exam or take the job. They do not indulge in cheating, especially when they know that they are being monitored.
- Give your candidates benefit of the doubt whenever you are not sure.
- Design system to study every signals carefully before flagging the candidate.
- Do not treat every signal equally, assign weight to different signals based on real life possibilities
Save minimum data
Data privacy is an important thing and users are well-aware of it. More data of storage means more information of user available, and it can be used for purpose other than proctoring too. So,
- System should have the ability to extract maximum information related to proctoring while storing minimum required data.
- System should be designed to process data in real time and save only the conclusion rather than saving all the data first and process it later.
- Security access around any such data should be adequate for obvious reasons.
Usability and Accessibility
Online Remote Proctoring solution should be very light and compatible across browsers and devices. Internet bandwidth is still a major issue in tier-2 and tier-3 cities.
- System should be capable of running on a low internet bandwidth as well, so that it does not disturb the functioning of the assessment.
- System must be light in size with optimized processing, so it can run easily on devices with lower CPU configuration across browsers. If not designed properly, proctoring solution may end up freezing candidate browsers and thus interrupting their assessment.
Eye In The Sky
While designing EITS, we kept all above principles in mind, and we can safely say that it has worked very well so far based on the feedback from candidates. EITS is built using lot of smart algorithms that help perform complex computations on candidate data before arriving to any conclusion.
- EITS is designed to have fewest possible pop up messages on candidate screen related to proctoring. If a candidate can focus on assessment well, candidate has higher chances of scoring well and we have a higher chance of finding a better candidate. It is more important to focus on good performing candidates than on candidates indulging in malpractice.
- EITS has set of algorithms for all types of malpractice indicators that study each signal and filter out unclear signals before flagging the user. Technology does not have the ability to understand context of the user action, so EITS combines technology inputs with specially designed algorithms to safeguard candidates from false positives.
- EITS stores derived information only related to proctoring rather than storing raw data. For all the webcam and ML/AI based features, EITS stores only images of the candidates which are taken at some intervals randomly for an audit purpose with consent of the candidate.
In the End
This technical field is an evolving field where lot of ideas are floating around and systems are improving continuously. But this field also touches the sensitive topic of skill measurement where end outcome can decide the fate of a candidate. Hence, it is important to have the right philosophy behind a technical product.
About the AuthorFollow on Linkedin More Content by Nilay Kothari