What is Sequential testing?
Sequential testing is a method of using a screening assessment (sequence 1), which all candidates attempt. The results are processed and candidates who do not achieve the pass mark (this is the sum of all the individual station cut scores) plus 2 Standard Errors of measurement (SEm).
The standard error of measurement (SEm) estimates how repeated measures of a person on the same instrument tend to be distributed around his or her “true” score. The true score is always an unknown, because no measure can be constructed that provides a perfect reflection of the true score.
SEm is directly related to the reliability of a assessment; the larger the SEm, the lower the reliability of the assessment and less precision in the measurement taken and scores obtained. Since all measurement contains some error, it is highly unlikely that any assessment will yield the same scores for a given person each time they are retested.
Candidates who have not achieved a clear pass are required to sit a second assessment containing different stations (sequence 2). These results are combined with sequence 1 and this gives a larger number of stations to assess the candidates on. The candidates are required to achieve the pass mark (sum of all the individual station cut scores) plus 1 SEm to pass the summative OSCE.
The 2015/16 academic year was the first time sequential testing had been introduced in Liverpool. The OSCE reliability for both sequences was 0.78 with only a small number of students failing both sequences which was less than previous years sitting the traditional resit assessment.
Why the move to Sequential testing?
The OSCE format has been shown to be both reliable and valid, but is expensive to run especially with large cohorts of students. The traditional method of assessment has candidates who fail the OSCE receiving short term remediation, which shows improved performance for the resit attempt. However these candidates do not appear to retain this level of remediation and continue to be around or below the borderline performance levels required to pass.