The Theory & Research Behind the Situational Judgment Test

Theory

Our situational judgment test (SJT) is designed with theoretical rigor and practical value in order to capture the personal attributes of prospective teachers. Our teacher selection model (see illustration) is being developed through a deductive (top-down) approach guided by theory and research, and an inductive (bottom-up) approach guided by expert panels and job analysis.

Our research draws on interdisciplinary educational and occupational psychology frameworks to develop an evidence-based teacher selection approach. In our model, a teacher’s personal attributes or non-cognitive competencies—their motives, traits, beliefs, and attitudes—have a causal relationship on teaching behaviours. SJTs can be used to tap into a teacher’s personal attributes, and are founded on the notion that situation-specific judgments and responses reflect implicit non-cognitive competencies that have a causal effect on job performance (i.e., teaching behaviours).

Motivation Theories Diagram

In this model we have included three illustrative motivation theories—self-efficacy theory, basic psychological needs mini-theory from self-determination theory, and teachers’ goal orientation (Butler, 2012). Other theories such as Duckworth’s theory of self-control and grit, Sternberg’s successful intelligence theory or, Keller and colleagues (2014) teacher enthusiasm, could be included as well.

According to these theoretical foundations, a person’s personality traits and motivation tendencies are often implicit, but expressed through behaviour in real-life situations, or responses to tests built on real-life scenarios that reveal implicit tendencies. As illustrated by the intersecting circles, implicit motives operate outside of conscious awareness, and may be more-or-less aligned with explicit (conscious) motives. A person benefits when implicit and explicit motives, traits, beliefs and attitudes are closely aligned, since they strive on both the conscious and unconscious level towards the same goals.

Governments all around the world are increasingly calling for non-cognitive attributes to be assessed as part of the selection process for teacher education courses. The research literature supports the governments' call as non-cognitive attributes have been shown to play an essential role in predicting positive educational outcomes

Situational judgment tests (SJTs) present applicants with a set of hypothetical work-relevant scenarios reflecting real-life situations applicants will face in the job. SJTs are shown to be superior to conventional personality tests as the most effective format for predicting job performance in a range of professional settings. SJTs are state-of-the-art, cost-effective and well-received by applicants and selectors. The tests are now used in trainee selection in health fields in the UK, but have not yet been developed for trainee selection in education.Governments all around the world are increasingly calling for non-cognitive attributes to be assessed as part of the selection process for teacher education courses. The research literature supports the governments' call as non-cognitive attributes have been shown to play an essential role in predicting positive educational outcomes.

Above: The Situational Judgement Test as a Teacher Selection Tool by Tracy Durksen 

Research

We propose using SJTs as one part of the selection process which may include a range of other information-rich sources, including tests of literacy and numeracy skills, reviews of academic background, and individual and group interviews. In selection for medicine in the UK and Belgium, SJTs do not replace other information sources, but rather complement other tools such as cognitive tests and interviews, showing high levels of predictive validity compared to other tools. In education, there is little validity evidence for current selection procedures. For example, conventional personality tests show a weak relationship with teacher effectiveness, considerably lower than the relationship between non-cognitive attributes measured by SJTs and job performance.

We propose that SJTs might be used as an initial screening tool along with government-mandated assessments of numeracy and literacy skills. The advantage of using SJTs for screening non-cognitive attributes at an early stage is that they show high levels of predictive and incremental validity, fewer inter-group differences, and can be efficiently administered to large numbers of applicants.

Preliminary Pilot Results

We piloted an initial version of our teacher selection SJT with 304 candidates in 2013-2014. Our preliminary pilot results are very encouraging (June 2014). Results show that the SJT is reliable (reliability coefficient > 0.80), valid (significant correlations with 7/8 interview criteria, and is capable of differentiating between applicants (i.e., scores are near-normally distributed). Furthermore, the SJT was reviewed favourably by applicants, with high ratings for fairness and relevance (Klassen, Durksen, Rowett, & Patterson, 2014).