In the minds of OSCE examiners: uncovering hidden assumptions

Chahine, Saad; Holmes, Bruce; Kowalewski, Zbigniew
August 2016
Advances in Health Sciences Education;Aug2016, Vol. 21 Issue 3, p609
Academic Journal
journal article
The Objective Structured Clinical Exam (OSCE) is a widely used method of assessment in medical education. Rater cognition has become an important area of inquiry in the medical education assessment literature generally, and in the OSCE literature specifically, because of concerns about potential compromises of validity. In this study, a novel approach to mixed methods that combined Ordinal Logistic Hierarchical Linear Modeling and cognitive interviews was used to gain insights about what examiners were thinking during an OSCE. This study is based on data from the 2010 to 2014 administrations of the Clinician Assessment for Practice Program OSCE for International Medical Graduates (IMGs) in Nova Scotia. An IMG is a physician trained outside of Canada who was a licensed practitioner in a different country. The quantitative data were examined alongside four follow-up cognitive interviews of examiners conducted after the 2014 administration. The quantitative results show that competencies of (1) Investigation and Management and (2) Counseling were highly predictive of the Overall Global score. These competencies were also described in the cognitive interviews as the most salient parts of OSCE. Examiners also found Communication Skills and Professional Behavior to be relevant but the quantitative results revealed these to be less predictive of the Overall Global score. The interviews also reveal that there is a tacit sequence by which IMGs are expected to proceed in an OSCE, starting with more basic competencies such as History Taking and building up to Investigation Management and Counseling. The combined results confirm that a hidden pattern exists with respect to how examiners rate candidates. This study has potential implications for research into rater cognition, and the design and scoring of practice-ready OSCEs.


Related Articles

  • Comparison of Differential Item Functioning Determination Techniques: HGLM, LR and IRT-LR. Acar, Tülin; Kelecıoğlu, Hülya // Educational Sciences: Theory & Practice;Spring2010, Vol. 10 Issue 2, p639 

    The aim of this research is to determine differential item functioning (DIF) by hierarchical linear modeling (HGLM) on test items and comparing these results by the DIF results determined by logistic regression (LR) and IRT-likelihood ratio (IRT-LR) techniques. Investigating the concordance...

  • Multilevel logistic regression for polytomous data and rankings. Skrondal, Anders; Rabe-Hesketh, Sophia // Psychometrika;Jun2003, Vol. 68 Issue 2, p267 

    We propose a unifying framework for multilevel modeling of polytomous data and rankings, accommodating dependence induced by factor and/or random coefficient structures at different levels. The framework subsumes a wide range of models proposed in disparate methodological literatures. Partial...

  • Exploring the Hierarchy of Mobility Performance in High-Functioning Older Women. Weiss, Carlos O.; Fried, Linda P.; Bandeen-Roche, Karen // Journals of Gerontology Series A: Biological Sciences & Medical ;Feb2007, Vol. 62 Issue 2, p167 

    Background. Preventing mobility disability depends on matching interventions to individual needs. The purpose of this study is to improve targeting by determining whether mobility performance is associated with, and predicts, mobility disability hierarchically. The hypothesis is that poorer...

  • Inverse probability weighting for clustered nonresponse. Skinner, C. J.; D’arrigo // Biometrika;Dec2011, Vol. 98 Issue 4, p953 

    Correlated nonresponse within clusters arises in certain survey settings. It is often represented by a random effects model and assumed to be cluster-specific nonignorable, in the sense that survey and nonresponse outcomes are conditionally independent given cluster-level random effects. Two...

  • Application of the Rural Development Index to Analysis of Rural Regions in Poland and Slovakia. Michalek, Jerzy; Zarnekow, Nana // Social Indicators Research;Jan2012, Vol. 105 Issue 1, p1 

    The main purpose of this research was to construct a multi-dimensional (composite) index measuring the overall level of rural development and quality of life in individual rural regions of a given EU country. In the Rural Development Index (RDI) the rural development domains are represented by...

  • Sizing up information distortion: Quantifying its effect on the subjective values of choice options. DeKay, Michael; Stone, Eric; Sorenson, Clare // Psychonomic Bulletin & Review;Apr2012, Vol. 19 Issue 2, p349 

    When choosing between options, people often distort new information in a direction that favors their developing preference. Such information distortion is widespread and robust, but less is known about the magnitude of its effects. In particular, research has not quantified the effects of...

  • Accounting for Creativity in the European Union: A multi-level analysis of individual competence, labour market structure, and systems of education and training. Lorenz, Edward; Lundvall, Bengt-Åke // Cambridge Journal of Economics;Mar2011, Vol. 35 Issue 2, p269 

    Some workplaces offer employees opportunities to creatively use their own ideas while engaged in learning and problem-solving activity without much interference from managers and bosses. In this paper we analyse the preconditions for creative work. Using multilevel logistic modelling we examine...

  • Individual and Contextual Factors Associated with Low Childhood Immunisation Coverage in Sub-Saharan Africa: A Multilevel Analysis. Wiysonge, Charles S.; Uthman, Olalekan A.; Ndumbe, Peter M.; Hussey, Gregory D. // PLoS ONE;May2012, Vol. 7 Issue 5, p1 

    Background: In 2010, more than six million children in sub-Saharan Africa did not receive the full series of three doses of the diphtheria-tetanus-pertussis vaccine by one year of age. An evidence-based approach to addressing this burden of unimmunised children requires accurate knowledge of the...

  • Environmental clustering of drug abuse in households and communities: multi-level modeling of a national Swedish sample. Kendler, Kenneth; Ohlsson, Henrik; Sundquist, Kristina; Sundquist, Jan // Social Psychiatry & Psychiatric Epidemiology;Aug2015, Vol. 50 Issue 8, p1277 

    Purpose: Both epidemiological and genetically informative studies indicate that shared environmental influences contribute to resemblance in siblings for drug abuse (DA). To what degree do these influences arise from living in the same household versus residing in the same community? Methods: We...


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics