Measuring the Quality of Physician Practice by Using Clinical Vignettes: A Prospective Validation Study

Peabody, John W.; Luck, Jeff; Glassman, Peter; Jain, Sharad; Hansen, Joyce; Spell, Maureen; Martin Lee
November 2004
Annals of Internal Medicine;11/16/2004, Vol. 141 Issue 10, p771
Academic Journal
Background: Worldwide efforts are under way to improve the quality of clinical practice. Most quality measurements, however, are poorly validated, expensive, and difficult to compare among sites. Objective: To validate whether vignettes accurately measure the quality of clinical practice by using a comparison with standardized patients (the gold standard method), and to determine whether vignettes are a more or less accurate method than medical record abstraction. Design: Prospective, multisite study. Setting: Outpatient primary care clinics in 2 Veterans Affairs medical centers and 2 large, private medical centers. Participants: 144 of 163 eligible physicians agreed to participate, and, of these, 116 were randomly selected to see standardized patients, to complete vignettes, or both. Measurements: Scores, expressed as the percentage of explicit quality criteria correctly completed, were obtained by using 3 methods. Results: Among all physicians, the quality of clinical practice as measured by the standardized patients was 73% correct (95% CI, 72.1% to 73.4%). By using exactly the same criteria, physicians scored 68% (CI, 67.9% to 68.9%) when measured by the vignettes but only 63% (CI, 62.7% to 64.0%) when assessed by medical record abstraction. These findings were consistent across all diseases and were independent of case complexity or physician training level. Vignettes also accurately measured unnecessary care. Finally, vignettes seem to capture the range in the quality of clinical practice among physicians within a site. Limitations: Despite finding variation in the quality of clinical practice, we did not determine whether poorer quality translated into worse health status for patients. In addition, the quality scores are based on measurements from 1 patient-provider interaction. As with all other scoring criteria, vignette criteria must be regularly updated. Conclusions: Vignettes are a valid tool for measuring the quality of clinical practice. They can be used for diverse clinical settings, diseases, physician types, and situations in which case-mix variation is a concern. They are inexpensive and easy to use. Vignettes are particularly useful for comparing quality among and within sites and may be useful for longitudinal evaluations of interventions intended to change clinical practice.


Related Articles

  • MORE PHYSICIANS USING ELECTRONIC MEDICAL RECORDS.  // Indian Journal of Medical Sciences;Jul2006, Vol. 60 Issue 7, p308 

    The article reports on the percentage of physicians in the U.S. who used electronic medical records in keeping records of their patients. A survey on health statistics, which deals on the use of electronically operated gadgets in keeping records in medical community revealed that a growing trend...

  • Differences in Presenting Advance Directives in the Chart, in the Minimum Data Set, and Through the Staff's Perceptions. Cohen-Mansfield, Jiska; Libin, Alexander; Lipson, Steven // Gerontologist;Jun2003, Vol. 43 Issue 3, p302 

    Presents information on a study which clarified how advance directives are summarized in the medical record of the patient and how physicians perceive the same advance directives and formal orders. Prevalence of advance directives; Rate of agreement between sources; Rates of agreement between...

  • Application of Portable CDA for Secure Clinical-document Exchange. Kuo-Hsuan Huang; Sung-Huai Hsieh; Yuan-Jen Chang; Feipei Lai; Sheau-Ling Hsieh; Hsiu-Hui Lee // Journal of Medical Systems;Aug2010, Vol. 34 Issue 4, p531 

    Health Level Seven (HL7) organization published the Clinical Document Architecture (CDA) for exchanging documents among heterogeneous systems and improving medical quality based on the design method in CDA. In practice, although the HL7 organization tried to make medical messages exchangeable,...

  • good documentation: what it means for your physicians--and your revenue cycle. Swindle, Jennifer // hfm (Healthcare Financial Management);Sep2006, Vol. 60 Issue 9, p68 

    The article discusses the importance of E/M documentation to the physicians. Educating physicians and providing them with customized tools can significantly improve documentation accuracy. The goal of this E/M coding is to have an accurate medical record of the patients so that malpractice...

  • Hope And Hype: Predicting The Impact Of Electronic Medical Records. Himmeistein, David U.; Woolhandler, Steffie // Health Affairs;Sep/Oct2005, Vol. 24 Issue 5, p1121 

    The current fascination with electronic medical records (EMRs) is not new. For decades, vendors have capitalized on this enthusiasm. But hospitals and clinics have ended up with little to show for their large outlays. Indeed, computing at a typical hospital has not gotten much beyond what was...

  • The Cure for the Fatally Flawed EMR Software Model. Bush, Jonathan // Health Management Technology;Aug2006, Vol. 27 Issue 8, p40 

    The article focuses on the need for the U.S. healthcare industry to examine the current packaged software approach to electronic medical records systems (EMR), to ensure that people understand why the market demands the technology and to investigate other routes to achieving the benefits of...

  • Partners in Quality. Murphy, Dan // Health Management Technology;May2009, Vol. 30 Issue 5, p21 

    The article discusses the implementation of electronic prescribing (e-prescribing) at Heritage Valley Health System. A grant had been received by the health system from the Highmark Health information Technology Grant Program for the implementation of e-prescribing. The health system integrated...

  • Improving medicine usage through patient information leaflets in India. Gupta, Usha; Sharma, Sangeeta; Sheth, Prafull D.; Jha, Jeevan; Chaudhury, Ranjit Roy // Tropical Doctor;Jul2005, Vol. 35 Issue 3, p164 

    The impact of patient information leaflets (PILs) was measured in an outpatient department of a large tertiary care public health facility in India. Patients were allocated to control (114) or intervention group (118). Patients in the intervention group received PILs. The primary indicators...

  • Among medicine's most enduring myths. Monette, Michael // CMAJ: Canadian Medical Association Journal;9/18/2012, Vol. 184 Issue 13, pE705 

    The article discusses the general trend of patients in Canada to seek medical opinion for the second time. It informs that patients go for second medical opinion because they feel that their questions are not answered completely or the physician overlooked some information in his diagnosis....


Read the Article


Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics