How Cognitive Interviewing can Provide Validity Evidence of the Response Processes to Scale Items

Castillo-Díaz, Miguel; Padilla, José-Luis
December 2013
Social Indicators Research;Dec2013, Vol. 114 Issue 3, p963
Academic Journal
The current theory about validity reflected in the Standards for Educational and Psychological Testing (AERA et al. in Standards for educational and psychological testing, American Psychological Association, Washington, DC, ), offers no clear indications about the methods for gathering validity evidence about the response processes. Cognitive interviewing (CI) can play an important role answering the current demand about empirical and theoretical analyses of the response processes as a source of validity evidence in psychological testing. CI can provide validity evidence for investigating substantive aspects of construct validity and for contributing to the explanations for item and test scores (Zumbo in Handbook of statistics, vol 26, Elsevier, Amsterdam, pp. 45-79, ; The concept of validity: revisions, new directions and applications, IAP-Information Age Publishing Inc., Charlotte, NC, pp. 65-82, ). The aim of the study was to illustrate the use of cognitive interviewing method for gathering validity evidence on response processes. The search for evidence about the 'response process' was guided by an argument-based approach to validity (Kane in Psychological Bulletin ; Educational measurement, American Council on Education/Praeger, Washington, DC, pp. 17-64, ). 21 cognitive interviews were carried out during the cognitive testing of the APGAR psychological scale intended to measure the 'family support' construct. Cognitive interviewing provided validity evidence that explains how respondents interpret and respond to the APGAR items. Respondents maintained a shared interpretation of 'family concept' while answering the APGAR scale items. Nevertheless, they included in the concept of family not only family members they live with but also other family members and even friends. CI participants were also capable of classifying their answers about the family support perception following a polythomous response system. Lastly, the role of CI in the Kane's argument-based approach and Zumbo's contextualized view of validity will be discussed.


Related Articles

  • Validity Semantics in Educational and Psychological Assessment. Hathcoat, John D. // Practical Assessment, Research & Evaluation;Jun2013, Vol. 18 Issue 9, p1 

    The semantics, or meaning, of validity is a fluid concept in educational and psychological testing. Contemporary controversies surrounding this concept appear to stem from the proper location of validity. Under one view, validity is a property of score-based inferences and entailed uses of test...

  • Guest Editorial: Performance contracts and test errors. Stake, Robert E.; Wardrop, James L. // Reading Research Quarterly;Spring1971, Vol. 6 Issue 3, p323 

    The article comments on performance contracts and errors in the gain scores of children in a reading program in the U.S. Performance contracts pay off on an individual-student basis. The student is to be tested, taught, and retested with a commercially published achievement test. Scores can be...


    The Confluence Score (CON) is developed using 22 pairs of items from the test that are opposite or nearly opposite; so that if one item is marked true or false, and the second item in the pair is not scored in the opposite direction, the answers fail to agree with fact. The CON score is first an...

  • EMPIRICAL DATA ON THE SCORING OF TRUE-FALSE TESTS. Paterson, Donald G.; Langlie, T. A. // Journal of Applied Psychology;Aug25, Vol. 9 Issue 4, p339 

    The article examines the reliability and validity of true-false tests that utilize the traditional right minus wrong scoring formula. It tests the following assumptions by psychologists: guessing enters into true-false tests; the effect of guessing is counteracted by a right minus wrong scoring...

  • Fair, Reliable, Valid: Developing a Grammar Test Utilizing the Four Building Blocks. Efeotor, Voke // Arab World English Journal;2014, Vol. 5 Issue 4, p203 

    The purpose of this study is to develop and analyse a grammar test, focusing on validity, reliability and fairness. Effective test development requires a systematic, well-organized approach, thus ensuring the veracity of the proposed inferences from the test scores, (Downing, 2006a). This...

  • How to recheck an SAT score.  // Medical Economics;5/5/2006, Vol. 83 Issue 9, p17 

    The article provides information how to recheck the Scholastic Achievement Test score of one's high schooler. The College Board suggests the interested party to obtain a copy of the examinee's answers and to buy the Question-and-Answer Service (QAS) that costs $24. The QAS shows the copy of the...

  • The Assessment-Savvy Student. Popham, W. James // Educational Leadership;Nov2008, Vol. 66 Issue 3, p80 

    The article discusses the value of explaining the nature of educational assessments to students. The author suggests that students must understand assessment methodology to achieve educational independence. He recommends that students be made to understand limitations regarding the validity of...

  • THE INTERPRETATION OF SCORES IN TESTS AND EXAMINATIONS. Courtis, S. A. // Education Digest;Oct1938, Vol. 4 Issue 2, p18 

    The article presents a reprint of the article "The interpretation of scores in tests and examinations," by S.A. Courtis, which appeared in the May 1938 issue of the "Journal of Educational Research". The purpose of the article is to generate doubts about the conventional attitudes toward...

  • The Reliability, Validity, and Utility of Self-Assessment. Ross, John A. // Practical Assessment, Research & Evaluation;2006, Vol. 11, p1 

    Despite widespread use of self-assessment, teachers have doubts about the value and accuracy of the technique. This article reviews research evidence on student self-assessment, finding that (1) self-assessment produces consistent results across items, tasks, and short time periods; (2)...


Read the Article

Courtesy of

Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics