The potential of anchoring vignettes to increase intercultural comparability of non-cognitive factors |
| |
| Authors: | Tamara Marksteiner Susanne Kuger Eckhard Klieme |
| |
| Institution: | 1. Department of Psychology, University of Mannheim, Mannheim, Germanyt.marksteiner@uni-mannheim.de https://orcid.org/0000-0002-8522-8957;3. Department of Educational Quality and Evaluation, German Institute for International Educational Research (DIPF), Mannheim, Germanyhttps://orcid.org/0000-0002-7874-7556;4. Department of Educational Quality and Evaluation, German Institute for International Educational Research (DIPF), Mannheim, Germanyhttps://orcid.org/0000-0003-0728-4950 |
| |
| Abstract: | ABSTRACTWe investigate whether Anchoring Vignettes (AV) improve intercultural comparability of non-cognitive student-directed factors (e.g., procrastination). So far, correlation analyses for anchored and non-anchored scores with a criterion have been used to demonstrate the effectiveness of AV in improving data quality. However, correlation analyses are often used to investigate external validity of a scale. Nonetheless, before testing for validity, the reliability of the measurement of a construct should be examined. In the present study, we tested for measurement invariance across countries and languages and compared anchored and non-anchored student-directed self-reports that are highly relevant for the students’ self and their behaviour and performance. In addition, we apply further criteria for testing reliability. The results indicate that the data quality for some of the constructs can – in fact – be improved slightly by anchoring; whereas, for other self-reports, anchoring is less successful than was hoped. We discuss with regard to possible consequences for research methodology. |
| |
| Keywords: | Anchoring vignettes measurement invariance culture-specific responses multi-group confirmatory factor analysis |
|
|
![ORCID Icon]({"type":"image","src":"/templates/jsp/images/orcid.png"} "ORCID Icon")
https://orcid.org/0000-0002-8522-8957;3. Department of Educational Quality and Evaluation, German Institute for International Educational Research (DIPF), Mannheim, Germany