A framework for evidencing assessment validity in large-scale,high-stakes international examinations     

Stuart Shaw  Victoria Crisp  Nat Johnson 《Assessment in Education: Principles, Policy & Practice》2012,19(2):159-176

It is important for educational assessment bodies to demonstrate how they are seeking to meet the demands of validity. The approach to validity taken here assumes a ‘consequentialist’ view where the appropriacy of the inferences made on the basis of assessment results is seen as central. This paper describes the development of a systematic approach to the collection of evidence that can support claims about validity for general qualifications. An operational framework was developed drawing on Kane (2006 Kane, M.T. 2006. “Validation”. In Educational measurement, 4th ed., Edited by: Brennan, R.L. 17–64. Westport, CT: Praeger.  [Google Scholar]). The framework involves a list of inferences to be justified as indicated by a number of linked validation questions. For each question various data would be gathered to provide ‘evidence for validity’ and to identify any ‘threats to validity’. The structure is designed to be accessible for operational users. This paper describes the development of the proposed framework and the types of methods to be used to gather relevant evidence.  相似文献   

Teaching ‘Instant Experience’ with Graphical Model Validation Techniques     

Claus Thorn Ekstrøm 《Teaching Statistics》2014,36(1):23-26

Graphical model validation techniques for linear normal models are often used to check the assumptions underlying a statistical model. We describe an approach to provide ‘instant experience’ in looking at a graphical model validation plot, so it becomes easier to validate if any of the underlying assumptions are violated.  相似文献   

Studying School Architecture: some reflections     

Roy Lowe 《International Journal of Research & Method in Education》2013,36(1):45-52

Abstract

This article reports on the findings of an investigation in which 30 secondary schools rated the priority being given to and the likely impact on teaching and learning of a selection of 23 issues currently on the agenda for school improvement. Results are considered for the whole sample, and according to the role of respondents and school context. Schools are giving high priority to the implementation of national policies and school‐generated initiatives directed at improved classroom practice. However, the perceived impact of externally generated change is less than that of change for which school staff feel a sense of ownership. Policies directed at a consideration of resource issues, professional development and salary changes are accorded low priority and are not considered to have a marked impact on teaching and learning. There is a potential for frustration and loss of morale when policies that are thought to be of limited value within schools require time and sometimes extensive consideration. The article's conclusions for policy‐makers are directed at minimising this risk.  相似文献   

When Assessment Validation Neglects Any Strand of Validity Evidence: An Instructive Example from PISA     

David Pepper 《Educational Measurement》2020,39(4):8-20

The Standards for Educational and Psychological Testing identify several strands of validity evidence that may be needed as support for particular interpretations and uses of assessments. Yet assessment validation often does not seem guided by these Standards, with validations lacking a particular strand even when it appears relevant to an assessment. Consequently, the degree to which validity evidence supports the proposed interpretation and use of the assessment may be compromised. Guided by the Standards, this article presents an independent validation of OECD's PISA assessment of mathematical self-efficacy (MSE) as an instructive example of this issue. OECD identifies MSE as one of a number of “factors” explaining student performance in mathematics, thereby serving the “policy orientation” of PISA. However, this independent validation identifies significant shortcomings in the strands of validity evidence available to support this interpretation and use of the assessment. The article therefore demonstrates how the Standards can guide the planning of a validation to ensure it generates the validity evidence relevant to an interpretive argument, particularly for an international large-scale assessment such as PISA. The implication is that assessment validation could yet benefit from the Standards as what Zumbo calls “a global force for testing”.  相似文献   

Argumentation Surrounding Argument-Based Validation: A Systematic Review of Validation Methodology in Peer-Reviewed Articles     

Matthew Ryan Lavery  Jonathan D. Bostic  Lance Kruse  Erin E. Krupa  Michele B. Carney 《Educational Measurement》2020,39(4):116-130

Since it was formalized by Kane, the argument-based approach to validation has been promoted as the preferred method for validating interpretations and uses of test scores. Because validation is discussed in terms of arguments, and arguments are both interactive and social, the present review systematically examines the scholarly arguments which appear in 83 papers on argument-based validation methods published in peer-reviewed journals. Findings suggest that scholars generally agree on the nature and importance of argument-based validation but disagree on whether validation should be structured or unstructured, formal or informal. Implications are discussed, including promotion of the Standards for Educational and Psychological Testing (AERA, APA, and NCME) as a foundation for consensus in the field.  相似文献   

French Adaptation of the Short Form Youth Experiences Survey for Sport and Life Skills Scale for Sport     

Camille Sabourin  Christiane Trottier  Eric Frenette  Martin Camiré  Vicky Drapeau  Claude Goulet 《Measurement in physical education and exercise science》2020,24(1):33-48

ABSTRACT

Research on positive youth development and life skills in sport has gained considerable attention over the past decades. However, there is a lack of translated questionnaires examining youth development and life skills available to non-English researchers and practitioners. The purpose of this research project was to adapt in French two questionnaires and to accumulate validity evidence of these adaptations. The short form – Youth Experiences Survey for Sport and the Life Skills Scale for Sport were adapted using a procedure based on a transcultural validation methodology applied to psychological measures. This research comprises four studies, which led to the French adaptation and validation process of each questionnaire. In conclusion, the results obtained show that these adapted French questionnaires can now be used to evaluate the perceptions and lived experiences of French-speaking athletes regarding positive youth development and life skills in sport.  相似文献   

What does ratemyprofessors.com actually rate?     

Dennis E. Clayson 《Assessment & Evaluation in Higher Education》2014,39(6):678-698

This research looks closely at claims that ratemyprofessors.com creates a valid measure of teaching effectiveness because student responses are consistent with a learning model. While some evidence for this contention was found in three datasets taken from the site, the majority of the evidence indicates that the instrument is biassed by a halo effect, and creates what most accurately could be called a ‘likeability’ scale.  相似文献   

高校图书馆参与科学数据验证的前景分析     

黄如花  李楠 《图书情报工作》2016,60(3):15-20

[目的/意义]当前科学数据造假现象日益显著,高校图书馆作为科学研究活动的重要信息服务机构,参与科学数据验证有利于提高科学数据质量,净化学术环境。[方法/过程]详细分析高校图书馆参与科学数据验证的必要性和可行性,提出高校图书馆参与科学数据验证的具体途径,并对国外高校图书馆参与科学数据验证的成功案例进行研究。[结果/结论]高校图书馆具有参与科学数据验证的历史机遇和优势,可以从学科馆员到数据馆员、提供科学数据验证服务、参与科学数据管理的整个生命周期3条途径拓展科学数据验证业务。  相似文献   

Vertical jumping biomechanical evaluation through the use of an inertial sensor-based technology     

I. Setuain  J. Martinikorena  M. Gonzalez-Izal  A. Martinez-Ramirez  M. Gómez  J. Alfaro-Adrián 《Journal of sports sciences》2016,34(9):843-851

Progress in micro-electromechanical systems has turned inertial sensor units (IUs) into a suitable tool for vertical jumping evaluation. In total, 9 men and 8 women were recruited for this study. Three types of vertical jumping tests were evaluated in order to determine if the data provided by an IU placed at the lumbar spine could reliably assess jumping biomechanics and to examine the validity of the IU compared with force plate platform recordings. Robust correlation levels of the IU-based jumping biomechanical evaluation with respect to the force plate across the entire analysed jumping battery were found. In this sense, significant and extremely large correlations were found when raw data of both IU and force plate-derived normalised force–time curves were compared. Furthermore, significant and mainly moderate correlation levels were also found between both instruments when isolated resultant forces’ peak values of predefined jumping phases of each manoeuvre were analysed. However, Bland and Altman graphical representation demonstrated a systematic error in the distribution of the data points within the mean ±1.96 SD intervals. Using IUs, several biomechanical variables such as the resultant force–time curve patterns of the three different vertical jumps analysed were reliably measured.  相似文献   

Intercultural Communication Competence Scale: Invariance and Construct Validation in Portugal     

Gabriela Gonçalves  Lily A. Arasaratnam-Smith  Natércia Rodrigues  Raquel Carvalheiro 《Journal of Intercultural Communication Research》2020,49(3):242-262

ABSTRACT

This article explores the psychometric validation of an Intercultural Communication Competence (ICC) scale. Three studies were performed: (1) psychometric evidence pertaining to the scale’s reliability and factor structure and an analysis of the measurement invariance; (2) assessment of convergent and predictive validity; and (3) analyzes the association between ICC and affective and cognitive dimensions of cultural empathy. A three-factor model with eight items yielded the best fit to the data. The scale showed non-invariance between genders, but proved to be a predictor of intercultural contact. Convergent validity were demonstrated.  相似文献