A comparison of the approaches of generalizability theory and item response theory in estimating the reliability of test scores for testlet-composed tests |
| |
| Authors: | Guemin Lee and In-Yong Park |
| |
| Institution: | (1) Department of Education, Yonsei University, 134 Shinchon-Dong Seodaemoon-Ku, Seoul, South Korea |
| |
| Abstract: | Previous assessments of the reliability of test scores for testlet-composed tests have indicated that item-based estimation
methods overestimate reliability. This study was designed to address issues related to the extent to which item-based estimation
methods overestimate the reliability of test scores composed of testlets and to compare several estimation methods for different
measurement models using simulation techniques. Three types of estimation approach were conceptualized for generalizability
theory (GT) and item response theory (IRT): item score approach (ISA), testlet score approach (TSA), and item-nested-testlet
approach (INTA). The magnitudes of overestimation when applying item-based methods ranged from 0.02 to 0.06 and were related
to the degrees of dependence among within-testlet items. Reliability estimates from TSA were lower than those from INTA due
to the loss of information with IRT approaches. However, this could not be applied in GT. Specified methods in IRT produced
higher reliability estimates than those in GT using the same approach. Relatively smaller magnitudes of error in reliability
estimates were observed for ISA and for methods in IRT. Thus, it seems reasonable to use TSA as well as INTA for both GT and
IRT. However, if there is a relatively large dependence among within-testlet items, INTA should be considered for IRT due
to nonnegligible loss of information. |
| |
| Keywords: | |
| 本文献已被 SpringerLink 等数据库收录！ |
|
