Embedded Field Test Item Statistics: Can They Be Trusted for Estimating Student Proficiency? |
| |
| Authors: | Jeffrey T Steedle Kristin M Morrison |
| |
| Institution: | 1. Pearson, Austin, TX, USA;2. Georgia Institute of Technology, Atlanta, GA, USA |
| |
| Abstract: | Assessment items are commonly field tested prior to operational use to observe statistical item properties such as difficulty. Item parameter estimates from field testing may be used to assign scores via pre-equating or computer adaptive designs. This study examined differences between item difficulty estimates based on field test and operational data and the relationship of such differences to item position changes and student proficiency estimates. Item position effects were observed for 20 assessments, with items in later positions tending to be more difficult. Moreover, field test estimates of item difficulty were biased slightly upward, which may indicate examinee knowledge of which items were being field tested. Nevertheless, errors in field test item difficulty estimates had negligible impacts on student proficiency estimates for most assessments. Caution is still warranted when using field test statistics for scoring, and testing programs should conduct investigations to determine whether the effects on scoring are inconsequential. |
| |
| Keywords: | |
|
|
