首页 | 本学科首页   官方微博 | 高级检索  
相似文献
 共查询到20条相似文献，搜索用时 31 毫秒
1.
When we administer educational achievement tests, we want to be confident that the resulting scores validly indicate what the test takers know and can do. However, if the test is perceived as low stakes by the test taker, disengaged test taking sometimes occurs, which poses a serious threat to score validity. When computer-based tests are used, disengagement can be detected through occurrences of rapid-guessing behavior. This empirical study investigated the impact of a new effort monitoring feature that can detect rapid guessing, as it occurs, and notify proctors that a test taker has become disengaged. The results showed that, after a proctor notification was triggered, test-taking engagement tended to increase, test performance improved, and test scores exhibited higher convergent validation evidence. The findings of this study provide validation evidence that this innovative testing feature can decrease disengaged test taking.  相似文献   

2.
Disengaged responding is a phenomenon that often biases observed scores from achievement tests and surveys in practically and statistically significant ways. This problem has led to the development of methods to detect and correct for disengaged responses on both achievement test and survey scores. One major disadvantage when trying to detect disengaged responses on surveys is that, unlike on achievement tests, there are no correct answers. As a result, validating decision rules for detection methods is problematic. In this study, we condition results from a variety of detection methods used to identify disengaged survey responses on response times. We then show how this conditional approach may be useful in identifying where to set response time thresholds for survey items, as well as in avoiding misclassification when using other detection methods.  相似文献   

3.
There has been a growing research interest in the identification and management of disengaged test taking, which poses a validity threat that is particularly prevalent with low‐stakes tests. This study investigated effort‐moderated (E‐M) scoring, in which item responses classified as rapid guesses are identified and excluded from scoring. Using achievement test data composed of test takers who were quickly retested and showed differential degrees of disengagement, three basic findings emerged. First, standard E‐M scoring accounted for roughly one‐third of the score distortion due to differential disengagement. Second, a modified E‐M scoring method that used more liberal time thresholds performed better—accounting for two‐thirds or more of the distortion. Finally, the inability of E‐M scoring to account for all of the score distortion suggests the additional presence of nonrapid item responses that reflect less‐than‐full engagement by some test takers.  相似文献   

4.
Abstract

Work avoidance, or the goal to reduce effort in an achievement context, is related to disengagement and may offer insight into disengaged students and employees. To shed light on this issue, we investigated the antecedents of work avoidance in academic and work contexts to distinguish it from mastery, performance-approach, and performance-avoidance goals. Two samples of employed college students (N?=?207, N?=?233) completed questionnaires about their jobs and in one sample about their introductory psychology class. Across both contexts, proposed antecedents of alienation, low need for achievement, and perceiving one’s competence needs as not being met predicted work avoidance. Work avoidance also accounted for additional variance when combined with achievement goals to predict citizenship behaviors, perceiving work/classwork as meaningless and wanting to leave/quit.  相似文献   

5.
As computer‐based tests become more common, there is a growing wealth of metadata related to examinees’ response processes, which include solution strategies, concentration, and operating speed. One common type of metadata is item response time. While response times have been used extensively to improve estimates of achievement, little work considers whether these metadata may provide useful information on social–emotional constructs. This study uses an analytic example to explore whether metadata might help illuminate such constructs. Specifically, analyses examine whether the amount of time students spend on test items (after accounting for item difficulty and estimates of true achievement), and difficult items in particular, tell us anything about the student's academic motivation and self‐efficacy. While results do not indicate a strong relationship between mean item durations and these constructs in general, the amount of time students spend on very difficult items is highly correlated with motivation and self‐efficacy. The implications of these findings for using response process metadata to gain information on social–emotional constructs are discussed.  相似文献   

6.
Considerable evidence indicates that students’ academic motivation and engagement generally decline as they move through middle school and on to high school. This study applied social cognitive theory to explore how self-efficacy and perceived control—two main factors of personal agency—may play a role in mitigating this decline in engagement and further contribute to academic performance. We used dual change score modeling to examine the dynamic structure of personal agency and disengagement during grades 8–10 for a large sample of students from the Pacific Northwest in the U.S. In that model, we analyzed how those variables predicted grade point average and attendance for students at the end of 10th grade. Students did not necessarily become more disengaged as a result of lower perceptions of control, rather they became more disengaged without the resilience factor of self-efficacy. The actual influence of disengagement on attendance and academic performance appears to be far weaker than the role of personal agency factors. Our results indicate that when student’s self-efficacy drops, disengagement in school increases during the years transitioning to high school. Increased disengagement weakens perceived control and change in both the control and self-efficacy dimensions of personal agency drive academic performance. Schools should prioritize the development of personal agency in each student during the middle school to high school transition years.  相似文献   

7.
We investigate the relationship between teacher licensure test scores and student test achievement and high school course-taking. We focus on three subject/grade combinations—middle school math, ninth-grade algebra and geometry, and ninth-grade biology—and find evidence that a teacher's basic skills test scores are modestly predictive of student achievement in middle school math and highly predictive of student achievement in high school biology. A teacher's subject-specific licensure test scores are a consistent and statistically significant predictor of student achievement only in high school biology. Finally, we find little evidence that students assigned to middle school teachers with higher basic-skills test scores are more likely to take advanced math and science courses in high school.  相似文献   

8.
Whenever the purpose of measurement is to inform an inference about a student’s achievement level, it is important that we be able to trust that the student’s test score accurately reflects what that student knows and can do. Such trust requires the assumption that a student’s test event is not unduly influenced by construct-irrelevant factors that could distort his score. This article examines one such factor—test-taking motivation—that tends to induce a person-specific, systematic negative bias on test scores. Because current measurement models underlying achievement testing assume students respond effortfully to test items, it is important to identify test scores that have been materially distorted by non-effortful test taking. A method for conducting effort-related individual score validation is presented, and it is recommended that measurement professionals have a responsibility to identify invalid scores to individuals who make inferences about student achievement on the basis of those scores.  相似文献   

9.
This paper serves as an illustration of the usefulness of structurally incomplete designs as an approach to reduce the length of educational questionnaires. In structurally incomplete test designs, respondents only fill out a subset of the total item set, while all items are still provided to the whole sample. The scores on the unadministered items are subsequently dealt with by using methods for the estimation of missing data. Two structurally incomplete test designs — one recording two thirds, and the other recording a half of the potentially complete data — were applied to the complete item scores on 8 educational psychology scales. The incomplete item scores were estimated with missing data method Data Augmentation. Complete and estimated test data were compared at the estimates of total scores, reliability, and predictive validity of an external criterion. The reconstructed data yielded estimates that were very close to the values in the complete data. As expected the statistical uncertainty was higher in the design that recorded fewer item scores. It was concluded that the procedure of applying incomplete test designs and subsequently dealing with the missing values is very fruitful for reducing questionnaire length.  相似文献   

10.
Abstract

The effect of changing item responses on scores of elementary school children on a standardized achievement test was studied. Previous research, primarily involving non-standardized instruments and adult samples, indicates that changed responses are more likely to be correct than not. Subjects were 165 third grade students using the Metropolitan Reading Tests. Students received no special instructions regarding changing responses. Changes were identified visually and were independently verified. While frequency of response changes was low, such changes generally improved scores. Sex differences in number and success of changes were non-significant. The relationship between frequency of response change and test score was minimal. Responses to difficult items were changed more frequently with less success than changes on easy items. High scorers made more successful changes than did low scorers. Within the limits of the methodology, results clearly indicated that response changes of elementary students on multiple-choice items tend to improve test scores.  相似文献   

11.
Students' different educational pathways were examined in relation to their disengagement during adolescence. The participants were Icelandic youth (N?=?832) who were followed from age 14 to 22. Based on their academic achievement at age 15 and educational attainment at age 22 they were classified into groups that took expected versus unexpected paths. The findings indicate that adolescents' behavioral disengagement (negative school behaviors) and emotional disengagement (academic disinterest, disidentification with school) differentiated according to their pathways. At age 14, those “at risk” academically who graduated unexpectedly showed fewer negative behaviors than the expected dropouts. Moreover, high achievers who dropped out unexpectedly showed more behavioral (negative behaviors) as well as emotional (academic disinterest, disidentification with school) disengagement compared to expected graduates. The following year (age 15), in general, disengagement increased among unexpected dropouts but decreased among expected graduates. Males and students from lower-SES backgrounds were generally more disengaged, and males from those backgrounds became more emotionally disengaged during their last year in compulsory school.  相似文献   

12.
Abstract

This study demonstrated a procedural model that can be applied by any school to assess, guide, and account for the progress of its students as well as to analyze its own effectiveness. The model uses equivalent achievement tests to monitor student achievement in subject areas at grade levels, between grade levels, and across subgroups of students. Multiple regression analyses of test scores between grades identify factors associated with achievement Using sixth and eighth grade Comprehensive Tests of Basic Skills scores in a matched longitudinal sample of 208 students, the study found small differences in average achievement between boys and girls. Differences between corresponding sixth and eighth grade test means were higher in mathematics than in language. From the sixth grade to the eighth, there was a widening gap in average achievement between high and low I.Q. groups. In multiple regressions of eighth grade test scores on sixth grade measures, I.Q., study skills, and reading were prevalent in the regression equations, but clusters of measures associated with achievement differed between high and low’ LQ. groups. The results of the study have implications for developing and evaluating the achievement of students with varying mental abilities.  相似文献   

13.
Several studies have found that higher hope is associated with higher academic achievement. Although scholars have asserted that this association is the result of higher hope leading to an increased likelihood that students will engage success‐oriented behaviors (e.g., participating in class, completing homework assignments), very little empirical research has been done to test this assertion. In this study, cluster analyses yielded three clusters of hope (high, average, and low) in a high school sample (N = 447) and a college sample (N = 375). Differences among hope clusters were examined across three domains of indicators—engagement, disengagement, and motivational variables—associated with success‐oriented behavior in school. Results from both samples indicated that students with higher levels of hope reported higher engagement, higher motivation, and lower disengagement, with medium to large effect sizes. These results have implications for both hope research and scaling psychosocial interventions.  相似文献   

14.
A vast literature investigates academic disengagement among students, including its ultimate manifestation, dropping out of school. Research also shows that test disengagement can be a problem for many inferences educators and policymakers wish to draw from test scores. However, few studies consider whether academic and test disengagement are related. In this study, we examine whether behaviors indicative of academic disengagement like chronic absenteeism and course failures are related to behaviors indicative of test disengagement like rapidly guessing on items. We also examine whether social-emotional factors like low academic self-efficacy and self-management, which research suggests are the root causes of academic disengagement, are also related to rapid guessing behavior. Our results provide evidence that academic and test disengagement are related, including through a common association with poor self-management. The implications of this connection for measurement and practice are discussed.  相似文献   

15.
The current simulation study examined the effects of Item Parameter Drift (IPD) occurring in a short scale on parameter estimates in multilevel models where scores from a scale were employed as a time-varying predictor to account for outcome scores. Five factors, including three decisions about IPD, were considered for simulation conditions. It was revealed that IPD occurring in a relatively shorter scale led to a substantial increase in the amount of relative bias in parameter estimates. The bias was more prominent in the estimates of level-2 time-varying predictors relative to those of level-1 time-varying predictors. Regarding the decisions about IPD, keeping items exhibiting IPD was more appropriate than removing them based on the results from relative bias of standard errors of estimates. Based on the findings, it can be concluded that removing items exhibiting IPD may lead to an increase of Type II errors due to the underestimation of parameter estimates and overestimation of standard errors. The applied example showed findings consistent with those in the simulation study.  相似文献   

16.
This paper examines the impact of potential fetal malnutrition on the academic test scores of Muslim students in Denmark. We account for the endogeneity of fetal malnutrition by using exposure to the month of Ramadan as a natural experiment under the assumption that mothers of some of the Muslim students might have fasted during Ramadan when they were pregnant. We also complement our Muslim sample with a control group comprised of immigrant children from predominantly non-Muslim countries in a difference-in-differences framework. Our outcome measures are the standardized test scores from the national exams on the subjects of Danish, English, Mathematics, and Science administered by the Danish Ministry of Education. Our results indicate that fetal exposure to Ramadan is likely to have a negative impact on the achievement scores of Muslim students, especially among females. Our analysis further reveals that the estimated relationship is stronger among children with a relatively low socio-economic background. Our findings lend support for the importance of interventions designed to assist economically disadvantaged women during pregnancy.  相似文献   

17.
Item parameter drift (IPD) occurs when item parameter values change from their original value over time. IPD may pose a serious threat to the fairness and validity of test score interpretations, especially when the goal of the assessment is to measure growth or improvement. In this study, we examined the effect of multidirectional IPD (i.e., some items become harder while other items become easier) on the linking procedure and rescaled proficiency estimates. The impact of different combinations of linking items with various multidirectional IPD on the test equating procedure was investigated for three scaling methods (mean-mean, mean-sigma, and TCC method) via a series of simulation studies. It was observed that multidirectional IPD had a substantive effect on examinees' scores and achievement level classifications under some of the studied conditions. Choice of linking method had a direct effect on the results, as did the pattern of IPD.  相似文献   

18.
The rise of computer‐based testing has brought with it the capability to measure more aspects of a test event than simply the answers selected or constructed by the test taker. One behavior that has drawn much research interest is the time test takers spend responding to individual multiple‐choice items. In particular, very short response time—termed rapid guessing—has been shown to indicate disengaged test taking, regardless whether it occurs in high‐stakes or low‐stakes testing contexts. This article examines rapid‐guessing behavior—its theoretical conceptualization and underlying assumptions, methods for identifying it, misconceptions regarding its dynamics, and the contextual requirements for its proper interpretation. It is argued that because it does not reflect what a test taker knows and can do, a rapid guess to an item represents a choice by the test taker to momentarily opt out of being measured. As a result, rapid guessing tends to negatively distort scores and thereby diminish validity. Therefore, because rapid guesses do not contribute to measurement, it makes little sense to include them in scoring.  相似文献   

19.
Abstract

This study explored the nature of disengagement in young people serving custodial sentences. This was in order to gain new theoretical insights into strategies for their reengagement with education. This is because 90% of young offenders who drop out of school prior to incarceration are reportedly disengaged (dropout being an indicator of disengagement). This paper reports on a qualitative ethnographic case study in one secure children’s home in England. Semi-structured interviews with 16 incarcerated young people were conducted from which five participated in in-depth case studies along with teachers and care staff. Data were collected as participants engaged with a learning opportunity over a period of time. Extending emerging theory of disengagement, findings showed that disengagement oscillated between both active and passive forms within the same individual and pointed to the need for a range of strategies needed for reengagement. This paper suggests that disengagement is complex where the physical and social context shapes how disengagement manifests. This has pedagogical implications for education in custodial settings and other at-risk children in alternative or mainstream education provision.  相似文献   

20.
《教育实用测度》2013,26(1):15-35
This study examines the effects of using item response theory (IRT) ability estimates based on customized tests that were formed by selecting specific content areas from a nationally standardized achievement test. Subsets of items were selected from four different subtests of the Iowa Tests of Basic Skills (Hieronymus, Hoover, & Lindquist, 1985) on the basis of (a) selected content areas (content-customized tests) and (b) a representative sampling of content areas (representative-customized tests). For three of the four tests examined, ability estimates and estimated national percentile ranks based on the content-customized tests in school samples tended to be systematically higher than those based on the full tests. The results of the study suggested that for certain populations, IRT ability estimates and corresponding normative scores on content-customized versions of standardized achievement tests cannot be expected to be equivalent to scores based on the full-length tests.  相似文献   

设为首页 | 免责声明 | 关于勤云 | 加入收藏

Copyright©北京勤云科技发展有限公司  京ICP备09084417号