共查询到20条相似文献，搜索用时 734 毫秒
1.
Alexander Naumann Jan Hochweber Johannes Hartig 《Journal of Educational Measurement》2014,51(4):381-399
Students’ performance in assessments is commonly attributed to more or less effective teaching. This implies that students’ responses are significantly affected by instruction. However, the assumption that outcome measures indeed are instructionally sensitive is scarcely investigated empirically. In the present study, we propose a longitudinal multilevel‐differential item functioning (DIF) model to combine two existing yet independent approaches to evaluate items’ instructional sensitivity. The model permits for a more informative judgment of instructional sensitivity, allowing the distinction of global and differential sensitivity. Exemplarily, the model is applied to two empirical data sets, with classical indices (Pretest–Posttest Difference Index and posttest multilevel‐DIF) computed for comparison. Results suggest that the approach works well in the application to empirical data, and may provide important information to test developers. 相似文献
2.
Maria Araceli Ruiz‐Primo Richard J. Shavelson Laura Hamilton Steve Klein 《科学教学研究杂志》2002,39(5):369-393
We propose a multilevel‐multifaceted approach to evaluating the impact of education reform on student achievement that would be sensitive to context and small treatment effects. The approach uses different assessments based on their proximity to the enacted curriculum. Immediate assessments are artifacts (students' products) from the enactment of the curriculum; close assessments parallel the content and activities of the unit/curriculum; proximal assessments tap knowledge and skills relevant to the curriculum, but topics can be different; and distal assessments reflect state/national standards in a particular knowledge domain. To provide evidence about the sensitivity of the multilevel approach in ascertaining outcomes of hands‐on science programs we administered close, proximal, and distal performance assessments to evaluate the impact of instruction based on two Full Option Science System units—Variables, and Mixtures and Solutions—in a Bay Area school district. Results indicated that close assessments were more sensitive to the changes in students' pre‐ to post‐test performance than proximal assessments. © 2002 Wiley Periodicals, Inc. J Res Sci Teach 39: 369–393, 2002 相似文献
3.
In recent years, students’ test scores have been used to evaluate teachers’ performance. The assumption underlying this practice is that students’ test performance reflects teachers’ instruction. However, this assumption is generally not empirically tested. In this study, we examine the effect of teachers’ instruction on test performance at the item level using a hierarchical differential item functioning approach. The items are from the U.S. TIMSS 2011 4th-grade math test. Specifically, we tested whether students who had received instruction on a given item performed significantly better on that item compared with students who had not received such instruction when their overall math ability was controlled for, whether with or without controlling for student-level and class-level covariates. This study provides preliminary findings regarding why some items show instructional sensitivity and sheds light on how to develop instructionally sensitive items. Implications and directions for further research are also discussed. 相似文献
4.
To improve student science achievement in the United States we need inquiry-based instruction that promotes coherent understanding and assessments that are aligned with the instruction. Instead, current textbooks often offer fragmented ideas and most assessments only tap recall of details. In this study we implemented 10 inquiry-based science units that promote knowledge integration and developed assessments that measure student knowledge integration abilities. To measure student learning outcomes, we designed a science assessment consisting of both proximal items that are related to the units and distal items that are published from standardized tests (e.g., Trends in International Mathematics and Science Study). We compared the psychometric properties and instructional sensitivity of the proximal and distal items. To unveil the context of learning, we examined how student, class, and teacher characteristics affect student inquiry science learning. Several teacher-level characteristics including professional development showed a positive impact on science performance. 相似文献
5.
Science education needs valid, authentic, and efficient assessments. Many typical science assessments primarily measure recall of isolated information. This paper reports on the validation of assessments that measure knowledge integration ability among middle school and high school students. The assessments were administered to 18,729 students in five states. Rasch analyses of the assessments demonstrated satisfactory item fit, item difficulty, test reliability, and person reliability. The study showed that, when appropriately designed, knowledge integration assessments can be balanced between validity and reliability, authenticity and generalizability, and instructional sensitivity and technical quality. Results also showed that, when paired with multiple‐choice items and scored with an effective scoring rubric, constructed‐response items can achieve high reliabilities. Analyses showed that English language learner status and computer use significantly impacted students' science knowledge integration abilities. Students who took the assessment online, which matched the format of content delivery, performed significantly better than students who took the paper‐and‐pencil version. Implications and future directions of research are noted, including refining curriculum materials to meet the needs of diverse students and expanding the range of topics measured by knowledge integration assessments. © 2011 Wiley Periodicals, Inc. J Res Sci Teach 48: 1079–1107, 2011 相似文献
6.
Achievement modeling is carried out in groups of students characterized by heterogeneous instructional background. Extensions of item response theory models incorporate variables reflecting different amounts of opportunity-to-leam (OTL). The effects of these OTL variables are studied with respect to their influence on both the latent trait and the item performance directly. Such direct effects may reflect instructionally sensitive items. U.S. eighth-grade mathematics data from the Second International Mathematics Study are analyzed. Here, the same test is taken by students enrolled in typical instruction and students enrolled in elementary algebra classes. It is shown that the new analysis provides a more detailed way to examine the influence of instruction on responses to test items than does conventional item response theory. 相似文献
7.
This mixed‐methods investigation compared the relative impacts of instructional approach and context of nature of science instruction on preservice elementary teachers' understandings. The sample consisted of 75 preservice teachers enrolled in four sections of an elementary science methods course. Independent variables included instructional approach to teaching nature of science (implicit vs. explicit) and the context of nature of science instruction (as a stand‐alone topic vs. situated within instruction about global climate change and global warming). These treatments were randomly applied to the four class sections along a 2 × 2 matrix, permitting the comparison of outcomes for each independent variable separately and in combination to those of a control group. Data collection spanned the semester‐long course and included written responses to pre‐ and post‐treatment administrations of the VNOS‐B, semi‐structured interviews, and a variety of classroom artifacts. Qualitative methods were used to analyze the data with the goal of constructing profiles of participants' understandings of the nature of science and of global climate change /global warming (GCC/GW). These profiles were compared across treatments using non‐parametric statistics to assess the relative effectiveness of the four instructional approaches. Results indicated that preservice teachers who experienced explicit instruction about the nature of science made statistically significant gains in their views of nature of science regardless of whether the nature of science instruction was situated within the context of GCC/GW or as a stand‐alone topic. Further, the participants who experienced explicit nature of science instruction as a stand‐alone topic were able to apply their understandings of nature of science appropriately to novel situations and issues. We address the implications of these results for teaching the nature of science in teacher preparation courses. © 2010 Wiley Periodicals, Inc., Inc. J Res Sci Teach 48: 414–436, 2011 相似文献
8.
Jana A. Hovland Virginia G. Carraway‐Stage Artenida Cela Caitlin Collins Sebastián R. Díaz Angelo Collins Melani W. Duffrin 《Journal of Food Science Education》2013,12(4):81-86
Health professionals and policymakers are asking educators to place more emphasis on food and nutrition education. Integrating these topics into science curricula using hand‐on, food‐based activities may strengthen students’ understanding of science concepts. The Food, Math, and Science Teaching Enhancement Resource (FoodMASTER) Initiative is a compilation of programs aimed at using food as a tool to teach mathematics and science. Previous studies have shown that students experiencing the FoodMASTER curriculum were very excited about the activities, became increasingly interested in the subject matter of food, and were able to conduct scientific observations. The purpose of this study was to: (1) assess 4th graders food‐related multidisciplinary science knowledge, and (2) compare gains in food‐related science knowledge after implementation of an integrated, food‐based curriculum. During the 2009–2010 school year, FoodMASTER researchers implemented a hands‐on, food‐based intermediate curriculum in eighteen 4th grade classrooms in Ohio (n = 9) and North Carolina (n = 9). Sixteen classrooms in Ohio (n = 8) and North Carolina (n = 8), following their standard science curricula, served as comparison classrooms. Students completed a researcher‐developed science knowledge exam, consisting of 13 multiple‐choice questions administered pre‐ and post‐test. Only subjects with pre‐ and post‐test scores were entered into the sample (Intervention n = 343; Control n = 237). No significant differences were observed between groups at pre‐test. At post‐test, the intervention group scored (9.95 ± 2.00) significantly higher (p = 0.000) than the control group (8.84 ± 2.37) on a 13‐point scale. These findings suggest the FoodMASTER intermediate curriculum is more effective than a standard science curriculum in increasing students’ multidisciplinary science knowledge related to food. 相似文献
9.
This study investigated how professional development featuring evidence‐based customization of technology‐enhanced curriculum projects can improve inquiry science teaching and student knowledge integration in earth science. Participants included three middle school sixth‐grade teachers and their classes of students (N = 787) for three consecutive years. Teachers used evidence from their student work to revise the curriculum projects and rethink their teaching strategies. Data were collected through teacher interviews, written reflections, classroom observations, curriculum artifacts, and student assessments. Results suggest that the detailed information about the learning activities of students provided by the assessments embedded in the online curriculum motivated curricular and pedagogical customizations that resulted in both teacher and student learning. Customizations initiated by teachers included revisions of embedded questions, additions of hands‐on investigations, and modifications of teaching strategies. Student performance improved across the three cohorts of students with each year of instructional customization. Coupling evidence from student work with revisions of curriculum and instruction has promise for strengthening professional development and improving science learning. © 2010 Wiley Periodicals, Inc. J Res Sci Teach 47: 1037–1063, 2010 相似文献
10.
11.
Instructional practices in secondary science: How teachers achieve local and standards-based success
Beth A. Covitt Elizabeth Xeng de los Santos Qinyun Lin Christie Morrison Thomas Charles W. Anderson 《科学教学研究杂志》2024,61(1):170-202
This article reports on analyses of the instructional practices of six middle- and high-school science teachers in the United States who participated in a research-practice partnership that aims to support reform science education goals at scale. All six teachers were well qualified, experienced, and locally successful—respected by students, parents, colleagues, and administrators—but they differed in their success in supporting students' three-dimensional learning. Our goal is to understand how the teachers' instructional practices contributed to their similarities in achieving local success and to differences in enabling students' learning, and to consider the implications of these findings for research-practice partnerships. Data sources included classroom videos supplemented by interviews with teachers and focus students and examples of student work. We also compared students' learning gains by teacher using pre–post assessments that elicited three-dimensional performances. Analyses of classroom videos showed how all six teachers achieved local success—they led effectively managed classrooms, covered the curriculum by teaching almost all unit activities, and assessed students' work in fair and efficient ways. There were important differences, however, in how teachers engaged students in science practices. Teachers in classrooms where students achieved lower learning gains followed a pattern of practice we describe as activity-based teaching, in which students completed investigations and hands-on activities with few opportunities for sensemaking discussions or three-dimensional science performances. Teachers whose students achieved higher learning gains combined the social stability characteristic of local classroom success with more demanding instructional practices associated with scientific sensemaking and cognitive apprenticeship. We conclude with a discussion of implications for research-practice partnerships, highlighting how partnerships need to support all teachers in achieving both local and standards-based success. 相似文献
12.
Morgan S. Polikoff 《Educational Measurement》2010,29(4):3-14
Standards-based reform, as codified by the No Child Left Behind Act, relies on the ability of assessments to accurately reflect the learning that takes place in U.S. classrooms. However, this property of assessments—their instructional sensitivity—is rarely, if ever, investigated by test developers, states, or researchers. In this paper, the literature on the psychometric property of instructional sensitivity is reviewed. Three categories of instructional sensitivity measures are identified—those relying on item or test scores only, those relying on item or test scores and teacher reports of instruction, and strictly judgmental methods. Each method identified in the literature is discussed alongside the evidence for its utility. Finally, recommendations are made as to the proper role of instructional sensitivity in the evaluation of assessments used under standards-based reform. 相似文献
13.
This study explores measurement of a construct called knowledge integration in science using multiple-choice and explanation items. We use construct and instructional validity evidence to examine the role multiple-choice and explanation items plays in measuring students' knowledge integration ability. For construct validity, we analyze item properties such as alignment, discrimination, and target range on the knowledge integration scale using a Rasch Partial Credit Model analysis. For instructional validity, we test the sensitivity of multiple-choice and explanation items to knowledge integration instruction using a cohort comparison design. Results show that (1) one third of correct multiple-choice responses are aligned with higher levels of knowledge integration while three quarters of incorrect multiple-choice responses are aligned with lower levels of knowledge integration, (2) explanation items discriminate between high and low knowledge integration ability students much more effectively than multiple-choice items, (3) explanation items measure a wider range of knowledge integration levels than multiple-choice items, and (4) explanation items are more sensitive to knowledge integration instruction than multiple-choice items. 相似文献
14.
Gina N. Cervetti Jacqueline Barber Rena Dorph P. David Pearson Pete G. Goldschmidt 《科学教学研究杂志》2012,49(5):631-658
This study investigates the efficacy of an integrated science and literacy approach at the upper‐elementary level. Teachers in 94 fourth grade classrooms in one Southern state participated. Half of the teachers taught the treatment unit, an integrated science–literacy unit on light and energy designed using a curriculum model that engages students in reading text, writing notes and reports, conducting firsthand investigations, and frequent discussion of key concepts and processes to acquire inquiry skills and knowledge about science concepts, while the other half of the teachers taught a content‐comparable science‐only unit on light and energy (using materials provided by their districts) and provided their regular literacy instruction. Students in the treatment group made significantly greater gains on measures of science understanding, science vocabulary, and science writing. Students in both groups made comparable gains in science reading comprehension. © 2012 Wiley Periodicals, Inc. J Res Sci Teach 49: 631–658, 2012 相似文献
15.
16.
Jennifer Wilhelm 《International Journal of Science Education》2013,35(15):2105-2122
This paper reports an examination on gender differences in lunar phases understanding of 123 students (70 females and 53 males). Middle‐level students interacted with the Moon through observations, sketching, journalling, two‐dimensional and three‐dimensional modelling, and classroom discussions. These lunar lessons were adapted from the Realistic Explorations in Astronomical Learning (REAL) curriculum. Students’ conceptual understandings were measured through analysis of pre‐test and post‐test results on a Lunar Phases Concept Inventory (LPCI) and a Geometric Spatial Assessment (GSA). The LPCI was used to assess conceptual learning of eight science and four mathematics domains. The GSA was used to assess learning of the same four mathematical domains; however, the GSA test items were not posed within a lunar context. Results showed both male and female groups to make significant gains in understanding on the overall LPCI test scores as well as significant gains on five of the eight science domains and on three of the four mathematics domains. The males scored significantly higher than the females on the science domain, phase—Sun/Earth/Moon positions, and on the mathematics domain geometric spatial visualisation. GSA results found both male and female groups achieving a significant increase in their test scores on the overall GSA. Females made significant gains on the GSA mathematics domains, periodic patterns and cardinal directions, while males made significant gains on only the periodic patterns domain. Findings suggest that both scientific and mathematical understandings can be significantly improved for both sexes through the use of spatially focused, inquiry‐oriented curriculum such as REAL. 相似文献
17.
Mike Stieff 《科学教学研究杂志》2011,48(10):1137-1158
The present article discusses the design and impact of computer‐based visualization tools for supporting student learning and representational competence in science. Specifically, learning outcomes and student representation use are compared between eight secondary classrooms utilizing The Connected Chemistry Curriculum and eight secondary chemistry using lecture‐based methods. Results from the quasi‐experimental intervention indicate that the curriculum and accompanying visualization tool yield only small to modest gains in student achievement on summative assessments. Analysis of student representation use on pre‐ and post‐assessments, however, indicate the students in Connected Chemistry classrooms are significantly more likely to use submicroscopic representations of chemical systems that are consistent with teacher and expert representation use. The affordances of visualization tools in inquiry activities to improve students' representational competence and conceptual understanding of content in the science classroom are discussed. © 2011 Wiley Periodicals, Inc. J Res Sci Teach 48: 1137–1158, 2011 相似文献
18.
Morgan S. Polikoff 《Educational Assessment》2016,21(2):102-119
As state tests of student achievement are used for an increasingly wide array of high- and low-stakes purposes, evaluating their instructional sensitivity is essential. This article uses data from the Bill and Melinda Gates Foundation's Measures of Effective Project to examine the instructional sensitivity of 4 states' mathematics and English language arts assessments to 5 measures of pedagogical quality. The results indicate modest overall levels of sensitivity, with some state assessments insensitive to most or all measures of pedagogical quality. Furthermore, there are sometimes substantial differences across states in the sensitivity of their assessments. The article calls for more research on what makes state tests instructionally sensitive and cautions against the use of insensitive assessments for consequential decisions. 相似文献
19.
Brian Gong Richard Venezky David Mioduser 《Journal of Science Education and Technology》1992,1(3):157-176
The Educational Testing Service/University of Delaware Science Education project has addressed the challenge of promoting systemic change in science education through the use of instructional assessments. Instructional assessments integrate performance-based assessments with curriculum and instruction at the classroom level. Instructional assessment provides a promising lever for promoting professional development and improving middle school science curriculum, instruction, and assessment. 相似文献
20.
Evaluating the impact of instructional innovations and coordinating instruction, assessment, and testing present complex tensions. Many evaluation and coordination efforts aim to address these tensions by using the coherence provided by modern cognitive science perspectives on domain-specific learning. This paper introduces an alternative framework that uses emerging situative assessment perspectives to align learning across increasingly formal levels of educational practice. This framework emerged from 2 design studies of a 20-hr high school genetics curriculum that used the GenScope computer-based modeling software. The 1st study aligned learning across (a) the contextualized enactment of inquiry-oriented activities in GenScope, (b) “feedback conversations” around informal embedded assessments, and (c) a formal performance assessment; the 2nd study extended this alignment to a conventional achievement test. Design-based refinements ultimately delivered gains of nearly 2 SD on the performance assessment and more than 1 SD in achievement. These compared to gains of 0.25 and 0.50 SD, respectively, in well-matched comparison classrooms. General and specific assessment design principles for aligning instruction, assessment, and testing and for evaluating instructional innovations are presented. 相似文献