共查询到20条相似文献，搜索用时 265 毫秒
1.
Carol Van Zile-Tamsen 《Research in higher education》2017,58(8):922-933
The use of surveys, questionnaires, and rating scales to measure important outcomes in higher education is pervasive, but reliability and validity information is often based on problematic Classical Test Theory approaches. Rasch Analysis, based on Item Response Theory, provides a better alternative for examining the psychometric quality of rating scales and informing scale improvements. This paper outlines a six-step process for using Rasch Analysis to review the psychometric properties of a rating scale. The Partial Credit Model and Andrich Rating Scale Model will be described in terms of the pyschometric information (i.e., reliability, validity, and item difficulty) and diagnostic indices generated. Further, this approach will be illustrated through the example of authentic data from a university-wide student evaluation of teaching. 相似文献
2.
Using data from the 2006 cohort of the Wabash National Study of Liberal Arts Education, we examined the relationships between three approaches to measuring student learning outcomes (direct-assessment learning gains, self-reported gains, and college grades) and student persistence from the first to second year. Results from a series of logistic regressions indicated that students’ grade-point averages had the largest explanatory power in student persistence, followed by self-reported gains. Direct-assessment learning gains had the least power in explaining persistence. The findings have implications for the national conversation on student success in college. 相似文献
3.
4.
Louis M. Rocconi 《Research in higher education》2011,52(2):178-193
This study investigated the direct and indirect relationships between participating in a learning community, student engagement,
and self-reported learning outcomes. Using a sample of 241 freshmen at a single urban research university who took the College Student Experiences Questionnaire, the results indicate that after controlling for demographic characteristics and entering composite ACT score, the relationship
between learning community participation and learning outcomes are mediated by students’ levels of engagement. Learning community
participation was not directly related to educational gains but was indirectly related to educational gains through student
engagement. Student engagement in turn was strongly related to educational gains. 相似文献
5.
The present study examined the relationships between student engagement, represented by two versions of the National Survey of Student Engagement (NSSE) and self-reported gains in learning. The study drew on institutional-level data from participating institutions in 2011 and 2013. The objective of the research was to compare evidence of convergence and discrimination for the two versions of NSSE using canonical correlation analysis. Results indicated that both versions of NSSE provided clear evidence of convergence in that student engagement measures were significantly and positively related to perceived gains in learning. However, only the most recent version of NSSE provided strong evidence of discrimination (i.e., differential relationships between engagement measures and self-reported learning outcomes). Thus, the revised NSSE appears to offer substantial advantages for institutions interested in more nuanced understandings of the relationships between student engagement and perceived learning outcomes. Implications for educators, with goals of enhancing student learning, and for researchers, who often compare complex sets of data, are included. 相似文献
6.
张红霞 《安徽广播电视大学学报》2003,(3):47-51
TEM4听写采用的是较传统的数错扣分法。数错扣分法是负分法，其中存在一些问题。因此我们提出一种实验性的评分方法——部分得分制。实验数据有两组，分别采用TEM4听写评分制和新评分制。数据比较以及部分得分模型(Rasch模型之一)对实验量表效能的分析(如模型与数据拟合值、被试拟合值、信息函数等)说明，实验评分制能较好地测量大多数学生的听写水平。 相似文献
7.
Che-Yu Kuo Tsung-Hau Jen Ying-Shao Hsu 《International Journal of Science Education》2013,35(14):2326-2357
The potential of computer-based assessments for capturing complex learning outcomes has been discussed; however, relatively little is understood about how to leverage such potential for summative and accountability purposes. The aim of this study is to develop and validate a multimedia-based assessment of scientific inquiry abilities (MASIA) to cover a more comprehensive construct of inquiry abilities and target secondary school students in different grades while this potential is leveraged. We implemented five steps derived from the construct modeling approach to design MASIA. During the implementation, multiple sources of evidence were collected in the steps of pilot testing and Rasch modeling to support the validity of MASIA. Particularly, through the participation of 1,066 8th and 11th graders, MASIA showed satisfactory psychometric properties to discriminate students with different levels of inquiry abilities in 101 items in 29 tasks when Rasch models were applied. Additionally, the Wright map indicated that MASIA offered accurate information about students’ inquiry abilities because of the comparability of the distributions of student abilities and item difficulties. The analysis results also suggested that MASIA offered precise measures of inquiry abilities when the components (questioning, experimenting, analyzing, and explaining) were regarded as a coherent construct. Finally, the increased mean difficulty thresholds of item responses along with three performance levels across all sub-abilities supported the alignment between our scoring rubrics and our inquiry framework. Together with other sources of validity in the pilot testing, the results offered evidence to support the validity of MASIA. 相似文献
8.
AbstractAssessment feedback from teachers gains consistently low satisfaction scores in national surveys of student satisfaction, with concern surrounding its timeliness, quality and effectiveness. Equally, there has been heightened interest in the responsibility of learners in engaging with feedback and how student assessment literacy might be increased. We present results from a five-year longitudinal mixed methods enquiry, thematically analysing semi-structured interviews and focus groups with undergraduate students who have experienced dialogic feed-forward on a course in a British university. We use inferential statistics to compare performance pre and post-assessment intervention. The assessment consisted of submitting a draft coursework essay, which was discussed and evaluated face-to-face with the course teacher before a self-reflective piece was written about the assessment process and a final essay was submitted for summative grading. We evidence that this process asserted a positive influence on the student learning experience in a number of inter-related cognitive and affective ways, impacting positively upon learning behaviour, supporting student achievement and raising student satisfaction with feedback. We advocate a cyclic and iterative approach to dialogic feed-forward, which facilitates learners’ longitudinal development. Programme teams should offer systematic opportunities across curricula for students to understand the rationale for and develop feedback literacy. 相似文献
9.
Using data from the 2006 cohort of the Wabash National Study of Liberal Arts Education, we developed a student typology based on student responses to survey items on the National Survey of Student Engagement. We then examined the utility of this typology in understanding direct-assessment learning outcomes, self-reported gains, grade-point average, and persistence from the first to second year of college. Results from linear and logistic regression models indicated there were relationships between student types and the various outcomes, and that an engagement-based student typology could help deepen our understanding of the college student experience and college outcomes. 相似文献
10.
Stephen R. Porter 《Research in higher education》2013,54(2):201-226
Recent studies have asserted that self-reported learning gains (SRLG) are valid measures of learning, because gains in specific content areas vary across academic disciplines as theoretically predicted. In contrast, other studies find no relationship between actual and self-reported gains in learning, calling into question the validity of SRLG. I reconcile these two divergent sets of literature by proposing a theory of college student survey response that relies on the belief-sampling model of attitude formation. This theoretical approach demonstrates how students can easily construct answers to SRLG questions that will result in theoretically consistent differences in gains across academic majors, while at the same time lacking the cognitive ability to accurately report their actual learning gains. Four predictions from the theory are tested, using data from the 2006–2009 Wabash National Study. Contrary to previous research, I find little evidence as to the construct and criterion validity of SRLG questions. 相似文献
11.
Sheila Qureshi Venkat Rao Vishnumolakala Daniel C. Southam David F. Treagust 《International Journal of Science and Mathematics Education》2017,15(6):1017-1038
This research took place within the context of ongoing educational reforms to promote inquiry-based science instruction and a desire to draw evidence to inform adoptions of western pedagogical practices in a high-context culture like Qatar. We report on the outcomes from Process Oriented Guided Inquiry Learning (POGIL) in a foundation chemistry course based on students’ achievement, their perceived learning gains, and their self-efficacy. The study utilized quantitative data obtained from normalized content tests and instruments to measure perceived learning gains and attitudes and experience. Qualitative data from open-ended student questionnaires were analyzed to cross-validate findings from the study. Positive effects of POGIL during fall (semester 1) and spring (semester 2) semesters were evidenced by (a) improved mean scores and medium to large effect sizes for content test results, perceived learning gains, and self-efficacy levels and (b) a positive correlation between the measures of perceived learning gains and self-efficacy. Students self-reported increased self-efficacy, interest, and better understanding of concepts using the POGIL method. Comparing fall and spring semesters, student reluctance and negative perceptions of the POGIL approach gradually diminished. Students were able to adapt easily to POGIL—a method of teaching that they had not experienced before but which was compatible with the high-context culture in which they live. In addition, this study reflects the current condition of science learning in Qatar, where the emerging outcomes of educational reforms play an important role in preparing local students to transition into higher education. 相似文献
12.
《Africa Education Review》2013,10(4):563-583
AbstractSummative assessment qualifies the achievement of a student in a particular field of specialization at a given time. Questions should include a range of cognitive levels from Bloom's taxonomy and be consistent with the learning outcomes of the module in question. Furthermore, a holistic approach to assessment, such as the application of the principles of the Herrmann Whole Brain Model, needs to be used to accommodate learning style diversity. The purpose of this study was to analyse, assess and compare the summative assessment of two third year level modules in the Bachelor of Science degree programme, namely Biochemistry and Zoology as part of action research with a view to enhancing the professional development of the lecturers involved. The questions posed in summative assessments were classified in terms of Bloom's differentiation of cognitive levels and the four different learning styles determined by Herrmann. Spearman's non-parametric analysis indicated that no correlation existed in this study between cognitive level and student performance based on achievement. In addition, there was not much difference between the cognitive levels and student performance between the two disciplines. Although the students seemed to do better at application level questions, the authors need to reflect on whether the assessments were valid with respect to the learning outcomes, methods of facilitating learning, and the assessments based on cognitive levels and learning style preferences. We conclude that continuous action research must be taken to improve the formulation of learning outcomes and students' achievement of these outcomes and quality of student learning – the main aim being the successful completion of the modules. 相似文献
13.
Twan Huybers 《Assessment & Evaluation in Higher Education》2014,39(4):496-513
An important purpose of student evaluation of teaching is to inform an educator’s reflection about the strengths and weaknesses of their teaching approaches. Quantitative instruments are one way of obtaining student responses. They have traditionally taken the form of surveys in which students provide their responses to various statements using item-by-item agree/disagree ratings. Previous research has identified shortcomings of such rating scales, including response bias and the associated lack of discrimination amongst the items evaluated. In this paper, best–worst scaling is proposed as a novel method for quantitative teaching evaluation. The way in which best–worst scaling can be used in this context is illustrated in three different applications. Two applications demonstrate how it can be used for evaluations in a small-size classroom environment. The third application is a broader evaluation of university courses on a larger scale. In comparison with conventional rating scales, the best–worst scaling approach enables better highlighting of the differences between evaluation items. In doing so, it can provide enhanced guidance to educators in their reflection about their teaching. Moreover, implementation and analysis of a best–worst scaling evaluation is relatively straightforward, which establishes it a feasible method for teaching practitioners and researchers. 相似文献
14.
15.
Mike Stieff 《科学教学研究杂志》2011,48(10):1137-1158
The present article discusses the design and impact of computer‐based visualization tools for supporting student learning and representational competence in science. Specifically, learning outcomes and student representation use are compared between eight secondary classrooms utilizing The Connected Chemistry Curriculum and eight secondary chemistry using lecture‐based methods. Results from the quasi‐experimental intervention indicate that the curriculum and accompanying visualization tool yield only small to modest gains in student achievement on summative assessments. Analysis of student representation use on pre‐ and post‐assessments, however, indicate the students in Connected Chemistry classrooms are significantly more likely to use submicroscopic representations of chemical systems that are consistent with teacher and expert representation use. The affordances of visualization tools in inquiry activities to improve students' representational competence and conceptual understanding of content in the science classroom are discussed. © 2011 Wiley Periodicals, Inc. J Res Sci Teach 48: 1137–1158, 2011 相似文献
16.
17.
Developing Performance Indicators for Assessing Classroom Teaching Practices and Student Learning 总被引：1，自引：0，他引：1
Alberto F. Cabrera Carol L. Colbeck Patrick T. Terenzini 《Research in higher education》2001,42(3):327-352
Several states are requiring institutions to document changes in student outcomes. Regional and specialized accrediting agencies are also changing their review criteria from measuring inputs to assessing indicators of student learning. This article describes the results of an evaluation project that sought to develop performance indicators of learning gains for undergraduate engineering students. Specifically, the study investigated the relationship between classroom practices and students' gains in professional competencies. More than 1,250 students from 7 universities participated. Findings show that the instructional practices of Instructor Interaction and Feedback, Collaborative Learning, and Clarity and Organization are significantly and positively associated with gains in students' self-reported gains in problem-solving skills, group skills, and understanding of engineering as an occupation. The indicators meet several conditions recommended by the assessment literature. They are (1) meaningful to the user, (2) reliable and valid, and (3) index observable behaviors rather than subjective impressions. 相似文献
18.
《教育实用测度》2013,26(4):363-378
The purpose of this article is to recast the Rasch model in a generalizable form allowing for broader insights into the model's applicability beyond partial credit scoring. Applications of an extended logistic model to the study of dependencies within subtests of dichotomously scored items and the study of questionnaires in the Likert tradition are identified. One possible outcome of research using the extended logistic model is a better understanding of the cognitive processes in problem solving and other learning tasks. 相似文献
19.
Tingting Li I-Chien Chen Emily Adah Miller Cory Susanne Miller Barbara Schneider Joseph Krajcik 《科学教学研究杂志》2024,61(2):358-418
This longitudinal study examines the relationship between students' knowledge-in-use performance and their performance on third-party designed summative tests within a coherent and equitable learning environment. Focusing on third-grade students across three consecutive project-based learning (PBL) units aligned with the Next Generation Science Standards (NGSS), the study includes 1067 participants from 23 schools in a Great Lakes state. Two-level hierarchical linear modeling estimates the effects of post-unit assessments on end-of-year summative tests. Results indicate that post-unit assessment performances predict NGSS-aligned summative test performance. Students experiencing more PBL units demonstrate greater gains on the summative test, with predictions not favoring students from diverse backgrounds. This study underscores the importance of coherence, equity, and the PBL approach in promoting knowledge-in-use and science achievement. A systematically coherent PBL environment across multiple units facilitates the development of students' knowledge-in-use, highlighting the significance of designing science and engineering practices (SEPs) and crosscutting concepts coherently and progressively, with intentional revisitation of disciplinary core ideas (DCIs). The study also investigates how the PBL approach fosters equitable learning environments for diverse demographic groups, offering equitable opportunities through equity-oriented design. Contributions include a coherent assessment system that tracks and supports learning aligned with NGSS, emphasizing the predictive power of post-unit assessments, continuous monitoring and tracking. The implications of context similarity and optimal performance expectations within units are discussed. Findings inform educators, administrators, and policymakers about the benefits of NGSS-aligned PBL systems and the need for coherent and equitable learning and assessment systems supporting knowledge-in-use development and equitable opportunities for all learners. 相似文献
20.
The scoring matrix, a method used to facilitate community participation in collaboratively planning and monitoring development projects in natural resource management, was adapted to promote collaboration and reflection in a course in participatory resource management. The scoring matrix is described and its strengths and weaknesses in relation to key objectives are analysed. The matrix represents an innovative approach to evaluation that may be useful in a variety of fields. The authors argue, too, that the case is an example of how discipline (and profession-) specific tools can be adapted in an educational setting to serve the dual purposes of promoting experiential learning of particular key skills, and of monitoring and evaluating student learning. They suggest that academics in other fields may wish to consider participatory tools like the scoring matrix or adapt the tools of their own disciplines as ways of collaboratively evaluating teaching in their own disciplines/professions. 相似文献