Assessing approaches to learning with nonparametric multidimensional scaling     

Gerald Knezek  David Gibson  Rhonda Christensen  Ottavia Trevisan  Morgan Carter 《British journal of educational technology : journal of the Council for Educational Technology》2023,54(1):126-141

This article reports on a trace-based assessment of approaches to learning used by middle school aged children who interacted with NASA Mars Mission science, technology, engineering and mathematics (STEM) games in Whyville, an online game environment with 8 million registered young learners. The learning objectives of two games included awareness and knowledge of NASA missions, developing knowledge and skills of measurement and scaling, applying measurement for planetary comparisons in the solar system. Trace data from 1361 interactions were analysed with nonparametric multidimensional scaling methods, which permitted visual examination and statistical validation, and provided an example and proof of concept for the multidimensional scaling approach to analysis of time-based behavioural data from a game or simulation. Differences in approach to learning were found illustrating the potential value of the methodology to curriculum and game-based learning designers as well as other creators of online STEM content for pre-college youth. The theoretical framework of the method and analysis makes use of the Epistemic Network Analysis toolkit as a post hoc data exploration platform, and the discussion centres on issues of semantic interpretation of interaction end-states and the application of evidence centred design in post hoc analysis.

Practitioner notes

What is already known about this topic

• Educational game play has been demonstrated to positively affect learning performance and learning persistence.
• Trace-based assessment from digital learning environments can focus on learning outcomes and processes drawn from user behaviour and contextual data.
• Existing approaches used in learning analytics do not (fully) meet criteria commonly used in psychometrics or for different forms of validity in assessment, even though some consider learning analytics a form of assessment in the broadest sense.
• Frameworks of knowledge representation in trace-based research often include concepts from cognitive psychology, education and cognitive science.

What this paper adds

• To assess skills-in-action, stronger connections of learning analytics with educational measurement can include parametric and nonparametric statistics integrated with theory-driven modelling and semantic network analysis approaches widening the basis for inferences, validity, meaning and understanding from digital traces.
• An expanded methodological foundation is offered for analysis in which nonparametric multidimensional scaling, multimodal analysis, epistemic network analysis and evidence-centred design are combined.

Implications for practice and policy

• The new foundations are suggested as a principled, theory-driven, embedded data collection and analysis framework that provides structure for reverse engineering of semantics as well as pre-planning frameworks that support creative freedom in the processes of creation of digital learning environments.

  相似文献   

Early home learning support and home mathematics environment as predictors of children's mathematical skills between age 4 and 6: A longitudinal analysis using video observations and survey data     

Yiran Vicky Zhao  Jenny Louise Gibson 《Child development》2023,94(6):e377-e392

This large-scale and longitudinal study examines early home support for learning, formal/informal home mathematics activities, and their associations with children's mathematical development between age two and six. Data were collected in Germany between 2012 and 2018, N = 1184 (49% girls, 51% boys), and 15% of children had parents with a migration history. Linguistically and mathematically stimulating, attentive, and responsive parent–child engagement at age two predicted children's mathematical skills at age four and six (small-to-medium effect size). Both formal and informal home mathematical activities at age five predicted children's mathematical skills at age six (small effect size), and were associated with children's prior mathematics attainment. This study also provides indicators where individual differences and social circumstances are relevant to understanding different early mathematics outcomes.  相似文献