首页 | 本学科首页   官方微博 | 高级检索  
相似文献
 共查询到20条相似文献，搜索用时 31 毫秒
1.
Abstract

Applied researchers often find themselves making statistical inferences in settings that would seem to require multiple comparisons adjustments. We challenge the Type I error paradigm that underlies these corrections. Moreover we posit that the problem of multiple comparisons can disappear entirely when viewed from a hierarchical Bayesian perspective. We propose building multilevel models in the settings where multiple comparisons arise. Multilevel models perform partial pooling (shifting estimates toward each other), whereas classical procedures typically keep the centers of intervals stationary, adjusting for multiple comparisons by making the intervals wider (or, equivalently, adjusting the p values corresponding to intervals of fixed width). Thus, multilevel models address the multiple comparisons problem and also yield more efficient estimates, especially in settings with low group-level variation, which is where multiple comparisons are a particular concern.  相似文献   

2.
Multilevel modeling has been utilized for combining single-case experimental design (SCED) data assuming simple level-1 error structures. The purpose of this study is to compare various multilevel analysis approaches for handling potential complexity in the level-1 error structure within SCED data, including approaches assuming simple and complex error structures (heterogeneous, autocorrelation, and both) and those using fit indices to select between alternative error structures. A Monte Carlo study was conducted to empirically validate the suggested multilevel modeling approaches. Results indicate that each approach leads to fixed effect estimates with little to no bias and that inferences for fixed effects were frequently accurate, particularly when a simple homogeneous level-1 error structure or a first-order autoregressive structure was assumed and the inferences were based on the Kenward-Roger method. Practical implications and recommendations are discussed.  相似文献   

3.
In many applications of multilevel modeling, group-level (L2) variables for assessing group-level effects are generated by aggregating variables from a lower level (L1). However, the observed group mean might not be a reliable measure of the unobserved true group mean. In this article, we propose a Bayesian approach for estimating a multilevel latent contextual model that corrects for measurement error and sampling error (i.e., sampling only a small number of L1 units from a L2 unit) when estimating group-level effects of aggregated L1 variables. Two simulation studies were conducted to compare the Bayesian approach with the maximum likelihood approach implemented in Mplus. The Bayesian approach showed fewer estimation problems (e.g., inadmissible solutions) and more accurate estimates of the group-level effect than the maximum likelihood approach under problematic conditions (i.e., small number of groups, predictor variable with a small intraclass correlation). An application from educational psychology is used to illustrate the different estimation approaches.  相似文献   

4.
This article compares maximum likelihood and Bayesian estimation of the correlated trait–correlated method (CT–CM) confirmatory factor model for multitrait–multimethod (MTMM) data. In particular, Bayesian estimation with minimally informative prior distributions—that is, prior distributions that prescribe equal probability across the known mathematical range of a parameter—are investigated as a source of information to aid convergence. Results from a simulation study indicate that Bayesian estimation with minimally informative priors produces admissible solutions more often maximum likelihood estimation (100.00% for Bayesian estimation, 49.82% for maximum likelihood). Extra convergence does not come at the cost of parameter accuracy; Bayesian parameter estimates showed comparable bias and better efficiency compared to maximum likelihood estimates. The results are echoed via 2 empirical examples. Hence, Bayesian estimation with minimally informative priors outperforms enables admissible solutions of the CT–CM model for MTMM data.  相似文献   

5.
Multilevel Structural equation models are most often estimated from a frequentist framework via maximum likelihood. However, as shown in this article, frequentist results are not always accurate. Alternatively, one can apply a Bayesian approach using Markov chain Monte Carlo estimation methods. This simulation study compared estimation quality using Bayesian and frequentist approaches in the context of a multilevel latent covariate model. Continuous and dichotomous variables were examined because it is not yet known how different types of outcomes—most notably categorical—affect parameter recovery in this modeling context. Within the Bayesian estimation framework, the impact of diffuse, weakly informative, and informative prior distributions were compared. Findings indicated that Bayesian estimation may be used to overcome convergence problems and improve parameter estimate bias. Results highlight the differences in estimation quality between dichotomous and continuous variable models and the importance of prior distribution choice for cluster-level random effects.  相似文献   

6.
This Monte Carlo study investigated the impacts of measurement noninvariance across groups on major parameter estimates in latent growth modeling when researchers test group differences in initial status and latent growth. The average initial status and latent growth and the group effects on initial status and latent growth were investigated in terms of Type I error and bias. The location and magnitude of noninvariance across groups was related to the location and magnitude of bias and Type I error in the parameter estimates. That is, noninvariance in factor loadings and intercepts was associated with the Type I error inflation and bias in the parameter estimates of the slope factor (or latent growth) and the intercept factor (or initial status), respectively. As noninvariance became large, the degree of Type I error and bias also increased. On the other hand, a correctly specified second-order latent growth model yielded unbiased parameter estimates and correct statistical inferences. Other findings and implications on future studies were discussed.  相似文献   

7.
In psychological research, available data are often insufficient to estimate item factor analysis (IFA) models using traditional estimation methods, such as maximum likelihood (ML) or limited information estimators. Bayesian estimation with common-sense, moderately informative priors can greatly improve efficiency of parameter estimates and stabilize estimation. There are a variety of methods available to evaluate model fit in a Bayesian framework; however, past work investigating Bayesian model fit assessment for IFA models has assumed flat priors, which have no advantage over ML in limited data settings. In this paper, we evaluated the impact of moderately informative priors on ability to detect model misfit for several candidate indices: posterior predictive checks based on the observed score distribution, leave-one-out cross-validation, and widely available information criterion (WAIC). We found that although Bayesian estimation with moderately informative priors is an excellent aid for estimating challenging IFA models, methods for testing model fit in these circumstances are inadequate.  相似文献   

8.
Structural equation modeling (SEM) is now a generic modeling framework for many multivariate techniques applied in the social and behavioral sciences. Many statistical models can be considered either as special cases of SEM or as part of the latent variable modeling framework. One popular extension is the use of SEM to conduct linear mixed-effects modeling (LMM) such as cross-sectional multilevel modeling and latent growth modeling. It is well known that LMM can be formulated as structural equation models. However, one main difference between the implementations in SEM and LMM is that maximum likelihood (ML) estimation is usually used in SEM, whereas restricted (or residual) maximum likelihood (REML) estimation is the default method in most LMM packages. This article shows how REML estimation can be implemented in SEM. Two empirical examples on latent growth model and meta-analysis are used to illustrate the procedures implemented in OpenMx. Issues related to implementing REML in SEM are discussed.  相似文献   

9.
Appropriate model specification is fundamental to unbiased parameter estimates and accurate model interpretations in structural equation modeling. Thus detecting potential model misspecification has drawn the attention of many researchers. This simulation study evaluates the efficacy of the Bayesian approach (the posterior predictive checking, or PPC procedure) under multilevel bifactor model misspecification (i.e., ignoring a specific factor at the within level). The impact of model misspecification on structural coefficients was also examined in terms of bias and power. Results showed that the PPC procedure performed better in detecting multilevel bifactor model misspecification, when the misspecification became more severe and sample size was larger. Structural coefficients were increasingly negatively biased at the within level, as model misspecification became more severe. Model misspecification at the within level affected the between-level structural coefficient estimates more when data dependency was lower and the number of clusters was smaller. Implications for researchers are discussed.  相似文献   

10.
Conventional multilevel modeling works well with purely hierarchical data; however, pure hierarchies rarely exist in real datasets. Applied researchers employ ad hoc procedures to create purely hierarchical data. For example, applied educational researchers either delete mobile participants' data from the analysis or identify the student only with the last school attended while including an explanatory variable indicating whether a student is mobile. This simulation study compared the parameter and standard error estimates of these two ad hoc procedures for handling and assessing the influence of mobility on outcomes with results based on use of the multiple membership random effects model. Substantial bias was found for some parameters when multiple membership data structures were ignored.  相似文献   

11.
Most of the software that is available to implement Bayesian approaches uses Markov chain Monte Carlo (MCMC) methods. It is our impression that many researchers are primarily concerned with convergence as assessed by the Potential Scale Reduction (PSR) and that other aspects of MCMC are largely ignored. In this article, we argue that the precision with which the Bayesian estimates are approximated by summary statistics for the MCMC chain is essential to ensure good statistical properties. We discuss the Effective Sample Size (ESS), which indicates how well an estimate is approximated, and present evidence from two simulation studies and an example from organizational research to support our claim that researchers should be concerned not only with convergence but also with precision, particularly when a multilevel model is estimated. In addition, we demonstrate how Mplus can be modified so that users can control the ESS, and we conclude with recommendations.  相似文献   

12.
ABSTRACT

Educational researchers frequently study the impact of treatments or interventions on educational outcomes. However, when observational or quasiexperimental data are used for such investigations, selection bias can adversely impact researchers’ abilities to make causal inferences about treatment effects. One way to deal with selection bias is to use propensity score methods. The authors introduce educational researchers to the general principles underlying propensity score methods, describe 2 practical applications of these methods, and discuss their limitations.  相似文献   

13.
Meta-analytic structural equation modeling (MA-SEM) is increasingly being used to assess model-fit for variables' interrelations synthesized across studies. MA-SEM researchers have analyzed synthesized correlation matrices using structural equation modeling (SEM) estimation that is designed for covariance matrices. This can produce incorrect model-fit chi-square statistics, standard error estimates (Cudeck, 1989), or both for parameters that are not scale free or that describe a scale-noninvariant model unless corrected SEM estimation is used to analyze the correlations. This study introduced univariate and multivariate approximate methods for synthesizing covariance matrices for use in MA-SEM. A simulation study assessed the approximate methods by estimating parameters in a scale-noninvariant model using synthesized covariances versus synthesized correlations with and without the appropriate corrections. Standard error bias was noted only for uncorrected analyses of pooled correlations. Chi-square model-fit statistics were overly conservative except when covariance matrices were analyzed. Benefits and limitations of this approximate method are presented and discussed.  相似文献   

14.
The question of whether students’ school-year learning rates differ by race/ethnicity is important for monitoring educational inequality. Researchers applying different modeling strategies to the same data (the ECLS-K:99) have reached contrasting conclusions on this question. We outline the similarities and differences across three common approaches to estimating gains and heterogeneity in gains: 1) a gain score model (with intercept), 2) a first-difference (FD) model (in some cases equivalent to regression-through-the-origin [RTO] and student fixed effects models), and 3) a student random effects (RE) model. We show via simulation that FD/RTO and RE models produce estimates of learning rates – and group differences in learning rates – with more favorable RMSD compared to the gain score model with intercept. Using data from the ECLS-K:99, we demonstrate that these precision differences lead to contrasting inferences regarding learning rate heterogeneity, and likely explain the inconsistencies across previous studies.  相似文献   

15.
Multilevel modeling has grown in use over the years as a way to deal with the nonindependent nature of observations found in clustered data. However, other alternatives to multilevel modeling are available that can account for observations nested within clusters, including the use of Taylor series linearization for variance estimation, the design effect adjusted standard errors approach, and fixed effects modeling. Using 1,000 replications of 12 conditions with varied Level 1 and Level 2 sample sizes, the author compared parameter estimates, standard errors, and statistical significance using various alternative procedures. Results indicate that several acceptable procedures can be used in lieu of or together with multilevel modeling, depending on the type of research question asked and the number of clusters under investigation. Guidelines for applied researchers are discussed.  相似文献   

16.
This study examines the use of cross-classified random effects models (CCrem) and cross-classified multiple membership random effects models (CCMMrem) to model rater bias and estimate teacher effectiveness. Effect estimates are compared using CTT versus item response theory (IRT) scaling methods and three models (i.e., conventional multilevel model, CCrem, CCMMrem). Results indicate that ignoring rater bias can lead to teachers being misclassified within an evaluation system. The best estimates of teacher effectiveness are produced using CCrems regardless of scaling method. Use of CCMMrems to model rater bias cannot be recommended based on the results of this study; combining the use of CCMMrems with an IRT scaling method produced especially unstable results.  相似文献   

17.
Abstract

Educational stakeholders have long known that students might not be fully engaged when taking an achievement test and that such disengagement could undermine the inferences drawn from observed scores. Thanks to the growing prevalence of computer-based tests and the new forms of metadata they produce, researchers have developed and validated procedures for using item response times to identify responses to items that are likely disengaged. In this study, we examine the impact of two techniques to account for test disengagement—(a) removing unengaged test takers from the sample and (b) adjusting test scores to remove rapidly guessed items—on estimates of school contributions to student growth, achievement gaps, and summer learning loss. Our results indicate that removing disengaged examinees from the sample will likely induce bias in the estimates, although as a whole accounting for disengagement had minimal impact on the metrics we examined. Last, we provide guidance for policy makers and evaluators on how to account for disengagement in their own work and consider the promise and limitations of using achievement test metadata for related purposes.  相似文献   

18.
This article examines whether Bayesian estimation with minimally informed prior distributions can alleviate the estimation problems often encountered with fitting the true score multitrait–multimethod structural equation model with split-ballot data. In particular, the true score multitrait–multimethod structural equation model encounters an empirical underidentification when (a) latent variable correlations are homogenous, and (b) fitted to data from a 2-group split-ballot design; an understudied case of empirical underidentification due to a planned missingness (i.e., split-ballot) design. A Monte Carlo simulation and 3 empirical examples showed that Bayesian estimation performs better than maximum likelihood (ML) estimation. Therefore, we suggest using Bayesian estimation with minimally informative prior distributions when estimating the true score multitrait–multimethod structural equation model with split-ballot data. Furthermore, given the increase in planned missingness designs in psychological research, we also suggest using Bayesian estimation as a potential alternative to ML estimation for analyses using data from planned missingness designs.  相似文献   

19.
The purpose of this study was to investigate the methods of estimating the reliability of school-level scores using generalizability theory and multilevel models. Two approaches, ‘student within schools’ and ‘students within schools and subject areas,’ were conceptualized and implemented in this study. Four methods resulting from the combination of these two approaches with generalizability theory and multilevel models were compared for both balanced and unbalanced data. The generalizability theory and multilevel models for the ‘students within schools’ approach produced the same variance components and reliability estimates for the balanced data, while failing to do so for the unbalanced data. The different results from the two models can be explained by the fact that they administer different procedures in estimating the variance components used, in turn, to estimate reliability. Among the estimation methods investigated in this study, the generalizability theory model with the ‘students nested within schools crossed with subject areas’ design produced the lowest reliability estimates. Fully nested designs such as (students:schools) or (subject areas:students:schools) would not have any significant impact on reliability estimates of school-level scores. Both methods provide very similar reliability estimates of school-level scores.  相似文献   

20.
The present study examines bias in parameter estimates and standard error in cross-classified random effect modeling (CCREM) caused by omitting the random interaction effects of the cross-classified factors, focusing on the effect of a sample size within cells and ratio of a small cell. A Monte Carlo simulation study was conducted to compare the correctly specified and the misspecified CCREM. While there was negligible bias in fixed effects, substantial biases were found in the random effects of the misspecified model depending on the number of samples within a cell and the proportion of small cells. However, in the case of the correctly specified model, no bias occurred. The present study suggests considering the random interaction effects when conducting CCREM to avoid overestimation of variance components and to calculate an accurate value of estimation. The implications of this study are to illuminate the conditions of cross-classification ratio and to provide a meaningful reference for applied researchers using CCREM.  相似文献   

设为首页 | 免责声明 | 关于勤云 | 加入收藏

Copyright©北京勤云科技发展有限公司  京ICP备09084417号