of 24

Multiple-group comparisons vs. dierential item functioning: Comparing two tests for measurement equivalence

0 views
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Share
Description
Multiple-group comparisons vs. dierential item functioning: Comparing two tests for measurement equivalence To be presented at the 5 th Conference of the European Survey Research Association, July 15-19,
Transcript
Multiple-group comparisons vs. dierential item functioning: Comparing two tests for measurement equivalence To be presented at the 5 th Conference of the European Survey Research Association, July 15-19, 2013, Ljubljana Dominik Becker & Jasmin Schwanenberg Version of July 17, 2013 Authors' Aliation: Technical University of Dortmund Institute for School Development Research Martin-Schmeiÿer-Weg Dortmund Germany Contact: Multiple-group comparisons vs. dierential item functioning: Comparing two tests for measurement equivalence Idealiter, many concepts utilized in empirical social research should not be measured by manifest but in terms of latent variables (Bollen, 2002). If, in addition, the sample consists of several subgroups, and group-mean comparisons are intended, in a second step, measurement invariance of the implied latent variable model has to be tested for (Steenkamp and Baumgartner, 1998). As regards variable-centered latent variable models (in contrast to observation-centered latent variable models such as latent class analysis), two group-specic invariance tests can be referred to: multiple group comparisons in the framework of conrmatory f actor analysis (CFA; cf. Millsap and Yun-Tein, 2004), and tests for dierential item f unctioning (DIF) in the framework of item response theory (IRT; cf. Holland and Wainer, 1993). This contribution aims to compare tests for measurement equivalence based on a sample of 31 upper-track secondary s from the German federal state of North Rhine-Westphalia. Based on several indicators for parent involvement in (e.g., helping out with events; attending parentteacher meetings) as measured in the parent survey of our sample (N = 2729), we rst analyze whether CFA and IRT arrive at the same conclusion regarding the dimensionality of our latent variable 'parent involvement'. In a second step, for both CFA and IRT we simultaneously estimate measurement models for parents both with and without migration background to validate if the same factor structure holds in dierent subgroups. And third, we elaborate on how to test for dierences of parameter estimates between groups in both frameworks. 1 Introduction In survey research it is well known that group-specic attributes may aect individuals' response behavior (e.g. Alwin and Krosnick, 1991). This holds particularly true if these attributes are based on latent variables which implies that individuals from dierent groups base their judgements on a common scale (cf. already Jöreskog, 1971). For instance, individuals' cultural background can inuence individuals' responses and thereby also intercorrelations between the respective items in such a manner that the manifest, i.e. observed indicators reect 'true' group dierences on the level of the underlying latent construct only in a rather distorted manner. Yet, these distortions remain unrevealed unless the measurement equivalence of these latent scales is not explicitly tested for. Hence, it is of utmost importance to test for this measurement equivalence if the research question to be answered is aimed at between-group dierences. 1 Although a common framework of latent variable modeling has been proposed (Bollen, 2002; Muthén, 2002; Skrondal and Rabe-Hesketh, 2004, 2007; Kankara² et al., 2011; Bartholomew et al., 2011), practical applications of either survey researchers or psychometricians usually stick to conrmatory f actor analysis (CFA; cf. Jöreskog, 1969; Bollen, 1989) or item response theory (IRT Rasch, 1960; Lord, 1980; de Boeck and Wilson, 2004) as two particular types of latent variable models (that will be explained more thoroughly in chapter 2). Moreover, this 'constraint' holds also true regarding tests for measurement invariance in the respective framework which are called multiple group conrmatory f actor analysis (MGCFA) for the former (for an overview on the literature see Vandenberg and Lance, 2000) and tests for dierential item f unctioning (DIF) for the latter (Holland and Wainer, 1993; Zumbo, 2007). Regarding explicit group comparisons of latent constructs, only few studies asked if both types of tests for measurement invariane arrive at the same conclusion concerning (non)invariance of the underlying latent variable or particular manifest indicators (Reise et al., 1993; Raju et al., 2002; Meade and Lautenschlager, 2004). From a conceptual viewpoint, most of these studies are limited towards latent variables from management science or work satisfaction (Raju et al., 2002; Meade and Lautenschlager, 2004). Hence, the conceptual contribution of the study at hand is to extend the comparisons of MGCFA and DIF available hitherto onto a topic of growing interest in educational research, namely parent involvement (Fan and Chen, 2001; Jeynes, 2003, 2005, 2007; Hill and Tyson, 2009). Concretely, based on cross-section data of N = 2, 729 parents of German 5 th graders, we ask whether a given measurement model for parent involvement equally holds for students with dierent forms of migration background (both parents native German, one parent native German, both parents non-native). Our methodological contribution is that while all preceding MGCFA-DIF comparisons were restricted to one-dimensional latent variable models, we estimate multidimensional CFA and IRT models on which tests for measurement invariance is applied. The remainder of this paper is structured as follows: First, we provide a brief literature review on dierent forms of parent involvement wherefrom we deduce our multidimensional model of the underlying latent construct (section 2.1). Regarding statistical theory, we rst present the basic CFA and IRT models and notation (section 2.2) and then elaborate on the corresponding tests for measurement invariance (section 2.3). Having sharpened our research questions (section 2.4), we present our data, indicators for the measurement models, and covariates that will serve as grouping factors for testing measurement invariance (section 3). In section 4, we rst present our results from the unseparated measurement models where both CFA and IRT models are tted for the whole sample (4.1). Then we turn to the results from testing for measurement invariance in order to validate if tests for MGCFA and DIF lead to the same conclusion (section 4.2). Results show that while both CFA and IRT arrive at a three-dimensional factor model for the latent variable of parent engagement, CFA appears to be more sensitive as regards detecting mistting items. With respect to measurement invariance, though several dierences concerning particular items remain, also a 2 2 Theoretical and Methodological Background In this section, we will rst allude to the relevance of parent involvement in general, and we will review theoretical models on dierent forms of involvement as proposed hitherto that we synthesize into a rened three-tired model of parent involvement. Next, we summarize evidence regarding dierent forms of parent involvement in general, and to migration-specic dierences regarding parent involvement in particular. 2.1 Theory Relevance of parent involvement As demands on in general increase for instance, as countries have to face labor force challenges due to demographic shifts, public voices regularly plead for an intensied cooperation between and parental home, and particularly, for parents' participation in s' pedagogic work (Alba et al., 2011). In the (primarily Anglo-American) scientic literature, the term 'parent involvement' and 'parent engagement' are regularly used to refer to cooperation between parents and s (Finn, 1998; Cotton and Wikelund, 2001). Among those label, a variety of parental activities can be subsumed (Comer, 1993; Eccles and Harold, 1996; Finn, 1998; Epstein and Sanders, 2002): [P]arent involvement includes several dierent forms of participation in education and with the s. Parents can support their children's ing by attending functions [...]. They can become more involved in helping their children improve their work [...] and monitoring homework [...]. They can volunteer to help out with activities or work in the classroom. Or they can take an active role in the governance and decision making necessary for planning [...] (Cotton and Wikelund, 2001, p. 4) Although already previous studies tried to dierentiate between dierent forms of parent involvement, in the next paragraph, we present a rened multidimensional theoretical model that we believe to represent the underlying latent construct more adequately. Forms of parent involvement The perhaps most widespread theoretical model of parent involvement was proposed by Epstein (1987) and colleagues (Connors and Epstein, 1995; Epstein and Sanders, 2002). The authors dierentiate between -based involvement such as volunteering at or communication between parents and teachers, and home-based involvement, i.e. educational support at home (also see Hill and Tyson, 2009). A three-tired model of parent involvement that refers to the underlying psychological mechanisms was developed by Grolnick and Slowiaczek (1994) who dierentiate between behavioral involvement, cognitive-intellectual involvement, and personal involvement. Behavioral involvement includes activities like going to and participating at open houses which may eect that the child adapts her behavior towards the perceived importance of ing. Parents' personal involvement includes the child's aective experience that the parent cares about (Grolnick and Slowiaczek, 1994, 239). And 3 nally, cognitive-intellectual involvement refer to the child's exposure to cognitively stimulating activities and materials such as books and current events (ibid.) Both theoretical models can be criticized for several reasons: On the one hand, the Epstein (1987) model only addresses the lieu of involvement, but remains silent on different forms of involvement from a conceptual point of view. On the other hand, as Grolnick and Slowiaczek (1994) explicitly concede that parental behavioral involvement may shape the child's attitudes towards the importance of ing as well, the conceptual dierence of behavioral involvement from personal involvement remains obscure. Another shortcoming of the Grolnick and Slowiaczek (1994) model is that cognitiveintellectual involvement strongly interferes with the idea of cultural capital (Bourdieu, 1986; Lareau and Weininger, 2003; Goldthorpe, 2007) so that dierences in parental involvement as a function of parental cultural capital could not be assessed. 1 Focusing on the strengths rather than on the shortcomings of both present theoretical models and further empirical evidence (Epstein, 1987; Comer, 1993; Grolnick and Slowiaczek, 1994; Epstein, 1995; Eccles and Harold, 1996; Finn, 1998; Epstein and Sanders, 2002; Sacher, 2008; also see Fan and Chen, 2001; Jeynes, 2003, 2005, 2007; Hill and Tyson, 2009), we propose a theoretical model of parent involvement covering the following three dimensions: 2 Organizational involvement covers all aspects of parental voluntary activities at. This includes cooperation in preparing events, accompanying class journeys and supporting extra-curricular activities (Comer, 1993; Eccles and Harold, 1996; Finn, 1998; Epstein and Sanders, 2002; Sacher, 2008). Thus, organizational involvement covers some, but not all of the activities that Grolnick and Slowiaczek (1994) denoted as related. The remainder of -related involvement can be labeled as conceptual involvement. This refers to parental participation in governance activities, in committees, and in P arent-t eacher Associations (PTAs). Also Epstein (1995, 704) referred to related decision making . And nally, learning-related engagement includes all forms of related support parents provide either at home (e.g. helping with homework) or outside (e.g. attending teacher-parent meetings or information events at ). This form of learning-related involvement is referred to by a number of international studies as well (Comer, 1993; Grolnick and Slowiaczek, 1994; Epstein, 1995; Eccles and Harold, 1996; Finn, 1998; Epstein and Sanders, 2002). Hence, in sum, our three-tired theoretical model of parent involvement overcomes Epstein's mechanistic classication of -related and home-related involvement in making a more conceptual dierentiation between learning-related, organizational, and conceptual involvement. Furthermore, we sharpen the distinction between parent involvement on the one hand and cultural capital on the other hand by focusing on the educational dimension only. 1 A reason for this amalgamation could be that Grolnick and Slowiaczek (1994, 238) aim to integrate both developmental and and educational constructs in their denition of parental involvement. 2 This model was rst proposed by Schwanenberg et al. (2013). 4 Migration-specic dierences of parent involvement In spite of the growing interest in parent involvement in general, evidence concerning dierences with respect to migration background in general or according to specic ethnic groups in particular is still mixed. While some studies attributed a lesser extend of involvement by non-native parents to potentially diering beliefs about the role of parents in (e.g. McLanahan, 1985; Milne et al., 1986; Lareau, 1987; Stevenson and Baker, 1987; Delgado-Gaitan, 1991; Connell et al., 1994; Turney and Kao, 2009), others emphasized that this lesser amount of minority parents is limited towards particular forms of participation, whereas in other domains, they engage more frequently (e.g. Keith et al., 1993; Keith and Lichtman, 1994; Sui-Chu and Willms, 1996; Catsambis and Garland, 1997; also see Desimone, 1999). More recent contributions predominantly pointed towards migration dierences with respect to the eect of parent involvement on student achievement (see e.g. recent metaanalyses by Fan and Chen, 2001; Jeynes, 2003; Hong and Ho, 2005; Jeynes, 2005, 2007; Hill and Tyson, 2009). Among those, however, only very few studies tted latent variable models for parent involvement (Desimone, 1999; Hong and Ho, 2005) 3, whereas only Hong and Ho (2005) tested for measurement invariance across migration groups. 4 Hence, apart from the methodological contribution of comparing two dierent methods to test for measurement invariance, we regard it to be another crucial conceptual contribution to do so particularly for the subject matter of parent involvement. 2.2 Measurement Models While for a long time survey researchers and psychometricians remained restricted to their domain-specic latent variable models conrmatory f actor analysis (CFA) in case of survey researchers, and i tem r esponse theory (IRT) in case of psychometricians, a common framework of generalized latent variable models has been proposed in order to unify coexisting approaches, and to regard them as special cases of this more general framework. Its basic idea is that individual answers reect respondents' positions on the underlying latent variable, and that once the latent variable is known, manifest indicators should be independent from each other. This idea can be expressed in more formal terms: f(θ, y) = f(θ)f(y Θ) = f(θ) K f(y k Θ) (1) where Θ is a vector of latente variables, y is a vector of observed manifest indicators, and y k is the k th observed manifest variable (for analogous specications see Bartholomew 3 Note, however, that the study by Desimone (1999) suers from the circumstance that although she used conrmatory factor analysis to estimate latent variables of parent involvement, she did not extend her measurement model to a structural one. Instead she apparently used the factor scores of the measurement model to build composites that were then used as predictors in a conventional OLS regression. 4 Hong et al. (2010) tted CFAs for parent involvement as well, but only tested for measurement invariance across measurement points. k=1 5 et al., 2011; Skrondal and Rabe-Hesketh, 2004; Muthén, 2002; Kankara² et al., 2011; Bollen, 2002). These propositions also underly both CFA and IRT framework which will be desribed in the following. Conrmatory Factor Analysis The rst concept of conrmatory factor analysis was presented by Jöreskog (1969). Its idea is that each observed item x 1, x 2,..., x n is a function of one or more unobserved latent variables ξ that accounts for the variation in x, the factor loadings λ that express the magnitude of the relationship between ξ and x, and measurement error δ that expresses that even a well-chosen set of manifest items x will only be an imperfect approximation to the latent variable(s) ξ. Equation 2 shows the formal notation how a set of six items would be represented by a two-factor model, while items x 1 to x 3 load on the rst factor ξ 1, and items x 4 to x 6 on the second factor ξ 2. x 1 x 2 x 3 x 4 x 5 x 6 = λ 11 0 λ 21 0 λ λ 42 0 λ 52 0 λ 62 ( ξ1 ξ 2 ) + If the manifest indicators are continuous, maximum l ikelihood (ML) estimation can be used to minimize the dierence between the implied covariance matrix S and the observed covariance matrix Σ (Jöreskog, 1969, 184). If the manifest indicators are categorical, ML estimation as used for continuous indicators will lead to biased estimates. Here, a polychoric correlation matrix is used instead of a usual variance-covariance matrix, and coecients and their signicant values are obtained either via W eighted Least Squares estimation or via ML estimation with bootstrapped standard errors (Bollen, 1989, 433.). Item Response Theory Similar to CFA and in contrast to classical test theory, i tem r esponse theory also controls for measurement error. As the theoretical framework stems from psychometry and educational assessment research, the crucial parameters are labeled as person ability and item diculty. In the simple dichotomous Rasch (1960) model, the probability π of a test item i to be solved (π 1 ) by person n is modeled as π 1ni = exp(β n δ i ) 1 + exp(β n δ i ) where β refers to person ability and δ to item diculty. If items dispose of more than two categories, the partial credit model (Masters, 1982, 1988) can be applied which introduces an additional parameter x to estimate the probability to respond in a certain category. In more formal terms: δ 1 δ 2 δ 3 δ 4 δ 5 δ 6 (2) (3) 6 π nix = exp k j=0 n(β n δ ij ) m k=0 exp k j=0 n(β n δ ij ) for x = 0, m (4) where π nix is the probability of person n with ability β n to respond in category x (x = 0, 1,..., m) of item i (Masters, 1988, p. 284). Both the basic dichotomous Rasch (1960) model and the PCM (Masters, 1982, 1988) are estimated by means of the conditional maximum likelihood function (Andersen, 1970). 2.3 Testing for Measurement Invariance Testing for measurement invariance aims to answer the question if one instrument measures the same construct in dierent group of respondents. In the framework of generalized latent variables, this means that the manifest variables y must indicate the latent variables Θ in dierent groups g in the same way (Kankara² et al., 2011, p. 283): f(y Θ, g) = f(y Θ) (5) CFA: Multiple Group Comparison In the CFA framework, testing for measurement invariance is dissected into (at least) 5 three separate steps: testing for (partial) congural invariance testing for (partial) metric invariance testing for (partial scalar invariance Testing for congural invariance asks whether a common measurement model can be established in each group. For instance, if in the rst group, a two-factorial model suited the data best, and in the second group, a three-factorial model, this would violate the condition of congural invariance. Given a common measurement model of all groups has been arrived at, testing for metric invariance asks whether the factor loading of this common measurement model are equal in all groups. Technically, this is performed by constraining the factor loadings to be equal between groups and investigating whether the constrained model ts the data signicantly worse than the unconstrained one. If not, the ne
Related Search
Advertisements
Related Docs
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks