ŷ = b0 + b1x
) ω
2011-2012 Research Methodology Series
( ²
Statistics and Research Methodology Unit
σ 8
Nebraska Center for Research on Children, Youth, Families and Schools
= n
&
National Center for Research on Rural Education
F = (MST + MSS×I×T) ÷ (MST×S + MSI×T)
/ z
+ ²
Data Collection via Video: An Introduction to Software for Coding Research Videos Friday, Nov. 18, 11:30 a.m. - 1:00 p.m. 242 Mabel Lee Hall, UNL City Campus Frances Chumney, B.S. CYFS Statistics and Measurement Consultant
Video recordings are a common tool for collecting data on behaviors and human interactions in social science research. This method makes it possible for researchers to systematically record multiple or complex phenomena in contexts where live observations might be distracting or impossible due to resource limitations. In response to the widespread use of video recordings in research, several software platforms have been developed for the coding and analysis of video observations. This presentation describes the advantages and challenges of collecting data from video recordings; outlines important considerations in the development of coding schemes for video data collection; offers an overview of some of the most common and flexible software available; and includes demonstrations of select software with real research data examples. Frances Chumney earned her bachelor’s in psychology from East Tennessee State University in 2007. She is currently a doctoral student in Quantitative, Qualitative and Psychometric Methods in the Educational Psychology Program at the University of Nebraska–Lincoln. She serves in project management roles for the National Center for Research on Rural Education (R2Ed). Her research interests include estimation of structural equation models with small samples, categorical data analysis, and the translation of advanced statistical practices for applied researchers.
An Introduction to Secondary Data Analysis Friday, Dec. 9, 11:30 a.m. - 1:00 p.m. 234 Mabel Lee Hall, UNL City Campus Natalie Koziol, M.A. CYFS Statistics and Measurement Consultant Ann Arthur, M.S. CYFS Statistics and Measurement Consultant Secondary data analysis is the analysis of data collected by an external source. Analysis of largescale secondary data sets is appealing because it is inexpensive and accessible, and because samples tend to be much larger and more representative of target populations than samples obtained by individual researchers. Nevertheless, applied educational and psychological researchers analyzing secondary data often encounter additional challenges due to complex sampling designs. This presentation will (a) discuss the advantages and disadvantages of using secondary data, (b) introduce relevant methodological considerations such as sampling weights, standard error estimation, and suppression, and (c) illustrate these concepts with examples from data sets sponsored by the National Center for Education Statistics (NCES). Funding and training opportunities will be briefly addressed. Natalie Koziol earned her master’s in Quantitative, Qualitative, and Psychometric Methods in Educational Psychology from the University of Nebraska–Lincoln in 2010 and was awarded the 2011 Folsom Distinguished Master’s Thesis Award. She is currently a doctoral student in Quantitative, Qualitative and Psychometric Methods in the Educational Psychology Program at UNL, with appointments as a CYFS Statistics and Measurement Consultant and as instructor of an undergraduate-level statistical methods course. Her research interests include latent variable methods, particularly as they relate to issues of estimation. Ann Arthur earned her master’s in Survey Research and Methodology from the University of Nebraska-Lincoln in 2011. She is currently a doctoral student in Quantitative, Qualitative and Psychometric Methods in the Educational Psychology Program at UNL and serves as a CYFS Statistics and Measurement Consultant. Her research interests include latent variable methods and measurement in education and survey data.
2011-2012 Research Methodology Series
Not All Students Need to be Assessed: Research Designs Where Data are Intentionally Missing Friday, Jan. 27, 11:30 a.m. - 1:00 p.m. 265 Mabel Lee Hall, UNL City Campus James Bovaird, Ph.D. Director, CYFS Statistics and Research Methodology Unit Associate Professor, Department of Educational Psychology
It can be daunting, difficult, or outright impossible to gain access to adequately large participant samples in order to test research hypotheses under conventional thinking. This talk will outline research design decisions that may enable researchers to answer important questions with smaller assessment batteries or smaller sample sizes than what may traditionally be recommended. Several practical examples will be presented to illustrate two broad scenarios: All participants are assessed, but not on all instruments; or all participants are assessed on all instruments, but not all participants initially anticipated actually participate. The literature bases on planned missing data designs and sequentially designed experiments will be used to support empirical examples of potential resource savings. James Bovaird earned his doctorate in Quantitative Psychology from the University of Kansas in 2002. He currently serves as Associate Professor of Quantitative, Qualitative and Psychometric Methods in Educational Psychology at the University of Nebraska–Lincoln and Director of the CYFS Statistics and Research Methodology Unit. His research interests involve determining the proper use of latent variable methods—including structural equation modeling, item response theory, and multilevel modeling—and applying these methods to advance substantive research in the social and behavioral sciences.
Evaluation for Specific Users and Specific Purposes: The Utilization Focused Evaluation Friday, Feb. 24, 11:30 a.m. - 1:00 p.m. 265 Mabel Lee Hall, UNL City Campus Greg Welch, Ph.D. Research Assistant Professor, CYFS
There are many approaches and techniques available in an evaluator’s tool box. The Utilization Focused Evaluation (UFE) focuses on the utility of the evaluation. The goal of this approach is commonly phrased as “intended use by intended users,” meaning that the focus becomes to involve all stakeholders in the evaluation process. This presentation will discuss the basic premises behind the UFE approach, along with the role of UFE in different types of evaluations and across evaluation settings. In addition to addressing the steps involved in implementing the UFE approach, this talk will cover the strengths and weaknesses of the UFE. The importance of establishing relationships with stakeholders and empowering stakeholders in the evaluation process will be discussed. The presentation will also examine situations in which the UFE approach is and is not warranted. Greg Welch earned his doctorate in Research Methodology in Education from the University of Pittsburgh in 2007. He currently serves as Research Assistant Professor for the CYFS Statistics and Research Methodology Unit. His research interests include structural equation modeling, latent curve analysis and educational policy.
2011-2012 Research Methodology Series
Characterizing Changing Classifications: Practical Illustrations of Latent Transition Analysis Friday, March 30, 11:30 a.m. - 1:00 p.m. 139 Teachers College Hall, UNL City Campus Ji Hoon Ryoo, Ph.D. CYFS Postdoctoral Fellow Chaorong Wu, M.A. CYFS Statistics and Measurement Consultant Carina McCormick, M.A. CYFS Statistics and Measurement Consultant Sometimes true latent individual differences are categorical or qualitative rather than continuous. Likewise, change in measured category memberships over time may reflect transitions between latent classes rather than change in levels. Additional caveats may include the presence of unmeasured heterogeneity, or subgroups, among participants, or substantial measurement error leading to incorrect observed classifications. Latent transition analysis (LTA) is a latent variable modeling approach that enables the researcher to estimate the prevalence and degree of transitioning between latent class memberships over time. In contrast to a latent growth curve approach, which models change over an entire developmental period, LTA allows modeling of period-to-period change in status. This talk will demonstrate LTA with two empirical data sets, exploring change in psychological status and change in reading proficiency designation. Ji Hoon Ryoo earned his doctorate in Quantitative Methods in Education from the University of Minnesota in 2010. He currently serves as a CYFS Postdoctoral Fellow, working in the area of statistical approaches and research methods for the educational and social-behavioral sciences. He earned the 2011 University of Nebraska–Lincoln Outstanding Postdoc Award. His research interests include analysis of longitudinal and multilevel data, educational measurement, structural equation modeling and program evaluation. Chaorong Wu earned his master’s in Psychology from Peking University (China) in 2004. He is currently a doctoral student in Quantitative, Qualitative and Psychometric Methods in the Educational Psychology Program at the University of Nebraska–Lincoln and serves as a CYFS Statistics and Measurement Consultant. His research interests include multilevel modeling, categorical data analysis and finite mixture modeling. Carina McCormick earned her master’s in Quantitative, Qualitative, and Psychometric Methods in Educational Psychology from the University of Nebraska–Lincoln in 2009 and was awarded the 2010 Folsom Distinguished Master’s Thesis Award. She is currently a doctoral student in Quantitative, Qualitative and Psychometric Methods in the Educational Psychology Program at UNL, with appointments as a CYFS Statistics and Measurement Consultant and as instructor of an undergraduatelevel statistical methods course. Her research interests include applied psychometric issues in K-12 achievement testing, especially those involving validation, fairness, reading assessment, accountability and advanced modeling of results.
Compliance and Fidelity in Intervention Research Friday, April 27, 11:30 a.m. - 1:00 p.m. 265 Mabel Lee Hall, UNL City Campus James Bovaird, Ph.D. Director, CYFS Statistics and Research Methodology Unit Associate Professor, Department of Educational Psychology Chaorong Wu, M.A. CYFS Statistics and Measurement Consultant Participant compliance is a common confounding factor in randomized experiments conducted in the social and behavioral sciences and education. Compliance, or fidelity, means that participants in a randomized trial fully comply with the expectations of their role in research. Noncompliance may reduce the statistical power to detect intervention effects, thereby influencing the internal and external validity of an intervention. Therefore, the possibility of noncompliance should be considered during both planning and evaluation of randomized experiments, especially randomized control trials (RCTs). Methods for accounting for participant noncompliance include both methodological design-based strategies (e.g., usage of standardized guidelines, observation, self-report evaluations) and statistical control techniques, including the intention-to-treat (ITT) and complier average causal effect (CACE) models. This talk will discuss a number of issues surrounding the very definition of noncompliance, strategies for minimizing treatment noncompliance, and relevant measurement and statistical approaches. James Bovaird earned his doctorate in Quantitative Psychology from the University of Kansas in 2002. He currently serves as Associate Professor of Quantitative, Qualitative and Psychometric Methods in Educational Psychology at the University of Nebraska–Lincoln and Director of the CYFS Statistics and Research Methodology Unit. His research interests involve determining the proper use of latent variable methods—including structural equation modeling, item response theory, and multilevel modeling—and applying these methods to advance substantive research in the social and behavioral sciences. Chaorong Wu earned his master’s in Psychology from Peking University (China) in 2004. He is currently a doctoral student in Quantitative, Qualitative and Psychometric Methods in the Educational Psychology Program at the University of Nebraska–Lincoln and serves as a CYFS Statistics and Measurement Consultant. His research interests include multilevel modeling, categorical data analysis and finite mixture modeling.
2011-2012 Research Methodology Series
ŷ = b0 + b1x
CYFS.UNL.EDU
= n
σ 8
) ω
R2ED.UNL.EDU
F = (MST + MSS×I×T) ÷ (MST×S + MSI×T)
( ²
/ z
+ ²