integrative data analysis in clinical psychology research

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

We're Hiring!
Help Center

Integrative Data Analysis in Clinical Psychology Research

2013, Annual Review of Clinical Psychology

Related Papers

Psychological Methods

Andrea Hussong

Scott Hofer , ANDREA PICCININ

Developmental Psychology

Journal of Applied Psychology

Edward Levine

Daniel Quintana

Meta-analysis synthesizes a body of research investigating a common research question. Outcomes from meta-analyses provide a more objective and transparent summary of a research area than traditional narrative reviews. Moreover, they are often used to support research grant applications, guide clinical practice and direct health policy. The aim of this article is to provide a practical and nontechnical guide for psychological scientists that outlines the steps involved in planning and performing a meta-analysis of correlational datasets. I provide a supplementary R script to demonstrate each analytical step described in the paper, which is readily adaptable for researchers to use for their analyses. While the worked example is the analysis of a correlational dataset, the general meta-analytic process described in this paper is applicable for all types of effect sizes. I also emphasise the importance of meta-analysis protocols and pre-registration to improve transparency and help avoid unintended duplication. An improved understanding this tool will not only help scientists to conduct their own meta-analyses but also improve their evaluation of published meta-analyses.

Psychonomic bulletin & review

Catherine Fritz

The Psychonomic Society (PS) adopted New Statistical Guidelines for Journals of the Psychonomic Society in November 2012. To evaluate changes in statistical reporting within and outside PS journals, we examined all empirical papers published in PS journals and in the Experimental Psychology Society journal, The Quarterly Journal of Experimental Psychology (QJEP), in 2013 and 2015, to describe these populations before and after effects of the Guidelines. Comparisons of the 2013 and 2015 PS papers reveal differences associated with the Guidelines, and QJEP provides a baseline of papers to reflect changes in reporting that are not directly influenced by the Guidelines. A priori power analyses increased from 5% to 11% in PS papers, but not in QJEP papers (2%). The reporting of effect sizes in PS papers increased from 61% to 70%, similar to the increase for QJEP from 58% to 71%. Only 18% of papers reported confidence intervals (CIs) for means; only two PS papers in 2015 reported CIs for ...

David Moreau

Psychology researchers are rapidly adopting open science practices, yet clear guidelines on how to apply these practices to meta-analysis remain lacking. In this tutorial, we describe why open science is important in the context of meta-analysis in psychology, and suggest how to adopt the 3 main components of open science: preregistration, open materials, and open data. We first describe how to make the preregistration as thorough as possible-and how to handle deviations from the plan. We then focus on creating easy-to-read materials (e.g., search syntax, R scripts) to facilitate reproducibility and bolster the impact of a meta-analysis. Finally, we suggest how to organize data (e.g., literature search results, data extracted from studies) that are easy to share, interpret, and update as new studies emerge. For each step of the meta-analysis, we provide example templates, accompanied by brief video tutorials, and show how to integrate these practices into the Open Science Framework (https://osf.io/q8stz/). Translational Abstract Open science practices are gaining traction in the psychological research ecosystem. A number of contributions have been published recently with the goal to facilitate open science practices in a number of research designs; however, clear guidelines about how to integrate these practices in the context of meta-analyses have been lacking. Here, we describe why and how open science practices can be applied in the context of meta-analysis. After highlighting the benefits of the three main components of open science (preregistration, open materials, and open data), we propose a series of nine templates and video tutorials to help researchers with all aspects of an open meta-analysis. These materials are designed to facilitate the evaluation of empirical findings and help increase the impact of a meta-analysis-both important steps toward robust and valid inferences in psychological science. Meta-analyses play a central role in synthesizing research findings. With a high value-to-cost ratio, especially in fields where studies are typically onerous or expensive (e.g., clinical trials, interventions, fMRI studies), meta-analyses can maximize information and reduce the influence of biased studies, often providing the basis for decisions or policies. Yet, despite the recent rise of open science across fields of investigation (Federer et al., 2018; Nosek & Lindsay, 2018; Popkin, 2019), clear and well-defined guidelines about how to integrate meta-analyses within the open science ecosystem remain lacking in psychology. This gap in the literature has important consequences, as it suggests that some-if not all-of these practices are not crucial in the context of meta-analyses, or that the specificities of meta-analyses do not warrant customized suggestions. In this tutorial, we argue that open science practices are critical to ensure robust and reliable meta-analyses, and propose specific practical recommendations to facilitate preregistration, open materials and open data. For each component, we provide templates that can help standardization, while allowing the necessary flexibility for researchers to adapt them to a range of designs. With the many small decisions that go into a meta-analysis, careful planning and clear documentation of these decisions is critical for transparency. The templates provided help to facilitate this. For instance, registering the analysis script ensures consideration of statistical decisions in advance of the literature search; registering the data extraction file ensures early decision-making about which variables to extract from included studies. All templates-together with tips on how to best utilize them-are also illustrated with

Thomas Debray , Gert van Valkenhoef , Orestis Efthimiou , Johannes Reitsma

Individual participant data (IPD) meta-analysis is an increasingly used approach for synthesizing and investigating treatment effect estimates. Over the past few years, numerous methods for conducting an IPD meta-analysis (IPD-MA) have been proposed, often making different assumptions and modeling choices while addressing a similar research question. We conducted a literature review to provide an overview of methods for performing an IPD-MA using evidence from clinical trials or non-randomized studies when investigating treatment efficacy. With this review, we aim to assist researchers in choosing the appropriate methods and provide recommendations on their implementation when planning and conducting an IPD-MA.

Heike Gerger

Meta-analyses are an important tool to evaluate the literature. It is essential that meta-analyses can easily be reproduced to allow researchers to evaluate the impact of subjective choices on meta-analytic effect sizes, but also to update meta-analyses as new data comes in, or as novel statistical techniques (for example to correct for publication bias) are developed. Research in medicine has revealed meta-analyses often cannot be reproduced. In this project, we examined the reproducibility of meta-analyses in psychology by reproducing twenty published meta-analyses. Reproducing published meta-analyses was surprisingly difficult. 96% of meta-analyses published in 2013-2014 did not adhere to reporting guidelines. A third of these meta-analyses did not contain a table specifying all individual effect sizes. Five of the 20 randomly selected meta-analyses we attempted to reproduce could not be reproduced at all due to lack of access to raw data, no details about the effect sizes extrac...

RELATED PAPERS

Muhtadi Muhtadi

Journal of Environmental Chemical Engineering

Gary Caldwell

SSRN Electronic Journal

claudio Albanese

Journal of clinical and diagnostic research : JCDR

anisa kusuma

Onomázein Revista de lingüística filología y traducción

Susanne Ramadan

Mutiara eka Putri XII OTKP-3

Luis Schoueri

Ethnobotany Research and Applications

Prakash Phondani

Margherita Bolla , Roby Stuani

Journal Nursing Care and Biomolecular

valentina lumbantobing

IEEE Transactions on Biomedical Engineering

Nitish Thakor

Acta gastroenterologica Latinoamericana

Daniel Peralta

Tiago Sales

roman caspar

Biblionline

Keila Fernanda de Souza Cruz

Alysa Pomer

Journal of Molecular Structure

Jean-Louis Colot

International journal of Pharmacy and Pharmaceutical Sciences

International Journal of Pharmacy and Pharmaceutical Sciences (IJPPS) , Nischal Tyagi

Bryony Hoskins

Jurnal PADMA: Pengabdian Dharma Masyarakat

Muhammad rizky Maulana

Medicinska istraživanja

Stefan Dugalić

Najibul Khoir

Alessandro Poggi

PROGRAMA (completo) Congreso Internacional Virtual 21. Un año, dos conmemoraciones, diversas Historias de América, 1521-1821.

Juan Camilo Ritoré Perez

20 Integrative Data Analysis from a Unifying Research Synthesis Perspective

Published: January 2018
Cite Icon Cite
Permissions Icon Permissions

Integrative data analysis (IDA) is a promising new approach in psychological research and has been well received in the field of alcohol research. This chapter provides a larger unifying research synthesis framework for IDA. Major advantages of IDA of individual participant-level data include better and more flexible ways to examine subgroups, model complex relationships, deal with methodological and clinical heterogeneity, and examine infrequently occurring behaviors. However, between-study heterogeneity in measures, designs, and samples and systematic study-level missing data are significant barriers to IDA and, more broadly, to large-scale research synthesis. Based on the authors’ experience working on the Project INTEGRATE data set, which combined individual participant-level data from 24 independent college brief alcohol intervention studies, it is also recognized that IDA investigations require a wide range of expertise and considerable resources and that some minimum standards for reporting IDA studies may be needed to improve transparency and quality of evidence.

Signed in as

Institutional accounts.

GoogleCrawler [DO NOT DELETE]
Google Scholar Indexing

Personal account

Sign in with email/username & password
Get email alerts
Save searches
Purchase content
Activate your purchase/trial code

Institutional access

Sign in with a library card Sign in with username/password Recommend to your librarian
Institutional account management
Get help with access

Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:

IP based access

Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.

Sign in through your institution

Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institutionâ€™s website and Oxford Academic.

Click Sign in through your institution.
Select your institution from the list provided, which will take you to your institution's website to sign in.
When on the institution site, please use the credentials provided by your institution. Do not use an Oxford Academic personal account.
Following successful sign in, you will be returned to Oxford Academic.

If your institution is not listed or you cannot sign in to your institutionâ€™s website, please contact your librarian or administrator.

Sign in with a library card

Enter your library card number to sign in. If you cannot sign in, please contact your librarian.

Society Members

Society member access to a journal is achieved in one of the following ways:

Sign in through society site

Many societies offer single sign-on between the society website and Oxford Academic. If you see â€˜Sign in through society siteâ€™ in the sign in pane within a journal:

Click Sign in through society site.
When on the society site, please use the credentials provided by that society. Do not use an Oxford Academic personal account.

If you do not have a society account or have forgotten your username or password, please contact your society.

Sign in using a personal account

Some societies use Oxford Academic personal accounts to provide access to their members. See below.

A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.

Some societies use Oxford Academic personal accounts to provide access to their members.

Viewing your signed in accounts

Click the account icon in the top right to:

View your signed in personal account and access account management features.
View the institutional accounts that are providing access.

Signed in but can't access content

Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.

For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.

Our books are available by subscription or purchase to libraries and institutions.

About Oxford Academic
Publish journals with us
University press partners
What we publish
New features
Open access
Rights and permissions
Accessibility
Advertising
Media enquiries
Oxford University Press
Oxford Languages
University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

Copyright © 2024 Oxford University Press
Cookie settings
Cookie policy
Privacy policy
Legal notice

This Feature Is Available To Subscribers Only

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Introduction to the Special Issue on Innovations and Applications of Integrative Data Analysis (IDA) and Related Data Harmonization Procedures in Prevention Science

Published: 09 November 2023
Volume 24 , pages 1425–1434, ( 2023 )

Cite this article

Antonio A. Morgan-López ORCID: orcid.org/0000-0003-4706-9964 1 ,
Catherine P. Bradshaw 2 , 3 &
Rashelle J. Musci 3

260 Accesses

Explore all metrics

This paper serves as an introduction to the special issue of Prevention Science entitled, “Innovations and Applications of Integrative Data Analysis (IDA) and Related Data Harmonization Procedures in Prevention Science.” This special issue includes a collection of original papers from multiple disciplines that apply individual-level data synthesis methodologies, including IDA, individual participant meta-analysis, and other related methods to harmonize and integrate multiple datasets from intervention trials of the same or similar interventions. This work builds on a series of papers appearing in a prior Prevention Science special issue, entitled “Who Benefits from Programs to Prevent Adolescent Depression?” (Howe, Pantin, & Perrino, 2018 ). Since the publication of this prior work, the use of individual-level data synthesis has increased considerably in and outside of prevention. As such, there is a need for an update on current and future directions in IDA, with careful consideration of innovations and applications of these methods to fill important research gaps in prevention science. The papers in this issue are organized into two broad categories of (1) evidence synthesis papers that apply best practices in data harmonization and individual-level data synthesis and (2) new and emerging design, psychometric, and methodological issues and solutions. This collection of original papers is followed by two invited commentaries which provide insight and important reflections on the field and future directions for prevention science.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Advancing Intervention and Prevention Research for Behavioral Health Problems Through Data Synthesis

Jane L. Pearson & Belinda E. Sims

Retrospective Psychometrics and Effect Heterogeneity in Integrated Data Analysis: Commentary on the Special Issue

George W. Howe & C. Hendricks Brown

Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials

Ahnalee Brincks, Samantha Montag, … C. Hendricks Brown

Ayer, L., Stevens, C., Reider, E., Sims, B., Colpe, L., & Pearson, J. (2023). Preventing youth suicide: Potential “crossover effects” of existing school-based programs. Prevention Science, 24 , 382–392.

Article PubMed Google Scholar

Barker, D. H., Bie, R., & Steingrimsson, J. (2023). Addressing systematic missing data in the context of causally interpretable meta-analysis. Prevention Science .

Bauer, D. J. (2017). A more general model for testing measurement invariance and differential item functioning. Psychological Methods, 22 , 507–.

Bauer, D. J., & Hussong, A. M. (2009). Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models. Psychological Methods, 14 , 101–.

Berry, V., Melendez-Torres, G. J., Axford, N., Axberg, U., de Castro, B. O., Gardner, F., & Leijten, P. (2022). Does social and economic disadvantage predict lower engagement with parenting interventions? An integrative analysis using individual participant data. Prevention Science .

Brincks, A., Perrino, T., Howe, G., Pantin, H., Prado, G., Huang, S., & Brown, C. H. (2018). Preventing youth internalizing symptoms through the Familias Unidas intervention: Examining variation in response. Prevention Science, 19, 49–59.

Brown, C. H., Brincks, A., Huang, S., Perrino, T., Cruden, G., Pantin, H., & Sandler, I. (2018). Two-year impact of prevention programs on adolescent depression: An integrative data analysis approach. Prevention Science, 19 , 74–94.

Cole, V. T., Hussong, A. M., Gottfredson, N. C., Bauer, D. J., & Curran, P. J. (2022). Informing harmonization decisions in integrative data analysis: Exploring the measurement multiverse. Prevention Science .

Connell, A. M., Seidman, S., Ha, T., Stormshak, E., Westling, E., Wilson, M., & Shaw, D. (2022). Long-term effects of the family check-up on suicidality in childhood and adolescence: integrative data analysis of three randomized trials. Prevention Science .

Connell, A. M., Stormshak, E., Dishion, T., Fosco, G., & Van Ryzin, M. (2018). The Family Check Up and adolescent depression: An examination of treatment responders and non-responders. Prevention Science, 19 , 16–26.

Article PubMed PubMed Central Google Scholar

Cooper, H., & Patall, E. A. (2009). The relative benefits of meta-analysis conducted with individual participant data versus aggregated data. Psychological Methods, 14 , 165–.

Curran, P. J. (2009). The seemingly quixotic pursuit of a cumulative psychological science: Introduction to the special issue. Psychological Methods, 14 , 77.

Curran, P. J., Hussong, A. M., Cai, L., Huang, W., Chassin, L., Sher, K. J., & Zucker, R. A. (2008). Pooling data from multiple longitudinal studies: The role of item response theory in integrative data analysis. Developmental Psychology, 44 , 365–.

Dagne, G. A., Brown, C. H., Howe, G., Kellam, S. G., & Liu, L. (2016). Testing moderation in network meta‐analysis with individual participant data. Statistics in Medicine, 35 , 2485–2502.

Diener, E., Northcott, R., Zyphur, M., & West, S. G. (2022). Beyond experiments. Perspectives in Psychological Science , 17 ,1101–1119 .

Dong, N., Herman, K. C., Reinke, W. M., Wilson, S. J., & Bradshaw, C. P. (2022). Gender, racial, and socioeconomic disparities on social and behavioral skills for K-8 students with and without interventions: An integrative data analysis of eight cluster randomized trials. Prevention Science .

Garber, J., Weersing, V. R., Hollon, S. D., Porta, G., Clarke, G. N., Dickerson, J. F., & Brent, D. A. (2018). Prevention of depression in at-risk adolescents: Moderators of long-term response. Prevention Science, 19 , 6–15.

Gardner, F., Leijten, P., Mann, J., Landau, S., Harris, V., Beecham, J., ... & Scott , S. (2017). Could scale-up of parenting programmes improve child disruptive behaviour and reduce social inequalities? Using individual participant data meta-analysis to establish for whom programmes are effective and cost-effective. Public Health Research, 5 (10).

Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5 (10), 3–8.

Article Google Scholar

Glass, G. V. (2000). Meta-analysis at 25.

Goldstein, A. B., & Avenevoli, S. (2018). Strength in numbers. Prevention Science, 19 , 109–111.

Gottfredson, N. C., Sterba, S. K., & Jackson, K. M. (2017). Explicating the conditions under which multilevel multiple imputation mitigates bias resulting from random coefficient-dependent missing longitudinal data. Prevention Science, 18 , 12–19.

Grant, S., Wendt, K. E., Leadbeater, B. J., Supplee, L. H., Mayo-Wilson, E., Gardner, F., & Bradshaw, C. P. (2022). Transparent, open, and reproducible prevention science. Prevention Science, 23 (4), 701–722. https://doi.org/10.1007/s11121-022-01336-w

Hensums, M., De Mooij, B., Kuijper, S. C., Fekkes, M., & Overbeek, G. (2022). What works for whom in school-based anti-bullying interventions? An individual participant data meta-analysis. Prevention Science .

Howe, G. W., & Brown, C. H. (this issue) Retrospective psychometrics and effect heterogeneity in integrated data analysis: Commentary on the special issue. Prevention Science .

Howe, G. W., Pantin, H., & Perrino, T. (2018). Programs for preventing depression in adolescence: Who benefits and who does not? An introduction to the supplemental issue. Prevention Science, 19 , 1–5.

Huh, D., Mun, E. Y., Walters, S. T., Zhou, Z., & Atkins, D. C. (2019). A tutorial on individual participant data meta-analysis using Bayesian multilevel modeling to estimate alcohol intervention effects across heterogeneous studies. Addictive Behaviors, 94 , 162–170.

Hussong, A. M., Curran, P. J., & Bauer, D. J. (2013). Integrative data analysis in clinical psychology research. Annual Review of Clinical Psychology, 9 , 61–89.

Keller, B. T., & Enders, C. K. (2021). Blimp user’s guide (Version 3). Retrieved From www.appliedmissingdata.com/multilevel-imputation.html

Kerr, D. C. R., Reinke, W. M., & Eddy, L. M. (2012). Trajectories of depressive symptoms and externalizing behaviors across adolescence: Associations with histories of suicide attempt and ideation in early adulthood. Suicide and Life-Threatening Behavior, 43 (1), 50–66.

Kush, J. M., Masyn, K. E., Amin-Esmaeili, M., Susukida, R., Wilcox, H. C., & Musci, R. J. (2023). Utilizing moderated non-linear factor analysis models for integrative data analysis: A tutorial. Structural Equation Modeling: A Multidisciplinary Journal, 30 , 149–164.

Lubinski, D., & Humphreys, L. G. (1996). Seeing the forest from the trees: When predicting the behavior or status of groups, correlate means. Psychology, Public Policy, and Law, 2 , 363–376.

MacKinnon, D. P., Valente, M. J., & Gonzalez, O. (2020). The correspondence between causal and traditional mediation analysis: The link is the mediator by treatment interaction. Prevention Science, 21 , 147–157.

Magee, K. E., Connell, A., Hipwell, A. E., Shaw, D., Westling, E., Keenan, K., & Stepp, S. (2022). Developmental models of depression, externalizing problems, and self-regulatory processes: Integrated data analysis across four longitudinal studies of youth. Prevention Science .

Mauricio, A. M., Mazza, G. L., Berkel, C., Tein, J. Y., Sandler, I. N., Wolchik, S. A., & Winslow, E. (2018). Attendance trajectory classes among divorced and separated mothers and fathers in the New Beginnings Program. Prevention Science, 19 , 620–629.

McDaniel, H. L., Saavedra, L. M., Morgan-López, A. A., Bradshaw, C. P., Lochman, J. E., Kaihoi, C. A., & Yaros, A. C. (2023). Harmonizing social, emotional, and behavioral constructs in prevention science: Digging into the weeds of aligning disparate measures. Prevention Science .

McNeish, D. (2022). Psychometric properties of sum scores and factor scores differ even when their correlation is 0.98: A response to Widaman and Revelle. Behavior Research Methods, 1–22.

Morgan-López, A. A., Hien, D. A., Saraiya, T. C., Saavedra, L. M., Norman, S. B., Killeen, T. K., & Consortium on Addiction, Stress and Trauma (CAST). (2022a). Estimating posttraumatic stress disorder severity in the presence of differential item functioning across populations, comorbidities, and interview measures: Introduction to Project Harmony. Journal of Traumatic Stress, 35 , 926–940.

Morgan-López, A. A., McDaniel, H. L., Bradshaw, C. P., Saavedra, L. M., Lochman, J. E., Kaihoi, C. A., & Yaros, A. C. (2022b). Design and methodology for an integrative data analysis of coping power: Direct and indirect effects on adolescent suicidality. Contemporary Clinical Trials, 115 , 106705.

Morgan-Lopez, A. A., Saavedra, L. M., Ramirez, D. D., Smith, L. M., & Yaros, A. C. (2022c). Adapting the multilevel model for estimation of the reliable change index (RCI) with multiple timepoints and multiple sources of error. International Journal of Methods in Psychiatric Research, 31 (2), e1906.

Morgan‐López, A. A., Saavedra, L. M., Hien, D. A., Norman, S. B., Fitzpatrick, S. S., Ye, A., & Back, S. E. (2023). Differential symptom weighting in estimating empirical thresholds for underlying PTSD severity: Toward a “platinum” standard for diagnosis? International Journal of Methods in Psychiatric Research , e1963.

Mun, E. Y., Jiao, Y., & Xie, M. (2016). Integrative data analysis for research in developmental psychopathology. Developmental Psychopathology: Theory and Method, 1 , 1042–1087.

Google Scholar

Mun, E. Y., Zhou, Z., Huh, D., Tan, L., Li, D., Tanner-Smith, E. E., ... & Larimer, M. E. (this issue). Brief Alcohol Interventions are Effective through 6 Months: Findings from marginalized zero-inflated poisson and negative binomial models in a two-step IPD meta-analysis. Prevention Science .

Musci, R. J., Kush, J. M., Masyn, K. E., Esmaeili, M. A., Susukida, R., Goulter, N., & Wilcox, H. C. (2023). Psychosis symptom trajectories across childhood and adolescence in three longitudinal studies: An integrative data analysis with mixture modeling. Prevention Science.

Muthén, L. K., & Muthén, B. O. (1998–2017). Mplus user’s guide. Eighth Edition. Los Angeles, CA: Muthén & Muthén.

Pearson, J. L., & Sims, B. E. (2023). Commentary on the special issue “innovations and applications of integrative data analysis (IDA) and related data harmonization procedures in prevention science.” Prevention Science .

Reider, E. E., & Sims, B. E. (2016). Family-based preventive interventions: Can the onset of suicidal ideation and behavior be prevented? Suicide and Life-Threatening Behavior, 46 , S3–S7.

Rosenbaum, P. R. (1984). The consequences of adjustment for a concomitant variable that has been affected by the treatment. Journal of the Royal Statistical Society: Series A (general), 147 (5), 656–666.

Russell, M. A., Coatsworth, J. D., Brown, A., Zaharakis, N., Mennis, J., Rodriguez, G. C., & Mason, M. J. (2022). Peer network counseling effects on substance use: An individual participant data meta-analysis integrating three randomized controlled trials. Prevention Science .

Saavedra, L. M., Lochman, J. E., Morgan-López, A. A., McDaniel, H. L., Bradshaw, C. P., Powell, N. P., Qu, L., Budavari, A. C., & Yaros, A. C. (in press) Collateral effects of coping power on caregiver symptoms of depression and long-term changes in child behavior. Development and Psychopathology .

Saavedra, L. M., Morgan-Lopez, A. A., Hien, D. A., Lopez-Castro, T., Ruglass, L. M., Back, S. E., & Hamblen, J. (2021). Evaluating treatments for posttraumatic stress disorder, alcohol and other drug use disorders using meta-analysis of individual patient data: Design and methodology of a virtual clinical trial. Contemporary Clinical Trials, 107 , 106479.

Saavedra, L. M., Morgan-López, A. A., West, S. G., Alegría, M., & Silverman, W. K. (this issue). Mitigating multiple sources of bias in a quasi-experimental integrative data analysis: Does treating childhood anxiety prevent substance use disorders in late adolescence/young adulthood?. Prevention Science.

Saavedra, L. M., Silverman, W. K., Morgan-Lopez, A. A., & Kurtines, W. M. (2010). Cognitive behavioral treatment for childhood anxiety disorders: Long-term effects on anxiety and secondary disorders in young adulthood. Journal of Child Psychology and Psychiatry, 51 , 924–934.

Schweer-Collins, M. L., Parr, N. J., Saitz, R., & Tanner-Smith, E. E. (2023). Investigating for whom brief substance use interventions are most effective: An individual participant data meta-analysis. Prevention Science .

Seidman, S., Connell, A., Stormshak, E., Westling, E., Ha, T., & Shaw, D. (2022). Disrupting maternal transmission of depression: Using integrative data analysis (IDA) to examine indirect effects of the family check-up (FCU) across three randomized trials. Prevention Science .

Siddique, J., de Chavez, P. J., Howe, G., Cruden, G., & Brown, C. H. (2018). Limitations in using multiple imputation to harmonize individual participant data for meta-analysis. Prevention Science, 19 , 95–108.

Stewart, L. A., & Parmar, M. K. (1993). Meta-analysis of the literature or of individual patient data: Is there a difference? The Lancet, 341 , 418–422.

Article CAS Google Scholar

Stuart, E. A., Cole, S. R., Bradshaw, C. P., & Leaf, P. J. (2011). The use of propensity scores to assess the generalizability of results from randomized trials. Journal of the Royal Statistical Society Series a: Statistics in Society, 174 , 369–386.

Tanner-Smith, E., Darlington, T., Gelberg, L., & Saitz, R. (2018). Brief substance use counseling interventions to reduce consumption and consequences among patients in general healthcare settings: Understanding variability in effects. https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42018086832

Tanner-Smith, E. E., Grant, S., & Mayo-Wilson, E. (2022). Modern meta-analytic methods in prevention science: Introduction to the special issue. Prevention Science, 23 , 341–345.

Tiberio, S. S., Pears, K. C., Buchanan, R., Chamberlain, P., Leve, L. D., Price, J. M., & Hussong, A. M. (2023). An integrative data analysis of main and moderated crossover effects of parent-mediated interventions on depression and anxiety symptoms in youth in foster care. Prevention Science .

Van Buuren, S., & Groothuis-Oudshoorn, K. (2011). mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45 , 1–67.

Vasilenko, S. A., Odejimi, O. A., Glassman, J. R., Potter, S. C., Drake, P. M., Coyle, K. K., & Clark, L. F. (2022). Who benefits from school-based teen pregnancy prevention programs? Examining multidimensional moderators of program effectiveness across four studies. Prevention Science .

Vidot, D. C., Huang, S., Poma, S., Estrada, Y., Lee, T. K., & Prado, G. (2016). Familias Unidas’ crossover effects on suicidal behaviors among Hispanic adolescents: Results from an effectiveness trial. Suicide and Life-Threatening Behavior, 46 , S8–S14.

Wall, M. M., Park, J. Y., & Moustaki, I. (2015). IRT modeling in the presence of zero-inflation with application to psychiatric disorder severity. Applied Psychological Measurement, 39 , 583–597.

Zhao, X., Coxe, S., Sibley, M. H., Zulauf-McCurdy, C., & Pettit, J. W. (2022). Harmonizing depression measures across studies: A tutorial for data harmonization. Prevention Science .

Zurovaca, J., Cook, T. D., Deke, J., Finucane, M. M., Chaplin, D., Coopersmith, J. S., & Forrow, L. V. (2021). Absolute and relative bias in eight common observational study designs: Evidence from a meta-analysis. arXiv preprint arXiv:2111.06941

Download references

Acknowledgements

This paper benefitted greatly from input and conversations with Drs. Lissette M. Saavedra and Stephen G. West.

Support for the writing of this paper comes in part from grants from the National Institute of Mental Health (3R01MH124438-03S1 and 3R01MH124438), and the Institute of Education Sciences (R305A220244).

Author information

Authors and affiliations.

Community Health Research Division, RTI International, Research Triangle Park, NC, USA

Antonio A. Morgan-López

School of Education and Human Development, University of Virginia, Charlottesville, VA, USA

Catherine P. Bradshaw

Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA

Catherine P. Bradshaw & Rashelle J. Musci

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonio A. Morgan-López .

Ethics declarations

Ethics approval.

This article does not include human subjects.

Consent to Participate

Conflict of interest.

Co-author Catherine Bradshaw is the editor of the journal Prevention Science , and both Antonio A. Morgan-López and Rashelle Musci are associate editors of Prevention Science ; however, another associate editor not involved with this paper managed the peer-review process. The authors have no other conflicts of interests or competing interests to report.

The views expressed are those of the authors and do not reflect those of either institute.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Morgan-López, A.A., Bradshaw, C.P. & Musci, R.J. Introduction to the Special Issue on Innovations and Applications of Integrative Data Analysis (IDA) and Related Data Harmonization Procedures in Prevention Science. Prev Sci 24 , 1425–1434 (2023). https://doi.org/10.1007/s11121-023-01600-7

Download citation

Accepted : 10 October 2023

Published : 09 November 2023

Issue Date : November 2023

DOI : https://doi.org/10.1007/s11121-023-01600-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Integrative data analysis
Meta-analysis
Evidence synthesis
Find a journal
Publish with us
Track your research

Trajectories and predictors of response in youth anxiety CBT: Integrative data analysis

Affiliations.

1 Department of Psychology.
2 Department of Clinical Psychology.
3 Department of Educational Statistics and Evaluation.
4 Department of Child and Adolescent Studies.
5 Research Institute Child Development and Education.
6 Child Study Center.
7 Department of Education and Psychiatry.
8 Department of Psychiatry and Human Behavior.
9 Division of Learning, Development and Diversity.
10 RTI Institute.
11 Department of Statistics and Biostatistics.
PMID: 30570308
DOI: 10.1037/ccp0000367

Objective: Integrative data analysis was used to combine existing data from nine trials of cognitive-behavioral therapy (CBT) for anxious youth ( N = 832) and identify trajectories of symptom change and predictors of trajectories.

Method: Youth- and parent-reported anxiety symptoms were combined using item-response theory models. Growth mixture modeling assessed for trajectories of treatment response across pre-, mid-, and posttreatment and 1-year follow-up. Pretreatment client demographic and clinical traits and treatment modality (individual- and family-based CBT) were examined as predictors of trajectory classes.

Results: Growth mixture modeling supported three trajectory classes based on parent-reported symptoms: steady responders, rapid responders, and delayed improvement. A 4-class model was supported for youth-reported symptoms: steady responders, rapid responders, delayed improvement, and low-symptom responders. Delayed improvement classes were predicted by higher number of diagnoses (parent and youth report). Receiving family CBT predicted membership in the delayed improvement class compared to all other classes and membership in the steady responder class compared with rapid responders (youth report). Rapid responders were predicted by older age (parent report) and higher number of diagnoses (parent report). Low-symptom responders were more likely to be male (youth report).

Conclusions: Integrative data analysis identified distinct patterns of symptom change. Diagnostic complexity, age, gender, and treatment modality differentiated response classes. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Anxiety Disorders / psychology
Anxiety Disorders / therapy*
Cognitive Behavioral Therapy*
Treatment Outcome

Grants and funding

National Institute of Mental Health
Netherlands Organisation for Health Research and Development
National Science Foundation

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
HHS Author Manuscripts

Integrative data analysis of self-efficacy in four clinical trials for alcohol use disorder

Eric s. kruger.

1 University of New Mexico, Albuquerque NM, USA

Kelsey N. Serier

Rory a. pfund, james r. mckay.

2 University of Pennsylvania, Philadelphia, PA, USA

3 Philadelphia VA Medical Center, Philadelphia, PA, USA

Katie Witkiewitz

Associated data, background:.

Self-efficacy has been proposed as a key predictor of alcohol treatment outcomes and a potential mechanism of success in achieving abstinence or drinking reductions following alcohol treatment. Integrative data analysis, where data from multiple studies are combined for analyses, can be used to synthesize analyses across multiple alcohol treatment trials by creating a commensurate measure and controlling for differential item functioning (DIF) to determine if alcohol treatments improve self-efficacy.

The current study used moderated nonlinear factor analysis (MNLFA) to examine the effect of treatment on self-efficacy across four different treatment studies ( N =3,720; 72.5% male, 68.4% non-Hispanic white). Self-efficacy was measured using the Alcohol Abstinence Self-Efficacy Scale (AASE) in the COMBINE Study ( n= 1,383) and Project MATCH ( n= 1,726), and the Drug Taking Confidence Questionnaire (DTCQ) in two studies of Telephone Continuing Care (TEL Study 1: n =303; TEL Study 2: n =212). DIF was examined across time, study, treatment condition, marital status, age, and sex.

We identified 12 items from the AASE and DTCQ to create a commensurate measure of self-efficacy using MNLFA. Results indicated all active treatments, including cognitive-behavioral treatment, combined behavioral intervention, medication management, motivation enhancement treatment, telephone continuing care, twelve-step facilitation, and relapse prevention, were associated with significant increases in self-efficacy from baseline to posttreatment and these changes were maintained for up to a year. Importantly, treatment as usual in community settings, which consisted of weekly group therapy that included addiction counseling and twelve-step recovery support, was not associated with significant increases in self-efficacy.

Conclusions:

Alcohol self-efficacy increases following treatment and numerous evidence-based treatments are associated with significant increases in self-efficacy, which are maintained over time. Community treatment that focuses solely on addiction counseling and twelve-step support may not promote increases in self-efficacy.

Introduction

Self-efficacy is defined as an individual’s belief in their ability to organize and execute behaviors to achieve a desired outcome in prospective situations ( Bandura, 1977 ). In the context of alcohol treatment, an individual with high self-efficacy is confident in their ability to abstain or reduce heavy drinking in potential alcohol use situations, such as social situations where alcohol is offered or in situations when an individual has previously used alcohol to cope with negative affect. On the other hand, an individual with low self-efficacy is unsure of his or her ability to abstain or reduce heavy drinking in these situations ( Marlatt and Gordon, 1985 ). Previous research consistently finds higher self-efficacy is associated with better drinking outcomes following alcohol treatment ( Kavanagh et al., 1996 ; Maisto et al., 2000 ; Sitharthan and Kavanagh, 1991 ), such that individuals with higher self-efficacy maintain abstinence longer ( Vielva and Iraurgi, 2001 ), report fewer days of alcohol use ( Brown et al., 2002 ), and show lower risk of returning to alcohol use after a period of abstinence ( Allsop et al., 2000 ) than individuals with lower self-efficacy. Given these findings, self-efficacy may represent an important treatment mechanism leading to better alcohol treatment outcomes. The question remains whether specific treatments are more or less likely to increase self-efficacy.

Previous research has examined the differential effects of theoretically distinct behavioral treatments for alcohol use on self-efficacy. For example, much of the research examines differences in self-efficacy between cognitive-behavioral treatment and twelve-step oriented treatment. Theoretically, cognitive-behavioral treatment involves learning specific coping behaviors that help individuals navigate situations where the risk for alcohol use is high (Kadden et al., 1994), while twelve-step oriented treatment assumes that sustained abstinence is the only effective remedy to problematic alcohol use ( Nowinski et al., 1995 ). These theoretical differences steer expectations for cognitive-behavioral therapy to exert greater effects on self-efficacy than twelve-step oriented treatment. However, two studies found no significant differences in self-efficacy between cognitive-behavioral treatment and twelve-step treatment ( Glasner-Edwards et al., 2007 ; Johnson et al., 2006 ). Such findings are unexpected and would suggest that distinct behavioral treatments do not differ in their effects on self-efficacy.

A smaller but parallel line of research examines the differences in self-efficacy between behavioral treatments and medication. Theoretical discussions warn that medication may undermine an individuals’ ability to build self-efficacy during treatment because an individual may attribute changes in alcohol use to medication rather than changes in their own behavior ( Moncrieff and Drummond, 1997 ). Despite this warning, empirical research indicates the opposite may be true – medication produces greater changes in self-efficacy than behavioral treatments ( Hartzler et al., 2011 ).

Collectively, these unexpected findings may be due to participants’ differential response to alcohol treatment. Recent discussions in alcohol treatment precision medicine initiatives suggest specific subgroups of participants may benefit more from specific treatments than other subgroups of participants (e.g., Litten et al., 2012 ). Related to self-efficacy, there is evidence that distinct treatments may exert differential effects on self-efficacy based on simple demographic characteristics. For example, in a secondary analysis of Project MATCH ( Project MATCH Research Group, 1997 ), Maisto and colleagues (2015) found that being male predicted greater increases in self-efficacy during treatment. However, no studies have comprehensively examined the effect of distinct alcohol treatments on self-efficacy, while adjusting for potential subgroup differences based on participant characteristics.

The lack of investigations into potential subgroups differences in measuring the construct of alcohol self-efficacy is problematic given that previous research has found differences in measures of general self-efficacy (Bonsaken et al., 2013; Peter et al., 2014 ) and domain-specific self-efficacy ( Makransky et al., 2015 ; Riazi et al., 2014 ). Previous studies of self-efficacy in other domains have identified differences in item functioning by sex, age, time, and marital status, and that differential item functioning may also exist related to treatment received (e.g., receiving medication management vs CBT).

To examine the effect of treatment on self-efficacy across different treatment studies it is imperative to have similar measures of self-efficacy and outcomes in each study. Integrative data analysis (IDA) is an innovative framework for creating a cumulative science by accumulating knowledge from multiple studies via coordinated measurement or pooling samples and identifying and testing commensurate measures across studies ( Curran and Hussong, 2009 ; Hofer and Piccinin, 2009 ; Hussong et al., 2013 ; Marcoulides and Grimm, 2017 ; McArdle et al., 2009 ; Shrout, 2009 ; Wilcox and Wang, 2021 ). Similar to meta-analysis, IDA extends findings beyond isolated clinical trials. IDA permits clinical research to generalize findings beyond a specific sample or study to maximize sample sizes, maximize sample heterogeneity, and examine differences across different studies and studied populations ( Curran et al., 2014 ; Hofer and Piccinin, 2009 ; Wilcox and Wang, 2021 ). IDA also allows for larger sample sizes and permits testing hypotheses in specific subgroups (e.g., by sex and age groups) and to test effects of treatments that may differ by different subgroups (e.g., does treatment “X” have a greater effect on self-efficacy than treatment Y?) ( Hussong et al., 2013 ). IDA can also be used to examine whether different study characteristics or participant characteristics within studies impact the measurement of constructs relevant to treatment outcomes ( Hussong et al., 2013 ; Shrout, 2009 ; Wilcox and Wang, 2021 ).

IDA has increasingly been used in alcohol research to combine data from multiple studies, including investigations of alcohol use and sexual risk behavior ( Walsh et al., 2017 ), changes in adolescent alcohol use over time ( Silins et al., 2018 ), and brief alcohol interventions for college students ( Mun et al., 2015 ). Given the lack of comprehensive understanding on alcohol treatments’ effects on self-efficacy, the present study sought to use IDA to pool data across samples and examine the effect of various alcohol treatments on self-efficacy across studies and subgroups of participants ( Curran and Hussong, 2009 ).

Participants

Data used for this study were collected from four alcohol clinical trials ( N = 3,720). These studies included the Project MATCH aftercare ( n = 774) and outpatient samples ( n = 952; Project MATCH research Group, 1997 ), the COMBINE study ( n = 1383; COMBINE Study Group, 2003 ), and two studies of Telephone Continuing Care (TEL; TEL 1, n = 359; McKay et al., 2005 ; TEL2, n = 252; McKay et al., 2011 ). All four studies were randomized clinical trials. Participants in the total combined sample were, on average, 42.21 years old ( SD = 10.51), predominantly male (72.5%), non-Hispanic White (68.4%) or African American (21.2%), and not married or cohabitating (66.2%).

The COMBINE study ( COMBINE Study Group, 2003 ) randomized participants with alcohol dependence ( n = 1383) from eleven research sites across the United States into nine treatment groups consisting of a combination of medical management (MM) or combined behavioral intervention (CBI) and medications (acamprosate, naltrexone, or placebo versions of each drug). Inclusion was determined by meeting the following criteria (1) alcohol dependence based on the Diagnostic and Statistical Manual of Mental Health Disorders, Fourth Edition (DSM-IV; American Psychiatric Association, 1994 ), (2) 4 to 21 days of abstinence; and (3) greater than 14 drinks (women) and 21 drinks (men) with at least 2 heavy drinking days--greater than or equal to 4 drinks (women) and 5 drinks (men) during a consecutive 30-day period within 90 days prior to baseline evaluation. Participants in COMBINE received treatment for 16 weeks (9 sessions of MM and up to 20 sessions of CBI). For evaluating predictor effects of study, the COMBINE study was used as the reference group.

Project MATCH ( Project MATCH Research Group, 1997 ) randomized outpatients ( n = 952) and aftercare patients (recruited from inpatient treatment; n = 774) with alcohol dependence or alcohol abuse from nine research sites across the United States into three treatment groups: cognitive behavioral therapy (CBT), motivation enhancement therapy (MET), or twelve-step facilitation (TSF). Eligibility was determined by meeting a diagnosis of alcohol dependence or abuse based on the Diagnostic and Statistical Manual of Mental Health Disorders, Revised Third Edition (DSM-III-R; American Psychiatric Association, 1987 ). Participants received up to 12 sessions of CBT and TSF and up to 4 sessions of MET over 12 weeks.

The TEL 1 study ( McKay et al., 2005 ) recruited participants with cocaine and/or alcohol dependence ( n = 359) after completing four weeks of intensive outpatient treatment in two outpatient treatment programs and randomized them to three continuing care treatments delivered over 12 weeks: weekly telephone based monitoring and brief counseling combined with weekly group support (TEL), weekly individual and group relapse prevention (RP), or treatment as usual (TAU, weekly group therapy that included addiction counseling and twelve-step recovery support). Participants were included if they had a previous diagnosis of cocaine or alcohol dependence (based on the DSM-IV criteria) at entrance to treatment and completed an intensive outpatient program (IOP) immediately prior.

The TEL 2 study ( McKay et al., 2011 ) recruited participants with alcohol dependence ( n= 252) after completing three weeks of intensive outpatient treatment in two outpatient treatment programs and were randomized to three conditions: telephone-based monitoring for up to 18 months (TEL), up to 18 months of telephone-based monitoring and individual counseling using motivational interviewing and CBT sessions (TMC), or TAU for 3-4 months (once weekly group therapy that included addiction counseling and twelve-step recovery support). Participants were included if they had a previous diagnosis of alcohol dependence (based on the DSM-IV criteria) at entrance to treatment and completed an intensive outpatient program (IOP) immediately prior.

Alcohol Abstinence Self-Efficacy Scale (AASE; DiClemente et al., 1994 ).

The AASE is a self-report measure designed to assess the construct of self-efficacy as it applies to abstinence from alcohol. The measure asks participants to rate both perceived confidence and ability to abstain across 20 different situations on a 5-point Likert-type scale (1 = not at all confident, 5 = extremely confident). Participants in Project MATCH and COMBINE completed the AASE. In Project MATCH, the AASE was administered at baseline, post-treatment (3 months post-baseline), 6-month follow-up (9 months post-baseline), and 12-month follow-up (15 months post-baseline). In the COMBINE study, the AASE was administered at baseline, post-treatment (16 weeks post-baseline), and the 6-month follow-up (26 weeks, approximately 6.5 months, post-baseline). Internal consistency reliability of the AASE at baseline was α = 0.95 in MATCH and α = 0.93 in COMBINE. Reliability exceeded α = 0.97 at follow-up time points in MATCH and COMBINE.

Drug Taking Confidence Questionnaire (DTCQ; Annis et al., 1997 ).

The DTCQ is a 50-item self-report measure that ask participants to rate their perceived ability to resist the urge to drink heavily and cope in different situations. Participants rated their confidence on a 6-point Likert-type scale (0 = not at all confident and 100 = very confident). Participants in TEL1 and TEL2 completed the DTCQ at baseline, 3-months post-baseline, 6-months post-baseline, and 12-months post-baseline. The internal consistency reliability of the 50 item DTCQ exceeded α = 0.97 in TEL1 and TEL2 at all time points.

Data Analysis

Moderated nonlinear factor analysis (MNLFA) is one data analysis approach for pooling data in IDA ( Curran and Hussong, 2009 ), in which indicators from different instruments across different studies can be combined. Further, MNLFA allows for testing differential item functioning (DIF) across potential covariates on the latent construct and individual indicators across multiple studies ( Curran et al., 2014 , 2018 ). All models described below were estimated with Mplus Version 8 ( Muthén and Muthén, 2017 ) using maximum likelihood estimation.

Identification and Harmonization of Similar Items.

To perform the MNLFA participants were screened for missing data on demographic variables, those with missing data were excluded. The final sample size used in the MNLFA was 3,581. As outlined in the Supplementary Code and MNLFA input and output statements provided online at https://osf.io/wa3tn , the first step in MNLFA was to identify similar items across the two self-efficacy measures. Similar items were identified and independently rated by two domain experts as items belonging to different measures that assessed similar content. We identified 12 self-efficacy items, see Table 1 for a description of items from the AASE and DTCQ. Since the AASE and DTCQ are scored on different Likert-type scales the DTCQ (scored from 0 to 100) was converted to the AASE scale (scored from 1 to 5): 0 (DTCQ) = 1 (AASE), 20 = 2, 40 = 2, 60 = 3, 80 = 4, and 100 = 5. The harmonization of Likert-type scales was based on distributions of the items in both samples. We also harmonized the time points across studies with baseline occurring before treatment in all studies and end of treatment occurring at 3 months post-baseline in the MATCH and TEL studies, and 4 months post-baseline in COMBINE.

Description of Items that Were Rated to be Similar

Note . AASE = Alcohol Abstinence Self-Efficacy Scale; DTCQ = Drug Taking Confidence Questionnaire.

Next, the data from the four RCTs were combined, which created a total of 20 treatment arms, therefore individual study arms were further combined based on similarity of interventional components into six treatment variables: Medical Management (MM; COMBINE MM arm), cognitive behavioral therapy (CBT; MATCH CBT and COMBINE CBI arms), motivational enhancement therapy (MET; MATCH MET arm), twelve-step facilitation (TSF; MATCH TSF), relapse prevention (RP; TEL1 RP arm), telephone based monitoring and counseling (TEL; TEL1 TEL arm; and TEL2 TEL and TMC arms), and treatment as usual (TAU; TEL1 and TEL2 TAU arms). All treatment variables were dummy coded with TAU as the reference.

Self-Efficacy Factor Structure.

Once the similar items and treatment variables were coded, the construct validity of the AASE, DTCQ, and self-efficacy items was evaluated by exploratory and confirmatory factor analysis (EFA and CFA, respectively) on a calibration sample. The calibration data set was created by randomly selecting one time-point for each participant. We elected to evaluate a parsimonious single factor model despite other studies using models with other factor configurations ( DiClemente et al., 1994 ; Glozah et al., 2015 , Kim et al., 2015 ; Ramo et al., 2009 ; Reilly et al., 1995 ; Sklar et al., 1997 ) to be consistent with previous studies of alcohol self-efficacy ( Maisto et al., 2015 ; Mensinger et al., 2007 ; Witkiewitz et al., 2012 ). For each measure model fit was examined according a priori cutoff criteria (Comparative Fit Index (CFI) > 0.90; Tucker-Lewis Index (TLI) > 0.90; Root Mean Square Error of Approximation (RMSEA) < 0.08; and Standardized Root Mean Square Residual (SRMR) < 0.08; Hu and Bentler, 1999 ).

Moderated Nonlinear Factor Analysis.

Using the calibration data set, we first evaluated the effect of the covariates on the factor means and variances and then estimated 12 models, one model for each of the 12 similar items to test the effects of covariates on item intercepts and loadings. Specifically, to examine differential item functioning (DIF), we included covariates of time, study, treatment condition, marital status, age, and sex. These covariates were selected given prior studies of self-efficacy in other domains have identified differences in item functioning by sex, age, time, and marital status (Bonsaken et al., 2013; Makransky et al., 2015 ; Peter et al., 2014 ; Riazi et al., 2014 ) and because we were specifically interested in DIF by treatment received. Importantly, race was confounded with study, with nearly all African American participants recruited for the TEL studies, thus we did not examine race/ethnicity in the current study. Next all significant covariate effects ( p < 0.10) from examining (1) means and variances and (2) intercepts and loadings were included in a single model. From this single model an iterative model trimming process was conducted which consisted of removing non-significant covariates if they had a p -value > 0.05 and then rerunning the model. If a dummy coded covariate with multiple levels was significant (e.g., treatment) then all treatment dummy variables were included. This process continued until a final model was reached with only significant covariates remaining in the model.

Once the final model was estimated we derived expectation a posteriori (EAP) self-efficacy scores. The EAP is a predicted latent score of self-efficacy for each observation that are informed by covariate effects ( Curran et al., 2018 ). Sensitivity of EAP scores were evaluated by correlating EAP self-efficacy values with values for the unadjusted raw self-efficacy commensurate items (derived from the AASE and DTCQ).

Self-Efficacy Latent Scores and Change in Self-Efficacy over Time.

After establishing a commensurate measure of alcohol self-efficacy, two separate analyses were conducted to examine (1) time and treatment related self-efficacy changes and (2) differences in self-efficacy across studies and treatments using the EAP scores. First, EAP self-efficacy means and variances were analyzed for linear trends over time and differences in self-efficacy between studies (COMBINE as reference) and across treatments (TAU as reference) were also examined. Mixed-effect models were used to analyze change in self-efficacy across all time points (collapsed across treatments) and between the baseline evaluation and end of treatment (3 months) for each treatment using EAP scores. Results of mixed effects models are reported as standardized mean difference ( M Δ /SD ). Changes in unadjusted raw scores on the AASE, DTCQ, and raw items across time and within treatments were also analyzed and results are reported in supplemental tables .

Identification of Similar Items

Twelve items from the DTCQ and AASE demonstrated significant construct overlap and were identified as similar items (see Table 1 ). Items between the AASE and DTCQ were matched based on theoretical overlap between items.

Self-Efficacy Factor Structure

Unidimensionality of the items was evaluated with exploratory factor analysis (EFA) using robust weighted least squares estimation with goemin rotation of all available data to examine dimensionality and item loadings and then subsequently with CFA using robust weighted least squares estimation. There was only one eigenvalue greater than 1.0 (first eigenvalue = 8.689, second eigenvalue = 0.718). Model fit of the one factor EFA model was not excellent, based on a significant χ 2 [ χ 2 = 5520.62 (54), p< 0.001] and RMSEA = 0.19 (90% CI: 0.183, 0.192), however CFI = 0.953 and SRMR = 0.054 were acceptable and all item loadings exceeded 0.78. The one-factor CFA was also not excellent due to a significant χ 2 [ χ 2 = 5520.63 (54), p< 0.001] and a RMSEA = 0.19 (90% CI: 0.183, 0.192), however the CFI = 0.953 and SRMR = 0.042 were acceptable.

Moderated Nonlinear Factor Analysis

A unidimensional MNLFA was fit to the 12-item, 1-factor model of alcohol self-efficacy items, and included moderation by demographic variables (age, marital status, and sex), time, study membership, and treatment condition to test for the effects of these covariates on factor mean, factor variance, item intercepts, and item loadings. 1

There were three items with significant DIF on item intercepts and/or loadings: “feeling depressed,” “urge to try just one,” and “social pressure.” Item intercepts reflect the level of self-efficacy associated with endorsement of an item (see Table 2 ). Additionally, there was DIF on the loadings for study for the items “feeling depressed” (MATCH and TEL2), “urge to try just one drink” (MATCH and sex), and “social pressure” (MATCH and TEL1). The significant loadings indicate an interaction between study characteristics, measurement characteristics, or participant characteristics with levels of latent self-efficacy and thus probability of endorsement is non-uniform based on study characteristics and sex. Expectation a posteriori (EAP) scores for the alcohol self-efficacy latent factor were then created for each observation.

Results from final MNLFA Alcohol Self-Efficacy Model with Significant Covariate Effects on Item Intercepts and Item Loadings

Note . COMBINE is the reference group for dummy coded study contrasts.

Self-Efficacy Latent Scores and Change in Self-Efficacy over Time

Only time, study, and treatment condition were significantly associated with the latent self-efficacy factor mean and variance. Latent self-efficacy mean and variance both increased significantly over time, mean: B ( SE ) = 0.319(0.035), p < 0.001; variance B ( SE ) = 0.346 (0.033), p < 0.001. Individuals in the MATCH sample, had higher average self-efficacy than COMBINE, B ( SE ) = 0.181(0.087), p = 0.037), and greater variance in self-efficacy, B ( SE ) = 0.340(0.105), p = 0.001. Mean EAP self-efficacy scores were significantly higher in TEL, RP, and TAU treatment conditions, which is consistent with recruitment of individuals after already completing intensive outpatient treatment in the TEL1 and TEL2 studies. Specifically, compared to TAU, individuals in MM, CBT, MET, and TSF had lower average self-efficacy (MM: B ( SE ) = −0.936(0.265), p = 0.001; CBT/CBI: B ( SE ) = −0.898(0.262), p = 0.001; MET: B ( SE ) = −0.930(0.279), p = 0.001; TSF: B ( SE ) = −0.963(0.280), p = 0.001, and individuals in any TEL and RP conditions had significantly higher average self-efficacy compared to TAU, B ( SE ) = 0.777(0.216), p < 0.001, and all other treatments tested in COMBINE and MATCH. Compared to TAU, the variance of self-efficacy was significantly smaller for the MM, CBT/CBI, and MET conditions, MM: B ( SE ) = −0.369 (0.170), p = 0.030; CBT/CBI: B ( SE ) = −0.390(0.165), p = 0.018; MET: B (SE) = −0.391(0.188), p = 0.038, indicating less variability in the self-efficacy construct for those who received these treatments.

EAP Self-Efficacy Scores over Time.

As demonstrated in Table 3 , the EAP self-efficacy scores demonstrated a significant increase from baseline to all other time points and did not significantly decrease across time points (all p s > 0.21). When comparing the effect of treatments from baseline to end of treatment (3 months), all treatments except TAU were associated with significant increases in self-efficacy (see Figure 1 , Table 4 ).

An external file that holds a picture, illustration, etc.
Object name is nihms-1740127-f0001.jpg

Note: Error bars denote 95% confidence intervals; CBT = Cognitive Behavioral Therapy, MM = Medication Management, TAU = Treatment as Usual, TSF = Twelve Step Facilitation, MET = Motivation Enhancement Therapy, RP = Relapse Prevention, and TEL = Telehealth, & ALL = grand mean across all individuals and conditions.

Alcohol Self-Efficacy EAP Scores at Each Time Point

Note . EAP = Expectation a posteriori scores from moderated nonlinear factor analysis; bracketed diagonal = standardized value at each time point; below diagonal is the standardized difference between time-points; above diagonal is the p -value.

Alcohol Self-Efficacy EAP Scores From Start to End Of Treatment Using MNLFA to Pool Studies and Treatment Conditions

Note: EAP = expectation a posteriori scores from moderated nonlinear factor analysis ; CBT = cognitive behavioral therapy; MET = motivational enhancement therapy; MM = medical management; RP relapse prevention; TAU = treatment as usual; TEL = telehealth; TSF = twelve-step facilitation.

Observed Self-Efficacy Scores over Time.

Analyses of scores on the observed (i.e., raw) self-efficacy items (i.e., summing the 12 items into a total self-efficacy score, unadjusted for covariate effects) produced somewhat different results than what was found using the EAP scores. First, the unadjusted observed self-efficacy scores indicated an initial significant increase in self-efficacy followed by a steady decline in observed self-efficacy scores between three and six (p <0.001) and 12-months (p < 0.001), but no decline was observed in the EAP scores. Examining change in the unadjusted observed self-efficacy scores over time within each study by individual treatment conditions produced results largely consistent with the EAP self-efficacy scores with two exceptions: individuals assigned to the RP and TEL treatment conditions in the TEL1 study did not demonstrate a significant increase in observed self-efficacy scores (see Supplement Tables 3 - 6 ).

In this study, we performed IDA by pooling data from four clinical trials to assess the effect of study treatments on alcohol self-efficacy. The active treatments, including cognitive-behavioral treatment, combined behavioral intervention, medication management, motivation enhancement treatment, telephone continuing care, twelve-step facilitation, and relapse prevention, were associated with a significant increase in self-efficacy from baseline to post-treatment and these changes were maintained up to one year. Importantly, treatment as usual in community settings, which consisted of weekly group therapy that included addiction counseling and twelve-step recovery support, was not associated with increased self-efficacy, above and beyond the levels of self-efficacy already achieved in intensive outpatient treatment.

Self-efficacy has been widely studied as one potential mechanism of change in alcohol treatment and has been shown to be a robust predictor of alcohol treatment outcomes ( Allsop et al., 2000 ; Brown et al., 2002 ; Kavanagh et al., 1996 ; Maisto et al., 2000 ; Sitharthan and Kavanagh, 1991 ; Vielva and Iraurgi, 2001 ). Adamson and colleagues (2009) conducted a review of predictors of alcohol treatment outcome and found self-efficacy was the most consistent predictor. Additionally, similar to other studies examining treatment-related changes in self-efficacy over time ( McKellar et al., 2008 ), increases in self-efficacy were maintained at follow-up. The current study demonstrated that across studies and treatment conditions, alcohol self-efficacy increased.

The effects of CBT, MET, RP, MM, TEL, and TSF on self-efficacy were robust with significant increases in self-efficacy from baseline to 3-months following treatment. These findings suggest that each of the specific treatments resulted in increases in self-efficacy from baseline to post-treatment. The fact that diverse treatments are effective in mobilizing self-efficacy could indicate that self-efficacy is a common target across treatments, or could indicate that changes in self-efficacy correspond to or are a byproduct of changes in behavior (i.e., drinking reductions achieved during treatment; Kadden and Litt, 2011 ). The fact that medication management was also effective in increasing self-efficacy suggests the latter explanation may be more likely. An experimental study that attempted to causally manipulate self-efficacy among individuals who were motivated to quit smoking found those assigned to a high self-efficacy condition were more likely to quit smoking, but changes in self-reported self-efficacy did not mediate the effects of the intervention on smoking outcomes ( Shadel et al., 2017 ). Interestingly, individual outcome expectancies did significantly mediate the effects of the self-efficacy intervention on smoking outcomes. Future work should continue to focus on whether self-efficacy is a mechanism of change, whether it is potentially a byproduct of behavior change ( Kadden and Litt, 2011 ), and/or whether self-reported self-efficacy is independent from other cognitive factors that may explain behavior change.

Although there was a high degree of concordance between the EAP and unadjusted self-efficacy scores, there were some differences between EAP scores and unadjusted scores. Specifically, unadjusted self-efficacy scores demonstrated a significant decrease in self-efficacy from immediate post-treatment to subsequent follow-up measurements. These findings would suggest that unadjusted self-efficacy scores decrease following treatment, however the analyses with the EAP scores do not indicate the erosion of self-efficacy across time following treatment. Similarly, the effect of RP and TEL on the change in self-efficacy from baseline to post-treatment was not significant in the TEL1 study using the unadjusted self-efficacy score, but did produce a significant effect using the EAP scores. Thus, findings diverged for the effect of RP and TEL on change in self-efficacy. Prior simulation work has found that raw scores may lead to biased inferential tests due to covariate effects that are not estimated, whereas EAP scores seem to mitigate these biases ( Curran et al., 2018 ). Three of the 12 of the items used to develop the EAP self-efficacy scores demonstrated differential item functioning by gender and study. Generally, these findings suggest that certain situations are more applicable for self-efficacy for different study populations and in different treatment groups.

Limitations

The current study utilized data collected from large and highly rigorous alcohol clinical trials, which is a strength and also a limitation. Combining treatments across studies prevents us from considering the randomized effective treatment. In other words, even though the individuals in each trial were randomized, the combination of individuals across treatments were not randomized. The current study was more focused on within-person changes in self-efficacy following treatment, however we cannot say that one treatment was better in a way that capitalizes on the randomized clinical trial design methodology used within each study.

All four clinical trials had exclusion criteria that limit generalizability of the current findings. The samples were also predominantly male and had a large representation of white participants, which limited our ability to assess for subgroup differences by race and ethnicity. It is also important to note that study and race were confounded, such that participants in the TEL studies were predominantly Black/African American. This study level difference was not explicitly captured in the analyses, which is a significant limitation. Similarly, the AASE used in the MATCH and COMBINE studies assessed abstinence self-efficacy, whereas the DTCQ used in the TEL studies assessed self-efficacy to resist urges to drink heavily. It is unclear if higher scores in the TEL studies may reflect greater self-efficacy to resist heavy drinking or other study level differences. The poor model fit, based on RMSEA and significant chi-square, is also a limitation. We proceeded with testing the one factor model given prior work has used a total score (assuming a one factor model) and given complexities in testing MNLFA models with multiple factors, however future work could consider examining a multidimensional structure based on the subscales of the AASE and DTCQ to examine whether subscales are invariant across participant characteristics and time. Additionally, future work may consider testing the models using a multilevel IDA framework ( Shrout, 2009 ; Wilcox and Wang, 2021 ).

Comparing the results of harmonized EAP scores with raw scores that did not adjust for covariates and time, leads to different conclusions in some cases. This is most evident for the effect of time, where the unadjusted raw self-efficacy scores demonstrate a significant decrease from immediate post-treatment to subsequent follow-up measurements. These findings would suggest that unadjusted self-efficacy scores decrease following treatment, however the analyses with the EAP scores across time following treatment show a more optimistic pattern of stability of self-efficacy following treatment. These findings should be considered preliminary and require replication, given the small number of studies included and the lack of multiple common items across studies.

Conclusions

The current study provides novel information about the measurement of self-efficacy in alcohol clinical trials. Future work should replicate these findings, and include other demographic factors, particularly race and ethnicity. Research should continue to examine the measurement of constructs that are central to alcohol treatment outcomes research, such as affect, craving, executive function, and social support. Examining measures across patient populations and in different settings and contexts has the potential to improve alcohol use disorder treatment and precision medicine approaches to matching patient populations to specific alcohol use disorder treatment protocols ( Kranzler and McKay, 2012 ).

Supplementary Material

This research was supported by grants funded by the National Institute on Alcohol Abuse and Alcoholism [T32AA018108 (McCrady, PI), R01 AA022328 and R01 AA025539 (Witkiewitz, PI)]. The content is solely the responsibility of the authors and does not necessarily reflect the views of NIH.

Declaration of interests: None.

1 Model fit of the MNLFA model was also estimated using weighted least squares estimation to obtain fit indices that are comparable to the CFA models. Results indicated acceptable model fit based on CFI = 0.98 and TLI = 0.98, and a better fitting model than the CFA results based on RMSEA = .095 (90% CI 0.094, 0.097).

Adamson SJ, Sellman JD, Frampton CM (2009) Patient predictors of alcohol treatment outcome: a systematic review . J Subst Abuse Treat 36 :75–86. [ PubMed ] [ Google Scholar ]
Allsop S, Saunders B, Phillips M (2000) The process of relapse in severely dependent male problem drinkers . Addiction 95 :95–106. [ PubMed ] [ Google Scholar ]
American Psychiatric Association (1987) Diagnostic and statistical manual of mental disorders . 3 rd ed. American Psychiatric Association, Washington, DC. [ Google Scholar ]
American Psychiatric Association (1994) Diagnostic and statistical manual of mental disorders . 4 th ed. American Psychiatric Association, Washington, DC. [ Google Scholar ]
Annis HM, Sklar SM, Turner NE (1997) The Drug-Taking Confidence Questionnaire (DTCQ): User's Guide . Toronto, Canada: Addiction Research Foundation, Centre for Addiction and Mental Health. [ Google Scholar ]
Bandura A (1977) Self-efficacy: Toward a unifying theory of behavioral change . Psychol Rev 84 :191–215. [ PubMed ] [ Google Scholar ]
Bonsaksen T, Kottorp A, Gay C, Fagermoen MS, Lerdal A (2013) Rasch analysis of the General Self-Efficacy Scale in a sample of persons with morbid obesity . Health Qual Life Outcomes 11 :202. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Brown TG, Seraganian P, Tremblay J, Annis H (2002) Process and outcome changes with relapse prevention versus 12-Step aftercare programs for substance abusers . Addiction 97 :677–689. [ PubMed ] [ Google Scholar ]
COMBINE Study Research Group (2003) Testing combined pharmacotherapies and behavioral interventions in alcohol dependence: rationale and methods . Alcohol Clin Exp Res 27 :1107–1122. [ PubMed ] [ Google Scholar ]
Curran PJ, Cole VT, Bauer DJ, Rothenberg WA, Hussong AM (2018) Recovering predictor-criterion relations using covariate-informed factor score estimates . Struct Equ Modeling 25 :860–875. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Curran PJ, Hussong AM (2009) Integrative data analysis: the simultaneous analysis of multiple data sets . Psychol Methods 14 :81–100. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Curran PJ, McGinley JS, Bauer DJ, Hussong AM, Burns A, Chassin L, Sher K, Zucker R (2014) A moderated nonlinear factor model for the development of commensurate measures in integrative data analysis . Multivariate Behav Res 49 :214–231. [ PMC free article ] [ PubMed ] [ Google Scholar ]
DiClemente CC, Carbonari JP, Montgomery RP, Hughes SO (1994) The alcohol abstinence self-efficacy scale . J Stud Alcohol 55 :141–148. [ PubMed ] [ Google Scholar ]
Glasner-Edwards S, Tate SR, McQuaid JR, Cummins K, Granholm E, Brown SA (2007) Mechanisms of action in integrated cognitive-behavioral treatment versus twelve-step facilitation for substance-dependent adults with comorbid major depression . J Stud Alcohol Drugs 68 :663–672. [ PubMed ] [ Google Scholar ]
Glozah FN, Adu NAT, Komesuor J (2015) Assessing alcohol abstinence self-efficacy in undergraduate students: psychometric evaluation of the alcohol abstinence self-efficacy scale . Health Qual Life Outcomes 13 :189. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Hartzler B, Witkiewitz K, Villarroel N, Donovan D (2011) Self-efficacy change as a mediator of associations between therapeutic bond and one-year outcomes in treatments for alcohol dependence . Psychol Addict Behav 25 :269–278. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Hofer SM, Piccinin AM (2009) Integrative data analysis through coordination of measurement and analysis protocol across independent longitudinal studies . Psychol Methods 14 :150–164. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Hu L, Bentler PM (1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives . Struct Equ Model 6 :1–55. [ Google Scholar ]
Hussong A, Curran PJ, Bauer DJ (2013) Integrative data analysis in clinical psychology research . Annu Rev Clin Psychol 9 :61–89. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Johnson JE, Finney JW, Moos RH (2006) End-of-treatment outcomes in cognitive-behavioral treatment and 12-step substance use treatment programs: do they differ and do they predict 1-year outcomes? J Subst Abuse Treat 31 :41–50. [ PubMed ] [ Google Scholar ]
Kadden RM, Carroll KM, Donovan DM, Cooney N, Monti P, Abrams D, Litt M, Hester R (1995) Cognitive-behavioral coping skills therapy manual . National Institute on Alcohol Abuse and Alcoholism, Rockville, MD. [ Google Scholar ]
Kadden RM, Litt MD (2011) The role of self-efficacy in the treatment of substance use disorders . Addict Behav 3612 :1120–1126. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Kavanagh DJ, Sitharthan T, Sayer GP (1996) Prediction of results from correspondence treatment for controlled drinking . Addiction 91 :1539–1545. [ PubMed ] [ Google Scholar ]
Kim SY, Ahn S, Lim TS (2015) Predicting alcohol misuse among college students in the US and South Korea . J Ethn Subst Abuse 14 :308–322. [ PubMed ] [ Google Scholar ]
Kranzler HR, McKay JR (2012) Personalized treatment of alcohol dependence . Curr Psychiatry Rep 14 :486–493. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Litten RZ, Egli M, Heilig M, Cui C, Fertig JB, Ryan ML, Falk DE, Moss H, Huebner R, Noronha A (2012) Medications development to treat alcohol dependence: a vision for the next decade . Addict Biol 17 :513–527. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Maisto SA, Connors GJ, Zywiak WH 2000. Alcohol treatment changes in coping skills, self-efficacy, and levels of alcohol use and related problems 1 year following treatment initiation . Psychol Addict Behav 14 :257–266. [ PubMed ] [ Google Scholar ]
Maisto SA, Roos CR, O'Sickey AJ, Kirouac M, Connors GJ, Tonigan JS, Witkiewitz K (2015) The indirect effect of the therapeutic alliance and alcohol abstinence self-efficacy on alcohol use and alcohol-related problems in Project MATCH . Alcohol Clin Exp Res 393 :504–513. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Makransky G, Rogers ME, Creed PA (2015) Analysis of the construct validity and measurement invariance of the career decision self-efficacy scale: a Rasch model approach . J Career Assess 23 :645–660. [ Google Scholar ]
Marcoulides KM, Grimm KJ (2017). Data integration approaches to longitudinal growth modeling . Educ Psychol Meas 77 : 971–989. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Marlatt GA, Gordon JR. (1985) Relapse prevention: maintenance strategies in addictive behavior change . Guilford Press, New York. [ Google Scholar ]
McCardle JJ, Grimm K, Hamagami F, Bowles R, Meredith W (2009). Modeling life-span growth curves of cognition using longitudinal data with multiple samples and changing scales of measurement . Psychol Methods 14 : 126–149. [ PMC free article ] [ PubMed ] [ Google Scholar ]
McKay JR, Lynch KG, Shepard DS, Pettinati HM (2005) The effectiveness of telephone-based continuing care for alcohol and cocaine dependence: 24-month outcomes . Arch Gen Psychiatry 622 :199–207. [ PubMed ] [ Google Scholar ]
McKay JR, Van Horn D, Oslin DW, Ivey M, Drapkin ML, Coviello DM, Yu Q, Lynch KG (2011) Extended telephone-based continuing care for alcohol dependence: 24-month outcomes and subgroup analyses . Addiction 106 :1760–1769. [ PMC free article ] [ PubMed ] [ Google Scholar ]
McKellar J, Ilgen M, Moos BS, Moos R (2008) Predictors of changes in alcohol-related self-efficacy over 16 years . J Subst Abuse Treat 35 :148–155. [ PubMed ] [ Google Scholar ]
Mensinger JL, Lynch KG, TenHave TR, McKay JR (2007) Mediators of telephone-based continuing care for alcohol and cocaine dependence . J Consult Clin Psychol 75 :775–784. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Moncrieff J, Drummond DC (1997) New drug treatments for alcohol problems: a critical appraisal . Addiction 92 :939–947. [ PubMed ] [ Google Scholar ]
Mun EY, de la Torre J, Atkins DC, White HR, Ray AE, Kim SY, Jiao Y, Clarke N, Huo Y, Larimer ME, Huh D, The Project INTEGRATE Team. (2015). Project INTEGRATE: An integrative study of brief alcohol interventions for college students . Psychol Addict Behav , 29 :34–48. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Muthén LK, Muthén BO. (1998-2017). Mplus User's Guide . Eigth Edition. Los Angeles, CA: Muthén & Muthén. [ Google Scholar ]
Nowinski J, Baker S, Carroll KM (1995) Twelve step facilitation therapy manual . National Institute on Alcohol Abuse and Alcoholism, Rockville, MD. [ Google Scholar ]
Peter C, Cieza A, Geyh S (2014) Rasch analysis of the General Self-Efficacy Scale in spinal cord injury . J Health Psychol 19 :544–555. [ PubMed ] [ Google Scholar ]
Pilkonis PA, Yu L, Dodds NE, Johnston KL, Lawrence SM, Daley DC (2016) Validation of the alcohol use item banks from the Patient-Reported Outcomes Measurement Information System (PROMIS) . Drug Alcohol Depend 161 :316–322. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Project MATCH Research Group (1997) Matching alcoholism treatments to client heterogeneity . J Stud Alcohol 58 :7–29. [ PubMed ] [ Google Scholar ]
Ramo DE, Myers MG, Brown SA (2009) Psychometric properties of a revised form of the Drug-taking Confidence Questionnaire for use with adolescents . J Child Adolesc Subst Abuse 18 :24–42 [ PMC free article ] [ PubMed ] [ Google Scholar ]
Reilly PM, Sees KL, Shopshire MS, Hall SM, Delucchi KL, Tusel DJ, Banys P, Clark HW, Piotrowski NA (1995) Self-efficacy and illicit opioid use in a 180-day methadone detoxification treatment . J Consult Clin Psychol 63 :158–162. [ PubMed ] [ Google Scholar ]
Riazi A, Aspden T, Jones F (2014) Stroke Self-efficacy Questionnaire: a Rasch-refined measure of confidence post stroke . J Rehab Med 46 :406–412. [ PubMed ] [ Google Scholar ]
Shadel WG, Martino SC, Setodji C, Cervone D, Witkiewitz K (2017) Does self-efficacy causally influence initial smoking cessation? An experimental study . Addict Behav 73 :199–203. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Shrout P (2009). Short and long views of integrative data analysis: Comments on conributions to the special issue . Psychol Methods 14 :177–181. [ PubMed ] [ Google Scholar ]
Silins E, Horwood LJ, Najman JM, Patton GC, Toumbourou JW, Olsson CA, Hutchinson DM, Degenhardt L, Fergusson D, Becker D, Boden JM, Borschmann R, Plotnikova M, Youssef GJ, Tait RJ, Clare P, Hall WD, Mattick RP; Cannabis Cohorts Research Consortium. (2018). Adverse adult consequences of different alcohol use patterns in adolescence: an integrative analysis of data to age 30 years from four Australasian cohorts . Addiction . 113 :1811–1825 [ PubMed ] [ Google Scholar ]
Sitharthan T, Kavanagh DJ (1991) Role of self-efficacy in predicting outcomes from a programme for controlled drinking . Drug Alcohol Depend 27 :87–94. [ PubMed ] [ Google Scholar ]
Sklar SM, Annis HM Turner NE (1997) Development and validation of the drug-taking confidence questionnaire: A measure of coping self-efficacy . Addict Behav 22 :655–670. [ PubMed ] [ Google Scholar ]
Vielva I, Iraurgi I (2001) Cognitive and behavioural factors as predictors of abstinence following treatment for alcohol dependence . Addiction 96 :297–303. [ PubMed ] [ Google Scholar ]
Walsh JL, Weinhardt LS, Kalichman SC, Carey MP.(2017) Using Integrative Data Analysis to Examine Changes in Alcohol Use and Changes in Sexual Risk Behavior Across Four Samples of STI Clinic Patients . Ann Behav Med . 51 ( 1 ):39–56. [ PMC free article ] [ PubMed ] [ Google Scholar ]
Wilcox KT, Wang L (2021, July 29). Modeling approaches for cross-sectional integrative data analysis: Evaluations and recommendations . Psychol Methods . Advance online publication. 10.1037/met0000397 [ PubMed ] [ CrossRef ] [ Google Scholar ]
Witkiewitz K, Donovan DM, Hartzler B (2012) Drink refusal training as part of a combined behavioral intervention: effectiveness and mechanisms of change . J Consult Clin Psychol 803 :440–449. [ PMC free article ] [ PubMed ] [ Google Scholar ]

School of Public Health
Research Methods

Research Synthesis: Meta-analysis and Integrative Data Analysis

Meta Analysis Integrative Data Analysis10

Meta-analysis and integrative data analysis are research synthesis methods that aim to provide large-scale evidence by pooling data from multiple independently conducted studies.

Dr. Eun-Young Mun
Dr. Zhengyang Zhou

Clarke, N., Kim, S.-Y., White, H. R., Jiao, Y., & Mun, E.-Y. (2013). Associations between alcohol

use and alcohol-related negative consequences among Black and White college men and women. Journal of Studies on Alcohol and Drugs, 74 (4), 521-531.

Huh, D., Mun, E.-Y., Larimer, M. White, H. R., Ray, A. E., Rhew, I., Kim, S.-Y., Jiao, Y., & Atkins, D. C. (2015). Brief motivational interventions for college student drinking may not be as powerful as we think: An individual participant-level data meta-analysis. Alcoholism: Clinical and Experimental Research, 39 (5), 919-931.

Huh, D., Li, X., Zhou, Z., Walters, S. T., Baldwin, S. A., Tan, Z., … & Mun, E. Y. (2022). A structural equation modeling approach to meta-analytic mediation analysis using individual participant data: Testing protective behavioral strategies as a mediator of brief motivational intervention effects on alcohol-related problems. Prevention Science , 23 (3), 390-402.

Jiao, Y., Mun, E.-Y., Trikalinos, T., & Xie, M. (2020). A CD-based mapping method for combining multiple related parameters from heterogeneous intervention trials. Statistics and Its Interface, 13 (4), 533-549.

Mun, E. Y., Li, X., Lineberry, S., Tan, Z., Huh, D., Walters, S. T., … & Project INTEGRATE Team. (2022). Do brief alcohol interventions reduce driving after drinking among college students? A two-step meta-analysis of individual participant data. Alcohol and Alcoholism , 57 (1), 125-135.

Mun, E.-Y., & Ray, A. E. (2018). Integrative data analysis from a unifying research synthesis perspective. In H. E. Fitzgerald & L. I. Puttler (Eds.), Alcohol use disorders: A developmental science approach to etiology (pp. 341-353). New York: Oxford University Press.

Mun, E.-Y., Atkins, D. C., & Walters, S. T. (2015). Is motivational interviewing effective at reducing alcohol misuse in young adults? A critical review of Foxcroft et al. (2014). Psychology of Addictive Behaviors, 29 (4), 836-846.

Mun, E.-Y., de la Torre, J., Atkins, D. C., White, H. R., Ray, A. E., Kim, S.-Y., Jiao, Y., Clarke, N., Huo, Y., Larimer, M. E., Huh, D., & The Project INTEGRATE Team (2015). Project INTEGRATE: An integrative study of brief alcohol interventions for college students. Psychology of Addictive Behaviors, 29 (1), 34-48.

Mun, E.-Y., Jiao, Y., & Xie, M. (2016). Integrative data analysis for research in developmental

psychopathology. In D. Cicchetti (Ed.), Developmental psychopathology (3rd ed., Vol. 1, pp. 1042-1087). New York: Wiley.

Mun, E.-Y., Zhou, Z., Huh, D., Tan, L., Li, D., Tanner-Smith, E. E., Walters, S. T., & Larimer, M. E. (2022). Brief alcohol interventions are effective through six months: Findings from marginalized zero-inated Poisson and negative binomial models in a two-step IPD meta-analysis. Prevention Science . https://doi.org/10.1007/s11121-022-01420-1

White, H. R., Jiao, Y., Ray, A. E., Huh, D., Atkins, D. C., Larimer, M. E.,… Mun, E.-Y. (2015). Are there secondary effects on marijuana use from brief alcohol interventions for college students? Journal of Studies on Alcohol and Drugs, 76 (3), 367-377.

White, H. R., Anderson, K., Ray, A. E., & Mun, E.-Y. (2016). Do drinking motives distinguish extreme drinking college students from their peers? Addictive Behaviors, 60 , 213-218.

This page was last modified on October 23, 2023

Quick links

Alumni Relations
Media Contact
Accessibility Statement
Accreditation Summary
Bondholder Information
Careers at HSC
HSC Trust Line
Mental Health Services
Minors on Campus
Notice of Nondiscrimination
Privacy Policy
Report Behavioral Misconduct
Report Fraud, Waste or Abuse
Report Sexual Misconduct, Intimate Partner Violence and Stalking
Texas Veterans Portal

Connect with us

3500 Camp Bowie Blvd.
Fort Worth, Texas, 76107
(817) 735-2000

Social media

Student Life
Patient Care
After-Action Review
Continuing studies
COVID-19 information
Student services

Open access
Published: 08 July 2020

The ADHD teen integrative data analysis longitudinal (TIDAL) dataset: background, methodology, and aims

Margaret H. Sibley 1 , 2 &
Stefany J. Coxe 3

BMC Psychiatry volume 20 , Article number: 359 ( 2020 ) Cite this article

5919 Accesses

13 Citations

1 Altmetric

Metrics details

The Attention Deficit Hyperactivity Disorder (ADHD) Teen Integrative Data Analysis Longitudinal (TIDAL) dataset integrates data from four randomized trials.

Participants with ADHD ( N = 854; 72.5% male, 92.5% racial/ethnic minority, ages 10–17) were assessed three times across 12 months. Data includes parent, self, and teacher ratings, observations, and school records. The battery was harmonized using an Integrative Data Analysis (IDA) approach to form variables that assign unique values to all participants.

The data will be used to investigate: (1) profiles that organize the heterogeneous population into clinically meaningful subgroups, (2) whether these profiles predict treatment response, (3) heterogeneity in treatment response and variables that predict this response, (4) how treatment characteristics and adjunctive supports predict treatment response, and (5) mediators of treatment and whether these mechanisms are moderated by treatment characteristics.

Conclusions

The ADHD TIDAL Dataset will be openly shared with the field to maximize its utility.

Peer Review reports

The teenage years are a critical period for ADHD intervention. Longitudinal studies indicate a close relationship between adolescent functioning and ADHD persistence into adulthood [ 1 , 2 , 3 ]. Moreover, high neuroplasticity during adolescence may potentiate skill learning that maintains long-term [ 4 ], while negative adolescent life events (i.e., legal troubles, dropout, teen pregnancy) derail adult trajectories [ 1 ]. The last 10 years witnessed a proliferation of empirically-supported psychosocial interventions for adolescents with ADHD [ 5 , 6 ]. These interventions teach adolescents compensatory skills to mitigate the effects of executive functioning deficits on daily life while training adult stakeholders to use age-appropriate contingency management to reduce the effects of rewards processing deficits [ 7 , 8 ]. Psychosocial treatments are a strong developmental-fit to the adolescent period because: (1) adolescents with ADHD often dislike stimulant medication and desist use [ 9 , 10 ] and (2) adolescent psychosocial treatments outperform medication in mitigating ADHD-related impairment, showing medium to large effects [ 6 ].

Despite their efficacy, implementation and utilization of adolescent ADHD treatments are poor [ 11 , 12 , 13 , 14 ]. Thus, efforts are needed to identify patient and service delivery barriers and facilitators. Understanding who fails to engage in treatment and why will signal opportunities to decrease treatment disparities.

The clinical profiles of adolescents with ADHD are highly heterogeneous [ 15 , 16 , 17 ]; yet, almost nothing is known about which treatments work best for whom, when, where, and how. Practitioners have few guidelines for treatment optimization—critical choices that influence a treatment credibility and patient retention in care [ 18 ]. Studies investigating questions of moderation and mediation are necessary to inform nuanced clinical care decisions [ 19 , 20 ]. Unlike treatments for childhood ADHD [ 21 ], there are almost no large-scale trials of treatment for adolescent ADHD. Much could be gleaned from a large-scale investigation of mechanisms of change in adolescent ADHD treatment, identification of treatment-relevant phenotypes that influence response, and tracing the role of adjunctive supports (i.e., medication, parent involvement style, school accommodations) in enhancing or detracting from psychosocial treatment. Rich clinical information could be derived from investigating how treatment response varies as a function of person- and service delivery-level variables [ 22 ].

Pursuing these aims require person-level approaches (i.e., mixture modeling, latent class analysis) [ 23 ]. As the field of child and adolescent mental health moves toward precision medicine [ 24 ], personal-level analyses are essential to ensuring maximally effective care. Precision medicine studies have led to important advances in the treatment of childhood conduct problems [ 25 ] and adult depression [ 19 ], among other disorders. These analyses require large sample sizes that have not yet been available in the treatment of adolescent ADHD.

Overview of the ADHD TIDAL dataset

To increase data resources in the field of adolescent ADHD, we constructed the ADHD Teen Integrative Data Analysis Longitudinal (ADHD TIDAL) dataset. The ADHD TIDAL dataset will be openly shared with the field. We integrated data from four randomized trials ( N = 854; ages 10–17) [ 26 , 27 , 28 , 29 ] that cumulatively tested the comparative efficacy of unique five treatment conditions (i.e., evidence-based parent-teen therapy, group parent training and teen organization skills training, intensive summer treatment, usual care psychotherapy, no treatment) and included data from three unique settings (university clinic, schools, community mental health). These data were combined using an Integrative Data Analysis [ 30 ] framework, resulting in a comprehensive dataset. In the current paper, we describe the dataset, our methodology, planned analyses to pursue a linked series of person-level research questions, and additional research questions that scientific investigators might pursue using the dataset.

Construction and content

Integrative data analysis.

Integrative data analysis is a relatively new technique that allows researchers to pool raw data from multiple studies to produce cumulative scientific knowledge [ 30 ]. IDA differs from more well-known techniques for combining information, such as meta-analysis, in that IDA analyzes pooled raw data from each study rather than summary statistics. IDA has several advantages over separate analysis of each study [ 31 ], including increased statistical power, management of sample heterogeneity, and increased frequency of low base-rate behaviors. IDA framework required us to: (1) code study characteristics, (2) harmonize measures and/or create commensurate measures, and (3) select a type of IDA. The four included studies vary on a variety of characteristics (i.e., treatment and comparison groups, referral source, and time of year for treatment). A major task for IDA is carefully coding each study on these characteristics. Per Hussong and colleagues [ 31 ], we coded each study based on sampling approach, history, design characteristics, and measurement. Codes are integrated into analyses as dictated by research questions.

To conduct an analysis on the combined dataset, the same variables must be present in some form in all studies. The four studies were conducted by the same investigators, so many measures of interest are common across studies (e.g., Diagnostic Interview Schedule for Children) [ 32 ] and require no additional work to use in an IDA (though IDA provides an opportunity to explore measurement invariance across studies). Other measures, such as parent depression (e.g., Patient Health Questionnaire-9,Symptom Checklist-90-Revised, World Health Organization Quality of Life) [ 33 , 34 , 35 ] and ADHD symptoms (i.e., DSM-IV-TR vs. DSM-5 symptom checklists) [ 6 ] are not identical, requiring development of commensurate measures. Commensurate measures typically involve item response theory (IRT) analysis to create common scale scores [ 36 , 37 ].

IDA allows for either random or fixed effects models, depending on the number of studies and whether a study is conceptualized as a random sample from the population of interest. Fixed effects IDA conceives of each study as a known, specific sample from the population and can be conducted with as few as two studies. Random effects IDA conceives of each study as a random sample from the population and requires a minimum of 20 to 30 studies. The ADHD TIDAL dataset includes four studies, so all analyses are conducted within a fixed-effects IDA framework. This means that dummy codes indicating study membership are included in each analysis to account for differences between studies. We interpret all results within a fixed effects IDA framework indicating that: (1) we can only make inferences back to the specific studies, not to similar studies on this population and (2) we cannot fully disaggregate some between- and within-study effects, due to study-specific code variables in analyses (i.e., whether variance attributed to “summer treatment program” is due to time of year or dose).

Study designs

Common elements.

Our four studies were chosen for the IDA because they shared common characteristics that promoted successful harmonization. From 2010 to 2019, the research team conducted seven longitudinal treatment outcome studies of psychosocial treatment for adolescents with ADHD. We sought to include studies in the IDA that: (1) included participants from the large local school district with standardized attendance, grades, and disciplinary data, (2) possessed inclusion criteria that all participants meet DSM criteria for ADHD during a comprehensive psychiatric evaluation that included a structured parent interview (Diagnostic Interview Schedule for Children; DISC) [ 32 ] and parent and teacher symptom and impairment ratings that were integrated and reviewed by licensed clinical psychologists; (3) possessed a randomized controlled trial design; and (4) included baseline, post-treatment, and follow-up data points. Based on these criteria, two of the research teams studies were excluded from the IDA because they did not possess a follow-up data point and one was excluded because it did not possess a randomized control group. Comparison of the four included studies indicated additional common features that suit the IDA framework: (1) Autism Spectrum Disorders were exclusionary in all studies (participants with other comorbidities were included) and (2) all study participants were permitted to continue stimulant medication and special education interventions at school. These adjunctive treatments were monitored carefully and can be included as time varying covariates in analyses. An overview of study design features is provided in Table 1 .

In study A (see Table 1 ) [ 26 ] middle school students were randomized to Supporting Teens’ Autonomy Daily (STAND) [ 38 ] in the university clinic or a treatment as usual control group in which no treatment was offered to participants by the research team. Admission to the study used a cohort design with students receiving 10 weeks of treatment in the spring (cohort 1 and cohort 2) or the fall (cohort 3) of the academic year. At post-treatment, all participants had data from at least one source and 95% of participants had data from at least two sources. At follow-up, 97% had data from at least one source and 87% had data from at least two sources [ 26 ].

Study B (see Table 1 ) [ 27 ] randomly assigned rising 6th or 9th graders with ADHD to the intensive Summer Treatment Program-Adolescent (STPA) or group STAND (STAND-G). School district personnel delivered the STP-A, which was held in district schools with bus transportation provided. In the fourth year of the study, a no treatment comparison group of 107 students was recruited and tracked using the same assessment schedule to contextualize group differences between the two active treatment arms. Retention converged at 90–95% across sources, time points, and groups [ 27 ].

Middle or high school students with ADHD (see Table 1 ; N = 123) were randomly assigned to STAND or STAND-G using a stratified randomization procedure within study wave [ 28 ]. Study enrollment occurred in six waves with approximately ten participants per modality per wave. Recruitment occurred across 24 months with each wave occurring approximately 4 months apart. Treatment was delivered on a rolling basis throughout the academic year, with a pause in recruitment and treatment over the summer months. Retention was strong at post-treatment (95.1–97.6%) and follow-up (85.4–91.9%).

This trial (see Table 1 ) tested the effectiveness of STAND versus Usual Care in a sample of middle and high school students with ADHD ( N = 278) who were incoming patients at four community mental health clinics. Over 3 years, treatment was provided by agency employees who were randomly assigned to receive STAND training and supervision or treat cases using UC practices. Adolescents were also randomized to STAND vs. UC. Retention was 99.3% at post-treatment and 97.5% at follow-up (data from at least one informant).

Heterogeneity

Combining the data across the four studies increases sample heterogeneity and allows for examination of between-study heterogeneity. Heterogeneity (i.e., variance) is an advantage when trying to find relationships between variables. Particularly in studies of clinical populations, restricted range can reduce statistical power and impede detection of relationships between variables. The larger, more heterogeneous combined sample improves statistical power. In addition, traditionally under-represented groups (e.g., girls with ADHD) and behaviors (e.g., conduct disorder) are well-represented in the IDA sample.

While the four studies are similar in scope, they also differ in several ways. Study B includes school-referred youth, study D includes patients in community agencies, and studies A and C included patients at a university clinic. Studies A and B utilize a no treatment control group; studies B, C, and D compare active treatments, including therapist-selected intervention in community mental health (i.e., agency usual care). Study B included summer treatment, while studies A, C, and D included treatment delivered at various points during the school year.

Demographic Characteristics

Demographic characteristics of the full sample ( N = 854) are presented in Tables 2 and 3 . The larger combined dataset allows for examination of typically low base-rate sample characteristics. There are many clinically-meaningful behaviors that are infrequently exhibited, even in clinical samples of adolescents with ADHD, such as superior IQ, predominantly hyperactive/impulsive presentation, or conduct disorder. Tables 2 and 3 illustrate that typically underrepresented patient populations, such as females and African-Americans with ADHD can be pooled across studies to create cell sizes that are now sufficient for analysis.

Treatment conditions

Supporting teens’ autonomy daily (stand).

STAND is an engagement-focused psychosocial treatment for adolescent ADHD. STAND is manualized and consists of 10 weekly 60-min sessions attended by the adolescent and parent. Skill instruction is blended with Motivational Interviewing [ 39 ] and guided parent-teen behavioral contracting [ 40 ]. Treatment targets family, behavioral, and academic impairment. In the engagement phase, MI is used to increase awareness of personal values and goals, identify strengths, and recognize ways to achieve goals by acting consistently with values. The skills phase is designed to teach parent-teen communication, parent behavioral strategies, and organization, time management and planning skills applied to homework, school, and chores. Planning sessions teach families to integrate skills into a daily routine, transfer new habits to school settings, and build a final parent-teen contract. In all studies, STAND was offered in either English or Spanish. Therapists are offered 3 days of training.

STAND-group

STAND-Group (STAND-G) is manualized and consists of eight 90-min weekly sessions attended by the adolescent and parent [ 38 ]. Parents and adolescents meet in separate groups for the first 75 min and a blended parent-teen group for the final 15 min. Parent training employs the community-based model [ 41 ], alternating between didactic instruction, and small and full group discussions. Parents are exposed to the same skills as in STAND, including how to monitor academics, set a daily routine, apply behavioral principles to homework, and create a parent-teen contract. The adolescent skills group alternated didactic instruction (e.g., introduction of a new skill), hands on activities (e.g., organizing one’s backpack with a peer), and discussion exercises (e.g., debating the pros and cons of writing in a planner). Adolescents and parents are given suggested exercises to practice skills at home between sessions (e.g., negotiating a homework plan, organizing one’s backpack and scheduling parent backpack checks). Therapists receive 1 day of training prior to implementing treatment. In study B, school consultation was offered in addition to STAND-G; however, almost no participants received a meaningful dose of this intervention component [ 14 ].

Summer treatment program-adolescent

The 8-week STP-A [ 27 ] includes 45 h of youth directed treatment per week. Intervention includes rotating group modules targeting materials management, time management, planning, homework completion, note-taking, study skills, writing skills, self-monitoring, decision-making (including LifeSkills© Training) [ 42 ], social pragmatics, and independently managing responsibilities in a vocational program. Contingency management is incorporated to enhance adolescent motivation to practice skills. A two-week training includes didactics, discussions, tests, role-playing, and practice. Each day, lead counselors telephone parents to provide a verbal summary of the adolescent’s performance on daily treatment goals and offer coaching on home contingency management. Parents also receive an eight-week manualized parent training curriculum [ 43 ] as described for STAND-G.

Usual Care (UC) therapists at community mental health agencies were instructed to treat study cases using usual procedures in the agency and the treatments they believed would be most effective. They received weekly supervision from agency supervisors according to typical agency practices. Complex analyses of UC practices have been proposed as a future direction; at present, UC psychotherapy for adolescent ADHD remains a black box.

No treatment

In Study A, a treatment as usual comparison group was offered no treatment by the study team. In Study B, an untreated comparison group was followed in the fourth year of the study. In these conditions, participants were permitted to pursue naturalistic treatment in their communities.

Measures and available data

Table 4 lists available data for each of the measures by administration schedule.

Adolescent academic problems checklist

The self, parent, and teacher-report versions of the 24-item Adolescent Academic Problems Checklist (AAPC) measure observable secondary-school specific organization, time management, and planning (OTP) problems and are validated for use in samples of adolescents with ADHD [ 44 ]. The AAPC possesses two distinct factors (academic skills and disruptive behavior) and a total score, with strong internal reliability and concurrent validity [ 44 ]. One item was removed during scale development (locker organization) but is included.

Conflict behavior questionnaire

The parent and teen Conflict Behavior Questionnaire-20 (CBQ-20) assessed the quality of the parent-teen relationship. Respondents were asked to rate statements on a five-point scale from 1 ( strongly agree ) to 5 ( strongly disagree ) [ 45 ]. The CBQ-20 is a 20-item scale adapted from the 73-item CBQ. The CBQ-20 contains the CBQ items that best discriminated distressed and nondistressed families. It yields a single score that correlates .96 with the CBQ [ 45 ].

Disruptive behavior disorders rating scale

In the DSM-IV-TR era, the parent and teacher Disruptive Behavior Disorders Ratings Scale (DBD-RS) [ 46 ] measured Inattention (IN), Hyperactivity/Impulsivity (HI), ODD, and CD severity. Respondents were asked to rate symptoms as 0 ( not at all present ), 1 ( just a little ), 2 ( pretty much ), or 3 (v ery much ). To calculate an index of symptom severity the average level (0–3) of each item on the ADHD subscales is obtained. The psychometric properties of the DBD rating scale are very good, with support for internally consistent subscales [ 47 ].

DSM-5 ADHD rating scale

In the DSM-5 era, IN and HI were measured using a DSM-5 ADHD Rating Scale completed by adolescents, parents, and teachers [ 48 ]. Respondents rated symptoms of ADHD as 0 ( not at all ) to 3 (v ery much ). Symptom severity is the mean level (0–3) of ADHD subscale items. Psychometric properties of the measure are very good, with empirical support for internally consistent IN and HI subscales [ 48 ]. The DSM-5 ADHD rating scale includes the adolescent/adult symptom modifiers that were introduced in the DSM-5 [ 49 ].

Child behavior checklist and youth self report

The parent-reported Child Behavior Checklist (CBCL) and Youth Self Report (YSR) were administered as broadband youth psychopathology scales [ 49 ]. These scales are well-validated measures of psychosocial adjustment problems. T-scores for the full range of clinical, diagnostic, and competence scores are included in the dataset.

Observed organization skills

Observations of planner use assessed the degree to which students recorded homework assignments. Percentage of classes in which homework was recorded (or some indication of no homework) was calculated for the last 5 days of school. Planner use was calculated as the mean of daily scores. Photocopies or screenshots were obtained to document use. If the adolescent did not record any homework, he/she received a score of zero. This measure demonstrates high inter-rater reliability (intraclass correlation was .98 in past trials) [ 26 ]. Observations of bookbag organization were obtained using an adaptation of the Organization Checklist [ 50 ]. Trained research assistants assessed dichotomously scored items on the organization checklist such as “Is the adolescent’s bookbag free from loose papers?” Organization checklist scores are shown to correlate with teacher ratings of impairment in adolescents with ADHD [ 50 ].

Report cards were obtained directly from the school district at the end of each academic quarter. GPA for each quarter was calculated by converting academic grades (e.g., English, Math, Science, Social Studies) to a 5-point scale (i.e., 4.0 = A to 0.0 = F). Grades were not weighted for the difficulty of the class. GPA provides an objective and ecologically valid measure of school performance that is meaningful to parents and schools. The average grade on each completed assignment was also calculated. Assignments included any mandatory academic work turned in by the student except for tests, quizzes, and exams (i.e., homework, classwork, projects, presentations). Extra credit assignments and class participation were not counted towards this average. Missing assignments were also not weighted in the average. The average grade on each test, quiz, or exam was calculated. The percentage of assignments turned in calculated by dividing turned-in assignment count by the total number of assignments due.

Disciplinary incidents

The school district provided records of student disciplinary incidents at the end of each year. Counts of each type of disciplinary incident (e.g., detention, in-school suspension) were calculated and coded according to Robb and colleagues [ 51 ]. Minor disciplinary incidents included detentions, warnings, and being sent to an administrator or counselor due to behavioral issues. Major incidents included suspensions and expulsions.

IQ and academic achievement

The Wechsler Abbreviated Scale of Intelligence (WASI or WASI-II) was administered to participants as an index of IQ. Full-scale IQ was measured using a composite score from the Matrix Reasoning and Vocabulary subtests (Full-2) or all four subscales (Full-4) of the Wechsler Abbreviated Scale of Intelligence-2nd Edition (WASI-II) [ 52 ]. The WASI-II is a well-established test that has been validated for use with children, adolescents and adults. The WIAT-III is a standardized comprehensive academic achievement battery [ 53 ]. It has strong psychometric properties. The Numerical Operations subtest measured math achievement and the Word Reading subtest measured reading achievement. Standard scores are available for WASI and WIAT scores.

Impairment rating scale

The Impairment Rating Scale (IRS) was administered to parents and teachers (IRS) [ 54 ]. Parents and teachers indicated the adolescent’s impairment severity in seven domains on a Likert scale ranging from “0 = no problem” to “6 = extreme problem.” The IRS demonstrates strong psychometrics and accurately identifies ADHD-related impairment across settings and informants [ 54 ].

Diagnostic interview schedule for children (DISC)

The DISC is a structured interview that was administered to assess ADHD, ODD, and CD diagnoses. The DISC queries the presence of each symptom (0 = No, 1 = Yes). The ADHD module contains supplemental probes for symptom-specific impairment [ 32 ]. Symptom presence is evaluated for each symptom of ADHD, ODD, and CD using parent reports.

Parent academic management scale

The PAMS is a 16-item checklist that measures the frequency of adaptive and maladaptive parental involvement strategies related to adolescent OTP skills [ 55 ]. Parents indicate the number of days during the typical school week (0 to 5) that they performed each activity. PAMS possesses strong psychometric properties as evidenced by good internal consistency, concurrent validity, and predictive validity [ 55 ]. In 2016, an item was added to the PAMS querying the number of hours the parent spends each week in activities related to the adolescent’s academics.

Caregiver strain questionnaire

Parent strain stemming from the parent-adolescent relationship was measured by the 21 item Caregiver Strain Questionnaire (CSQ) [ 56 ]. The parent indicates how his/her child’s problems affected the parents and family over the past 4 weeks. Responses were scored on a 5-point scale ranging from not at all to very much a problem. The CSQ shows strong psychometric properties and the measure correlates well with other measures of family functioning.

Adult ADHD self-report scale

The Adult ADHD Self-Report Scale (ASRS) measured parental ADHD [ 57 ]. Eighteen adult-specific ADHD symptoms were rated on a five-point scale (0 = Never to 4 = Very Often). The ASRS correlates highly with clinician ADHD ratings and displays strong internal consistency [ 57 ]. Parental ADHD severity is calculated as the mean score of ASRS items.

Symptom Checklist-90-revised

The Symptom Checklist-90-Revised (SCL-90-R) is a 90-item broadband scale of adult psychopathology that measures nine symptom domains using a 5-point Likert scale [ 34 ]. The SCL-90-R has good internal consistency for each subscale and possesses convergent, discriminant, and predictive validity [ 58 ]. Individual items and T-scores from the SCL-90-R are included in the dataset.

Patient health Questionnaire-9

The Patient Health Questionnaire-9 depression scale has excellent internal reliability as well as criterion and construct validity [ 33 ]. Parents reported on whether they experienced a range of depressive symptoms during the past 2 weeks, rating symptom frequency from 0-not at all to 3-nearly every day.

World Health Organization quality of life questionnaire

Parents completed the World Health Organization (WHO) Quality of Life Questionnaire, a multidimensional profile of quality of life for cross cultural use. The English version is self-administered and covers 25 facets of quality of life within six broad domains. It captures positive and negative aspects of quality of life and possesses strong psychometric properties [ 35 ].

Harmonization

Harmonization of measures followed methods of Bauer (2017) [ 36 ] and Curran et al. (2008) [ 37 ].

Parent depression

Parent depression was assessed by the SCL-90 (studies A and B), PHQ-9 (study C), and the WHO QOL (study D). Specific items from the SCL-90 (13 of 90 items) and the WHO QOL (5 of 26 items) that reflected symptoms of major depression were selected; the PHQ-9 measures the nine DSM symptoms of major depression and all items were included. Each participant’s item scores were combined and recoded to reflect endorsement of the nine DSM-5 major depression symptoms. Nonlinear moderated factor analysis (NLMFA) was used to determine item-specific characteristics (i.e., discrimination and difficulty) within the IRT framework, how those item characteristics vary across study, and to provide each participant with a common-scale score of depression [ 36 ]. This harmonized, common-scale score can be used in any analyses involving the total IDA sample.

ADHD severity

Adolescent ADHD symptoms were measured using DSM-IV criteria in studies A and B and using DSM-5 criteria in studies C and D; both parent and teacher report of adolescent ADHD symptoms were collected. The DSM-IV-TR criteria (e.g., often fails to give close attention to details or makes careless mistakes in schoolwork, work, or during other activities) omits corresponding adolescent/adult specifiers added to the DSM-5 ADHD criteria (e.g. … overlooks or misses details, work is inaccurate). For each participant, item scores were recoded to reflect endorsement the 18 ADHD symptoms. Since ADHD symptoms are grouped into IN and HI, a two-factor model was used. Nonlinear moderated factor analysis (NLMFA) was used to determine item-specific characteristics (i.e., discrimination and difficulty) within the IRT framework, how those item characteristics vary across study, and to provide each participant with common-scale scores of IN and HI ADHD symptoms [ 36 ]. These harmonized, common-scale scores can be used in any analyses involving the total IDA sample. Separate models and scores were created for parent report and teacher report of symptoms.

Utility and discussion

Our research team has several analyses planned using the ADHD TIDAL dataset. However, numerous opportunities for data analysis exist beyond our specific aims. We believe that pursuing personalized medicine questions for adolescents with ADHD will provide useful information that promotes improved treatment engagement and response—leading to meaningful changes in long-term outcomes. We invite additional research teams to utilize the ADHD TIDAL dataset, which is publicly available for use at the National Institute of Mental Health, National Data Archive ( https://nda.nih.gov ).

Investigator aims

Our first aim is to identify clinical and family-risk profiles that divide the heterogeneous population into clinically meaningful subgroups. In doing so, we will identify unobserved groups of individuals who differ from one another on a combination of baseline measures. Latent profile analysis (LPA) will be used to identify treatment-relevant phenotypes and environmental factors based on relevant individual (e.g., gender, race/ethnicity, age, ODD/CD severity, ADHD subtype, depression severity, anxiety severity, organization skills, % of school work turned in, average test/assignment grades, school attendance, school disciplinary incidents, IQ, achievement) and family context variables (e.g., parent education level/SES, parent English skills, parent marital status, parent-teen conflict, parental ADHD, parental well-being, family size). Variables that demonstrate superior psychometric properties when modeled as an observed variable will not be modeled in the context of the latent profiles.

In a second aim, we will examine whether baseline latent profile and observed variables predict treatment engagement and response. The first analyses will describe who is most at risk for treatment disengagement (i.e., medication and psychosocial). Finally, we will examine whether baseline latent profile and observed variables predict treatment response, with primary outcomes (ADHD symptoms, parent-teen conflict, and GPA) as the distal outcomes.

Our third aim will examine heterogeneity in treatment response over time. This aim will identify latent, unobserved groups of individuals who differ from one another in terms of their outcome (ADHD symptoms, parent-teen conflict, GPA, OTP problems) trajectory over time. This analysis will allow us to examine this heterogeneity and determine if clinical profile, family context, adjunctive supports (e.g., medication status, parent involvement, class placement, school accommodations), and treatment characteristics (e.g., time of year, setting of treatment, % of treatment attended, content of treatment) predict treatment response.

Our fourth aim will be to identify key treatment mediators and moderators of the relationship between treatment group and change in key outcomes (ADHD symptoms, GPA, parent-teen conflict, OTP problems). Potential mediators of the treatment effect on outcomes include teen organization skills, parent contingency management, parent-teen conflict, and parental well-being. Potential moderators of the treatment effect on outcomes include individual, treatment, and family variables (as noted above).

Sensitivity analysis

Given the fixed sample size, we present sensitivity analyses that provide the smallest effect that can be detected, rather than power analyses. For latent profile analyses and growth mixture models proposed in aims 1 through 3, simulation studies indicate that the Bayesian Information Criterion and bootstrap likelihood-ratio test perform best at determining the correct number of classes [ 59 , 60 ]. These studies show that these measures have at least 80% power to detect the correct number of classes when sample size is greater than 500, if at least 8 indicators of the latent class are used. With a combined sample size of 854, we expect to have sufficient power to correctly identify emergent latent classes based on baseline variables. In aim 4 we will examine questions of moderated mediation with a sample size of 854. For moderation analyses, treatment level variables, the individual level moderator, and their interaction will predict change over time in the outcome variable. A simplified power analysis for a repeated-measures ANOVA design with an interaction between group and time (a much less powerful model than a latent growth model) suggests that for N = 854, 9 treatment groups (across all four studies), and 3 measurement occasions, the required effect size is approximately d = 0.12, a very small effect. In previous analyses of individual studies included in this IDA, we found effect sizes of d = 0.5 or higher for moderation of the treatment effect on ADHD symptoms (i.e., Sibley et al., 2016). For the mediation analyses, we used the tables generated by Fritz and MacKinnon [ 61 ]. With a sample size of N = 854 and using the preferred method of bootstrap confidence intervals, we have greater than 80% power to detect even small effects for both the treatment to mediator and mediator to outcome slope. In previous analyses of individual studies included in this IDA, we found effect sizes of d = 0.5 or higher for the moderating effect of treatment on ADHD symptoms (equivalent to the effect of a mediating variable on the outcome slope).

Additional research directions

The ADHD TIDAL dataset is suitable for examining treatment outcome questions that expand on those noted above by selecting independent and dependent variables, moderators, and mediators that our team did not incorporate into our planned analyses. Given the broad age range of participants, cross-sequential analyses with the ADHD TIDAL dataset could reveal important information about the nature of ADHD symptoms and related impairments. The samples demographic diversity also may support research questions related to gender, cultural, or socioeconomic differences in the expression of ADHD. The broad range of data available in the ADHD TIDAL dataset also may help research teams estimate effect sizes for power analyses and conduct pilot analyses prior to data collection studies, as well as further integration with existing datasets.

Availability of data and materials

The datasets generated during and/or analysed during the current study are available in the National Institute of Mental Health, National Data Archive repository, https://nda.nih.gov .

Abbreviations

Attention deficit/hyperactivity disorder

Integrative data analysis;

Grade point average

Teen integrative data analysis longitudinal

Diagnostic interview schedule for children

Supporting teens’ autonomy daily

Supporting teen’s autonomy daily-group

Organization, time management, and planning

Disruptive behavior disorder rating scale

Inattention

Hyperactivity/impulsivity

Diagnostic and statistical manual

Child behavior checklist

Youth self report

Wechsler abbreviated scale of intelligence

Intelligence quotient

Wechsler Individual achievement test

Adult ADHD self report scale

Symptoms checklist-90-revised

Patient health questionnaire-9

World health organization

Quality of life

Nonlinear moderated factor analysis

Oppositional defiant disorder

Conduct disorder

Barkley RA, Murphy KR, Fischer M. ADHD in adults: what the science says. New York: Guilford. 2008. https://doi.org/10.1177/1087054709333321 .

Molina BS, Pelham Jr WE, Cheong J, Marshal MP, Gnagy EM, Curran PJ. Childhood ADHD and growth in adolescent alcohol use: the roles of functional impairments, ADHD symptom persistence, and parental knowledge. J Abnorm Psychol 2012;121(4):922. https://doi.org/10.1037/a0028260 .

Sibley MH, Pelham Jr WE, Molina BS, Coxe S, Kipp H, Gnagy EM, Meinzer M, Ross JM, Lahey BB. The role of early childhood ADHD and subsequent CD in the initiation and escalation of adolescent cigarette, alcohol, and marijuana use. J Abnorm Psychol 2014;123(2):362. https://doi.org/10.1037/a0036585 .

Blakemore SJ, Choudhury S. Development of the adolescent brain: implications for executive function and social cognition. J Child Psychol Psychiatry. 2006;47 https://doi.org/10.1111/j.1469-7610.2006.01611.x .

Chan E, Fogler JM, Hammerness PG. Treatment of attention-deficit/hyperactivity disorder in adolescents: a systematic review. Jama. 2016;315(18):1997–2008. https://doi.org/10.1001/jama.2016.5453 .

Sibley MH, Kuriyan AB, Evans SW, Waxmonsky JG, Smith BH. Pharmacological and psychosocial treatments for adolescents with ADHD: an updated systematic review of the literature. Clin Psychol Rev 2014;34(3):218–232. https://doi.org/10.1016/j.cpr.2014.02.001 .

Castellanos FX, Sonuga-Barke EJ, Milham MP, Tannock R. Characterizing cognition in ADHD: beyond executive dysfunction. Trends Cogn Sci 2006;10(3):117–123. https://doi.org/10.1016/j.tics.2006.01.011 .

Sonuga-Barke EJ. Causal models of attention-deficit/hyperactivity disorder: from common simple deficits to multiple developmental pathways. Biol Psychiatry 2005;57(11):1231–1238. https://doi.org/10.1016/j.biopsych.2004.09.008 .

Brinkman WB, Simon JO, Epstein JN. Reasons why children and adolescents with attention-deficit/hyperactivity disorder stop and restart taking medicine. Acad Pediatr 2018;18(3):273–280. https://doi.org/10.1016/j.acap.2017.09.005 .

Molina BS, Hinshaw SP, Swanson JM, Arnold LE, Vitiello B, Jensen PS, Epstein JN, Hoza B, Hechtman L, Abikoff HB, Elliott GR. The MTA at 8 years: prospective follow-up of children treated for combined-type ADHD in a multisite study. J Am Acad Child Adolesc Psychiatry 2009;48(5):484–500. https://doi.org/10.1097/CHI.0b013e31819c23d0 .

Barkley RA, Edwards G, Laneri M, Fletcher K, Metevia L. The efficacy of problem-solving communication training alone, behavior management training alone, and their combination for parent–adolescent conflict in teenagers with ADHD and ODD. J Consult Clin Psychol 2001;69(6):926. https://doi.org/10.1037/0022-006X.69.6.926 .

Bussing R, Zima BT, Mason DM, Porter PC, Garvan CW. Receiving treatment for attention-deficit hyperactivity disorder: do the perspectives of adolescents matter?. J Adolesc Health 2011;49(1):7–14. https://doi.org/10.1016/j.jadohealth.2010.08.014 .

Evans SW, Serpell ZN, Schultz BK, Pastor DA. Cumulative benefits of secondary school-based treatment of students with attention deficit hyperactivity disorder. Sch Psychol Rev 2007;36(2):256–273. https://doi.org/10.1080/02796015.2007.12087943 .

Sibley MH, Olson S, Morley C, Campez M, Pelham Jr WE. A school consultation intervention for adolescents with ADHD: barriers and implementation strategies. Child Adolesc Mental Health 2016;21(4):183–191. https://doi.org/10.1111/camh.12159 .

Fair DA, Bathula D, Nikolas MA, Nigg JT. Distinct neuropsychological subgroups in typically developing youth inform heterogeneity in children with ADHD. Proc Natl Acad Sci 2012;109(17):6769–6774. https://doi.org/10.1073/pnas.1115365109 .

Martel MM, Roberts B, Gremillion M, Von Eye A, Nigg JT. External validation of bifactor model of ADHD: explaining heterogeneity in psychiatric comorbidity, cognitive control, and personality trait profiles within DSM-IV ADHD. J Abnorm Child Psychol 2011;39(8):1111. https://doi.org/10.1007/s10802-011-9538-y .

Wåhlstedt C, Thorell LB, Bohlin G. Heterogeneity in ADHD: neuropsychological pathways, comorbidity and symptom domains. J Abnorm Child Psychol 2009;37(4):551–564. https://doi.org/10.1007/s10802-008-9286-9 .

Gray JM. The shift to personalised and population medicine. Lancet 2013;382(9888):200–201. https://doi.org/10.1016/S0140-6736(13)61590-1 .

Frank E, Cassano GB, Rucci P, Thompson WK, Kraemer HC, Fagiolini A, Maggi L, Kupfer DJ, Shear MK, Houck PR, Calugi S. Predictors and moderators of time to remission of major depression with interpersonal psychotherapy and SSRI pharmacotherapy. Psychol Med 2011;41(1):151–162. https://doi.org/10.1017/S0033291710000553 .

Langberg JM, Evans SW, Schultz BK, Becker SP, Altaye M, Girio-Herrera E. Trajectories and predictors of response to the challenging horizons program for adolescents with ADHD. Behav Ther 2016;47(3):339–354. https://doi.org/10.1016/j.beth.2016.01.001 .

The MTA Cooperative Group. A 14-month randomized clinical trial of treatment strategies for attention-deficit/hyperactivity disorder. Arch Gen Psychiatry 1999;56(12):1073–1086. https://doi.org/10.1001/archpsyc.56.12.1073 .

Wallace ML, Frank E, Kraemer HC. A novel approach for developing and interpreting treatment moderator profiles in randomized clinical trials. JAMA Psychiatry 2013;70(11):1241–1247. https://doi.org/10.1001/jamapsychiatry.2013.1960 .

Collins LM, Lanza ST. Latent class and latent transition analysis: with applications in the social, behavioral, and health sciences. New York: Wiley; 2010. https://doi.org/10.1002/9780470567333 .

Kazdin AE. Evidence-based psychosocial treatment: advances, surprises, and needed shifts in foci. Cogn Behav Pract 2016;23(4):426–430. https://doi.org/10.1016/j.cbpra.2015.11.003 .

Pelham WE, Dishion TJ, Tein JY, Shaw DS, Wilson MN. What doesn’t work for whom? Exploring heterogeneity in responsiveness to the family check-up in early childhood using a mixture model approach. Prev Sci 2017;18(8):911–922. https://doi.org/10.1007/s11121-017-0805-1 .

Sibley MH, Graziano PA, Kuriyan AB, Coxe S, Pelham WE, Rodriguez L, Sanchez F, Derefinko K, Helseth S, Ward A. Parent–teen behavior therapy+ motivational interviewing for adolescents with ADHD. J Consult Clin Psychol 2016;84(8):699. https://doi.org/10.1037/ccp0000106 .

Sibley MH, Coxe SJ, Campez M, Morley C, Olson S, Hidalgo-Gato N, Gnagy E, Greiner A, Coles EK, Page T, Pelham WE. High versus low intensity summer treatment for ADHD delivered at secondary school transitions. J Clin Child Adolesc Psychol 2018;47(2):248–265. https://doi.org/10.1080/15374416.2018.1426005 .

Sibley, MH, Rodriguez, LM, Coxe, SJ, Page, T, & Espinal, K . Parent-teen group versus dyadic treatment for adolescent ADHD: what works for whom? J Clin Child Adolesc Psychol. 2019: https://doi.org/10.1080/15374416.2019.1585257 .

Sibley, MH, Graziano, PA, Bickman, L, Coxe, SJ, Martin, P, Rodriguez, LM, Fallah, N, Ortiz, M (in press). Implementing parent-teen motivational interviewing + behavior therapy for ADHD in community mental health. Prev Sci.

Curran PJ, Hussong AM. Integrative data analysis: the simultaneous analysis of multiple data sets. Psychol Methods 2009;14(2):81. https://doi.org/10.1037/a0015914 .

Hussong AM, Curran PJ, Bauer DJ. Integrative data analysis in clinical psychology research. Annu Rev Clin Psychol 2013;9:61–89. https://doi.org/10.1146/annurev-clinpsy-050212-185522 .

Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-Stone ME. NIMH diagnostic interview schedule for children version IV (NIMH DISC-IV): description, differences from previous versions, and reliability of some common diagnoses. J Am Acad Child Adolesc Psychiatry 2000;39(1):28–38. https://doi.org/10.1097/00004583-200001000-00014 .

Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606. https://doi.org/10.1046/j.1525-1497.2001.016009606.x .

Article CAS PubMed PubMed Central Google Scholar

Derogatis LR. SCL-90-R: symptom Checklist-90-R: administration, scoring, and procedures manual. New York: NCS Pearson; 1996.

Whoqol Group. The world health organization quality of life assessment (WHOQOL): position paper from the world health organization. Soc Sci Med 1995;41(10):1403–1409. https://doi.org/10.1016/0277-9536(95)00112-K .

Bauer DJ. A more general model for testing measurement invariance and differential item.functioning. Psychol Methods. 2017;22(3):507. https://doi.org/10.1037/met0000077 .

Curran PJ, Hussong AM, Cai L, Huang W, Chassin L, Sher KJ, Zucker RA. Pooling data from multiple longitudinal studies: the role of item response theory in integrative data analysis. Dev Psychol 2008;44(2):365. https://doi.org/10.1037/0012-1649.44.2.365 .

Sibley MH. Parent-teen therapy for executive function deficits and ADHD: building skills and motivation. New york: Guilford Publications; 2016.

Miller WR, Rollnick S. Motivational interviewing: helping people change. New york: Guilford press; 2013.

Patterson GR, Forgatch M. Parents and adolescents living together: part 1: the basics. Eugene, OR: Castalia; 1987.

Google Scholar

Cunningham CE. Large group, community based, family-centered parent training. Attention deficit hyperactivity disorder: A clinical workbook 2005:480–98. https://doi.org/10.1016/j.ambp.2006.05.003 .

Botvin GJ, Griffin KW. Life skills training: empirical findings and future directions. J Prim Prev 2004;25(2):211–232. https://doi.org/10.1023/B:JOPP.0000042391.58573.5b .

Sibley MH, Altszuler AR, Ross JM, Sanchez F, Pelham Jr WE, Gnagy EM. A parent-teen collaborative treatment model for academically impaired high school students with ADHD. Cogn Behav Pract 2014;21(1):32–42. https://doi.org/10.1016/j.cbpra.2013.06.003 .

Sibley MH, Altszuler AR, Morrow AS, Merrill BM. Mapping the academic problem behaviors of adolescents with ADHD. Sch Psychol Q 2014;29(4):422. https://doi.org/10.1037/spq0000071 .

Robin AL, Foster SL. Negotiating parent–adolescent conflict: A behavioral–family systems approach;1989.

Pelham Jr WE, Gnagy EM, Greenslade KE, Milich R. Teacher ratings of DSM-III-R symptoms for the disruptive behavior disorders. J Am Acad Child Adolesc Psychiatry 1992;31(2):210–218. https://doi.org/10.1097/00004583-199203000-00006 .

Evans SW, Brady CE, Harrison JR, Bunford N, Kern L, State T, Andrews C. Measuring ADHD and ODD symptoms and impairment using high school teachers' ratings. J Clin Child Adolesc Psychol 2013;42(2):197–207. https://doi.org/10.1080/15374416.2012.738456 .

Sibley MH, Kuriyan AB. DSM-5 changes enhance parent identification of symptoms in adolescents with ADHD. Psychiatry Res 2016;242:180–185. https://doi.org/10.1016/j.psychres.2016.05.036 .

Achenbach TM. The Achenbach system of empirically based assessment (ASEBA): development, findings, theory, and applications. Burlington: University of Vermont, Research Center for Children, Youth, & Families; 2009. https://doi.org/10.1002/9781118625392.wbecp150 .

Evans SW, Schultz BK, White LC, Brady C, Sibley MH, Van Eck K. A school-based organization intervention for young adolescents with attention-deficit/hyperactivity disorder. Sch Ment Heal 2009;1(2):78–88. https://doi.org/10.1007/s12310-009-9009-6 .

Robb JA, Sibley MH, Pelham WE, Foster EM, Molina BS, Gnagy EM, Kuriyan AB. The estimated annual cost of ADHD to the US education system. Sch Ment Heal 2011;3(3):169–177. https://doi.org/10.1007/s12310-011-9057-6 .

Wechsler D. WASI-II: Wechsler abbreviated scale of intelligence. San Antonio: PsychCorp; 2011.

Wechsler D. Manual for the Wechsler individual achievement test-(WIAT-III). San Antonio; 2009.

Fabiano GA, Pelham, Jr WE, Waschbusch DA, Gnagy EM, Lahey BB, Chronis AM, Onyango AN, Kipp H, Lopez-Williams A, Burrows-MacLean L. A practical measure of impairment: psychometric properties of the impairment rating scale in samples of children with attention deficit hyperactivity disorder and two school-based samples. J Clin Child Adolesc Psychol 2006;35(3):369–385. https://doi.org/10.1207/s15374424jccp3503_3 .

Sibley MH, Campez M, Perez A, Morrow AS, Merrill BM, Altszuler AR, Coxe S, Yeguez CE. Parent management of organization, time management, and planning deficits among adolescents with ADHD. J Psychopathol Behav Assess 2016;38(2):216–228. https://doi.org/10.1007/s10862-015-9515-9 .

Brannan AM, Heflinger CA, Bickman L. The caregiver strain questionnaire: measuring the impact on the family of living with a child with serious emotional disturbance. J Emot Behav Disord 1997 ;5(4):212–222. https://doi.org/10.1177/106342669700500404 .

Adler LA, Spencer T, Faraone SV, Kessler RC, Howes MJ, Biederman J, Secnik K. Validity of pilot adult ADHD self-report scale (ASRS) to rate adult ADHD symptoms. Ann Clin Psychiatry 2006;18(3):145–148. https://doi.org/10.1080/10401230600801077 .

Derogatis LR, Cleary PA. Confirmation of the dimensional structure of the SCL-90: a study in construct validation. J Clin Psychol 1977;33(4):981–989. https://doi.org/10.1002/1097-4679(197710)33:4<981::AID-JCLP2270330412>3.0.CO;2-0 .

Nylund KL, Asparouhov T, Muthén BO. Deciding on the number of classes in latent class analysis and growth mixture modeling: a Monte Carlo simulation study. Struct Equ Model. 2007;14(4):535–69.

Article Google Scholar

Tein JY, Coxe S, Cham H. Statistical power to detect the correct number of classes in latent profile analysis. Struct Equ Model Multidiscip J. 2013;20(4):640–57.

Fritz MS, MacKinnon DP. Required sample size to detect the mediated effect. Psychol Sci. 2007;18(3):233–9.

Download references

Acknowledgements

The authors acknowledge Dr. Patrick Curran for his advisement with respect to the harmonization of research data. The authors also acknowledge Ms. Mercedes Ortiz for her assistance organizing study data.

This research was funded by the National Institute of Mental Health R03 MH116397. It was also supported by IES R324A120169, R01 MH106587, R34 MH092466, and the Klingenstein Third Generation Foundation Fellowship in ADHD. The funding agencies played no role in the design, administration or interpretation of research data for this study.

Author information

Authors and affiliations.

Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle Children’s Research Insitute, 2001 8th Ave., Suite 400, Seattle, WA, 98117, USA

Margaret H. Sibley

Department of Psychiatry & Behavioral Health, Florida International University, 11200 SW 8th Street, Miami, FL, 33199, USA

Department of Psychology, Florida International University, 11200 SW 8th Street, Miami, FL, 33199, USA

Stefany J. Coxe

You can also search for this author in PubMed Google Scholar

Contributions

MHS pooled data, built the initial dataset, and contributed to design of the study aims noted here in. SJC conducted data coding and statistical analyses related to harmonization of the dataset. MHS and SJC contributed to paper writing. All authors have read and approved this manuscript.

Corresponding author

Correspondence to Margaret H. Sibley .

Ethics declarations

Ethics approval and consent to participate.

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee (Florida International University, Social and Behavioral Institutional Review Board; IRB00008169; Reference #: IRB-18-0415) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Written informed consent was obtained from all individual participants included in the study. Written parental consent and written youth assent was obtained for all participants under the age of 18.

Consent for publication

Not applicable.

Competing interests

MHS receives book royalties from Guilford Press for a treatment manual described herein. SJC reports no conflicts of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Sibley, M.H., Coxe, S.J. The ADHD teen integrative data analysis longitudinal (TIDAL) dataset: background, methodology, and aims. BMC Psychiatry 20 , 359 (2020). https://doi.org/10.1186/s12888-020-02734-6

Download citation

Received : 02 March 2020

Accepted : 12 June 2020

Published : 08 July 2020

DOI : https://doi.org/10.1186/s12888-020-02734-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Longitudinal data
Adolescence
Intervention

BMC Psychiatry

ISSN: 1471-244X

Submission enquiries: [email protected]
General enquiries: [email protected]

integrative data analysis in clinical psychology research

COMMENTS

Integrative Data Analysis in Clinical Psychology Research
Integrative data analysis (IDA), a novel framework for conducting the simultaneous analysis of raw data pooled from multiple studies, offers many advantages including economy (i.e., reuse of extant data), power (i.e., large combined sample sizes), the potential to address new questions not answerable by a single contributing study (e.g., combining longitudinal studies to cover a broader swath ...
PDF Integrative Data Analysis in Clinical Psychology Research
measurement harmonization, pooled data analysis, collaborative science, data sharing, research methods, cumulative science. Abstract. Integrative data analysis (IDA), a novel framework for conducting the simultaneous analysis of raw data pooled from multiple studies, offers many advantages including economy (i.e., reuse of extant data), power ...
Integrative data analysis in clinical psychology research.
Integrative data analysis (IDA), a novel framework for conducting the simultaneous analysis of raw data pooled from multiple studies, offers many advantages including economy (i.e., reuse of extant data), power (i.e., large combined sample sizes), the potential to address new questions not answerable by a single contributing study (e.g., combining longitudinal studies to cover a broader swath ...
Integrative Data Analysis in Clinical Psychology Research
Abstract and Figures. Integrative data analysis (IDA), a novel framework for conducting the simultaneous analysis of raw data pooled from multiple studies, offers many advantages including economy ...
Integrative Data Analysis: The Simultaneous Analysis of Multiple Data
The authors define integrative data analysis (IDA) as the analysis of multiple data sets that have been pooled into one. ... Measurement invariance in longitudinal clinical research assuming change from development and intervention. Journal of Consulting and Clinical Psychology. 1994; 62:450-462. [Google Scholar] Pfeffermann D. The role of ...
Integrative data analysis: the simultaneous analysis of multiple data
The authors define integrative data analysis (IDA) as the analysis of multiple data sets that have been pooled into one. Although variants of IDA have been incorporated into other scientific disciplines, the use of these techniques is much less evident in psychology. In this article the authors present an overview of IDA as it may be applied ...
Integrative data analysis in clinical psychology research
Proposed solutions to methodological challenges associated with IDA are outlined, future avenues for developing IDA as a framework for studies in clinical psychology are described, and potential solutions to these challenges are described. Integrative data analysis (IDA), a novel framework for conducting the simultaneous analysis of raw data pooled from multiple studies, offers many advantages ...
Integrative Data Analysis in Clinical Psychology Research
Integrative data analysis (IDA), a novel framework for conducting the simultaneous analysis of raw data pooled from multiple studies, offers many advantages inc ... Andrea M. and Curran, Patrick J. and Bauer, Daniel J., Integrative Data Analysis in Clinical Psychology Research (March 2013). Annual Review of Clinical Psychology, Vol. 9, pp. 61 ...
Integrative Data Analysis in Clinical Psychology Research
Meta-analysis synthesizes a body of research investigating a common research question. Outcomes from meta-analyses provide a more objective and transparent summary of a research area than traditional narrative reviews. Moreover, they are often used to support research grant applications, guide clinical practice and direct health policy.
Informing Harmonization Decisions in Integrative Data Analysis
In practice, both logical and analytic harmonization require the research team to make multiple decisions, and thus, IDA is better considered a set of research methodologies rather than a single analytic approach. ... Integrative data analysis in clinical psychology research. Annual Review of Clinical Psychology, 9, 61-89.
Harmonizing Depression Measures Across Studies: a Tutorial for Data
There has been increasing interest in applying integrative data analysis (IDA) to analyze data across multiple studies to increase sample size and statistical power. Measures of a construct are frequently not consistent across studies. ... Integrative data analysis in clinical psychology research. Annual Review of Clinical Psychology, 9, 61-89.
[PDF] Integrative data analysis: the simultaneous analysis of multiple
The authors define integrative data analysis (IDA) as the analysis of multiple data sets that have been pooled into one. Although variants of IDA have been incorporated into other scientific disciplines, the use of these techniques is much less evident in psychology. ... Integrative data analysis in clinical psychology research. A. Hussong P ...
Integrative Data Analysis from a Unifying Research Synthesis
The purpose of this chapter is to present a broader unified framework for integrative data analysis (IDA; Curran & Hussong, 2009; Hussong, Curran, & Bauer, 2013) and to provide an overview of the strengths of IDA as well as future directions for IDA.IDA, defined as "the statistical analysis of a single data set that consists of two or more separate samples that have been pooled into one ...
Advantages of Integrative Data Analysis for Developmental Research
Data integration can be visualized on a continuum from analysis of single study raw data at one extreme (least integrative) to combining summary statistics from many studies using meta-analysis at the other (most integrative); IDA lies in between these two extremes (Curran & Hussong, 2009). All of these approaches play important roles within ...
Steps Toward an Integrative Clinical Systems Psychology
Perspectives. Applications of constructs of systems science in the field of psychology can be focused on concepts like "structure" or "dynamics" as will be done below. Several models - e.g., control loop concepts, synergetics, chaos theory - can be used to re-conceptualize essential findings in (clinical) psychology.
Introduction to the Special Issue on Innovations and Applications of
This paper serves as an introduction to the special issue of Prevention Science entitled, "Innovations and Applications of Integrative Data Analysis (IDA) and Related Data Harmonization Procedures in Prevention Science." This special issue includes a collection of original papers from multiple disciplines that apply individual-level data synthesis methodologies, including IDA, individual ...
Trajectories and predictors of response in youth anxiety CBT
Objective: Integrative data analysis was used to combine existing data from nine trials of cognitive-behavioral therapy (CBT) for anxious youth (N = 832) and identify trajectories of symptom change and predictors of trajectories. Method: Youth- and parent-reported anxiety symptoms were combined using item-response theory models. Growth mixture modeling assessed for trajectories of treatment ...
Integrative data analysis of self‐efficacy in 4 clinical trials for
Alcoholism: Clinical and Experimental Research. Volume 45, Issue 11 p. 2347-2356. ... Department of Psychology, University of New Mexico, Albuquerque, NM, USA. ... Integrative data analysis, where data from multiple studies are combined for analyses, can be used to synthesize analyses across multiple alcohol treatment trials by creating a ...
Integrative Data Analysis and the Study of Global Health
This chapter introduces Integrative Data Analysis (IDA), a novel framework for simultaneous analysis of individual-level data pooled from multiple studies, and describes the IDA methodological framework, emphasizing unique issues in measurement harmonization and hypothesis testing. In this chapter, we introduce Integrative Data Analysis (IDA) for use in the field of Global Health. IDA is a ...
Integrative data analysis of self-efficacy in four clinical trials for
Integrative data analysis, where data from multiple studies are combined for analyses, can be used to synthesize analyses across multiple alcohol treatment trials by creating a commensurate measure and controlling for differential item functioning (DIF) to determine if alcohol treatments improve self-efficacy. ... IDA permits clinical research ...
Research Synthesis: Meta-analysis and Integrative Data Analysis
Meta-analysis and integrative data analysis are research synthesis methods that aim to provide large-scale evidence by pooling data from multiple independently conducted studies. Dr. ... An individual participant-level data meta-analysis. Alcoholism: Clinical and Experimental Research, 39(5), ... Psychology of Addictive Behaviors, 29(1), 34-48.
The ADHD teen integrative data analysis longitudinal (TIDAL) dataset
The Attention Deficit Hyperactivity Disorder (ADHD) Teen Integrative Data Analysis Longitudinal (TIDAL) dataset integrates data from four randomized trials. Participants with ADHD (N = 854; 72.5% male, 92.5% racial/ethnic minority, ages 10-17) were assessed three times across 12 months. Data includes parent, self, and teacher ratings, observations, and school records.
Advantages of Integrative Data Analysis for Developmental Research
An integrative data analysis combined observations over ages 2 through 17 from two longitudinal studies of children of alcoholic parents and matched controls recruited from the community found no support for time-varying effects, but proximal effects of mothers' alcohol-related consequences on child-reported internalizing symptoms were found and distal effects of mother and father alcoholism ...

Integrative Data Analysis in Clinical Psychology Research

Related Papers

RELATED PAPERS

RELATED TOPICS

20 Integrative Data Analysis from a Unifying Research Synthesis Perspective

Signed in as

Personal account

Institutional access

IP based access

Sign in through your institution

Sign in with a library card

Society Members

Sign in through society site

Sign in using a personal account

Viewing your signed in accounts

Signed in but can't access content

This Feature Is Available To Subscribers Only

Introduction to the Special Issue on Innovations and Applications of Integrative Data Analysis (IDA) and Related Data Harmonization Procedures in Prevention Science

Cite this article

Access this article

Similar content being viewed by others

Advancing Intervention and Prevention Research for Behavioral Health Problems Through Data Synthesis

Retrospective Psychometrics and Effect Heterogeneity in Integrated Data Analysis: Commentary on the Special Issue

Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials

Acknowledgements

Author information

Corresponding author

Ethics declarations

Consent to Participate

Additional information

Rights and permissions

About this article

Share this article

Trajectories and predictors of response in youth anxiety CBT: Integrative data analysis

Grants and funding

Integrative data analysis of self-efficacy in four clinical trials for alcohol use disorder

Kelsey N. Serier

Katie Witkiewitz

Conclusions:

Introduction

Participants

Alcohol Abstinence Self-Efficacy Scale (AASE; DiClemente et al., 1994 ).

Drug Taking Confidence Questionnaire (DTCQ; Annis et al., 1997 ).

Data Analysis

Identification and Harmonization of Similar Items.

Self-Efficacy Factor Structure.

Moderated Nonlinear Factor Analysis.

Self-Efficacy Latent Scores and Change in Self-Efficacy over Time.

Identification of Similar Items

Self-Efficacy Factor Structure

Moderated Nonlinear Factor Analysis

Self-Efficacy Latent Scores and Change in Self-Efficacy over Time

EAP Self-Efficacy Scores over Time.

Observed Self-Efficacy Scores over Time.

Limitations

Conclusions

Supplementary Material

Research Synthesis: Meta-analysis and Integrative Data Analysis

Quick links

Connect with us

Social media

The ADHD teen integrative data analysis longitudinal (TIDAL) dataset: background, methodology, and aims

Conclusions

Overview of the ADHD TIDAL dataset

Construction and content

Study designs

Heterogeneity

Demographic Characteristics

Treatment conditions

STAND-group

No treatment

Measures and available data

Disruptive behavior disorders rating scale

DSM-5 ADHD rating scale

Child behavior checklist and youth self report

Observed organization skills

Disciplinary incidents

IQ and academic achievement

Diagnostic interview schedule for children (DISC)

Adult ADHD self-report scale