Why Mixed Methods?

ADDING FOCUS GROUPS DOES NOT A MIXED METHODS STUDY MAKE.

Our experience has shown that an alternative to “qualitative tokenism” is to consider whether qualitative methods can help us understand individuals’ experiences and the meanings they attribute to them without trying to “fit” them into previously defined constructs.[1] By allowing individuals to describe their own reality, we learn how people experience and define their distress differently than researchers, or how an intervention is adapted to specific contexts. Differing perspectives across patient cultural groups, families, clinician experiences and interests that act to affect how people respond to attempts to improve care demand that we consider new ways to carry out services research. Often investigators are only trained and comfortable with a single methodology, so the approaches to problems and the types of questions asked are constrained by training limited to one “world view”.[2] Reliance on a single methodological stance is no longer tenable in an increasingly complex multicultural and interdisciplinary context, or in the translation and dissemination of population and behavioral research to broader applications and conditions.

THE TRANSLATION CONTINUUM: FROM INTERVENTION DEVELOPMENT TO DISSEMINATION AND MAINTENANCE

Mixed methods fit into implementation science across the translational continuum.[3,4] We recognize that intervention development may require initial phases under optimum conditions (efficacy or effectiveness studies), but to facilitate translation, development of interventions may best be carried out with the participation of the settings in which the intervention is to be delivered.[5] Participants in the MMRTP may be engaged in intervention research, a key arena for mixed methods applications in mental and behavioral research. Mixed methods qualitative data can augment an RCT or intervention design by gathering exploratory data before, during, or after a trial.[6] These applications are becoming increasingly well-known, and published mixed methods studies provide insight into qualitatively exploring with subjects before an intervention trial or explaining the outcomes of the trial with qualitative data follow up.[7] Implementation, dissemination, and maintenance of interventions deserve attention as well (Figure 1, adapted from[8,9]). Implementation is defined as “a set of activities designed to put into practice” an intervention program[10] (e.g., the core intervention components of the chronic care model include a patient registry and self-management strategies[11]). A common confusion in the arena of implementation science concerns the failure to differentiate between the core intervention components (usually studied in efficacy or effectiveness studies under controlled conditions unlike the conditions in which the intervention will be employed) and the core implementation components (the elements needed to implement the core intervention components, such as coaching[10]). Efficacious and effective interventions only affect patient outcomes in the context of good implementation -- both an efficacious intervention and good implementation are needed. “Diffusion” of interventions is something of a misnomer because users adapt interventions to their local context.[12] Failure to adapt treatments in ways that increase systems, organizational, and community “fit” may explain why research-proven interventions are often not disseminated or maintained.[13] In multilevel behavioral interventions, investigators need to understand “contextually nested systems” for a more complete understanding of how implementation is occurring.[14] Following the IOM systems approach, investigators can focus on the “whole picture and not just a single element” typical of solely quantitatively- or qualitatively-driven investigations.[15] Mixed methods contribute to implementation research by providing information about what core components are adopted and by whom (quantitative methods) and why (qualitative methods) to develop interventions adapted to the practice environment and sustainable.

We expect that participants will be working on translational research projects that bring qualitative, contextualized data into aspects of the translational continuum (see Figure 1). Discussions about fitting qualitative data into this continuum have surfaced in 22 studies addressing practices that effectively prevent or treat mental health problems.[16] This review highlighted the importance of qualitative research to a) depict process b) capture the words of personal experience c) examine fluid and dynamic processes d) factor in patients’ views. In the translation process, mixed methods designs come into the process between effectiveness studies and implementation research.[5] Mixed methods can facilitate movement of interventions along the translational continuum because the intervention or implementation components are designed while eliciting the experiences of the persons in the contexts in which the interventions are to be carried out.

DISSEMINATION AND IMPLEMENTATION

Dissemination and implementation has received recent attention and is an arena some scholars will work in.[17] Much of patient-centered and community-centered research is actually evaluation of current practices, protocols and policies: How well do current interventions work, for whom do they fail, what adaptations are made when an intervention is deployed in real world settings, how can they be improved? Clearly the stories behind the numbers are critically important in making these assessments and testing alternative approaches. The use of mixed methods in areas such as clinical or quality issues, healthcare performance, health care decisions, and the implementation of innovations has been documented in several fields.[18,19] Community-based research, involving key stakeholders in investigations in mental and behavioral research provides a culturally sensitive model for data collection and responsiveness to the diversity found in many communities.[20]

Mixed methods are particularly advantageous in developing new and better environmental and contextual (or social-ecological, sociocultural) interventions, gearing investigations to community-identified cultural and ethnic factors that influence illness presentation and acceptance of treatment.[21] As an important approach to adapt interventions to communities or cultures, mixed methods research has increasingly been used to address mental and behavioral health issues across the lifespan. Mixed methods has applicability to understand suicide,[22] functional status of individuals in humanitarian crises,[23] caregiving to family members affected by HIV and AIDS,[24] depression status among older patients,[25] coping with trauma,[26] and health disparities,[27] among many others. Mixed methods enables investigators conceptually and analytically to integrate qualitative research and qualitative data (e.g., semi-structured interviews, observations, focus groups) with traditional epidemiological and quantitative methods of research to facilitate translation. Mixed methods help understand, not just whether an intervention works, but how, why, and for whom.

PATIENT-CENTERED CARE AND COMMUNITY-CENTERED SERVICES

The public health impact of interventions is often not realized because the stakeholders have not been involved in the development of the preventive strategies or interventions intended for them.[28] Reports from NIH, including the Roadmap initiative, have emphasized the need to restructure the research enterprise for more relevance to consumers. The Institute of Medicine Quality Chasm reports called attention to the need for “outside the box” thinking related to re-design of health care, including a strong focus on preferences and patient (person) –centered care, and evidence-based decision-making.[29,30] A public health model places more emphasis on diversity in samples, treatment that involves mixed, not “pure” modalities, assessing outcomes of importance to stakeholders, and collaboration of members of the community (individuals, organizations, and policymakers) early in the services and intervention development process.

Because mixed methods research designs place high value on the stories behind the numbers – both in exploratory designs where the experiences and insights of the community under study inform the quantitative investigation, and in explanatory designs where they illuminate the quantitative data – mixed methods are especially attractive to community partners who are more interested in improving practice and outcomes than in advancing knowledge. As such, mixed methods provide a valuable bridge between two largely separate and sometimes antagonistic groups, both of whom need effective research partnerships. We have found that project-oriented mixed methods research workshops - structured to provide the opportunity to design research proposals - provide an excellent platform for training. Such workshops, combining didactic sessions on principles and models of mixed methods research and facilitated, structured breakout groups focusing on specific research designs, can be proactive and effective starting points for helping researchers and community partners find one another and embark on projects together. For community-based participatory research to reach its full potential faculty must be mentored to obtain skills and experiences to be successful in carrying out such research.[31]

The Federal Coordinating Council for Comparative Effectiveness Research defined CER broadly, asserting that it is patient-centered, “real world” research that can help patients, clinicians, and other decision makers assess “the relative benefits and harms of strategies to prevent, diagnose, treat, manage, or monitor health conditions and the systems in which they are made.”[32,33] A misleading assumption is that all participants respond to an intervention in the same way, no matter the context. In the real world, people come to the treatment with preconceived notions about what is wrong and what to do about it. Tension between “patient-centeredness” and application of an “evidence base” -- between incorporating context and general applicability of evidence (“generalizability”) – keeps treatments that might be beneficial (e.g., depression treatment) from getting to people who could benefit (e.g., persons with medical comorbidity such as diabetes who are sometimes poorly adherent to lifestyle changes and medical regimens[34,35]). Mixed methods strategies are essential to understanding what must be adapted in evidence-based models to make them successful with different patients and in different communities.

Mixed methods hold promise to advance the strategic goals of several institutes to develop interventions that are culturally relevant, to translate interventions into practice, and to carry out research that leads to improved patient- and community-centered care (see Table 1 for examples). Themes such as reducing health disparities, increasing the reach and acceptability of interventions, and enhancing the patient-centeredness of care are common to the strategic plans of several institutes calling for the integration of behavioral and social science research with clinical investigation (http://report.nih.gov/strategicplans/). For example, strategic plans of NHLBI (Goal 3, to translate research into practice), NIMH (Strategic Objective 3, to incorporate diverse needs and circumstances of people with mental illnesses), NCCAM (Goal 3, to increase understanding of “real world” patterns and outcomes, and what is important to patients), and NIA (Research Goal E, to develop interventions to improve culturally appropriate health care delivery), all emphasize the need for understanding the sociocultural context of the patient, family, and providers to achieve mission goals. Statistical methods may be employed to study adherence patterns or to estimate treatment effects in the face of non-adherence to treatment regimens, but to account for preferences, to consider alternative concepts of disorder and etiology, and to incorporate community perspectives into how an intervention or service is configured -- issues that affect conditions as varied as use of cancer screening strategies, glaucoma detection and treatment, lifestyle changes in diet and physical activity, and accepting treatment for mental disorders -- investigators are recognizing that mixed methods can be a powerful tool. The value of mixed methods were highlighted in a monograph from APHA, “Improving Aging and Public Health Research: Qualitative and Mixed Methods” edited by Dr. Leslie Curry (a consultant for this program).^[38] While quantitative approaches can characterize and measure patient outcomes, use of mixed methods can enhance quantitative analyses to identify unmeasured factors that might be associated with poor response to interventions,^[43] or factors that account for people who do not “fit the model,” that is outliers,44 or provide clues to how interventions can incorporate the diverse needs and circumstances of people (“personalized interventions”).^[38] Mixed methods increase the ability to address research questions about what puts people at risk in the first place (“at risk for risk”).^[45] The complexity of the problems faced by public health needs the perspectives of multiple disciplines.^[46,47] With Scholars working across the translation continuum, incorporating both quantitative and qualitative data that are integrated in mixed methods designs, the proposed training program supports the NIH’s scientific mission “to foster fundamental creative discoveries, innovative research strategies, and their applications as a basis for ultimately protecting and improving health.”

<sub>1. Patton M. Qualitative Evaluation and Research Methods. 2 ed. Newbury Park, California: Sage; 1990.
2. Tashakkori A, Teddlie C, eds. Handbook of Mixed Methods in Social & Behavioral Research. Thousand Oaks, California: Sage Publications; 2003.
3. Glasgow RE, Emmons KM. How can we increase translation of research into practice? Types of evidence needed. Annual review of public health. 2007;28:413-433.
4. Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. National Institutes of Health Approaches to Dissemination and Implementation Science: Current and Future Directions. American Journal of Public Health. 2012;102(7):1274-1281.
5. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care. 2012;50:217-226.
6. Creswell JW, Plano Clark VL. Designing and Conducting Mixed Methods Research. 2nd ed. Washington, D.C.: Sage Publications; 2011.
7. Farquhar MC, Ewing G, Booth S. Using mixed methods to develop and evaluate complex interventions in palliative care research. Palliative medicine. 2011;25(8):748-757.
8. Woolf SH. The meaning of translational research and why it matters. JAMA. 2008;299(2):211-213.
9. Wilson KM, Brady TJ, Lesesne C. An organizing framework for translation in public health: the Knowledge to Action Framework. Prev Chronic Dis. 2011;8(2):A46.
10. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation Research: A Synthesis of the Literature. Tampa, Florida: University of South Florida;2005.
11. Wagner EH, Austin BT, Von Korff M. Organizing care for patients with chronic illness. Milbank Q. 1996;74(4):511-544.
12. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. DIffusion of innovations in service organizations: Systematic review and recommendations. Milbank Q. 2004;82:581-629.
13. Hoagwood, K., Burns BJ, Weiss JR. A profitable conjunction: From science to service in children's mental health. In: Burns BJ, Hoagwood K, eds. Community Treatment for Youth: Evidence-based interventions for severe emotional and behavioral disorders. New York: Oxford University Press; 2002:327-338.
14. Dziak JJ, Nahum-Shani I, Collins LM. Multilevel factorial experiments for developing behavioral interventions: Power, sample size, and resource considerations. Psychological Methods. 2012;17(2):153-175.
15. Institute of Medicine. Accelerating progress in obesity prevention: Solving the weight of the nation. Washington, DC: National Academies Press; 2012.
16. Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Administration and policy in mental health. 2011;38(1):44-53.
17. Palinkas LA, Soydan H. Mixed methods research. In: Palinkas LA, Soydan H, eds. Translation and Implementation of Evidence-Based Practice. New York: Oxford University Press; 2012:125-149.
18. Wisdom JP, Cavaleri MA, Onwuegbuzie AJ, Green CA. Methodological Reporting in Qualitative, Quantitative, and Mixed Methods Health Services Research Articles. Health Services Research. 2012;47(2):721-745.
19. Curry LA, Krumholz HM, O'Cathain A, Plano Clark VL, Cherlin E, Bradley EH. Mixed methods in biomedical and health services research. Circ Cardiovasc Qual Outcomes. 2013;6:119-123.
20. Israel BA, Eng E, Schulz AJ, Parker EA, eds. Methods for Community-Based Participatory Research for Health. Somerset, New Jersey: Jossey-Bass; 2012.
21. Landsverk J, Brown CH, Chamberlain P, et al. Design and Analysis in Dissemination and Implementation Research. In: Brownson RC, Colditz GA, Proctor EK, eds. Dissemination and Implementation Research in Health: Translating Science to Practice. New York: Oxford University Press; 2012:225-260.
22. Ting L, Jacobson JM, Sanders S. Available supports and coping behaviors of mental health social workers following fatal and nonfatal client suicidal behavior. Social Work. 2008;53:211-221.
23. Bolton P, Bass J, Murray L, Lee K, Weiss W, McDonnell SM. Expanding the scope of humanitarian program evaluation. Prehosp Disaster Med. 2007;22:390-395.
24. Petros SG. Use of a mixed methods approach to investigate the support needs of older caregivers to family members affected by HIV and AIDS in South Africa. Journal of Mixed Methods Research. 2011;6:275-293.
25. Wittink MN, Barg FK, Gallo JJ. Unwritten rules of talking to doctors about depression: integrating qualitative and quantitative methods. Annals of Family Medicine. 2006;4(4):302-309.
26. Banyard VL, Williams LM. Women’s voices on recovery: A multi-method study of the complexity of recovery from child sexual abuse. Child Abuse & Neglect. 2007;31:275-290.
27. Stewart M, Makwarimba E, Barnfather A, Letourneau N, Neufeld A. Researching reducing health disparities: Mixed-methods approaches. Social Science & Medicine. 2008;66:1406-1417.
28. Anderson NB, Bulataa RA, Cohen B, eds. Critical Perspectives on Racial and Ethnic Differences in Health in Late Life. Washington, D.C.: National Academies Press; 2004.
29. Committee on Crossing the Quality Chasm: Adaptation to Mental Health and Addictive Disorders. Improving the Quality of Care for Mental and Substance-Use Conditions: Quality Chasm Series. Washington D.C.: National Academies Press; 2006.
30. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.
31. Ahmed SM, Beck B, Maurana CA, Newton G. Overcoming barriers to effective community-based participatory research in US medical schools. Education for Health. 2004;17:141-151.
32. Congressional Budget Office of the Congress of the United States. Research on the Comparative Effectiveness of Medical Treatments. 2007.
33. National Institutes of Health. National Institutes of Health. Research Portfolio Online Reporting Tools (RePORT): NIH RePORTER,.
34. Ciechanowski PS, Katon WJ, Russo JE. Depression and diabetes: impact of depressive symptoms on adherence, function, and costs. Arch Intern Med. 2000;160(21):3278-3285.
35. Gonzalez JS, Peyrot M, McCarl LA, et al. Depression and diabetes treatment nonadherence: a meta-analysis. Diabetes Care. 2008;31(12):2398-2403.
36. Puschel K, Thompson B. Mammogram screening in Chile: using mixed methods to implement health policy planning at the primary care level. Breast. 2011;20 Suppl 2:S40-45.
37. Greysen SR, Allen R, Lucas GI, Wang EA, Rosenthal MS. Understanding transitions in care from hospital to homeless shelter: a mixed-methods, community-based participatory approach. J Gen Intern Med. 2012;27:1484-1491.
38. Curry L, Shield R, Wetle T, eds. Improving Aging and Public Health Research: Qualitative and Mixed Methods. Washington, D.C.: American Public Health Association; 2006.
39. Lipsky S, Cristofalo M, Reed S, Caetano R, Roy-Byrne P. Racial and ethnic disparities in police-reported intimate partner violence perpetration: a mixed methods approach. J Interpers Violence. 2012;27:2144-2162.
40. Rosen RK, Morrow KM, Carballo-Diéguez A, et al. Acceptability of tenofovir gel as a vaginal microbicide among women in a phase I trial: a mixed-methods study. J Women's Health. 2008;17:383-392.
41. Curran GM, Bauer M, Mittman B, Pyne JM, C. S. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50:217-226.
42. Plano Clark VL, Schumacher K, West C, et al. Practices for embedding an interpretive qualitative approach within a randomized clinical trial. Journal of Mixed Methods Research. 2013;7:219-242.
43. Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q. 2004;82(4):661-687.
44. Kawachi I, Berkman LF. Neighborhoods and Health. Oxford: Oxford University Press; 2003.
45. Glass TA, McAtee M. Behavioral science at the crossroads in public health: Extending horizons, envisioning the future. Social Science & Medicine. 2006;62:1650-1671.
46. Committee on Future Directions for Behavioral and Social Sciences Research at the National Institutes of Health. New Horizons in Health: An Integrative Approach. Washington D.C.: National Academies Press; 2001.
47. Committee on Facilitating Interdisciplinary Research. Facilitating Interdisciplinary Research. Washington, D.C.: National Academies Press; 2004.</sub>