Systematic Review

Susan M. Wilczynski, in A Practical Guide to Finding Treatments That Work for People with Autism, 2017

Systematic reviews require a careful analysis of the quality, quantity, and consistency of research findings (Moher, Liberati, Tetzlaff, Altman, & The PRISMA Group, 2009; Slocum et al., 2012). The process of initiating a systematic review typically begins with a team of experts who are motivated to answer one of two types of questions. The experts may ask:

1.

“Is a given treatment effective (The term “efficacious” is often used to describe treatments that are effective under research conditions and “effective” is used for treatments that work in real-world settings. In this practitioner’s guide, the term “effective” is used to describe both type of treatments that work.)?” (Method #1). In this case the experts examine the evidence of all populations with respect to a single treatment (e.g., functional communication training (FCT) or picture schedules).

2.

“Are there any effective treatments for a given population (e.g., Autism Spectrum Disorder)?” (Method #2). In this case the experts examine the evidence regarding all treatments that have been studied for the population.

Read full chapter
URL: https://www.sciencedirect.com/science/article/pii/B9780128094808000017

Neurological Rehabilitation

Pam Enderby, in Handbook of Clinical Neurology, 2013

Stable dysarthria

A systematic review and a case series investigated speech and language therapy for patients with stable dysarthria, that is, dysarthria associated with nonprogressive conditions.

The systematic review (Palmer and Enderby, 2007), which included 23 research studies investigating the effect of different speech therapy treatments where participants had longstanding nonprogressive dysarthria as a result of traumatic brain injury, following stroke, or cerebral palsy, was unable to draw conclusions about the efficacy of the different treatments for stable dysarthria as all of the studies incorporated small numbers of subjects and different treatment techniques. The authors summarized the different treatment techniques and placed them into categories targeting rate, resonance, oromotor, articulation, and prosody, including compensatory strategies and treatment programs.

Read full chapter
URL: https://www.sciencedirect.com/science/article/pii/B9780444529015000228

Meta-Analysis of Clinical Trials in Bipolar Disorder

John R. Geddes, in Clinical Trial Design Challenges in Mood Disorders, 2015

Systematic reviews provide reliable syntheses of the evidence on efficacy and safety for interventions in bipolar disorder. Pooling data between studies using meta-analysis can produce precise estimates of treatment effects. Meta-analysis can also identify sources of variation between studies and explore the reasons for these. In the clinical area of bipolar disorder, meta-analysis has been used in many areas in which other study designs are not feasible including unbiased estimation of the protective effect of lithium on risk of suicide. More recent developments in research synthesis include network, or multiple treatments, meta-analysis, which allows the estimation of the comparative efficacy of multiple treatments by combining direct and indirect evidence. Overall, the application of meta-analysis in bipolar disorder has been particularly important for the appropriate establishment of the place of lithium in management.

Read full chapter
URL: https://www.sciencedirect.com/science/article/pii/B9780124051706000026

COMT Genotypes, Cannabis Use, and Psychosis: Gene-Environment Interaction Evidence from Human Populations, and Its Methodological Concerns

M. Fatjó-Vilas, ... L. Fañanás, in Handbook of Cannabis and Related Pathologies, 2017

Abstract

Systematic reviews of prospective studies have consistently evidenced the role of cannabis use as a risk factor for the emergence of psychosis. However, as expected in multifactorial complex diseases, cannabis use is not sufficient or necessary to cause schizophrenia or related disorders The sensitivity to the cannabis effects is proposed to be mediated by the genetic variability observed in human populations for the Catecol-O-methyltransferase gene (COMT), one of several enzymes that degrade catecholamines, such as, dopamine. The Dunedin Multidisciplinary Health and Development Study reported for the first time that the relationship between adolescent cannabis use and the risk of developing psychotic disorders is modulated by COMT Val158Met genotype. These results have triggered several other studies, however, only some have replicated the interaction, while others have failed or have offered controversial results. Environmental factors, other genetic interactions and several methodological concerns and challenges are discussed in the present chapter.

Read full chapter
URL: https://www.sciencedirect.com/science/article/pii/B9780128007563000314

The Role of Comparative Effectiveness Research

Joe V. Selby, ... Jean R. Slutsky, in Principles and Practice of Clinical Research (Fourth Edition), 2018

Evidence Synthesis in CER

To this point, this chapter has focused largely on primary research approaches in CER, in which a research team identifies a question, selects a study population, comparators and study design, conducts the study and reports its findings. In medical and clinical science, however, one study rarely produces results that are sufficiently definitive to change practice. In most cases, knowledge develops through a series of experiments or observations that cumulatively impact our shared understanding.

Secondary research, or evidence synthesis, is a body of methods that has emerged as part of evidence-based medicine and aims to extend knowledge through aggregating results or data from multiple individual studies. It seeks to determine what is known in a field of inquiry across all the available, relevant primary research studies and to estimate the variability or consistency of the evidence. These tools allow summary of “what we know” (consistent conclusions) and “how surely we know it” (i.e., confidence that the conclusions are valid, precise, and unlikely to change with future research) to most appropriately inform decision-makers in health-care policy and practice or determine the needs for future research.66 As the methodology of evidence synthesis, including systematic reviews and other approaches, has developed, “secondary” research has gained a prominent and influential role in CER, health-care policy and clinical practice worldwide, especially as a knowledge foundation for clinical guidelines.

There are several approaches under the umbrella of evidence synthesis. The most well-known of these, systematic reviews, was promoted in the early 1990s as an antidote to the selective citation of findings in support of an expert's opinion in a given clinical area.67,68 Systematic reviews have gained acceptance among decision-makers as an approach that uses comprehensive, rigorous, explicit, and ostensibly reproducible methods and includes critical appraisal of each study's design and conduct. As defined by the IOM, systematic reviews are “essential for clinicians who strive to integrate research findings into their daily practices” and are thus a critical component of trustworthy clinical practice guidelines.69

The production of systematic reviews is currently supported by the international Cochrane Collaboration, the AHRQ Evidence-based Practice Centers, and many others. Systematic reviews are distinguished from narrative reviews and characterized by clearly specified, objective methods for locating, critically appraising, summarizing, and reporting all research relevant to a particular question. In 2011, the IOM specified 21 standards with 82 performance elements for conducting systematic reviews of CER; these standards were intended to ensure that systematic reviews are objective, transparent, and scientifically valid.69 PCORI has adopted the IOM standards almost entirely into PCORI's Methodology Standards focusing on the standards for evidence synthesis (see Text Box).

Standards for Evidence Synthesis

1.

Initiate a team with appropriate expertise and experience to conduct the systematic review and ensure user and stakeholder input into the systematic review design and conduct, while appropriately managing conflicts of interest in all participants.

2.

Formulate the systematic review topic, develop, and peer-review the review protocol, and publish the final protocol with timely amendments as warranted.

3.

Conduct and document a comprehensive, systematic search for evidence, with attention to addressing potential sources of bias in research results reporting.

4.

For individual studies:

a.

Assess and document assessment of individual studies for inclusion/exclusion according to protocol; and

b.

Conduct and document critical appraisal of individual studies for bias, relevance, and fidelity of interventions using prespecified criteria.

5.

Use standard and rigorous data collection and management approaches.

6.

Synthesize the body of evidence qualitatively and, if warranted, quantitatively, using prespecified methods.

7.

Evaluate the body of evidence on characteristics-related overall quality and confidence in the estimates of effect on prespecified outcomes.

8.

Report the results using a structured format, peer-review the draft report (including public comment period), and publish the final report to allow free public access.

These standards reflect current scientific consensus and are likely to be supplemented or revised periodically as major producers of systematic reviews, such as the Cochrane, AHRQ, International Health Technology Assessment (HTA), and others, continue to work with decision-makers and conduct additional empirical studies of the impact of these standards on the production of unbiased, relevant systematic reviews. New standards also will be developed and current standards revised through further empirical methods development for analytic approaches including mixed treatment comparisons, network meta-analysis, and individual patient data meta-analysis (see Chapter 22).

As with primary studies, systematic reviews were initially focused mostly on questions of efficacy rather than comparative effectiveness. However, the principles and methods of systematic review apply equally well for synthesizing results from CER studies. Care must be taken to assure the equivalence of the comparisons being synthesized across studies. When the same (or reasonably similar) treatment-comparison contrasts are available in a series of studies, traditional metaanalytic techniques can be appropriate for combining results.

HTAs were early exemplars of the application of systematic review methods to comparing the benefits and harms of new technologies with existing alternatives. HTA research aims to provide evidence for decision-making on the incorporation of new health technologies and uses an interdisciplinary approach to evaluate the impact of these technologies in clinical practice.70 The “technologies” assessed include pharmaceuticals as well as devices, procedures, and other interventions. The United States had an Office of Technology Assessment from 1972 to 1995 that provided nonpartisan information on a wide range of scientific and technological issues, including health care, to Congress. Although the United States no longer has this national agency, HTA remains a robust enterprise internationally .71 Within the United States, HTA continues in some state initiatives, such as the Drug Effectiveness Review Project in Oregon, which synthesizes clinical evidence and was originally intended for drug-class decisions under Medicaid.72 HTA also has been supported by payers and health system decision-makers through such entities as the Blue Cross–Blue Shield Technology Evaluation Center.73 The AHRQ also funds some HTA through its Evidence-based Practice Centers, and these often are used by Medicare as part of their national coverage decisions.

As systematic review has gained traction, many innovations or adaptations have developed to address needs for more timely or robust summarized evidence that is applicable to and can comprehensibly inform decision-making across a range of contexts. In many cases, there are not multiple CER studies featuring direct comparisons of two or more treatments. Indirect comparisons represent inferences about treatment A versus treatment B through synthesizing results of studies that do not directly compare the two, such as by comparing results in studies of A versus no treatment and B versus no treatment. A number of caveats apply when making indirect comparisons, but quantitative methods known as Mixed Treatment Comparisons have been developed to address the challenges of conducting statistically valid syntheses based on indirect comparisons. One of the better known methods, network meta-analysis, is increasingly applied in protocols and reports from CER reviews.74–76 Network meta-analysis leverages both the direct and indirect evidence available for comparisons of two or more interventions; this methodology maintains as much evidence as possible when conducting a systematic review across trials with a common comparator.77 There are multiple caveats that must be kept in mind, including the observational nature of the indirect comparisons.78 Nonetheless, this method holds promise, particularly when ideal evidence is not available to address urgent decision-making needs.

Most metaanalyses and systematic reviews are based on aggregate data, i.e., the results from multiple studies. These data are readily accessed from the published medical literature with caveats to protect against publication bias favoring overrepresentation of positive findings in published results.79 Despite their relative accessibility, aggregate data are not as robust as individual participating data for synthesizing research results. Individual participant data (IPD) meta-analysis has been called the gold standard since it allows for characterization of results according to individual participant characteristics (such as age, sex, disease risk, or comorbidity), which cannot be adequately investigated using aggregated study results. IPD meta-analysis is a particularly powerful technique for addressing heterogeneous treatment effects and for targeting treatments to those most likely to receive benefit (or least likely to be harmed). IPD meta-analysis has not been applied as often as might be expected for a “gold standard” research technique, largely due to the considerable challenges in gaining access to original trial data. However, application of this method is growing and should be expected to accelerate further as the ethos of data sharing and open science gain momentum.79–81

Synthesis of existing evidence is always limited by the scope and rigor of the existing evidence, and limitations are most severe for newly emerging, often expensive treatments. There may be little or no evidence on comparisons with existing, less expensive alternatives, due to in part to a lack of requirements or incentives for this type of research prior to approval. In such cases, traditional methods of evidence synthesis will not be able to quickly support decision-makers' needs, although attempts at synthesis may be useful for identifying the relevant gaps in the CER literature.82

There is growing advocacy and appreciation for a broader range of approaches that can be considered part of the family of evidence synthesis methods. One such approach is decision modeling. In the absence of empiric evidence on important comparative effectiveness questions, decision-makers may determine that mathematical models incorporating estimates from available empiric evidence and reasonable assumptions for missing parameters provide a worthwhile substitute. Such models can estimate the likely comparative effectiveness research results needed by decision-makers. In some instances, these models extend to incorporate cost-effectiveness analyses, to more conveniently assess the relative value of various alternatives. Cost-effectiveness models typically incorporate outcomes metrics that can facilitate estimation of relative value across diseases or conditions. These include life-years saved, quality- or disability-adjusted life-years saved and others. While the advantages and disadvantages of such approaches are beyond this chapter, it is important to acknowledge that these are active and important areas that those interested in policy-relevant CER may be called upon to understand and address.

Read full chapter
URL: https://www.sciencedirect.com/science/article/pii/B9780128499054000198

Creativity Training

J.J. Caughron, ... M.D. Mumford, in Encyclopedia of Creativity (Second Edition), 2011

Conclusion

Systematic reviews of literature on creativity training demonstrated that using training to improve creativity can be highly effective. Using a cognitive approach in the development of creativity training was the only approach that consistently contributed to the effectiveness of creativity training. Specifically, training that emphasizes idea generation, problem finding, conceptual combination, and idea generation were particularly important for promoting creativity.

In addition to characteristics that promote a systematic, cognitive approach to creativity, scholars have also identified delivery methods that appear to promote creativity. Making sure training materials are challenging, include real world, domain-specific exercises, and cover well-researched cognitive models of creativity in an in-depth fashion should promote trainee creativity. Training should involve some lecture-based components along with multiple opportunities for practice by trainees. Naturally, training that involves an in-depth, theoretically based consideration of creativity along with multiple opportunities for trainee practice on challenging, realistic, domain-specific exercises is likely to be somewhat lengthy. In fact, lengthy training sessions have been found to be associated with higher levels of creativity.

When designing creativity training, incorporating elements that have been shown to promote trainee creativity that have been examined empirically and systematically is likely to enhance the effectiveness of creativity training. Investigating the use of techniques that are less structured like brainstorming or the use of imagery may be interesting, but should be avoided when designing creativity training courses, at least until the real utility of these techniques can be determined. It is our hope that this brief review of findings regarding creativity training has given the reader confidence that training to enhance creativity can, in fact, have notable effects. Additionally, we hope this effort has also informed the reader of specific topics and techniques that when used in creativity training courses can be effective in promoting creativity generally as well as specific aspects of creativity such as divergent thinking, problem-solving, performance, and attitudes towards creative efforts.

Read full chapter
URL: https://www.sciencedirect.com/science/article/pii/B9780123750389002260

Case Conceptualization and Treatment: Adults

Alan Carr, in Comprehensive Clinical Psychology (Second Edition), 2022

6.04.19.1 Children and Adolescents

Systematic reviews and meta-analyses support the effectiveness of FBPIs as an adjunct to medical care for children and adolescents with asthma (Brinkley et al., 2002), diabetes (Hilliard et al., 2016) and functional abdominal pain (Sprenger et al., 2011). In a systematic review of 20 studies, Brinkley et al. (2002) concluded that family-based interventions for asthma spanning up to 8 sessions, were more effective than individual therapy. They included psychoeducation to improve understanding of the condition, medication management and environmental trigger management; relaxation training to help young people reduce physiological arousal; skills training to increase adherence to asthma management programs; and conjoint family therapy sessions to empower family members to work together to manage asthma effectively. The adaption of Multisystemic Therapy for treating high-risk families of asthmatic adolescents is an important development in this field. In a randomized controlled trial, compared with family support, Multisystemic Therapy led to significant improvements in adolescents' asthma management and lung function, parental self-efficacy, and parental beliefs in the value of asthma-related positive parenting behaviors (Naar-King et al., 2014).

Systematic reviews and meta-analyses of the impact of FBPIs for adolescents with type 1 diabetes point to the following conclusions (Farrell et al., 2002; Hilliard et al., 2016; Hood et al., 2010; Lohan et al., 2015; Savage et al., 2010). Effective FBPIs spanning 2–6 months led to improvements in a range of domains including family members knowledge about diabetes and its management, adherence, glycemic control, hospitalization, quality of family relationships, and well-being of children and parents. Different types of programs were appropriate for families at different stages of the lifecycle. For families of young children newly diagnosed with diabetes, psychoeducational programs which helped families understand the condition and its management were particularly effective. Family-based behavioral programs, where parents rewarded youngsters for adhering to their diabetic regimes, were particularly effective with pre-adolescent children, whereas family-based communication and problem-solving skills training programs were particularly effective for families with adolescents, since these gave families skills for negotiating diabetic management issues in a manner appropriate for adolescence. Multisystemic Therapy adapted for families of adolescents with type 1 diabetes (Ellis et al., 2012) is particularly well supported.

Narrative reviews and meta-analyses show that cognitive behavioral family therapy is effective in alleviating recurrent functional abdominal pain, often associated with repeated school absence, and for which no biomedical cause is evident (Banez and Gallagher, 2006; Sprenger et al., 2011; Weydert et al., 2003). Such programs involve family psychoeducation about functional abdominal pain and its management, relaxation and coping skills training to help children manage abdominal pain which is often anxiety-based, and contingency management implemented by parents to motivate children to engage in normal daily routines, including school attendance.

Read full chapter
URL: https://www.sciencedirect.com/science/article/pii/B9780128186978001989

The World Café as a methodology for examining disability issues: Review and recommendations

Jennifer L. Bumble, Erik W. Carter, in International Review of Research in Developmental Disabilities, 2020

6 Conclusions

This systematic review explored how the World Café methodology has been employed to address issues important in the lives of people with disabilities and their families. We developed and applied a framework of quality indicators to available studies to examine the rigor of World Café research. Overall, articles varied widely in the clarity of their reporting and the rigor of their approaches. At the same time, the studies highlight the flexibility of the World Café as a methodology and the myriad ways that qualitative and quantitative approaches can be incorporated. We recommend that future research adopt relevant aspects of these quality indicators. Moreover, we encourage additional examination of the impact of these events on the outcomes of individuals with disabilities.

Read full chapter
URL: https://www.sciencedirect.com/science/article/pii/S2211609520300087

The Evidence Base for Health Information Exchange

William R. Hersh, ... Marian S. McDonagh, in Health Information Exchange, 2016

Comparison With Other Reviews

The present systematic review of HIE can be compared with two other systematic reviews of HIE: one by Rudin et al. [16] and another by Rahurkar et al. [17]. All three systematic reviews used generally similar approaches, with similar definitions of HIE and focus on studies of HIE impact, excluding system descriptions and simple case studies. The three reviews differed, however, in their scope and inclusiveness. Our review was the broadest in scope and the most inclusive in the search for evidence. In addition to patient and population health outcomes, economic, utilization process outcomes, and barriers and facilitators to implementation and use, our review also included studies concerned with use and usability of HIE. We also included studies beyond the United States and those reporting on public health and surveillance as well as exchange of administrative and financial information. The overall result is that we examined a more diverse and more inclusive collection of evidence, especially with respect to usability and use as well as assessing public health settings, although we came to largely similar conclusions. Rahurkar et al. performed a multivariate analysis that found study design was the only characteristic associated with finding a beneficial effect, with the most rigorous studies being less likely to report benefits of HIE [17].

Read full chapter
URL: https://www.sciencedirect.com/science/article/pii/B9780128031353000141

Health Psychology

Christel Hendrieckx, ... Frank J. Snoek, in Comprehensive Clinical Psychology (Second Edition), 2022

8.14.4.3.1 Depression

Systematic reviews of epidemiological studies have demonstrated an increased prevalence of depression in people with diabetes relative to the general population, with more women affected than men (Renn et al., 2011; Roy and Lloyd, 2012).

There are two main hypotheses explaining the observed association between diabetes and depression. First, depression is assumed to elevate diabetes risk (Musselman et al., 2003) by mechanisms that are not yet fully understood. In the case of type 2 diabetes there is evidence pointing at a causal role for unhealthy behaviors, elevated cortisol and inflammation; for type 1 diabetes, research into the role of serious life events and stress as precursors of diabetes is inconclusive (Sharif et al., 2018). Second, the “hardship hypothesis” that posits that depression results from chronic stress due to the burden of living with and managing diabetes (Egede and Ellis, 2010). This explanation concords with the observation that depression prevalence is also elevated in other chronic medical conditions. Research suggests that higher rates of depression found in diabetes are not attributable to confounding by diabetes-related somatic symptoms (Harding et al., 2019). Interestingly, statistical analyses of symptom profiles in large data sets, suggest different depression profiles, including a subtype that is closely linked to obesity and metabolic disturbances, referred to as immuno-metabolic depression (Lamers et al., 2020).

Read full chapter
URL: https://www.sciencedirect.com/science/article/pii/B9780128186978002065