Main findings: Present the key findings with a specific focus on theory building and testing, 15. If you have access to a journal via a society or association membership, please browse to your society journal, select an article to view, and follow the instructions in this box. Cochrane update: ‘Scoping the scope’ of a Cochrane review. SCIE Systematic mapping guidance. We extracted data on these 19 items. Strengths, limitations, and future research directions: b) Discuss the limitations, 17. Complex interventions: how ‘out of control’ can a randomised controlled trial be? Some authors referred to an interface (e.g., OVID, EBSCO), rather than a particular database, which made it difficult to evaluate in which database they had searched. Similarly, without theoretical or methodological justifications, realist reviews may also be suspected of offering a partial and potentially misleading account of the evidence. Abstracts and protocols (a protocol is a document that describes the plan for conducting a study, including purpose and methods) of planned or ongoing reviews that appeared to meet the inclusion criteria were included on a separate list, but data from these were not extracted. Disagreements were resolved by re-examination of the review and subsequent discussion. If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Interestingly, while realist reviews are easily identified as such through either the title or the abstract, three of the reviews included in our study failed to use the term realist in both instances, identifying instead as “explanatory review,” “theory-led narrative review,” or simply as “review.” Similarly, given realist review is a theory-driven approach, it was perplexing that most reviews lacked a rationale as to why the realist review was a suitable approach to follow. Realist review - a new method of systematic review designed for complex policy interventions Ray Pawson, Ph.D , Trisha Greenhalgh, M.D , Gill Harvey, Ph.D , and Kieran Walshe, Ph.D Journal of Health Services Research & Policy 2016 10 : 1_suppl , 21-34 A recurring element was a call for additional research, usually without specification of research directions. Please check you selected the correct society from the list and entered the user name and password you use to log in to your society website. With respect to languages searched, 7 had no limits (15 per cent), 21 limited the search to English, and 4 limited the search to English and one more language, either French or German. Bates, S., Clapton, J., & Coren, E. (2007). (2011). Discovery of Grounded Theory. Details of 71 Publications Included in This Study, Understanding the effectiveness of school-based interventions to prevent suicide: A realist review, Child and Adolescent Mental Health, 18(3), 131-139, Interventions to reduce the prevalence of female genital mutilation/cutting in African countries, A realist synthesis of controlled studies to determine the effectiveness of interventions to prevent genital cutting of girls, Paediatrics and International Child Health, 33(4), Large-system transformation in health care: A realist review, A realist review of art Therapy for clients with depression, Towards a framework for enhancing the performance of NHS boards: A synthesis of the evidence about board governance, board effectiveness and board development, Health Service Delivery Research, 1(6), 1-158. Members of _ can log in with their society credentials below, Journal of Health Services Research & Policy. It will not tell policy-makers or managers whether something works or not, but will provide the policy and practice community with the kind of rich, detailed and highly practical understanding of complex social interventions which is likely to be of much more use to them when planning and implementing programmes at a national, regional or local level. A typical description of the analysis and synthesis processes that we marked as inadequate is quoted below: Data synthesis was undertaken by [the main author], and synthesis results were regularly shared and discussed with the research team to ensure validity and consistency. Lastly, we note that only a third of the reviews mentioned involving stakeholders at any point in the review process. When necessary, we contacted the original authors to obtain further information. Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005). Sharing links are not available for this article. As such, a systematic realist literature review facilitates an analysis that is more explanatory in nature, discovers crosscut- Realist review (also known as realist synthesis) is particularly helpful for this task. The remaining eight reviews included primarily quantitative studies in addition to other types of documents, such as documents titled research papers or conceptual papers. Complying with the standard should overcome idiosyncratic reporting of how the documents are identified. Analysis and synthesis process: (a) Describe the analysis process in detail. By continuing to browse There is inadequate transparency about methods for selection, appraisal, analysis, and synthesis of studies. We have not extracted data from these protocols. Realist review - a new method of systematic review designed for complex policy interventions Ray Pawson, Ph.D , Trisha Greenhalgh, M.D , Gill Harvey, Ph.D , and Kieran Walshe, Ph.D Journal of Health Services Research & Policy 2016 10 : 1_suppl , 21-34 Realist review aims to answer “what works for whom in what circumstances, in what respects and how?” with respect to social intervention programs (Pawson et al., 2004, p. 3). RAMESES publication standards: Realist syntheses. Deriving from Aristotle’s tutelage at the feet of the great idealist Plato, realism has an important influence on education. Access to society journal content varies across our titles. The final section of the RAMESES Publication Standards is labeled Discussion and includes five items (Items 15-19). Nine reviews (20 per cent) explained how they searched for grey literature, such as performing searches in Google and examining government reports. Bravata, D. M., McDonald, K. M., Shojania, K. G., Sundaram. This approach was chosen because it is designed to synthesise evidence on social programmes and policies which are complex. Funding: (c) Provide details of any conflict of interest of the reviewers, Notes. Inter-reviewer agreement: An analysis of the degree to which agreement occurs when using tools for the appraisal, extraction and meta-synthesis of qualitative research findings. We found that the initial process of exploratory scoping of the literature was described in only 58 per cent of the reviews. A realist review and a call for place-based research, Human resource management interventions to improve health workers' performance in low and middle income countries: A realist review, Health Research Policy and Systems, 7(7), 1-13, Realist review and synthesis of retention studies for health workers in rural and remote areas, Interprofessional shared decision making in NICU: A mixed methods study [Dissertation], School of Nursing, University of Ottawa, Canada. Similar to what Dixon-Woods and colleagues (2007) found for qualitative evidence syntheses, there is some intransparency about how the data are analyzed and synthesized in realist reviews. Login failed. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Only seven of the reviews included in our study, all published in 2014 or 2015, cited and stated that they used the RAMESES publication standards. Lang, L. A., & Teich, S. T. (2014). International Journal of Behavioral Nutrition and Physical Activity, 8, 46, Improving organizational capacity to address health literacy in public health: A rapid realist review, Internet-based medical education: A realist Clapton, J., Rutter, D., & Sharif, N. (2009). Lastly, as all reviewers assessing published reports, we were limited by what is reported. Provide details on all the sources accessed for information in the review [d], 9. Norwegian Institute of Public Health, Oslo, NORWAY; When methods are not sufficiently justified or are poorly undertaken or reported, and the approach subsequently appears to exist in many variants, it becomes harder to understand what is meant when the term realist review is used. Evaluation, 8(3), 340-358. Such reviews take stock of the research available in a field of study, such that one can decide how to develop the field further (Arksey & O’Malley, 2005; Armstrong, Hall, Doyle, & Waters, 2011; Bates, Clapton, & Coren, 2007; Clapton, Rutter, D., & Sharif, 2009; Oakley, Gough, Oliver, & James, 2005; Shepherd et al., 2006). To ensure that the search was as comprehensive as possible, we applied neither methodology search filters nor language delimiters. In order to make the process of realist reviews rigorous and transparent, a set of 19 publication standards has been developed by a team of scholars (i.e., the RAMESES [Realist and Meta-narrative Evidence Syntheses: Evolving Standards] publication standards, Wong et al., 2013). Realist reviews follow most of the same standard steps as systematic reviews (Pawson et al., 2004): Pawson and colleagues (2004) stipulate that the process should be equally rigorous and transparent (i.e., auditable), where every judgment is written down. ESRC Centre for Evidence-Based Policy and Practice. The politics of evidence and methodology: Lessons from the EPPI-Centre. NHS Modernisation Agency Clinical Governance Support Team. It is possible that more development or clarification of the standards is needed. The Method section of the RAMESES Publication Standards includes seven items (Items 5-11). Furthermore, one third of the reviews in our study made no reference to appraisal of the documents included, and those that did, generally ignored accounting for appraisal results in the synthesis. In realist review methodology, exploring interventions (in this case delegation) as a multifaceted, social process enabled the authors to account for context, mechanisms, and outcomes associated with the evidence. It was also challenging to designate some of the data points. Journal of Clinical Epidemiology, 59, 697-703. British Journal of Music Therapy, Vol. Quality collaboratives: lessons from research, Evaluation of quality improvement programmes, Systematically reviewing qualitative and quantitative evidence to inform management and policy making in the health field, Meta-narrative mapping: a new approach to the synthesis of complex evidence. Scoping the literature: Describe and justify the initial process of exploratory scoping of the literature [d], 8. . [c] A lack of reference to changes may not indicate that no changes were made; therefore, we only noted when such changes were explicitly stated Thus it is disquieting that only about one in five of the reviews in our study had selection, appraisal, or data extraction done independently by two persons. The number of electronic databases searched varied greatly, from 1 to 25 (mean=15, median=6). We then look for empirical evidence to populate this theoretical framework, supporting, contradicting or modifying the programme theories as it goes. [d] We did not assess rationale/justification Please read and accept the terms and conditions and check the box to generate a sharing link.