Before I start writing this blog I am mindful that many of those who read this may have some experience of systematic reviews. Thus, before I start here is my disclaimer:
I have conducted and written one systematic review as a lead author and been involved with two others, and this is the extent of my experience.
A systematic review is a rigorous method to searching for information. I guess I might compare it to sorting out my wardrobe. For not only is it logical, it’s also rather satisfying, but can be slightly painful somewhere in the middle. Ultimately though, it points you in the right direction so you can understand where the gaps are (in your wardrobe or the research literature), and what works and what doesn’t (in your wardrobe and the research literature). It allows you to make a more evidence based decision, rather than going on feel alone. In a recent blog posted on the Nature journal website, an Associate Professor of chemical and biochemical engineering explained that systematic reviews can be a great way to understand the history in the area you are working. The remaining contributors to this blog also highlighted valid points such as using a review as an introduction to a field or science, a good source of research experience and a way of understanding critique. For interventionists a systematic review is also an important part of a bigger picture.
Prior to completing my systematic review, I was delivering interventions to people with dementia, consulting the research evidence and using my clinical judgement to identify perceived gaps to guide my research ideas. In 2008, Craig et al, published the Medical Research Council guidance for the development of complex interventions. They provided a really practical and structured guideline to researchers, especially those developing complex interventions, on how to systematically go about developing a new intervention. A key stage is to understand what has gone before, and whether the intervention one is developing is likely to have a worthwhile effect. Doing a systematic literature review is a key component of this process.
A complex intervention is one that has multiple interacting components, this basically constitutes almost all behavioural interventions, particularly speech and language therapy interventions, and interventions for dementia. Craig et al explain that the first step in developing a new intervention is to “identify what is already know about similar interventions and the methods that have been used to evaluate them”. When I started my PhD back in 2015, there hadn’t been any reviews of this kind examining the functional interventions for people with language led dementias (primary progressive aphasia, and by functional I mean interventions focused on activity and participation, including environmental modifications). What there was focused on impairment based interventions (that aim to remediate, alleviate or improve symptoms). Thus the first thing I did was to do just this, a systematic review of functional communication interventions for people with primary progressive aphasia and their family members.
I attended a training course on how to do systematic reviews, which was incredibly useful. I put together a protocol for my review, and registered it online with PROSPERO. This was extremely helpful as it forced me to stick with what I said I would do. Having found 12,309 records on searching the databases I needed to be extremely strict with myself and not get distracted by any interesting articles. It may be surprising that I had found so many articles, but actually there is so much variability in the terminology around primary progressive aphasia and the interventions that I needed to cast a big net to capture the right articles. After going through all the titles and abstracts, with another colleague (an independent reviewer) looking at 10% of all of these and again at 10% of the included items I was left with only 19 studies.
Technology is a key component in helping manage the unwieldy numbers, and plethora of data under scrutiny. I used Mendeley and Endnote to manage the articles I found. Since then I also used Rayyan, and would actually really recommend this https://rayyan.qcri.org/welcome. Above and beyond this I recommend using excel and being a rigorous documenter. It’s all about organisation and knowing where you left off. If you can make sure it is clear and concise and you can’t get lost, then that will work for you. (Oh and I do love a traffic light system of highlighting red for excluded, orange for unsure and green for included- so satisfying!). The Nature blog has a handy table about tech at the bottom.
Once I had my 19 studies my plan had been to complete a statistical analysis of the results of the studies. Dutifully I collated the data from the outcomes of the studies, but found (unsurprisingly unfortunately) that each study used a completely different set of measures that were not at all comparable. My other plan had always been to extract the key components of the interventions to inform the development of my own. I did indeed do this, using an adapted version of the Intervention Taxonomy (ITAX; O’Rourke, Power, O’Halloran, & Rietdijk, 2018).
The information I gathered from the review was absolutely key to the development of my intervention. Alongside this, I surveyed 105 clinical speech and language therapists, held focus groups with people with primary progressive aphasia and their families and co-developed an intervention called Better Conversations with PPA. The systematic review had flagged the key common components of the interventions to date which included having a communication partner in the intervention (usually a family member) and focusing on building on existing strategies. These were both components that we embedded into the development of Better Conversations with PPA. We have almost finished the Phase II pilot-feasibility study of the intervention across 11 NHS sites (due to close on the 31st January 2021. The next stage of the complex intervention guidelines points me toward an effectiveness evaluation and implementation work- so watch this space.
- Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. Bmj, 337.
- O’Rourke, A., Power, E., O’Halloran, R., & Rietdijk, R. (2018). Common and distinct components of communication partner training programmes in stroke, traumatic brain injury and dementia. International Journal of Language & Communication Disorders, 53(6), 1150-1168.
- Volkmer, A., Spector, A., Meitanis, V., Warren, J. D., & Beeke, S. (2020). Effects of functional communication interventions for people with primary progressive aphasia and their caregivers: a systematic review. Aging & mental health, 24(9), 1381-1393.
Dr Anna Volkmer is a Speech and Language Therapist and researcher in Language and Cognition, Department of Psychology and Language Sciences, University College London. Anna is researching Speech and language therapy interventions in language led dementia and was once voted scariest speech and language therapist (even her children agree).
What advice would you give to someone undertaking their first systematic review? Let us know in the comments box below, just hit reply.