Print This PostNo matter how well you plan research projects or activities, there can still be issues that crop up to challenge you. Whether you’re new to research or have 20 years’ experience, you can’t always predict what might happen in different situations, but a bit of experience might affect how well you cope when things do go awry and how you respond. Here, we reflect on a number of challenges that we encountered during a particular project, and we’ll show how we dealt with them and our lessons learned.
A bit about the project
We’re not going to name names, but as a brief overview we were commissioned to evaluate how well a new staff role worked in a housing scheme and the impact it had on the people living in that scheme. The overall plan was to conduct surveys with the residents at different timepoints during the two-year evaluation period, conduct case study interviews with six residents/dyads to explore the impact in more detail, interview staff members to gain their perspectives, and use reflective diaries from the person in the new role to see how the role was working in practice. Nothing unusual, nothing complicated, nothing we hadn’t done before on other projects, so it should have been relatively straightforward, right?
Research challenges
The new role changed, even before a person was appointed. While this in itself wasn’t an issue as it didn’t affect our methodology or measures, it wasn’t a great start as it introduced an element of uncertainty and confusion. In addition to creating a new role, the housing scheme was still being built and experienced delays, which had a knock-on effect on residents moving in. Understandably, the focus for the scheme was around resolving teething issues with the building and finding new residents, so it took a while for the new role to be recruited. This meant that the project didn’t begin on time as there was nothing for us to actually evaluate. Cue the moveable/flexible timeline.
There was also confusion about our role and relationship with the housing scheme. Residents believed we employed and managed the person in the new role. Even the person in the new role though we were their manager! And all this despite stating clearly in all interactions that we were completely independent from the scheme and only there to evaluate the new role.
Imagine the scenario when we receive an email to say the new member of staff is resigning after three months and not being replaced. Not quite what had been envisaged.
It’s quite tricky to evaluate a role that doesn’t exist, so we had to do a quick pivot and decide to compare what happened during the first three months and what happened after they left. Cue solution-focused, flexible research.
Two not so great views, for different reasons
Data collection issues
One of the issues we encountered around data collection was that we were working at a distance as we were based nowhere near the scheme so were not able to visit that often. This meant that some of our data collection had to be facilitated by staff at the housing scheme. We had already met the residents, chatted at their coffee morning and explained the surveys we were going to distribute. Residents had also volunteered to distribute booklets and encourage completion. However, we soon spotted that, despite reassurances of anonymity, residents felt that pairing a signed consent sheet with their completed booklet would enable housing scheme staff to see their answers. Cue a low return rate and making changes to the consent process for subsequent distribution of booklets.
We also had one of the case study dyads withdrawing part way through the evaluation – as is their right – but luckily we had recruited an additional resident as a case study so it didn’t affect things too much.
Due to changes within the project affecting what we were looking at, we also had to adapt our data collection in response, such as the questions we were asking in our surveys and case study interviews to reflect the evolving situation, and no longer having regular reflective diaries being completed once the person had resigned. This resulted in a need to submit an ethics amendment (not once but twice!) to make sure that everything was done properly, with us providing a clear rationale for making each change.
It was also useful for us as researchers to refer back to our ethics approval as we found that lots of interesting information came up during informal interactions when we were visiting the scheme, but we had to be clear about what could and could not be included in the research. We did not have approval to carry out observations or include anecdotal evidence as part of our data collection, only information captured via the planned activities. As a researcher, this can be frustrating because many great things happened in those micro-interactions we observed in the communal spaces. Remember the KitKat advert where the photographer is trying to take a photo of pandas? Watch it on YouTube and you’ll see what we mean. Thankfully, by using several different data collection activities there was already a richness to the data that we did capture.
Practicalities
On a more practical note, to make the most out of our visits we found that the best approach was for two researchers to be at the scheme for three days at a time. This meant that travel and accommodation had to be factored in. Well sourced accommodation turned out to be right next to a retail park – which was still being built. Not ideal and a retail park doesn’t really have the best views, but we did get to see how the construction progressed on subsequent visits.
We also had issues with the hire car on one visit (warnings about the brakes are always fun), but successfully used a ‘divide and conquer’ approach with one researcher going to swap cars while the other continued with interviews at the scheme.
Illness was also a factor, unfortunately on more than one occasion. One scheme visit had to be rescheduled at the last moment due to one researcher being unwell and unable to work, while another visit had to be cut short due to a researcher finding out they had Covid on the second day. While we want to reassure everyone that no one caught it at the scheme (or even the second researcher), it made for an interesting journey back in terms of logistics and keeping everyone safe.
What helped to make the project work?
Despite the variety of challenges encountered, the project did conclude successfully and on reflection this was due to a few key factors.
- Having a ‘spare’ – there was a lot of benefit in terms of having two researchers involved throughout. It meant that we were able to discuss and form a plan of action when things went wrong and could still keep research activities going while also resolving issues (e.g. swapping the car). Using multiple types of data collection activity was useful as we were not reliant on one particular method. For example, it meant that even without reflective diaries after the first three months, we were still able to collect a good amount of data. Recruiting more case studies than we needed helped to reduce the impact of one dyad withdrawing. Similarly, conducting multiple visits at different timepoints meant that if one was cancelled or delayed we didn’t lose everything. Longer visits allowed us to have ‘down time’ at the scheme so that we could spend it with residents and allow them to get to know us, rather than trying to cram everything into two days.
- Developing good relationships – linked to the previous point, engaging with residents during down time meant that we got to know people and were remembered between visits, so they became more comfortable and relaxed with us being there. We also developed good relationships with the staff which made it easier to ask them to help out with arranging interviews etc., and when any visits were affected they knew we weren’t messing them around for no good reason. The ethics panel also came to know us quite well (unsurprisingly!) as we had to get plans changed and reapproved more than once.
- Clear communication – there was a key contact on both sides (i.e. within the project team and at the scheme) so all communication went through them. During our visits we made sure to correct any misconceptions about our role, and also spent time in communal areas to answer any questions that residents had. This helped, to some extent, to reinforce messages around our independence from the scheme and reassure residents about how their survey responses would be used. We also produced regular interim reports and held Advisory Group meetings to ensure that everyone was aware of progress, changes, and potential issues, so there were no surprises at the end of the project.
- Being flexible – it was important for us to recognise that things happen and that’s ok. We just have to be prepared to adapt and go along with it, changing our plans accordingly. At a scheme level, it was useful for us to remember that we were going into people’s homes and residents have their own lives, so even though an interview might have been arranged for a particular time it may need to be rearranged on the day if it was no longer convenient or appropriate. This links back to having spare time during our visits as it meant schedule changes could be accommodated rather than our plans being too rigid or time pressured.
- Good preparation – make sure you’ve got contact details for the scheme staff, hotel, car hire etc., just in case!
And finally
So it sounds like we went through a lot on this project, and what we started with didn’t necessarily match what we ended up with, but we did complete an evaluation that everyone was happy with. Did we jokingly wonder what else could go wrong? Yes, on many occasions. Was it a little bit stressful at times? Also yes, but accepting and adapting rather than fighting against things beyond our control made a huge difference. Did we learn a lot that will help us in our future work? Yes.
Despite the experience we brought into this project – that undoubtably helped us to manage some tricky situations – there’s always something new to take away.
And hopefully our experience on this project gives you a few ideas to think about too.
Authors
Jennifer Bray is a Research Assistant at the University of Worcester’s Association for Dementia Studies. Her work focuses on improving quality of life for people affected by dementia, with recent projects on environmental design and carer support.
Dr Teresa Atkinson is a Senior Research Fellow at the University of Worcester’s Association for Dementia Studies. Her work focuses on improving quality of life for people living with dementia and those who support them, drawing on over 20 years of qualitative research experience.
Ajantha Abey
Dr Emma Law
Dr Clíona Farrell
Adam Smith
Rebecca Williams