Reflections from working on an evaluation project: Your Experience in Mind
by Julie Billsborough
How well were local Minds doing in 2014?
To answer this, Mind commissioned the McPin Foundation back in April 2014 to carry out an evaluation of selected local Minds from across the country. We used a survey to capture the views of people using a range of Local Mind services and supports. I worked on this project, called ‘Your Experience in Mind’, and we have put together our first methods paper describing our approach. you can read it here: Survey Champion methods paper July 2015
The Survey Champion Model
We developed an approach to carrying out this evaluation which we called the Survey Champion model. We employed people who had used their local Mind to promote the survey for us, help us make sense of the data we received and comment on the final report. I was responsible for supporting the Survey Champions, chatting to them about the survey, giving guidance and letting them know they could get in touch with me at any time if they had any problems with the survey. Not all the local Minds had a Survey Champion. 17 local Minds took part in the evaluation and out of these, 12 had a Survey Champion. Not all the local Minds could easily identify a local service user who wanted to undertake the role.
I was really pleased to see Survey Champions rise to the challenge, promoting the survey using our posters, writing articles or blogs for their local Mind’s newsletter and / or website, encouraging local Mind staff to promote the survey, organising coffee mornings, and promoting the chance to provide feedback on the services they received via social media such as Facebook and Twitter. One real advantage of using Survey Champions was their local knowledge and the time they could spend promoting the survey. They used a very ‘hands-on’ approach, talking to as many service users as possible, visiting social groups and activity sessions.
We found this approach really helped recruitment, and the local Minds with a Survey Champion had an average of 69 surveys returned, while those that didn’t have a Champion had 52 surveys returned. We don’t know how many people could have completed the survey at each of the local Minds but the figures are encouraging and suggest the Survey Champion model was a good way of working. Feedback we received from the Survey Champions confirmed this when we heard about the impact on the individuals who had worked with us.
Impact for Champions
After we’d finished the survey, I chatted to some of the Survey Champions about how they felt about working with us. Some of them told me how they’d found the work rewarding:
“I believed in it and enjoyed giving people that chance to say something confidential; I think that’s really important”
Others told me that taking part had improved their mental wellbeing and one Survey Champion said it gave him a “purpose for the day”. Some Champions told me how they enjoyed the feeling of responsibility and being treated as a colleague by staff at their local Mind. “Being made to feel welcome did me a lot of good”.
What worked and what didn’t
After the evaluation ended, the Survey Champions were very useful in helping us review the process and in particular think about the survey tool. They explained what worked and what didn’t work so well about their role. In particular, they felt that being local had really helped, making it so much easier for them to chat to service users directly, and to promote the survey using their local Mind website and newsletters. We had really wanted them to reach people who might not easily respond to a paper copy survey or an online form.
They also explained that the survey didn’t always go smoothly. Being local wasn’t always as helpful as some had first thought. Some of the local Minds covered large areas, which made it difficult for them to talk to service users face-to-face, and not all local Mind services were easily accessible due to confidentiality issues. This restricted how much promotion work they could do in some places.
I have worked on several studies at the McPin Foundation. We put lessons learned from the previous evaluation of local Minds into practice for this year’s survey. For example I am able to provide the Survey Champions with better and more detailed guidance in light of the lessons learned from last year’s survey.
It felt good to be part of the survey team and to help the Survey Champions. Many inputted into the findings sections, and helped us complete the report for the funder. I was really pleased to see them do so well and to get so much out of working on the project. We are currently working on the 2015 data collection for ‘Your Experience in Mind’, this time working with 20 local Minds with at least one Survey Champion in each. I hope they also have a positive experience working with us!
Julie Billsborough, Researcher, The McPin Foundation