A seven-minute explanatory video? In a quant survey? Yes – it worked!

Screenshot of explanatory video

One of the challenges we face when trying to measure opinion on difficult policy areas is that quantitative surveys can miss the complexities and nuance of the issues. In contrast, deliberative research enables us to give a small number of people the time and information they need to come up with an informed ‘citizen’ perspective – something that is not possible in a 15-minute survey… or is it?! Well – not quite, but we recently tried something that came a bit closer to getting a more informed opinion from a nationally representative sample.

The Legal Services Board (LSB) faces a difficult decision about whether regulators should do more to ensure that lawyers remain competent throughout their careers (rather than just at the point of qualification). To support its thinking, we ran a ‘Citizens Jury’ style exercise online (more about that here) taking 23 people through an in-depth and evidence-heavy three weeks of deliberation.

The LSB was mindful of the level of scrutiny that both its decisions and the research will face, so wanted to quantify the opinion of the wider public. However – how could we ensure that over 1,000 members of the public had enough understanding of this contentious issue, to give a sound, considered opinion?

We took the decision to strip out half of the questions in the 15-minute survey, replacing them instead with a video setting out key issues, different viewpoints, and trade-offs. We knew there was potential for high drop-out with a 7-minute video on a complicated subject, particularly when respondents typically are used to quick fire survey questions. We worked hard with the LSB to make sure that the video was engaging, succinct and balanced.

In fact, it worked really well: we had low drop-out, and only 11% of participants did not continue with the survey at the video stage. In addition, on the key ‘difficult’ questions, only a small minority felt unable to offer an opinion – we’re pretty sure that there would have been many more had we not shown them the video. We also had feedback that respondents enjoyed the challenge of a slightly more thought-provoking survey.

So in our experience, with the right approach, it is possible to quantify opinion on challenging policy issues with a sizeable nationally representative survey. You can find out more about what we did and the results of the research here.