Round 10 self-completion experiment funded in UK

The European Social Survey (ESS) has been allocated over £250,000 by the Economic and Social Research Council (ESRC), part of UK Research and Innovation, to conduct an experiment during the latest round of data collection.

This new project will allow social scientists to understand the impact of choosing a self-completion rather than interviewer-administered approach for repeat cross-sectional surveys.

The project will also help to answer questions about the use of conditional incentives.

Specifically, the funding will enable the ESS to collect a second Round 10 dataset in the United Kingdom (UK) from respondents using only self-completion modes (online and postal).

This second dataset will provide an experimental comparison with the usual ESS dataset already being collected through face-to-face interviews undertaken by the UK data collector, NatCen Social Research.

A self-completion survey will be launched with a sample of 8,000 cases from the Post Office Address File, with each address randomly allocated a conditional incentive group (None, £5, £10).

Every address will be sent a £5 unconditional cash incentive to encourage the next person in the household – aged at least 18 and who has the next birthday – to take part.

The letter will contain a simple website address which will take respondents to the first page of a survey hosted by web-based survey tool, Qualtrics.

Up to two further reminders will be sent to each household, with the final correspondence containing a paper questionnaire with a return envelope. Later the researchers will try to relaunch the survey after a break to see if additional respondents can be recruited.

Once completed, the two datasets will allow analysts to examine how similar the estimates from the self-completion survey are to those from the face-to-face survey.

Face-to-face fieldwork is becoming more expensive in the UK and many countries in Europe, whilst response rates have been declining in most participating countries.

Other surveys are already considering whether a switch to using self-completion modes instead of interviewer-administered interviews offer a more efficient, digital future for data collection.

However, there are concerns about some aspects of self-completion surveys and worries as to whether it is possible to maintain a reliable time series when switching mode.

By conducting the ESS using a ‘push-to-web’ design at the same time as the usual face-to-face survey, this project will provide an important publicly available dataset to help alleviate concerns.

It will allow survey researchers to assess whether a general social survey can transition mode whilst still maintaining its time series, whilst also facilitating direct comparisons in areas such as sample composition, measurement, data quality and costs.

A recent ESS experiment conducted in Austria found similar response rates, representativeness and data quality for a 50-, 35- and 20-minute self-completion survey. By fielding the full 50-minute survey in the UK, the feasibility of conducting longer surveys without interviewers will be addressed.

The dataset is likely to be of significant interest to UK social scientists and will be deposited to the UK Data Service (UKDS).

Professor Rory Fitzgerald, Director of ESS ERIC at City, University of London said: “This experiment comes at a time when we are seeing more and more pressure on face-to-face fieldwork capacity. It is crucial that we prepare for a future where more interviews are performed without an interviewer being present in a respondents’ home.

“This funding from UK Research and Innovation through ESRC will allow us to really understand what self-completion methods would mean for a repeat cross-sectional attitudinal survey in the UK.”