Estimation of Response Propensities Using the Previous Survey

Many surveys are carried out yearly, and the implementation of the surveys remains the same from year to year. Experience from a previous survey is useful when planning a new survey, because the response behavior usually remains quite the same in subsequent years. This paper studies how response propensities, estimated using the dataset of the previous survey, predict actual response rates. In this study, two consecutive datasets of the European Social Survey were available. The both datasets contained same register variables. Response propensities were estimated to the older dataset using a logistic regression model. Then the propensities were imputed to the newer dataset using a donor-recipient method. The imputation was based on the explanatory variables of the logistic regression model so that the donor and the recipient had the same values in the variables. Then it was examined if there was a connection between the imputed response propensities and actual response rates. The result was that the imputed response propensities predicted the response behavior quite well. People with low response propensities were often nonrespondents, and people with high response propensities were often respondents. Using the previous survey, it is possible to calculate response propensities for a new sample before the data collection of the survey has been started. Then challenging respondents are known before the data collection, and this information is useful for data collection.


Introduction
Nonresponse is an increasing problem in surveys internationally [1]. Response rate is widely used and important indicator of the quality of the survey, but it is alone a poor indicator of any nonresponse bias [2]. It is also important to think representativeness based on auxiliary variables. The set of respondents is representative, if the response propensities are same for all units in the sample [3].
Many surveys are carried out annually. The implementation of the surveys and response behavior remain quite similar in consecutive years. For example, many survey organizations use simple random sampling every year, and collect the data similarly from year to year. The set of respondents may be unbalanced from year to year. For example, older age groups and high-educated people often are more willing to participate in the surveys than young and low-educated people. If a sample has been drawn using simple random sampling, the sample represents the target population, but the set of respondents differs from the population according to auxiliary variables. This causes bias to the estimates.
The survey organizations often archive their old survey datasets. Then it is possible to utilize information of previous years when planning a new survey. If a survey has conducted several times previously, it may be possible to determine different optimal designs for different subgroups based on the experience [4]. Sampling design and data collection can be implemented.
An easy way to utilize the previous survey is to calculate response rates from the previous dataset. The new sample may be drawn using information about the previous response rates [5]. For example, stratified sampling can be used so that the new sample contain more young and low-educated people, whose response rates have been low in the previous survey. Respectively, the new sample can contain less old and high-educated people, if they have had high response rates earlier. If the previous sample has been drawn using simple random sampling, this kind of stratified sampling can be a better choice, because then the set of respondents will probably represent better the target population.
Another possibility to utilize the previous survey is to use responsive or adaptive survey designs. If the response rates of the previous survey are known, phone interviewing may be targeted to certain groups. The phone interviewers can contact more young and low-educated people, if these groups have had low response rates in the previous year. Schouten et al. [6] present adaptive survey design which offers several methods for data-driven tailoring of data collection. Researchers are interested in effectiveness of these designs. According to Chun et al. [7], it is inevitable that responsive and adaptive survey designs will be a natural component of many data collections in the future. Responsive and adaptive survey designs may mitigate the risk for nonresponse bias and control data collection costs [8,9]. However, practical implementation of these designs is not necessarily easy [10,11]. One reason is that interviewer resources are often limited. The interviewers play an important role in the success of the adaptive and responsive survey designs.
Alternative possibility for responsive data collection is to estimate response propensities before the data collection of the survey, using the dataset of the previous year. The response propensities can be imputed to the new sample. Then the response propensities can be utilized in responsive data collection so that the phone interviewers try specially to contact people with low response propensities.
This paper shows how response propensities can be estimated to a dataset, using the dataset of the previous year. The study was carried out using Statistics Finland's data. Two datasets were available: datasets of European Social Survey from rounds 7 (2014) and round 8 (2016). In this paper, ESS7 means the dataset of round 7 and ESS8 means the dataset of round 8. Response propensities were estimated to the ESS7 using a logistic regression model. After this, the propensities were imputed from the ESS7 to the ESS8. The both datasets included same register variables, and the imputation were implemented using register variables. An interesting question was how the imputed propensities predicted actual response rates. Were individuals with low response propensities often non-respondents? On the other hand, were individuals with high response propensities often respondents?
Utilizing response propensities in responsive and adaptive survey designs is an interesting topic. Luiten et al. [12] used older datasets to predict contact and co-operation propensities in the Survey of Consumer Sentiment. They used the logistic regression model and found following connections: the higher the predicted contact propensity, the higher the actual contact rates and, the higher the predicted co-operation propensity, the higher the actual co-operation rate.

ESS Datasets
The European Social Survey is an academically driven, large-scale, European, cross-national survey that has established in 2001 [13]. The last round was in 2018. The target population of the ESS consists of all residents 15 years or older who are residents of the country within private households. The ESS is conducted every two years using face-to-face interviews.
Statistics Finland has participated in the all ESS rounds from 2002. Samples for ESS and other person surveys at Statistics Finland are drawn for the Statistics Finland's population database. It is a person register which is based on the Central Population Register. Therefore, persons are selected directly from the register, and multi-level sampling is usually not required at Statistics Finland. The population database also contains useful variables for surveys, like gender, age, education and region. These variables are known for all people in the frame, so the variables may be utilized in sampling, weighting, estimation and modelling. It is possible to combine these variables to all survey datasets. Calculating response propensities and imputation of the propensities into other datasets is possible when same register variables are in all survey datasets.
Statistics Finland archives its old datasets, so it is possible to use them and combine register variables to the datasets. In Finland, the sample size of the ESS was 3400 in the both rounds, 7 and 8. Sampling method in the both rounds was single stage equal probability systematic sampling, where the population database were ordered by gender, domicile code and date of birth before the sampling. This method brought similar sample like simple random sampling, but the distributions of gender and municipality were very similar to the population. Response rate was 62.7% in the round 7 and 57.7% in the round 8. The both datasets included same register variables, for example gender, age, education, marital status and different area variables. The both datasets also included a binary response indicator (1 = respondent, 0 = non-respondent).

Fitting a Response Propensity Model
A response propensity model was fitted to the ESS7. The model was a logistic regression model where the dependent variable was the response indicator. Laaksonen gives more information about response propensity models and link functions [14,15]. Explanatory variables of the model were selected from the register variables. The model was made using SAS program, which offers different automatic methods to select good explanatory variables to the model. In the final model, there were four classified explanatory variables for the response indicator with following classes: gender (1 = male, 2 = female), age group (1 = under 30, 2 = 30-44, 3 = 45-59 and 4 = over 59 years), education (1 = basic, 2 = secondary and 3 = tertiary) and municipality group (1 = urban municipalities, 2 = semi-urban municipalities, 3 = rural municipalities). Table 1 shows information about the explanatory variables.  Table 1 shows number of classes of the explanatory variables. These four variables had 2*4*3*3 = 72 combinations. Therefore, the ESS7 could contain up to 72 probabilities. However, the ESS7 contained only 69 different response propensities, so the explanatory variables had three combinations with no people. The size of the data was 3400, so it is possible to calculate that on average 49 people had same response propensity in the ESS7. Table 1 also shows pvalues of Wald Chi-Square tests. Every explanatory variable had statistically significant effect on response (significance level 0.05). Age group, education and municipality group were the best explanatory variables, but also gender had impact on response.

Imputation of Response Propensities
After modeling, the response propensities were imputed from the ESS7 to the ESS8. The imputation was based on the four variables presented in Table 1. These register variables were in the both datasets. Imputation method was a donorrecipient method. A donor in the ESS7 and a recipient in the ESS8 had same characteristics (same values in the four variables). Table 2 shows examples about response propensities in the ESS7. These response propensities were lowest and highest propensities in the ESS7. The values of the variables are described in section 3.1. On the top line the response propensity is 39,8%. All people in the ESS7 who had these characteristics (female, age 30-44, basic education and living in an urban municipality), had this response propensity. If the ESS8 contained similar people, their imputed response propensity was 39,8%. Similar way, all people in the ESS8 got imputed response propensities based on the four variables.

Imputation Problems
There were also problems in imputation. The ESS8 contained recipients with unique combination of imputation variables: the ESS7 did not include similar donor persons. For example, the ESS8 contained following people: gender = male, age group = under 30, education = high and municipality group = semi-urban. The ESS7 did not include similar people, and these donor persons in the ESS8 did not get any response propensity in the imputation process. Response propensities had to be added manually for this kind of people. This meant looking for people with very similar combination and borrowing response propensity from these people. Imputation problems cause that a lot of explanatory variables cannot be used to calculate response propensities. The model in this study contained four variables that had 69 combinations. It is a good number, when the sample size is 3000-4000. If five explanatory variables are used, the variables may have more than 200 combinations. Then the imputation process would be challenging because there would be a lot of recipient persons without similar donor persons. Besides, when three or four best explanatory variables are selected to the model, the model will produce response propensities that are good enough. If a lot of explanatory variables are used in the model, the response propensities will hardly change, because the best explanatory variables have the greatest impact on response propensities.

Evaluation of Response Propensities
The ESS8 were divided into groups according to imputed response propensities. Actual response rates in the six response propensity groups in the ESS8 are shown in Figure 1. Response rate is lowest in the group where response propensities (rp) are lower than 45%. The response rate in this group is 40%. Response rate is 75% in the group where response propensities are over 75%. Figure 1 shows that imputed response propensities predict actual response rates quite well. The curve is rising, so there is a clear link between the imputed response propensities and the actual response rates. Figure 1 also shows that response behavior remained fairly the same in the ESS7 and the ESS8. Table 2 shows examples which kind of persons have lowest response propensities (under 45%). These people are low-educated, under 45 years and live in urban areas. Their response rate was low in the ESS7, and the Figure 1 shows that their response rate was low in the ESS8, too. Table 3 presents estimated response propensities and actual response rates in the ESS8 in groups where response propensities are lowest and highest. The groups are same as in Table 2. Table 3 also contain information about the number of people in the groups (respondents and non-respondents in total).  Table 3 shows that there is a clear connection between response propensities and actual response rates. In the group where the response propensities were 88,9%, all people were respondents. In the group where the response propensities were the lowest, the response rate was only 25,7%.

Conclusion
The objective of this study was to find out how estimated response propensities predicted the actual response rates. The response propensities were constructed using the older dataset of the same survey. Then the response propensities were imputed to the newer dataset, and the imputed response propensities were compared to the actual response rates in different groups. There was a clear connection between the imputed response propensities and the actual response rates. People with low response propensities were often nonrespondents, whereas people with high response propensities were often respondents. The purpose of the study was to simulate the situation where the data collection of the new survey is about to begin. In this kind of situation, estimation of response propensities for the new sample is possible if the dataset of the previous survey is available.
Estimation of response propensities is useful in surveys that are carried out annually, and where the sampling designs and the implementation of the surveys generally remains the same in subsequent years. When potential challenging respondents are known in advance, it is possible to plan the data collection in a new way and consider motivation letters or incentives for some respondents, for example.
In telephone interview surveys, imputed response propensities can determine the number of calls. The lower response propensity, the more contact attempts. However, the implementation of the responsive data collection may be challenging in many statistical organizations. The design requires enough interviewers so that the interviewers had enough time to make contact attempts. The interviewer resources may be very limited in many survey organizations, and then the interviewers may find it difficult to implement the data collection even in the normal way. They do not necessarily have time to call many times for challenging respondents. Another problem is that the interviews are often conducted using case management systems that are programmed for "normal" data collection. When using responsive survey design, it may be difficult to make changes to the case management systems. However, it could be interesting to test this kind of responsive data collection which is based on the previous survey. This could be a topic for a further study.
Other possibility to utilize the response propensities is to compare interviewers to each other during the data collection. When response propensities are known for a new sample, a score may be calculated for each person in the sample. The score may be, for example, the inverse of the response propensity. Table 3 shows that the response propensities in the ESS8 are between 39,8% -88,9%. If the score is the inverse of the response propensity, the scores of the ESS8 are between 1,12 -2,51. "Challenging" respondents have high scores, and "easy" respondents have low scores. When interviewers manage in getting respondents, the scores of the respondents are summed together. Total scores of interviewers are compared at different times during the data collection. An interviewer with the best score has not necessarily conducted most interviews, but he or she may have succeeded with challenging respondents. The best interviewers may be rewarded. This may encourage the interviewers to call challenging respondents and work harder.