Social survey/questionnaire

What is the method? Social surveys are a questionnaire-based method of research that can produce both qualitative and quantitative information depending on how they are structured and analysed. This section focuses on the use of surveys to collect and analyse qualitative data. Many of the issues and considerations are the same as for the quantitative use of surveys, and more detail can be found in the earlier section of this handbook. When should it be used? Questionnaire surveys can be used in a wide range of settings and to gather a variety of different types of information. You may be evaluating a programme in which a wide range of projects have been commissioned, and want to gather the views of a wide range of project managers, or you may be measuring the impact of an initiative on the business community in a specific geographical area. A small-scale qualitative survey may be conducted to explore in more detail the findings of qualitative research. What do I need to consider? Many of the considerations for a social survey are the same as for a quantitative survey, however we define a social survey as one where less statistical rigour is required, where sample sizes are not as large, and with results not expected to be significant of the wider population. A social survey may have a greater focus on collecting rich and detailed qualitative data. Population A number of questions about the proposed population for a social survey need to be considered. Such as are there language issues? And Qualitative Research Methods Is this the real life? yes/no If no, please answer question 2 2. Is it just fantasy? yes/no Please provide reasons for your answer .................................................. .................................................. .................................................. .................................................. ..................................................what are the geographic restrictions? These are the same issues as for quantitative surveys. Sampling The sample is the section of the wider population that will be engaged in the survey. Detailed consideration of sampling still needs to be made even when not striving for statistical significance. It is still important to understand who the respondent is and what your sampling frame is going to be. Format A social survey will usually be a cross-sectional survey used to gather information on a small sample population at a single point in time. An example of a cross-sectional survey would be a questionnaire that collects data on peoples’ experiences of a particular initiative. However, a qualitative survey could equally be used in a longitudinal study, perhaps returning to particular individuals over time to measure the impact of an intervention on the direction of someone’s life. Questions There are a whole range of questions to be asked in relation to survey design, such as: What types of questions can be asked? How complex will the questions be? Will screening questions be needed? Can question sequence be controlled? Will lengthy questions be asked? Will long response scales be used? A social survey will be more interested in qualitative findings, in recording peoples’ opinions and perceptions, and therefore will make more use of open questions where respondents can give their own responses to a set question. Open questions will begin with what, why, how, or describe, to elicit rich qualitative information. Open questions can be used in a variety of ways: Usage Example As a follow-on from closed questions, to develop a more detailed response. ‘If answering yes to question 7, please provide the reasons for this’ To find out more about a person, their thoughts, needs, problems, etc. Why is that so important to you? To get people to realise the extent of their problems. What effect does this have on your family life? To get people to reflect on the impact of something or some change. How has this made a difference to you? Administration The costs, required facilities, time, and personnel needed to conduct an effective survey are often underestimated, even when it is not on a large scale. There should be an administrative system in place to deal with the questionnaires for when they are returned/completed. This may include numbering the questionnaires, recording what action has been taken with them, entering the results into a spreadsheet/database etc. How should it be used? Surveys can be carried out by phone, post, email, website or face-toface, for detailed pros and cons of these delivery methods see the earlier section on qualitative surveys. In collecting rich qualitative survey data, the most effective method would be via face to face, administered surveys, as the researcher would be able to use prompts to encourage people to give more detailed answers. This does however introduce a bias, which needs to be understood and controlled as much as possible, i.e. by using standard prompts. In qualitative surveys, it is necessary that the interviewer conduct the interview with total objectivity, so that respondents are not influenced by any outside source in their responses. For this reason, interviews should be conducted by welltrained and qualified interviewers. What is the output? The data that a social survey can produce is very much dependent on how the questionnaire is constructed. However, the data can be very useful for providing an overall picture of the way in which a project or programme is being implemented and how effectively it is impacting upon its target audience. Qualitative data output will be in a text, audio or picture format, and each answer may be very different from another. This can make collection of data more difficult, and a way of collating data needs to be considered early in the process. How should it be analysed ? The Quantification of Qualitative Survey Data Surveys can be analysed by collating the frequency of responses to each of the questions on the survey form. This can be done manually using a “frequency table”, which can be easily set up on an Excel spreadsheet to analyse descriptive statistics. QSR NUD*IST and NVIVO are qualitative data analysis packages, which enable non-statistical information from interviews, group work, observations, audio, video, pictures or documents to be analysed according to chosen criteria. For example, it is possible to use the package to ‘pull out’ all material relating to key words or phrases (e.g. neighbourhood renewal) and then sub-divide the data into more specific areas of analysis (e.g. statement of use, problems, projects). This is a powerful piece of software that can provide clarity to wide range of often complicated written or media materials. Case study: Using surveys to evaluate a project A programme targeted on helping young people back into work through training wants to evaluate how well it is achieving its objectives. It uses a survey to canvas the views of young people who have been on the programme to date. The survey asks them closed questions about what training they have attended and how useful they have found the training (on a scale of 1:4). The survey also uses open questions to ask young people about what their plans are for the future as a result of the training (i.e. has it helped them to consider applying for full time work? Or further education opportunities?). The qualitative data is analysed and this shows that the young people have gained in confidence, are looking to go into further education or training or have already secured job interviews in a range of occupational fields, however there is a distinct focus on work in the field of construction. The results of the survey are analysed and this provides conclusions about overall success of the programme, which allows the programme manager to draw conclusions and consider design issues for making the programme more effective in the future. Interviews What is the method? One of the most popular and frequently used methods of gathering information from people about anything is by interviewing them. It is also the most popular method used within the social sciences. There is a continuum of formality around interviewing and it covers a multitude of techniques, from informal “chats” maybe arranged as “vox-pops” right through to highly structured, formal interviews, taped and transcribed. The different types and styles of interview elicit very different types of information. Conducting interviews is an interpersonal process and as an investigator you must be very aware of your own behaviours and assumptions in the context. Interviews are not “neutral” social spaces and you must be respectful and maintain appropriate boundaries at all times. What do I need to consider ? Interviews are a qualitative method of research often used to obtain the interviewees’ perceptions and attitudes to the issues. The key issue with interviewing is making decisions about who are the key people to talk to and what type of interview are you going to use. Interview Style There are three clearly identifiable styles of interview- structured, semistructured and unstructured: Structured - Follows a set of specific questions, which are worked through systematically. This type of interview is used when the researcher wishes to acquire information where the responses are directly comparable. Semi-structured - This is a more commonly used interview technique that follows a framework in order to address key themes rather than specific questions. At the same time it allows a certain degree of flexibility for the researcher to respond to the answers of the interviewee and therefore develop the themes and issues as they arise. Unstructured - This method of interview does not follow any predetermined pattern of questions or themes. Rather, the interviewer will address the issues as they emerge in the interview. The method is useful when the researcher wishes to explore the full breadth of a topic. Interview Type These are some of the types of interviews: Fact finder - This type of interview is used to obtain specific information from an interviewee and usually includes structured or standardised interview questions (the wording of the questions and the order in which they are asked is the same). It is used when some information is already known and there is a need to gain a more in-depth insight. An example of when a fact finder interview would be appropriate is when interviewing a project officer as part of an evaluation of their project. Quantitative (or ‘hard’) information is usually already known (such as outputs and funding data), therefore the interview could be used to discover qualitative information that the hard data cannot portray, such as the ‘softer’ outcomes of the project. Idea generator - In many respects, this type of interview is the opposite of the fact finder interview. It is used when the interviewer has no preconceptions about what might be discovered over the course of the interview and results can be used to set the parameters or framework for the study. Interview questions are loosely structured allowing maximum flexibility to explore a range of issues. Idea generator interviews are usually applied at the start of a research project in order to discover and explore issues from a particular group or community. For example, in order to develop a community cohesion strategy, idea generator interviews may be used to find out what community cohesion means to different groups in the community. Exploratory - These are the most frequently used type of interview as they are relevant to most types of research project. They are usually conducted with representatives that have a strategic role to play in the research. These types of interview require some degree of prior knowledge about the research subject as they are about testing hypotheses, making connections between other elements of the research, ensuring the strategic fit and progressing the findings of the research forward (e.g. senior officials from a local authority may be interviewed using this method in order to find out future plans and priorities and how they fit in with others’ plans and priorities). Experiential - This type of interview aims to draw out people’s feelings, perceptions and experiences over a specific period of time (e.g. the duration of a regeneration programme or project). This provides rich, in-depth material about how the subject under investigation has affected an individual’s life on a personal level. Experiential interviews may be used to elicit information from people who have benefited from a community project or who live in an area that has received regeneration monies.