MEANING OF RESEARCH DESIGN

The formidable problem that follows the task of defining the research problem is the preparation of the design of the research project, popularly known as the “research design”. Decisions regarding what, where, when, how much, by what means concerning an inquiry or a research study constitute a research design. “A research design is the arrangement of conditions for collection and analysis of data in a manner that aims to combine relevance to the research purpose with economy in procedure.”1 In fact, the research design is the conceptual structure within which research is conducted; it constitutes the blueprint for the collection, measurement and analysis of data. As such the design includes an outline of what the researcher will do from writing the hypothesis and its operational implications to the final analysis of data. More explicitly, the desing decisions happen to be in respect of: (i) What is the study about? (ii) Why is the study being made? (iii) Where will the study be carried out? (iv) What type of data is required? (v) Where can the required data be found? (vi) What periods of time will the study include? (vii) What will be the sample design? (viii) What techniques of data collection will be used? (ix) How will the data be analysed? (Xi) In what style will the report be prepared? Keeping in view the above stated design decisions, one may split the overall research design into the following parts: (a) the sampling design which deals with the method of selecting items to be observed for the given study;Claire Selltiz and others, Research Methods in Social Sciences, 1962, p. 50. (b) the observational design which relates to the conditions under which the observations are to be made; (c) the statistical design which concerns with the question of how many items are to be observed and how the information and data gathered are to be analysed; and (d) the operational design which deals with the techniques by which the procedures specified in the sampling, statistical and observational designs can be carried out. From what has been stated above, we can state the important features of a research design as under: (i) It is a plan that specifies the sources and types of information relevant to the research problem. (ii) It is a strategy specifying which approach will be used for gathering and analysing the data. (iii) It also includes the time and cost budgets since most studies are done under these two constraints. In brief, research design must, at least, contain—(a) a clear statement of the research problem; (b) procedures and techniques to be used for gathering information; (c) the population to be studied; and (d) methods to be used in processing and analysing data. NEED FOR RESEARCH DESIGN Research design is needed because it facilitates the smooth sailing of the various research operations, thereby making research as efficient as possible yielding maximal information with minimal expenditure of effort, time and money. Just as for better, economical and attractive construction of a house, we need a blueprint (or what is commonly called the map of the house) well thought out and prepared by an expert architect, similarly we need a research design or a plan in advance of data collection and analysis for our research project. Research design stands for advance planning of the methods to be adopted for collecting the relevant data and the techniques to be used in their analysis, keeping in view the objective of the research and the availability of staff, time and money. Preparation of the research design should be done with great care as any error in it may upset the entire project. Research design, in fact, has a great bearing on the reliability of the results arrived at and as such constitutes the firm foundation of the entire edifice of the research work. Even then the need for a well thought out research design is at times not realised by many. The importance which this problem deserves is not given to it. As a result many researches do not serve the purpose for which they are undertaken. In fact, they may even give misleading conclusions. Thoughtlessness in designing the research project may result in rendering the research exercise futile. It is, therefore, imperative that an efficient and appropriate design must be prepared before starting research operations. The design helps the researcher to organize his ideas in a form whereby it will be possible for him to look for flaws and inadequacies. Such a design can even be given to others for their comments and critical evaluation. In the absence of such a course of action, it will be difficult for the critic to provide a comprehensive review of the proposed study. FEATURES OF A GOOD DESIGN A good design is often characterised by adjectives like flexible, appropriate, efficient, economical and so on. Generally, the design which minimises bias and maximises the reliability of the data collected and analysed is considered a good design. The design which gives the smallest experimental error is supposed to be the best design in many investigations. Similarly, a design which yields maximal information and provides an opportunity for considering many different aspects of a problem is considered most appropriate and efficient design in respect of many research problems. Thus, the question of good design is related to the purpose or objective of the research problem and also with the nature of the problem to be studied. A design may be quite suitable in one case, but may be found wanting in one respect or the other in the context of some other research problem. One single design cannot serve the purpose of all types of research problems. A research design appropriate for a particular research problem, usually involves the consideration of the following factors: (i) the means of obtaining information; (ii) the availability and skills of the researcher and his staff, if any; (iii) the objective of the problem to be studied; (iv) the nature of the problem to be studied; and (v) the availability of time and money for the research work. If the research study happens to be an exploratory or a formulative one, wherein the major emphasis is on discovery of ideas and insights, the research design most appropriate must be flexible enough to permit the consideration of many different aspects of a phenomenon. But when the purpose of a study is accurate description of a situation or of an association between variables (or in what are called the descriptive studies), accuracy becomes a major consideration and a research design which minimises bias and maximises the reliability of the evidence collected is considered a good design. Studies involving the testing of a hypothesis of a causal relationship between variables require a design which will permit inferences about causality in addition to the minimisation of bias and maximisation of reliability. But in practice it is the most difficult task to put a particular study in a particular group, for a given research may have in it elements of two or more of the functions of different studies. It is only on the basis of its primary function that a study can be categorised either as an exploratory or descriptive or hypothesis-testing study and accordingly the choice of a research design may be made in case of a particular study. Besides, the availability of time, money, skills of the research staff and the means of obtaining the information must be given due weightage while working out the relevant details of the research design such as experimental design, survey design, sample design and the like. IMPORTANT CONCEPTS RELATING TO RESEARCH DESIGN Before describing the different research designs, it will be appropriate to explain the various concepts relating to designs so that these may be better and easily understood. 1. Dependent and independent variables: A concept which can take on different quantitative values is called a variable. As such the concepts like weight, height, income are all examples of variables. Qualitative phenomena (or the attributes) are also quantified on the basis of the presence or absence of the concerning attribute(s). Phenomena which can take on quantitatively different values even in decimal points are called ‘continuous variables’.* But all variables are not continuous. If they can only be expressed in integer values, they are non-continuous variables or in statistical language ‘discrete variables’.Age is an example of continuous variable, but the number of children is an example of non-continuous variable. If one variable depends upon or is a consequence of the other variable, it is termed as a dependent variable, and the variable that is antecedent to the dependent variable is termed as an independent variable. For instance, if we say that height depends upon age, then height is a dependent variable and age is an independent variable. Further, if in addition to being dependent upon age, height also depends upon the individual’s sex, then height is a dependent variable and age and sex are independent variables. Similarly, readymade films and lectures are examples of independent variables, whereas behavioural changes, occurring as a result of the environmental manipulations, are examples of dependent variables. 2. Extraneous variable: Independent variables that are not related to the purpose of the study, but may affect the dependent variable are termed as extraneous variables. Suppose the researcher wants to test the hypothesis that there is a relationship between children’s gains in social studies achievement and their self-concepts. In this case self-concept is an independent variable and social studies achievement is a dependent variable. Intelligence may as well affect the social studies achievement, but since it is not related to the purpose of the study undertaken by the researcher, it will be termed as an extraneous variable. Whatever effect is noticed on dependent variable as a result of extraneous variable(s) is technically described as an ‘experimental error’. A study must always be so designed that the effect upon the dependent variable is attributed entirely to the independent variable(s), and not to some extraneous variable or variables. 3. Control: One important characteristic of a good research design is to minimise the influence or effect of extraneous variable(s). The technical term ‘control’ is used when we design the study minimising the effects of extraneous independent variables. In experimental researches, the term ‘control’ is used to refer to restrain experimental conditions. 4. Confounded relationship: When the dependent variable is not free from the influence of extraneous variable(s), the relationship between the dependent and independent variables is said to be confounded by an extraneous variable(s). 5. Research hypothesis: When a prediction or a hypothesised relationship is to be tested by scientific methods, it is termed as research hypothesis. The research hypothesis is a predictive statement that relates an independent variable to a dependent variable. Usually a research hypothesis must contain, at least, one independent and one dependent variable. Predictive statements which are not to be objectively verified or the relationships that are assumed but not to be tested, are not termed research hypotheses. 6. Experimental and non-experimental hypothesis-testing research: When the purpose of research is to test a research hypothesis, it is termed as hypothesis-testing research. It can be of the experimental design or of the non-experimental design. Research in which the independent variable is manipulated is termed ‘experimental hypothesis-testing research’ and a research in which an independent variable is not manipulated is called ‘non-experimental hypothesis-testing research’. For instance, suppose a researcher wants to study whether intelligence affects reading ability for a group of students and for this purpose he randomly selects 50 students and tests their intelligence and reading ability by calculating the coefficient of correlation between the two sets of scores. This is an example of non-experimental hypothesis-testing research because herein the independent variable, intelligence, is not manipulated. But now suppose that our researcher randomly selects 50 students from a group of students who are to take a course in statistics and then divides them into two groups by randomly assigning 25 to Group A, the usual studies programme, and 25 to Group B, the special studies programme. At the end of the course, he administers a test to each group in order to judge the effectiveness of the training programme on the student’s performance-level. This is an example of experimental hypothesis-testing research because in this case the independent variable, viz., the type of training programme, is manipulated. 7. Experimental and control groups: In an experimental hypothesis-testing research when a group is exposed to usual conditions, it is termed a ‘control group’, but when the group is exposed to some novel or special condition, it is termed an ‘experimental group’. In the above illustration, the Group A can be called a control group and the Group B an experimental group. If both groups A and B are exposed to special studies programmes, then both groups would be termed ‘experimental groups.’ It is possible to design studies which include only experimental groups or studies which include both experimental and control groups. 8. Treatments: The different conditions under which experimental and control groups are put are usually referred to as ‘treatments’. In the illustration taken above, the two treatments are the usual studies programme and the special studies programme. Similarly, if we want to determine through an experiment the comparative impact of three varieties of fertilizers on the yield of wheat, in that case the three varieties of fertilizers will be treated as three treatments. 9. Experiment: The process of examining the truth of a statistical hypothesis, relating to some research problem, is known as an experiment. For example, we can conduct an experiment to examine the usefulness of a certain newly developed drug. Experiments can be of two types viz., absolute experiment and comparative experiment. If we want to determine the impact of a fertilizer on the yield of a crop, it is a case of absolute experiment; but if we want to determine the impact of one fertilizer as compared to the impact of some other fertilizer, our experiment then will be termed as a comparative experiment. Often, we undertake comparative experiments when we talk of designs of experiments. 10. Experimental unit(s): The pre-determined plots or the blocks, where different treatments are used, are known as experimental units. Such experimental units must be selected (defined) very carefully. DIFFERENT RESEARCH DESIGNS Different research designs can be conveniently described if we categorize them as: (a) Research design in case of exploratory research studies. (b) Research design in case of descriptive and diagnostic research studies. (c) Research design in case of hypothesis-testing research studies. 1. Research design in case of exploratory research studies: Exploratory research studies are also termed as formulative research studies. The main purpose of such studies is that of formulating a problem for more precise investigation or of developing the working hypotheses from an operational point of view. The major emphasis in such studies is on the discovery of ideas and insights. As such the research design appropriate for such studies must be flexible enough to provide opportunity for considering different aspects of a problem under study. Inbuilt flexibility in research design is needed because the research problem, broadly defined initially, is transformed into one with more precise meaning in exploratory studies, which fact may necessitate changes in the research procedure for gathering relevant data. Generally, the following three methods in the context of research design for such studies are talked about: (a) the survey of concerning literature; (b) the experience survey (c) the analysis of ‘insight-stimulating’ examples. The survey of concerning literature happens to be the most simple and fruitful method of formulating precisely the research problem or developing hypothesis. Hypotheses stated by earlier workers may be reviewed and their usefulness be evaluated as a basis for further research. It may also be considered whether the already stated hypotheses suggest new hypothesis. In this way the researcher should review and build upon the work already done by others, but in cases where hypotheses have not yet been formulated, his task is to review the available material for deriving the relevant hypotheses from it. Besides, the bibliographical survey of studies, already made in one’s area of interest may as well as made by the researcher for precisely formulating the problem. He should also make an attempt to apply concepts and theories developed in different research contexts to the area in which he is himself working. Sometimes the works of creative writers also provide a fertile ground for hypothesis formulation and as such may be looked into by the researcher. Experience survey means the survey of people who have had practical experience with the problem to be studied. The object of such a survey is to obtain insight into the relationships between variables and new ideas relating to the research problem. For such a survey people who are competent and can contribute new ideas may be carefully selected as respondents to ensure a representation of different types of experience. The respondents so selected may then be interviewed by the investigator. The researcher must prepare an interview schedule for the systematic questioning of informants. But the interview must ensure flexibility in the sense that the respondents should be allowed to raise issues and questions which the investigator has not previously considered. Generally, the experience collecting interview is likely to be long and may last for few hours. Hence, it is often considered desirable to send a copy of the questions to be discussed to the respondents well in advance. This will also give an opportunity to the respondents for doing some advance thinking over the various issues involved so that, at the time of interview, they may be able to contribute effectively. Thus, an experience survey may enable the researcher to define the problem more concisely and help in the formulation of the research hypothesis. This survey may as well provide information about the practical possibilities for doing different types of research. Analysis of ‘insight-stimulating’ examples is also a fruitful method for suggesting hypotheses for research. It is particularly suitable in areas where there is little experience to serve as a guide. This method consists of the intensive study of selected instances of the phenomenon in which one is interested. For this purpose the existing records, if any, may be examined, the unstructured interviewing may take place, or some other approach may be adopted. Attitude of the investigator, the intensity of the study and the ability of the researcher to draw together diverse information into a unified interpretation are the main features which make this method an appropriate procedure for evoking insights. Now, what sort of examples are to be selected and studied? There is no clear cut answer to it. Experience indicates that for particular problems certain types of instances are more appropriate than others. One can mention few examples of ‘insight-stimulating’ cases such as the reactions of strangers, the reactions of marginal individuals, the study of individuals who are in transition from one stage to another, the reactions of individuals from different social strata and the like. In general, cases that provide sharp contrasts or have striking features are considered relatively more useful while adopting this method of hypotheses formulation. Thus, in an exploratory of formulative research study which merely leads to insights or hypotheses, whatever method or research design outlined above is adopted, the only thing essential is that it must continue to remain flexible so that many different facets of a problem may be considered as and when they arise and come to the notice of the researcher. 2. Research design in case of descriptive and diagnostic research studies: Descriptive research studies are those studies which are concerned with describing the characteristics of a particular individual, or of a group, whereas diagnostic research studies determine the frequency with which something occurs or its association with something else. The studies concerning whether certain variables are associated are examples of diagnostic research studies. As against this, studies concerned with specific predictions, with narration of facts and characteristics concerning individual, group or situation are all examples of descriptive research studies. Most of the social research comes under this category. From the point of view of the research design, the descriptive as well as diagnostic studies share common requirements and as such we may group together these two types of research studies. In descriptive as well as in diagnostic studies, the researcher must be able to define clearly, what he wants to measure and must find adequate methods for measuring it along with a clear cut definition of ‘population’ he wants to study. Since the aim is to obtain complete and accurate information in the said studies, the procedure to be used must be carefully planned. The research design must make enough provision for protection against bias and must maximise reliability, with due concern for the economical completion of the research study. The design in such studies must be rigid and not flexible and must focus attention on the following: (a) Formulating the objective of the study (what the study is about and why is it being made?) (b) Designing the methods of data collection (what techniques of gathering data will be adopted?) (c) Selecting the sample (how much material will be needed?) (d) Collecting the data (where can the required data be found and with what time period should the data be related?) (e) Processing and analysing the data. (f) Reporting the findings. In a descriptive/diagnostic study the first step is to specify the objectives with sufficient precision to ensure that the data collected are relevant. If this is not done carefully, the study may not provide the desired information. Then comes the question of selecting the methods by which the data are to be obtained. In other words, techniques for collecting the information must be devised. Several methods (viz., observation, questionnaires, interviewing, examination of records, etc.), with their merits and limitations, are available for the purpose and the researcher may user one or more of these methods which have been discussed in detail in later chapters. While designing data-collection procedure, adequate safeguards against bias and unreliability must be ensured. Whichever method is selected, questions must be well examined and be made unambiguous; interviewers must be instructed not to express their own opinion; observers must be trained so that they uniformly record a given item of behaviour. It is always desirable to pretest the data collection instruments before they are finally used for the study purposes. In other words, we can say that “structured instruments” are used in such studies. In most of the descriptive/diagnostic studies the researcher takes out sample(s) and then wishes to make statements about the population on the basis of the sample analysis or analyses. More often than not, sample has to be designed. Different sample designs have been discussed in detail in a separate chapter in this book. Here we may only mention that the problem of designing samples should be tackled in such a fashion that the samples may yield accurate information with a minimum amount of research effort. Usually one or more forms of probability sampling, or what is often described as random sampling, are used. To obtain data free from errors introduced by those responsible for collecting them, it is necessary to supervise closely the staff of field workers as they collect and record information. Checks may be set up to ensure that the data collecting staff perform their duty honestly and without prejudice. “As data are collected, they should be examined for completeness, comprehensibility, consistency and reliability.”2 The data collected must be processed and analysed. This includes steps like coding the interview replies, observations, etc.; tabulating the data; and performing several statistical computations. To the extent possible, the processing and analysing procedure should be planned in detail before actual work is started. This will prove economical in the sense that the researcher may avoid unnecessary labour such as preparing tables for which he later finds he has no use or on the other hand, re-doing some tables because he failed to include relevant data. Coding should be done carefully to avoid error in coding and for this purpose the reliability of coders needs to be checked. Similarly, the accuracy of tabulation may be checked by having a sample of the tables re-done. In case of mechanical tabulation the material (i.e., the collected data or information) must be entered on appropriate cards which is usually done by punching holes corresponding to a given code. The accuracy of punching is to be checked and ensured. Finally, statistical computations are needed and as such averages, percentages and various coefficients must be worked out. Probability and sampling analysis may as well be used. The appropriate statistical operations, along with the use of appropriate tests of significance should be carried out to safeguard the drawing of conclusions concerning the study. Last of all comes the question of reporting the findings. This is the task of communicating the findings to others and the researcher must do it in an efficient manner. The layout of the report needs to be well planned so that all things relating to the research study may be well presented in simple and effective style. Thus, the research design in case of descriptive/diagnostic studies is a comparative design throwing light on all points narrated above and must be prepared keeping in view the objective(s) of the study and the resources available. However, it must ensure the minimisation of bias and maximisation of reliability of the evidence collected. The said design can be appropriately referred to as a survey design since it takes into account all the steps involved in a survey concerning a phenomenon to be studied. Definition of a case study It is an understatement that there is confusion among students, teachers, researchers, and methodologists about the definition and the main characteristics of case study research. Case study research is presented by some as a strictly exploratory research strategy in which nothing can be proven, most often by referring to the alleged impossibility to “generalize”. Others, such as Yin (1984, 1994, 2003), have claimed that the problem of “generalization” can be solved and that, therefore, theories can also be tested in (preferably) “multiple case studies”. A major difficulty for students and novice case study researchers is that proponents of these different perspectives give different meanings to similar methodological terms without clearly defining these meanings, making it almost impossible to grasp the nature of the debate and to infer solutions to problems in designing their own research. Ragin (1992) has argued that the work of any given case study researcher often is characterized by some hybrid of various approaches, which are usually difficult to disentangle. Most definitions of case study research, as found in the literature, are statements about the most frequently used measurement techniques (such as using “multiple sources of evidence”, or “qualitative methods”) and research objectives (such as “exploration”). Such definitions are attempts to capture in one statement the most important practical characteristics of a diverse array of studies that present themselves 1.1as case studies. Yin’s (2003: 13–14) definition is an example of such an all-inclusive descriptive definition: “A case study is an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between object of study and context are not clearly evident. It copes with the technically distinctive situation in which there will be many more variables of interest than data points, and as one result relies on multiple sources of evidence, with data needing to converge in a triangulating fashion, and as another result benefits from the prior development of theoretical propositions to guide data collection and analysis”. But one methodological characteristic by which a case study is distinct from other research strategies such as the survey is not captured in Yin’s work, or most other definitions found in the literature, namely the fact that a case study basically is an inquiry of only one single instance (the case), or sometimes a small number of instances, of the object of study. Yin’s and others’ definitions only highlight another distinctive characteristic of the case study, namely that in a case study the object of study or its environment are not manipulated (“real life context”).This definition wants to capture both, and the two really distinctive features of the case study in comparison to the survey and the experiment create our definition of the case study: A case study is a study in which (a) one case (single case study) or a small number of cases (comparative case study) in their real life context are selected, and (b) scores obtained from these cases are analysed in a qualitative manner. With “study” we mean a research project in which a practice-oriented or theory-oriented research objective is formulated and achieved. With a case we mean an instance of an object of study. (We will explain our concept of “object of study” in Chapter 3.) With “real life context” we mean the object of study as it occurs (or has occurred) in reality, without manipulation. With “analysis in a qualitative manner” we mean an analysis based on visual inspection of the scores of the case (in contrast to a statistical analysis). We distinguish two main types of case study: the single case study, a case study in which data from one instance is enough to achieve the research objective, and the comparative case study, a case study that requires data from two or more instances to achieve the research objective. The difference between the experiment and the case study is that the experiment manipulates instances, whereas the case study does not. An experiment is a study in which one or more variable characteristics of an object of study are manipulated in one or multiple (“experimental”) instances of an object of study and in which scores obtained in the experimental instance or instances are analysed. The survey also studies instances in their real life context. A survey is a study in which (a) a single population in the real life context is selected, and (b) scores obtained from this population are analysed in a quantitative (statistical) manner. Our definition of the case study reflects our idea that the survey and the case study are different in two aspects; (a) the number of instances from which data are collected for the analysis, and, consequently, (b) the method of data analysis. The instances and data can be available from earlier studies (allowing for a secondary analysis) or it may be necessary to select new instances and collect new data. The case study draws conclusions on the basis of a “qualitative” analysis (“visual inspection”) of scores from one single instance (single case study) or from a small number of instances (comparative case study), whereas the survey draws conclusions on the basis of a quantitative (statistical) analysis of data from a population with a large number of instances.The definition of the case study does not include statements on data collection or measurement techniques. In our view research strategies do not differ, in principle, in terms of methods of measurement. For all three research strategies discussed here, the data analysed can be quantitative or qualitative! Measurement methods that are usually associated with case studies, such as the “qualitative” interview and using “multiple sources of evidence”, could also be used in the other research strategies. Similarly, measurement methods that are usually associated with other research strategies, such as standardized questionnaires in surveys and quantitative measurements in experiments, could also be used in case studies. Principles of measurement and the quality criteria that apply to it, such as reliability and validity, apply to any measurement in any research strategy. Although in a case study quantitative data can be used to generate the scores to be analysed, the interpretation of scores of the (small number of) cases in order to generate the outcome of the study is done qualitatively (by visual inspection) and not statistically. The definition of the case study is applicable also to the study of instances (cases) of objects of study that existed or occurred in the past. Therefore, the study of instances of an object of study as occurring “in its real-life context” (as formulated in our definition) includes both the study of contemporary instances and of past instances. In this book, thus, we discuss the case study as a research strategy defined by the number of instances (N 1 or N small) that is studied as well as the “qualitative” or non-statistical method of analysis of all kinds of (quantitative and qualitative) data. Social survey/questionnaire What is the method? Social surveys are a questionnaire-based method of research that can produce both qualitative and quantitative information depending on how they are structured and analysed. This section focuses on the use of surveys to collect and analyse qualitative data. Many of the issues and considerations are the same as for the quantitative use of surveys, and more detail can be found in the earlier section of this handbook. When should it be used? Questionnaire surveys can be used in a wide range of settings and to gather a variety of different types of information. You may be evaluating a programme in which a wide range of projects have been commissioned, and want to gather the views of a wide range of project managers, or you may be measuring the impact of an initiative on the business community in a specific geographical area. A small-scale qualitative survey may be conducted to explore in more detail the findings of qualitative research. What do I need to consider? Many of the considerations for a social survey are the same as for a quantitative survey, however we define a social survey as one where less statistical rigour is required, where sample sizes are not as large, and with results not expected to be significant of the wider population. A social survey may have a greater focus on collecting rich and detailed qualitative data. Population A number of questions about the proposed population for a social survey need to be considered. Such as are there language issues? And Qualitative Research Methods Is this the real life? yes/no If no, please answer question 2 2. Is it just fantasy? yes/no Please provide reasons for your answer .................................................. .................................................. .................................................. .................................................. ..................................................what are the geographic restrictions? These are the same issues as for quantitative surveys. Sampling The sample is the section of the wider population that will be engaged in the survey. Detailed consideration of sampling still needs to be made even when not striving for statistical significance. It is still important to understand who the respondent is and what your sampling frame is going to be. Format A social survey will usually be a cross-sectional survey used to gather information on a small sample population at a single point in time. An example of a cross-sectional survey would be a questionnaire that collects data on peoples’ experiences of a particular initiative. However, a qualitative survey could equally be used in a longitudinal study, perhaps returning to particular individuals over time to measure the impact of an intervention on the direction of someone’s life. Questions There are a whole range of questions to be asked in relation to survey design, such as: What types of questions can be asked? How complex will the questions be? Will screening questions be needed? Can question sequence be controlled? Will lengthy questions be asked? Will long response scales be used? A social survey will be more interested in qualitative findings, in recording peoples’ opinions and perceptions, and therefore will make more use of open questions where respondents can give their own responses to a set question. Open questions will begin with what, why, how, or describe, to elicit rich qualitative information. Open questions can be used in a variety of ways: Usage Example As a follow-on from closed questions, to develop a more detailed response. ‘If answering yes to question 7, please provide the reasons for this’ To find out more about a person, their thoughts, needs, problems, etc. Why is that so important to you? To get people to realise the extent of their problems. What effect does this have on your family life? To get people to reflect on the impact of something or some change. How has this made a difference to you? Administration The costs, required facilities, time, and personnel needed to conduct an effective survey are often underestimated, even when it is not on a large scale. There should be an administrative system in place to deal with the questionnaires for when they are returned/completed. This may include numbering the questionnaires, recording what action has been taken with them, entering the results into a spreadsheet/database etc. How should it be used? Surveys can be carried out by phone, post, email, website or face-toface, for detailed pros and cons of these delivery methods see the earlier section on qualitative surveys. In collecting rich qualitative survey data, the most effective method would be via face to face, administered surveys, as the researcher would be able to use prompts to encourage people to give more detailed answers. This does however introduce a bias, which needs to be understood and controlled as much as possible, i.e. by using standard prompts. In qualitative surveys, it is necessary that the interviewer conduct the interview with total objectivity, so that respondents are not influenced by any outside source in their responses. For this reason, interviews should be conducted by welltrained and qualified interviewers. What is the output? The data that a social survey can produce is very much dependent on how the questionnaire is constructed. However, the data can be very useful for providing an overall picture of the way in which a project or programme is being implemented and how effectively it is impacting upon its target audience. Qualitative data output will be in a text, audio or picture format, and each answer may be very different from another. This can make collection of data more difficult, and a way of collating data needs to be considered early in the process. How should it be analysed? The Quantification of Qualitative Survey Data Surveys can be analysed by collating the frequency of responses to each of the questions on the survey form. This can be done manually using a “frequency table”, which can be easily set up on an Excel spreadsheet to analyse descriptive statistics. QSR NUD*IST and NVIVO are qualitative data analysis packages, which enable non-statistical information from interviews, group work, observations, audio, video, pictures or documents to be analysed according to chosen criteria. For example, it is possible to use the package to ‘pull out’ all material relating to key words or phrases (e.g. neighbourhood renewal) and then sub-divide the data into more specific areas of analysis (e.g. statement of use, problems, projects). This is a powerful piece of software that can provide clarity to wide range of often complicated written or media materials. Case study: Using surveys to evaluate a project A programme targeted on helping young people back into work through training wants to evaluate how well it is achieving its objectives. It uses a survey to canvas the views of young people who have been on the programme to date. The survey asks them closed questions about what training they have attended and how useful they have found the training (on a scale of 1:4). The survey also uses open questions to ask young people about what their plans are for the future as a result of the training (i.e. has it helped them to consider applying for full time work? Or further education opportunities?). The qualitative data is analysed and this shows that the young people have gained in confidence, are looking to go into further education or training or have already secured job interviews in a range of occupational fields, however there is a distinct focus on work in the field of construction. The results of the survey are analysed and this provides conclusions about overall success of the programme, which allows the programme manager to draw conclusions and consider design issues for making the programme more effective in the future. Further Reading See Question Bank for details of question design - http://qb.soc.surrey.ac.uk/ Computer Assisted Qualitative Data Analysis (CAQDA) – http://caqdas.soc.surrey.ac.uk/ - provides practical support, training and information in the use of a range of software programs designed to assist qualitative data analysis. Also provides various platforms for debate concerning the methodological and epistemological issues arising from the use of such software packages. Research Observatory, University of the West of England - http://ro.uwe.ac.uk/RenderPages/RenderHomePage.aspx - the site is divided into topic areas with each topic area containing a number of learning units and a collection of resources about a particular subject related to research. Interviews What is the method? One of the most popular and frequently used methods of gathering information from people about anything is by interviewing them. It is also the most popular method used within the social sciences. There is a continuum of formality around interviewing and it covers a multitude of techniques, from informal “chats” maybe arranged as “vox-pops” right through to highly structured, formal interviews, taped and transcribed. The different types and styles of interview elicit very different types of information. Conducting interviews is an interpersonal process and as an investigator you must be very aware of your own behaviours and assumptions in the context. Interviews are not “neutral” social spaces and you must be respectful and maintain appropriate boundaries at all times. What do I need to consider? Interviews are a qualitative method of research often used to obtain the interviewees’ perceptions and attitudes to the issues. The key issue with interviewing is making decisions about who are the key people to talk to and what type of interview are you going to use. Interview Style There are three clearly identifiable styles of interview- structured, semistructured and unstructured: Structured - Follows a set of specific questions, which are worked through systematically. This type of interview is used when the researcher wishes to acquire information where the responses are directly comparable. Semi-structured - This is a more commonly used interview technique that follows a framework in order to address key themes rather than specific questions. At the same time it allows a certain degree of flexibility for the researcher to respond to the answers of the interviewee and therefore develop the themes and issues as they arise. Unstructured - This method of interview does not follow any predetermined pattern of questions or themes. Rather, the interviewer will address the issues as they emerge in the interview. The method is useful when the researcher wishes to explore the full breadth of a topic. Interview Type These are some of the types of interviews: Fact finder - This type of interview is used to obtain specific information from an interviewee and usually includes structured or standardised interview questions (the wording of the questions and the order in which they are asked is the same). It is used when some information is already known and there is a need to gain a more in-depth insight. An example of when a fact finder interview would be appropriate is when interviewing a project officer as part of an evaluation of their project. Quantitative (or ‘hard’) information is usually already known (such as outputs and funding data), therefore the interview could be used to discover qualitative information that the hard data cannot portray, such as the ‘softer’ outcomes of the project. Idea generator - In many respects, this type of interview is the opposite of the fact finder interview. It is used when the interviewer has no preconceptions about what might be discovered over the course of the interview and results can be used to set the parameters or framework for the study. Interview questions are loosely structured allowing maximum flexibility to explore a range of issues. Idea generator interviews are usually applied at the start of a research project in order to discover and explore issues from a particular group or community. For example, in order to develop a community cohesion strategy, idea generator interviews may be used to find out what community cohesion means to different groups in the community. Exploratory - These are the most frequently used type of interview as they are relevant to most types of research project. They are usually conducted with representatives that have a strategic role to play in the research. These types of interview require some degree of prior knowledge about the research subject as they are about testing hypotheses, making connections between other elements of the research, ensuring the strategic fit and progressing the findings of the research forward (e.g. senior officials from a local authority may be interviewed using this method in order to find out future plans and priorities and how they fit in with others’ plans and priorities). Experiential - This type of interview aims to draw out people’s feelings, perceptions and experiences over a specific period of time (e.g. the duration of a regeneration programme or project). This provides rich, in-depth material about how the subject under investigation has affected an individual’s life on a personal level. Experiential interviews may be used to elicit information from people who have benefited from a community project or who live in an area that has received regeneration monies.