IPDET 
                                             Handbook 
                                             Module 8 
                                             Data Collection Methods 
                                                                         Introduction 
                                             You have now learned about evaluation questions and 
                                             evaluation designs to match these questions. In this module, 
                                             you will be looking at how you will collect the data to answer 
                                             your evaluation questions. The module begins with information 
                                             you need to know about data collection and ends with a 
                                             “Toolkit” of ways to collect data. 
                                             This module has five topics. They are: 
                                                  •   Data Collection Strategies 
                                                  •   Data Collection General Rules 
                                                  •   Key Issues about Measures 
                                                  •   Quantitative and Qualitative Data 
                                                  •   Common Data Collection Approaches: The Toolkit. 
                                                                                                           Participatory Data
                         Intervention                                              Collect                 Available Data 
                                          Evaluation                                                       Observation 
                              or                        Design    Approaches                               Surveys 
                                          Questions                                                        Questionnaires 
                            Policy                                                                         Focus Groups 
                                                                                                           Diaries, Journals, Self- 
                                                                                                              reported Checklists 
                                                                                                           Expert Judgement 
                                                                                                           Delphi Technique 
                                                                                                           Citizen Report Cards 
                           Module 8 
                                              
                                             Learning Objectives 
                             Â By the end of the module, you should be able to: 
                                                  •   describe data collection strategies 
                                                  •   list the general rules for data collection 
                                                  •   describe key issues about measures, including: 
                                                      credibility, reliability, validity, and precision 
                                                  •   describe common approaches to data collection and 
                                                      when to use these approaches, including: participatory 
                                                      data, available data, observation, surveys, interviews, 
                                                      questionnaires, focus groups, diaries, journals, and self-
                                                      reported checklists, expert judgment, and citizen report 
                                                      cards.   
                           Page 346                              International Program for Development Evaluation Training − 2007 
                                                          Data Collection Methods 
                          Key Words 
                          You will find the following key words or phrases in this module. 
                 ³ Watch for these and make sure that you understand what they 
                          mean and how they are used in the course. 
                                participatory data collection 
                                structured data collection 
                                unstructured data collection 
                                validity 
                                reliability 
                                precision 
                                quantitative approach 
                                qualitative approach 
                                bias 
                           triangulation 
                                global positioning system (GPS) 
                           available data 
                           unobtrusive observer 
                           participant observer 
                           obtrusive observer 
                           in-person interview 
                           self-administered questionnaire 
                                pilot or pilot test 
                           structured survey 
                           semi-structured survey 
                           response rate 
                           cover letter 
                                volunteer sample or self-selected sample 
                           focus group 
                           diary 
                           self-reported checklist 
                           rich data 
                           Delphi technique 
                                nominal group technique 
                                citizen report cards 
                International Program for Development Evaluation Training − 2007 Page 347 
                           Module 8 
                           Data Collection Strategies 
                                             You have a number of ways to collect data but there is no one 
                                             single best way. The decision about which approaches to use 
                                             depends upon: 
                                                  •   what you need to know 
                                                  •   where the data reside 
                                                  •   resources and time available 
                                                  •   complexity of the data to be collected 
                                                  •   frequency of data collection. 
                                             If you need to know whether villagers who participated in an 
                                             adult literacy intervention can read and write better than those 
                                             who did not participate, you will need to find a way to collect 
                              1 
                                             data about their reading and writing skills. Maybe you can get 
                                             access to some samples of their writing before and after the 
                                             intervention, or perhaps you can give them a reading and 
                                             writing test and compare scores before and after the program. 
                                             If you need to know if literacy intervention participants are 
                                             more actively engaged in their children’s education, you might 
                              1 
                                             try observing parent-child interactions, or you might ask 
                                             children, parents, and teachers whether this is the case, before 
                                             and after the program. 
                                             If you need to know if the literacy program participants were 
                                             satisfied with the quality of the literacy workshops and follow-
                                             up, then you will need to gather the opinions of the 
                              1 
                                             participants. A structured interview of participants is one 
                                             option (or a survey, if literacy levels are high enough).  
                                             As mentioned briefly in and earlier module, the choice of 
                                             methods hinges partly on the evaluation question to be 
                                             answered, partly on how well the intervention is understood, 
                                             and partly on the time and resources available. There is a 
                                             trade-off here between in-depth understanding that comes 
                                             from a case study, for example, and having data collected in a 
                                             systematic and precise way through a survey that allows valid 
                                             comparisons to be made. 
                                             As each evaluation question has its own mini-design, different 
                                             data collection methods may be used to answer different 
                                             questions within the over-all evaluation framework. 
                           Page 348                              International Program for Development Evaluation Training − 2007