Customizing Survey Methods to the Target Population: Innovative Approaches to Improving Response Rates and Data Quality among Hispanics Brian Evans, Rosanna Quiroz, Leslie Athey, Joseph McMichael, Vicky Albright RTI International Michelle O Hegarty, Ralph S. Caraballo, Centers for Disease Control and Prevention (CDC) May 18, 2008 RTI International is a trade name of Research Triangle Institute 1 AAPOR May 2008
Overview Background on growth of Hispanics in U.S. Traditional survey methods of Hispanics Hispanic/Latino Adult Tobacco Survey Challenges and Solutions Results of survey Applying methods to other work AAPOR May 2008 2
Hispanic growth U.S. Census reporter in 1970, there were 9.6 million Hispanics in the U.S. 1 As of July 1, 2006, grown to 44.3 million Hispanics accounted for 50% of nation s growth between 2000 2006 Hispanic growth rate was 3 times that of the total population between 2000 2006 Given this growth, Hispanics are becoming increasingly important as a survey population 1 Owen, Anna M. Hispanics in the U.S. http://www.census.gov/population/www/socdemo/hispanic/hispanic_pop_presentation.html AAPOR May 2008 3
Traditional Hispanic Survey Methods While American and Hispanic cultures differ, interviewed using same methods Translation Single direct translation Back translation Translation by committee Pre-testing Cognitive interviews However, interviewing Hispanics requires much more than just translation of the questionnaire AAPOR May 2008 4
Hispanic/Latino Adult Tobacco Survey Conducted for the Office of Smoking and Health (OSH) in the Centers for Disease Control (CDC) Adult Tobacco Survey is national survey conducted by telephone Hispanic/Latino Adult Tobacco Survey (H/LATS) created to address under-representation concerns of Hispanics in National Survey In-person PAPI survey conducted in the colonias in El Paso, TX AAPOR May 2008 5
Colonias in El Paso AAPOR May 2008 6
Colonias (cont d) Unincorporated areas along the Texas-Mexico border Many immigrants to the United States Many households lack public utilities such as water, electricity, etc. Wide range of household structures Fear of outsiders AAPOR May 2008 7
Colonias (cont d) AAPOR May 2008 8
Colonias (cont d) AAPOR May 2008 9
Four Main Challenges Sample Creation Communication Barriers Respondent Experience with Surveys Reluctance to Participate AAPOR May 2008 10
Sample Creation Challenge Needed to create area probability sample on budget that didn t afford it Colonias added following challenges; Unknown/insufficient telephone coverage We could not use postal lists because most of the residents use P.O. Boxes to collect mail Field enumeration ( Counting and listing ) was not an option AAPOR May 2008 11
Sample Creation - Solution We invented Geoframe TM which allowed for creation of sample frame on time and within budget Consists of digital photograph of dwelling unit coupled with GPS coordinates Allows for creation of dwelling unit sheets that allows interviewers to easily identify and locate selected dwelling units More detailed presentation about GeoFrame TM by Leslie Athey et. al. titled Geoframe TM : A New Method of Field Enumeration AAPOR May 2008 12
CASEID: 12345678 AREA: South Side AAPOR May 2008 13
AAPOR May 2008 14 Right Side [B01_0002] Right Side [B01_0003] Left Side [B01_0005] Left Side [B01_0006]
Communication Barriers - Challenge Survey conducted with Spanish as primary language Standard terms we use in English are not necessarily the same in Spanish Technical terms could be difficult for respondents to understand Culturally differences could be problematic in development of questionnaire AAPOR May 2008 15
Communication Barriers - Solution OSH met with 10 researchers with expertise in tobacco control and research experience with Hispanic/Latino populations H/LATS was tailored to ensure understanding of questionnaire in Hispanic communities Translated by 3 professional translators Survey was cognitively tested in 7 major cities AAPOR May 2008 16
Communication Barriers Solution (cont d) How was it tailored? Researchers studied: Dialects of Latin American countries Educational systems of Latin American countries Acculturation among Latin Americans Specific questions specific to Hispanic/Latino populations were added to the survey Country of birth, educational and income levels, language preference AAPOR May 2008 17
Respondent Experience with Surveys - Challenge Assumed that residents of the colonias did not have much experience with surveys We did not know much about the colonias, the culture, or whether our standard interviewing approaches would work Did discover that the 2000 Census had very difficult time interviewing in the colonias 2 Assumed that interviewers and respondents might have difficulty with: Skip patterns, scales, appropriate response categories, etc. 2 The Enumeration of Colonias in Census 2000: Perspectives of Ethnographers and Census Enumerators. Final Report. Manuel de la Puenta, David Stemper. U.S. Census Bureau. September 22, 2003. AAPOR May 2008 18
Respondent Experience with Surveys - Solution Texas A&M Colonias Program Promotoras provide outreach and services to the residents of the colonias Very familiar with the culture of the colonias Recognized in the community and program has a good reputation in the colonias Compared to other outreach programs, regarded as neutral agency (i.e. not in it for themselves) AAPOR May 2008 19
Respondent Experience with Surveys Solution (cont d) Interviewer training Conducted entire training in Spanish Used multiple training techniques Multiple training sections on questionnaire administration Multiple training sections on avoiding bias Solicited interviewer feedback in training about what questions might be difficult to administer AAPOR May 2008 20
Reluctance to Participate - Challenge Assumed the following reasons why residents of the colonias might be reluctant to participate 3 Suspicion of government involvement Research topic Low level of self-disclosure Legitimacy of the study Type of benefit to be received Interviewer appearance 3 Research with Hispanic Populations (Applied Social Research Methods). By Gerardo Marin, Barbara Vanoss. Volume 23. 1991 AAPOR May 2008 21
Reluctance to Participate - Solution Interviewer Training Sections of training focused on door approaches and effective contacting techniques to alleviate concerns about Border Patrol, Tax Collectors, etc. Allowed interviewers a little freedom in explaining research topic Reviewed in training benefits study would have to the residents of the colonias Respondent incentive AAPOR May 2008 22
Reluctance to Participate Solution (cont d) Texas A&M Colonias Program Provided uniforms and badges for interviewers Required they work in teams of 2, mainly for safety reasons Benefits included display of importance of survey, assistance in answering questions In some instances, provided vehicles with Texas A&M logo Recognized by respondents in the colonias AAPOR May 2008 23
Results Achieved following provisional rates: ~ 75% response rate 92% eligibility rate 84% screening rate Only 1.2% refusal rate (26 refusals out of 2100 cases) Only 1 case where they could not find the household Higher quality of data than expected AAPOR May 2008 24
How solutions can be used in future work Continuing to test GeoFrame TM in other circumstances Works well in areas with no mailing list coverage and little/no telephone coverage Specialized development of questionnaire We found that a little extra effort and preparation provided a questionnaire that was culturally acceptable and understandable for respondents AAPOR May 2008 25
How solutions can be used in future work (cont d) Training Specialize training to target population Conducted entire training in Spanish Local groups For studies in specific areas, these groups can provide many insights to assist in administration of study Understand population s culture and needs AAPOR May 2008 26
Contact Brian Evans RTI International evans@rti.org AAPOR May 2008 27