Incorporating a Finite Population Correction Factor into American Community Survey Variance Estimates

Transcription

1 Sectio o Survey Research Methods JSM Icorporatig a Fiite Populatio Correctio Factor ito America Commuity Survey Variace Estimates Michael Starsiic U.S. Cesus Bureau, Washigto, DC, 33 Abstract The America Commuity Survey (ACS) produced its first atioide 5-year estimates i, usig sample data from 5 through 9. With five years' orth of sample, the combied sample size i some areas ould be large eough that a fiite populatio correctio (FPC) factor might have a oticeable impact o variaces. This paper discusses the methodology used to icorporate a FPC factor ito the 5-year ACS variace estimates, ad ho the method as adapted to accout for the subsamplig of orespodets. Results comparig the impact o the variace of usig the FPC across a broad spectrum of estimates ad geographic areas are also preseted. Prelimiary ork idicated improvemets i the stadard error estimates of betee to ad four percet could be achieved. Key Words: America Commuity Survey, variace estimatio, fiite populatio correctio. Itroductio The America Commuity Survey (ACS) is a cotiuous mothly survey that collects the data historically collected by the deceial cesus log form sample. Full implemetatio of the ACS bega i Jauary 5, ith the sample expadig to a size of approximately three millio housig uit addresses, ith sample selected from all couties ad couty equivalets i the 5 states, the District of Columbia, ad Puerto Rico. A sigle year s orth of sample i the ACS is ot adequate to publish estimates for all geographic areas for hich log form estimates ere published i Cesus. Istead, sigle-year estimates are published oly for geographic areas ith a populatio of at least 65,. For smaller areas, several years of ACS sample are pooled together to create period estimates. The first estimates based o three years of pooled ACS data ere published i 8 for all areas ith a populatio of at least, usig data from 5 through 7. All geographic areas, icludig Cesus tracts ad block groups, ca be published usig five years orth of pooled ACS data. The five-year data ere first published i usig sample from the years 5-9. (U.S. Cesus Bureau, b) The ACS had ot previously used a fiite populatio correctio (FPC) factor i its variace estimatio methodology. Oe-year ACS samples are ot large eough for a This report is released to iform iterested parties of ogoig research ad to ecourage discussio of ork i progress. The vies expressed are those of the author ad ot ecessarily those of the U.S. Cesus Bureau. 36

2 Sectio o Survey Research Methods JSM FPC to have much impact o variaces of published estimates. Hoever, ith 5-year ACS estimates, up to 5 percet of housig uits i certai blocks may have bee i sample over the 5-year period. The Cesus log form, hich used essetially the same variace estimatio methodology as the ACS does curretly, did iclude such a factor. The motivatio behid this research ito applyig a FPC factor as the impedig release of the first ACS 5-year sample data (usig sample from 5 through 9), ad the belief that usig a FPC factor could eable more accurate estimates of the variace, particularly for small areas. Usig a FPC should decrease most variace estimates. The true samplig error is, of course, uchaged, but the FPC should allo a more accurate estimate of the variace. This paper ill describe the FPC methodology used by the ACS, ad ill discuss the improvemets i the estimatio of stadard errors see for the 5-year 5-9 ACS data products due to the use of the FPC.. Statistical Backgroud of the FPC For a simple radom sample ithout replacemet, the variace of a sample estimate is ( ) N s v Yˆ N here is the sample size, N is the populatio size, /N is the samplig fractio, ad the ( /N) multiplicative factor is the FPC. (Cochra, 977) As the sample size approaches the populatio size, the factor ad the variace decrease. This makes ituitive sese if a large proportio of a populatio is icluded i sample, the variability of the resultig estimates should be reduced. If all of the populatio is i sample, the FPC ad the variace are zero, as there is o samplig variability. If the samplig fractio is small, the the FPC makes very little differece to the variace calculatio. For may large surveys, here N may be the populatio of the Uited States, is much, much smaller, ad the FPC is igored he calculatig the variace. Cochra (977) suggests that the FPC ca be igored for samplig fractios up to percet. I 9, the umber of iitially selected addresses for the ACS sample as,897,56, hile the July, 9 official estimate of housig uits as 9,99,96, for a samplig rate of about.3 percet. (U.S. Cesus Bureau, a) Usig this rate i a FPC ould decrease the stadard error (the square root of the variace) by about. percet, hich is ot much of a impact. FPC theory, hoever, is depedet o complete respose. If a 9 percet sample of the populatio is selected, but oly five percet respods, a FPC of 9/. is t appropriate. Istead, let the effective sample size, eff, be defied as the umber of actual respodets, or r, here r is the respose rate. The, eff ould be used i the FPC istead of. ( Y ) ˆ r N s N r v r 36

3 Sectio o Survey Research Methods JSM The orespose meas that this versio of the variace estimator ith the adusted FPC is o loger a ubiased estimate of the variace, hoever. 3. Cesus Log Form FPC Applicatio The Cesus log form selected a sample of about -i-6 housig uits. Hoever, the samplig rate varied by Cesus block, ith target rates of -i-8, -i-6, -i-, ad -i-. (U.S. Cesus Bureau, 3) With samplig rates this large, a FPC could have a oticeable impact o the magitude of the variace estimates. The log form variaces ere calculated usig the successive differeces replicatio methodology (Fay & Tra 995), hich as developed to better hadle systematically selected samples such as the log form, the ACS, ad the Curret Populatio Survey. A pre-specified umber of replicate eights (5 for the log form, for the ACS) are created by first applyig sets of replicate factors to each observatio s iitial base eight, ad the reprocessig the eightig methodology idepedetly o each set of replicate iitial eights. The variace of a estimate is calculated from the sum of the squared differeces betee the productio estimate (hich as calculated ithout replicate factors) ad the replicate estimates, created from the sets of replicate eights (deoted by k, belo). Var k k ( x ) ( x i x ) i The replicate factors, f (i replicate #, sample uit #), i a typical successive differeces variace estimator are defied as follos:.5.5 f + ai, a + / / here a ± ad a ± are the appropriate cells from a Hadamard matrix. The log form applied a FPC factor directly to the replicate factors: f.5.5 ( a a ) FPC, here FPC N i, + here eff is defied as the observed umber of log form sample respodets, ad N is defied as the ucorrected cesus cout (Gbur & Fairchild, ). The FPC is typically applied as a multiplicative factor outside the variace formula. Hoever, uder certai simplifyig assumptios, the variace usig the replicate factors after applyig the FPC factor is equal to the origial variace multiplied by the FPC factor. We ca explore the algebra associated ith this FPC adustmet usig the simplified assumptio that there are o further eightig adustmets after the iitial eights are assiged. First, defie.5.5 g ai, a eff 363

4 Sectio o Survey Research Methods JSM So, f + g. Let, be the productio eight for the th sample uit. The the replicate eight is, f, ( + g ) No let s defie the estimate ad replicate estimates as eighted sums over the uits i sample. x, ad x i Usig the ACS ith its replicates for this example, the variace of x is the Var( x ) ( x ) i x, (, ) i (, ( + g ), ) i i, g The FPC-adusted replicate factor is.5.5 f + a a i i ( ) FPC + g FPC i, The variace usig the FPC-adusted replicate factors is Var ( x SE ( x ) ) ( xi x ), (, ) i i i FPC i FPC Var( x ) FPC SE( x ( ( + g FPC ) ),, g ), g i FPC, i So, i this simplified example, applyig the FPC to the replicate factors yields exactly the origial stadard error (SE) multiplied by the square root of the FPC. It is expected that the reductio i the variace estimate ill carry though the eightig, ad ill be see he the fial eights are used. 36

5 Sectio o Survey Research Methods JSM Why as the FPC adustmet made to the replicate factor ad ot o the outside of the variace calculatio? For simplicity, this example oly icluded oe FPC adustmet. The log form applied idividual FPCs for each log form eightig area (frequetly equivalet to a cesus tract). For a estimate that crossed multiple eightig areas, the FPCs ould have to be adusted depedig o hat eightig areas the observatios comprisig the estimate ere icluded i. By applyig the FPC to the replicate factor, that step has already bee take care of, ad the FPC adustmet does ot eed to be recalculated for each estimate.. Applicatio of the FPC to the ACS Like the log form, the ACS target samplig rates differ at the cesus block level. Although the overall samplig rate for 9 as ust.3 percet, target block rates raged from.9 percet to percet. The block-level samplig rates ere based i part o the expected umber of occupied housig uits i the govermetal uits (e.g. couties, places, mior civil divisios (MCDs), America Idia areas, etc.) that cotais the give block. The percet samplig rate is assiged to blocks that are cotaied ithi a govermetal uit hich has or feer estimated occupied housig uits. For these blocks, the ACS 5-year estimates, based o 5-years orth of sample, ill have combied samplig rates of up to about 5 percet. Table shos selected 5-9 combied samplig rates. Table : ACS 5-9 Samplig Rates for Selected Areas 5-9 Combied Samplig Rate Category Samplig Rate Blocks i smallest govermetal uits 5.% (< occupied housig uits) Blocks i smaller govermetal uits 33.5% (-799 occupied housig uits) Blocks i small govermetal uits 6.9% (- occupied housig uits) Source: 5-9 America Commuity Survey, The samplig rates for those three classes of blocks seemed large eough that tryig to apply a FPC could be orthhile. All other blocks have a combied 5-year samplig rate of about percet or less. (U.S. Cesus Bureau, b) Research (ad experiece) has sho that ACS variace estimates for small areas ca be very high. (Starsiic & Tersie, 7) The highest samplig rates are i blocks i small govermetal uits. So, the applicatio of the FPC should result i the largest reductios i the estimate of the SEs i those small govermetal uits. The ACS differs from the log form i havig a additioal phase of samplig. Housig uits i the iitial sample hich do ot respod by mail or by a computer-assisted telephoe itervie (CATI) are subsampled for a computer-assisted persoal itervie (CAPI) at rates roughly betee -i-3 ad -i-3. (U.S. Cesus Bureau, a) Because of the ACS CAPI sub-samplig, a sigle FPC factor as ot deemed suitable. To develop a ACS applicatio e cosidered basic features of the ACS samplig ad eightig methodology, maily the samplig of mail/cati o-respodets. To develop a alterate method, e cosidered the mail/cati respose uder a fixed respose 365

6 Sectio o Survey Research Methods JSM model. The fixed respose model assumes to distict populatios, oe cotaiig uits that, if i sample, ould alays respod, ad the other cotaiig uits that ould ever respod. (Bethlehem, 9) Those i the ACS sample ad respodig by mail or CATI ould be i the first stratum, hile the CAPI respodets (ho ere o-respodets by mail ad CATI), ould be i the secod stratum. Separate FPC factors ould be computed for each stratum. mail + CATI F, ad N R CAPI F, here N R ( ) R mail + samp CATI Here, N is the ueighted sample uiverse cout, ad mail, CATI, ad CAPI are the ueighted couts of respodets by mode. R is the eighted proportio (usig the ubiased samplig eights) of those ho respod by mail or CATI, so N R is a estimate of the populatio of the respodet stratum, ad N (-R) is a estimate of the populatio of the orespodet stratum. For mail ad CATI respodets, FPC mail/cati F ad for CAPI respodets, FPC CAPI F Weightig areas for the ACS are defied at the couty level, ulike the log form. Sice the ACS iitial samplig rate variatio occurs at smaller areas tha couty, it as decided to calculate the FPC factors at the cesus tract level. Tracts ere thought to be small eough to capture some of the variability, ad are geerally equivalet to the log form's eightig areas. Also, the FPC as oly applied to housig uits ad the housig uit populatio. No FPC as applied to persos i group quarters If the decisio had bee made to use a overall FPC i frot of the variace estimatio formula, FPC mail/cati ould have bee.895, ad FPC CAPI ould have bee.96. A combied, aive FPC value that igored the subsamplig for o-respose ould have bee.96. This ould have reduced SEs about 3.8 percet. Hoever, this ould have igored the local variability i the samplig rate that e ere tryig to capture usig the replicate factor adustmet method described above. With the methodology decided, a simulatio as coducted usig year data. A set of replicate eights ere created usig the FPC factor. SEs for a umber of estimates ere computed for selected geographic types ad areas, ad directly compared agaist the productio SEs hich did ot have the FPC factors. The ith-fpc SEs ere, o average,.8 to.5 percet less tha the ithout-fpc SEs, hich as about the chage that as expected. The expectatio as that the decrease i the 5-year SEs ould be eve larger. Also, very fe 3-year estimates sa a icrease i their SEs. With the 366

7 Sectio o Survey Research Methods JSM successful results from the simulatio, the decisio as made to apply the FPCs to the 3- year 7-9 ad 5-year 5-9 data tabulatios. Oly aalysis of the 5-year 5-9 SEs is icluded i this paper. 5. Evaluatio of 5-Year FPC Impact As ith the simulatio, the aalysis of the productio data used to sets of SEs: the actual 5-year 5-9 productio data created from replicate eights ith the FPC adustmets, ad oe set for this research usig replicate eights ithout the FPC adustmets. (Sice the FPC oly affects the replicate eights, the estimates from the ith-fpc ad ithout-fpc datasets are idetical.) The evaluatio ould be coducted o the ACS data profile. The data profile cotais estimates of over uique characteristics, ad covers a broad set of subects (such as age, race, educatio, ad icome) ad estimate types (icludig couts, percets, meas, ad ratios). Published data profiles (usig the FPC) ere available for each of the geography types e ated to ivestigate atio, state, couty, MCD (icludig Cesus Couty Divisios), place, tract, ad America Idia/Alaska Native (AIAN) area, hich also iclude Haaiia home lads. The profile s SEs ere recreated for these geographic areas usig the ithout-fpc set of replicate estimates, to provide a direct compariso ith the published SEs. Certai estimate values ere omitted from the evaluatio because their SEs are hadled as special cases that do ot use the replicate eight values. These iclude estimates of zero (for couts ad percets), estimates of percet, ad estimates hich are cotrolled durig the eightig process to equal a value from the idepedet populatio cotrols used by the ACS. Table shos the distributio of the ratio of the SE usig the FPC to the SE ithout the FPC. Sice the FPC is less tha oe, this ratio should be less tha oe as ell, if the adustmet orked as expected. Table : Distributio of SE Ratio by Summary Level Summary Level st 5th Media 75th 99th Natio State Couty MCD Tract Place AIAN Area Source: 5-9 America Commuity Survey (special tabulatio), As expected, the atio ad state SEs are the least affected by the adustmet, ith media ratios aroud.97, or, equivaletly, a 3 percet decrease. The largest decreases are see at the MCD ad AIAN Area levels. Small govermetal uits are eligible for the highest iitial samplig rates, so it makes sese that these areas also see the largest chage. A relatively small umber of estimates do sho a icrease i the SE for several of the geographic area types, the 99th percetile of the SE ratio is at or over., 367

8 Sectio o Survey Research Methods JSM but these are ot large chages. Hoever, e could fid o systematic patter i the estimates ith icreased SEs. We believe this is ust the variace of the variace. Does the FPC adustmet have a differetial impact o differet types of estimates? For example, maybe SEs of percet estimates ere less affected tha SEs of couts. Table 3 shos the media of the ratio of SEs by the type of estimate for tracts ad places. Table 3: Media of SE Ratio by Estimate Type for Tracts ad Places Estimate Type Tract Media Place Media Household Cout Housig Uit Cout Populatio Cout Percet Ratio or Mea Source: 5-9 America Commuity Survey (special tabulatio), There appears to be o appreciable differece betee couts, percets, ad ratios for either tracts or places. Other geographic types sho a similar distributio. Does the FPC adustmet have a differetial impact o differet characteristics at loer levels of geography? The may estimates i the data profile ca be easily broke do ito broad topics, hich may iclude oly oe estimate or may. The SE ratio distributio for 7 selected topic groupigs at the tract level are preseted i Table. Table : Distributio of SE Ratio for Selected Topic Groupigs for Tracts Topic st 5th Media 75th 99th AIAN Groups Gradparets Acestry Asia Groups Housig Value Household Icome Laguage Spoke at Home Educatioal Attaimet Poverty (Persos) Household Type Age & Sex Residece Oe Year Ago Teure Total Populatio Total Housig Uits Total Households Vacacy Rate Source: 5-9 America Commuity Survey (special tabulatio), The media SE ratios rage from.953 for America Idia tribal groups to.9 for vacacy rates. The distributio of the media values amog the topics is fairly smooth, ith o large outliers ith very high or lo media ratios. The FPC adustmet has less 368

9 Sectio o Survey Research Methods JSM of a impact o the populatio, housig uit, ad household totals, but SEs for those characteristics are geerally small to begi ith. Does the FPC have a differetial impact o the stadard error for smaller or larger geographic areas? Table 5 shos the SE ratio distributio for the tract ad place summary levels, ith each broke ito populatio size groupigs. The populatio size rages ere chose so that approximately percet of that type of geographic area is cotaied ithi i. For example, about percet of all tracts have a populatio betee,9 ad,6. Table 5: Distributio of SE Ratio by Populatio Size for Tracts ad Places Size Rage Geo st 5th Media 75th 99th -,9 Tract ,9-,6 Tract ,6-3, Tract ,-3,6 Tract ,6-, Tract ,-,7 Tract ,7-5, Tract ,-6,3 Tract ,3-7,7 Tract >7,7 Tract Place Place Place Place ,39 Place ,39-,6 Place ,7-3,66 Place ,66-6,95 Place ,95-6,3 Place >6,3 Place Source: 5-9 America Commuity Survey (special tabulatio), For both geography types the smaller areas i populatio geerally have a smaller SE ratio, idicatig a larger decrease i the value of the SE after applyig the FPC. This is more proouced for places, as places ith small populatios are likely to be sampled at higher rates, ad have correspodigly higher samplig fractios ad smaller FPC factors. The published ACS data profiles ad other products iclude the 9 percet margi of error (MOE) for each estimate, that is, the SE multiplied by.65. Percet estimates ad their margis of error are preseted ith oe decimal place of accuracy. If a user had ithout-fpc ad ith-fpc margis of error side-by-side, ould the small decreases i the SEs e have see make a oticeable differece i the values i the published data profile? Table 6 asers this questio for several summary levels. It shos that, for may areas, the as-published MOE ould be loer ith the FPC tha ithout the FPC. 369

10 Sectio o Survey Research Methods JSM Table 6: Observable Chage i MOE for Percet Estimates by Summary Level Summary Level MOE Higher No Chage MOE Loer Natio.%.%.% State.% 9.7% 8.3% Couty.%.% 58.9% MCD.3%.6% 88.% Tract.% 3.3% 86.6% Place.%.7% 8.% AIAN Area.%.% 87.5% Source: 5-9 America Commuity Survey (special tabulatio), More tha percet of MCDs, tracts, places, ad AIAN areas ould display a smaller MOE. At the atio ad state, levels, ith loer SEs due to more sample, feer percet estimates ould sho a chage i the MOE. If the SE is small eough such that the ithout FPC MOE is. percet, the o amout of decrease due to the FPC ould make a differece i the published MOE. 6. Coclusios ad Further Research After iitial research, the FPC factor methodology as applied to the 3-year 7-9 ad 5-year 5-9 ACS data published i. This evaluatio of the 5-year 5-9 data idicates that the chage i estimates of the SEs, as measured by the ratio of the ith-fpc to ithout-fpc SEs, shos a cosistet patter of improvemet (i.e. loer SEs ith the FPC). The chages are ithi a reasoable rage of values, ad fe estimates have SEs that icrease at all, much less substatially. The largest reductios are see at geographic types (MCDs, places, ad AIAN areas) as ould be expected based o the ACS samplig methodology. We pla to cotiue usig this methodology for the 3-year ad 5-year ACS data published i, ad to cotiue to moitor its impact. There are curretly o plas to apply this to the -year ACS data, but further research could idicate its utility. Ackoledgemets The author ould like to ackoledge the ork doe by Alfredo Navarro ad Mark Asiala o this research. Refereces Bethlehem, J. (9). Applied Survey Methods: A Statistical Perspective. Hoboke, NJ: Joh Wiley & Sos. Cochra, W. (977). Samplig Techiques, 3rd Editio. Ne York, NY: Joh Wiley & Sos. Fay, R., ad G. Trai. (995). Aspects of Survey ad Model-Based Postcesal Estimatio of Icome ad Poverty Characteristics for States ad Couties. Proceedigs of the Sectio o Govermet Statistics. Alexadria, VA: America Statistical Associatio, pp

11 Sectio o Survey Research Methods JSM Gbur, P., ad L. Fairchild. (). Overvie of the U.S. Cesus Log Form Direct Variace Estimatio. Proceedigs of the Sectio o Survey Research Methods. Alexadria, VA: America Statistical Associatio, pp. 39. Starsiic, M., ad A. Tersie. (7). Aalysis of Variace Estimates from America Commuity Survey Multiyear Estimates. Proceedigs of the Sectio o Survey Research Methods. Alexadria, VA: America Statistical Associatio, pp U.S. Cesus Bureau (), Summary File 3 Techical Documetatio, U.S. Cesus Bureau. (a). America Commuity Survey Accuracy of the Data (9). ccuracy_of_data_9.pdf U.S. Cesus Bureau. (b). America Commuity Survey Multiyear Accuracy of the Data (3-year 7-9 ad 5-year 5-9). aracsaccuracyofdata9.pdf 363