XII. Demographic Analysis and Other Quality Measures

Coverage Estimation is only one way to judge the accuracy of the census. Census also uses Demographic Analysis, operational assessments, evaluations, experiments, and other specialized studies. These tools answer different questions and rely on different sources of evidence.

Demographic Analysis builds an independent estimate of the population from birth and death records, estimates of international migration, Medicare enrollment, and other data. Census compares that estimate with the census count to measure net undercounts and overcounts nationally for selected demographic groups. Because it does not begin with the census address list or depend on household participation, it can reveal some errors that a post-enumeration survey may not detect.

But Demographic Analysis cannot provide a complete account of census quality. Its results are strongest for populations well represented in vital statistics and other source records. Historical limits in those records restrict the available race and ethnicity detail, while characteristics such as disability, language, housing situation, sexual orientation, and gender identity generally cannot be assessed through the traditional method. Demographic Analysis also measures net error, so omissions in one population may be offset by duplicates or erroneous counts elsewhere.

Operational assessments and evaluations serve a different purpose. Assessments document how an operation performed, including workloads, completion rates, costs, and lessons learned. Evaluations analyze whether particular census methods worked and how they affected coverage and data quality. Experiments test alternative approaches that may inform the next census. Census plans to use these studies both to assess the 2030 Census and to shape early planning for 2040.

These forms of evidence should be considered together. Demographic Analysis may show that a population was undercounted without explaining which operation failed. An operational assessment may show that Census completed nearly every assigned case without revealing that some housing units never entered the workload. A national evaluation may find that a method worked overall while concealing weaker performance for particular communities.

This chapter examines the improvements planned for Demographic Analysis, opportunities to expand what it can measure, and the limits that require other forms of evaluation. It also considers whether Census’s assessments and research reports will be designed and released in time to identify differential quality problems and inform the next census.

A. Planned Improvements

Demographic Analysis provides a second independent way to assess census coverage. Rather than interviewing a separate sample of households, it builds population estimates from birth and death records, estimates of international migration, Medicare enrollment data, and other sources. Census then compares those estimates with the census counts to identify possible net undercounts and overcounts.

The 2030 Operational Plan identifies three planned improvements.

First, Census intends to produce estimates with more race and ethnicity detail. Historically, Demographic Analysis has been limited by the categories available in older birth and death records. The 2020 estimates were largely restricted to Black and non-Black populations and to Hispanic and non-Hispanic populations for selected ages. More recent vital records contain greater detail, creating an opportunity to assess coverage for additional populations.

Census should explain what “more detail” will mean in practice. Important questions include which racial and ethnic populations will receive estimates, which age ranges will be covered, whether people reporting more than one race can be represented, and how the estimates will align with the standards used for the 2030 Census questionnaire.

Second, Census plans to develop formal measures of uncertainty. Demographic Analysis depends on assumptions and imperfect source data, particularly for international migration. Uncertainty measures can show the range within which the population is reasonably likely to fall and help prevent small differences between an estimate and the census from being interpreted too confidently.

The measures should identify which components contribute most to uncertainty and how uncertainty changes across age, race, ethnicity, and geography. Census should publish enough methodological information for outside researchers to understand whether an apparent coverage difference is statistically meaningful.

Third, Census plans to produce official state- and county-level estimates for young children. The 2020 program produced the first experimental local Demographic Analysis estimates for children ages 0 through 4. Making these estimates a regular part of the 2030 program could show where the persistent undercount of young children is concentrated rather than limiting the analysis to a national rate.

These local estimates could be especially useful when paired with information about housing instability, response mode, language access, administrative enumeration, and local census operations. They could help distinguish a broad national problem from the particular conditions contributing to omissions in different communities.

Planned improvements

More detailed race and ethnicity estimates, formal uncertainty measures, and official state- and county-level estimates for young children.

Open design questions

Which populations and age ranges will receive detailed estimates, how uncertainty will be presented, and how Census will validate the new local estimates.

B. Opportunities for Expansion

The planned young-child estimates raise a broader question: can Demographic Analysis or related independent methods provide useful subnational coverage information for other populations?

Other age groups may be a logical starting point. Birth records provide a particularly strong foundation for estimating children, but Census should examine whether state or county estimates are feasible for older children, young adults, or older adults. Each group presents different methodological challenges, especially as migration becomes more important and historical records become less detailed.

Census should also explore additional race and ethnicity estimates. Greater detail at the national level would be valuable, but many coverage concerns are geographically concentrated. A national estimate may conceal a serious undercount affecting a particular population in only a few states or metropolitan areas.

The ability to produce local estimates will depend on the size of the population and the consistency of race and ethnicity information across decades of vital and administrative records. Where direct estimates are not possible, Census could consider combining years, producing estimates for broader regions, or conducting targeted studies in places with known coverage concerns.

Disability and other characteristics present a different challenge. Birth, death, migration, and Medicare records were not designed to produce a complete measure of disability, language use, sexual orientation, gender identity, or housing instability. Census should not infer sensitive characteristics from records that do not measure them directly.

Instead, advocates should ask whether the broader evaluation program can develop complementary methods. Those might include specialized surveys, voluntary questions in follow-up studies, linkage to appropriate statistical datasets, or focused qualitative and quantitative research. The objective should be to measure coverage where sound methods exist, not to force every population into the traditional Demographic Analysis model.

Census should also consider targeted estimates for populations with known coverage concerns. These could include people living in areas affected by disasters or displacement, residents of tribal lands, people in group quarters, and populations experiencing unusually high mobility. The most suitable method may differ by group.

The Bureau should publish a feasibility agenda identifying:

  • which additional populations it has considered;

  • which data sources and methods might support estimates;

  • what geographic and demographic detail may be possible;

  • which estimates cannot be produced reliably; and

  • what alternative evaluation methods will be used where Demographic Analysis is unsuitable.

Advocacy opportunity

Use the planned local estimates for young children as a model for asking where other geographically concentrated coverage problems can be measured through Demographic Analysis or complementary research.

C. Limits of Demographic Analysis

Demographic Analysis is powerful because it does not depend on household participation in the census or on the census address list. But it can assess only the populations and characteristics that its independent sources can represent reliably.

The method is strongest where vital records are complete and the demographic accounting is relatively direct. Birth records provide a strong basis for estimating children born in the United States, while death records allow Census to remove people from earlier birth cohorts. Medicare records are especially useful for estimating the older population.

International migration is especially difficult to estimate in the current environment. Immigration and emigration can change quickly in response to enforcement policies, deportations, changes in legal status, and decisions by people to enter or leave the country. Census has already revised its recent population-estimates methods to account for policy-driven increases in emigration and a sharp decline in net international migration. These rapid shifts are difficult to capture through surveys that become available only after a delay and administrative records that provide an incomplete picture of migration. The resulting uncertainty affects estimates for foreign-born populations and other groups for whom international migration is a major source of population change.

Race and ethnicity estimates are also constrained by historical records. The categories used on birth and death records have changed substantially over time. Older records provide very limited race information, and Hispanic origin was not recorded widely enough to support estimates for all ages. As a result, traditional Demographic Analysis has been unable to reproduce the full detail available through self-reported census responses.

Many other characteristics are not captured directly at all. Demographic Analysis cannot readily estimate coverage by disability, language, sexual orientation, gender identity, household relationship, or housing stability because its principal source records do not contain consistent and appropriate measures of those characteristics.

It also cannot easily distinguish people according to their living situation on Census Day. The underlying records may help estimate the total number of people of a given age and sex, but they generally do not establish whether those people lived in households, college housing, prisons, nursing facilities, shelters, or other group quarters. Separate evaluation methods are needed to assess coverage within those settings.

Demographic Analysis primarily measures net coverage error. An estimate showing little net error does not necessarily mean that everyone was counted correctly. An undercount of one population can be offset by duplicates or erroneous counts elsewhere. The method also does not generally identify which particular people were missed, where an operational failure occurred, or why the error happened.

These limits do not reduce the importance of Demographic Analysis. They define the questions it can answer. It provides a strong independent check on the overall population and on selected demographic groups, but it must be interpreted alongside Coverage Estimation, operational assessments, specialized studies, and evidence from communities.

Why advocates should care

No single evaluation method can measure every census error. Census should state clearly which populations Demographic Analysis can assess reliably and ensure that people outside that scope are not left without any meaningful coverage evaluation.

D. Operational Assessments and Evaluations

Coverage estimates show whether the census appears to have counted too many or too few people. Operational assessments and evaluations help explain how and why those errors occurred.

The Operational Plan distinguishes several kinds of research. Operational assessments document how well an operation was conducted, including its workload, completion rates, costs, and lessons learned. Evaluations examine the effectiveness of particular methods or design choices, while experiments compare alternative approaches. Census expects generally all operations needed to conduct the census to produce an assessment, although the final inventory will depend on the completed 2030 design.

These studies will be particularly important because the 2030 Census includes major changes that cannot be evaluated through a national coverage rate alone. Census will need to examine when administrative records replaced interviews, whether targeted fieldwork found unexpected errors, how no-ID responses were matched, and whether combined field operations worked equally well across living situations.

Advance Study Plans

Census should publish study plans before major tests and production operations begin. A useful plan should identify the research questions, comparison groups, data sources, methods, limitations, and criteria for judging success.

Advance publication allows outside experts and affected communities to identify missing questions while the study can still be improved. It also reduces the risk that measures are selected after Census already knows whether an operation appeared successful.

The Operational Plan says Research Support will develop standard study-plan and report templates, conduct quality reviews, and facilitate the release of final study plans and reports through the memorandum series. Advocates should seek a commitment that these materials will be released publicly and early, not merely completed for internal governance.

Publication Schedules

Census should publish a schedule showing when each assessment, evaluation, and experiment will be released. The schedule should include research from the 2026 Census Test, the 2028 Dress Rehearsal, peak census operations, Coverage Estimation, and Demographic Analysis.

Reports from tests must be released soon enough to inform the next design decision. Findings from the 2026 Test should be available before the 2028 design is fixed, and Dress Rehearsal results should be available while meaningful changes to 2030 operations remain possible.

Post-census reports should likewise be completed early enough to shape the first major decisions about the 2040 Census. The plan expressly identifies evaluations and operational assessments as inputs into early 2040 planning.

Measures of Differential Quality

Overall completion rates and national averages will not show whether an operation worked equally well across communities. Assessments should report results by relevant population, geography, language, housing type, response mode, and enumeration method.

For example, an assessment of in-office enumeration should not report only the number of cases completed. It should examine whether administrative enumeration produced less complete household rosters for young children, recent movers, or complex households. An assessment of CQA should examine wait times and successful response by language. Address-frame studies should report what kinds of housing units remained missing.

The relevant subgroups will vary by operation, but every study plan should ask whether aggregate success could conceal concentrated quality problems.

Timely and Complete Reporting

Reports should document unsuccessful tests as carefully as successful ones. A proposed innovation that did not improve quality, could not be implemented reliably, or created unexpected disparities provides essential information for future planning.

Census should publish abandoned approaches, inconclusive results, implementation failures, and design changes made after testing. Otherwise, the public may see only the methods that advanced and have no way to understand which alternatives were considered or why they were rejected.

Reports should also distinguish between a method that failed under realistic conditions and one that could not be evaluated because the test was too small, implementation was incomplete, or necessary data were unavailable. Those are different lessons.

Finally, Census should maintain a public system linking research recommendations to later decisions. The Operational Plan anticipates tracking recommendations and action plans internally. External reporting should show whether Census accepted, modified, deferred, or rejected each important recommendation and provide a brief explanation.

Advocacy priorities

Publish study plans before implementation, commit to release schedules, measure differential quality, report results in time to inform subsequent decisions, and document unsuccessful and inconclusive tests as well as successful ones.

Why advocates should care

A census can achieve its national production targets while particular operations work poorly for specific populations. Transparent and timely evaluation is necessary to reveal those differences and prevent them from being carried into the next census.

Where to Look: See section 3.3.2, “Demographic Analysis,” pp. 56–57 of the 2030 Census Operational Plan. It describes the use of vital statistics, international-migration estimates, Medicare enrollment records, and other independent sources. It also identifies the principal planned improvements for 2030: greater race and ethnicity detail, uncertainty measures, and official state- and county-level estimates for young children.

Section 3.3.1, “Research Support,” pp. 54–56, describes Census’s plans for evaluations, experiments, and operational assessments. It covers the role of the Decennial Research Objectives and Methods working group, standardized study plans and reports, recommendation tracking, assessments of the 2026 Census Test and 2028 Dress Rehearsal, and the use of research findings to inform early 2040 planning. The final inventory of 2030 evaluations and assessments has not yet been established.

Advocates should monitor the 2030 Census Memorandum Series for study plans, methodological decisions, test reports, and operational assessments. The 2030 Census Research Project Explorer and the Census research-and-testing webpage provide additional information about ongoing projects and the questions Census is studying before the design is finalized.

For the prior-decade evidence base, see the 2020 Demographic Analysis materials, including the national estimates by age, sex, race, and Hispanic origin and the experimental state- and county-level estimates for children ages 0 through 4. The local young-child tables and methodology provide an especially useful benchmark for evaluating the official subnational estimates planned for 2030.

The 2020 Census Evaluations and Experiments webpage contains the inventory of operational assessments, evaluations, experiments, and quality-control studies. It is useful both as a source of findings about 2020 and as a benchmark for assessing whether the 2030 program publishes study plans and results earlier, measures differential quality more consistently, and releases enough information for outside review.