BEYOND ISOLATED HEADLINES — REPLICATION AND SUPPORTING MATERIALS

Supporting archive for
"Beyond Isolated Headlines: A Framework for Coupling Scorekeeping and Explanatory
Modeling in Public Political Polls."

This package separates two kinds of evidence used in the paper so that the introductory
release audits are not confused with the 1963 proof-of-concept analyses.

CONTENTS

  audit-materials/
    Replication materials for the introductory FiveThirtyEight and New York Times evidence
    and Appendix A.3. This folder contains the archived poll files (consolidated 2026-09-12 with 2025
    Internet Archive captures of the FiveThirtyEight endpoints), record-count, sponsor, and
    sample-draw scripts, audit workbooks, crosswalks, prescreen output and scripts, and the
    archived copy of the one release coded as disclosing a model. See audit-materials/README.txt for the
    audit design, second-coder checks, provenance notes, and file-level checksums.

  1963-model-materials/
    Analytical output supporting the 1963 Harris–Newsweek proof-of-concept, specifically
    the Model Builder's all-candidate Standalone vs Incremental output used to inspect
    omitted measured variables, including party identification. See that folder's README.

  1963-replication/
    The 1963 analysis file, the codebook the companion site uses for it, and a standalone
    script that reproduces every table and figure from the 1963 demonstration, plus the
    Appendix A.2 weight summary and race-raking sensitivity check, the Appendix B.7
    bootstrap, all three routes of the Appendix E.2 triangulation example, the Auto-Build
    search behind the Appendix E.3 alternative specification, and the Appendix E.4
    Nonresponse Test, Variable-Ranking Tool, and synthetic-variable figures, without the
    companion site's software. See that folder's README for how to run it, what each output
    corresponds to, and what each author-created variable in the analysis file is.

SCOPE

The folders serve different manuscript purposes. The release audits document how rarely
an explanatory model is attached to public poll-release materials. The 1963 folders support the
historical demonstration. They are not part of either release audit and should not be
interpreted as audit evidence.

The paper's prospective framework is broader than the contents of this companion archive.
The companion site's software is not included. If the article is accepted, the audit
materials, the 1963 analysis file, and the replication script will be deposited in POQ's
permanent Harvard Dataverse archive before typesetting.

See MANIFEST-SHA256.txt for checksums of the package's input, script, and
documentation files. The files in 1963-replication/output/ are not listed:
replicate_1963.py regenerates and overwrites them when run as documented, so
they are verified by the script's --check against the published values rather
than by checksum.

CHANGE HISTORY
  The entries below are a dated change history, newest first. Check totals in older
  entries describe earlier versions of the package. The current --check total is
  130 values (121 when the network route is unavailable).

REVISED 2026-10-08 (manifest scope)
  - This README now states what MANIFEST-SHA256.txt covers: the package's
    input, script, and documentation files. The 23 files in
    1963-replication/output/ were never listed; the sentence previously read
    "the files in this package," which overstated the manifest's scope. The
    outputs are regenerated and overwritten by replicate_1963.py when run as
    documented, so listing their shipped hashes would fail verification after
    any rerun; the script's --check verifies them against the published
    values instead.
  - MANIFEST-SHA256.txt updated for this README. No other hash changed.
  - No script, data file, workbook, or output changed.

REVISED 2026-10-05 (manifest hash; archive structure)
  - MANIFEST-SHA256.txt: the entry for 1963-replication/README.txt carried a
    stale hash from before the 2026-10-03 revision of that file; the line now
    matches the README actually shipped (current check totals 130 and 78). No
    file content changed; every other listed hash verified unchanged.
  - The ZIP now extracts into a single top-level directory instead of two
    identically named nested ones. Relative paths within the package are
    unchanged, so MANIFEST-SHA256.txt verifies as before.
  - MANIFEST-SHA256.txt updated for this README.
  - No script, data file, workbook, or output changed.

REVISED 2026-10-03 (observed approval by civil-rights position)
  - The paper now reports, in the evaluation discussion of "Toward Routine
    Adoption," the observed weighted approval rates by position on the
    civil-rights bill in the 1,159-case analytic sample: 73 percent among
    supporters and 31 percent among opponents. replicate_1963.py computes
    both in table_1 and adds them to the expected-values table and to the
    --core-check subset; --check now compares 130 values (121 when the
    network route is unavailable), and --core-check 78.
  - sample_summaries.csv gains two rows (the two observed rates). Every
    other previously written table and figure is byte-identical.
  - 1963-replication/README.txt and this file: current check totals
    updated to 130.
  - MANIFEST-SHA256.txt updated for the three changed files.
  - No data file changed.

REVISED 2026-10-01 (quick verification flag)
  - 1963-replication/replicate_1963.py adds --core-check, a quick
    verification that recomputes and compares 76 core manuscript values -
    the reference-model sample and baseline, the principal Table 1
    scenarios and intervals, the Table 2 standalone-incremental
    comparisons, the Table B1 diagnostics, the Table C1 civil-rights
    coefficient, the Republican subgroup result, the Table E1 sample and
    Appendix E.3 common-sample comparison, the Appendix A.2 weight and
    race-raking quantities, the Appendix E.2 model- and propensity-route
    results, the nonresponse tipping calculations, and the propensity
    support rule - in a few seconds, without the computationally intensive
    sensitivity analyses (the Appendix B.7 bootstraps, Bayesian-network
    analyses, synthetic-variable seeds, full variable ranking, and
    Appendix E.3 Auto-Build search), which its output names as not
    included. The core values are a tagged subset of the script's single
    expected-values table, so the two checks compare against the same
    published entries and cannot drift apart. --core-check writes no
    output files and never imports the network libraries. --check is
    unchanged and remains the authoritative test: 128 values (119 when
    the network route is unavailable), rerun after this revision with
    128/128 reproduced, and every previously written table and figure is
    byte-identical.
  - 1963-replication/README.txt: HOW TO RUN documents the flag and its
    scope.
  - MANIFEST-SHA256.txt updated for the two changed files.
  - No data file changed.

REVISED 2026-09-25 (Appendix E.3 Auto-Build search)
  - 1963-replication/replicate_1963.py now reruns the Auto-Build search (Auto:
    Actionable path) that produced the Appendix E.3 specification, reimplemented
    from the companion site's routine without its software or wall-clock budgets:
    candidate screens, the standalone weighted-Tjur and random-forest rankings,
    the forward pass on cross-validated Tjur R-squared with the collinearity gate
    and separation screen, and the backward pass. It selects q31_9, q31_1, q31_20,
    q31_4, q20a, q31_10, q31_3 in that order on 1,176 complete cases - the seven
    predictors of Table E1 - and records that the selection is unchanged with the
    random-forest additions switched off. It writes e3_autobuild_steps.csv and
    e3_autobuild_pool.csv and adds the search's summary to sample_summaries.csv;
    --check now compares 128 values (119 when the network route is unavailable).
    The script takes a new --codebook argument (default jfk_codebook.csv). No
    earlier output changed: every previously written table and figure is
    byte-identical.
  - 1963-replication/jfk_codebook.csv added: the companion site's codebook for the
    1963 file, the source of the actionability tiers the search uses. Its README
    entry names the three columns the script reads.
  - 1963-replication/README.txt: PURPOSE, FILES, HOW TO RUN, OUTPUT, and METHOD
    NOTES describe the search; the NOT INCLUDED paragraph on the search is removed.
  - This README's 1963-replication entry and check-total note updated.
  - MANIFEST-SHA256.txt updated for the changed and added files.
  - No data file changed.

REVISED 2026-09-24 (change-history note)
  - This README adds the CHANGE HISTORY note above, stating that check totals in
    older entries refer to earlier versions of the package. No script, data file,
    workbook, or output changed.
  - MANIFEST-SHA256.txt updated for this README.

REVISED 2026-09-23 (E-value removed from the checked values)
  - The paper no longer reports the Appendix E.2 E-value, so 1963-replication/
    replicate_1963.py no longer checks it: --check now compares 126 values (117 when the
    network route is unavailable). The script still computes the quantity and writes it
    to the e_value column of e2_propensity.csv, as 1963-replication/README.txt describes.
    No data, estimate, or other output changed.
  - 1963-replication/README.txt states the new count.

REVISED 2026-09-20 (B.7 common basis; tracker zero-text pages)
  - 1963-replication/replicate_1963.py adds the Appendix B.7 common-basis comparisons:
    each increment the main text compares is recomputed as a drop-one loss from one model
    containing both items (civil rights 0.019 vs corruption 0.005 on 1,156 cases; moving
    the country ahead 0.016 vs Vietnam 0.009 and race 0.002 on 1,153 cases), with a
    1,000-resample bootstrap in which moving ahead exceeds race in 99 percent and Vietnam
    in 76 percent. It writes b7_common_basis.csv and adds the shares to
    sample_summaries.csv; --check now compares 127 values (118 when the network route is
    unavailable). No earlier output changed.
  - 1963-replication/README.txt documents the addition.
  - audit-materials/tracker-audit-1104.xlsx: the two archived HTML documents that returned
    zero extractable text (cato.org; publicpolicypolling.com), previously left in the
    screen-negative pool unread, were opened in a browser and coded individually; neither
    disclosed a model. Their coding_route is now "zero-text HTML (2)". URLs coded
    individually 128 -> 130; screen-negative documents not read individually 658 -> 656.
    Records covered by individual coding now reads 188: the earlier 183 was a Summary and
    README error (Records_1104 marked 186 before this change). No verdict changed;
    assessable records (1,083) and disclosures (1 release, 6 records) are unchanged.
  - audit-materials/README.txt: EXTRACTION COVERAGE describes the individual coding.
  - MANIFEST-SHA256.txt updated for the changed files.
  - No data file changed.

REVISED 2026-09-20 (robust covariance label)
  - 1963-replication/replicate_1963.py: the sandwich covariance is now requested and
    described as HC0 (Huber-White, no finite-sample correction) instead of HC1.
    statsmodels applies no finite-sample correction to GLM results under any HCx label,
    so the two options return the same matrix: the earlier label was inaccurate, the
    numbers were not. The docstring now says that Stata multiplies the same robust
    variance by N/(N-1), which for the reference model makes its standard errors larger
    by at most 0.0004. Figure 1's "+0.4 pts" label is now placed inside the plot frame.
    No computed value or output table changed: with every package installed the run
    still reports 119/119 reproduced and exits 0.
  - 1963-replication/README.txt: METHOD NOTES describes the covariance the same way.
  - MANIFEST-SHA256.txt updated for the changed files.
  - No data file changed.

REVISED 2026-09-20 (network skip notice)
  - 1963-replication/replicate_1963.py: when the Appendix E.2 network route cannot
    run, the script now names the package that is actually missing. Installing
    pgmpy with --no-deps and skipping requirements.txt leaves pgmpy importable but
    the packages it uses absent; the script previously reported that case as
    "pgmpy not installed" and printed an install command that was cut off. The
    notice, the nine SKIP lines, and the summary line now all read "network route
    unavailable" with the same reason, and point to requirements.txt when pgmpy
    itself is present. No computed value, output file, or exit status changed:
    with every package installed the run still reports 119/119 reproduced and
    exits 0; with one missing it reports 110/110 and exits 2.
  - 1963-replication/README.txt: HOW TO RUN describes both install lines and the
    two skip cases.
  - MANIFEST-SHA256.txt updated for the changed files.
  - No data file changed.

REVISED 2026-09-20 (E.2 network route; E.4 nonresponse and ranking)
  - 1963-replication/replicate_1963.py adds the Bayesian-network route of the Appendix E.2
    example (40.8 percent; -15.8 points, band -19.9 to -11.5; civil rights and Khrushchev as
    the only direct antecedents of approval), the Appendix E.4 Nonresponse Test figures, and
    the Variable-Ranking Tool results for all 123 candidates, with the E.4 summary
    statistics, and the synthetic-variable probe's 30-seed range. It writes e2_network.csv,
    e2_network_edges.csv, e4_nonresponse.csv, e4_ranking.csv, e4_synthetic.csv, and
    e4_propensity_support.csv (which of the 25 single-level scenarios meet the propensity
    route's quintile rule); --check now compares 119 values. No earlier output changed.
  - 1963-replication/requirements.txt adds the network route's dependencies;
    1963-replication/README.txt documents installation and the new outputs.
  - MANIFEST-SHA256.txt updated for the changed files.
  - No data file changed.

REVISED 2026-09-19 (E.2 propensity side)
  - 1963-replication/replicate_1963.py adds the propensity side of the Appendix E.2
    triangulation example, now computed by Rosenbaum-Rubin subclassification on propensity
    quintiles with raw within-quintile rates (Civil Rights "Oppose": 45.6 percent adjusted
    approval; -11.0 points, bootstrap 95% CI -17.7 to -6.4; gap with the model route -0.6
    points, 95% CI -5.0 to +1.9) and writes e2_propensity.csv; --check now compares 65
    values. No other output changed.
  - 1963-replication/README.txt documents the addition.
  - MANIFEST-SHA256.txt updated for the changed files.
  - No data file changed.

REVISED 2026-09-18 (E.2 scenario)
  - 1963-replication/replicate_1963.py: the Appendix E.2 triangulation example now
    reproduces the scenario the manuscript reports in Appendix E.2 and "Four
    Vulnerabilities": Civil Rights "Oppose" for every respondent (356 respondents
    giving that answer; 46.2 percent; -10.4 points, 95% CI -14.5 to -6.4). It replaces
    the earlier two-item scenario (Peace "Excellent" with Civil Rights "Favor"), which
    the manuscript no longer reports. e2_scenario.csv changes accordingly; --check
    still compares 51 values. No other output changed.
  - 1963-replication/README.txt documents the change.
  - MANIFEST-SHA256.txt updated for the changed files.
  - No data file changed.

REVISED 2026-09-16 (run commands and E.2 scenario)
  - audit-materials/README.txt adds a HOW TO RUN section with the exact command lines
    for record-count.py, sponsor-count.py, and draw-sample.py, and the values each
    should return.
  - 1963-replication/replicate_1963.py adds the model side of the Appendix E.2
    triangulation example (Peace "Excellent" with Civil Rights "Favor": 244 respondents
    giving both answers; 70.9 percent; +14.3 points, 95% CI +8.7 to +20.0) and writes
    e2_scenario.csv; --check now compares 51 values. Earlier outputs are unchanged.
  - 1963-replication/README.txt documents the addition.
  - MANIFEST-SHA256.txt updated for the changed files.
  - No data file changed.

REVISED 2026-09-16 (file metadata)
  - audit-materials/tracker-audit-1104.xlsx and
    audit-materials/superseded-2026-07/link-audit-400.xlsx: removed the
    "last modified by" name from each workbook's document properties.
    Metadata only; no sheet, cell, or coding changed.
  - MANIFEST-SHA256.txt updated for those two files and this README.
  - No data file changed.

REVISED 2026-09-16 (Figure 1)
  - 1963-replication/replicate_1963.py now marks the same logit pairs as the published
    Figure 1 (0 to 1, +23.1 points; 5 to 6, +0.4 points) instead of 0 to 1 and 2.5 to 3.5,
    and labels each change to one decimal. figure_1.png and figure_1_points.csv change
    accordingly; --check still compares 46 values, all unchanged.
  - 1963-replication/README.txt states the two marked pairs.
  - No data file changed.

REVISED 2026-09-16
  - 1963-replication/replicate_1963.py adds the Appendix B.7 bootstrap (seeded, 1,000
    resamples) and writes b7_bootstrap.csv; --check now compares 46 values. Earlier
    outputs are unchanged.
  - 1963-replication/README.txt documents the bootstrap.
  - No data file changed.

REVISED 2026-09-15
  - audit-materials/draw-sample.py now writes an uncoded link-audit-400-draw.xlsx and refuses
    to write to the coded link-audit-400-v2.xlsx or to replace any existing file without
    --overwrite. The draw itself is unchanged.
  - link-audit-400-v2.xlsx: the second-coder verdicts already in Second_coder_50 are copied
    to Sample_400.second_coder_verdict. No primary verdict changed.
  - tracker-audit-1104.xlsx: a second_coder_note on the disclosure URL explains that the
    second coder's "partial" matches the primary coding. No verdict changed.
  - 1963-replication/replicate_1963.py adds the Appendix A.2 weight summary and race-raking
    sensitivity check; --check now compares 42 values. Earlier outputs are unchanged.
  - 1963-replication/README.txt documents every author-created column in opster6.csv.
  - No data file changed.
