S5 pre-registration (completed wave)

Design, exclusion rules, and results from the locked S5 wave (N=392, filed on OSF).

The preregistration, in plain English

OSF record: https://osf.io/br3u2 (posted before we analyzed the data.) Wave: S5. Last refreshed: 2026-06-04.

Why this matters

Before we looked at the results, we posted the design, the exclusion rules, and the analysis plan publicly with a timestamp. That is the integrity point. It means we could not have picked the rules or the cutoffs after seeing the numbers to make the result look better. A validator can read the OSF record and confirm the plan predates the analysis.

A note on the hypotheses

The honest framing: the headline of this study is descriptive. The finding that matters is the support level (about 90% support, above 88% in every party bloc) and that it holds up after careful exclusions and across the political spectrum. The five pre-registered outcomes below are a pre-commitment device, not the point of the study. Their value is that we fixed the bounds in advance, so you can see we are not moving the goalposts. Whether each one technically "passes" matters far less than the fact that the rules were locked before analysis and the data reproduces from the raw rows.

What "supported" (or "reliable") means for each test

For every outcome below, we do not rely on the point estimate alone. A result counts as supported only when the 95% confidence interval stays on the correct side of the pre-registered bound:

  • For a floor (such as "at least 80%"), the lower edge of the interval must clear it.
  • For a ceiling (such as "no more than 10%"), the upper edge must clear it.
  • For an equivalence claim (the shift is near zero), the whole interval must sit inside the pre-registered band, tested with TOST (two one-sided tests).
  • For the "modal choice" claim, the winning option's share must sit above the upper confidence bound of the next-best option.

Confidence intervals are Wilson intervals for proportions. (Wilson is a standard, well-regarded interval for proportions and is in no way a weaker choice than the normal approximation; it behaves better at high percentages like ours.)

The five pre-registered primary outcomes

#OutcomePre-registered boundResult
aPrinciple agreementAt least 95% of clean respondents have a Section B principle-mean of 5 or higher (1 to 7 scale).Pass. Lower CI bound clears the 95% floor.
bBallot supportAt least 80% support the ballot initiative (rated 5 or higher of 7).Pass. Lower CI bound clears the 80% floor.
cAbstract-to-specific shiftThe within-respondent shift is equivalent to zero, inside ±0.5 Likert points (TOST).Pass. Mean shift near zero, whole interval inside ±0.5.
dMitigation "no" rateNo more than 10% say the named protections do not answer their concerns.Pass. Upper CI bound clears the 10% ceiling.
eModal review designThe citizen routing panel is the modal forced choice, its share above every other option's upper 95% CI.Pass. Modal share exceeds the next option's upper bound.

All five pre-registered outcomes passed under the locked rules. The exact current values are in lib/study-snapshots/rthg-s5.json and reproduce from the raw data with the commands in START_HERE.md.

The exclusion rules (S5), in plain English

We remove a response from the analysis cohort if any of these is true. The same rules are encoded in lib/rthg-study-flags-s5.ts and applied by the reproduce script:

  • It failed the instructed-attention check (a Likert item that told people exactly which box to tick).
  • It failed a comprehension check (for example, the ballot clearly stated a two-year setup window and the check asked for it back).
  • It failed the final demographics-section attention check.
  • It failed a method check on the ballot pages.
  • It was answered implausibly fast (the principles page or the mechanism page under 45 seconds, or the arm page under 30 seconds), which indicates clicking through without reading.
  • It showed acquiescence on the reverse-coded item (agreed that officials should be allowed to lie and agreed they should explain honestly, a self-contradiction).
  • For the headline US-adult cohort, the respondent indicated they were not eligible to vote in 2024 (used as a citizenship proxy in S5; S6 replaces this with an explicit citizenship question).

What changed for S6 (so you are not surprised later)

S6 keeps the same instrument family and analysis pipeline. The exclusion rules were deliberately revised: the reverse-coded check was relaxed so it only fires on genuine self-contradiction (agree-agree), and the 2024-vote eligibility proxy is replaced by a direct citizenship question. S6 reports two tiers: a headline cohort (everything except the reverse-coded check) and a stricter confirmatory cohort (everything). One enforcement change: in S5 the instructed-attention check was applied only after the fact, while S6 also gates it in-survey (one retry, then a recorded screen-out), with the post-hoc rule kept as a backstop. The gate is safe because the instructed answer is content-neutral ("Neither"), so it filters non-readers without coaching the result; every other check stays recorded-not-blocking. Details and the rationale are in Documents/V3/S6_EXCLUSION_PREREG_NOTES.md.