What changed from S5 to S6
This document summarizes every design change between the completed S5 wave
(N=392, locked April 2026, filed at osf.io/br3u2) and the current S6 instrument.
Ballot face (the text respondents read and vote on)
| Element | S5 | S6 |
|---|
| "The right" paragraph | States the right to honest government. | Adds "and accountability" to the fiduciary duty. |
| "Courts and consequences" paragraph | Describes consequences for dishonest service. | Adds a recall sentence: citizens could ask courts to remove officials in serious cases. |
| "Who reviews" paragraph | Mechanism-agnostic bridge to the mechanism page ("There are several reasonable ways to design it..."). | Same neutral wording. |
All other ballot paragraphs (The requirement, When there is doubt,
How the review works, How findings are made, The goal, Lasting protection,
Implementation) are unchanged.
Mechanism page (the four review-process designs)
| Change | Detail |
|---|
| Named concept rename | "Source Credibility Framework" (S5 term) renamed to "Source Reliability Framework" (S6 term). Tooltip body bullets unchanged; only the display name changed. |
| Shared callout bullet 3 | S5: "Any AI tool used in a review must be vetted and human-checked." S6: "Advanced analytical tools are encouraged to speed research and analysis, but tools never decide; the human responsible for the public record verifies the evidence and reasoning personally and signs off on it." |
| Mechanism options wording | "Vetted AI tools" language replaced with "modern tools under human responsibility" across all four mechanism descriptions. |
| Mitigation "Expertise support" | S5: "Independent counsel runs vetted AI tools and advisors at the panel's direction." S6: "Independent counsel runs analytical tools and advisors at the panel's direction, under human responsibility." |
| Mitigation "Speed" | S5: "Vetted tools and a written record speed review. Courts follow enforceable ethics rules, and must explain any departure from the panel record in writing." S6: "Modern technology and a written record speed review." (The courts-and-ethics sentence was relocated to the new Uniform Fiduciary Duty bullet below.) |
| Mitigation new bullet | S6 adds "Uniform Fiduciary Duty. Public officials of every party and branch are held to the same fiduciary duty standard to act with loyalty, care, and good faith on behalf of the public. Courts follow enforceable ethics rules, and must explain any departure from the panel record in writing." (The courts sentence formerly lived in the Speed bullet.) |
The four mechanism designs themselves (Courts only, Citizen grand jury,
Citizen routing panel, Public audit with court referral) are structurally
unchanged.
The post-mechanism mitigation-feedback question changed shape (David 2026-06-14):
the 4-option multiple choice ("how well did the protections answer your
concerns") is kept, but its follow-up open text is no longer conditional on
which option was chosen. There is now a single always-shown follow-up with the
same wording for everyone ("Which protections mattered most for you? What
concerns were not addressed, if any? Or if no concerns, why did you feel
confident already?"), required, with an explicit "N/A"/"None" escape so it stays
inline on the mechanism page.
Section B randomization
| Element | S5 | S6 |
|---|
| Fixed slots | B-bal at 1, B-att at 7, B-rev at 11. | Same three fixed slots. |
| Principle ordering | Fisher-Yates shuffle with a "no three consecutive items from the same thematic cluster" redraw rule. | Plain Fisher-Yates shuffle, cluster rule dropped. |
S6 drops the cluster constraint so the live (Azure) instrument and the Qualtrics
/ Verasight instrument randomize identically; Qualtrics cannot reproduce the
cluster rule. The three fixed diagnostic slots and per-respondent randomization
are unchanged. S5 keeps its cluster rule (it was fielded with it). In code, the
constraint is now opt-in: orderSectionBItems(seed, { clusterConstraint }),
with S6 call sites passing false.
Re-vote page (after seeing the mechanisms)
| Element | S5 | S6 |
|---|
| Re-vote reason | Conditional open text ("what shifted"), required only if the vote moved or the respondent said something changed. | Single always-shown, always-required reason merging the former "main reason" and "what shifted" prompts: "What's the main reason for the way you voted? If anything changed, what specifically changed when you saw the mechanism options?" Kept on both full S6 and s6-short. |
Panel composition page
| Element | S5 | S6 |
|---|
| "Minimum share of reserved seats" | Conditional sub-question, shown only if the respondent picked the mostly-random-with-reserved option. | Promoted to an always-shown, required question on the same page, with a "Not applicable (I did not choose reserved seats)" option. Worded conditionally so it reads naturally, but kept inline because Qualtrics pushes true conditional text-entry onto its own page. |
Welcome page, consent, and contact opt-in
| Element | S5 | S6 |
|---|
| Time estimate copy | "About 12-20 minutes." | "About 12 to 20 minutes." for full S6 and "About 8 to 15 minutes." for s6-short, rendered as a word range (no dashes). |
| Follow-up contact opt-in | "Yes, contact me through Prolific." | Provider-neutral: "Yes, you may contact me through the survey provider." (so the same instrument can run on Prolific, Verasight, or any panel). |
Exclusion rules
| Rule | S5 | S6 |
|---|
| B-att (instructed attention check) | Post-hoc exclusion only: recorded, non-blocking; a failer completes the survey and is dropped at analysis time. | In-survey hard gate: one retry, then terminate as a recorded screen-out. Enforced identically on the live site (SectionBPrinciples.tsx) and the Qualtrics/Verasight survey (branch-and-terminate flow). The post-hoc B-att exclusion is retained as a backstop. Justified because the instructed answer ("Neither") is content-neutral, so gating cannot coach the substantive result. |
| B-rev (reverse-coded acquiescence) | Exclude if B-rev >= 3 (slightly disagree or higher). | Exclude only if B-rev >= 5 AND B-bal >= 5 (genuine acquiescence: agreeing officials should be allowed to lie AND should explain honestly). |
| Vote-2024 "not eligible" | Hard exclude. | Removed. Replaced by explicit citizenship screener. |
| Two-tier architecture | Single sample. | Two samples reported side by side: headline (excludes on all checks except B-rev) and confirmatory (also excludes B-rev acquiescence). |
On the locked S5 data, the S6 rules recover 53 respondents at the same
support level (0.3pp difference), improving precision without moving the
headline.
Citizenship handling
| S5 | S6 |
|---|
| Relied on vote-2024 "not eligible" as a proxy for non-citizenship. | Explicit "Are you a U.S. citizen?" screener on the welcome page. "No" hard-blocks participation. Prolific pre-screener also filters at recruitment. |
New arm: s6-short
S6 introduces a trimmed, low-cost variant designed for large-N fielding at
$1 per participant (vs. $5 for full S6).
| What s6-short drops | Rationale |
|---|
| Forced rate-all-four on mechanism designs | Replaced with opt-in (rate at least one). Forced choice is retained. |
| Panel composition page | Made optional/collapsed (skip or expand). |
| Self-reported "did anything change" yes/no on revote | Dropped. Net movement is still computable from vote vs. revote. The one-line re-vote reason is kept on s6-short (required), so the qualitative signal is not lost. |
| Reflection page (recall, pull-pause, attention open text) | Dropped entirely. |
| Per-mechanism open text and mitigation feedback | Dropped. |
Demographic region | Dropped for all S6 arms (not s6-short specific). Reconstructable from state. |
Pre-divergence measures (H1 principle agreement, H2 initial ballot support)
are identical across S6 and s6-short and are poolable. Post-divergence
measures (H3, H4) are reported primary-from-S6 with s6-short as large-N
corroboration, pooled only if a preregistered arm-effect check passes.
Debrief
| Change | Detail |
|---|
| Data-release paragraph | S6 includes a data-release paragraph that mirrors the publication and quoting disclosure shown at consent on the welcome page: "We will publish the combined results from this study along with the analysis code used to produce them, so anyone can verify how the findings were calculated. Any data we release is de-identified. We may quote short written answers in anonymized form." |
Unchanged across S5 and S6
- 14-item Section B principles Likert (same items, same quality checks at the same three fixed positions). Note: the principle-ordering constraint changed (see "Section B randomization" above); the cluster redraw rule was dropped for S6.
- Per-respondent stratified randomization with Fisher-Yates shuffle and three fixed diagnostic slots (the shuffle mechanism is unchanged; only the S5 cluster redraw rule was removed)
- Neutral sponsor label ("American Institutions Study 2026")
- Four mechanism designs and their forced-choice preference measure
- Panel composition questions
- Demographics battery (party, vote-2024, age, gender, state, race/ethnicity, education)
- Final attention check
- Debrief with sponsor reveal and follow-up opt-in
- De-identification and publication of combined results with analysis code