{"type":"ScholarlyArticle","title":"White-Hat Testing for the Ballot Box: A Framework for Election AI Auditing","authors":[{"name":"Chendi Wang","slug":"chendi-wang","given":"Chendi","family":"Wang","affiliation":""},{"name":"Jieying Chen","slug":"jieying-chen","given":"Jieying","family":"Chen","affiliation":""}],"abstract":"Recent research shows that conversational AI can shift voter preferences, with effects persisting for weeks. Yet frontier models exhibit a documented \"persuasion-reliability tradeoff\", producing hallucinated or systematically distorted election information. Despite these risks, election officials lack standardized tools to systematically evaluate AI systems before deployment. We propose CivicAudit-Bench, a stakeholder-guided auditing framework to stress-test large language models for civic hallucinations, false confidence, jurisdiction-dependent failure, and asymmetric refusals/accuracy. This framework introduces a modular, counterfactual, and severity-aware auditing methodology that integrates roll-call–based alignment modeling, entity-swap probing, and jurisdiction-conditional correctness criteria. Informed by engagement with the U.S. Election Assistance Commission, the toolkit consists of three modules: (1) PoliBias-US, a multi-indicator alignment screen combining Congressional roll-call ideology scaling with party-cue counterfactual sensitivity, persona robustness, and narrative-framing alignment; (2) HalluBias-Election, an evidence-linked benchmark that measures hallucinations,  severity-weighted critical errors, and asymmetries via Entity-Swap Counterfactual Probing and a jurisdiction-safe completion criterion; and (3) Disclosure-Test, pre-registered experiments assessing whether transparency and calibrated-uncertainty disclosures reduce overreliance and attenuate persuasion without blocking legitimate civic information. CivicAudit-Bench outputs versioned audit scorecards and a coordinated white-hat disclosure workflow, advancing UN SDG~16 by strengthening democratic information integrity.","doi":"10.24963/ijcai.2026/832","url":"https://ijcai.org/proceedings/2026/832","pdf":"https://ijcai.org/proceedings/2026/0832.pdf","markdown":"https://ijcai.org/proceedings/2026/832.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/832","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"AI and Social Good","track":"AI and Social Good","number":832,"pages":{"first":7483,"last":7491},"keywords":["AI Ethics, Trust, Fairness: AI Ethics, Trust, Fairness","Multidisciplinary Topics and Applications: Multidisciplinary Topics and Applications","Uncertainty in AI: Uncertainty in AI","Humans and AI: Humans and AI"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}