The Alignment Bottleneck in Decomposition-Based Claim Verification
Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Main Track. Pages 5458-5467.
https://doi.org/10.24963/ijcai.2026/608
Keywords
- Natural Language Processing: Resources and evaluation
- Natural Language Processing: Interpretability and analysis of models for NLP
- Multidisciplinary Topics and Applications: News and media
- Natural Language Processing: Text classification
Abstract
Structured claim decomposition is often proposed as a solution for verifying complex, multi-faceted claims, yet empirical results have been inconsistent. We argue that these inconsistencies stem from two overlooked bottlenecks: evidence alignment and sub-claim error profiles. To better understand these factors, we introduce a new dataset of real-world complex claims, featuring temporally bounded evidence and human-annotated sub-claim evidence spans. We evaluate decomposition under two evidence alignment setups: Sub-claim Aligned Evidence (SAE) and Repeated Claim-level Evidence (SRE). Our results reveal that decomposition brings significant performance improvement only when evidence is granular and strictly aligned. By contrast, standard setups that rely on repeated claim-level evidence (SRE) fail to improve and often degrade performance as shown across different datasets and domains (PHEMEPlus, MMM-Fact, COVID-Fact). Furthermore, we demonstrate that in the presence of noisy sub-claim labels, the nature of the error ends up determining downstream robustness. We find that conservative "abstention" significantly reduces error propagation compared to aggressive but incorrect predictions. These findings suggest that future claim decomposition frameworks must prioritize precise evidence synthesis and calibrate the label bias of sub-claim verification models.
- [Atanasova et al., 2020] Pepa Atanasova, Jakob Grue Simon- sen, Christina Lioma, and Isabelle Augenstein. Generating fact checking explanations. In Proc. of the 58th Annual Meeting of the ACL, pages 7352-7364, Online, July 2020.
- ACL.
- [Bastings et al., 2019] Jasmijn Bastings, Wilker Aziz, and Ivan Titov. Interpretable neural predictions with differen- tiable binary variables. In Proc. of the 57th Annual Meeting of the ACL, pages 2963-2977, Florence, Italy, July 2019.
- ACL.
- [Bilal et al., 2024] Iman Munire Bilal, Preslav Nakov, Rob Procter, and Maria Liakata. Generating zero-shot abstrac- tive explanations for rumour verification, 2024.
- [Brodersen et al., 2010] Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M. Buhmann. The balanced accuracy and its posterior distri- bution. In 2010 20th Int. Conf. on Pattern Recognition, pages 3121-3124, 2010.
- [Chen et al., 2022] Jifan Chen, Aniruddh Sriram, Eunsol Choi, and Greg Durrett. Generating literal and implied subquestions to fact-check complex claims. In Proc. of the 2022 Conf. on EMNLP, pages 3495-3516, Abu Dhabi, United Arab Emirates, December 2022. ACL.
- [Chen et al., 2024] Jifan Chen, Grace Kim, Aniruddh Sriram, Greg Durrett, and Eunsol Choi. Complex claim verification with evidence retrieved in the wild. In Proc. of the 2024 Conf. of the NAACL: Human Language Technologies, pages 3569-3587, Mexico City, Mexico, June 2024. ACL.
- [Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proc. of the 2019 Conf. of the NAACL: Human Language Technologies, pages 4171-4186, Minneapolis, Minnesota, June 2019. ACL.
- [Dougrez-Lewis et al., 2022] John Dougrez-Lewis, Elena Kochkina, Miguel Arana-Catania, Tom Sherborne, Yulan Lei, and Maria Liakata. PHEMEplus: Enriching social media rumour verification with external evidence. In Proc. of the Fifth Fact Extraction and VERification Workshop (FEVER), pages 1-13, Dublin, Ireland, May 2022. ACL.
- [Dougrez-Lewis et al., 2025] John Dougrez-Lewis, Mah- mud Elahi Akhter, Federico Ruggeri, Sebastian Löbbers, Yulan He, and Maria Liakata. Assessing the reasoning ca- pabilities of llms in the context of evidence-based claim verification, 2025.
- [Ge et al., 2025] Ziyu Ge, Yuhao Wu, Daniel Wai Kit Chin, Roy Ka-Wei Lee, and Rui Cao. Resolving conflicting evidence in automated fact-checking: A study on retrieval- augmented llms. ArXiv, abs/2505.17762, 2025.
- [Glockner et al., 2022] Max Glockner, Yufang Hou, and Iryna Gurevych. Missing counter-evidence renders NLP fact-checking unrealistic for misinformation. In Proc. of the 2022 Conf. on EMNLP, pages 5916-5936, Abu Dhabi, United Arab Emirates, December 2022. ACL.
- [Gong et al., 2024] Haisong Gong, Huanhuan Ma, Qiang Liu, Shu Wu, and Liang Wang. Navigating the noisy crowd: Finding key information for claim verification. CoRR, abs/2407.12425, 2024.
- [Guo et al., 2022] Zhijiang Guo, Michael Schlichtkrull, and Andreas Vlachos. A survey on automated fact-checking. Transactions of the ACL, 10:178-206, 2022.
- [Hu et al., 2022] Xuming Hu, Zhijiang Guo, GuanYu Wu, Ai- wei Liu, Lijie Wen, and Philip Yu. CHEF: A pilot Chinese dataset for evidence-based fact-checking. In Proc. of the 2022 Conf. of the NAACL: Human Language Technologies, pages 3362-3376, Seattle, United States, July 2022. ACL.
- [Hu et al., 2024] Qisheng Hu, Quanyu Long, and Wenya Wang. Decomposition dilemmas: Does claim decomposi- tion boost or burden fact-checking performance?, 2024.
- [Huang et al., 2024] Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet, 2024.
- [Jolly et al., 2022] Shailza Jolly, Pepa Atanasova, and Is- abelle Augenstein. Generating fluent fact checking ex- planations with unsupervised post-editing. Information, 13(10):500, 2022.
- [Kamoi et al., 2023] Ryo Kamoi, Tanya Goyal, Juan Diego Rodriguez, and Greg Durrett. WiCE: Real-world entailment for claims in Wikipedia. In Proc. of the 2023 Conf. on EMNLP, pages 7561-7583, Singapore, December 2023. ACL.
- [Kochkina and Liakata, 2020] Elena Kochkina and Maria Li- akata. Estimating predictive uncertainty for rumour verifi- cation models. In Proc. of the 58th Annual Meeting of the ACL, pages 6964-6981, Online, July 2020. ACL.
- [Lei et al., 2016] Tao Lei, Regina Barzilay, and Tommi Jaakkola. Rationalizing neural predictions. In Proc. of the 2016 Conf. on EMNLP, pages 107-117, Austin, Texas, November 2016. ACL.
- [Li et al., 2024] Miaoran Li, Baolin Peng, Michel Galley, Jianfeng Gao, and Zhu Zhang. Self-checker: Plug-and-play modules for fact-checking with large language models. In Findings of the ACL: NAACL 2024, pages 163-181, Mexico City, Mexico, June 2024. ACL.
- [Liu et al., 2024] Hui Liu, Wenya Wang, Haoru Li, and Hao- liang Li. TELLER: A trustworthy framework for explain- able, generalizable and controllable fake news detection. In Findings of the ACL: ACL 2024, pages 15556-15583, Bangkok, Thailand, August 2024. ACL.
- [Liu et al., 2025] Xinyu Liu, Junhao Yu, Jing Zhao, and Zhouhan Li. Bidev: Bidirectional decomposition for com- plex claim verification, 2025.
- [Lyu et al., 2025] Xiucheng Lyu, Chengyu Cao, Mingwei Sun, and Ruifeng Xu. Mitigating error propagation in multi- hop fact verification with logic reasoning. International Journal of Machine Learning and Cybernetics, 2025.
- [Ma et al., 2025] Jiatong Ma, Linmei Hu, Rang Li, and Wenbo Fu. Local: Logical and causal fact-checking with llm-based multi-agents. In Proc. of the ACM on Web Con- ference 2025, page 1614-1625, New York, NY, USA, 2025. ACM.
- [Min et al., 2023] Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. FactScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proc. of the 2023 Conf. on EMNLP, pages 12076-12100, Singapore, December 2023. ACL.
- [Nakov et al., 2021] Preslav Nakov, David Corney, Maram Hasanain, Firoj Alam, Tamer Elsayed, Alberto Barrón- Cedeño, Paolo Papotti, Shaden Shaar, and Giovanni Da San Martino. Automated fact-checking for assisting human fact-checkers. In Proc. of the 30th IJCAI, pages 4551-4558. IJCAI, 2021. IJCAI
- [Pan et al., 2023] Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min-Yen Kan, and Preslav Nakov. Fact-checking complex claims with program-guided reasoning. In Proc. of the 61st Annual Meeting of the ACL, pages 6981-7004, Toronto, Canada, July 2023. ACL.
- [Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In Proc. of the 40th ACL, ACL '02, pages 311-318, USA, 2002. Associa- tion for Computational Linguistics.
- [Quelle and Bovet, 2023] Dorian Quelle and Alexandre Bovet. The perils and promises of fact-checking with large language models. In Proc. of the 2023 ACM Conf. on Fairness, Accountability, and Transparency, 2023.
- [Saakyan et al., 2021] Arkadiy Saakyan, Tuhin Chakrabarty, and Smaranda Muresan. COVID-fact: Fact extraction and verification of real-world claims on COVID-19 pandemic. In Proc. of the 59th ACL and the 11th IJCNLP, pages 2116- 2129, Online, August 2021. Association for Computational Linguistics.
- [Schlichtkrull et al., 2023a] Michael Schlichtkrull, Zhijiang Guo, and Andreas Vlachos. Averitec: A dataset for real- world claim verification with evidence from the web. In Advances in NeurIPS 36, 2023.
- [Schlichtkrull et al., 2023b] Michael Schlichtkrull, Zhijiang Guo, and Andreas Vlachos. Averitec: A dataset for real- world claim verification with evidence from the web. ArXiv, abs/2305.13117, 2023.
- [Sundriyal et al., 2022] Megha Sundriyal, Parantak Rani, Sriparna Saha, Pushpak Bhattacharyya, and Tanmoy Chakraborty. Empowering the fact-checkers! automatic identification of claim spans on Twitter. In Proc. of the 2022 Conf. on EMNLP, pages 7701-7715, Abu Dhabi, United Arab Emirates, December 2022. ACL.
- [Tang et al., 2024] Liyan Tang, Philippe Laban, and Greg Durrett. MiniCheck: Efficient fact-checking of LLMs on grounding documents. In Proc. of the 2024 Conf. on EMNLP, pages 8818-8847, Miami, Florida, USA, Novem- ber 2024. ACL.
- [Wang and Shu, 2023] Haoran Wang and Kai Shu. Explain- able claim verification via knowledge-grounded reason- ing with large language models. In Findings of the ACL: EMNLP 2023, pages 6288-6304, Singapore, December 2023. ACL.
- [Wang, 2024] Yuxia et al. Wang. Factcheck-bench: Fine- grained evaluation benchmark for automatic fact-checkers. In Findings of the ACL: EMNLP 2024, pages 14199-14230, Miami, Florida, USA, November 2024. ACL.
- [Wanner et al., 2024] Miriam Wanner, Seth Ebner, Zheng- ping Jiang, Mark Dredze, and Benjamin Van Durme. A closer look at claim decomposition. In Proc. of the 13th Joint Conf. on Lexical and Computational Seman- tics (*SEM 2024), pages 153-175, Mexico City, Mexico, June 2024. ACL.
- [Xu et al., 2025] Wenyan Xu, Dawei Xiang, Tianqi Ding, and Weihai Lu. Mmm-fact: A multimodal, multi-domain fact- checking dataset with multi-level retrieval difficulty, 2025.
- [Yang et al., 2024] Sohee Yang, Jonghyeon Kim, Joel Jang, Seonghyeon Ye, Hyunji Lee, and Minjoon Seo. Improving probability-based prompt selection through unified evalu- ation and analysis. Transactions of the ACL, 12:664-680, 2024.
- [Yang et al., 2025] An Yang, Anfeng Li, Baosong Yang, Be- ichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. Accessed: 2026-01-15.
- [Zhang and Gao, 2023] Xuan Zhang and Wei Gao. Towards LLM-based fact verification on news claims with a hier- archical step-by-step prompting method. In Proc. of the 13th IJCNLP and the 3rd AACL, pages 996-1011, Bali, Indonesia, November 2023. ACL.
- [Zhang et al., 2019] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Eval- uating text generation with bert. ArXiv, abs/1904.09675, 2019.
- [Zhang et al., 2024] Zhihao Zhang, Yixing Fan, Ruqing Zhang, and Jiafeng Guo. A claim decomposition bench- mark for long-form answer verification, 2024.
- [Zhao et al., 2024] Xiaoyan Zhao, Lingzhi Wang, Zhanghao Wang, Hong Cheng, Rui Zhang, and Kam-Fai Wong. PACAR: Automated fact-checking with planning and cus- tomized action reasoning using large language models. In Proc. of the 2024 LREC-COLING, pages 12564-12573, Torino, Italia, May 2024. ELRA and ICCL.
- [Zubiaga et al., 2016] Arkaitz Zubiaga, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, and Peter Tolmie. Analysing how people orient to and spread rumours in social media by looking at conversational threads. PLOS ONE, 11(3):1-29, 2016.
- [Zubiaga et al., 2018] Arkaitz Zubiaga, Ahmet Aker, Kalina Bontcheva, Maria Liakata, and Rob Procter. Detection and resolution of rumours in social media: A survey. ACM Computing Surveys, 51(2):1-36, 2018.
References were extracted automatically from the PDF and may contain errors.
Full text extracted automatically from the PDF; figures, tables and equations may differ. The PDF is the version of record.
1 Introduction
The task of claim verification involves determining the veracity for a given claim based on associated evidence. It is crucial for a number of different tasks, such as rumour or misinformation verification and fact-checking [Zubiaga et al., 2018; Glockner et al., 2022; Guo et al., 2022; Pan et al., 2023]. Claim verification is particularly challenging when dealing with complex claims which conflate multiple facts, requiring granular reasoning over evidence for accurate verification. To verify complex claims, humans identify several sub-parts (sub-claims), gather multiple pieces of evidence and perform multi-step reasoning [Nakov et al., 2021].
In such cases mapping the claim to a complete set of evidence is non-trivial; A common approach is to disentangle the sub-claims from the initial claim, (claim decomposition), which is then followed by sub-claim verification and rationale generation for each resulting sub-claim [Chen et al., 2022; Pan et al., 2023]. Although laborious, this process offers more transparency in the decision-making process, especially in cases of insufficient or contradictory evidence relevant to different parts of a claim. However, this introduces an additional error propagation factor since errors in sub-claim decomposition or sub-claim labels can mislead claim-level inference.
Performing claim verification using Large Language Models (LLMs) [Wang and Shu, 2023; Quelle and Bovet, 2023] has recently gained popularity. Apart from single LLMs, Agentic systems have also been applied to perform complex claim verification [Ma et al., 2025; Liu et al., 2025]. However, these still inherit the error propagation caveats due to decomposition and misalignment of granular evidence.
Real world claim verification requires complex multi-hop reasoning steps. Reasoning with LLMs suffers from important shortcomings. For instance, [Dougrez-Lewis et al., 2025] show that LLMs struggle with abductive reasoning while LLM performance on claim verification is very much domain-dependent. Specifically, LLMs are better at handling simple claims fromWikipedia compared to real-world data which manifest complex claims and hierarchical structure. This leads to a major bottleneck in claim verification and fact-checking since most datasets, where the state-ofthe-art (SOTA) is trained and evaluated, are sourced from Wikipedia [Guo et al., 2022]. Studies that have evaluated the benefits of claim decomposition for fact-checking report conflicting results [Tang et al., 2024; Wanner et al., 2024; Chen et al., 2024]. According to [Tang et al., 2024] claim decomposition is not necessary to improve LLM performance on fact checking. By contrast, [Wanner et al., 2024] show that the impact of decomposition is method-specific, especially when considering LLM-based decomposition. [Hu et al., 2024] carried out an in-depth analysis of claim decomposition, revealing several deciding factors, including how the decomposition process is defined and how evidence information is associated with each sub-claim. Thus, the benefits of claim decomposition for complex claims are currently poorly understood. We argue these conflicting findings largely stem from two factors: (i) how evidence is aligned and aggregated at the sub-claim level, and (ii) whether sub-claim veracity signals are reliable or noisy. Both of these can directly affect error propagation to claim-level inference. Based on this, we hypothesize that decomposition helps only when evidence is aligned at the sub-claim level, and sub-claim veracity labels are reliable; otherwise, decomposition can degrade performance in claim verification.

Here we investigate how sub-claim decomposition impacts complex claim verification in real-world data. Fig. 1 illustrates our approach. We assume that a complex claim C corresponding to an evidence set EC is decomposed into sub-claims {sj}mj=1 , each of which are paired with corresponding evidence sets {Ej}mj=1 and annotated with sub-claim veracity labels {yj}mj=1 . These structured triples (sj , Ej , yj) serve as inputs for sub-claim-level prediction and, when aggregated, form the basis for claim-level veracity inference under both oracle and noisy labels. Our approach connects human annotation, claim decomposition, and model evaluation into a unified framework for complex claim verification. We show that decomposition can improve performance under ideal labels (oracle), but can be neutral or harmful when labeling is noisy. Furthermore, to test the generalisability of the decomposition effect, we evaluate the same pipelines not only on temporally bounded rumour verification ( [Dougrez-Lewis et al., 2022]), but also on two additional fact-checking datasets (MMM-Fact [Xu et al., 2025] and COVID-Fact[Saakyan et al., 2021]), which differ with respect to domain and evidence characteristics. We make the following contributions,
- We introduce a dataset of real-world claims decomposed into sub-claims accompanied by manually annotated subclaim-level evidence and fine-grained veracity labels (§3).
- We formalize two complementary experimental settings (i) claim veracity under oracle sub-claim labeling (upper bound) (§5.1) and under predicted sub-claim labeling (real-world scenario) (§5.4),and (ii) sub-claim veracity (§5.3) as the upstream task; together these quantify how sub-claim labeling errors propagate to claim-level verification.
- We show decomposition improves performance under sub-claim aligned evidence and reliable sub-claim signals, but can also be neutral or harmful under noisy labeling and coarse evidence. Thus, we highlight that the benefits of decomposition are conditional, dependent on
evidence quality and sub-claim error signals.
2 RelatedWork
Automatic ClaimDecomposition. Several works have proposed decomposing fact-checking claims into atomic text segments. Such segments (sub-claims/atomic facts/atomic propositions), are typically extracted by prompting LLMs [Zhang and Gao, 2023], via hybrid approaches relying on keywords [Gong et al., 2024] or predefined templates [Pan et al., 2023; Liu et al., 2024]. Prompts are either textual propositions [Kamoi et al., 2023; Tang et al., 2024] or queries for information retrieval [Chen et al., 2022; Chen et al., 2024]. Apart from prompting methods, orchestrated multi-agent systems have also been used for decomposition-based claim verification [Zhao et al., 2024; Ma et al., 2025]. [Hu et al., 2024] provide valuable insights for evaluating the effects of claim decomposition. However, their work pertains to datasets containing automatically extracted sub-claim evidence and does not include real-world complex claims, unlike the task we address here.
ClaimDecomposition Datasets. Researchers have applied claim decomposition for several purposes, including finegrained evidence retrieval [Kamoi et al., 2023; Zhang and Gao, 2023], and factual verification of generated content to mitigate hallucinations [Min et al., 2023; Wang, 2024; Zhang et al., 2024].
To this effect, several fact-checking datasets containing subclaims have been proposed. Notable examples are BingCheck, for evaluating the actuality of LLM-generated texts in response to fact-checking [Li et al., 2024], LLM-AggreFact [Tang et al., 2024] containing a suite of fact-checking corpora, ClaimDecomp centered on extracting explicit and implicit subclaims [Chen et al., 2022], WiCE for fine-grained evidence retrieval on Wikipedia [Kamoi et al., 2023], and AVERITEC for real-world claim verification [Schlichtkrull et al., 2023a].
ClaimDecomp [Chen et al., 2022] and WiCE [Kamoi et al., 2023] are the closest to our setting. In ClaimDecomp, subclaims are presented as questions to help evaluate the veracity of complex claims extracted from Politifact. Both the claim and the evidence-based justification, written by journalists, are used to reverse-engineer the verification process, thus introducing both literal (produced from the claim) and implied sub-claims (produced from evidence). Here we adhere to a more realistic setting of decomposing claims with no prior knowledge of existing evidence. In WiCE, claim decomposition is carried out via LLMs, and each sub-claim is associated with evidence manually annotated from Wikipedia articles. Similarly, our dataset provides manually annotated evidence texts for sub-claims but unlike WiCE, we focus on real-world claim verification, consisting of social media content, where rumours often emerge and spread [Zubiaga et al., 2018].
3 Dataset
3.1 Task Overview
Our task involves assessing the veracity of complex claims through structured decomposition into sub-claims, each supported by relevant evidence. We first decompose claims into distinct sub-claims, then independently verify each sub-claim using associated evidence. We then analyze how this pipeline behaves under different evidence association strategies (claimlevel vs sub-claim-aligned evidence) and different supervision regimes (oracle vs predicted sub-claim labels), to quantify error propagation to claim-level inference.
3.2 Dataset Creation
We filter complex claims within the PHEME dataset [Zubiaga et al., 2016] by removing claims containing fewer than three verbs and two sentences. This heuristic is a proxy method for multi-fact composition introduced by [Chen et al., 2022]. PHEME contains a collection of claims posted on Twitter as time-sensitive breaking news events were unfolding within a specific time-frame. It contains 1972 rumourous claims related to five events and each of the rumours are annotated with a veracity label (True, False, Unverified). To decompose complex claims into sub-claims, we adopted a fixed decomposition procedure to isolate the effects of evidence association and label noise on verification performance, rather than optimizing the decomposition model itself. We used an LLM-based schema introduced by FactScore [Min et al., 2023] with gpt-3.5-turboinstruct to break down text into short statements covering only one piece of information. This is followed by a manual evaluation process using the claim detection guidelines in [Sundriyal et al., 2022] to ensure that the resulting sub-claims contain check-worthy information and are comprehensive when considered without the original claim. In total, we extract 423 complex claims from PHEME, covering five distinct events. We then leveraged PHEMEplus [Dougrez-Lewis et al., 2022] annotations to map these claims to relevant evidence from news articles. PHEMEplus contains evidence-providing articles for the claims within PHEME in a time constrained manner (within 2 days of each event). We obtained 399 claims after accounting for evidence quality and dataset artifacts in PHEMEplus. We filtered out image-only evidence, hyperlinks and embedded ads. We do not consider claims with associated images in the tweet since decomposing such claims result in non-sense sub-claims (URL links or pictures).
Domain generalisation. We additionally use MMM-Fact and COVID-Fact to test generalisability of our findings beyond event-centric rumour verification. MMM-Fact is a large-scale multi-domain fact-checking benchmark with multisource evidence. It has a full size of 125,449 samples with evidence grouped into retrieval-difficulty levels (Basic/Intermediate/Advanced). We use intermediate level which had 6-10 evidence sources per claim (21,873). COVIDFact is a domain-specific (health/COVID-19) real-world factchecking dataset with 4,086 claims paired with single source evidence. Their goal was to test multi-document synthesis (MMM-Fact), to better assess evidence alignment and to see the decomposition and alignment effect in a narrower-domain with less structural complexity (COVID-Fact).
3.3 Annotation Details
Given a claim, associated sub-claims and evidence providing articles for the original claim, the task was to label the veracity of the sub-claims and highlight relevant spans of evidence supporting the assigned veracity label. The highlighted evidence was later extracted as sub-claim level evidence. In particular, each sub-claim was labeled as true (T), false (F), and Unverified (U). Starting with 1169 sub-claims, these were evenly distributed to three annotators, Phd students in Computer Science, fluent in English, so that 300 samples were annotated by all. The latter were used to calculate the Inter Annotator Agreement (IAA) score. In short, we double-annotated 300 sub-claims for IAA; the remaining sub-claims were singleannotated and distributed evenly, resulting in ≈290 unique items per annotator. Annotators labeled a sub-claim as (T) if directly supported by the evidence, (F) if contradicted, and (U) if the evidence was insufficient or ambiguous. Evidence spans were selected as the minimal set of sentences justifying the label. We computed IAA as Bennett's S score to account for label imbalance, resulting in a score of 0.81. We also analyzed sentence-level overlap among annotators using BLEU and BERTScore. The Average BLEU[Papineni et al., 2002] ranged from ≈ 0.40-0.48 across pairs, and average symmetric BERTScore [Zhang et al., 2019] F1 was around 0.49-0.56, indicating moderate to strong overlap in selected evidence spans.
| Category | Split | T% | U% | F% |
|---|---|---|---|---|
| Claim Veracity | Total Train Test | 48.37 47.98 50.00 | 31.33 31.78 29.49 | 20.3 20.25 |
| Sub-claim Veracity | Total Train Test | 57.66 57.11 59.75 | 34.05 34.70 31.54 | 8.3 8.19 8.71 |
| 20.51 |
Unlike Averitec, [Schlichtkrull et al., 2023b], our verification process was not constructed as a QA process but rather the annotators had to verify sub-claims based on the provided evidence. Contrary to Factcheck-Bench's [Wang, 2024] automatic evidence retrieval, we manually selected evidence spans to isolate the verification problem from retrieval errors[Ge et al., 2025].
Our dataset differs from previous work in that we provide: (i) human-annotated evidence texts per extracted sub-claim rather than evidence automatically extracted by LLMs and (ii) evidence documents per claim with a closed retrieval setting. In particular, we consider only evidence documents that were restricted to articles published within a fixed temporal window around the event [Dougrez-Lewis et al., 2022], preventing leakage from post-hoc reporting and ensuring a realistic realtime verification setting. [Zubiaga et al., 2018]. Furthermore, this controlled evidence setting allows us to isolate how subclaim evidence alignment and sub-claim label noise affect decomposition-based verification, without conflating effects from open-domain retrieval.
3.4 Dataset Details
Dataset statistics are in Table 1, including distribution of veracity labels for both complex claims and sub-claims, with 399 complex claims corresponding to 1169 sub-claims. We used the full dataset for oracle and ablation setups in Tables 2 and 3 with 929 samples for training the baseline and GNN
models in §4.3 and 240 as testing. The results in Tables 3, 4, and 5, were calculated using this test set. On MMM-Fact we extracted 1181 (5.40% of intermediate subset) complex claims and 163 complex claims in COVID-Fact (3.99% full dataset). These resulted in 2326 and 447 sub-claims respectively for each dataset. Note that these datasets do not contain manually annotated sub-claim labels or corresponding sub-claim aligned evidence.
4 Experiments
While claim verification has increasingly been addressed alongside explanation generation [Atanasova et al., 2020; Jolly et al., 2022; Bilal et al., 2024], here we focus on assessing the merits of claim decomposition given (i) claim-level verification and (ii) sub-claim verification labels, while considering two evidence alignment settings (SRE Eq. (5) vs SAE Eq. (6)) ((§5.3)) and two levels of labeling quality for sub-claims (oracle vs predicted/noisy). Evidence is temporally bounded and pre-linked (no open-domain retrieval) to isolate decomposition effects from open retrieval errors.
We denote as C a complex claim and as EC = {ECi}ni=1 the complete evidence set corresponding to the original claim C. C is decomposed into a set of m sub-claims {sj}mj=1 , each associated with an evidence set {Ej}mj=1 and a veracity label yj ∈ {T, F,U}. Here, yj denotes either oracle subclaim labels (upper bound), predicted labels (realistic), or is omitted in label-free ablation variants (Eq. (7) & Eq. (8)). The overall claim-level veracity YC will depend on the veracity and evidence of its sub-claims as follows:
where each sub-claim evidence Ej derives from some claim evidence ECi. To ensure temporal and contextual fidelity, evidence sets are restricted to information available before the claim timestamp:
where D is the document collection, t(e) is the publication time of evidence e, and t(C) is the time when the claim C emerged. This constraint prevents open-domain retrieval to avoid compromising the time-sensitive nature of the dataset as it would otherwise retrieve evidence not available at the time of unfolding events; events happening in real time usually generate unverified claims that are later resolved within a few hours to a few days. As our goal was not to propose a new retrieval method, we fix evidence to measure the causal effect of decomposition choices instead of compounding it with retrieval errors.
4.1 ClaimVeracity
Experiment details. Here the task is to verify a claim (label as T/F) given a set of evidence texts, known to be relevant to the claim where evidence documents are pre-linked to each claim via PHEMEPLUS within a temporally bounded window. We do not consider unverified (U) labels and drop them as these were not typically supported by corresponding evidence within the PHEMEPlus subset. All experiments at claimlevel are conducted using the Qwen3-14B [Yang et al., 2025] reasoning model.
We first perform claim-level veracity prediction using oracle sub-claim labels [Huang et al., 2024; Yang et al., 2024] to establish an upper bound on performance. Here, yj the gold sub-claim is provided as part of the structured input to test the best-case scenario for decomposition; these are later replaced with predicted labels Eq. (10) or omitted in ablations (Eq. (7) & Eq. (8)) to reflect realistic usage. The model predicts claim veracity as:
where E represents the evidence configuration used. We consider the following evidence alignment configurations per setting: Claim-level evidence (ECi), for the Vanilla and Subclaims with Repeated Claim-level Evidence (SRE) experimental settings, and Sub-claim level evidence (Ej) for the Sub-claims with Aligned Evidence (SAE) setting:
In practice, SRE repeats the same claim-level evidence block after each sub-claim in the prompt, whereas SAE inserts the sub-claim-specific evidence block Ej next to its sub-claim sj .
Ablation. We conduct a set of targeted ablation studies, with the aim of isolating the impact of the oracle sub-claim veracity labels on downstream model performance and quantifying the influence of evidence alignment. Here, we use the same model setting as Equation 3, but in the ablations the evidence for SRE and SAE isn't paired with any sub-claim veracity labels.
Ablation SRE also mirrors real-world use cases where we may have sub-claims available, with the parent claim and corresponding evidence but no sub-claim labels or sub-claim level evidence. Indeed, Ablation SRE is the setup used in the experiments for the MMM-Fact and COVID-Fact datasets. The SAE setting was not possible for these as contrary to the dataset introduced in this paper, they lack human-annotated sub-claim evidence spans, a resource-intensive annotation process.
Metrics and statistical significance. Claim-level verification performance (See Table 2) is gauged using two primary metrics: macro-averaged F1-score and balanced accuracy [Brodersen et al., 2010]. We use both due to the class imbalance between claim veracity labels. Models were run over three independent random seeds (six for ablations).
Statistical Significance: We applied paired statistical testing on claims across model configurations to evaluate the statistical significance of our findings. We performed a paired bootstrap analysis (Primary Performance ∆) to assess the differences in F1 and Accuracy metrics. Claims were selected with replacement (N = 1000 resamples) to compute the performance difference (∆ = MetricSAE/SRE - Metricbaseline) for each resample. We report the two-sided bootstrap p-value (pboot) (how often ∆s crosses zero). To evaluate the significance of changes in correct/incorrect predictions, we utilized McNemar's test (Categorical Agreement) on the 2 × 2 table of paired correctness on the same claims. This focuses specifically on instances where (SAE/SRE)) and baseline disagree. We report both the McNemar p-value and the Odds Ratio (OR) of the disagreeing pairs (b01/b10) as a measure of effect size, indicating how many times more likely the model is to correct a baseline error than to introduce a new one.
4.2 Sub-claimVeracity
Experiment details: Sub-claim veracity is formulated as a classification task [Guo et al., 2022] where individual subclaims are classified with the claim-level evidence (claim level evidence is repeated per sub-claim). Each sub-claim is considered to be a discrete instance. At the sub-claim level, each sub-claim sj is independently classified as:
We predict ˆ yj ∈ {T, F, U}, since decomposed sub-claims can remain unresolved even when the overall claim is verifiable. We evaluate using the macro-averaged F1-score. We do not consider sub-claim veracity prediction given sub-claim evidence as this deviates from real-world scenarios where only claim-level evidence is available e.g. to journalists.
Predicted sub-claim veracity labels are then aggregated to infer claim-level veracity under noisy sub-claim labels (setting SRE_Noisy in Table 3):
Here, f is the classifier that aggregates the claim and subclaim information. This setup allows us to assess how noisy sub-claim predictions ˆ yj influence overall claim verification. We use an LLM-based aggregator, and leave a systematic comparison against deterministic or learned aggregation rules (e.g., conjunctive/disjunctive logic over ˆ yj , or meta-classifiers) as future work.
4.3 Models for Sub-claimVeracity Prediction
Graph Neural Network Baseline. We chose GNNs as a strong structured baseline. Our Graph Neural Network (GNN) is a variant of [Bilal et al., 2024], able to leverage evidence from both news articles and Twitter threads, tested on PHEMEplus. It is structured as a bipartite graph to perform sub-claim veracity prediction with two distinct node sets: tweet nodes (preserving thread structure) (xt ∈ Rn×512 ) and evidence nodes (xe ∈ Rm×128 ), interconnected via dedicated edges. Tweet and evidence texts are encoded into initial node embeddings using pretrained encoders. The training was done with leave-one-event-out [Kochkina and Liakata, 2020] (learning rate=5e-5 , weight decay= 1e-4 , batch size=20).
Encoder Models. We use CHEF [Hu et al., 2022] a BERTbased sub-claim verifier that performs evidence selection before veracity classification. We report the two strongest CHEF variants: CHEFlatent (hard selection via Hard Kumaraswamy [Bastings et al., 2019]) and CHEFrl(Bernoulli selection optimized with RL [Lei et al., 2016]). We additionally use BERT[Devlin et al., 2019] as a strong encoder baseline: larger encoders tended to overfit (low-resource setting) in our preliminary trials, so we use BERT as a competitive and standard control. We chunk long evidence to fit the model context (learning rate=5e-5 , weight decay=1e-4 , batch size=8).
| Setup | Dataset | F1 ± std | ∆ | pboot | OR / McNemar(p) | Balanced Acc. ± std | ∆ | pboot | OR / McNemar(p) |
|---|---|---|---|---|---|---|---|---|---|
| Vanilla Oracle_SRE | PHEMEplus | 0.5643 ± 0.0091 0.5872±0.0127 0.6268 ± 0.0098 | - | - | - / 0.0227 | 0.6072 ± 0.0074 0.6558 ± 0.0135 | - | - 0.774 | - 1.52 / 0.0227 2.13 / 0.0000 |
| Vanilla | +0.0229 | 0.124 | 1.52 | 0.6117±0.0136 | +0.0045 | ||||
| Oracle_SAE | +0.0625 | 0.0000 | 2.13 / 0.0000 | +0.0486 | 0.0000 | ||||
| CovidFact | 0.7365 ± 0.0248 | - | - | - | 0.7559 ± 0.0240 | - | - | - | |
| Ablation_SRE | 0.7252±0.0161 | -0.0113 | 0.549 | 0.96 / 0.5599 | 0.7533 ± 0.0148 | -0.0326 | 0.115 | 0.86 / 0.1153 | |
| Vanilla | MMM-Fact | 0.7550 ± 0.0069 | - | - | - | 0.7451 ± 0.0065 | - | - | - |
| Ablation_SRE | 0.6878±0.0064 | -0.0672 | 0.0000 | 0.05 / 0.0000 | 0.7120 ± 0.0127 | -0.0377 | 0.0000 | 0.05 / 0.0000 |
Large Language Model. For both claim and sub-claim veracity prediction our LLM of choice is Qwen3-14B reasoning model. We evaluate Qwen3-14B in zero-shot mode with a fixed prompt (temperature=0.3). We initially used Llama 3.1 8B-instruct and Phi4 Reasoning-14B but found both failed at structured input based prediction.
5 Results
5.1 ClaimVeracity Results
As Table 2 shows, the benefits of decomposition depend heavily on how the evidence is aligned. On PHEMEPlus, decomposition offers a clear advantage when sub-claim evidence is available and aligned with sub-claims (SAE). We see statistically significant gains over the Vanilla baseline (+ 0.0625 F1 and +0.0486 balanced accuracy). However, simply repeating claim-level evidence (SRE) on the same dataset, without associated sub-claim evidence to match the sub-claims and their labels, produces negligible changes, suggesting that usefulness of decomposition is conditional upon evidence alignment. The limitations of SRE become even clearer on MMM-Fact and COVID-Fact where it fails to improve verification on either dataset. Vanilla slightly edges out on COVID-Fact, while on MMM-Fact, the drop from providing noisy sub-claims (without labels or evidence) is substantial (macro-F1 ∆ : -0.0672, balanced accuracy ∆ : -0.0521). These findings validate our core hypothesis that decomposition works when evidence is tightly coupled with valid sub-claim signals. Without such alignment, the process tends to introduce noise that harms the final inference, a phenomenon we analyze further in (§5.4). These findings highlight the need for proper evidence synthesis.
5.2 Ablation Results
Table 3 highlights a trade-off between evidence granularity and the quality of sub-claim labels. When using redundant claimlevel evidence but no sub-claim veracity labels (Ablation SRE), the model relies little on sub-claim veracity. However, in the aligned setting where sub-claim evidence is provided (Ablation SAE), the absence of sub-claim veracity labels is detrimental. The model performs significantly worse than standard SAE and even falls below the SRE baseline. This indicates that granular, sub-claim-level, evidence is insufficient on its own. Without the signal from correct sub-claim veracity labels, the model struggles to synthesize the fine-grained information, leading to inference errors that coarser evidence setups seem to avoid.
5.3 Sub-claimVeracity Results
Table 4 presents the sub-claim veracity results. The zeroshot Qwen3-14B sets the state of the art with 56.94% macroF1. The performance gap between the GNN (45.79%) and the encoder-based CHEF variants (≈ 28%) underscores the necessity of structured modeling for this task. While encoders struggle with overfitting and evidence integration, the GNN successfully captures evidence interactions, though it still falls short of the LLM's reasoning capabilities. However, the lowresource nature of the dataset makes it much harder for trained encoders to perform well compared to much larger LLMs.
5.4 Effects of Noisy Sub-claimPredictions
Table 3 shows that replacing oracle sub-claim labels with predicted ones reduces performance. However, the severity of this drop depends on the evidence configuration. Under the aligned SAE setting, the degradation is moderate (Qwen SAENoisy: ∆- 0.0304 F1). However, under SRE, the models collapse (dropping to ≈ 0.44 F1), confirming that redundant claim-level evidence exacerbates error propagation from noisy sub-claim labels. This error propagation pattern aligns with findings from [Lyu et al., 2025] in multi-hop verification, confirming that intermediate step errors compound through the pipeline. A deeper look at the per-model error analysis reveals distinct model biases. From Table 5 we can see that, during sub-claim prediction, when GNN commits (non-U), its accuracy on T/F (Accv) is comparable to Qwen (0.625 vs 0.646), but it abstains far more (35% vs 15.8%) and almost never detects refutations (RF 0.039 vs. 0.314). This gap is driven primarily by coverage on verifiable sub-claims (T/F). Qwen issues a polarity label for 85.7% of verified cases (Covver=0.857), while GNN commits on only 66.7% (0.667). Moreover, Qwen's increased refutation sensitivity comes with modest precision (PF 0.276 vs. 0.154), indicating more frequent false positives for F alongside higher RF . Thus, decomposition success depends not only on correctness when committing, but on the model's commit/abstain and refute/affirm bias. This distinction explains the downstream effects where in the presence of fine-grained evidence (SAE), the GNN's tendency to predict Unverified acts as a safety mechanism, but also misses crucial refutations. Ultimately, this analysis shows that sub-claim macro-F1 is a poor predictor of success and that error analysis, especially the balance between conservative abstention and aggressive detection determines whether decomposition can improve verification.
| Setup | F1Macro ± std | ∆F1 | Balanced Accuracy ± std | ∆Acc |
|---|---|---|---|---|
| Oracle_SRE | 0.5872 ± 0.0127 | - | 0.6117 ± 0.0136 | - |
| Ablation_SRE | 0.5808 ± 0.0064 | -0.0064 | 0.6259 ± 0.0052 | +0.0142 |
| Oracle_SAE | 0.6268 ± 0.0098 | - | 0.6558 ± 0.0132 | - |
| Ablation_SAE | 0.5485 ± 0.0052 | -0.0783 | 0.6220 ± 0.0065 | -0.0338 |
| Effect of Noisy sub-claim Predictions on Claim Verification (all ∆ vs. SAE_oracle) | ||||
| Oracle_SAE | 0.6268 ± 0.0098 | - | 0.6558 ± 0.0132 | - |
| Qwen Noisy_SAE | 0.5964 ± 0.0360 | -0.0304 | 0.6425 ± 0.0595 | -0.0133 |
| Qwen Noisy_SRE | 0.4335 ± 0.0439 | -0.1933 | 0.4411 ± 0.0512 | -0.2147 |
| GNNNoisy_SAE | 0.5839 ± 0.1202 | -0.0429 | 0.5963 ± 0.1361 | -0.0595 |
| GNNNoisy_SRE | 0.4416 ± 0.0489 | -0.1852 | 0.4399 ± 0.0609 | -0.2159 |
| Model | F1 Macro (%) |
|---|---|
| CHEFlatent CHEFrl Bert-Chunk GNN Qwen3-14B | 27.30±7.11 28.92 ± 6.56 41.90±0.57 45.79 ± 1.25 56.94±0.92 |
6 Discussion and FutureWork
The Alignment Bottleneck. Our experiments clarify a boundary condition for decomposition-based verification: claim decomposition is not a universal performance booster, but rather depends on evidence granularity. Decomposition improves performance only when sub-claims are paired with subclaim-aligned evidence (SAE setting). When evidence is repeated at the claim level (SRE), decomposition adds structure without resolving ambiguity and can degrade performance as seen on MMM-Fact and COVID-Fact, where the SRE setting failed to improve over Vanilla. This suggests that the primary real-world bottleneck is often evidence synthesis. Without reliable sub-claim-evidence alignment, decomposition offers limited value and may rather amplify noise.
Sensitivity to Error Profiles. Moving from oracle to predicted sub-claim labels shows upstream noise affects decomposition, but the type of labeling error matters more than overall accuracy. Under noise, SRE collapses, while SAE can remain comparatively stable depending on the predictor's label bias. We observe that the GNN behaves conservatively, frequently predicting Unverified; in a granular-evidence setting this can be beneficial, by avoiding incorrect polar signals. By contrast, Qwen is biased towards refutations and risks more false positives. Thus,stabilizing decomposition pipelines may require optimizing the sub-claim predictor's label bias rather than maximizing macro-F1 alone.
| Model | U | F | RF | PF | Cv | Av|¬u | Av |
|---|---|---|---|---|---|---|---|
| Qwen | 15.8 | 24.2 | .31 | .28 | .86 | .55 | .65 |
| GNN | 35.0 | 5.4 | .04 | .15 | .67 | .42 | .63 |
Future Directions: Human-Centric and Real-Time Verification. Beyond algorithmic accuracy, the ultimate utility of decomposition lies in its interpretability. A key avenue for future work is evaluating whether sub-claim-level rationales genuinely assist human fact-checkers (e.g., journalists) in identifying missing evidence or isolating disputed claims, rather than simply increasing cognitive load. Finally, our current setup uses temporally bounded but static evidence snapshots. In breaking news scenarios, such as those on social media, information evolves rapidly. Future systems must move towards just-in-time verification, utilizing timeline summarization to capture evolving narratives and explicitly representing the status of sub-claims as "unresolved" until sufficient evidence becomes available.
7 Conclusion
We investigate the role of structured sub-claim decomposition in complex claim verification. By leveraging a real-world dataset with human-annotated sub-claim evidence spans, we show that decomposition is not a universal remedy: it requires granular, sub-claim-aligned evidence and reliable veracity labels to be effective. However, our results reveal that when these ideal conditions are met, sub-claim decomposition leads to significant performance gains over vanilla methods that treat claims monolithically.
By contrast, when evidence is limited to the claim level or when sub-claim labels are noisy, the benefits of decomposition diminish or even become detrimental to performance, as shown with our findings on MMM-Fact and Covid-Fact. Our ablation experiments further confirm a critical interaction between evidence granularity and label reliability: providing granular evidence without reliable labels can confuse the verification model. Furthermore, downstream robustness depends heavily on the label bias of the verification model (e.g., the safety of abstaining via an Unverified label versus the risk of committing to incorrect polarity). Ultimately, we conclude that the primary bottleneck for deployment in real-world settings is not decomposition itself, but the capability for precise evidence synthesis and alignment.
Future research should prioritize developing robust subclaim predictors that optimize for label bias, specifically favoring calibrated abstention over incorrect polarity. Additionally, critical advancements are needed in evidence synthesis to produce sub-claim-aligned spans, particularly under temporal constraints. Ultimately, the goal is to design verifiers that can leverage structured decomposition while remaining resilient to noisy labeling inherent in real-world settings.
Acknowledgements
This work was supported by a UKRI/EPSRC Turing AI Fellowship to Maria Liakata (grant ref EP/V030302/1) and the Alan Turing Institute (grant ref EP/N510129/1). This work was also supported by the Engineering and Physical Sciences Research Council [grant number EP/Y009800/1], through funding from Responsible Ai UK (KP0016) as a Keystone project lead by Maria Liakata. F. Ruggeri is partially supported by the project European Commission's NextGeneration EU programme, PNRR - M4C2 - Investimento 1.3, Partenariato Esteso, PE00000013 - "FAIR - Future Artificial Intelligence Research" - Spoke 8 "Pervasive AI" and by the European Union's Justice Programme under Grant Agreement No. 101087342 for the project "Principles Of Law In National and European VAT". We also thank Queen Mary University of London Comp-Research Team for their infrastructure support.
@inproceedings{ijcai2026p608,
title = {The Alignment Bottleneck in Decomposition-Based Claim Verification},
author = {Akhter, Mahmud Elahi and Ruggeri, Federico and Bilal, Iman Munire and Procter, Robert and Liakata, Maria},
booktitle = {Proceedings of the Thirty-Fifth International Joint Conference on
Artificial Intelligence, {IJCAI-26}},
publisher = {International Joint Conferences on Artificial Intelligence Organization},
editor = {Diego Calvanese},
pages = {5458--5467},
year = {2026},
month = {8},
note = {Main Track},
doi = {10.24963/ijcai.2026/608},
url = {https://doi.org/10.24963/ijcai.2026/608},
}
TY - CPAPER TI - The Alignment Bottleneck in Decomposition-Based Claim Verification AU - Akhter, Mahmud Elahi AU - Ruggeri, Federico AU - Bilal, Iman Munire AU - Procter, Robert AU - Liakata, Maria T2 - Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26 PB - International Joint Conferences on Artificial Intelligence Organization PY - 2026 SP - 5458 EP - 5467 DO - 10.24963/ijcai.2026/608 UR - https://doi.org/10.24963/ijcai.2026/608 ER -Copied
Machine-readable versions of this paper, for tools and AI agents: