{"type":"ScholarlyArticle","title":"Hierarchical Conditional Energy Modeling for Medical Vision–Language Pretraining","authors":[{"name":"Chengsheng Mao","slug":"chengsheng-mao","given":"Chengsheng","family":"Mao","affiliation":""},{"name":"Yuan Luo","slug":"yuan-luo","given":"Yuan","family":"Luo","affiliation":""}],"abstract":"Contrastive vision–language pretraining models such as CLIP align images and text in a shared embedding space but do not explicitly model or evaluate the hierarchical semantics common in medical image interpretation. We propose HCE-CLIP (Hierarchical Conditional Energy CLIP), a vision–language pretraining framework that formulates medical image–text alignment as a hierarchical label-conditional energy modeling problem. HCE-CLIP encodes an image series using transformer-based aggregation and aligns it with free-text reports and structured label state prompts across multiple semantic levels. At each level, conditional energy functions favor clinically consistent label states while suppressing contradictory alternatives, enabling uncertainty-aware inference. To assess semantic coherence, we introduce a hierarchical contradiction-based metric that quantifies logical inconsistencies between fine-grained disease predictions and higher-level clinical summaries. Experiments on MIMIC-CXR and other public benchmarks show that HCE-CLIP outperforms existing medical vision–language pretraining methods in seen-label, zero-shot and linear-probe settings, while producing substantially fewer hierarchical contradictions.","doi":"10.24963/ijcai.2026/759","url":"https://ijcai.org/proceedings/2026/759","pdf":"https://ijcai.org/proceedings/2026/0759.pdf","markdown":"https://ijcai.org/proceedings/2026/759.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/759","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"AI and Health","track":"AI and Health","number":759,"pages":{"first":6823,"last":6831},"keywords":["Multimodal data: Multimodal data","Biomedical NLP: Biomedical NLP","LLM in medicine: LLM in medicine","Medical imaging: Medical imaging"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}