{"type":"ScholarlyArticle","title":"OrthKD: Extracting Generalized Clinical Knowledge from Heterogeneous Teachers for Lightweight Deployment","authors":[{"name":"Yi Xu","slug":"yi-xu","given":"Yi","family":"Xu","affiliation":""},{"name":"Cheng Chen","slug":"cheng-chen","given":"Cheng","family":"Chen","affiliation":""},{"name":"Mufan Cao","slug":"mufan-cao","given":"Mufan","family":"Cao","affiliation":""}],"abstract":"Deploying diabetic retinopathy (DR) screening models in primary care requires edge-efficient systems that remain accurate, safe, and reliable under domain shift. Multi-teacher knowledge distillation (KD) is a natural compression strategy, but existing approaches largely assume that all teachers provide equally trustworthy supervision. In our setting, this assumption fails: a strong CNN teacher (EfficientNet-B3, 0.876 QWK) and a weaker Transformer teacher (Swin-Base, 0.830 QWK) are complementary, yet the Transformer's logits can still mislead the student. We therefore propose OrthKD, a selective-trust distillation framework that transfers full supervision from the strong CNN, uses feature-only distillation from the weak ViT, and enforces orthogonality between teacher-specific student projections to encourage complementary rather than redundant evidence. This design preserves local lesion precision, injects global structural context, and improves robustness to distribution shift. On 132,049 retinal images, a 5.4M-parameter MobileNetV3 student reaches 0.885 QWK on EyePACS and improves zero-shot Messidor-2 performance from 0.507 to 0.728 QWK, while also achieving strong referral AUC and calibration. These results show that selectively distilling heterogeneous teachers can enable practical DR screening on resource-constrained devices.","doi":"10.24963/ijcai.2026/774","url":"https://ijcai.org/proceedings/2026/774","pdf":"https://ijcai.org/proceedings/2026/0774.pdf","markdown":"https://ijcai.org/proceedings/2026/774.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/774","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"AI and Health","track":"AI and Health","number":774,"pages":{"first":6958,"last":6965},"keywords":["Medical imaging: Medical imaging","Public health: Public health","Telehealth: Telehealth","Medical diagnosis: Medical diagnosis","Clinical decision support system: Clinical decision support system"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}