Trustworthy AI for Post-Stroke Care

1 On the Research Problem

As AI systems are increasingly deployed in high-stakes settings, their decisions can have significant consequences for individuals and society. Ensuring the trustworthiness of Artificial Intelligence (AI) is therefore a central concern [European Comission, 2019]. Two key dimensions of trustworthy AI are cautiousness, awareness of model limitations that supports risk-aware decision-making [Dubois and Prade, 2009], and explainability, the ability to provide human-understandable justifications for model decisions [Barredo Arrieta et al., 2020]. Both dimensions are especially critical in safety-sensitive applications. My PhD addresses this challenge along two complementary axes: theory and application.

On the theoretical side, I develop methods that enable cautiousness and explainability in models, both through mathematical formalisation and by implementing new algorithms. Cautiousness relies on uncertainty quantification, which can be modelled in several frameworks, including imprecise probability theory [Walley, 1991], possibility theory [Dubois and Prade, 1988], rough sets [Pawlak, 1982], fuzzy sets [Zadeh, 1965], and Dempster-Shafer evidence theory [Shafer, 1976]. Explainability enables users to understand, critique, and improve models. A common taxonomy distinguishes intrinsic methods-models designed to be interpretable-from post-hoc methods-explanations produced for already-trained black boxes [Carvalho et al., 2019]. Post-hoc techniques mainly fall into two families [Barredo Arrieta et al., 2020]: feature-relevance methods, which rank or quantify the influence of input features [Baehrens et al., 2010], and simplification methods, which approximate black-box classifiers with interpretable surrogates such as decision trees, rule lists, or linear models [Guidotti et al., 2018].

On the application side, the focus of my work is on poststroke rehabilitation, where trustworthy AI can directly support clinical decisions. My research is conducted within the ReArm project, a quadruple-blind, randomised, shamcontrolled, bi-centre, two-arm interventional study involving fifty-eight chronic stroke patients, designed to assess a treatment that combines virtual reality therapy with anodal transcranial direct current stimulation. The collected data include clinical assessments, EEG and fNIRS recordings, and continuous upper-limb monitoring in daily life using wearable accelerometers [Muller et al., 2021]. The goal of the application part of my PhD is to design trustworthy AI systems that exploit this multimodal information to produce personalised insights and support individualised rehabilitation planning.

2 Contributions

I summarize the main contributions of my PhD research:

• Explaining evidential clusterings with IEMM (best poster award at BFTA 2025; paper under review at ECML-PKDD 2026, preprint available [de Souza et al., 2025]). We formalise representativeness, a utilitybased criterion that encodes decision-makers' preferences over explanation misassignments, and evidential Mistakeness, a loss tailored to credal partitions. Building on these notions, the Iterative Evidential Mistakeness Minimization (IEMM) algorithm learns decisiontree surrogates for evidential clustering; we also provide conditions for effective explanations in both hard and evidential settings and validate the approach on synthetic and real-world datasets.
• Personalized Activity Profiling in Post-Stroke Care via Explainable Evidential Clustering (accepted at FUZZ-IEEE WCCI 2026). We design an explainable clustering pipeline for wrist-accelerometer data: fiveminute windows are transformed into clinically mean-

ingful arm-use features and clustered using either (i) Evidential C-Means with an IEMM decision-tree surrogate or (ii) K-means with a Belief Rule-Based Classification System. We then combine clustering results with patient-reported activity logs using evidence theory to derive activity-level profiles with associated uncertainty, producing clinically interpretable insights on real-world post-stroke data. We compare the two pipelines and discuss their respective advantages and limitations.

• Eliciting utilities under imprecision (under review at UAI 2026). We propose practical elicitation procedures that quantify how much imprecision a user is willing to accept when receiving set-valued predictions, i.e., how much precision to trade for robustness and accuracy. Our proposed iterative algorithm converges to the target parameter on both tabular and image-based benchmarks with a limited number of queries.
• Counterfactuals and adversarial examples for setvalued classifiers (accepted at IPMU 2026). We generalize counterfactual explanations through foiled predicates that specify conditions the current set-valued prediction fails to satisfy. We also define adversarial examples for set-valued outputs, distinguishing possible adversarial examples that induce uncertainty about the true class from certain adversarial examples that guarantee misclassification, and we illustrate the framework on different datasets.

3 Impact and Next Steps

Trustworthy AI ultimately aims to support human-machine cooperation: models should not merely output decisions but provide the elements required for users to critique them. In this view, mistakes are not only risks to mitigate, but can also trigger a deeper understanding of the data, modelling assumptions, and domain knowledge. Designing systems that can be interpreted, meaningfully challenged, and ultimately trusted is therefore a prerequisite for responsible deployment in safety-sensitive settings.

In post-stroke rehabilitation, this requirement is particularly important. Stroke remains a major cause of longterm disability and reduced autonomy in daily activities, and decision-support tools can only be useful if they are clinically sound and aligned with medical practice. For this reason, continuous dialogue with clinical experts is central in my work, both to validate the relevance of the extracted patterns and to ensure that the proposed explanations and uncertainty representations match practitioners' needs.

Next, I will strengthen the methodological foundations and broaden the scope of the application. This includes advancing the elicitation of cautious utilities, extending cautious and explainable approaches to multimodal rehabilitation data, and developing cautious explainable regression methods to identify factors that are associated with better post-stroke recovery outcomes.