{"type":"ScholarlyArticle","title":"HFFN-ID: A Hierarchical Feature Fusion Network with Bi-Phase Subject ID Modulation for EEG Mel-Spectrogram Reconstruction","authors":[{"name":"Chi Huang","slug":"chi-huang","given":"Chi","family":"Huang","affiliation":""},{"name":"Zhaohu Liu","slug":"zhaohu-liu","given":"Zhaohu","family":"Liu","affiliation":""},{"name":"Yong Peng","slug":"yong-peng","given":"Yong","family":"Peng","affiliation":""},{"name":"Wanzeng Kong","slug":"wanzeng-kong","given":"Wanzeng","family":"Kong","affiliation":""}],"abstract":"High-fidelity reconstruction of mel-spectrograms from EEG signals remains a formidable challenge, primarily due to the inherent inter-subject variability of neural patterns and the semantic gap between heterogeneous feature representations of these two modalities. To alleviate both issues, this paper proposes a Hierarchical Feature Fusion Network with bi-phase subject IDentifier modulation (HFFN-ID) for reconstructing mel-spectrograms from EEG signals. The primary improvements of the HFFN-ID framework are from two aspects, a Binary-Phase Subject-ID Modulation (BiPSM) mechanism for explicit subject conditioning and a Condition-guided Hierarchical Fusion (CHF) component for dynamic multi-layer feature synthesis, which respectively aim to address the problems of inter-subject variability in neural patterns and heterogeneous cross-modality feature fusion. Moreover, a speech envelope feature-based pre-training strategy is incorporated to initialize the parameter space, inspired by the shared low-level representations across different speech features. On the SparrKULee dataset, HFFN-ID establishes a new benchmark with a pearson correlation coefficient of 0.0723, representing a substantial 44% relative improvement over existing baseline. Importantly, HFFN-ID is highly efficient, achieving a 38.7% parameter reduction and a 2.5× training speedup. These results highlight the effectiveness of subject-conditioned and hierarchically fused architectures for advancing high-fidelity neural speech decoding.","doi":"10.24963/ijcai.2026/402","url":"https://ijcai.org/proceedings/2026/402","pdf":"https://ijcai.org/proceedings/2026/0402.pdf","markdown":"https://ijcai.org/proceedings/2026/402.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/402","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Humans and AI","track":"Main Track","number":402,"pages":{"first":3612,"last":3620},"keywords":["Humans and AI: Human-computer interaction","Humans and AI: Cognitive modeling","Humans and AI: Intelligent user interfaces"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}