AMR-LLM: Knowledge-Enhanced Multi-Modal Automatic Modulation Recognition via Large Language Models

Abstract

Existing multi-modal automatic modulation recognition (AMR) methods primarily focus on exploiting multi-view representations of raw signal data to improve performance, but still struggle to effectively model and exploit high-level human prior knowledge. Although recent studies attempt to introduce large language models (LLMs) to integrate the textual modality, most of them merely treat LLMs as feature extractors, without further exploiting the LLM's potential. To this end, we propose a knowledge-enhanced multi-modal automatic modulation recognition framework based on large language model (AMR-LLM). Specifically, we first specially design an AMR instruction construction mechanism based on knowledge to activate the LLM's potential for signal perception, which designates the LLM as a signal domain expert, introduces human prior knowledge as a supplement, and exploits the unique physical information of each data sample as connection guidance. Then, to enable the LLM to directly perceive digital signals, we introduce a progressive multi-modal fusion strategy mapping signal features into the space of LLMs. Experimental results on multiple benchmark datasets demonstrate that AMR-LLM achieves approximately 5% higher accuracy than current state-of-the-art methods. Moreover, it achieves the current domain performance with only 30% of the data.