DIRECT: Decentralized Intention and Latent Rule Emergence for Multi-Agent Cooperation Under Intermittent Communication

1 Introduction

The Centralized Training with Decentralized Execution (CTDE) is a widely adopted paradigm for cooperative multiagent reinforcement learning. Despite its progress, CTDE relies on purely local observations during execution, which introduces uncertainty about the state and teammates' information. This uncertainty frequently results in ineffective coordination and suboptimal joint strategies. Recently, more and more approaches increasingly incorporate explicit inter-agent communication into their algorithms to enhance coordination [Du et al., 2025; Meng and Tan, 2024; Du et al., 2024]. We divide these methods into two categories: i) state-aware coordination, where agents exchange their local observations or embeddings [Hu et al., 2024; Guo et al., 2023], and ii) intention-aware coordination, where agents communicate their long-term objectives [Xu et al., 2023b; Song et al., 2025] or sub-goals [Zhang et al., 2025; Dann et al., 2023] to teammates to enable strategic alignment. In this work, we focus on the latter strategy, as the former is often constrained by high communication costs, limiting its effectiveness in real-world applications with stringent bandwidth and real-time requirements [Roth et al., 2006]. Moreover, excessive communication can introduce irrelevant or even harmful information [Guan et al., 2022], potentially undermining the convergence of the learning process.

∗Corresponding authors

Although intention-aware approaches have achieved strong performance in several cooperative settings, their applicability becomes limited in denied environments. First, existing approaches assume reliable communication, whereas in denied environments characterized by the "fog of war" [Clausewitz, 2003], agents must coordinate under intermittent information [Yao et al., 2025; Liu et al., 2025]. Second, existing approaches generally encourage agents to reach global consensus to improve coordination [Li et al., 2025; Xu et al., 2023a]. However, achieving such agreement among all agents is often difficult. Resolving a local disagreement between two agents may trigger cascading conflicts with the actions of others. To address these issues, we propose a novel Intention ⊗Latent Rule paradigm. Our inspiration is drawn from human cooperative practices in traffic systems, as illustrated in the left panel of Fig. 1, where the agent efficiently resolves the local conflicts by coupling an intention (turn signal) with a shared traffic rule (yielding to through traffic), under which the turning vehicle yields unilaterally.

As shown in the right panel of Fig. 1, our method is driven by two core components. The Intention Forecaster Module maps the agent's local action history into a predictive intention signal, thereby establishing a communication grounding for future behavior [Lo et al., 2024; Lin et al., 2021; Udagawa and Aizawa, 2019]. Moreover, the Latent Rule Inference Module treats the communication link as a timevarying topology. By employing a gating mechanism to selectively process fragmentary signals from currently available neighbors, this module facilitates the emergence of the latent rules. Our main contributions are summarized as follows:

• We propose a beyond-consensus paradigm for decentralized multi-agent coordination, formalized as Intention⊗Latent Rule. This paradigm leverages latent rules to mitigate conflicts among shared intentions, thereby bypassing the need for global consensus.
• We develop DIRECT (Decentralized Intention and Latent Rule-Emergent CooperaTion), a framework that employs an action-grounded Intention Forecaster Module to project the agent's plan into a shared semantic space, ensuring that communicated intentions remain behaviorally grounded. These intentions are processed by an attention-based Latent Rule Inference Module, which jointly considers their temporal freshness and semantic relevance, thereby inferring latent-rule representations that facilitate coordinated decision-making.
• Empirical results on StarCraft II and MPE benchmarks demonstrate that DIRECT significantly outperforms state-of-the-art baselines, exhibiting coordination under varying levels of intermittent communication.

2 RelatedWork

Research on communication-enabled MARL can be broadly organized along several dimensions: when communication is triggered based on its expected utility [Guo et al., 2023], who communicates through structured or graph-based topologies [Du et al., 2025; Hu et al., 2024; Xu et al., 2023b; Xie et al., 2025], how received information is integrated into decision-making [Sun et al., 2024a], and what information is exchanged, including compact semantic representations of local observations or latent features [Meng and Tan, 2024; Du et al., 2024]. In this line of work, intention-based communication has emerged as a prominent paradigm, focusing on sharing representations of anticipated future behavior.

Recent intention-based approaches adopt diverse representations, ranging from explicit forecasts of future trajectories or action sequences [Kim et al., 2021; Fang et al., 2023], to more abstract sub-goals or task-critical intermediate states [Zhang et al., 2025], as well as consensus-driven formulations that infer shared group-level guidance without transmitting explicit plans [Li et al., 2025; Xu et al., 2023a]. These methods generally assume stable communication and promote coordination by driving agents toward global consensus. In contrast, our approach grounds communicated intentions in agents' executed actions and leverages an emergent latent rule to regulate conflicts among intentions, enabling coordination under intermittent communication.

From the perspective of communication constraints, another line of recent work studies multi-agent coordination under imperfect communication conditions, such as limited bandwidth and temporal delays. Bandwidth-oriented methods address transmission budgets and semantic compression [Saleh and Rayna, 2025; Yu et al., 2024; Chen et al., 2025; Sun et al., 2024b], while latency- and asynchronyaware approaches focus on mitigating delayed or misaligned information arrivals [Yuan et al., 2023; Song et al., 2025]. These methods improve robustness to compressed or delayed communication, but generally assume persistent link availability, where information eventually arrives. In contrast, our work considers nonpersistent connectivity, in which communication links may fail entirely and information can be absent for extended periods.

3 Preliminary

A Decentralized Partially Observable Markov Decision Process (Dec-POMDP) is defined by the tuple G = ⟨N, S,O,Ω,A,P,R, γ⟩, where N = {1, . . . , n} denotes the set of agents, and s ∈ S is the global state. At each timestep t, each agent i ∈ N receives a local observation oit ∼ Ω(· | st, i) and selects an action ait ∈ Ai . The resulting joint action at = ⟨a1t , . . . , ant ⟩ ∈ A, where A = ∏i∈N Ai , induces a state transition st+1 ∼ P(· | st, at) and yields a shared reward rt = R(st, at). The objective is to maximize the expected discounted return E [∑∞t=0 γ t rt].

In this paper, we formalize intermittent communication as a time-varying connectivity process, extending the standard formulation to an Intermittent Communication Dec-POMDP (IC-Dec-POMDP). This is defined by the augmented tuple GIC = ⟨N, S,O,Ω,A,P,R, γ, C⟩, where C characterizes the stochastic availability of communication links. Specifically, the connectivity for each directed pair (j → i) at timestep t is modeled as an independent Bernoulli process: C ji t ∼ Bernoulli(1 -ω). Here, C ji t = 1 denotes successful transmission, 0 indicates link failure, and ω ∈ [0, 1] represents the failure probability. Accordingly, the communication configuration C is formalized as C = {Cji t : t ≥ 0; i, j ∈ N}.

4 Method

In this section, we introduce DIRECT, a framework that realizes the Intention ⊗ Latent Rule formulation for decentralized coordination under intermittent communication. The key idea is to mitigate the impact of conflicting intentions via the emergent latent rule, rather than relying on a hand-crafted fixed rule. As illustrated in Fig. 2, DIRECT comprises two core components: an Intention Forecaster Module, which derives a communicated intention representation for each agent from its recent behavior and temporal context, and a Latent Rule Inference Module, which instantiates this shared logic into a context-dependent local representation to regulate how these intentions influence local decision-making.

4.1 Intention Forecaster Module

The Intention Forecaster Module is designed to derive an intention from the agent's executed action. At timestep t, the module conditions on the agent's immediate history by concatenating the recurrent hidden state hit-1 with the executed action ait-1 . The resulting vector xit = [hit-1 , ait-1 ] encodes a compact representation of the agent's recent behavior. This context vector xit is then passed through a non-linear encoder ϕenc to obtain a feature representation eit = ϕenc(xit ), which is used to parameterize the diagonal Gaussian distribution over the intention space via two linear heads ϕµ and ϕσ:

Then, the intention is obtained via the reparameterization trick to enable gradient-based optimization:

To regularize the intention distribution, we minimize the Kullback-Leibler (KL) divergence between the predicted distribution q(mit | xit ) and a standard Gaussian prior p(mit ):

To ensure a shared semantic interpretation of intentions, we construct a unified action embedding space that serves as a grounding interface between intentions and executable actions. Specifically, following the design of RODE [Wang et al., 2021], we pre-train a single action encoder ϕe that maps each action a ∈ A to a d-dimensional embedding za = ϕe(a). The resulting embedding matrix Z ∈ R|A|×d is shared by all agents. The encoder ϕe is trained within a predictive model to capture the dynamical and reward consequences of actions by jointly predicting the next local observation and the global reward, encouraging the embeddings to capture the functional effects of actions on environment dynamics. After pre-training, ϕe is frozen and used as a fixed semantic reference.

With the shared encoder ϕe frozen, we project the predicted intention mit ∈ R|A| into the shared action embedding space, yielding a grounded intention embedding mit ∈ Rd . Specifically, mit is calculated as the expected action embedding under the softmax distribution defined by mit :

where mit (a) denotes the logit score associated with action a, and τ is a temperature parameter.

4.2 Latent Rule Inference Module

The Latent Rule Inference Module operates under the intermittent communication setting defined in Sec. 3. In this setting, each agent i maintains an intention memory Mit = (M i t , T i t ), consisting of a content component M i t and a temporal index component T i t . For each teammate j, M i t [j] stores the most recently received grounded intention from agent j, while T i t [j] records the timestep at which that entry was received. This memory is updated according to the communication protocol C. Whenever information from agent j is successfully delivered to agent i at timestep t, the corresponding memory entry is refreshed; otherwise, the memory entry remains unchanged:

Building on the intention memory Mit , we derive rit as a local instantiation of the latent rule. In our framework, the latent rule is defined as a globally shared coordination logic parameterized by the Latent Rule Inference Module, while rit represents its private, context-dependent realization for agent i at timestep t. By weighting teammate intentions based on their current context and recency, this module allows each agent to adaptively modulate how diverse intentions influence local decision-making under intermittent communication. Specifically, the relevance score is computed as:

where WQ, WK ∈ Rd×d are learnable projections, and β > 0 controls temporal decay. The normalized weights are obtained via:

The resulting latent representation rit for agent i is then represented as a weighted combination of teammate intentions:

where WV ∈ Rd×d maps intentions into the latent rule space. Rather than a fixed rule, rit serves as a coordination prior that biases the agent's subsequent decision-making toward mitigating inter-agent conflicts.

4.3 Agent Policy Network

To incorporate the inferred latent rule representation into local decision-making, the agent conditions its policy on both the representation rit and the observation oit . Specifically, rit and oit are concatenated and passed through an MLP for feature projection, and the resulting representation is then used to update the internal hidden state at each timestep t as:

from which per-action value estimates are produced via:

During decentralized execution, the agent selects its action according to:

while centralized training aggregates individual action-value functions through the QMIX mixer [Rashid et al., 2020].

4.4 Training

The overall training objective augments the standard QMIX loss with an auxiliary regularization term on the latent intention space. Specifically, the global reinforcement learning objective minimizes the temporal-difference (TD) error:

where yt = rt+γ maxat+1 ¯Qtot(st+1, at+1) is the TD target, ¯Qtot is the target network, and D denotes the replay buffer of collected transitions.

The complete training objective is then given by:

where LKL is the auxiliary KL regularization term defined in Sec. 4.1, and λKL controls its relative contribution.

5 Experiment

In this section, we empirically evaluate DIRECT on two diverse benchmarks: the StarCraft II micromanagement (SMAC) benchmark [Samvelyan et al., 2019] and the MultiAgent Particle Environment (MPE) [Mordatch and Abbeel, 2018]. Through these experiments, we aim to answer three core questions regarding the framework's effectiveness: (1) Does DIRECT outperform state-of-the-art communicationand intention-based baselines, particularly in complex cooperative scenarios? (See Sec. 5.1). (2) Can the Intention ⊗ Latent Rule paradigm effectively translate communicated intentions into coordinated behavior? (See Sec. 5.2). (3) Can DIRECT robustly sustain coordinated behavior across different degrees of intermittent communication? (See Sec. 5.3).

Baselines. We compare DIRECT against a set of baseline methods drawn from three representative categories. First, we consider intention- and consensus-oriented methods, including M2I2 [Sun et al., 2024a], which models teammate intentions via masked state prediction, and COLA [Xu et al., 2023a], which learns the implicit consensus from local observations. Second, we further include general communication frameworks that focus on learning structured and efficient information exchange, such as ROCO [Xie et al., 2025], MAIC [Yuan et al., 2022], and NDQ [Wang et al., 2020]. Third, as non-communicative baselines, we adopt QMIX [Rashid et al., 2020] to assess the necessity of explicit information exchange.

5.1 Performance Comparison

To ensure fair robustness, each experiment was repeated with five random seeds, and we report the mean performance with 95% confidence intervals. Besides, to simulate intermittent communication, we set the link failure probability ω to 0.6 across all experiments. For the baselines, M2I2, COLA, ROCO, MAIC, and NDQ were implemented based on the open-source PyMARL framework. Additionally, we adapted QMIX to accommodate the communication setting by concatenating received information with their local observations. This ensures they operate under the same information availability constraints as other baselines.

The top row of Fig. 3 presents the performance on the SMAC benchmark. DIRECT consistently outperforms baselines across all maps, achieving higher final win rates and more stable training behavior. This advantage is most pronounced on the heterogeneous map 6h vs 8z, a scenario demanding precise coordination among agents with heterogeneous unit capabilities. While baselines struggle near a zero win rate-likely due to their inability to resolve conflicting intentions without regulation-DIRECT achieves a 60% win rate. This validates the efficacy of the latent rule in modulating disparate action-grounded signals. On symmetric maps like 5m vs 6m and 8m vs 9m, DIRECT similarly maintains its lead through more stable learning dynamics.

The bottom row of Fig. 3 reports results on continuous control tasks. Across all evaluated environments, DIRECT consistently achieves higher episode rewards than the baseline methods. In contrast, approaches such as COLA and ROCO exhibit pronounced performance oscillations, particularly under intermittent communication, whereas DIRECT maintains smoother reward trajectories throughout training. This consistent behavior indicates greater robustness to inconsistent teammate information and demonstrates that the overall design of DIRECT supports stable coordination in continuous control tasks, enabling agents to learn reliable policies even when communication is intermittent.

5.2 Case Study

To answer the second question, we use two visual case studies to show how the Intention ⊗ Latent Rule paradigm enables effective coordination from communicated intentions. Fig. 4 illustrates how coordination in DIRECT emerges through a latent rule in the SMAC 6h vs 8z scenario. At timestep t, allied Hydralisk 5 and Hydralisk 6 communicate retreat intentions, while Hydralisk 4 is surrounded by enemy Zealots. At timestep t + 1, Hydralisk 5 and Hydralisk 6 adjust their actions from retreat to attack, converging on a compatible response that alleviates the local disadvantage. This coordinated adjustment indicates that an emergent latent rule mitigates conflicting intentions by incorporating both communicated information and dynamic environmental state. The attention heatmap further shows that, during action selection, agents place higher emphasis on teammates whose communicated intentions are potentially incompatible with their own, indicating that the latent rule guides agents to adjust their actions when faced with incompatible teammate intentions, producing coherent joint actions.

Fig. 5 offers several interesting insights on how latent rules mediate the translation from communicated intentions to coordinated actions under intermittent communication. First, temporary mismatches between intentions and actions naturally arise from sudden changes in the environment, and the latent rule helps agents adjust their actions accordingly while preserving coordinated team behavior. At t = 32, allied Hydralisk 6 communicates an intention to pursue enemy Zealot 9, while Hydralisk 5 initially targets Zealot 12. As Zealot 12 moves out of favorable engagement range, Hydralisk 5 reallocates its target to Zealot 9 at t = 33, reflecting a coordinated adjustment driven by both teammate intention and updated local state. Second, when agents infer mutually compatible intentions, the latent rule reinforces this alignment and induces coordinated execution. At t = 45, allied Hydralisk 1 and Hydralisk 5 communicate aligned intentions toward enemy Zealot 13, which are directly realized at t = 46 as a coordinated joint attack. Third, over the latter half of the selected episode, communicated intentions exhibit a clear alignment with executed actions over time. As shown in the bottom panel of Fig. 5, this alignment is reflected by the dominant color correspondence between intention and action sequences, indicating that intentions serve as high-level behavioral abstractions that guide coordination without constraining execution. Together, these insights show that the latent rule both stabilizes coordination under local perturbations and accelerates alignment when shared objectives emerge.

5.3 Ablation and Sensitivity Analysis

We evaluate the impact of key components in Fig. 6. Removing the pre-trained action encoder (DIRECT a) leads to a significant performance collapse on the complex 6h vs 8z map, indicating that grounding communicated information in actions is essential for effective coordination. Without this grounding, agents struggle to interpret the intention behind information received from teammates. In contrast, removing the Latent Rule Inference Module (DIRECT b) results also degrades performance on complex tasks, showing that actiongrounded intentions alone are not sufficient. Without a mechanism to account for outdated information, coordination becomes unstable. By combining both components, DIRECT achieves consistently strong performance, demonstrating that action grounding and latent rule regulation play complementary roles in supporting robust coordination.

As shown in Fig. 7, we evaluate the robustness of DIRECT under varying link failure probabilities ω. As ω decreases, agents receive intention more frequently, which generally corresponds to better coordination performance. Interestingly, highly intermittent communication does not consistently outperform the no-communication setting with ω = 100%. One possible explanation is that sporadic intention updates can lead to mismatches between communicated intentions and the actions agents actually execute, causing the exchanged information to act as noise. In contrast, when communication is entirely absent, agents rely on a consistent local information stream, which can result in more stable learning behavior. Importantly, DIRECT does not exhibit noticeable performance degradation under such noisy communication conditions. Its performance remains comparable to the stable baseline, suggesting that the framework can effectively tolerate intention-action mismatches without compromising learning stability.

6 Conclusion

In this paper, we proposed DIRECT, a decentralized communication framework for cooperative multi-agent reinforcement learning under intermittent communication. Bypassing the need for global agreement, DIRECT follows the Intention ⊗ Latent Rule paradigm, in which agents share behavior-grounded intentions, and an emergent latent rule is used to mitigate conflicts among these communicated intentions when making decisions. By grounding intentions in executed actions and adapting to partial and temporally fragmented information, DIRECT ensures robust cooperation even in intermittent communication environments. Extensive empirical analyses demonstrate that DIRECT achieves robust coordination under intermittent communication. Rather than enforcing global consensus, the results show that coordination emerges from how the latent rule mitigates conflicts among communicated intentions during decision-making. In particular, when communicated intentions temporarily diverge from subsequent actions due to dynamic state changes, coordinated behavior is maintained despite these discrepancies. A promising direction for future work is to integrate Vision-Language-Action (VLA) models as a shared grounding interface for intentions. By aligning DIRECT's intention embeddings with the pre-trained action representations of VLA, the Intention ⊗ Latent Rule paradigm could be extended to open-ended embodied settings, enabling zero-shot coordination with previously unseen partners.

Acknowledgments

This work is supported by National Natural Science Foundation of China (Grant No. 62276238, U24A20326, 62325602, 62036010), the Foundation of Henan Educational Committee (Grant No. 25HASTIT034), the Natural Science Foundation of Henan (Grant No. 232300421095).