Doctoral Thesis

Thesis Title: Trustworthy Multi-Agent Learning: From Decentralized Anomaly Detection to Resilient Policy Learning

Main Supervisor: Prof. György Dán

Abstract:
Multi-agent systems provide a framework for solving complex tasks through the interaction and coordination of autonomous agents. They have become increasingly important in applications such as autonomous driving, robotics, and smart grids, where reliable cooperation is essential. However, the distributed nature of these systems also makes them vulnerable to adversarial behavior, including compromised agents, manipulated observations, and corrupted communication or recommendation signals. Such disruptions can undermine coordination and lead to significant performance degradation. Ensuring the reliability of multi-agent systems therefore requires both the ability to detect abnormal behavior and the ability to maintain effective cooperation in the presence of adversaries.

This thesis investigates two complementary approaches to improving the security and resilience of multi-agent systems: decentralized anomaly detection and robust policy learning. The first part focuses on identifying compromised agents and detecting adversarial manipulations using locally available information. In multi-agent reinforcement learning, we develope decentralized detection methods in which agents use recurrent neural networks to predict the expected action distributions of their observable neighbors based on their observation histories. The framework accommodates both discrete and continuous action spaces, modeling expected behavior using categorical and parameterized multivariate Gaussian distributions, respectively. Deviations from these predictions are quantified through normality scores and monitored using sequential detection methods, including the cumulative sum (CUSUM) procedure, enabling the detection of persistent behavioral changes. In parallel, we formulate dynamic attacker problems to examine how rational adversaries balance team degradation against detection risk. Broadening this detection paradigm and from a more theoretical perspective, the investigation is extended to strategic-form games, where quickest change detection is used to identify adversarial manipulations of correlated-equilibrium recommendations by a compromised mediator.

The second part addresses proactive defense through robust policy learning and test-time adaptation. To move beyond conventional approaches based on worst-case adversarial behavior, we introduce a Bayesian decentralized partially observable Markov decision process (Dec-POMDP) game model. By grouping unknown adversarial behaviors into disjoint adversarial types, the model supports the training of a single cooperative policy that can adapt to a diverse range of non-cooperative and malicious agents. Moreover, addressing the steep sample complexity inherent to model-free MARL, we develop MATRIX, a diffusion-based multi-agent world model that enables sample-efficient policy training in imagination. Because existing world models rely heavily on global state information unavailable during decentralized execution, MATRIX introduces a structured conditioning scheme inspired by classifier-free guidance that selectively drops global context. This design allows a single model to support sample-efficient centralized imagination during training while providing locally conditioned observation predictions at runtime. Operating with low computational overhead, individual agents evaluate incoming observations using a single-sample denoising loss and replace detected adversarial perturbations with locally conditioned diffusion rollouts, preserving coordination under test-time attacks.

Taken together, the contributions of this thesis advance a complementary approach to securing multi-agent systems by combining decentralized detection of adversarial behavior with robust and adaptive policy learning. The results provide methodological foundations for improving the reliability of cooperative decision-making in distributed environments where agents may face uncertainty, compromised information, or malicious interference.


Papers Included in the Thesis: 

1- K. Kazari, E. Shereen, and G. Dán, "Decentralized Anomaly Detection in Cooperative Multi-Agent Reinforcement Learning", in Proceedings of the 32nd International Joint Conference on Artificial Intelligence (IJCAI), 2023

2- K. Kazari, E. Shereen, and G. Dán, "Distributed Detection of Adversarial Attacks in Multi-Agent Reinforcement Learning with Continuous Action Space", in Proceedings of the 28th European Conference on Artificial Intelligence (ECAI), 2025

3- K. Kazari, A. Kanellopoulos, and G. Dán, "Quickest Detection of Adversarial Attacks Against Correlated Equilibria", in Proceedings of the 39th AAAI Conference on Artificial Intelligence (AAAI), 2025

4- K. Kazari and G. Dán, "Bayesian Robust Cooperative Multi-Agent Reinforcement Learning Against Unknown Adversaries", in Proceedings of the International Conference on Learning Representations (ICLR), 2026

5- K. Kazari, S. Skanda, and G. Dán, "Adversarially Robust Multi-Agent RL via Test-Time Imagination", under review

Papers Not Included in the Thesis: 
1- E.Shereen, K.Kazari, G.Dán, "A reinforcement learning approach to undetectable attacks against automatic generation control," IEEE Transactions on Smart Grid, 2023

2- E.Shereen, K.Kazari, G.Dán, "Adversarial Robustness of Multi-agent Reinforcement Learning Secondary Control of Islanded Inverter-based AC Microgrids",  in Proceedings of IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), 2023

3- A.Javeed, K.Kazari, G.Dán, "CooL-BO: Constraint-aligned Latent Bayesian Optimization for High-dimensional Problems", under review