Abstract: Multi-agent reinforcement learning (MARL) is an efficient cooperative training approach for adaptive traffic signal control (ATSC). With multiple agents seen as cooperated traffic intersections, and can only observe limited information in the real environment, posing a challenge for agent policy space exploration.In addition, they share the environment reward, which make it difficult to accurately measure contribution for individual agents. To tackle these problems, we propose the Graph Decomposition Action Reference (GDAR) framework based on Centralized Training Decentralized Execution (CTDE). Specifically, the multi-agent system is modeled as a graph structure, in which agents are regarded as nodes and relations as edges. To solve the problem of limited observation, Graph Neural Networks (GNN) is used to expand the receiving domain of the agents.   Meanwhile, we extract node representation to evaluate the individual contribution of each agent. In addition, we design Action Reference networks to improve the diversity of individual action choosing. We model the traffic conditions near the Nanjing Yangtze River Bridge in the Simulation of Urban MObility (SUMO) simulator. Experimental results show that GDAR adapts to ATSC tasks and is superior to advanced comparison algorithms.

With the rapid development of mobile communication technology and intelligent applications, the quantity of mobile devices and data traffic in networks have been growing exponentially, which poses a great burden to networks and brings huge challenge to servicing user demand. Edge caching, which utilizes the storage and computation resources of the edge to bring resources closer to end users, is a promising way to relieve network burden and enhance user experience. In this paper, we aim to survey the edge caching techniques from a comprehensive and systematic perspective. We first present an overview of edge caching, summarizing the three key issues regarding edge caching, i.e., where, what, and how to cache, and then introducing several significant caching metrics. We then carry out a detailed and in-depth elaboration on these three issues, which correspond to caching locations, caching objects, and caching strategies, respectively. In particular, we innovate on the issue “what to cache”, interpreting it as the classification of the “caching objects”, which can be further classified into content cache, data cache, and service cache. Finally, we discuss several open issues and challenges of edge caching to inspire future investigations in this research area.