Mobile Network Operator Collaboration using Deep Reinforcement Learning

Abstract

Fifth Generation Mobile Networks (5G) are designed to offer connectivity services for new types of applications that not only address the needs of private individuals, but also those of enterprise customers. Network requirements of such applications often transcend the capabilities of a single Mobile Network Operator (MNO). In such cases, the ability of MNOs to collaborate in order to provide connectivity services is essential. Existing methods for establishing collaborations between MNOs are reactive in nature and are set up using a Business Support System (BSS) that involves human decision-making. In contrast, this dissertation presents a proactive approach, which, in addition to automating collaboration establishment, uses machine learning to predict future requirements for connectivity services, and dynamically optimizes resource allocation through selection of collaborating MNOs.

The analysis begins by investigating the possibility for mobile networks to support the requirements of mission-critical enterprise applications. With assistance from an automotive industry partner, a remote vehicle driving application (teleoperation) was selected and the network's requirements were quantified in terms of throughput and latency. Existing Quality of Service (QoS) mechanisms in an already-deployed mobile network were used to set up the corresponding policies, and then the performance of the connectivity service was evaluated. The evaluation is conducted within the coverage area of a single radio base station and considered a single vehicle communicating with a remote driving station. The results show that it is possible even for the current generation of mobile networks to support this type of mission-critical application.

Following this quantitative assessment, wider deployments of the teleoperation application were considered, when the application's network requirements could not always be served by a single MNO. This dissertation uses deep Reinforcement Learning (RL), to build models that predict future network requirements and proactively suggest an MNO collaboration that fulfills these requirements. Experiments are conducted using two different approaches. First, a single-agent approach, wherein decisions for collaborations are provided by the model of an MNO-independent agent. In this case, all MNO share the same fairness-based policy of willingness to collaborate as long as they equally split connectivity service provisioning with their counterparts. Second, a multi-agent approach, wherein every MNO has its own agent following its own policy enforcing the terms to collaborate with other MNO. Both approaches are compared against the current state of predetermined collaboration structures or reactive collaboration approaches based on current - and unpredicted - connectivity service(s). 

The proposed approach is shown to provide up to a two-fold improvement in connectivity service requirement coverage over the state of art. Another benefit is resource optimization, as MNOs' capabilities  are better matched to connectivity service requirements. This resource optimization can contribute to a more sustainable growth of mobile network expansion. Finally, democratization of collaborations allows for new revenue streams for smaller MNO which do not have the capacity or resources to establish collaborations in the traditional way. Although a teleoperation application and MNO mobile networks are used for the evaluation, the method and findings are expected to be applicable to a broader range of applications across different types of wireless networks.

urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-289651