关键词： Mobile edge computing   Service migration   Hybrid learning   Deep reinforcement learning   Multi-user decisions  

摘要： Mobile edge computing, a new computing model, aims to reduce network latency and improve user experience by placing computing resources closer to the users. However, when users move away from the current service areas, the quality of services will decline. To guarantee the quality of services, service migration technique is proposed and studied. Due to the wireless resource contention among users, there is service migration decisionmaking coupling among multiple users. Meanwhile, making real-time service migration decisions for multiple users to minimize service cost with application deadlines remains an open challenge because of the huge state space. In this paper, we formalize the service migration problem with deadlines as a Markov decision process (MDP) and propose a hybrid learning based service migration strategy. The proposed approach integrates Monte Carlo method and DQN learning, which adopts online training without Q-values and neural network approximation to generate service migration decisions for multiple users. Experimental results demonstrate that the proposed hybrid learning based service migration strategy outperforms the existing migration strategies in terms of service success ratio and service cost.

关键词： Computer architecture   Task analysis   Approximation algorithms   Mobile handsets   Servers   Mathematical models   Delays   Approximation algorithm   cloud radio access network   mobile edge computing   task offloading   user statisfaction  

摘要： With the continuous development of the communication service industry, users pay more attention to the quality of network service. Previous studies on offloading problems, especially in the Cloud Radio Access Network (C-RAN) architecture with Mobile Edge Computing (MEC), are primarily focused on the economic perspective, with little consideration given to user-oriented satisfaction problems. To fill this gap, this article proposes a mathematical model for maximizing user satisfaction in the C-RAN architecture with multi-layer MEC. The problem is divided into two stages for solution. The first stage addresses the optimal connection problem between users and Remote Radio Heads (RRHs). The second stage then schedules user tasks reasonably based on the solution obtained in the first stage. The two-stage problems are all proved to be NP-Hard. Two efficient approximation algorithms, namely User-to-RRH Association Algorithm (URAA) and Maximum Satisfaction Algorithm (MSA), are proposed to solve the problems in different stages. This article proves and analyzes the theoretical performance of the two algorithms. Finally, the performance of the proposed algorithms is verified by simulation experiments. The experimental results demonstrate that the two proposed algorithms can achieve reasonable solutions to the problems, and the user satisfaction level can be maintained at a high level.

关键词： mobile edge computing   wireless body area network   task offloading   reinforcement learning   D3QN   blockchain   quality of experience  

摘要： In recent years, mobile edge computing has become one of the popular methods to provide computing resources for the body area network, but existing research only considers the problem of minimizing the cost of offloading when solving the optimization problem of task-offloading, ignoring the trust problem of edge computing nodes, and offloading tasks on edge nodes may cause user information disclosure and reduce the quality of user experience. In response to this situation, this study aims to minimize the average user cost and designs a task-offloading strategy based on the D3QN (dueling double deep Q-network) algorithm in conjunction with the blockchain information security storage model. This strategy uses deep reinforcement learning algorithms to obtain the minimum average offloading cost of the system while considering user latency, energy consumption, and data protection conditions. The experimental simulation results show that compared to traditional schemes and other reinforcement learning-based schemes, this scheme can more effectively reduce the average cost of the system, and the average cost is reduced by 31.25% when reaching convergence. In addition, as the complexity of the model increases, this scheme can provide users with better experience quality, with 53.7% of the 1000 users having a very good experience quality.

关键词： Task analysis   Servers   Delays   Multi-access edge computing   Uncertainty   Mobile handsets   Information sharing   Bandit feedback   mobile edge computing   multi-user offloading   uncertain system-side information  

摘要： Mobile edge computing facilitates users to offload computation tasks to edge servers for meeting their stringent delay requirements. Previous works mainly explore task offloading when system-side information is given (e.g., server processing speed, cellular data rate), or centralized offloading under system uncertainty. But both generally fall short of handling task placement involving many coexisting users in an uncertain environment. In this paper, we develop a multi-user offloading framework considering unknown yet stochastic system-side information to enable a decentralized user-initiated service placement under overlapping server coverage. Specifically, we formulate the dynamic task placement as an online multi-user multi-armed bandit process, and propose a decentralized epoch based offloading (DEBO) to optimize user rewards which are subjected under network delay. We consider both cases without and with neighboring edge feedback once users' tasks are processed, where the latter incorporates system-side information sharing among edge servers for an enhanced task placement. For both cases, we show that DEBO can gradually deduce the optimal user-server assignment during dynamic offloading, thereby achieving a close-to-optimal service performance and tight O(log(2)T) regret. Moreover, we generalize DEBO to various common scenarios such as unknown reward gap, dynamic entering or leaving of clients, and fair reward distribution, while further exploring when users' offloaded tasks require heterogeneous computing resources. Particularly, we accomplish a sub-linear regret for each of these instances. Real measurements based evaluations corroborate the superiority of our offloading schemes over state-of-the-art approaches in optimizing delay-sensitive rewards.

关键词： Mobile edge computing   augmented reality   reward maximization   online learning   multi-armed bandits  

摘要： In the coming era of Metaverse, Augmented Reality (AR) has become a key enabler of diverse applications including healthcare, education, smart cities, and entertainments. To provide users with interactive and immersive experience, most AR applications require extremely high responsiveness and ultra-low processing latency. Mobile edge computing (MEC) has demonstrated great potentials in meeting such stringent latency requirements and resource demands of AR applications, by implementing AR requests in edge servers within the proximity of users. In this paper, we investigate the reward maximization problem for AR applications with uncertain resource demands in an MEC network, such that the accumulative reward of services provided for AR applications is maximized, while ensuring that the responsiveness of AR applications is enhanced, subject to network resource capacity. To this end, we formulate an exact solution when the problem size is small, otherwise we devise an efficient approximation algorithm with a provable approximation ratio for the problem. We also develop an online learning algorithm with a bounded regret for the dynamic reward maximization problem without the knowledge of future arrivals of AR requests, by adopting the Multi-Armed Bandits (MAB) technique. Considering maximizing the reward may defer the implementations of some urgent yet low-award requests, we propose a fairness-aware online learning algorithm for the dynamic reward maximization problem, through a data rate prediction mechanism that adopts a multi-task and multi-timescale Long Short-Term Memory (MT2 -LSTM) method. Finally, we evaluate the performance of the proposed algorithms for AR applications by building a real test bed. Experimental results show that the proposed algorithms outperform existing studies by improving the award by 13%.

关键词： Mobile edge computing   Unmanned aerial vehicle   Service caching   Workload offloading   Deep reinforcement learning  

摘要： Internet of Things devices generate a large number of heterogeneous workloads in real-time that require specific application to tackle, and the inability to communicate between devices and communication base stations due to complex scenarios is a thorny issue. Service caching play a key role in managing specific-request workload from devices, and unmanned aerial vehicles with computation and communication functions can effectively solve communication barrier between devices and ground base stations. In addition, the joint optimization of workload offloading and service cache placement is a key issue. Accordingly, we design an unmanned aerial vehicle-enabled mobile edge computing system with multiple devices, unmanned aerial vehicles and edge servers. The proposed framework takes into account the randomness of workload arrival, the time-varying nature of channel states, the limitations of the hosting service caching, and wireless communication blocking. Furthermore, we designed workload offloading and service caching hosting decision-making optimization problems to minimize the long-term weighted average latency and energy consumption costs. To tackle this joint optimization problem, we propose a request-specific workload offloading and service caching decision-making scheme based on the medley deep reinforcement learning scheme. To this end, the proposed scheme is decomposed into two-stage optimization subproblems: the workload offloading decision-making problem and the service caching hosting selection problem. In terms of the first subproblem, we model each device as a learning agent and propose the workloads offloading decision-making scheme based on multi-agent deep deterministic policy gradient. For the second subproblem, we present the decentralized double deep Q-learning scheme to tackle the service caching hosting policy. According to the comprehensive experimental results, the proposed scheme is able to converge rapidly on various parameter configurations a

关键词： Internet of vehicles   Mobile edge computing   Offloading strategy   Resource allocation   Deep reinforcement learning  

摘要： To meet the stringent requirements for real-time performance of computing tasks on the Internet of Vehicles (IoV) scenario, the Mobile Edge Computing (MEC) technique is introduced to combine edge servers with vehicles that have storage and computing resources, thereby reducing latency. However, successfully accessing the channel and completing the offloading computation within the specified time remains a big challenge in scenarios with multitasking vehicles and multibase stations equipped with multiple channels. To address this problem, we propose the Multi-Agent Deep Deterministic Policy Gradient-based Offloading and Resource Allocation (MADDPG-RA) algorithm. First, a sub-optimal offloading strategy is determined using the MADDPG algorithm to minimize the sum of system latency and energy consumption. Specifically, this strategy determines whether each vehicle should perform local or offload computation, which MEC server to choose, and which channel to access if offloaded. Based on the above, a closed form expression for the optimal computational resource allocation for MEC is derived using Lagrange multipliers. The simulation results demonstrate that the proposed MADDPG-RA algorithm can effectively reduce the total system latency and energy consumption compared to the existing algorithms.

关键词： multiple UAVs   mobile edge computing   deep reinforcement learning   power inspection   offloading strategy   electric power IoT  

摘要： With the ongoing advancement of electric power Internet of Things (IoT), traditional power inspection methods face challenges such as low efficiency and high risk. Unmanned aerial vehicles (UAVs) have emerged as a more efficient solution for inspecting power facilities due to their high maneuverability, excellent line-of-sight communication capabilities, and strong adaptability. However, UAVs typically grapple with limited computational power and energy resources, which constrain their effectiveness in handling computationally intensive and latency-sensitive inspection tasks. In response to this issue, we propose a UAV task offloading strategy based on deep reinforcement learning (DRL), which is designed for power inspection scenarios consisting of mobile edge computing (MEC) servers and multiple UAVs. Firstly, we propose an innovative UAV-Edge server collaborative computing architecture to fully exploit the mobility of UAVs and the high-performance computing capabilities of MEC servers. Secondly, we established a computational model concerning energy consumption and task processing latency in the UAV power inspection system, enhancing our understanding of the trade-offs involved in UAV offloading strategies. Finally, we formalize the task offloading problem as a multi-objective optimization issue and simultaneously model it as a Markov Decision Process (MDP). Subsequently, we proposed a task offloading algorithm based on a Deep Deterministic Policy Gradient (OTDDPG) to obtain the optimal task offloading strategy for UAVs. The simulation results demonstrated that this approach outperforms baseline methods with significant improvements in task processing latency and energy consumption.

关键词： Mobile edge computing   Task offloading   Limited computational resources   Energy   Arithmetic optimization algorithm  

摘要： The emergence of Mobile Edge Computing (MEC) not only provides low-latency computing services for the User Equipment (UE), but also extends the battery life of the UE. However, the computational resources of MEC servers are usually limited, and how to efficiently offload UE's task and allocate the resources of MEC servers has become a research hotspot in MEC. In this paper, we develop an improved arithmetic optimization algorithm (IAOA) to optimize the convergence speed and convergence accuracy of the arithmetic optimization algorithm. Then a task offloading algorithm based on IAOA is designed to reduce the cost of offloading tasks in the framework including a single MEC server and multi-UE. The proposed algorithm jointly optimizes the task offloading strategy of the UEs and the resource allocation of the MEC server, meanwhile, models the weighted sum of delay and energy consumption as the system cost, with the goal of minimizing the system cost while satisfying the delay and energy consumption constraints of the tasks. Simulation results show that the proposed algorithm can effectively reduce the system cost and achieve a performance improvement of up to 20% compared with the benchmark algorithm.

关键词： Task analysis   Servers   Costs   Heuristic algorithms   Training   Scheduling algorithms   Deep learning   Deep reinforcement learning   deep q-learning   mobile edge computing   resource allocation   task offloading  

摘要： Mobile Edge Computing (MEC) is a promising solution to enhance the computing capability of resource-limited networks. A fundamental problem in MEC is efficiently offloading tasks from user devices to edge servers. However, there still exists a gap to deploy in real-world environments: 1) traditional centralized approaches needs complete information of edge network, ignoring the communication costs generated by synchronization, 2) previous works do not consider concurrent computation on edge servers, which may cause dynamic changes in the environment, and 3) the scheduling algorithm should deliver individualized decisions for different users independently and with high efficiency To solve this mismatch, we studied a multi-user task offloading problem where user devices make offloading decisions independently. We consider the concurrent execution of tasks and formulate a non-divisible and delay-aware task offloading problem to jointly minimize the dropped task ratio and long-term latency. We propose a decentralized task scheduling algorithm based on DRL that makes offloading decisions without knowing the information of other user devices. We employ Double-DQN, Dueling-DQN, Prioritized Replay Memory, and Recurrent Neural Network (RNN) techniques to improve the algorithm's performance. The results of simulation experiments show that our method can significantly reduce the long-term latency and dropped task ratio compared to the baseline algorithms.