Yuan Li,Jiaxuan Xie,Mu Xia,Qianqian Li,Meng Li,Lei Guoand Zhen Zhang

1Tsinghua University,Beijing,100084,China

2School of Automation,Northwestern Polytechnical University,Xi’an,710129,China

3University of Science and Technology Beijing,Beijing,100083,China

4Beijing University of Posts and Telecommunications,Beijing,100876,China

5Systems Engineering Institute AMS,Beijing,100071,China

6Audio Analytic,2 Quayside,Cambridge,CB5 8AB,UK

7Beijing Commsat Technology Development Co.,Ltd.,Beijing,100089,China

Abstract:The goal of delivering high-quality service has spurred research of 6G satellite communication networks.The limited resource-allocation problem has been addressed by next-generation satellite communication networks,especially multilayer networks with multiple low-Earth-orbit(LEO)and nonlow-Earth-orbit(NLEO)satellites.In this study,the resource-allocation problem of a multilayer satellite network consisting of one NLEO and multiple LEO satellites is solved.The NLEO satellite is the authorized user of spectrum resources and the LEO satellites are unauthorized users.The resource allocation and dynamic pricing problems are combined,and a dynamic gamebased resource pricing and allocation model is proposed to maximize the market advantage of LEO satellites and reduce interference between LEO and NLEO satellites.In the proposed model, the resource price is formulated as the dynamic state of the LEO satellites,using the resource allocation strategy as the control variable.Based on the proposed dynamic game model,an openloop Nash equilibrium is analyzed, and an algorithm is proposed for the resource pricing and allocation problem.Numerical simulations validate the model and algorithm.

Keywords: Resource pricing; resource allocation; satellite network; LEO;dynamic game; Nash equilibrium

1 Introduction

Research of 6G mobile communication has become a major direction for the upgrade of satellite networks for next-generation mobile communication [1].This integrated space-air-ground network includes satellite and ground communication.Satellite networks facilitate global communication services and enhance network accessibility in areas inaccessible to ground communication networks [2–5].

Many companies have launched low-Earth-orbit (LEO) satellites, the next generation in satellite communication [6], for commercial reasons [7].Telesat launched 28 innovative LEO satellites integrated with a data network.SpaceX and Commsat plan to launch tens of thousands of LEO satellites to build a global satellite Internet.However, the deployment of LEO satellites can affect satellites already in orbit, especially regarding spectrum resource efficiency.Traditional nonlow-Earth-orbit (NLEO) satellites are authorized users with priority to use spectrum resources,and LEO satellites are unauthorized.The use of spectrum resources by LEO satellites interferes with NLEO satellites.Optimum allocation of spectrum resources will help LEO satellites balance user service requirements and address interference problems.The allocation of resources and management of NLEO satellite interference are challenging problems that must be addressed,and research on spectrum resource allocation in satellite communication networks is critically needed [8,9].

Many studies [10–13] have explored the resource allocation of satellite communication networks.A Stackelberg-game-based resource allocation scheme was proposed to divide satellites into two groups based on their priorities [10].In another study [11], a beam-hopping scheme was used to maximize the network throughput of a cognitive satellite network.Heuristic algorithms were proposed to solve the spectrum distribution problems.Deep reinforcement learning (DRL) was used to dynamically allocate resources in satellite communication networks [12].A joint power and sub-channel allocation problem was solved using a novel optimization model [13] to address the interference between primary and secondary networks.

To solve resource allocation problems, a dynamic game-based resource allocation scheme in multilayer satellite communication networks, using resource pricing as the main control variable,is proposed.The contributions of this study follow.

1) A dynamic game-based resource pricing and allocation model for LEO satellites, to control service price and resources for mobile users, is proposed.

2) The resource price is formulated as the dynamic state of the LEO satellites, the variation of which is affected by the resource allocation strategy.

3) Each satellite controls the resource allocation based on the Nash equilibrium solution for the proposed model.The optimal strategy is obtained for each satellite.

The rest of this paper is organized as follows.In Section 2, a system model for the research of a multilayer satellite communication network is provided and the resource pricing and allocation problem formulated.In Section 3, the proposed model is analyzed.Numerical simulations and their results are discussed in Section 4.Section 5 concludes the paper.

2 System Model and Problem Formulation

2.1 System Model

Fig.1 shows a multilayer satellite system, multiple mobile users, multiple LEO satellites, and one NLEO satellite.The NLEO satellite is the authorized user, with the right to use the spectrum resources.The LEO satellites are unauthorized users that cannot use the spectrum resources.They only have access to the spectrum resources if the NLEO satellite is not using them.The satellites are running in orbits, and when an LEO satellite moves into the NLEO satellite coverage area,it shuts down communication to allocate spectrum resources to the NLEO satellite.When not in the NLEO satellite coverage area, LEO satellites can use the spectrum resources.In the proposed multilayer satellite system, the movement of the LEO satellites and periodic usage of spectrum resources cause interference in the NLEO satellite network.Thus, controlling the LEO satellite equipment is crucial to managing their service time.This is an effective solution to the interference problem when LEO satellites are in the NLEO satellite coverage area.

However, the shutdown of LEO satellites will interrupt connected mobile users and increase the waiting time for service.This reduces the quality of service (QoS) and decreases revenue from mobile users.Users whose services are interrupted must be placed in a waiting queue for service recovery, and time in the queue is controlled by LEO satellite service strategies, which are critical to balance QoS and stop interference due to a NLEO satellite.

The multilayer satellite system is formulated herein as a dynamic game system using a dynamic variable state and control variables to optimally allocate the LEO satellites.Fig.1 shows the multilayer satellite system.LEO satellites should control service prices and times to balance the service and interference due to a NLEO satellite.

Figure 1:System model

2.2 Problem Formulation

It is assumed that N LEO satellites are in the proposed multilayer satellite system.After paying for spectrum resources, the LEO satellites provide mobile services.The communication load of mobile users on LEO satellite i at time t is denotedmi(t), and the resource allocated by the satellite isri(t).The service latency time isuiis the unit cost of service time, and the cost brought by latency is

The service price provided by LEO satellite i is denotedpi(t), with i = 1, 2,..., N.LEO satellites charge users a constant pricec, which is their minimum operation cost, wherepi(t)≥c.Given the service price, LEO satellites earn a profit of

LEO satellites control their pricing strategies, which are considered state variables of the proposed satellite system.The resource allocation strategies are the main criteria used to define resource-pricing strategies because the allocation strategies can affect latency performance [14].If LEO satellites allocate more resources, then communication service requirements are better satisfied, improving latency performance and lowering the resource price.If the allocated resources for mobile users decrease, then latency performance will be worse, which raises the resource price.When service resources increase, LEO satellites should increase the service price.The relationship between the service price and allocated resources is expressed as

whereδi,αi, andβiare weighted parameters.

Spectrum resources are unavailable when LEO satellites are in a NLEO satellite coverage area, and mobile users are placed in a queue.Assuming the arrival rate of a NLEO satellite isλ, following a Poisson distribution, then the probability that the LEO satellites are in a NLEO satellite coverage area is

The objective of each LEO satellite is to maximize the revenue earned from mobile users,

subject to

The resource pricing and allocation problem in the multilayer satellite system is formulated as a dynamic game, as follows:

? The LEO satellites are the players.

? The system state is the resource price.

? The strategy of each LEO satellite consists of the allocated resources for mobile users.

3 Game Analysis

The optimal strategies for the proposed problem are now discussed.Based on the system model and problem formulation, a dynamic resource pricing and allocation model is provided, as shown in Fig.2.

Figure 2:Pricing and allocation model

The Bellman dynamic programming technique is used to solve the proposed dynamic game model.As the resource-pricing strategy is formulated as the system state and the resource allocation strategy is the control variable, the optimal pricing strategy is achieved once the optimal resource allocation strategy for each LEO satellite is obtained.The following definitions must be developed before obtaining the optimal strategies.

Definition 1The resource allocation strategyri*(t)is the optimal strategy for each LEO satellite, which, for all strategiesri(t)ri*(t), satisfies

To obtain the resource allocation strategy for each LEO satellite, a Hamiltonian function is constructed for the proposed dynamic game model.This is a key component of the Bellman dynamic programming technique [15], and is defined as follows.

Definition 2The Hamiltonian function of LEO satelliteiis

whereΛi(t)is the co-state function, which satisfies

From the Hamiltonian function above, the first derivative is calculated to find the optimal resource allocation strategy for each LEO satellite, as explained by the following theorem.

Theorem 1The optimal resource allocation strategyri*(t)is obtained based on the open-loop Nash equilibrium of the proposed dynamic game in (5) and (6), wherepi*(t)is the corresponding optimal resource pricing strategy if there is a constant functionΛi(t)for LEO satellite i that satisfies

Considering the optimal resource allocation and pricing problem given by (3) and (4), based on Pontryagin’s maximum principle, the Nash equilibrium solution is achieved for each LEO satellite, as given in the following theorem.

Theorem 2There is a unique open-loop Nash equilibrium for each LEO satellite in the resource pricing and allocation problem, with optimal resource allocation strategy

where [pi(t),Λi(t)] are the solutions to the following equations:

Proof:The partial derivative of the Hamiltonian function given in (8) is calculated, giving

Setting the partial derivative to zero,

whereri*(t)is the optimal resource allocation strategy for LEO satellite i.Λi(t)is the co-state function, which satisfies

Taking the optimal resource allocation strategy of LEO satellite i into the various functions of the service price,

The algorithm to obtain the optimal resource allocation strategy for a LEO satellite is expressed as Algorithm 1.

Algorithm 1:Optimal resource allocation of LEO satellite Input:Arrival rate of NLEO satellite Output:Optimal resource allocation strategies for LEO satellites 1.Initialize service price;2.for each LEO satellite do 3.Set objective function according to formula (5);4.end for 5.do 6.Set Hamiltonian function of each LEO satellite;

?

Based on the proposed algorithm and the Nash equilibrium solutions of the proposed dynamic game, each satellite controls the resource allocation strategy and optimizes the objective function.An optimal pricing strategy is obtained for each satellite.As the differential gamebased model is non-cooperative, satellites do not require cooperation.To optimize their objective functions, the satellites non-cooperatively control their resource allocation strategies.

4 Numerical Simulations

Numerical simulations of the optimal pricing strategies and resource allocation strategies of LEO satellites were conducted using one NLEO satellite, which is the authorized user of spectrum resources, and three LEO satellites.When the LEO satellites are in the coverage area of the NLEO satellite, their spectrum resources are retrieved.Tab.1 provides parameters for the simulations.

Table 1:Simulation Parameters

Fig.3 provides the optimal resource allocation strategies for LEO satellites when the arrival rate of the NLEO satellite is 0.15.The LEO satellites allocate more resources at the beginning of the game to attract mobile users and provide satisfactory service.After approximately five iterations, each LEO satellite decreases the allocated resources to reduce the interference caused by the NLEO satellite.When the resource allocation strategies of the LEO satellites converge,LEO-2 has the most allocated resources and LEO-1 has the least; however, LEO-1 has the fastest convergence rate.Before the resource allocation strategies converge, LEO-3 has the largest value of allocated resources.

Figure 3:Resource allocation strategies when λ= 0.5

The LEO satellite’s optimal resource allocation strategies when the arrival rate of the NLEO satellite is changed were also simulated, as shown in Figs.4a and 4b.The arrival rates of the NLEO satellite in Figs.4a and 4b are 0.5 and 0.85, respectively.It is shown that the LEO satellites increase the allocated resources at the beginning of the game.After five iterations, the LEO satellites decrease their allocated resources.Comparing the resource allocation strategies in Figs.4a and 4b with that in Fig.3, it is observed that the allocated resources decrease with the increase of the NLEO satellite’s arrival rate.

Figure 4:Resource allocation strategies (a) λ=0.5 (b) λ=0.85

Fig.5 shows the optimal resource-pricing strategies for LEO satellites when the arrival rate of the NLEO satellite is 0.15.The resource-pricing strategy converges to a stable value after five iterations, which is fast.Based on the resource allocation strategies, LEO-2 and LEO-3 will increase their resource price and LEO-1 will decrease its price.LEO-3 has the highest resource price and LEO-1 has the lowest.

Figure 5:Resource pricing strategies when λ= 0.15

5 Conclusions

The resource allocation problem of a multilayer satellite system was investigated and a dynamic game-based resource pricing and allocation model, using differential equations to formulate the resource-pricing strategy of each LEO satellite, proposed.Utility maximization based on an objective function was proposed for each LEO satellite, with the resource allocation strategy as the control variable.Bellman dynamic programming was used to maximize the objective function,and the Nash equilibrium solution of resource allocation of each LEO satellite was obtained.An algorithm was developed for the resource pricing and allocation model, and numerical simulations demonstrated its accuracy.

Funding Statement:This work is supported by the National Natural Science Foundation of China(Grant No.61971032) and Fundamental Research Funds for the Central Universities (Grant No.FRF-TP-18-008A3).

Conflicts of Interest:The authors declare that they have no conflicts of interest to report regarding the present study.