Shengwei MEI,Wei WEI?,Feng LIU

1.Department of Electrical Engineering and Applied Electronics Technology,Tsinghua University,Beijing 100084,China;

2.State Key Laboratory of Control and Simulation of Power Systems and Generation Equipments,Tsinghua University,Beijing 100084,China

On engineering game theory with its application in power systems

Shengwei MEI1,2,Wei WEI1,2?,Feng LIU1,2

1.Department of Electrical Engineering and Applied Electronics Technology,Tsinghua University,Beijing 100084,China;

2.State Key Laboratory of Control and Simulation of Power Systems and Generation Equipments,Tsinghua University,Beijing 100084,China

Due to its capability of solving decision-making problems involving multiple entities and objectives,as well as complex action sequences,game theory has been a basic mathematical tool of economists,politicians,and sociologists for decades.It helps them understand how strategic interactions impact rational decisions of individual players in competitive and uncertain environment,if each player aims to get the best payoff.This situation is ubiquitous in engineering practices.This paper streamlines the foundations of engineering game theory,which uses concepts,theories and methodologies to guide the resolution of engineering design,operation,and control problems in a more canonical and systematic way.An overview of its application in smart grid technologies and power systems related topics is presented,and intriguing research directions are also envisioned.

Decision making,engineering game theory,power system,smart grid technology

1 Introduction

Game theory is the study of rational decision making among strategic participants,who pursue best payoffs for themselves,while taking into account the interaction of their profits among others and their abilities to gather information.In short,a game is the collection of several interdependent decision-making problems,in which every decision maker,also called a player,must react to other players’choices rationally.

The first influential monograph[1]that established the foundations of game theory was published in 1944,by Von Neumann and Morgenstein,who systematically described the constituents,rules,and outcomes of a game for the first time.The matrix game formulation introduced in[1]not only provides a concise characterization for a certain class of zero-sum game,but also a canonical modeling and analyzing paradigm for decision making under uncertainties.In early 1950s’,Nash proposed the models and solutions,later called Nash equilibrium,for general non-cooperative games in[2]and[3],whichbrought him a Nobel prize in 1994.The third milestone was the evolutionary game theory[4–6],which was developed by Smith in 1970s.Motivated by observations of animal behaviors,evolutionary game provides a dynamic view on the equilibrium as the consequence of repeated actions and learning from a biological perspective.

Although game theory originated from economics studies,it also appeared to be a powerful approach to design robust controllers against worst-case disturbances[7].The past a few years have witnessed precipitously increasing interests in the engineering applications of game theory.Undoubtedly,game theory have been an indispensable methodology for modern strategic decision making.A recent monograph[8]reveals the connection between game theory and traditional methods for optimal decision making,and formalizes the procedure of applying game theory methods in engineering problems,which is referred to as the“Engineering Game Theory”.

In fact,the terminology “Engineering Game Theory”is inspired by the phylogeny of control theory.In 1948,Wiener published the famous bookCybernetics:Or Control and Communication in the Animal and the Machine[9](Cyberneticsfor short),which has been recognized as the cornerstone in the field of control theory.However,in the beginning,there had been a debate on its scientific value:scientists criticized its obscurity,while engineers criticized its lack of systematic design procedures.It was not until H.S.Tsien issued his bookEngineering Cybernetics[10],which reported important applications of Wiener’s theory in aviation,navigation,electronics,and communication technologies,and proposed canonical controller design procedures,that the merit ofCyberneticshad been widely acknowledged.In analogy to Tsien’s terminology and attempt which streamlined engineering design principles of control theory,engineering game theory refers to concepts,theories and methodologies which guide the resolution of engineering design,operation,and control problems with canonical and stylized procedures.

This introductory paper is organized as follows.Fundamentals of game theory are reviewed in Section 2,including cooperative game,non-cooperative game,and evolutionary game.The central idea and major attempts of engineering game theory are presented in Section 3.The overview of smart grid applications are surveyed in Section 4.Conclusions are drawn in Section 5.

2 Fundamental game-theoretical concepts

This section is dedicated to an outlook of three major branches of game theory:Noncooperative game theory,cooperative game theory,and evolutionary game theory.They can be further classified according to certain properties,such as the number of stages(static games,dynamic games,and differential games),the structure of information,(perfect and imperfect information games;complete and incomplete information games),type of strategies(pure strategy games and mixed strategy games),and so on.

2.1 Noncooperative game theory

This branch focuses on the strategic decision-making problem of selfish players with conflicting utilities over the strategy space.The payoff of each player is not only influenced by his own decision,but also depends on the actions of others.In a static setting,all players make decisions in one shot without knowing rivals’actions.Even if the strategies are not selected synchronously,the game is still called a simultaneous one because each player is not aware of decisions made by others.Static games are known as the normal form games.In contrast,in a dynamic setting,players can observe the outcome of the previous stage,and based on which react to rivals’decisions adaptively in the next round.Time plays a central role in dynamic games,which are usually referred to as sequential,extensive or repeated games.In what follows,the definition and solution concepts of non cooperative games will be reviewed.More details can be found in[8].

Definition 1A static non cooperative game has three components:the set of players N,the sets of strategies(Ai)i∈N,and the utility functions(ui)i∈N.Letaibe the pure strategy ofi∈ N,anda?ibe the strategies of remaining players except that of playeri.When playing the game,each player chooses an actionai∈Aiand endeavours to optimize his utility functionui(ai,a?i).A mixed strategy of playerirefers to a certain assignment of probability to each pure strategyai∈Ai,which allows a player to randomly select a pure strategy.

To formulate a dynamic game,one needs to specify additional components,such as information sets which reflect the awareness of knowledge among players,and histories which represent sets of past actions.Please bear in mind that the notion of action does not coincides with that of a strategy in a dynamic game.An action is a move taken by a player at a certain stage of the game;a strategy is a complete plan which informs a player how to move under every possible situation throughout the game(interested readers are referred to[11]for more detailed descriptions).While in a static game,the terms action and strategy can be used interchangeably.

Noncooperative game theory aims to characterize the outcome of such an interactive and computative decision-making process,and provide suitable methodologies and algorithms for computing the outcome.Nash equilibrium is the most important solution concept for static noncooperative games.It portrays a state at which no playercan benefitfrom changing his strategy unilaterally.Hence no one has the incentive to deviate from that state.The pure strategy Nash equilibrium is defined as follows.

Definition 2A(pure-strategy)Nash equilibrium is a vectora?∈ A(where A= ?i∈NAiis the Cartesian production of the strategy sets)such that

Nash equilibrium depicts a stable outcome of a noncooperative game that can be reached and maintained spontaneously by the players in a distributed manner without coordination.However,only the existence of a mixed-strategy Nash equilibrium can be guaranteed.Conditions which ensure the existence of a pure-strategy Nash equilibrium can be restrictive.In addition,a game can have multiple equilibria.Identifying a meaningful and desirable equilibrium is challenging and remains an open problem,especially in power system applications[12].

Nash equilibrium underlies the equilibrium concepts of several other types of noncooperative games.For instance,the equilibrium of an incomplete information static game is called Bayes-Nash equilibrium;the equilibrium of a complete information dynamic game is named sub-game perfect Nash equilibrium;the equilibrium of an incomplete information dynamic game is referred to as perfect Bayes-Nash equilibrium.

2.2 Cooperative game theory

Noncooperative games are completely competitive,in which players are unable to coordinate with each other due to the lack of binding agreements for cooperation.However,in the presence of certain protocols the players have agreed on,they may be able to improve their overall utility by acting as an alliance.Cooperative games investigates the incentive for individual players which encourages them form a coalition and the imputation of total coalition revenue.Cooperative game theory encompasses two branches:coalitional games and Nash bargaining games.

A coalitional game focuses on what group will emerge rather than what individual players can do.In a stable coalition,no subset of players can unilaterally improve their outcomes.Three solution concepts of a coalitional game are introduced below.

The first concept is core,whose modern definition was introduced in[13].Core is the set of feasible imputations under which no sub-coalition has a total payoff that is greater than the sum of its players’payoffs.The core may contain none,only one,and infinitely many elements.To avoid an empty core,the ε-core is introduced in[14],which is always non-empty.The value ε represents a penalty for leaving the grand coalition.

The second concept is nucleolus[16],which provides an imputation with minimum dissatisfaction of players.If the core is nonempty,the nucleolus is unique and resides in the core,thus it is group and individually rational.The nucleolus is a promising scheme,as it combines a number of fairness consideration with stability criteria.

The third concept is Shapley value[15],which assigns the total coalitional surplus among players according to their marginal contributions.Shapley showed it satisfied efficiency,symmetry and additivity,and most important of all,it could provide a unique solution of a cooperative game,no matter whether the core is empty or not.

Nash bargaining game appears in the situations where players have to reach an agreement on the bargaining set,which is usually a Pareto set,so as to avoid their worst-case outcome as much as possible.In essence,the Nash bargaining game focuses on the negotiation of a fair outcome when the utility is not transferable.

Nash proposed the first bargaining game model in early 1950s[17,18],which characterized a fair compromising outcome among players based on simply information about their preferences.Nash provided several axioms to guarantee the efficiency and fairness of a bargaining solution:Pareto optimality,symmetry,independence of irrelevant alternatives,and invariance with respect to linear utility transformations.Nash further showed that the bargaining solution is unique and can be recovered from a single objective optimization problem.

Nash bargaining game is an appropriate and convincing method to balance multiple conflicting design ob-jectives.In spite of its advantages,the bargaining theory relies on the convexity of bargaining region.Real world optimization models usually render non-convex feasible regions,which will give rise to a non-convex bargaining problem.In such circumstance,only a local solution can be found quickly.

2.3 Evolutionary game theory

Traditional game theory assumes that players are rational:they are always able to identify the best strategy,which is somehow restrictive.In 1970’s,it underwent a transition to evolutionary game theory after Maynard Smith revealed the connection between Darwin’s biological evolutionary theory and the dynamics to reach an equilibrium of a game[19],called the replicator dynamics.Evolutionary game theory puts more emphasis on the dynamical behavior of strategy change.

Different from a classical game where players are clearly defined and fixed during the course of play.In contrast,players in an evolutionary game are changing over time,and the ultimate driving force is a replicator who can make approximately accurate copies of itself.The substance of a replicator can be a gene,a strategy,a species,and so on.A solution of an evolutionary game is called an evolutionarily stable strategy(ESS)which,if adopted by a population in a gaming environment,cannot be invaded by alternative strategies(populations)that are initially rare.ESS has some connection with a mixed strategy equilibrium.Hawk-Dove game is a very famous example of ESS.The main concept of ESS helps develop deeper understanding of dynamical systems in biology and social sciences.

3 Basic scientific problems in engineering game theory

3.1 Motivations of engineering games theory

As a complicated cyber-physical network that integrates sophisticated energy production,system control,communication,and information technologies,the designing,planning,operation,and control of the modern smart grid renders multi-objective optimization problems with multiple decision makers and uncertainties.At the generation side,the high penetration of volatile renewable energies introduces notable uncertainties in energy production.At the distribution side,load aggregators are behaving more active in response to real time energy prices encouraged by demand response programs.In the merging trend of energy internet,the uncertain and competitive characteristics of the networked heterogeneous energy systems will become more prominent,which call for sophisticated and systematic methodology to overcome technical challenges at different levels.More precisely,there is an urgent need to develop novel methods endowed with the following functionalities:

1)Compromising multiple conflicting targets;

2)analyzing and coordinating strategies in interdependent optimization problems with multiple decision makers and complex action sequences;and

3)hedging against uncertainty.

The superiority of game theory is apparent in meeting the former two requirements.As for the last one,our motivation is explained as follows.On the one hand,as the human decision maker must determine his strategy without exact information on the future realization of uncertain factors,a prudent choice will be planning for the worst-case outcome,so as to be able to cope with all possible situations.In this regard,a zero-sum game can be formulated in which the uncertainty is regarded as a virtual player that always has an utility opposite to that of the human decision maker.On the other hand,with the uncertainty rolling in over time,the human decision maker can deploy corrective actions to compensate adverse effects brought by the uncertainty,which can be modeled as a dynamic game.In this context,game theory provides a plausible framework that can address many challenging problem involving multiple decision agents and uncertainties.In what follows,we summarize four major scientific problems that engineering game theory in[8]deals with via systematic synthesis.

3.2 How to solve multi-objective optimization problems via static games?

In a multi-objective optimization problem,several conflicting targets should be coordinately optimized.The solution concept refers to the Pareto optimal solution,or non-dominated solution,which is defined as a state in which it is unable to improve an individual objective without compromising at least one other objective.Traditional multi-objective optimization theory aims to seek the Pareto front,or a set of uniformly distributed Pareto optimal solutions.Due to the lack of a uniform criterion,it is sometimes difficult to determine a fair trade off from infinitely many Pareto solutions.The user has to decide one strategy for deploying according to his own preferences,which is somehow subjective.

To overcome this difficulty,the multi-objective optimization problem is considered as a static game,in which each objective plays the role of a player,whose utility is influenced by the strategies of other players.Despite of their similarities,a static game and a multi objective optimization also exhibits clear distinctions:in the former one,decision variables are separately controlled by individual players;in the latter one,one central authority coordinates individual objectives via adjusting all decision variables.This difference leads to the discrepancy between Nash equilibrium and Pareto solution.Engineering game theory provides two options to solve a multi-objective optimization problem[8].The first one treats it as a noncooperative game and recovers a single Nash equilibrium as the solution.The second one considers it as a Nash bargaining game and computes a bargaining solution,which is also Pareto optimal.Computing the Pareto front is not the main focus of engineering game theory.

3.3 How to solve robust optimization problems via concepts of zero-sum game?

It has been reported that optimal solutions of mathematical programs can be extremely sensitive to parameter perturbations,thus a solution in the nominal case often appears to be highly infeasible or suboptimal in practice[20].This is particularly the case for power system optimization problems with renewable energy generations.Optimization problems affected by uncertain parameters has been a focus of the operational research community for a long time.

Engineering game theory tackles such problems via zero-sum games[8].One player is the human decision maker who wishes to maximize his profit.The other one is the nature,who seeks the most threatening strategy by trying to minimizing its rival’s profit.This motivation has been clarified in Section 3.1.Engineering game theory offers flexible ways to model different decision-making patterns.If all strategies should be deployed before uncertainty is known,the situation can be formulated as a static game;If corrective actions are allowed with uncertainty being observed over time,the process can be depicted by a dynamic game.Although the decisions offered by zero-sum game models could be somehow conservative,it is still desired by power systems,as the consequence of failures may be catastrophic,and security is explicitly guaranteed by taking into account the worst-case outcome.

3.4 How to design robust controller via concepts of differential game?

Methods for solving optimal control problems rest on two different fundamental principles:Pontryagin’s maximum principle and Bellman’s optimality principle.They are suitable for deterministic problems,and can boil down to the Hamilton-Jacobi-Bellman(HJB)equation under certain conditions.However,uncertainties are inevitable in real-world engineering systems,such as inexact system parameter and unmodeled dynamics,which may have a great impact on the performance of the optimal controller.

It was Isaacs pioneered the theory of differential games[21],which enabled an extension of optimal control problems with multiple participants in a noncooperative game theoretical setting.Similar to the robust optimization,the most extreme case comes down to a zero-sum game,in which the nature pursues a diametrically opposite objective against the designer.The two person zero-sum differential game has been extensively studied in control theory in the sense of robust H∞control problem[22],where the equilibrium yields a saddle point.For linear dynamic systems,developing an analytical solution is equivalent to solving an algebraic Riccati inequality.For nonlinear dynamic systems,computing the saddle point relies on solving a Hamilton-Jacobi-Isaacs(HJI)partial differential inequality,which remains an open problem.Engineering game theory introduces three tractable methods for the challenging saddle point problem of affine nonlinear dynamic systems,including a variable-scale feedback linearization approach,a Hamilton system approach,and the approximate dynamic programming approach[8].They share a feature in common that a saddle point can be recovered without solving the HJI partial differential inequality.

3.5 How to solve multi-level optimization problems using concepts of dynamic game?

A multi-level optimization problem is a hierarchy of several optimization problems.The most representative one is the bilevel program,in which two decision makers act sequentially.The so-called leader optimizes his objective subject to physical constraints,which depend on his own strategy as well as the optimal choice of a socalled follower.The strategy of the leader also influences the feasible set and the utility of the follower.In view of such an interaction,when the leader makes decisions,he must consider the reaction from the follower.In fact,the first formulation of the bilevel program can be traced back to the Stackelberg game appeared in[23],and the modern mathematical formulation is proposed in[24].Due to its strength on describing sequential decision making problems,there have been fruitful outcomes in theoretical development and engineering applications of bilevel optimization in the past decades[25].

Bilevel programs belong to the category of nonconvex optimization problems.The complexity of solving a general bilevel program,and even validating local optimality for a given solution,have been shown to be NP-hard[25].Commonly used constraint qualifications are violated at every feasible point,preventing the direct application of commercial nonlinear programming solvers.

In light of the connection between bilevel programs and dynamic games,the analytical method of dynamic game theory,i.e.,the backward induction,has been used to develop the single-level equivalence of bilevel programs[8].More specifically,the follower’s optimization is replaced by its first-order optimality condition,say,the KKT optimality condition,which is included in leader’s constraints.This replacement portrays how the leader predicts the best response from the follower in game theoretical language.

Moreover,engineering game theory entails a generalization of the standard bilevel program to include multiple leaders in the upper level[8],resulting in a multi-leader-follower game[26],or an equilibrium program with equilibrium constraints[27].The Nash equilibrium among leaders can be computed using the best response iteration method similar to a traditional noncooperative game.Nevertheless,due to the non-convexity of the multi-leader-follower game,a pure-strategy Nash equilibrium may not exist;even if one does exist,the best-response procedure may not converge,unless the initial point is close enough to the true equilibrium.

3.6 Connections with other disciplinary

According to above discussions we can see that engineering game theory intertwines with a number of disciplines in control and optimization studies.A brief comparison is provided in Table 1.Game problems in engineering usually involve multiple players with partly or completely conflicting objectives,or uncertainties resulting from inexact parameter,unmodeled factors,and expected contingencies.Uncertainty is treated as a virtual player who pursues an opposite objective against the human decision maker.A traditional optimization problem can be regarded as a special game where only one player exists.It should be pointed out that despite several similarities between a noncooperative game and a multi-objective optimization problem,they are conceptually different because the latter describes the coordination among multiple payoffs of only one decision maker,which is more similar to a cooperative game.

Table 1 A comparison of game and other disciplines.

Needless to say,owing to the diversity of objectives,constraints,and competition patterns,the existence and uniqueness guarantee of equilibria of engineering games is non-trivial.Computing an equilibrium is also much more challenging than solving a traditional optimization problem.Approaches with pertinence to special problems are recommended,in accordance with the discussions in[8].For the Bargaining game model of multi objective optimization,it can be cast a traditional convex optimization problem.Fornon-cooperative games,if every single player’s problem is strongly convex and can be readily solved,the best response algorithm will be a good option;otherwise,the stationary method,which solves the concentrated KKT system consisting of KKT optimality conditions of individual players’problem,is one possible alternative.

4 Applications in power system

The modern power system encompasses various stakeholders(such as generation companies,grid operators,distribution utilities,and auxiliary service providers),conflicting optimization targets(including production cost,emission reduction,security enhancement),and uncertainties(resulting from renewable power generation and contingencies).To implement reliable and efficient power generation,transmission,and distribution,solid mathematical tools that help make scientific decisions at different levels from planning,operation,and control are in great need.The heterogeneous and uncertain nature of modern power systems inspires the exploration of engineering game theory for mitigating technical challenges at different levels.In this context,this section provides an overview on engineering game theory applications in relevant topics of power systems and smart grid technologies,which exhibits a clear outlook on the success and possible challenges of adopting game theoretical methods in future energy systems and more broader classes of engineering decision making problems.A more thorough and detailed survey can be found in[28].

4.1 Power market

Due to its capability of capturing interactions among interdependent decision-making problems,game theory has been the canonical and prevailing paradigm adopted in power market studies.For example,the celebrated C our not model and Bertrand model for quantity and price competition[29],respectively,are basic means for market analysis.

In more dedicated wholesale power market research,generation companies bid their offering prices,and the independent system operator clears the market according to an optimal power flow problem with fixed bidding prices,determining the contract power of each units as well as the locational marginal prices of electricity[12].Given this bilevel structure,the strategic bidding problem gives rise to multi-leader-follower games,which have been studied in[12,27,30,31].A common feature is that the market clearing problem is modeled as a linear program or a convex quadratic program in the lower level,which can be replace by the KKT optimality condition,and the market equilibrium can be found by solving mathematical programs with complementarity constraints through best-response iterations.More recently,the bilevel game approach has been applied to the analysis and design of energy markets with renewable generations[32–35],showing its strong vitality in this line of research.

4.2 Power system planning

Power system planning is one of the most important subject of electrical engineering.It provides references for sitting and sizing of generation/storage equipment as well as upgrading of the network components,such as substations and transmission lines.A proper expansion planning can bring enormous social and economic benefits.Because the typical time frame of planning problem goes up to one or two decades,the decision maker must balance short-term and long-term revenues,investment,and reliability.Moreover,if multiple investors are engaged,the power system planning is an emblematical multi-agent and multi-objective decision-making problem with uncertainty.In this regard,engineering game theory is playing an increasingly important role in this area.

The Cournot model is used in[36]to formulate the competitive generation expansion planning,in which each firm maximizes his profit given all other firms’quantity decisions.The Shapley value is employed in[37]to coordinate the expansion income most efficiently.The competitive generation expansion planning is formulated as a multi-leader-follower game with incomplete information in[38],in which the upper level decides investors’planning and energy/reserve bidding strategies.The lower level involves the market clearing problem which determines the electricity prices.Simultaneous generation and transmission expansion planning is studied in a similar manner in[39].

Noncooperative game is introduced for the planning of a grid-connected hybrid power system comprised of wind turbines,photovoltaic panels,and storage batteries[40].Four different coalition forms are also investigated.Imputation schemes are discussed in[41].The static reserve capacity planning with high penetration of wind power is formulated as a zero-sum game in[42],which aims to find the minimum reserve capacity while keeping the system reliability index within a desired value.

4.3 Power system dispatch

The main target of power system dispatch is to maintain a stable,reliable,and economic operation condition.However,high penetration of renewables introduces remarkable uncertainty in the generation side,which greatly challenges load balancing in real time.In engineering game theory,uncertainties of renewable power output is treated as a virtual player,i.e.,the nature,which seeks the most unfavorable strategy of the grid,while the system operator has to find out a feasible generation dispatch in response to the strategy of the nature.

The gaming interpretation of power system robust dispatch is discussed in[43],in which a zero-sum two stage dynamic game model is suggested for power system dispatch with electric vehicles.In order to model both preventive and corrective actions,the adaptive robust optimization,which is a three-stage dynamic game,is firstly applied to unit commitment problem[44–46].The three-stage dynamic game approach for more general power systems operation problems is investigated in[47],and further applied to a joint energy and reserve dispatch problem in[48].

The three-stage dynamic game can be solve by the Benders decomposition algorithm[45,46],or the constraint and column generation algorithm[49].To generate a cut,it is necessary to solve a two-stage dynamic game,which can be transformed into a bilinear program[50].Bilinear programs are NP-hard.To retrieve a global optimal solution,the mixed-integer linear programming method in[48,51]can be used;otherwise,the mountain climbing method[52]can find a local solution efficiently.

For a multi-objective dispatch problem,Nash bargaining criterion is employed to compromising conflicting targets[53,54],such as generation cost and carbon emission.Uncertainty is taken into account in a multi objective dispatch problem in[55].

4.4 Power system control

Power systems may experience disturbances during operation frequently,such as load perturbation,measurement error,line tripping,generator outage.Robust control is the most representative application of differential game theory.Excitation system disturbance attenuation is studied in[56,57].Robust excitation controller for larger generation units is designed based on the saddle point principle in[58,59].Approximate dynamic programming is used to compute the saddle point,and design load-frequency controller with large-scale wind power integration in[60],and supplementary reactive power control for wind farms to enhance power system stability in[61].

A game theoretic approach for controlling sources and loads in small-scale direct current power systems is proposed in[62].Moreover,Nash bargaining approach is recommended for improving the efficiency of the equilibrium of a non-cooperative game between two loads.A differential game based approach is devised in[63]for the optimal control of load players during a cold start.It minimizes losses and achieves a desired steady-state operating point based on local measurements without communications.A differential game based cooperative control scheme is proposed in[64]for coordinating load frequency control and tie-line scheduling.It is shown that the cooperative scheme is promising to reduce the regulation burden and improve system dynamic performance.

4.5 Micro-grids and distributed generation

Micro-grid consists of a connected group of distributed generators such as photovoltaic panels and wind turbines and local demands in a small geographical area.It can be operated both in grid-connection orisl and mode.Different from a traditional power grid,defining a centralized operation and control objective for microgrids is more difficult,because of the heterogeneous nature of the utilities,which often include electric vehicles cars,heating devices,energy storage equipment,diesel generators,distributed renewable units,and so on.To this end,it is natural to investigated the application of distributed coordination techniques enabled by game theory in microgrid operation and control.The game theory implementation is also motivated by the fact that microgrid participants can respond very quickly to changing operating conditions.In this respect,four research directions are open for engineering game theory,which have already received wide attentions.

1)Energy exchange between among DGs,microgrids,and distribution networks,electricity pricing,distribution control and optimization via cooperative and noncooperative games[65–68].

2)Grid impacts of uncertain generation resources,robust energy management via zero-sum game and robust optimization[69–72].

3)Electricity pricing,generation dispatch,and equilibrium in the retail market[73–76].

4)Communication structure and cyber-physical security of networked microgrids[77–79].

4.6 Demand response

Demand-side management is an essential practice in the smart grid which provide opportunities to smooth the load profile over time by shaving peaks and filling valleys.Provided with financial incentives,customers would like to reduce part of their energy consumptions or shift electricity usages during peak hours.Demandside management entails interplays between two entities:utility companies and consumers.The latter often includes smart buildings,storage companies,and electric vehicles.

Efficient pricing schemes that could encourage load reshaping plays a central role in demand response programs.A Stackelberg game based pricing approach is proposed for electricity retailers in a demand response market[80].The similar method is applied in[81]to determine the optimal contract price of distributed gen-erators in distribution systems.An optimal time-of-use pricing scheme is described in[82].Utility functions for the provider and users are properly designed,and the equilibrium is derived using backward induction.More pricing methods can be found in[83].

A distributed energy management approach is developed in[84]based on game theory.A best response algorithm is used to find the Nash equilibrium while ensuring fairness and protecting privacy.The similar problem with load prediction and real-time adjustment is model as a generalized Nash equilibrium problem in[85].An iterative algorithm is employed to implement the distributed energy management,which converges to the variational equilibrium,i.e.,a special Nash equilibrium at which global constraints share the same shadowing prices in each players’problem.A Stackelberg game between the utility company and end-users is introduced in[86].A distributed algorithm which converges to the unique equilibrium with only local information is devised.

4.7 Power system security

Connected by the ultra-high voltage transmission network,the statewide power grid in China is made up of tens of thousands of buses and lines,as well as thousands of generators.With the deepened reforming of energy industry policy and proliferation of power electronics based convertors,both static operation and dynamic control of such a complicated power system are becoming more and more challenging.Moreover,existing stability and reliability evaluation methods are orientated towards given failure.They are not eligible for assessing the consequence of malicious attacks,not to mention cascading failure prevention and interdiction.

Engineering game theory enables a multi-stage description of the interaction between the attacker and system defender[8],creating a new branch of engineering games:network security game,a zero-sum dynamic game with two or three stages.The former encompasses the attacker-defender model and defenderattacker model[87–91],depending on the sequence of actions;The latter allows both preventive and correction actions deployed by the defender,and gives rise to a defender-attacker-defender model[92–95].The equilibria of network security games suggest the most vulnerable components in the system,as well as the best defensive strategy when the network is under attack.

In short,engineering game theory opens a new way to protect complex networked system against deliberated attack,and enhance system vulnerability and resilience.

4.8 Power system evolution

As an important research direction in power system engineering,the study of power grid evolution not only helps gain deeper insights on the history of grid development,but also offers useful suggestions on power planning and topology design.Existing power system planning gives optimal expansion strategies based on certain optimization models.The lack of dynamic and variational perspectives makes it less competitive in revealing the evolutionary mechanism of power systems.The three-generation power grid theory in[96]explains the evolution of power system from a biological point of view.In this context,evolutionary game theory inspired research on power system evolution include two basic problems

1)General principles on power grid’s evolution,including driving factor identification,evolving model and analytical method,based on complex network theory[97].

2)The power grid evolutionary game model,whose evolutionary stable strategy balances safety,cost,and environmental impact.

Research in this direction provides a holistic view on the power grid evolution history,and helps related authority make scientific and reasonable decisions on power network planning and grid operation.

5 Conclusions

We have explained basic motivations of engineering game theory,summarized its scientific orientations,and provided a comprehensive overview on its applications in power system pertinent problems.We hope that through developing engineering game theory,we can build a bridge between theorists and engineers,such that game theory should show its promising potential to provide analytical foundations and decision support tools for practical problems in a variety of engineering subjects in energy industry,wireless power transfer and communication technology,information science,etc.,in spite of possible challenges and difficulties,rather than sinking into pure logical and mathematical studies.Nevertheless,engineering game theory is still in its infancy stage.More work should be done to improve its theoretical soundness and practical relevance.

[1]J.von Neumann,O.Morgenstern.Theory of Games and Economic Behavior.Princeton:Princeton University Press,1944.

[2]J.F.Nash.Equilibrium points in n-person games.Proceedings of the National Academy of Sciences,1950,36(1):48–49.

[3]J.F.Nash.Non-cooperative games.Annals of Mathematics,1951,54(2):286–295.

[4]J.M.Smith,G.R.Price.The logic of animal conflict.Nature,1973,246:15–18.

[5]J.M.Smith.Game theory and the evolution of behavior.Proceedings of the Royal Society of London–Series B:Biological Sciences,1979,205(1161):475–488.

[6]J.M.Smith.Evolution and the Theory of Games.Cambridge:Cambridge University Press,1982.

[7]X.Yang,B.Ye.Linear and NonlinearH∞Control.Tabei:Chuan Hwa Publishing Ltd.,1997.

[8]S.Mei,F.Liu,W.Wei.Foundations of Engineering Game Theory and its Applications in Power Systems(in Chinese).Beijing:Science Press,2016.

[9]N.Wiener.Cybernetics or Control and Communication in the Animal and the Machine.Cambridge:MIT Press,1948.

[10]H.S.Tsien.Engineering Cybernetics.New York:McGraw-Hill,1954.

[11]T.Basar,G.J.Olsder.Dynamic Noncooperative Game Theory.Philadelphia:SIAM,1999.

[12]S.A.Gabriel,A.J.Conejo,J.D.Fuller,et al.Complementarity Modeling in Energy Markets.New York:Springer,2013.

[13]D.B.Gillies.Solutions to general non-zero-sum games.A.W.Tucker&H.W.Kuhn(Eds.).Contributions to the Theory of Games.Annals of Mathematical Studies 40.Princeton:Princeton University Press:47–85.

[14]L.S.Shapley,M.Shubik.Quasi-cores in a monetary economy with nonconvex preferences.Econometrica:Journal of the Econometric Society,1966,34(4):805–827.

[15]L.S.Shapley.A value for n-person games.H.W.Kuhn&A.W.Tucker(Eds.).Contributions to the Theory of Games.Annals of Mathematical Studies 28.Princeton:Princeton University Press:307–317.

[16]D.Schmeidler.The nucleolus of a characteristic function game.SIAM Journal on Applied Mathematics,1969,17(6):1163–1170.

[17]J.F.Nash.The bargaining problem.Econometrica:Journal of the Econometric Society,1950,18(2):155–162.

[18]J.F.Nash.Two person cooperative games.Econometrica:Journal of the Econometric Society,1953,21(1):128–140.

[19]M.Smith.Evolutionary Genetics.New York:Oxford University Press,1989.

[20]A.Ben-Tal,A.Nemirovski.Robust solutions of linear programming problems contaminated with uncertain data.Mathematical Programming,2000,88(3):411–424.

[21]R.Isaacs.Differential Games.Hoboken:Wiley,1967.

[22]T.Basar,P.Bernhard.H∞Optimal Control and Related Minimax Design Problems.Boston:Birkhauser,2008.

[23]H.von Stackelberg.Marktform und Gleichgewicht.Berlin:Springer,1934.The Theory of the Market Economy(English translated).Oxford:Oxford University Press,1952.

[24]J.Bracken,J.McGill.Mathematical programs with optimization problems in the constraints.Operations Research,1973,21(1):37–44.

[25]S.Dempe,V.Kalashnikov,G.A.Perez-Valdes,et al.Bilevel Programming Problems:Theory,Algorithms and Applications to Energy Networks.Berlin:Springer,2015.

[26]J.S.Pang,M.Fukushima.Quasi-variational inequalities,generalized Nash equilibria,and multi-leader-follower games.Computational Management Science,2005,2(1):21–56.

[27]C.L.Su.Equilibrium Problems with Equilibrium Constraints:Stationarities,Algorithms,and Applications.Stanford:Stanford University,2005.

[28]W.Saad,Z.Han,H.V.Poor,et al.Game-theoretic methods for the smart grid:An overview of microgrid systems,demandside management,and smart grid communications.IEEE Signal Processing Magazine,2012,29(5):86–105.

[29]N.Singh,X.Vives.Price and quantity competition in a differentiated duopoly.Rand Journal of Economics,1984,15(4):546–554.

[30]B.F.Hobbs,C.B.Metzler,J.S.Pang.Strategic gaming analysis for electric power systems:An MPEC approach.IEEE Transactions on Power Systems,2000,15(2):638–645.

[31]X.Hu,D.Ralph.Using EPECs to model bilevel games in restructured electricity markets with locationalprices.Operations Research,2007,55(5):809–827.

[32]M.Banaei,M.O.Buygi,H.Zareipour.Impacts of strategic bidding of wind power producers on electricity markets.IEEE Transactions on Power Systems,2016,31(6):4544–4553.

[33]S.J.Kazempour,H.Zareipour.Equilibria in an oligopolistic market with wind power production.IEEE Transactions on Power Systems,2014,29(2):686–697.

[34]T.Dai,W.Qiao.Optimal bidding strategy of a strategic wind power producer in the short-term market.IEEE Transactions on Sustainable Energy,2015,6(3):707–719.

[35]E.G.Kardakos,C.K.Simoglou,A.G.Bakirtzis.Optimal offering strategy of a virtual power plant:A stochastic bi-level approach.IEEE Transactions on Smart Grid,2016,7(2):794–806.

[36]A.S.Chuang,F.Wu,P.Varaiya.A game-theoretic model for generation expansion planning:Problem formulation and numerical comparisons.IEEE Transactions on Power Systems,2001,16(4):885–891.

[37]N.I.Voropai,E.Ivanova.Shapley game for expansion planning of generating companies at many non-coincident criteria.IEEE Transactions on Power Systems,2006,21(4):1630–1637.

[38]J.Wang,M.Shahidehpour,Z.Li,et al.Strategic generation capacity expansion planning with incomplete information.IEEE Transactions on Power Systems,2009,24(2):1002–1010.

[39]M.Jenabi,G.Fatemi,Y.Smeers.Bi-level game approaches for coordination of generation and transmission expansion planning within a market environment.IEEE Transactions on Power Systems,2013,28(3):2639–2650.

[40]S.Mei,Y.Wang,F.Liu,et al.Game approaches for hybrid power system planning.IEEE Transactions on Sustainable Energy,2012,3(3):506–517.

[41]Y.Wang,S.Mei,F.Liu.Imputation schemes for the cooperative game in the hybrid power system planning.Journal of Systems Science and Complexity,2012,32(4):418–428(in Chinese).

[42]S.Mei,D.Zhang,Y.Wang,et al.Robust optimization of static reserve planning with large-scale integration of wind power:A game theoretic approach.IEEE Transactions on Sustainable Energy,2014,5(2):535–545.

[43]S.Mei,W.Guo,Y.Wang,et al.A game model for robust optimization of power systems and its application.Proceedings of the CSEE,2013,33(19):47–56(in Chinese).

[44]R.Jiang,J.Wang,Y.Guan.Robust unit commitment with wind power and pumped storage hydro.IEEE Transactions on Power Systems,2012,27(2):800–810.

[45]L.Zhao,B.Zeng.Robust unit commitment problem with demand response and wind energy.IEEE Power and Energy SocietyGeneral Meeting.San Diego:IEEE,2012:DOI 10.1109/PESGM.2012.6344860.

[46]D.Bertsimas,E.Litvinov,X.A.Sun,et al.Adaptive robust optimization for the security constrained unit commitment problem.IEEE Transactions on Power Systems,2013,28(1):52–63.

[47]W.Wei.Power System Robust Dispatch:Models and Applications.Beijing:Tsinghua University,2013(in Chinese).

[48]W.Wei,F.Liu,S.Mei,et al.Robust energy and reserve dispatch under variable renewable generation.IEEE Transactions on Smart Grid,2015,6(1):369–380.

[49]B.Zeng,L.Zhao.Solving two-stage robust optimization problems using a column-and-constraint generation method.Operations Research Letters,2013,41(5):457–461.

[50]J.E.Falk.A linear maxmin problem.Mathematical Programming,1973,5(1):169–188.

[51]M.Zugno,A.J.Conejo.A robust optimization approach to energy and reserve dispatch in electricity markets.European Journal of Operational Research,2015,247(2):659–671.

[52]H.Konno.A cutting plane algorithm for solving bilinearprograms.Mathematical Programming,1976,11(1):14–27.

[53]W.Wei,F.Liu,S.Mei.Nash bargain and complementarity approach based environmental/economic dispatch.IEEE Transactions on Power Systems,2015,30(3):1548–1549.

[54]W.Wei,J.Wang,S.Mei.Convexification of the Nash bargain based environmental-economic dispatch.IEEE Transactions on Power Systems,2016,31(6):5208–5209.

[55]W.Wei,F.Liu,J.Wang,et al.Robust environmental-economic dispatch incorporating wind power generation and carbon capture plants.Applied Energy,2016,183:674–684.

[56]Q.Lu,S.Mei,W.Hu,et al.Nonlinear decentralized disturbance attenuation excitation control via new recursive design for multimachine power systems.IEEE Transactions on Power Systems,2001,16(4):729–736.

[57]Q.Lu,S.Mei,W.Hu,et al.Decentralised nonlinear H∞excitation control based on regulation linearization.IEE ProceedingsGeneration,Transmission and Distribution,2000,147(4):245–251.

[58]Q.Lu,S.Zheng,S.Mei,et al.NR-PSS(nonlinear robust power system stabilizer)for large synchronous generators and its large disturbance experiments on real time digital simulator.Science in China Series E:Technological Sciences,2008,51(4):337–352.

[59]S.Mei,W.Wei,S.Zheng,et al.Development of an industrial non-linear robust power system stabiliser and its improved frequency-domain testing method.IET Generation,Transmission and Distribution,2011,5(12):1201–1210.

[60]W.Guo,F.Liu,J.Si,et al.Online supplementary ADP learning controller design and application to power system frequency control with large-scale wind energy integration.IEEE Transactions on Neural Networks and Learning Systems,2016,27(8):1748–1761

[61]W.Guo,F.Liu,J.Si,et al.Approximate dynamic programming based supplementary reactive power control for DFIG wind farm to enhance power system stability.Neurocomputing,2015,170:417–427.

[62]W.W.Weaver,P.T.Krein.Game-theoretic control of small-scale power systems.IEEE Transactions Power Delivery,2009,24(3):1560–1567.

[63]N.C.Ekneligoda,W.W.Weaver.Game-theoretic cold-start transient optimization in DC microgrids.IEEE Transactions on Industrial Electronics,2014,61(12):6681–6690.

[64]H.Chen,R.Ye,X.Wang,et al.Cooperative control of power system load and frequency by using differential games.IEEE Transactions on Control Systems Technology,2015,23(3):882–897.

[65]J.Kim,J.Jeon,S.Kim,etal.Cooperative controlstrategy of energy storage system and microsources for stabilizing the microgrid during islanded operation.IEEE Transactions on Power Electronics,2010,25(12):3037–3048.

[66]Z.Wang,B.Chen,J.Wang,et al.Coordinated energy management of networked microgrids in distribution systems.IEEE Transactions on Smart Grid,2015,6(1):45–53.

[67]Z.Wang,B.Chen,J.Wang.Decentralized energy management system for networked microgrids in grid-connected and islanded modes.IEEE Transactions on Smart Grid,2016,7(2):1097–1105.

[68]Q.Wang,C.Zhang,J.Wang,et al.Real-time trading strategies of proactive DISCO with heterogeneous DG owners.IEEE Transactions on Smart Grid::DOI 10.1109/TSG.2016.2597263.

[69]W.Wei,F.Liu,S.Mei.Dispatchable region of the variable wind generation.IEEE Transactions on Power Systems,2015,30(5):2755–2765.

[70]W.Wei,F.Liu,S.Mei.Real-time dispatchability of bulk power systems with volatile renewable generations.IEEE Transactions on Sustainable Energy,2015,6(3):738–747.

[71]Y.Zhang,N.Gatsis,G.Giannakis.Robustenergy management for microgrids with high-penetration renewables.IEEE Transactions on Sustainable Energy,2013,4(4):944–953.

[72]T.Ding,S.Liu,W.Yuan,et al.A two-stage robust reactive power optimization considering uncertain wind power integration in active distribution networks.IEEE Transactions on Sustainable Energy,2016,7(1):301–311.

[73]G.Asimakopoulou,A.Dimeas,N.Hatziargyriou.Leader-follower strategies for energy management of multi-microgrids.IEEE Transactions on Smart Grid,2013,4(4):1909–1916.

[74]W.Wei,F.Liu,S.Mei.Energy pricing and dispatch for smart grid retailers under demand response and market price uncertainty.IEEE Transactions on Smart Grid,2015,6(3):1364–1374.

[75]S.Mei,W.Wei.Hierarchal game and its applications in the smart grid.Journal of Systems Science and Mathematical Sciences,2014,34(11):1331–1344(in Chinese).

[76]J.Lee,J.Guo,J.Choi,et al.Distributed energy trading in microgrids:A game-theoretic model and its equilibrium analysis.IEEE Transactions on Industrial Electronics,2015,62(6):3524–3533.

[77]N.C.Ekneligoda,W.W.Weaver.Game-theoretic communication structures in microgrids.IEEE Transactions on Smart Grid,2015,6(2):1064–1072.

[78]P.Chen,S.Cheng,K.Chen.Smart attacks in smart grid communication networks.IEEE Communications Magazine,2012,50(8):24–29.

[79]S.Backhaus,R.Bent,J.Bono,et al.Cyber-physical security:A game theory model of humans interacting over control systems.IEEE Transactions on Smart Grid,2013,4(4):2320–2327.

[80]M.Zugno,J.M.Morales,P.Pinson,et al.A bilevel model for electricity retailers’participation in a demand response market environment.Energy Economics,2013,36:182–197.

[81]J.M.Lopez-Lezama,A.Padilha-Feltrin,J.Contreras,et al.Optimal contract pricing of distributed generation in distribution networks.IEEE Transactions on Power Systems,2011,26(1):128–136.

[82]P.Yang,G.Tang,A.Nehorai.A game-theoretic approach for optimaltime-of-use electricity pricing.IEEE Transactions on Power Systems,2013,28(2):884–892.

[83]J.S.Vardakas,N.Zorba,C.V.Verikoukis.A survey on demand response programs in smart grids:pricing methods and optimization algorithms.IEEE Communications Surveys&Tutorials,2015,17(1):152–178.

[84]H.Mohsenian-Rad,V.Wong,J.Jatskevich,et al.Autonomous demand side management based on game-theoretic energy consumption scheduling for the future smart grid.IEEE Transactions on Smart Grid,2010,1(3):320–331.

[85]I.Atzeni,L.G.Ordonez,G.Scutari,et al.Noncooperative day-ahead bidding strategies for demand-side expected cost minimization with real-time adjustments:A GNEP approach.IEEE transactions on Signal Processing,2014,62(9):2397–2412.

[86]S.Maharjan,Q.Zhu,Y.Zhang,et al.Dependable demand response management in the smart grid:A Stackelberg game approach.IEEE Transactions on Smart Grid,2013,4(1):120–132.

[87]A.J.Holmgren,E.Jenelius,J.Westin.Evaluating strategies for defending electric power networks against antagonistic attacks.IEEE Transactions on Power Systems,2007,22(1):76–84.

[88]G.Chen,Z.Dong,D.J.Hill,et al.Exploring reliable strategies for defending power systems against targeted attacks.IEEE Transactions on Power Systems,2011,26(3):1000–1009.

[89]L.Zhao,B.Zeng.Vulnerability analysis of power grids with line switching.IEEE Transactions on Power Systems,2013,28(3):2727–2736.

[90]M.P.Scaparra,R.L.Church.A bilevel mixed-integer program for critical infrastructure protection planning.Computers&Operations Research,2008,35(6):1905–1923.

[91]J.M.Arroyo.Bilevel programming applied to power system vulnerability analysis under multiple contingencies.IET Generation,Transmission&Distribution,2010,4(2):178–190.

[92]G.Brown,M.Carlyle,J.Salmeron,et al.Defending critical infrastructure.Interfaces,2006,36(6):530–544.

[93]Y.Yao,T.Edmunds,D.Papageorgiou,et al.Trilevel optimization in power network defense.IEEE Transactions on Systems,Man,and Cybernetics–Part C:Applications and Reviews,2007,37(4):712–718.

[94]W.Yuan,L.Zhao,B.Zeng.Optimal power grid protection through a defender-attacker-defender model.Reliability Engineering&System Safety,2014,121:83–89.

[95]N.Alguacil,A.Delgadillo,J.M.Arroyo.A trilevel programming approach for electric grid defense planning.Computers&Operations Research,2014,41:282–290.

[96]X.Zhou,S.Chen,Z.Lu.Review and prospect for power system development and related technologies:a concept of three-generation power systems.Proceedings of the CSEE,2013,36(22):2–11.

[97]S.Mei,X.Zhang,M.Cao.Power Grid Complexity.Berlin:Springer,2011.

9 December 2016;revised 16 December 2016;accepted 16 December 2016

DOI 10.1007/s11768-017-6186-y

?Corresponding author.

E-mail:wei-wei04@mails.tsinghua.edu.cn.

This work was supported by National Natural Science Foundation of China(No.51621065).

?2017 South China University of Technology,Academy of Mathematics and Systems Science,CAS,and Springer-Verlag Berlin Heidelberg

Shengwei MEIis currently a Professor with Tsinghua University,Beijing,China.His research interests include power system complexity and control,game theory and its application in power systems.E-mail:meishengwei@tsinghua.edu.cn.

Wei WEIis currently an Assistant Professor with Tsinghua University.His research interests include applied optimization,energy economics,and interdependent energy networks.E-mail:weiwei04@mails.tsinghua.edu.cn.

FengLIUis an Associate Professor with Tsinghua University.His research interests include power system distributed control and optimization.E-mail:lfeng@tsinghua.edu.cn.