Mehdi Firouznia and Qing Hui, Member, IEEE

Abstract—Motivated by the converse Lyapunov technique for investigating converse results of semistable switched systems in control theory, this paper utilizes a constructive induction method to identify a cost function for performance gauge of an average,multi-cue multi-choice (MCMC), cognitive decision making model over a switching time interval. It shows that such a constructive cost function can be evaluated through an abstract energy called Lyapunov function at initial conditions. Hence, the performance gauge problem for the average MCMC model becomes the issue of finding such a Lyapunov function, leading to a possible way for designing corresponding computational algorithms via iterative methods such as adaptive dynamic programming. In order to reach this goal, a series of technical results are presented for the construction of such a Lyapunov function and its mathematical properties are discussed in details. Finally, a major result of guaranteeing the existence of such a Lyapunov function is rigorously proved.

I. INTRODUCTION

FOR many critical infrastructure systems, such as power transmission networks, water distribution systems, and gas pipeline networks, human operators are naturally part of the system decision making process in which they have absolute authority to hold off the decision made by automation or control systems. The question of how the human operators’decision will affect the performance of such systems is the main challenge in designing functional, human-intelligent control systems [1]. Also the idea of exploring the possibility of optimizing human-in-the-loop decision making by mainly controlling the parts that may affect human decision performance sounds plausible with the ever increasing presence of artificial intelligence in daily life. For this purpose, the first logical step is to develop appropriate models for human decision process in the control system by integrating cognitive perspectives on human decision making.

The concept of temporal integration of evidence through sequential probability ratio test (SPRT) has been widely employed in decision-making modeling studies [2]. According to the drift diffusion model (DDM) to which SPRT converges in its continuum limit, decisions are made by accumulating noisy stimulus information until the decision variable reaches either positive or negative threshold in two-alternative forced choice (2AFC) tasks. The DDM has been proven to be successful in emulating the process of decision, and due to its connection to SPRT, it is optimal in a sense of maximizing any reward criterion that is monotonically decreasing with respect to decision time [3]. In other words, DDM renders the shortest possible decision time, given a specific accuracy.However, as pointed out by [4], this optimality description does not consider any cost associated with behavior, or a cost function for gathering information dynamically.

By increasing the number of decision choices and attributes in multi-choice multi-cue tasks (MCMC), race models with mutual inhibition were adopted to depict the decision process[3]. Although these models are intuitively plausible in describing the decision process, the notion of asymptotic optimality of SPRT [5] cannot be applied. Note that for the specific case of two-choice tasks, inhibition models can be reduced to DDM, and hence, renders the optimal solution under specific circumstances [6]. This leads to the question of how to address optimality for general MCMC tasks. Even before we talk about optimality, a cost associated with optimal performance needs to be defined and its evaluation needs to be tackled.

In this work we take a control-theoretic approach to tackle the performance evaluation of MCMC tasks modeled by mutual inhibition race pools, as discussed later in Section II.Our focus is to construct a performance gauge, which accounts for the performance of average MCMC and at the same time deals with the potential sources of deviation from optimality at the psychological level [7]. To do so, we first briefly review the mathematical abstract models in decision making in Section II and then formulate our proposed problem in Section III. In Section IV, some mathematical preliminaries of a novel method, called converse Lyapunov approach, are developed to prepare for construction of such a performance gauge later, which is based on converse Lyapunov results for switched and nonlinear semistable systems. As a major contribution of the paper, a performance gauge function is constructed in Section V. Finally, some conclusion about this work is drawn in Section VI.

II. REVIEW OF DECISION MAKING MODELS

In cognitive and behavioral sciences, models of describing the process of making decision in human brain start with analyzing the simple 2AFC decision task. Using the optimal SPRT test, the process on its continuum limit converges to the DDM if symmetric threshold is assumed. In this case, the decision variable for a noisy evidence can be modeled by onedimensional Wiener process bounded by positive and negative thresholds, θAand θB, in which an integrator accumulates the difference of evidences between two choices [3].

For 2AFC, only one cue is concerned. In real world situations, always several cues are involved. One method is to combine and integrate all cues in favor of each choice into single source of evidence and this source is being used throughout the decision process. More involved treatment includes separate processes for each cue. In this approach the order of considering the cues and the process time devoted to each cue are two important aspects. The time frame of the decision process is divided to subintervals with different lengths during which the attention focus is only one cue [8].

In order to model multi-choice tasks, a more general race model, which is comprised of separate leaky competing integrators, representing each choice, with mutual inhibition,was proposed in [9], where each integrator gathers information in favor of or against the associated choice based on the value of cue. Based on this idea, in our preliminary work [10], we have proposed the following leaky integrator race model to describe the dynamics of MCMC tasks. This model combines the race model and time and order scheduling concept as follows:

III. PROBLEM FORMULATION

In this paper, we consider the mean or average dynamics of(2) under the case where its initial condition xi(0) is random with a mean that may not be zero, meaning that the initial perception may be biased. Let zi(t)=E[xi(t)], where E denotes the expectation operator. Then the mean or first moment equation of (2) is given by

or in vector form

It has been shown in [3] that the DDM solves 2AFC problems optimally: It will on average return a decision in the shortest possible time for a specified level of accuracy.However, as pointed out by [4], this optimality description does not consider any cost associated with behavior, or a cost function for gathering information dynamically. Thus, in this paper, we will focus on the performance cost function that involves dynamic behavior of information gathering for the average MCMC model (4). The core question that this paper attempts to address is

Question 1: Does there exist a cost function associated with(4) to gauge its dynamic performance over finite or infinite horizon?

The answer to this question is imperative since it can serve as the first step toward finding optimal decision making strategies for MCMC tasks by evaluating their performance gauge. We will give a positive answer to this question by constructing such a performance cost function for average MCMC models. Before we show this main result, some mathematical preliminaries are needed in the next section.

IV. MATHEMATICAL PRELIMINARIES

In this section, we will present some mathematical preliminaries for a general switched linear system motivated by (5). Specifically, consider the following switched linear system:

V. PERFORMANCE GAUGE: MAIN RESULT

Fig. 1. State trajectories versus time.

VI. CONCLUSION

This paper utilized a converse Lyapunov approach to solve a longstanding problem regarding the performance gauge of average MCMC models for decision making. The developed result can be used to evaluate the cost for performance comparison of decision making under various inputs and initial conditions, and hence, lead to possible algorithmic ways of finding optimal performance for average MCMC decision making via a computational scheme. This idea seems tangible due to the extensive development of semidefinite programming methods [30] for searching appropriate Lyapunov functions within the past two decades, and hence,the proposed approach lays a theoretical foundation and suggests a possible path toward seeking optimal decision making in MCMC situations.

APPENDIX