張偉豪

(中國西南電子技術(shù)研究所，成都 610036)

1 Introduction

Weapon systems are essential for national defense.They are concerned by different countries’military more and more.What is weapon system?All weapons of the army;weapons used in some battle;even one naval vessel or combat aircraft.To research weapon equipment system，the first is to research its effectiveness.One method is researching system’s combat capability firstly，and then computing combat effectiveness based on input variable of combat capability［1］.Sometimes，combat capability and combat effectiveness are confused by more researchers，such as an author thinks equipment’s combat capability equals to combat effectiveness in its book［2］.Another author defines combat capability as the combination of combat effectiveness and command effectiveness［3］.The author of Reference［4］researches combat effectiveness’s evaluation methods and its supporting technology systematically.Combat capability is the“skill”of weapon equipment in systematology，which is a static concept and unrelated to combat process.But combat effectiveness is a dynamic concept and the effect of combat capability.

Combat capability is a broad concept and different weapons have different capabilities.But target identification capability is a public capability of all weapon systems.Many authors discuss index of target identification in their treatises［5－6］.He Jun and his team survey measures of ATR performance evaluation［6］，including confusion matrix，probability index，rate index and ROC index;however，they do not construct index frame.This paper mainly concerns capability index of weapon systems，discusses definition of target identification and its hierarchy nexus，and then constructs and analyzes target identification capability’s index framework.

2 Basic Concept of Target Identification

Target identification，the basic concept of which is some discrimination of target such as target attribute and its state，belongs and develops to mode identification.According to number of data sources，target identification is distinguished to single source identification and multisource identification.The former uses target signals acquired by sensors to process and identify target attribute or state.The latter combines multisource data and then identifies target’s identity(ID)based on prior knowledge.Commonly，weapon systems identify target’s attribute including model，type，friend－orfoe，military or civil use，nation and so on.Target’s state defines as target motion state specially.

3 Identification Capability Index Framework

Generally，in the combat process of weapon equipment system，target identification can be divided into three stages，which are target detection，target classification and target confirmation.Aiming at every stage，this paper analyzes its capability and constructs capability index，then synthesizes index of three stages to form target identification capability index framework of weapon systems.

(1)Target detection stage.Relying on field signals，images and other features acquired by sensors，weapon systems detect potential targets from environment noise.

(2)Target classification stage.Weapon systems extract target features(signal features，image features，or others)and match with features in the database.Sometimes，search strategy and matching algorithm are used to classify targets.The final purpose is to judge target’s type，category or friend－or－foe.

(3)Target confirmation stage.In this stage，weapon systems must extract target’s fine features，and then confirm target’s ID.

Figure 1 gives weapon equipment system’s target identification capability index frame，which includes target detection index，target classification index and target confirmation index.In addition，any weapon system must consume some resource to reach some kind of identification capability.Therefore，cost index of identification system is added to the index framework.

Fig.1 Target identification capability index frame圖1 目標(biāo)識別能力指標(biāo)體系

3.1 Target Detection Capability Index

Target Detection Capability Index includes confusion matrix，ROC curve，detection probability and false alarm probability.These indexes reflect identification capability of weapon systems from background noise from different aspects.

(1)Confusion Matrix

Confusion matrix is a widely used performance index，its basic form is Two Dimension Table built up by row and column which also can be a matrix.Confusion matrix describes classifier’s detection identification capability and reflects which target is falsely detected by classifier.But，Confusion Matrix can only give results in a particular decision threshold and can’t embody classifier’s effectiveness changes with different decision threshold.

Taking friend－or－foe detection for example，how to compute confusion matrix is introduced.Friend－orfoe confusion matrix A(k)3×4can be acquired by analyzing friend－or－foe detection results.Aij(k)means the confidence level of that target i(foe，friend，neutral)is identified to target j(foe，friend，neutral，unknown).Specially，when the detection result is hard decision，Aij(k)means the number of that target i is identified to target j.Then adding up confusion matrix A(k)(k=1，2，…，N)of every time，the overall matrix A is

(2)ROC Curve

ROC curve，firstly used in Radar target detection，was used to describe the relation between receiver’s detection probability and false alarm probability.Target detection capability based on ROC curve can embody identification algorithm’s performance working in several thresholds.For a simple binary－class，supposing PTPmeans probability of right identification and PFPmeans probability of false identification.PTPis different from PFPfor different threshold.It is not robust for directly using the two indexes to evaluate.ROC curve reveals there lation between PTP(θ)and PFP(θ)，when θ changes.The main methods to get ROC curve are parameter method or nonparametric method［7］.

ROC curve has many advantages.Firstly，ROC curve strikes an average between PTP(θ)and PFP(θ)when threshold changes，it directly shows algorithm’s total performance under multi－threshold.Secondly，due to PTP(θ)and PFP(θ)is independent of prior probability，curve’s shape doesn’t change with different target’sprior probability and only reflects algorithm’s performance.Lastly，ROC curve is irrelevant cost.But ROC curve can only describe two algorithm’s performance.

(3)Detection Probability and False Alarm Probability

These two belong to probabilistic index like identification rate and error rate.Probabilistic index defines occurrence probability of particular event in identification process.The following gives the two indexes’definition and statistic computing formula.

Detection Probability PD:probability of identification algorithm detecting target from all targets and nontargets

where，N means the number of all targets and nDmeans target number of detection.

False Alarm Probability PFA:probability of identification algorithm misjudging non－target to target

where，M means the number of non－targets and nFAmeans target number of mis judgement.

3.2 Target Classification Capability Index

Target classification is a process of hierarchical classing including target attribute classification，target category classification and target type classification.So，target classification capability index can divide into three classes:attribute identification rate，category identification rate and type identification rate.Every can define right identification rate，error identification rate and identification rejection rate.Taking target category identification for an example，defining indexes as follows.

(1)Category right rate PCr:probability of algorithm identifying target’s category of detection targets rightly

where，nCrmeans target number of right category identification.

(2)Category error rate PCe:probability of algorithm identifying target’s category of detection targets falsely

where，nCemeans target number of error category identification.

(3)Category rejection rate PCj:probability of target not being classed to any category

where，nCjmeans target number unable to class.

Therefore，target category classification comprehensive index PCcan be defined as

where，weight coefficients C1/C2/C3，which reflect different stressing by researchers，can be acquired according to actual situation.

For attribute classing and type classing，the same performance indexes can be defined.

Totally，target classification capability PCCindex can be defined:

where，PA，PC，PTmeans target attribute，category，type classification comprehensive index;CA，CC，CTare weight coefficients.

3.3 Target Confirmation Capability Index

Target confirmation capability is the performance of confirming target’s ID by weapon system.The capability index is personal recognition rate，including following indexes:

(1)ID recognition right rate PUc:probability of algorithm recognizing target’s ID in detection targets rightly

where，nUcmeans target number of right ID recognition.

(2)ID recognition error rate PUe:probability of algorithm recognizing target’s ID in detection targets rightly falsely

where，nUemeans target number of error ID recognition.

(3)ID rejection rate PUr:probability of target not being recognized to any ID

where，nUrmeans target number of unable to recognition.

Therefore，target personal recognition comprehensive index PUcan be defined as

where，weight coefficients C1，C2，C3can be acquired according to actual situation.

3.4 System Process Cost Index

Cost index is an important index of performance evaluation in the process of evaluating weapon systems’target identification comprehensive performance.Identification system’s costs are mainly time index and memory index.

Target identification time:defined as an average time from target entering into weapon system’s detection zone to target identification confidence reaching the given affirmance threshold.

Data sampling time:defined as demand data sampling time in process of system designing，training and testing.

Memory capacity:defined as computer memory used by identification work such as data sampling and data processing.

4 Simulation and Analysis

Using simulation platform，one scenario of air and sea battlefield is generated.Targets judging results are given in Table 1 based on statistic method.

Table 1 Targets judging results by statistics表1 目標(biāo)統(tǒng)計判決結(jié)果

Table 1 also can be regarded as Confusion Matrix.Using formula(2)and(3)，PDand PFAare given in Table 2.

Table 2 Calculation results of PDand PFA表2 檢測概率和虛警概率

Take target category classification calculating as an example，in Table 1 targets can be depart into two categories:sea targets and air targets.Then PCr，PCeand PCjare computed using formula(4)～(7)，and the results are PCr=71.9%，PCe=19.1%，PCj=9.0%.Supposing weight coefficients C1/C2/C3equal to 0.45/0.35/0.2，we can get the comprehensive index PC=0.41.The other target identification indexes PA，PTand PUcan be botained as the same way.

From the theory analysis and simulation，the index comparison is given as following.

Confusion Matrix is mainly used to record identification statistic results of weapon systems.This index has no question of computability because of the two－dimensional tables used to record.Using Confusion Matrix will be hard to differ two similar systems when system performance is close to each other.For weapon systems，the level of detection probability and false alarm probability impact Confusion Matrix.So，in real application，it is best to test system’s detection probability based on different false alarm probability，and test and record the confusion matrix.

ROC curve can reflect weapon system’s ability of discriminating targets from non－targets.This index can also be used to evaluate the overall performance better when the system wants to solve the conflict of high detection and low false alarm.But the index’s calculation and comparison are relatively complex，which limits its application.Therefore，this paper does not simulate ROC curve due to its little use.

Probabilistic index，such as PD，PFA，PCC，PUand so on，can be well calculated.We can get better comparison based on reasonable approximation model.The main disadvantage is the test set of large sample size while small size will reduce the index value and the reliability of the results.

5 Conclusions

From weapon system target identification’s generalized process to start，this paper firstly analyzes capability of identification stages，and then constructs a scientific，complete and relatively comprehensive target identification index framework.Every index’s definition and computation or statistic method are discussed.The framework enriches relative theory and method for evaluating combat capability and combat effectiveness of weapon equipment system.It can form the basis of expanding relative evaluation work.However，weapon equipment systems have many types and weapon system’s performance is also affected by combat environment.The next work is extracting more all－around identification capability indexes and researching index’s evaluation method.

［1］Bouthonnier V，Levis A.Effectiveness analysis of C3 system［J］.IEEE Transactions on Systems，Man，and Cyberneticcs，1984，14(1):50－55.

［2］ZHU Bao－liu，ZHU Rong－chang，XIONG Xiao－fei.Effectiveness evaluation of combat aircraft［M］.Beijing:A-viation Industry Press，1993.(in Chinese)朱寶鎏，朱榮昌，熊笑非.作戰(zhàn)飛機效能評估［M］.北京:航空工業(yè)出版社，1993.

［3］XIONG Shao－h(huán)ua，LIU Hong－bin.Study on the Estimation Indexes Set for Force Capability of Radar Information Network ［J］.Systems Engineering and Electronics，2002，22(3):94－96.(in Chinese)熊少華，劉洪彬.雷達情報網(wǎng)作戰(zhàn)能力評價指標(biāo)體系研究［J］.系統(tǒng)工程與電子技術(shù)，2002，22(3):94－96.

［4］HUANG Yan－yan.Research on Robust Evaluation Methodology of Weapon Equipment Operational Effectiveness and its supporting technology［D］.Changsha:National University of Defense Technology，2006.(in Chinese)黃炎焱.武器裝備作戰(zhàn)效能穩(wěn)健評估方法及其支撐技術(shù)研究［D］.長沙:國防科學(xué)技術(shù)大學(xué)，2006.

［5］MA Chun－ting，ZHENG Jian，CHEN Dong－gen，et al.The Evaluation Guideline of Multi－target Recognition Model of the Battlefield Reconnaissance［J］.Journal of Detection ＆ Control，2006，28(1):6－9.(in Chinese)馬春庭，鄭堅，陳東根，等.地面戰(zhàn)場偵察系統(tǒng)多目標(biāo)識別的評價指標(biāo)［J］.探測與控制學(xué)報，2006，28(1):6－9.

［6］HE Jun，ZHAO Hong－zhong，F(xiàn)U Qiang.A Survey of Measures on ATR Performance Evaluation［J］.Telecommunication Engineering，2007，47(5):32－37.(in Chinese)何峻，趙宏鐘，付強.自動目標(biāo)識別性能評估指標(biāo)簡述［J］.電訊技術(shù)，2007，47(5):32－37.

［7］Johnson A Y，Bobick A F.Relationship between identification metrics:expected confusion and area under a ROC curve［C］//Proceedings of 16th International Conference on Pattern Recognition.Alamitors.CA:IEEE，2002:662－666.