Amer I.Al-Omari,Amal S.Hassan,Heba F.Nagy,Ayed R.A.Al-Anziand Loai Alzoubi

1Faculty of Science,Department of Mathematics,Al al-Bayt University,Mafraq,25113,Jordan

2Faculty of Graduate Studies for Statistical Research,Cairo University,Giza,12613,Egypt

3Department of Mathematics,College of Science and Human Studies at Hotat Sudair,Majmaah University,Majmaah,11952,Saudia Arabia

Abstract:This paper deals with the Bayesian estimation of Shannon entropy for the generalized inverse exponential distribution.Assuming that the observed samples are taken from the upper record ranked set sampling(URRSS)and upper record values(URV)schemes.Formulas of Bayesian estimators are derived depending on a gamma prior distribution considering the squared error,linear exponential and precautionary loss functions,in addition,we obtain Bayesian credible intervals.The random-walk Metropolis-Hastings algorithm is handled to generate Markov chain Monte Carlo samples from the posterior distribution.Then,the behavior of the estimates is examined at various record values.The output of the study shows that the entropy Bayesian estimates under URRSS are more convenient than the other estimates under URV in the majority of the situations.Also, the entropy Bayesian estimates perform well as the number of records increases.The obtained results validate the usefulness and efficiency of the URV method.Real data is analyzed for more clarifying purposes which validate the theoretical results.

Keywords: Shannon entropy; generalized inverse exponential distribution;Bayesian estimators; loss function; ranked set sampling; markov chain

1 Introduction

Record values are crucial in many areas of real life applications comprising data relating to weather, sports, economics and life testing studies.Reference [1] constructed the theory of record values as a model for successive extremes in a sequence of independently and identically distributed (iid) random variables.Reference [2] mentioned that an observation is called upper(lower) record value if its value more (less than) that all of the preceding observations.Letxi,i≥1 be a sequence of iid random variables with a cumulative distribution function (CDF), sayF(x),and probability density function (PDF), sayf(x), an observationxiis called upper record value if its value exceeds all the preceding values, i.e.,xiis an URV ifxi ＞xj, wherei ＞j.

LetT1=t1,T2=t2,...,Tn=tnbe the firstnURV arising from any distribution with a certain PDF and CDF, the joint PDF of the first nURVis given by:

whereΘis the parameter space andθ∈Θmay be a vector.

Another record sampling scheme, known as upper record ranked set sampling, has been provided by [3].This scheme is valuable in some situations when the used observations are the last record data such as athletic, weather and Olympic data.The URRSS can be described as follows:

Considernindependent sequences of continuous random variables, theith sequence sampling is stopped when theith record value is noticed.The only observations that are handled by analysis are the last record values in each sequence.Let the last record value of theith sequence in this situation, sayTi,i, then the accessible observations areT=(T1,1,T2,2,...,Tn,n)T, that is

whereT(i)jis theith record in thejth cycle.LetTi,ibe a set of observed URRSS, then according to [3], the joint PDF ofTi,i, is given by

For information about ranked set sampling, see [4] for imputation of the missing observations using RSS and [5] for mean estimation based on modified robust extreme ranked set sampling.References [6–8] for partial, mixed and varied RSS methods, respectively.Reference [9] proposed a new RSS technique for mean and variance estimation, as well as [10] investigated the estimation of a symmetric distribution function in multistage ranked set sampling.

Some researchers have considered inference about different distributions based on records.For instance, Bayesian estimators and predictions for some life distributions from record values are discussed by [11].Stress-strength reliability estimator of exponentiated inverted Weibull distribution values has been discussed by [12] based on lower record.Reference [13] considered Bayesian and non-Bayesian estimators from power Lomax distribution using URV.Estimation of the twoparameter bathtub-shaped distribution is discussed by [14] from record data.Bayesian estimators of the generalized inverse exponential (GIE) distribution are discussed by [15] via URV.Stress strength reliability estimator for independent GIE distributions using URRSS is handled by [16].Reference [17] discussed estimation and prediction for Nadarajah-Haghighi distribution based on record.Statistical inference for the power Lindley model is studied by [18] from record values and inter-record times.Reference [19] handled reliability estimator for Weibull distribution for multicomponent system based on URV.

Reference [20] introduced the concept of entropy as a measure of information, which provides a quantitative measure of the uncertainty.It is also considered as a measure of randomness of a probabilistic system.LetXbe a non-negative random variable with cumulative distribution functionF(x) and probability density functionf(x).The Shannon entropy, denoted bySH(X), of the random variable is defined by

It is seen that a very sharply peaked distribution has very low entropy, whereas if the probability is spread out, the entropy is much higher.In this sense,SH(X) is a measure of uncertainty associated withf(x).Entropy estimation for some life distributions has been discussed by many authors.For example, [21] obtained an entropy estimator using URV from the generalized halflogistic distribution.References [22,23] suggested some entropy estimators based on RSS and double RSS methods, respectively.Reference [24] investigated entropy estimation and goodnessof-fit tests for the Laplace and inverse Gaussian distributions based on pair RSS.Reference [25]discussed the entropy Bayesian estimators of Weibull distribution based on generalized progressive hybrid censoring scheme.Reference [26] proposed new measures of entropy and [27] discussed the entropy maximum likelihood and Bayesian estimators of inverse Weibull distribution under generalized progressive hybrid censoring scheme.Reference [28] provided an exact expression for entropy information contained in both types of progressively hybrid censored data and applied it in exponential distribution.Reference [29] discussed the estimation of entropy for generalized exponential distribution via record values.Reference [30] discussed entropy estimators of a continuous random variable using RSS.Reference [31] obtained the maximum likelihood estimator of Shannon entropy for inverse Weibull distribution under multiple censored data and [32] proposed entropy Bayesian estimators of Lomax distribution using record data, and [33] considered extropy properties of RSS.

To our knowledge, in the literature, there are no studies that had been performed about entropy estimation in view of URRSS.So, our interest in this study is estimating the Shannon entropy of the GIE distribution using Bayesian approach from URRSS and URV.The Shannon entropy Bayesian estimator is considered using gamma priors.The Bayesian estimator of entropy is induced related to symmetric and asymmetric loss functions.The proposed loss functions are squared error loss function (SELF), linear exponential loss function (LINEX) and precautionary loss function (PRLF).Bayesian entropy estimators under symmetric and asymmetric loss functions have complicated expressions, so we implemented the Markov Chain Monte Carlo (MCMC) technique.

The following sections are organized as follows.Formula of Shannon entropy for GIE distribution is provided in Section 2.Entropy Bayesian estimator is derived using URRSS from symmetric and asymmetric loss functions in Section 3.Based on URV, entropy Bayesian estimator for GIE distribution is discussed using the proposed loss functions in Section 4.Simulation issue and application to real data are given in Sections 5 and 6, respectively.The paper ends with some concluding remarks in Section 7.

2 Expression of Shannon Entropy

The two-parameter GIE distribution is provided by [34] which has many applications in various areas such as, accelerated life testing, queues, horse racing, sea currents and wind speeds.The PDF of the GIE model with the shape parameterθand scale parameterβis given by

The CDF of the GIE distribution is given by

LetXbe a random variable follows a GIE distribution with PDF given in (4), hence the Shannon entropy ofXis obtained by substituting (4) in (3) as follows:

where

whereγ= 0.577 is Euler constant.To obtainI2, lety= 1 -e-β/x, thenI2=θ(θ- 1)

Also,I3is obtained as follows

SubstitutingI1,I2, andI3in (6), we obtain the Shannon entropy for GIE distribution as follows:

which is a function of the parametersθandβ.

3 Entropy Bayesian Estimation Based on URRSS

In this section, Bayesian estimator of the Shannon entropy for the GIE model is discussed in view of URRSS.Firstly, the Bayesian estimators of parameters must be computed in order to get the entropy Bayesian estimator.Then, entropy Bayesian estimator is obtained using (7) according to the invariance property.The Bayesian estimator based on gamma priors is considered.Three Bayesian estimators are obtained according to SELF, LINEX and PRLF.Furthermore, the Bayesian credible intervals are constructed.

Let=(t1,1,t2,2,...,tn,n)be a set of observed URRSS from GIE distribution, then the likelihood function denoted byL1, is obtained by inserting PDF in (4) and CDF in (5) in (2), as follows

Assuming that the prior of parametersθandβhas a gamma distribution with parameters(a,b)and(c,d), respectively.Hence, the joint prior distribution of parameters, denoted byπ(θ,β),assuming independence of parameters is as follows

The joint posterior under the assumption thatβandθare independent gamma priors is

Hence, the marginal posterior distributions ofβandθare given by

where

Therefore, the Bayesian estimators ofβandθunder SELF, denoted byand, depending on URRSS are obtained as the posterior mean as follows:

The Bayesian estimators ofβandθunder LINEX, denoted byandare given by

and

whereδis a real number.Additionally, the Bayesian estimators ofβandθunder PRLF, sayandare given as follows

and

The integrals (9)–(13) are very hard to be solved analytically according to their convoluted forms.Therefore, we employ the MCMC technique to approximate these integrations.The Bayesian estimates together with credible intervals width under SELF, LINEX and PRLF loss functions are implemented using Metropolis-Hastings (M-H) algorithm.Therefore, the Bayes estimate ofSH(X), denoted byS?H1(x)under SELF is obtained as follows

Consequently, the Bayesian estimator ofSH(X) under LINEX and PRLF are obtained by similar way after setting their estimators in (7).Additionally, we get the Bayesian credible interval of entropy using the same algorithm proposed by [35].

4 Entropy Bayesian Estimation Based on URV

This section provides the Bayesian estimators ofθandβfor the GIE distribution based on URV.The Bayesian estimators are obtained assuming that the gamma priors are independent using SELF, LINEX and PRLF.Let1,t2,...,tn)benobserved URV from GIE distribution with PDF in (4) and CDF in (5), then the likelihood function, sayL2, of the GIE distribution is obtained by inserting (4) and (5) in (1), as follows:

Assuming that the prior ofθandβhas a gamma distribution with parameters(a,b)and(c,d), respectively.Hence, the joint prior distribution of parameters, assuming independence is considered as provided in (8).Therefore, the joint posterior can be expressed as follows:

Consequently, expressions for the marginal posterior distributions ofθandβare as follows:

where

Hence, Bayesian estimators ofθandβ, under SELF, sayand, can be obtained as posterior mean as follows:

Also, under LINEX, the Bayesian estimators ofθandβ, sayand, are obtained as follows:

Furthermore, considering PRLF, the Bayesian estimators ofθandβ, sayandare given as follows:

Again, the MCMC procedure is provided to approximate the integrals (14)–(16) based on M-H algorithm to compute the estimates and credible interval width considering symmetric and asymmetric loss functions.

Regarding to Eq.(7), the Bayesian estimator ofSH(x), denoted byunder SELF is obtained as follows

By similar way, the Bayesian estimator ofSH(X) under LINEX and PRLF are obtained after setting their estimators in (7).Furthermore, the Bayesian credible interval is obtained as mentioned in the Section 3.

5 Simulation Study

In this section, a simulation investigation is carried out to compare the performance of the entropy estimate of the GIE distribution based on URV and URRSS.The relative absolute bias(RAB), estimated risk (ER) and width (WD) of credible intervals for the Shannon entropy based on URV and URRSS for GIE distribution are used to evaluate the behaviour of the Bayesian estimates.In the simulation setup, the number of records are selected asn=4,5,6,7.The values of parameters are selected as(θ,β)=(4,2),(2,2)and (0.5, 2), where the associated true values of entropy areSH(x) = 0.8452, 1.3584 and 3.2896, respectively.The hyper-parameters for gamma prior are selected asa=b=2 andc=d=2.Also, we takeδ=-2, 2 for LINEX loss function.M-H algorithm will be used via R 3.1.2 program.

The M-H algorithm procedure is described as follows:

Letg(.) be the density of subject distribution.

Initialize a starting valuex0and the number of samplesN

fori=2 toN

setx=xi-1

generateufrom U(0, 1)

generateyfromg(.)

Tabs.1–3 summarize the Bayes estimates and their measures (RAB, ER and WD) based on URV and URRSS.From the numerical outcomes given in Tabs.1–3 and Figs.1–6, we can conclude the following:

Table 1:Entropy estimates, RAB, ER and WD based on URV and URRSS at (θ,β)=(4,2) and SH(x) = 0.8452

Table 2:Entropy estimates, RAB, ER and WD based on URV and URRSS at (θ,β)=(2,2) and SH(x) = 1.3584

? The ER of entropy estimates under SELF and LINEX based on URRSS is smaller than that of the corresponding under URV atn=6 for all values of(θ,β)(see Figs.1 and 2).

Figure 1:ER of entropy estimate based on URV and URRSS at SELF and n=6 for all values of (θ,β)

Figure 2:ER of entropy estimate based on URV and URRSS at n=6, and LINEX (δ=2) for all values of (θ,β)

? The ER of entropy estimates under LINEX (δ=2) under URRSS is smaller than that of the corresponding under URV atn=7 for all values of(θ,β)(see Fig.3).

Figure 3:ER of entropy estimate based on URV and URRSS at n=7 and LINEX (δ=2) for all values of (θ,β)

? The ER of entropy estimates based on URSS is smaller than the corresponding under URV atn=5, and(θ,β)=(0.5, 2) for different loss functions (see Fig.4).

Figure 4:ER of entropy estimate based on URV and URRSS at n = 5, SH(x) = 3.289 and different loss functions

Table 3:Entropy estimates, RAB, ER and WD based on URV and URRSS at (θ,β)=(0.5,2)and SH(x) = 3.2896

? The ER of entropy estimates based on URRSS is smaller than the corresponding under URV atn=7,(θ,β)=(2,2)for the proposed loss functions except LINEX (δ=-2) (see Fig.5).

Figure 5:ER of entropy estimate based on URV and URRSS at n = 7, SH(x) = 1.358 and different loss functions

? The WD of entropy estimates based on URV is smaller than the corresponding under URRSS atn=4 under PRLF for all values of(θ,β)(see Fig.6).

Figure 6:WD of entropy estimate based on URV and URRSS at n=4 and PRLF for all values of (θ,β)

? In general, asnincreases, the ER, RAB and WD of estimate decrease for both record schemes.

? As the true valueSH(x) increases, the ER increases in most of the situations.

6 Application to Real Data

In this section, a real data set is analysed for illustrative purposes.The suggested data represent the lifetimes of steel specimens tested at different stress levels (for more details see [36].Some preliminary data analysis is performed.The Kolmogorov-Smirnov (K-S) test is used for the data set to the fitted model.It is observed that the K-S distance are 0.083 with the corresponding P-value 0.917.It indicates that the GIE model provides reasonable fit to this data set.Also, the estimated PDF, CDF and PP plots for data are represented in Fig.7.From these figures it can be concluded that the GIE distribution is an adequate model to fit these data.

Figure 7:Estimated PDF, CDF and PP plots of the GIE distribution for lifetimes of steel specimens data

The extracted records from a part of this data are presented as

?

Based on the above record data, it can be shown that URRSS of sizen= 4 is(t1,1,...,t4,4)=(60, 128, 194, 394) and the URV of sizen= 4 is(t1,...,t4)=(60, 83, 140, 186).Considering this record data, the entropy Bayes estimator atn= 4 under SELF, LINEX and PRLF are obtained and listed in Tab.4.

From Tab.4, we can conclude that ER of entropy estimates under URRSS gets the smallest values compared to the corresponding under URV in case of PRLF and LINEX(δ=-2)at true valueSH(X) = 3.2896.While at true valueSH(X) = 1.3584, it is noted that the ER of entropy estimates under URRSS is smaller than the corresponding under URV in case of LINEX(δ=2)and SELF.Furthermore, one can conclude that the ER of entropy estimates under URRSS are smaller than the corresponding counterparts URV at LINEX(δ=-2)for true valueSH(X)=0.8452.

Table 4:Entropy estimates and their RAB, ER and WD of steel specimens at different stress levels data based on URV and URRSS

7 Summary and Conclusion

This paper provides Bayesian estimation of the Shannon entropy for the generalized inverse exponential distribution using URRSS and URV shemes.The entropy Bayesian estimators are considered using gamma prior functions for symmetric (SELF) and asymmetric (LINEX and PRLF) loss functions.In order to obtain the Bayesian estimators, we employed Markov Chain Monte Carlo method based on Metropolis-Hastings algorithm.The performance of the entropy estimates for the GIE distribution is investigated in terms of their relative absolute bias, estimated risk and the width of credible intervals.From simulation results, it turns out that, the entropy Bayesian estimator approaches the true value as the number of record increases.Generally, the entropy and ERs are directly proportional, that is; if the real value of entropy increases, the ERs increase.The WD of Bayes credible intervals for estimated values of entropy URRSS is smaller than the corresponding estimated values based on URV for all loss functions for most values of record values in the majority of the cases.A data real example has been considered to illustrate the applicability of the proposed methodology for the considered record schemes.

Acknowledgement:The authors are grateful to the Editor and anonymous reviewers for their valuable comments and suggestions.

Funding Statement:A.R.A.Alanzi would like to thank the Deanship of Scientific Research at Majmaah University for financial support and encouragement.

Conflicts of Interest:The authors declare that they have no conflicts of interest to report regarding the present study.