Paul Swamidhas Sudhakar Russell,Priya Mary Mammen,Satya Raj Shankar,Shonima Aynipully Viswanathan,Grace Rebekah,Sushila Russell,Richa Earnest,Swetha Madhuri Chikkala

Paul Swamidhas Sudhakar Russell,Priya Mary Mammen,Satya Raj Shankar,Shonima Aynipully Viswanathan,Sushila Russell,Richa Earnest,Swetha Madhuri Chikkala,Child and Adolescent Psychiatry Unit,Christian Medical College,Vellore 632 002,Tamil Nadu,India

Grace Rebekah,Department of Biostatistics,Christian Medical College,Vellore 632 002,Tamil Nadu,India

Abstract BACKGROUND Emergence delirium (EmD) is a troublesome motoric,emotional,and cognitive disturbance associated with morbidity.It is often misdiagnosed despite being present in a substantial proportion of children and adolescents during emergence from anesthesia.AIM To evaluate the summary diagnostic accuracy of Pediatric Anesthesia Emergence Delirium Scale (PAEDS) for EmD among children and adolescents.METHODS Two researchers electronically and hand searched the published literature from May 2004 to February 2021 that evaluated the diagnostic accuracy of PAEDS for EmD among children and adolescents,using appropriate terms.Two independent researchers extracted the diagnostic parameters and appraised the study quality with QUADAS-2.Overall,the diagnostic accuracy of the measures was calculated with the summary receiver operating characteristic curve (SROC),the summary sensitivity and specificity,and diagnostic odds ratio (DOR) for EmD.Various diagnostic cut-off points were evaluated for their diagnostic accuracy.Heterogeneity was analyzed by meta-regression.RESULTS Nine diagnostic accuracy studies of EmD that conformed to our selection criteria and PRISMA guidelines were included in the final analysis.There was no publication bias.The area under the SROC was 0.97 (95% confidence interval [CI]:95%-98%).Summary sensitivity and specificity were 0.91 (95%CI:0.81-0.96;I2=92 .93%) and 0.94 (95%CI:0.89-0.97;I2=87.44%),respectively.The summary DOR was 148.33(95%CI:48.32-455.32).The effect size for the subgroup analysis of PAEDS cut-off scores of＜10,≥10,and≥12 was 3.73,2.19,and 2.93,respectively;they were not statistically significantly different.The setting of the study and reference standard were statistically significantly related to the sensitivity of PAEDS but not specificity.CONCLUSION The PAEDS is an accurate diagnostic measure for the diagnosis of EmD among children and adolescents.Further studies should document its clinical utility.

Key Words:Anesthesia;Children;Emergence delirium;Diagnostic accuracy;Measure;Meta-analysis

lNTRODUCTlON

Emergence delirium (EmD) is seen in up to 80% of children and adolescents in post anesthesia care units[1,2].This troublesome motoric,mental,and cognitive disturbance is often missed or misdiagnosed[3].It can last from under 0.5 h to 2 d,and potentially can result in significant morbidity including transient neurological deficits[1,4],longer hospital stays,and regression of milestones if not identified early in its presentation[5].Fortunately,the use of psychometrically validated measures improves the early diagnosis and effective treatment of delirium in intensive care settings[6].However,despite the existence of more than 20 measures for EmD,many of them have not been validated[7].Among the validated and widely used measures for EmD are the WATCHA Scale,Cravero Scale,and Pediatric Anesthesia Emergence Delirium Scale (PAEDS)[7];the latter scale has been recommended for use in the identification of EmD among children and adolescents[3,8].Nonetheless,the diagnostic accuracy parameters of PAEDS in individual studies have ranged widely from a sensitivity of 64%-100% and specificity of 80%-98%[7,9].These wide ranges of results warrant the analysis of the pooled diagnostic accuracy data of PAEDS for EmD.Hence,we conducted this meta-analysis of published data to evaluate the pooled global diagnostic accuracy of PAEDS,its specific diagnostic accuracy parameters of pooled sensitivity and specificity,the diagnostic accuracy of various PAEDS total cut-off points,and the effect of the setting of the use of PAEDS,sample size,age of the juveniles,and the reference standard on the effect size of sensitivity and specificity by meta-regression.

MATERlALS AND METHODS

Literature search

Two researchers (RE and SMC),independently and electronically,searched for the diagnostic accuracy studies of PAEDS in English in the Scopus,PubMed,and Cochrane Data published between May 2004(from the time of development of PAEDS and publication of its first validation study) to February 2021(date of last literature update for final analysis).The term “Pediatric Anesthesia Emergence Delirium Scale” was combined with “diagnostic accuracy” and “validation” as ("pediatrics"[All Fields] OR"pediatrics"[MeSH Terms] OR "pediatrics"[All Fields] OR "pediatric"[All Fields] OR "pediatric"[All Fields]) AND ("emergence delirium"[MeSH Terms] OR ("emergence"[All Fields] AND "delirium"[All Fields]) OR "emergence delirium"[All Fields]) AND ("scale s"[All Fields] OR "scaled"[All Fields] OR"scaling"[All Fields] OR "scalings"[All Fields] OR "weights and measures"[MeSH Terms] OR("weights"[All Fields] AND "measures"[All Fields]) OR "weights and measures"[All Fields] OR"scale"[All Fields] OR "scales"[All Fields]) AND ("diagnosis"[MeSH Terms] OR "diagnosis"[All Fields]OR "diagnostic"[All Fields] OR "diagnostical"[All Fields] OR "diagnostically"[All Fields] OR"diagnostics"[All Fields]) AND ("accuracies"[All Fields] OR "accuracy"[All Fields]);and("paediatrics"[All Fields] OR "pediatrics"[MeSH Terms] OR "pediatrics"[All Fields] OR "paediatric"[All Fields] OR "pediatric"[All Fields]) AND ("emergence delirium"[MeSH Terms] OR ("emergence"[All Fields] AND "delirium"[All Fields]) OR "emergence delirium"[All Fields]) AND ("scale s"[All Fields] OR"scaled"[All Fields] OR "scaling"[All Fields] OR "scalings"[All Fields] OR "weights and measures"[MeSH Terms] OR ("weights"[All Fields] AND "measures"[All Fields]) OR "weights and measures"[All Fields]OR "scale"[All Fields] OR "scales"[All Fields]) AND ("valid"[All Fields] OR "validate"[All Fields] OR"validated"[All Fields] OR "validates"[All Fields] OR "validating"[All Fields] OR "validation"[All Fields]OR "validational"[All Fields] OR "validations"[All Fields] OR "validator"[All Fields] OR "validators"[All Fields] OR "validities"[All Fields] OR "validity"[All Fields]).

The electronic search did not incorporate any search filter to improve the retrieval of as many articles as possible.After a review of the identified titles and abstracts,those articles deemed potentially relevant were collected.We augmented our electronic search with a hand search for additional relevant articles in reference lists of collected articles and from conference abstracts.

Study selection,data extraction,and quality appraisal

Two other researchers (Mammen PM and Shankar SR) extracted the required details independently,resolved any difference in extraction by consultation with another researcher (PSSR),and entered the information as electronic data.They extracted the information including participants,index measure,comparative reference measure,and outcome of diagnostic accuracy details.To be included in the final meta-analysis,studies had to compare the ability of PAEDS as the index test and DSM IV/DSM-IVTR/DSM 5/ICD-10 or clinical consensus/clinical observation as the reference standard (using clinical interview,semi-structured interview,or interviewing schedules) among children and adolescents (1-18 years).Those diagnostic accuracy studies of PAEDS to identify EmD only were included and studies on PAEDS in the context of other emergent conditions like emergent agitation and emergent pain were excluded.Finally,the study had to report sufficient data to construct 2 x 2 tables for calculating the true positive,false positive,false negative,and true negative values.Two researchers (SR and SAV)appraised the quality of the studies with Quality Assessment of Diagnostic-Accuracy Studies,version 2(QUADAS-2);differences in appraisal were resolved by consensus with the third researcher (Russell PSS).

Statistical analysis

We constructed the true positive,false positive,false negative,and true negative values,for each included study using 2 × 2 tables.We calculated the area under the curve (AUC) using the summary receiver operating characteristic curve (SROC) to establish the global diagnostic accuracy for all PAEDS cut-offs together;we calculated the confidence and prediction contour for the SROC as well[10].The pooled sensitivity and specificity were estimated.We calculated the pooled diagnostic odds ratio (DOR)as the diagnostic accuracy parameter for various PAEDS cut-off scores and presented it as a forest plot.AnI2value of＞50 was considered as substantial heterogeneity.For exploring the heterogeneity and subgroup analysis,the effect of the setting of the use of PAEDS,sample size,reference standard,as well as age of children and adolescents (as independent variables) on the effect size of sensitivity and specificity (as dependent variables) was done using univariate meta-regression.In addition,as the heterogeneity was substantial,it was reasoned that the summary statistics might not represent the individual studies adequately.Therefore,as apost hoctest to parametrise the summary DOR,we conducted a leave-one-out cross validation.We calculated the 95% confidence interval (95%CI) when indicated.The analyses were done with the METANDI module of STATA (version 16).We conducted the leave-one-out cross validation using the software Open-Meta meta-analysis software (Brown University,Providence RI,United States)[11].

RESULTS

Literature search

Totally we identified 232 studies from all the data bases,and nine studies (K=9;n=1251) were included for the final meta-analysis[7,9,12-17].Two studies were excluded as they did not satisfy the selection criteria[18,19].Augmentation strategies of checking the cross references and conference abstracts did not supplement to the eligible article list.The PRISMA flowchart of studies for the final meta-analysis is represented in Figure 1.

Figure 1 PRlSMA flow chart of studies included in the diagnostic meta-analysis for Pediatric Anesthesia Emergence Delirium Scale.

The studies were conducted either in the out-patient (K=2) or in-patient settings (K=7) and the sample size varied from 90-260 participants.Four studies had children as participants and the remaining five had children as well as adolescents.Six studies had used a PAEDS cut-off of＜10,two studies≥10,and two studies≥12 for the diagnosis of EmD;except two studies,all had used clinical observation by trained professionals in identifying EmD as the reference standard (Table 1).

Table 1 Data on methodology and epidemiology of included studies

Publication bias and quality appraisal

The quality appraisal using QUADAS-2 is pictorially represented for individual studies and across studies in Figure 2A and 2B,respectively;the most common bias across studies was documenting the reference standards and applicability of the reference standards.The Deek’s plot did not show publication bias [coefficient=39.10 (95%CI:-6.05-84.25);P=0.08] for the studies included in the final analysis as noted in Figure 3.

Figure 2 Quality appraisal using the revised diagnostic accuracy studies (quality assessment of diagnostic accuracy studies-2) for individual studies (A) and average quality across studies (B).

Figure 3 Deek’s plot for publication bias among studies included in the diagnostic meta-analysis for Pediatric Anesthesia Emergence Delirium Scale.

Diagnostic accuracy

The AUC for the HSROC was 0.97 (95%CI:95%-98%) (Figure 4).The summary sensitivity and specificity(95%CI for sensitivity/specificity;I2for heterogeneity) for the PAEDS were 0.91 (95%CI:0.81-0.96;I2=92.93%) and 0.94 (95%CI:0.89-0.97;I2=87.44%),respectively,for diagnosing EmD.When we analyzed the sensitivity-specificity pair within studies,most of the studies had a higher specificity than sensitivity[8,12,15,16,18].However,two studies each had a higher sensitivity than specificity[9,17] or equal sensitivity and specificity[13,14].

Figure 4 Diagnostic accuracy of the Pediatric Anesthesia Emergence Delirium Scale based on the summary receiver operating characteristic curve.

The summary DOR for all PAEDS cut-off scores together was 148.33 (95%CI:48.32-455.32).With the leave-one-out cross validation,the individual studies significantly contributed to the summary DOR in a descending order from the study by Sikichet al[8] at the top [DOR=152.23 (95%CI:76.23-304.82)],followed by Bajwaet al[13] [DOR=148.48 (95%CI:82.18-268.27)],Bonget al[12] [DOR=134.04 (95%CI:66.53-270.02)],Somainiet al[17] [DOR=133.30 (95%CI:66.95-265.41)],Janssenet al[14] [DOR=131.35(95%CI:64.70-266.64)],Locatelliet al[15] [DOR=121.36 (95%CI:59.72-249.32)],Simonsenet al[18] [DOR=117 (95%CI:76.23-304.82)],Jooet al[16] [DOR=111.78 (95%CI:62.25-200.73)],and finally Blankespooret al[9] [DOR=111.72 (95%CI:63.47-196.65)].

The effect size for the subgroup analysis of PAEDS cut-off scores of＜10,≥10 and≥12 was 3.73,2.19,and 2.93 respectively.Although the＜10 PEDS cut-off score had the largest effect size,the three studied cut-off scores were not statistically significantly different in their diagnostic accuracy;however,they were statistically significantly different when individual studies with varying cut-off PAEDS scores were studied (Figure 5).

Figure 5 Forest plot for the diagnostic odds ratio presenting the subgroup analysis by cut-off scores and individual studies included in the diagnostic meta-analysis for Pediatric Anesthesia Emergence Delirium Scale.

Meta-regression

In the meta-regression,the setting of the study and reference standard used were statistically significantly related to the sensitivity of PAEDS and not to its specificity,but the age of the children and adolescents and the sample size of the studies were neither related to the sensitivity nor specificity(Figure 6).

Figure 6 Meta-regression and subgroup analysis on sensitivity and specificity of Pediatric Anesthesia Emergence Delirium Scale.

DlSCUSSlON

Currently,the diagnostic methods for EmD are evolving,and there is more clarity in differentiating EmD from other emergent phenomena.This meta-analysis included only those studies where PAEDS was used as a diagnostic measure for EmD only.This meta-analysis on PAEDS supports the evidence obtained from previously documented diagnostic accuracy parameters based on individual studies that the measure can be used as an effective diagnostic measure for EmD among children and adolescents.

There was no publication bias.The quality appraisal showed that the most common bias across studies was documenting the reference standards and applicability of the reference standards.Overall,the studies were of moderate quality.The absence of very large studies,duplicated data sets,same study sample/population,and similar selection process of participants or same group of authors with similar interpretation of results has minimized the skewing of our summary findings.

The AUC-SROC for PAEDS in diagnosing EmD was 0.97.As this AUC is much above the random predictor value of 0.5,the classification of EmD by PAEDS is not by random chance of 50% or toss of a coin but instead the classification is because of the excellent inherent global diagnostic accuracy of PAEDS.Thus,PAEDS succeeds as a diagnostic test for pediatric EmD with the various diagnostic cutoff scores used currently.

The pooled sensitivity of PAEDS in our study was 91%,which is an excellent sensitivity meaning that 91/100 children with EmD were correctly identified.Similarly,the pooled specificity of PAEDS was 94%,which is an excellent specificity and it means that 94/100 healthy children were identified as not having EmD.Such excellent sensitivity and specificity again support the use of PAEDS as a diagnostic measure for EmD among the pediatric population.This pooled sensitivity and specificity are comparable with the data documented in individual diagnostic accuracy studies of PEDS[9,11,14-16].

The overall DOR calculated from sensitivity and specificity was 148.In theory,the DOR ranges in value from zero to infinity,with higher values indicating better discriminatory performance of the test.This binary classification is not dependent on the prevalence of EmD and hence can be applied in various pre-test probability contexts[20].When the subgroup analysis of the DOR based on the PEDS cut-off scores was performed,although the lowest of the threshold scores more accurately diagnosed EmD,there was no statistically significant difference among them.However,when a range of cut-off scores,from＞8 to＞16,were used,the lowest score showed a statistically significant diagnostic accuracy than higher scores[9].This speculatively could be because in higher PAEDS scores,the motoric combined with cognitive items possibly identify the symptoms of emergence agitation and emergence pain as well[20,21];this hypothesis has to be further tested.

However,some of the above findings should be interpreted in the context of the study limitations and strengths.There was substantial heterogeneity in the diagnostic accuracy parameters of the PAEDS,which was partly explained by the setting of the occurrence of EmD and the reference standard used.The role of each individual study in the summary DOR was further explored with a range of 111-152,adding strength to the method of this meta-analysis.The PAEDS threshold effect has to be further studied with larger meta-analysis.Expecting heterogeneity to start with,the use of random effects models,exploring the heterogeneity by meta-regression,subgroup analysis,and the leave-one-out cross validation have strengthened the meta-analysis.Furthermore,in order not to compromise the diagnostic accuracy of PAEDS for EmD from other post-anesthetic emergent problems like pain and agitation,we excluded those studies with such conditions in this meta-analysis.

From a clinical-utility perspective,PAEDS has the global and specific diagnostic accuracy characteristics to be used as a diagnostic measure for EmD among both children and adolescents.It has documented that integrated use of PAEDS in post-anesthesia care unit improves the identification of ED better than other measures[7];our study encourages the integration of this measure for the diagnosis of ED.

CONCLUSlON

In conclusion,PAEDS has excellent diagnostic accuracy for emergent delirium among children and adolescents.

ARTlCLE HlGHLlGHTS

Research background

There are various measures to identify emergence delirium (EmD) among children and adolescents as they recover from anesthesia.Pediatric Anesthesia Emergence Delirium Scale (PAEDS) is one such measure and has been found to have varying accuracy for diagnosing EmD.

Research motivation

The diagnosis of EmD is often missed or misdiagnosed.This can result in significant morbidity.The widely used PAEDS across the world has been proven to have the ability of early identification of EmD.

Research objectives

The aims of this meta-analysis were to document the summary global and specific diagnostic accuracy parameters of PAEDS,diagnostic accuracy for various diagnostic threshold scores of the measure,and factors associated with these summary parameters of PAEDS in diagnosing EmD.

Research methods

Nine studies were included in the analysis following the PRISMA guidelines.We used the summary area under the receiver operating characteristic curve,with a random effects model,to summarize the global diagnostic accuracy of PAEDS along with its diagnostic odds ratio,sensitivity,and specificity.

Research results

The area under the SROC was 0.97 (95%CI:95-98%).The summary sensitivity and specificity were 0.91(95%CI:0.81-0.96;I2=92.93%) and 0.94 (95%CI:0.89-0.97;I2=87.44%),respectively.The summary DOR was 148.33 (95%CI:48.32-455.32).The effect size for the subgroup analysis of PAEDS cut-off scores of＜10,≥10,and≥12 was 3.73,2.19,and 2.93,respectively;they were not statistically significantly different.The setting of the study and reference standard were statistically significantly related to the sensitivity of PAEDS but not specificity.

Research conclusions

The authors have established the summary global diagnostic accuracy of PAEDS for EmD among children and adolescents.

Research perspectives

The PAEDS could be used for diagnosing EmD among children and adolescents.The specific diagnostic cut-off scores have to be further studied.

ACKNOWLEDGEMENTS

We acknowledge with gratitude Ms.Mary Pauline Paul and Mr.George Devadoss in coordinating the certification processes.

FOOTNOTES

Author contributions:Russell PSS and Mammen PM conceived and designed the study;Chikkala SM and Earnest R did the literature search and collected the data;Mammen PM and Shankar SR extracted the data;Viswanathan SA and Russell S appraised the quality of the studies;Mammen PM resolved the conflicts in data extraction and quality appraisal;Russell PSS and Rebekah G did the statistical analyses;all authors contributed to the writing and approval of the final manuscript.

Conflict-of-interest statement:All authors declare that there are no any conflicts of interest to disclose.

PRlSMA 2009 Checklist statement:The authors have read the PRISMA 2009 Checklist,and the manuscript was prepared and revised according to the PRISMA 2009 Checklist.

Open-Access:This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers.It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BYNC 4.0) license,which permits others to distribute,remix,adapt,build upon this work non-commercially,and license their derivative works on different terms,provided the original work is properly cited and the use is noncommercial.See:https://creativecommons.org/Licenses/by-nc/4.0/

Country/Territory of origin:India

ORClD number:Paul Swamidhas Sudhakar Russell 0000-0001-6463-2750;Priya Mary Mammen 0000-0001-7182-6959;Satya Raj Shankar 0000-0001-9454-7610;Shonima Aynipully Viswanathan 0000-0002-3792-0009;Grace Rebekah 0000-0001-6279-4326;Sushila Russell 0000-0003-0055-1542;Richa Earnest 0000-0003-4389-3913;Swetha Madhuri Chikkala 0000-0002-1462-1751.

S-Editor:Ma YJ

L-Editor:Wang TQ

P-Editor:Cai YX