Pshlis Gvriiliis , , Bj?rn Ewin , Egiijus Plnis , Ernst Hilgo , Niol ’Anglis ,Riro Mmo , Lu Alrightti , Rort P Sutli

a Department of Hepato-Pancreato-Biliary and Liver Transplant Surgery, Queen Elizabeth University Hospitals Birmingham NHS Foundation Trust, B15 2TH,UK

b The Intervention Centre and Department of HPB Surgery, Oslo University Hospital and Faculty of Medicine, University of Oslo, Oslo, Norway

c Department of Hepato-Pancreatico-Biliary Surgery and Transplantation, Hospital Universitari Vall d’Hebron, Barcelona, Spain

d Department of Digestive Surgery, University Hospital Henri Mondor (AP-HP), 94010 Créteil and University of Paris Est, Créteil, France

e Department of Hepatobiliary and Pancreatic Surgery, Miulli Hospital, Acquaviva delle Fonti, Bari 70021, Italy

f Division of Hepatobiliary Surgery, San Raffaele Hospital, Via Olgettina 60, Milan 20132, Italy

Keywords:Navigated Hepatic surgery 3D Computer assistance Image guidance Image guided surgery Indocyanine green 3D print Visual simulation Virtual reality Augmented reality Real-time navigated liver surgery

ABSTRACT

Introduction

A successful hepatic resection (HR) requires being safe, precise and efficacious. For it to be safe, it must be ensured that suffi-cient liver remnant is left to avoid post-resection liver failure and that concept involves adequate functional volume, inflow and outflow [1] . Precision implies an optimal definition of vascular structures and its relationships with tumors. On the other hand, efficacy will be determined by appropriate clearance with clear margins at the transection surface. This overall balance might be difficult to achieve, especially when multiple resections are contemplated or large tumors are resected.

The implementation of advance imaging to develop a satisfactory virtual reality which will then translate into augmented reality once is applied in real-time into the patient’s procedure, to navigate within the liver, replicating the global positioning system(GPS) concept [2-4] . The process begins with interrogating the liver anatomy and tumor distribution, based by and large on contrastenhanced computed tomography (CT) imaging and to lesser extend magnetic resonance imaging (MRI). The 2D obtained information is then processed. Based on the advanced imaging of the virtual and augmented reality we can tackle complex liver anatomy and consequently optimize the diagnosis and tailor individual strategic plans for our patients [5] . Using data of CT and MRI scans virtual reality software can generate 3D liver models of individual patients [ 5 , 6 ].

Radiologic workstations can provide the patient-specific liver anatomy either with direct volume rendering (DVR) or surface rendering. The main privilege of DVR is its easy usability and availability on personal computers. On the other hand, its major disadvantage is that it does not permit the free and interactive modeling of individual anatomical structures and organs. In contrast, with the help of 3D model rendering view individual structures and organs can be selectively removed or manipulated to further improve the visualization.

Furthermore, the combination of the virtual reality and surface rendering leads to the state of augmented reality which enables the profound and detailed understanding of specific patient’s anatomy demonstrating all the anatomic variants and complex anatomic relationships that might be missed using conventional medical imaging modalities [2-5] .

3D modeling contributes essentially to further advancing of precision in hepatic surgery by permitting estimation of liver volume and the functional liver remnant, delineation of resection lines along the liver segments and evaluation of tumor margins.This advanced visualization (virtual reality) can be overlaid onto real life images (augmented reality), during open or minimally invasive procedures, by matching structures on the operative image with ones in the 3D model [4-6] .

However, the principal challenge when performing hepatic surgery is the liver deformation and motion during its manipulation and/or respiration and consequently, this can cause imprecise navigation [2-5] . Recent publications have also demonstrated the essential contribution to further understanding of the individualized liver anatomy with the help of 3D printed liver models [ 5 , 6 ].

Additionally, augmented reality can be enhanced by real-time detection and identification of liver cancers (its anatomy) generated by the indocyanine green (ICG) and its unique characteristic to generate fluorescence emissions. This imaging technique is based on the ability of ICG to bind to plasma protein and produce a light with a peak wavelength of approximately 830 nm when illuminated with near-infrared light [7] . In addition, it is only removed from circulation exclusively by the hepatocytes (into the biliary tree). The applicability and essential usefulness of fluorescent imaging techniques of ICG in surgery were first described for intraoperative patency assessment of coronary artery bypass grafts [8] .

This review discusses the most recent developments in digital imaging technology as well as applications of ICG fluorescence in hepatobiliary and liver transplantation surgery.

Literature search-strategy

A literature search was performed from inception until January 2021 in MEDLINE (PubMed), Embase, Cochrane library and database for systematic reviews (CDSR), Google Scholar, and National Institute for Health and Clinical Excellence (NICE) databases using free text and MeSH terms [navigated, liver resection, hepatic resection, virtual and augmented reality, three-dimensional computer-assisted surgical planning, computer-assisted surgical planning (CASP), hybrid operating room, image guidance, liver, 3D,registration, laparoscopic, parenchyma sparing, hepatocellular carcinoma (HCC), colorectal liver metastases (CRLM), mixed reality,extended reality, segmentation, image guided surgery, planning,intraoperative imaging, navigation, anatomical, planning, hepatectomy, surgical margin, ICG, demarcation line]. References cited in the retrieved articles were manually checked for further analysis.Any disagreements between the authors were resolved by consensus.

Questions

What is the current state of navigated liver surgery and what issues remain unsolved? What are the future research directions in navigated liver surgery that may maximize the practical impact on the field?

Definitions

Intraoperative navigated liver surgery (NLS) has been defined as the visualization of perioperatively acquired imaging and/or postprocessed (computer-modeled) data presented in relation to the patient’s anatomy and the surgical instruments.

The hybrid operating theater (HOT) is equipped with conebeam computed tomography (CBCT) and/or CT, which facilitates intraoperative imaging of the liver such as liver lesions and vascular structures during the operation.

Real life is the real physical surrounding in the world.

Registration is defined the process of combining two different modalities such as 3D models and operative images.

Augmented reality is defined as the overlay of the digital information onto the real world through a screen.

Mixed reality is defined as the merging of both the real world and the digital world with real-time human and environment interaction.

Virtual reality is defined as the immersing of a user in a completely digital world. Additional term and further explanations can be found in the Strasbourg International Consensus Study from 2020 [9] .

3D modeling

Despite the best cross-sectional available imaging modalities,3% to 17% of CRLM and HCC can only be diagnosed microscopically during pathologic examination after a resection [ 8 , 10-12 ]. In particular, HCC smaller than 2 cm with indistinct margins can be overlooked even by pathologic examination because microscopic and detailed examination of the whole specimen is not feasible [12-14] .One of the essential prerequisites in parenchyma sparing liver resection is the spatial understanding by the surgeons of the precise tumor location within the specific segment along with its relation with the intrahepatic vessels. Currently, the vast majority of surgeons study contrast enhanced CT and/or MRI scans. Accordingly,they will try to mentally reconstruct the information acquired from 2D images into 3D for a spatial understanding of the liver anatomy and the location of the tumor.

The available medical software (Syngo.via Liver Analysis, Slicer,VR-Planning, Osirix on MacOS, VR RENDER, Myrian, MeVis) allows to obtain 3D models with liver parenchyma, hepatic artery, portal and hepatic veins, bile ducts and lesions from conventional 2D medical images. These models can be used for visualization of anatomy, volume calculations as well as being split into anatomical segments. With specialized tools, these patient-specific 3D liver models can also be used to plan resections, utilizing annotated 3D structures to optimize a parenchyma-sparing approach while preserving sufficient surgical margins [15] . 3D modeling helps for detailed evaluation of the biliary and vessel anatomy at the hepatic hilum for preoperative planning for Bismuth-Corlette type III hilar cholangiocarcinoma. Operations performed with the help of 3D modeling demonstrated significantly shorter operative time, less blood loss, and fewer specimen margin positive cases compared to those without [16] .

Fig. 1. Surgical setup during laparoscopic liver resection with navigation at The Intervention Centre, Oslo University Hospital (approval was obtained from the Oslo group).

Fig. 2. HoloLens in use to visualize patient-specific anatomy during laparoscopic liver resection at The Intervention Centre, Oslo University Hospital (approval was obtained from the Oslo group).

In Oslo (Norway) a network of different hospitals and surgical departments have established a system for sharing holograms(mixed reality using Microsoft HoloLens, hereafter HoloLens) obtained from preoperative 2D conventional imaging [17] . A study was conducted to describe and evaluate their experience using these new tools in workflows, showing very high interest and acceptance by the users both in local and international institutions [18] ( Figs. 1 and 2 ).

Moreover, Pelanis et al. [19] from the same group, investigated whether the surgeons’ understanding increased by using liver holograms with HoloLens compared to 2D screen view of multiplanar reconstruction of MRI images. They reported similar accuracy of lesion localization although the median time for correct identification was significantly shorter using Hololens [6 min (range 1-35)]compared to MRI images [23.5 min (range 4-138)].

Some of the previously mentioned approaches for creating 3D models and surgical plans can also be brought into the operating room as interactive maps shown on a computer screen and controlled by a supporting technician [ 15 , 20 ].

Though still in early phase of clinical research, holograms can also be used intraoperatively - eliminating the need for screens and allowing surgeons to interact by themselves with the liver models [ 15 , 21 ].

Commercially available, Cascination Cas-One medical device provides navigation software for open and laparoscopic liver surgery. After a registration procedure, tracked instruments can be shown on a patient-specific liver model on an interactive screen or as augmented reality on-top of laparoscopic camera view [ 22 , 23 ].

So far, there have been reported very few liver resections either open or laparoscopic-assisted by such navigation [ 6 , 24-26 ].Ntourakis et al. [6] reported promising results of the use of augmented reality to identify disappearing CRLM following neoadjuvant chemotherapy. By comparing and superimposing pre- and post-treatment images with the surgical image (exoscopic camera VITOM?), they were able to completely resect the sites of disappearing CRLM. Application of the augmented reality navigation in open liver resections should affect the deformations caused by the mobilization and manipulation of the liver. This deformation can deform the numerical mesh and will require elastic registration to provide adequate augmentation. Golse et al. [27] proposed a non-rigid registration method, integrating a physics-based elastic model of the liver, computed in real-time using efficient finite element method. To fit the actual deformations, the model was driven by data provided by a single RGB-D camera. They reported 7.9 mm root mean square error for the registration of the internal landmarks and the setup installation time was less than 10 minutes. Consequently, they concluded that the non-rigid registration system demonstrated anatomical accuracy and clinical feasibility.However, their results should be tested by future studies investigating the oncological outcomes of complex liver resections. Several upcoming navigation solutions are being introduced and researched for minimally invasive liver resections, like pure laparoscopic and robot-assisted [ 28 , 29 ].

Fluorescent imaging

Novel fluorescent imaging utilization and new techniques with high detective sensitivity have been developed based on the visualization of the characteristic disordered biliary excretion of ICG from the surrounding noncancerous tissues of CRLM and from within the HCC itself [30] . Challenges in liver surgery that have been potentially addressed or even resolved by fluorescent imaging can be summarized in six points. First, complete resection of small HCCs less than 2 cm (early or very early HCC) without distinct capsule can be achieved by assessing the presence of residual fluorescence on the liver parenchymal resection surface [ 13 , 14 , 18 , 30 , 31 ]. Second,fluorescent techniques contributed to the detection of disappearing metastases after neoadjuvant treatment. In such cases, a good indicator for complete resection of CRLM that partially or completely responded to neoadjuvant treatment is the complete absence of residual fluorescence on the cutting surface [31] . Third,during laparoscopic resection of CRLM guided by the ICG, capture ring of the metastases can help to perform R0 resection without the use of intraoperative ultrasound [32] . Furthermore, the inability of laparoscopic surgery to provide tactile perception, can be overcome by the above technique. Fourth, ICG retention can delineate areas of cholestasis from bile duct tumor infiltration and in return suggest the adequate extension to achieve clear oncological resection [33] . Fifth, ICG has been successfully used to detect biliary leaks from the hepatic resection cutting surface [34] . Sixth, it has been reported that precise real-time definitions of liver segments and subsegments can be seen with the help of ICG and its fluorescence imaging under near-infrared light [ 35 , 36 ]. In particular, for laparoscopic hepatic resections, the intraoperative delineation of segments V, VI, VII, and VIII is of great importance.Japanese teams in HOT reached the targeted hepatic arterial branch through catheterization of the right femoral artery. Consequently,after confirmation of the perfused area by arteriography, embolic solution containing ICG was injected, and the branch was embolized, while the limits of the ICG colored segment were observed using a near-infrared imaging system. As a consequence, precise laparoscopic resection of the segment was performed to confirm the intra-parenchymal boundary by observing ICG fluorescence on the cutting surface [ 37 , 38 ] ( Fig. 3 ).

Another point of contention is the optimization of the timing and dose of ICG for intraoperative imaging of hepatic neoplasms.In 2016, pioneers of the method reported their initial experiences with fusion ICG fluorescence images in hepatectomies. Takahashi et al. administered 7.5 mg ICG intravenously (i.v.) two days preoperatively [39] and they recommended 0.5 mg/kg i.v. within two days before surgery [32] . In a later publication, Zhou et al. [40] reported that i.v. administration of the ICG at a dose of 0.25 mg/kg,3-5 days preoperatively, was associated with the highest success rate in visualization of HCCs during laparoscopic hepatectomy. Interestingly, an experimental study on pigs demonstrated that direct injection of ICG into the gallbladder produced better quality images of the biliary anatomy compared to i.v. injection images [41] .

The main limitation in applying ICG fluorescence imaging is its restricted penetration depth of only 5 to 6 mm [42] . Future research should focus on increasing the penetration depth of ICG.

In addition, given that the biliary excretion of ICG takes approximately 30 minutes after intravenous injection, we can obtain ICG-based fluorescence images of the biliary tree, which are very helpful in complex biliary procedures [42] . Operations performed with the help of 3D modeling demonstrated significantly shorter operative time, less blood loss, and fewer specimen margin positive cases compared to those without [42] .

3D modeling in liver transplantation

Living donor liver transplantation (LDLT) is the established operation for end-stage liver diseases, particularly in countries with a shortage of cadaveric donors. One of the principal concerns of this procedure is the determination of the adequate donor vessel’s cut lines in order to avoid major complications such as stenosis of the donor’s vessels and bile leakage from the cutting surface. Furthermore, anastomosis mismatching and consequent stenosis may cause graft malfunction.

To date, the optimal management of the middle hepatic vein(MHV) for the right lobe graft in LDLT remains controversial [43] .The 3D liver models have contributed significantly to detail preoperatively the anatomy and variations of the MHV. Simulated hepatectomies can be performed preoperatively and this facilitates the optimal management during donor hepatectomy and its reconstruction ( Fig. 4 ) [44] . It is possible that the 3D printed models are superior to conventional printing or even screen visualization and its intrahepatic vessels can be seen through the transparent parenchyma from any angle. Depending on the 3D print material used, liver model can have several unique characteristics, such as elasticity, softness, and feel of texture similar to native liver parenchyma. Surgeons can visualize the positional relationship between the vessels and the surface of the liver even after mobilization and rotation of the liver during the operation. Moreover, using the hilar vascular model of the recipient and the 3D model of the donor liver, the whole team can discuss and plan any step of operative and reconstructive procedures.

Fig. 3. Liver surface image after segment VIII was embolized with staining solution. A: Overlaid image of ICG fluorescence and RGB color image; B: indigo carmine blue image. Cutting surface image during liver resection, overlaid image of ICG fluorescence and RGB color image ( C ), indigo carmine blue image ( D ). Reprinted with permission from Ueno et al. [37] .

Fig. 4. A : Front view. B: Bottom view. C: Preoperative simulation of right-lobe donor hepatectomy using the 3D model. D: Intraoperative navigation using the sterilized 3D model. Reprinted with permission from Kuroda et al. [44] .

The main drawbacks of the technique of the 3D printed liver model areas are as follows: first, there are specific skills that need to be mastered by the technicians, with senior radiological support, in order to process the 3D data; and second, the materials of each model might cost $700 with the processing time being ≥10 days [45] . In order to counter the high cost, Oshiro et al. [46] proposed a 3D-printed liver model, without using a transparent loading material, surrounded by frames modeled on the surface of the liver ( Fig. 5 ).

In pediatric LDLT, large-for-size syndrome and hepatic venous twisting secondary to graft growth are two causes of major complications [ 47 , 48 ]. Therefore, meticulous surgical planning based on precise knowledge of liver section volume, intrahepatic vessels, and their corresponding perfusing and draining territories are essential for surgical planning. Wang et al. [48] reported that 3D touchable liver models with transparent parenchyma and abdominal cavity models facilitated surgical planning and procedures(LDLT setting), particularly in the management of hepatic venous reconstruction and preventing of large-for-size syndrome. After the placement of the virtual resection plane on the 3D liver model, the donor’s liver was divided into graft and remnant components. Consequently, the graft was tested into the 3D reconstructed abdominal cavity model to confirm whether the graft was implantable without the risk of large-for-size syndrome [48] ( Fig. 6 ). The same team also reported significantly shorter operative time and lower inpatient costs for donors compared to those for the cohort without the use of 3D printed models [48] .

Fig. 5. A novel 3D-printed liver model. The 3D-printed liver model was originally surrounded by frames that were modeled on the surface of the liver. Reprinted with permission from Oshiro et al. [46] .

Conclusions

3D models can help in detailed anatomical evaluation and spatial relationships between the tumor and biliary and intrahepatic vessels of patients. The 3D models are displayed on a twodimensional screen, and objects can occlude each other, which hampers spatial comprehension of the depth or complex structures. Personnel, such as medical specialists, need to be trained to develop the skills to process medical images such as CT or MRI through a segmentation process to create patient specific 3D models. The models can successfully be used for increased understanding of anatomy, planning procedures and to some extent brought guiding the surgery.

Application of navigation solutions during laparoscopic liver resection is still in early stage of use with specialized research and industrial groups solving remaining challenges.

One of the weaknesses of virtual 3D models is that they lack a sense of tactility for surgeons. In liver transplantation and especially in LDLT, 3D printed models of the donor’s liver and models of the recipient’s hilar anatomy can contribute further to improving the results. In particular, pediatric LDLT abdominal cavity models can help to manage the largest challenge of this procedure, namely large-for-size syndrome. The creation of a 3D printed liver model may cost $700 and require at least 10 days for construction.

As an alternative for the elaborate effort to further utilize radiological imaging modalities, ICG fluorescence imaging techniques can contribute essentially to the real-time definition of liver segments and subsegments. As a result, precise hepatic resection and specimen negative margins can be guided by the presence and

Fig. 6. A: Actual left lateral segment graft and preoperatively 3D-printed left lateral segment graft. The short black arrows point to the hepatic vein, the long black arrows point to the portal vein, and the double black arrows point to the hepatic artery. B, C: The 3D-printed abdominal cavity model and liver model confirmed that the left lobe graft was implantable into the recipient’s abdominal cavity. Reprinted with permission from Wang et al. [48] .

Acknowledgments

None.

CRediT authorship contribution statement

Paschalis Gavriilidis: Conceptualization, Formal analysis, Investigation, Methodology, Software, Validation, Writing – original draft, Writing – review & editing. Bj?rn Edwin: Formal analysis, Investigation, Methodology, Software, Supervision, Validation,Writing – original draft, Writing – review & editing. Egidijus Pelanis: Formal analysis, Investigation, Methodology, Software, Validation, Writing – original draft, Writing – review & editing.Ernest Hidalgo: Formal analysis, Investigation, Methodology, Software, Validation, Writing – original draft, Writing – review & editing. Nicola de’Angelis: Formal analysis, Investigation, Methodology,Software, Validation, Writing – original draft, Writing – review &editing. Riccardo Memeo: Formal analysis, Investigation, Methodology, Software, Validation, Writing – original draft, Writing – review & editing. Luca Aldrighetti: Formal analysis, Investigation,Methodology, Software, Validation, Writing – original draft, Writing – review & editing. Robert P Sutcliffe: Formal analysis, Investigation, Methodology, Software, Validation, Writing – original draft,Writing – review & editing.

Funding

None.

Ethical approval

Not needed.

Competing interest

All authors hereby declare that they have no conflicts of interest to disclose. Bj?rn Edwin and Egidijus Pelanis are co-inventors of technology licensed by the company HoloCare AS and also hold shares in the company indirectly through Inven2 AS.