Mixed Reality Assisted Radical Endoscopic Thyroidectomy

Radical endoscopic thyroidectomy is associated with various surgical complications. This study utilizes mixed reality techniques to assist surgeons in performing radical endoscopic thyroidectomy, aiming to enhance its safety and lower the surgical threshold.

Radical endoscopic thyroidectomy (ET) offers superior cosmetic outcomes and enhanced visibility of the surgical field compared to open surgery. However, the thyroid's unique physiological functions and intricate surrounding anatomy may result in various surgical complications. Mixed reality (MR), a real-time holographic visualization technology, enables the creation of highly realistic 3D models in the real world and facilitates multiple human-computer interactions. MR can be utilized for both preoperative evaluation and intraoperative navigation. First, semi-automatic 3D reconstruction of the neck from enhanced computed tomography images is performed using 3Dslicer. Next, the 3D model is imported into Unity3D to create a virtual hologram that can be displayed on an MR helmet-mounted display (HMD). During surgery, surgeons can wear the MR HMD to locate lesions and surrounding anatomy through the virtual hologram. In this study, patients requiring radical ET were randomly assigned to either the experimental group or the control group. Surgeons performed MR-assisted radical ET in the experimental group. A comparative analysis of surgical outcomes and the results of scales was conducted. This study successfully developed the neck 3D model and the virtual hologram. According to the NASA Task Load Index Scale, the experimental group exhibited significantly higher scores in 'Own Performance' and lower scores in 'Effort' compared to the control group (p = 0.002). Additionally, on the Likert Subjective Evaluation Scale, the mean scores for all questions exceeded 3. Although the incidence of surgical complications was lower in the experimental group than in the control group, the differences in surgical outcomes were not statistically significant.MR is beneficial for enhancing performance and alleviating the burden of surgeons during the perioperative period. Furthermore, MR has demonstrated the potential to enhance the safety of ET. Therefore, it is essential to further investigate the surgical applications of MR.

Over the past decade, the global incidence of thyroid nodules has risen to 29.29%, reflecting an increase of 7.76% compared to the previous decade. This trend positions thyroid nodules as one of the most prevalent diseases¹. Concurrently, the incidence of thyroid cancer has reached 20/100,000, ranking it seventh among all cancers and third among females. Radical surgery remains the preferred treatment approach².

Thyroidectomy can be categorized into open, endoscopic, and robotic surgeries. Endoscopic thyroidectomy (ET) is a form of cosmetic surgery that employs the length of endoscopic instruments to create subcutaneous tunnels through incisions made in the areola, axilla, oral cavity, and other concealed areas, thereby facilitating a scarless appearance in the neck. This approach is particularly appealing to women, who constitute the majority of thyroid cancer patients³. However, ET still requires further optimization from a surgical perspective. Firstly, ET is challenged by limited surgical space and a lack of tactile feedback, which complicates the operation and contributes to a prolonged learning curve⁴. Secondly, the thyroid possesses special physiological functions and complex local anatomy, making ET susceptible to serious complications, such as hypoparathyroidism and recurrent laryngeal nerve (RLN) injury⁵. Additionally, thyroid cancer often presents with occult lymphatic metastasis, and incomplete cervical lymph node dissection may further elevate the risk of postoperative lymphatic recurrence⁶, leading to the necessity for secondary or even multiple surgeries in some patients.

Artificial intelligence (AI) is an emerging field that seeks to simulate and enhance human intelligence through computer algorithms. Extended reality (XR) is a category of AI techniques that can deliver highly realistic audiovisual information in real-time, particularly suited for surgical applications⁷. Mixed reality (MR), a subset of XR, integrates virtual holograms with real-world entities, enabling users to interact seamlessly between reality and virtual environments⁸. Three-dimensional (3D) reconstruction is a computer graphics technique that transforms two-dimensional (2D) images composed of pixels into 3D models made up of voxels. The presentation of 3D models is crucial, and the primary application of MR in surgery involves displaying 3D models constructed from tomographic images, such as computed tomography (CT) and magnetic resonance imaging (MRI).

In this study, MR was utilized to assist in radical ETwith the aim of enhancing its efficacy and safety while reducing surgical complexity. Patients diagnosed with thyroid cancer who met specified inclusion and exclusion criteria were randomly assigned to either the experimental group or the control group. Those in the experimental group underwent MR-assisted radical ET. A comparative analysis of the surgical outcomes between the two groups was conducted. Furthermore, the National Aeronautics and Space Administration Task Load Index Scale (NASA-TLX) and the Likert Subjective Rating Scale were employed to assess the impact of MR-assisted ET on the surgeons. NASA-TLX is recognized as the gold standard for subjective workload assessment and comprises six dimensions: mental demand (MD), physical demand (PD), temporal demand (TF), own performance (OP), effort (EF), and frustration (FR)⁹. Each dimension is rated independently on a scale of 0 to 100. The Likert Subjective Rating Scale includes a series of questions with five response levels ranging from very satisfied (5 points) to very dissatisfied (1 point), with the intermediate levels being satisfied (4 points), acceptable (3 points), and dissatisfied (2 points).

The protocol follows the guidelines of the human research ethics committee of Sun Yat-Sen University. No specific ethics approval is required since this treatment was performed in routine clinical care.

1. 3D reconstruction

1. Import DICOM data from neck-enhanced CT scans into 3DSlicer¹⁰, utilizing a window width of 350 Hounsfield units (HU) and a window level of 40 HU.
2. Segment and reconstruct the neck 3D model using a semi-automatic approach.
   1. Segment and reconstruct the skin using the Threshold and Hollow functions. First, create a segmentation for CT values greater than -250 HU with the Threshold function. Then, utilize the Hollow function to remove the interior of this segmentation.
   2. Segment and reconstruct the thyroid, lesion, trachea, and esophagus using the Grow from seeds function based on either arterial or venous phase images. Manually delineate seeds within the target structures across cross-sectional, sagittal, and coronal images. Following this, automatically generate the target segmentation using the Grow from seeds function.
   3. Segment and reconstruct the bone using the Threshold function based on plain scanning images. Create this segmentation for CT values exceeding 200 HU.
   4. Segment and reconstruct arteries using the Local threshold function based on arterial phase images. Manually delineate seeds within the arteries in the coronal images. Subsequently, create a segmentation with a CT value range consistent with the seeds using the Local threshold function, and repeat the procedure to gradually expand the segmentation.
   5. Segment and reconstruct veins using the Local threshold function based on venous phase images. Use the specific operations identical to those described in the previous step.
3. Automatically reconstruct the neck 3D model using volume rendering.
   1. Select either arterial phase images or venous phase images within the volume rendering module.
   2. Opt for an optimal automatic reconstruction protocol, typically the coronary enhanced CT reconstruction protocol, emphasizing visualizing the superior and inferior thyroid vessels.
   3. Minimize unnecessary reconstruction by precisely adjusting the region of interest.

2. Constructing the neck virtual hologram

1. Export the semi-automatically reconstructed neck 3D model from 3DSlicer as an OBJ file.
2. Create a new project in Unity3D utilizing the Mixed Reality Toolkit (MRTK) and configure the necessary components.
   1. Add a control border by utilizing the Box collider component.
   2. Implement a movable cursor using the Cursor Context Object Manipulator component.
   3. Add movement, scaling, and rotation functions through the Object manipulator, Near interaction Grabbable, and Min Max Scale Constraint components.
   4. Enable transparency control by utilizing the Slider Transparency Controller component.
3. Import the OBJ file into the project and associate the aforementioned components with the neck 3D model.
4. Debug the neck virtual hologram on the MR head-mounted device (HMD) using the Holographic Remoting program, followed by packaging and exporting the project from Unity3D¹¹.
5. Install the neck virtual hologram onto the MR HMD using the Visual Studio program.

3. MR device manipulation

1. Wear the MR HMD before surgery or have circulating nurses assist in wearing the HMD during surgery.
2. Use MR HMD to manipulate the neck virtual holograms.
   1. Control the movement, scaling, and rotation of the neck virtual hologram using a Grab gesture.
   2. Adjust the transparency of the neck virtual hologram by dragging the corresponding virtual slider.
   3. Align the neck virtual hologram with the patient's position, ensuring proper alignment of anatomical markers such as the mandible and clavicle.
3. Import clinical data such as neck CT images, ultrasonography and laboratory examination results into the MR HMD to access this information at any time during surgery.
4. Control the HMD to capture MR images and videos through voice commands.
5. Share the first-person perspective through the MR HMD via a Wi-Fi connection.

4. Preoperative stage

1. Inclusion criteria
   1. Include patients with papillary thyroid carcinoma (PTC) with a maximum diameter ≤ 3 cm.
   2. Include patients who express a requirement for beauty.
2. Exclusion criteria
   1. Exclude patients who are unable to tolerate surgery or general anesthesia.
   2. Exclude patients exhibiting extrathyroidal invasion or distant metastasis of PTC.
   3. Exclude patients with a history of thyroid surgery, ablation, or radioactive iodine treatment.
   4. Exclude patients with chest or clavicle deformities.
   5. Exclude patients with hypoparathyroidism or vocal cord dysfunction.
3. Auxiliary examination
   1. Conduct neck-enhanced CT and ultrasonography to assess the PTC and its surrounding anatomy.
   2. Conduct chest radiography, electrocardiogram, complete blood count, liver and kidney function tests, and coagulation tests to exclude any absolute surgical contraindications.
   3. Conduct thyroid and parathyroid function tests, along with thyroglobulin levels.
   4. Conduct fiberoptic laryngoscopy to evaluate vocal cord function.
4. Preoperative preparation
   1. Fast patients for 10 h and administer intravenous glucose saline or Ringer's solution.
   2. Administer midazolam, propofol, sufentanil, and atracurium intravenously to induce general anesthesia, followed by transoral tracheal intubation. Avoid using muscle relaxants during the maintenance phase of anesthesia.
   3. Perform urethral catheterization.
   4. Connect the intraoperative neuromonitoring (IONM) device.
   5. Position the patient in a split-leg configuration and extend their neck by tilting their head back. Position the endoscope system at the front of the patient's head.

5. Surgical procedure (breast approach)

1. Disinfect the surgical field, lay down the aseptic operating towels, and connect the endoscope, electrocoagulation hook, suction unit, and ultrasound knife.
2. Identify the observation hole located 2 cm to the right of the midpoint of the line connecting the bilateral nipples. Inject expansion fluid subcutaneously and dissect using a stripping rod.
3. Place a 10 mm Trocar and introduce carbon dioxide to establish an operating space with internal pressure maintained at less than 6 mmHg.
4. Use the upper edge of both areolas as the operating channel. Dissect the subcutaneous cavities on both sides with two 5 mm or 10 mm Trocar needles under 30° laparoscopic visualization. Insert the ultrasonic knife and scatheless forceps, and extend the subcutaneous space towards the front of the neck.
5. Separate the cervical white line to free the thyroid gland. Suspend the ribbon muscles with sutures to extend the field of view.
6. Inject carbon nanoparticles into the thyroid gland to stain it and the cervical lymph nodes.
7. Disconnect the isthmus of the thyroid to expose the trachea and detach the suspensory ligament of the thyroid.
8. Utilize the ultrasonic knife to coagulate the superior thyroid artery and vein, inferior thyroid artery, and middle thyroid vein. Secure the larger blood vessels with absorbable clamps.
9. Perform auto-transplantation of PGs into the homolateral sternocleidomastoid muscle if the PG is removed or its blood supply is destroyed.
10. Identify the RLN and superior laryngeal nerve using the IONM probe. Protect the nerves with wet gauze to prevent thermal injury.
11. Carefully separate and excise the lesional thyroid gland, ensuring it is completely retrieved using a specimen bag.
12. Perform homolateral lymphadenectomy in the cervical central region. If indicated, proceed with contralateral thyroidectomy and lymphadenectomy.
13. Ensure all bleeding is thoroughly controlled, and rinse the surgical field with sterile distilled water. Insert a drainage tube in the operating channel and suture the incision layer by layer.

6. Postoperative management

1. Provide electrocardiogram monitoring and oxygen therapy as patients return to the ward. Subsequently, administer treatments that include fluid replacement, intravenous calcium, pain relief, inhalation drug therapy, and phlegm clearance. If required, administer hemostatic drugs and initiate physical therapy.
2. On the first postoperative day, discontinue electrocardiogram monitoring and oxygen therapy and remove the urinary catheter. Patients may resume a full diet and take daily oral doses of levothyroxine and calcium.
3. On the second or third postoperative day, remove the drainage tube and prepare for patient discharge.
4. Instruct the patient to visit the outpatient department for follow-up during the first week after surgery, as well as at 1 month, 3 months, 6 months, and 12 months postoperatively. Routinely review thyroid function and thyroglobulin levels, and perform neck ultrasonography.

This study successfully constructed the neck 3D model of patients with PTC (Figure 1) and performed 14 cases of MR-assisted radical ET (Figure 2).

A total of 32 ETs were performed by a senior surgeon, who completed the NASA-TLX and the Likert Subjective Rating Scale (Table 1 and Table 2) following the surgeries. In the NASA-TLX, the experimental group exhibited a significantly higher OP score and lower EF score compared to the control group (p = 0.002), indicating that the surgeon felt it easier to achieve better performance in MR-assisted radical ET. In the Likert Subjective Rating Scale, the average score for all questions exceeded 3 points, suggesting that the surgeon regarded the protocol as feasible and effective.

The clinical results of this study are summarized in Table 3. None of the patients experienced recurrence. Although the incidence of surgical complications in the experimental group was lower than in the control group, this difference was not statistically significant (p = 0.426). As a preliminary study, the differences in all surgical outcomes were not statistically significant due to the small sample size.

In summary, MR is beneficial for enhancing the intraoperative experience and alleviating the subjective burden on surgeons during the perioperative period. Furthermore, MR has demonstrated potential for improving the safety of ET.

Figure 1: 3D reconstruction of the neck using enhanced CT imaging. (A) Semi-automated segmentation and reconstruction of the neck 3D model. (B) Automated reconstruction of the neck 3D model through volume rendering. Please click here to view a larger version of this figure.

Figure 2: Manipulation of the MR HMD during surgery. (A,B) The surgeon utilized the MR HMD to access the neck CT images. (C,D) The surgeon employed the MR HMD to present the virtual hologram of the neck. Please click here to view a larger version of this figure.

Table 1: Results of the NASA-TLX for this protocol.

Table 2: Questions and scores from the Likert Subjective Rating Scale for this protocol.

Table 3: Comparative analysis of surgical outcomes between the experimental group and the control group.

Table 4: Existing clinical studies on MR-assisted surgery.

MR is a cutting-edge AI technology based on various algorithmic models and sensing devices. The purpose of this protocol is to utilize MR to assist in radical ET. Moreover, the key procedure is the construction of the neck 3D model and virtual hologram. Subjective scales indicate positive results, demonstrating that MR is benefitial for surgeons in performing radical ET with ease. Additionally, MR exhibits potential advantages in preventing complications associated with ET. However, the small sample size limits the significance of the differences observed in all surgical outcomes.

MR is employed as a real-time presentation method for the neck 3D model. This protocol incorporates two 3D reconstruction methods¹²: volume rendering, which effectively displays thyroid blood vessels, and semi-automatic reconstruction, which accurately presents small structures such as lesions and lymph nodes. The 3D reconstruction of neck-enhanced CT images can be completed within one hour, while the production of the neck virtual hologram takes only 15 min. Consequently, the preparation phase of this protocol is simple and can be implemented in routine clinical settings.

MR can significantly enhance the clinical utility of tomographic imaging in surgical settings. On the one hand, the MR HMD works wirelessly and incorporates various innovative human-computer interaction techniques, making it particularly suitable for sterile environments such as operating rooms¹⁰. Surgeons can interact with the virtual hologram at any time during surgery. On the other hand, MR offers greater convenience and efficacy compared to other visualization methods, including 3D printing. It significantly enhances the visualization capabilities of 3D models, which can accurately and in real-time depict the spatial distribution of critical structures, such as target organs, lymph nodes, and blood vessels⁸. Consequently, surgeons can optimize surgical protocols promptly before and during operations, thereby increasing their confidence. To date, MR has been utilized in various surgical disciplines, including orthopedics, neurosurgery, urology, and hepatobiliary surgery. The existing studies on MR-assisted surgery are summarized in Table 4^{13,14,15,16,17,18,19,20,21}. It can be seen that MR-assisted ET is realized for the first time in this protocol. Furthermore, many of these studies utilized subjective rating scales.

However, MR-assisted surgery is not without its limitations. Firstly, the intense lighting in operating rooms may compromise the MR device's ability to accurately recognize and analyze the surrounding environment, thereby diminishing the quality of the virtual hologram presentation²². Secondly, this protocol requires manual alignment of the virtual hologram with the patient for effective localization and navigation. Nevertheless, every alignment method has inherent errors. As a result, MR should be regarded as an auxiliary tool for perioperative assessment rather than a standalone basis for modifying surgical plans without anatomical exposure²³. Additionally, in contrast to other surgical procedures, the approach and field of radical ET are relatively fixed, limiting the potential for optimization through MR.

MR holds significant promise for surgical applications, as it can be integrated with various visual, auditory, and tactile sensors to assist surgeons in conducting realistic preoperative simulation exercises²⁴. Additionally, MR can be combined with deep learning-based image recognition technology to develop an intraoperative navigation system. This enables surgeons to more efficiently and accurately discern the relationship between lesions and surrounding anatomical structures²⁵. MR HMD can present not only virtual 3D models in real-time but also integrate endoscopic system data to display virtual split screens from any position and angle²⁶. During surgery, surgeons can access clinical data such as patient history and auxiliary examination results at any time through the MR HMD and share their first-person perspective with others to facilitate bedside teaching and telemedicine.

In summary, investigating the surgical applications of MR is of considerable importance. The advancement of AI technology is anticipated to promote a transformative shift in surgical procedures in the future. In the context of radical ET, MR has the potential to enhance the surgeon's intraoperative experience, reduce the surgical threshold, and possibly improve safety. However, a large-scale, prospective, multi-center clinical study is necessary to further assess the impact of MR on the efficacy and safety of ET.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

This study was funded by the National Natural Science Foundation Cultivation Program of The Third Affiliated Hospital of Sun Yat-sen University (2023GZRPYMS08) and funding by Science and Technology Projects in Guangzhou (SL2023A03J01216). The authors would also like to acknowledge the joint funding project of the Third Affiliated Hospital of Sun Yat-sen University and Chaozhou Central Hospital.