Skip to main content

Advertisement

SpringerLink
Log in

Augmented reality and cone beam CT guidance for transoral robotic surgery

  • Original Article
  • Published:
Journal of Robotic Surgery Aims and scope Submit manuscript

Abstract

In transoral robotic surgery preoperative image data do not reflect large deformations of the operative workspace from perioperative setup. To address this challenge, in this study we explore image guidance with cone beam computed tomographic angiography to guide the dissection of critical vascular landmarks and resection of base-of-tongue neoplasms with adequate margins for transoral robotic surgery. We identify critical vascular landmarks from perioperative c-arm imaging to augment the stereoscopic view of a da Vinci si robot in addition to incorporating visual feedback from relative tool positions. Experiments resecting base-of-tongue mock tumors were conducted on a series of ex vivo and in vivo animal models comparing the proposed workflow for video augmentation to standard non-augmented practice and alternative, fluoroscopy-based image guidance. Accurate identification of registered augmented critical anatomy during controlled arterial dissection and en bloc mock tumor resection was possible with the augmented reality system. The proposed image-guided robotic system also achieved improved resection ratios of mock tumor margins (1.00) when compared to control scenarios (0.0) and alternative methods of image guidance (0.58). The experimental results show the feasibility of the proposed workflow and advantages of cone beam computed tomography image guidance through video augmentation of the primary stereo endoscopy as compared to control and alternative navigation methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Zhen W, Karnell LH, Hoffman HT, Funk GF, Buatti JM, Menck HR (2004) The National Cancer Data Base report on squamous cell carcinoma of the base of tongue. Head Neck 26:660–674

    Article  PubMed  Google Scholar 

  2. Weinstein GS, O’Malley BW Jr, Magnuson JS, Carroll WR, Olsen KD, Daio L, Moore EJ, Holsinger FC (2012) Transoral robotic surgery: a multicenter study to assess feasibility, safety, and surgical margins. Laryngoscope 122:1701–1707

    Article  PubMed  Google Scholar 

  3. Weinstein GS, Quon H, Newman HJ, Chalian JA, Malloy K, Lin A, Desai A, Livolsi VA, Montone KT, Cohen KR, O’Malley BW (2012) Transoral robotic surgery alone for oropharyngeal cancer: an analysis of local control. Arch otolaryngol-head and neck surg 138:628–634

    Article  Google Scholar 

  4. Van de Kelft E, Costa F, Van der Planken D, Schils F (2012) A prospective multicenter registry on the accuracy of pedicle screw placement in the thoracic, lumbar, and sacral levels with the use of the O-arm imaging system and StealthStation Navigation. Spine 37:E1580–E1587

    Article  PubMed  Google Scholar 

  5. Su LM, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD (2009) Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 73:896–900

    Article  PubMed  Google Scholar 

  6. Hughes-Hallett A, Mayer EK, Marcus HJ, Cundy TP, Pratt PJ, Darzi AW, Vale JA (2013) Augmented reality partial nephrectomy: examining the current status and future perspectives. Urology 83:266–273

    Article  PubMed  Google Scholar 

  7. Volonte F, Buchs NC, Pugin F, Spaltenstein J, Jung M, Ratib O, Morel P (2013) Stereoscopic augmented reality for da Vinci robotic biliary surgery. Int J Surg case rep 4:365–367

    Article  PubMed Central  PubMed  Google Scholar 

  8. Cabrilo I, Sarrafzadeh A, Bijlenga P, Landis BN, Schaller K (2014) Augmented reality-assisted skull base surgery. Neurochirurgie 60(6):304–306

    Article  CAS  PubMed  Google Scholar 

  9. Caversaccio M, Garcia Giraldez J, Thoranaghatte R, Zheng G, Eggli P, Nolte LP, Gonzalez Ballester MA (2008) Augmented reality endoscopic system (ARES): preliminary results. Rhinology 46(2):156–158

    PubMed  Google Scholar 

  10. Caversaccio M, Langlotz F, Nolte LP, Häusler R (2007) Impact of a self-developed planning and self-constructed navigation system on skull base surgery: 10 years experience. Acta Otolaryngol 127(4):403–407

    Article  PubMed  Google Scholar 

  11. Chen X, Wang L, Fallavollita P, Navab N (2013) Precise X-ray and video overlay for augmented reality fluoroscopy. Int J Comput Assist Radiol Surg 8:29–38

    Article  CAS  PubMed  Google Scholar 

  12. Volonte F, Buchs NC, Pugin F, Spaltenstein J, Schiltz B, Jung M, Hagen M, Ratib O, Morel P (2013) Augmented reality to the rescue of the minimally invasive surgeon. The usefulness of the interposition of stereoscopic images in the da Vinci robotic console. Int J Med Robotics + Comp Assisted Surg : MRCAS 9:e34–38

    Article  Google Scholar 

  13. Mirota DJ, Uneri A, Schafer, S, Nithiananthan S, Reh, DD, Gallia GL, Taylor RH, Hager GD, Siewerdsen JH (2011) High-accuracy 3D image-based registration of endoscopic video to C-arm cone-beam CT for image-guided skull base surgery. In: Wong, K.H., Holmes Iii, D.R. (eds.) SPIE Medical Imaging, vol 7964, pp 79640 J-79610. SPIE, Lake Buena Vista, FL

  14. Liu WP, Mirota DJ, Uneri A, Otake Y, Hager GD, Reh DD, Ishii ML, Siewerdsen JH (2012) A clinical pilot study of a modular video-CT augmentation system for image-guided skull base surgery. In: SPIE Medical Imaging 2012: Image-Guided Procedures, Robotic Interventions, and Modeling, pp 8316–8112

  15. Pratt P, Edwards E, Arora A, Tolley N, Darzi A.W, Yang G.-Z (2012) Image-guided transoral robotic surgery for the treatment of oropharyngeal cancer. In: Hamlyn Symposium

  16. Falk V, Mourgues F, Vieville T, Jacobs S, Holzhey D, Walther T, Mohr FW, Coste-Maniere E (2005) Augmented reality for intraoperative guidance in endoscopic coronary artery bypass grafting. Surgical technology international 14:231–235

    PubMed  Google Scholar 

  17. Pietrabissa A, Morelli L, Ferrari M, Peri A, Ferrari V, Moglia A, Pugliese L, Guarracino F, Mosca F (2010) Mixed reality for robotic treatment of a splenic artery aneurysm. Surg Endosc 24:1204

    Article  PubMed  Google Scholar 

  18. Suzuki N, Hattori A, Suzuki S, Otake Y (2007) Development of a surgical robot system for endovascular surgery with augmented reality function. Stud Health Technol Inform 125:460–463

    PubMed  Google Scholar 

  19. Herrell, SD, Kwartowitz, DM, Milhoua, PM, Galloway, RL (2009) Toward image guided robotic surgery: system validation. J Urol 181, 783–789; discussion 789–790

  20. Edwards PJ, King AP, Hawkes DJ, Fleig O, Maurer CR Jr, Hill DL, Fenlon MR, de Cunha DA, Gaston RP, Chandra S, Mannss J, Strong AJ, Gleeson MJ, Cox TC (1999) Stereo augmented reality in the surgical microscope. Stud Health Technol Inform 62:102–108

    CAS  PubMed  Google Scholar 

  21. Surgery Toward Intraoperative Image-Guided Transoral Robotic (2013) Liu, W.P., Reaungamornrat, S., A., D., Sorger, J.M., Siewerdsen, J.H., Richmon, J.D., Taylor, R.H. Robotic Surgery 7:217–225

    Article  Google Scholar 

  22. Badani KK, Shapiro EY, Berg WT, Kaufman S, Bergman A, Wambi C, Roychoudhury A, Patel T (2013) A Pilot Study of Laparoscopic Doppler Ultrasound Probe to Map Arterial Vascular Flow within the Neurovascular Bundle during Robot-Assisted Radical Prostatectomy. Prostate cancer 2013:810715

    Article  PubMed Central  PubMed  Google Scholar 

  23. Liu WP, Reaungamornrat SAD, Sorger JM, Siewerdsen JH, Richmon JD, Taylor RH (2013) Toward intraoperative image-guided transoral robotic surgery. Robotic Surg. 7:217–225

    Article  Google Scholar 

  24. Reaungamornrat S, Liu WP, Wang AS, Otake Y, Nithiananthan S, Uneri A, Schafer S, Tryggestad E, Richmon J, Sorger JM, Siewerdsen JH, Taylor RH (2013) Deformable image registration for cone-beam CT guided transoral robotic base-of-tongue surgery. Phys Med Biol 58:4951–4979

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  25. Yushkevich Paul A, Piven Joseph, Hazlett Heather Cody, Smith Rachel Gimpel, Ho Sean, Gee James C, Gerig Guido (2006) User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability. Neuroimage 31(3):1116–1128

    Article  PubMed  Google Scholar 

  26. Deguet A, Kumar R, Taylor RH, Kazanzides P (2008) The cisst libraries for computer assisted intervention systems. MICCAI Workshop https://trac.lcsr.jhu.edu/cisst/

  27. Jung MY, Balicki M., Deguet A, Taylor RH, Kazanzides P (2014) Lessons learned from the development of component-based medical robot systems. software engineering for robotics 5(2):25–41

  28. Pieper S, Lorenson B, Schroeder W, Kikinis R (2006) The NA-MIC kit: ITK, VTK, pipelines, grids, and 3D Slicer as an open platform for the medical image computing community. Proc. IEEE Intl. Symp. Biomed. Imag 698–701

  29. Reiter A, Allen PK, Zhao T (2012) Feature classification for tracking articulated surgical tools. Med Image Comput Comput Assist Interv 15:592–600

    PubMed  Google Scholar 

  30. Liu WP, Reaugamornrat S, Sorger JM, Siewerdsen JH, Taylor RH, Richmon JD (2014) Intraoperative image-guided transoral robotic surgery: pre-clinical studies. Int J Med Robot 11(2):256–267

    Article  PubMed  Google Scholar 

  31. Rieger A, Blum T, Navab N, Friess H, Martignoni ME (2011) Augmented reality: merge of reality and virtuality in medicine. Dtsch Med Wochenschr 136:2427–2433

    Article  CAS  PubMed  Google Scholar 

  32. Stayman JW, Otake Y, Prince JL, Khanna AJ, Siewerdsen JH (2012) Model-Based tomographic reconstruction of objects containing known components. IEEE Trans Med Imaging 31(10):1837–1848

    Article  PubMed Central  PubMed  Google Scholar 

Download references

Acknowledgments

The authors extend sincere thanks to support provided by Intuitive Surgical Inc., Johns Hopkins, NIH-R01-CA-127444, and the Swirnow Family Foundation. The SAW software infrastructure used in this work was developed under NSF grants EEC9731748, EEC0646678, MRI0722943, and NRI1208540 and under Johns Hopkins University internal funds.

Author information

Authors and Affiliations

  1. Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA

    Wen P. Liu & Russell H. Taylor

  2. Department of Otolaryngology-Head and Neck Surgery, Johns Hopkins Hospital, Baltimore, MD, USA

    Jeremy D. Richmon

  3. Intuitive Surgical Inc., Sunnyvale, CA, USA

    Jonathan M. Sorger & Mahdi Azizian

Authors
  1. Wen P. Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeremy D. Richmon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jonathan M. Sorger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mahdi Azizian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Russell H. Taylor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wen P. Liu.

Ethics declarations

Conflict of interest

Wen P. Liu, PhD, is a student fellow sponsored by Intuitive Surgical, Inc., Jeremy D. Richmon, MD, is a proctor for Intuitive Surgical, Inc., Jonathan M. Sorger, PhD, is an employee of Intuitive Surgical, Inc., Mahdi Azizian, PhD, is an employee of Intuitive Surgical, Inc., Russell H. Taylor presents no conflict of interest.

Ethical studies

All institutional and national guidelines for the care and use of laboratory animals were followed.

Informed consent

Statement of informed consent was not applicable since the manuscript does not contain any patient data.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, W.P., Richmon, J.D., Sorger, J.M. et al. Augmented reality and cone beam CT guidance for transoral robotic surgery. J Robotic Surg 9, 223–233 (2015). https://doi.org/10.1007/s11701-015-0520-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11701-015-0520-5

Keywords

Search

Navigation