 |
|
Telemedicine
The New Must for Surgery
Peter M. Schlag, MD, PhD;
K. Thomas Moesta, MD;
Stefan Rakovsky, PhD;
Georgi Graschew, PhD
Arch Surg. 1999;134:1216-1221.
ABSTRACT
The vision of telesurgery comprises a multitude of new communicative elements influencing the way surgeons will treat their patients in the future. The first prerequisite for effective telecommunication is to digitize surgical data. Many medical imaging modalities provide primarily digital data sets, and digital image communication is already entering clinical practice under the labels of teleradiology and telepathology. However, for any surgical purpose, images must refer to tissues. Three-dimensional image reconstruction is warranted, and if such data shall be useful during surgery, different image sources must be combined into some virtual, multiparametric body model and matched to an intraoperatively distorted organ contour. A multitude of detail problems arise, beginning with image standards, data interfaces, data transport, image fusion, registering, contour matching, and, once the data are integrated, all the aspects of surgery-suitable data display and interaction. We refer here to several demonstration projects illustrating such a complex surgical data set and its interactive telecommunication. In all instances, telecommunication was to enable a concentration of distributed medical intelligence at the site where the patient was treated. With further technological development, such telesurgical applications will have a growing influence on patient management and surgical decision making. In the very near future, computer-aided navigation and robotic assistance, based on the same surgical data sets, will be available to all fields of surgery. How decisive the role these methods will play for specific procedures or diseases needs to be determined.
Arch Surg. 1999;134:1216-1221
INTRODUCTION
Zurich, Switzerland, 2032. After a hard 6-hour workday, Theodor Billroth VI (pseudonyms became very popular after the institution of the Worldwide Medical Court) orders the computer to close the microaccess ports of his third duodenopancreatectomy in a row and pulls his head out of the virtual operative environment. The first 2 cases went well, owing to the extensive virtual planning by his assistants; however, this last case. . . . The guy had such a bleed when he inexplicably lost contact with his second intra-abdominal microbot. Fortunately, Alan Whipple IV from New Sloan Kettering in New York, NY, was available to assist him for 30 minutes, but the fees would again wipe out any profit he had otherwise made this day. In a bad temper, he came down from his study to dine with his family, while the patient was cared for by the intensive care robot at his hospital in Sidney, Australia. . . .
Vision or nightmare, modern information technology renders this scenario possible within the not-too-distant future. Whether such development is profitable for the patient and society, what steps should be taken, and which ones should be avoided will be discussed here. Of course, not the least of the conjectures is whether there will still be any diseases in this future requiring radical surgical procedures. However, during this century, there has been little change in the need for surgery and the great breakthrough from "molecular" technologies is apparently not to occur immediately. Information technologies are actually revolutionizing society and some argue that the same technological development will revolutionize surgery to the same extent that antisepsis and general anesthesia did 100 years ago.1-2 However, 100 years ago, technological development was driven by specific demand. Today, technological development drives the applications and, therefore, great care must be taken not to succumb to technocratic excitation, but rather to carefully select for surgical application only those pieces of technological innovation that promise a clear benefit to patients. Considering telemedical applications, the entire surgical datastructure is to be questioned. Telesurgery, in the broadest sense, becomes possible only as surgery itself becomes digital.
DIGITIZATION, VISUALIZATION, AND VIRTUAL REALITY
The transmission of medical image information by electronic means (in the beginning not necessarily in a digital format) is about 40 years old.3 However, digital data formats are a prerequisite for image communication meeting medical quality requirements. As of today, most preoperative information about a given pathology is primarily digital (magnetic resonance imaging, computed tomographic scans, and sonographic imaging). To print this information onto plastic films constitutes a reduction of the informational content and availability. Only standards,4 time, and financial resources are necessary to use this information in its original format. It is then obvious that we need to develop tools of enhanced data viewing, ie, 3-dimensional reconstruction,5-6 volume rendering,7-8 virtual endoluminal views,9-10 and image fusion,11-12 to enable the best possible preoperative assessment of the available information. An improvement of surgical planning is evident, even if patient benefit can not be proven by proper clinical trial techniques. More difficulties may arise if the surgeon, during an operative procedure, wants to review the data or even wishes to superimpose digital information on his or her conventional view of the surgical site. The first problem may be solved by the development of surgery-adapted interfaces based on voice or motion control. The second would involve the construction of a virtual reality environment fusing real-time optical stereoscopic images with stored 3-dimensional data. For open-access general surgery, a multitude of problems arise from such a scenario, some of them purely technical, some of them rather human. Technologically, the first major difficulty arises from the structural and positional mobility of the involved soft tissues, which prevents a precise fusion of preoperative with intraoperative information. The immediate problem, however, lies in the quality of our eyes, their dynamic range, their high-resolution, large weighted field, and spatial optical data acquisition that is so far not equaled by any technical solution. However, the obvious benefits of minimal access surgery, at least for a limited number of diseases, have made surgeons all over the world accept a clearly inferior video image of their work field, even loss of the third dimension, when technological solutions for 3-dimensional viewing were available. Thus, if the benefit from virtual reality surgery is large enough, there is no fundamental obstacle to it. A good example for such benefit comes from cardiac surgery, where considerable progress has been made toward a virtual heart arrest that may enable the surgeon to work on a motionless image of the heart13 while the motion of the beating heart is removed electronically from the video image. Far more difficult, of course, is the then necessary addition of the cyclic spatial reconfiguration of the beating heart to a robotic interface between the surgeon working on the motionless image on the screen and the real moving surgical site.
Closer to reality are developments in digital data processing for surgical purpose. A project of 3-dimensional volume rendering for radiological data at our institution was baptized BERTA (bidirectional enhanced rendering for telemedical applications). The software performs real-time stereoscopic visualization of reconstructed radiological data by displaying off-line rendered views.14 On an imaginary sphere, enclosing the data cube (originated by computed tomography, magnetic resonance imaging, or other modalities), about 5000 views of the object are precalculated. The corresponding views can be created off-line by any method rendering volume data. As a result of the achievable high frame rate, the user is able to rotate the data cubes in real time (50 stereoscopic frames per second) by using different interactive devices (keyboard, mouse, joystick, or voice control). The user can create a region of interest that specifies a cut plane. This cut plane can be pushed through the voxel data set along the line of sight by the interactive device. The visible cut of the plane with the data cube is computed in real time as a gray-level image of the original voxel information (Figure 1). The display of the precalculated views can be used as a rough guide for prescanning the radiological data.
|
|
|
|
Figure 1. The 3-dimensional reconstruction tool bidirectional enhanced rendering for telemedical applications (BERTA) visualizes computed tomographic, magnetic resonance imaging, and ultrasound scans stereoscopically. In a region of interest, the original computed tomographic data are shown. (Source: OP 2000 Surgical Research Unit OP 2000, Robert-Rössle-Klinik am Max-Delbrück-Centrum, Berlin, Germany.)
|
|
|
SURGICAL PLANNING AND SIMULATION
Once the tools are developed to integrate and handle the multitude of anatomical, physiological, and pathological body data within a digital environment, it becomes possible to manipulate this virtual patient by virtual therapies. The most obvious application may be seen in planning the plane of a surgical resection, for example, in parenchymatous organs, optimizing the relationship between radicality and organ damage. For this approach, the governing conditions for remaining organ function and postoperative morbidity must be known and translated into evaluating and self-optimizing algorithms. Again, based on such outcome prediction models, surgical training for the individual case, as well as general surgical training, becomes possible. The evaluating algorithms themselves may represent the individual medical intelligence and experience of an institution, which again can be communicated to others.
The actual technological development first integrates haptic interfaces into what at this point are still rather simple models of virtual interaction with existing morphological data. The Responsive Workbench shown in Figure 2 is a suitable presentation technology for medicine, especially for surgery (in cooperation with Virtual Media and System Design, and Gesellschaft für Mathematik und Daten Verarbeitung, St Augustin, Germany). In the Responsive Workbench concept, virtual objects and control tools are located on a real workbench, in our case a surgical table. To get this effect, stereoscopy is required. The objects, displayed as computer-generated stereoscopic images, are projected onto the surface of the table (in cooperation with Belgian-American Radio Corporation, Kortrijk, Belgium). This view corresponds to the conventional working situation in medical environments. The computer is invisibly integrated into the user's world. The idea behind it is to create a realistic virtual surgical environment where planning and training can be performed without a change in the surgical routine, with movements, gestures, and words being perceived by cameras and microphones, thus adapting the technology to the surgeon, not the other way around (Figure 3).
|
|
|
|
Figure 2. Virtual surgical table for medical simulations with haptic interfaces and voice control. (Source: OP 2000 RRK/MDC, Berlin, Germany and GMD, St Augustin, Germany.)
|
|
|
|
|
|
|
Figure 3. Virtual interactive laparoscopy. A team of surgeons is captured by 2 tracked cameras (from a blue box) and projected into a virtual operating room. Manipulations on the virtual patient are illustrated by projecting videos of former laparoscopic investigations. (Source: OP 2000 RRK/MDC, Berlin, Germany and GMD, St Augustin, Germany.)
|
|
|
TELECONSULTING AND TELEPRESENCE
Teleradiology is developing as digital data communication acquires the necessary speed and quality to communicate radiological data. While at the beginning of this decade comparative trials of radiological diagnoses based on plain radiographs and digitized images displayed on a video monitor resulted in an inferior digitized image,15-16 reports today seem to prove equivalence.17 Especially for military applications, teleradiology systems have already proven their value.18-19 Once legal and financial issues are solved,20-21 the acceptance of this model of collegial reassurance and specialist consultation will certainly grow. In pathology, mailing some histology slides or an embedded specimen to a colleague for a second opinion is widely accepted and digital data communication will simply replace this already common practice.22-23 While some comparative trials yield a good diagnostic accuracy based on computerized still images24-25 the sampling error in static telepathological systems is a concern. In a survey of Austrian pathologists, most would require an interaction with the slide for image selection.26 Such interaction is feasible by using telecommandable microscopes, based on 2-way communication.27-28 Thus, the idea of interaction enters teleconsultation. Teleconsulting in surgery is certainly more difficult, because a multitude of different data have to be communicated. For some kinds of data, even the sensor technology is not readily available (tactile sense, texture, elasticity, or degree of mobility), far less the representation algorithms on the recipient side. However, in those areas of surgery where the surgeon's perception is already limited to the videoscopic image or an optical microscope, teleconsultation will only depend on the availability of data communication technology at a sufficient quality level. The adaptation of high-resolution stereoscopic video imaging to surgical microscopes is one step in that direction (Figure 4), the integration of different data sources into a stereoscopic workstation another. We have developed the technological system architecture to display, communicate, and manipulate the complex surgical data set (Figure 5) required for surgical teleconsulting at our surgical research unit, OP 2000 (a project to design the operating room [OP] of the next century).29
|
|
|
|
Figure 4. Stereoscopic surgical microscope with 2 independent remote channels for second opinion. (Source: SRU OP 2000 RRK/MDC, Berlin, Germany.)
|
|
|
|
|
|
|
Figure 5. Stereoscopic workstation with interactive telemedical facilities. Three video sources are displayed simultaneously: 3-dimensional room camera, 3-dimensional laparoscope, and biopsy examination. (Source: SRU OP 2000 RRK/MDC, Berlin, Germany.)
|
|
|
The OP 2000/SICONET (Stereoscopic Image Communication Network, in cooperation with Deutsche Telekom, Darmstadt, Germany) telecommunication module is suitable for the connection of 2 or more locations with stereoscopic broadcast video, audio, and data, as shown in several applications (Figure 6). This technique is able to transfer live video data for medical purposes. Figure 7 shows the components of this module, which is able to transmit 2 video channels (left and right) in digital serial component form (digital serial component, 2 x 270 megabits per second [MBit/s]), 4 audio channels, and additional data channels for computer networking, multiplexed to a 17-Mbit/s (satellite) or 34-Mbit/s (asynchronous transmission mode) data stream.
|
|
|
|
Figure 6. Working team in OP 2000 (a surgical research unit project) at the Robert Roessle Hospital in Berlin, Germany. (Source: SRU OP 2000 RRK/MDC, Berlin, Germany.)
|
|
|
|
|
|
|
Figure 7. Diagram of asynchronous transmission mode and satellite transmission with OP 2000/stereoscopic image communication network.
|
|
|
The network for the first field trials was implemented using the experimental 20/30-gigahertz (GHz) transponder of the German national satellite "DFS Kopernikus." This method of linking the codecs was chosen for the easy availability, great flexibility, and the possibility of making point to multipoint connections. Actually, a switch to the more commercially used 12/14-GHz transmission frequency band also exists. The operating rooms, the computer graphics system, endoscopic unit, and pathological laser laboratories at the Robert-Rössle-Klinik, Berlin, Germany, are connected by a central digital video router. The router's stereoscopic video outputs (DSC) are coded and transmitted by the OP 2000/SICONET telecommunication system (Figure 7).
In several demonstrations, different telemedical applications have been shown. The performance of the 17-MBit/s satellite link was successfully demonstrated by the first European stereoscopic telesurgery between Berlin and Paris, France (1996 Meeting of Computer Assisted Radiology and Surgery, Paris) and by the first transatlantic stereoscopic interactive telesurgery between Berlin and Chicago, Ill (50th Annual Meeting of the American Society of Surgical Oncology, 1997, Chicago).
In 1997, the OP2000/SICONET module was adapted to asynchronous transmission mode (ATM) channels. We have used an ATM fiberglass connection of about 80 km to connect different departments of the University Hospital Charité, which are distributed within the city of Berlin. At the 1997 Meeting of Computer Assisted Radiology and Surgery, Berlin, we demonstrated an intraoperative, multipoint consultation system via a 34-Mbit/s ATM for a patient with cancer that involved a pathological department (Charité Mitte) and a radiological department (Charité Wedding) assisting the surgeons at the Robert-Rössle-Klinik (Charité Buch) during the surgical intervention (Figure 8). Not only were video and audio connections provided, but interactive manipulations were performed remotely.
|
|
|
|
Figure 8. University Hospital Charité/Berlin, Germany's OP 2000–distributed medical intelligence in surgery, radiology, and pathology.
|
|
|
At the 1998 meeting of the European Society of Surgical Oncology, Lausanne, Switzerland, multiple operating theaters, 3 in Berlin and 2 in Lausanne, were interactively connected via satellite.
At the final demonstration of the European project PANORAMA (Package for New Operational Autostereoscopic Multiview systems and Applications; in cooperation with 12 European science centers, universities, and companies), an interactive telesurgical scenario using stereoscopic image sources and stereoscopic visualization was presented by OP2000/SICONET via ATM.
Telemedicine will help to focus on the advantages of expert knowledge by rendering access to all available data. These possibilities should be accessible during a consultation, an examination, or in the course of a surgical intervention. Real-time modalities, eventually including the possibility of direct interaction with the data, are referred to as telepresence and exceed by far an electronic version of the patient's medical folder.4, 30 Especially in oncology, interdisciplinary collaboration is immensely important for successful therapy. Preoperative diagnostic data still need to be evaluated according to the intraoperative findings. At this decisive moment, it is necessary to involve specialists of other oncological disciplines. Real-time communication devices have to be present to transfer image data and clinical observations and ensure the best possible transmission quality to resident and geographically distant experts.
TELESURGERY
Theoretically, there is only a small step from the virtual plan of a surgical procedure to its execution in the manner it was planned. However, taking the scalpel out of the surgeon's hand and putting it into the mechanical front end of a robot involves more than one might expect, and is not within the focus of this article. The creation of a robotic interface between the surgeon and the patient is a prerequisite for the scenario that was drawn up in the introduction. Increasing the distance between the surgeon guiding the procedure and the robotic execution of the orders from meters to kilometers turns computer aided surgery into telesurgery, with all of the implications of immediate teleconsulting and teleassistance. However, traditional medical ethics should limit the allowance of distance between surgeons and their patients to situations of war31 or to areas that would otherwise not receive medical care owing to limited resources.
CONCLUSIONS
Theodor Billroth VI may still stand at an operating table, to the right of his patient, performing open access surgery with a scalpel in his hand. However, he will certainly have vizualized the preoperative data through some kind of computed, digital data–rendering interface, eventually even prior to the patient consultation. Medical liability issues and ongoing subspecialization may have motivated him to consult with colleagues, all of them having access to the same data, and some of these consultations might take place during the surgical procedure. Also, at least some of the indications that today would be treated by open access surgery would in that future be treated by robots using minimal access procedures, and the surgeon might not touch the patient.
It is the task of our generation to preserve the traditional qualities of surgery while keeping an open mind to the benefits of a society-revolutionizing technology.
AUTHOR INFORMATION
Corresponding author: Peter M. Schlag, MD, PhD, Direktor der Klinik für Chirurgie und Chirurgische Onkologie, Robert-Rössle-Klinik am Max-Delbrück-Centrum für Molekulare Medizin, Universitätsklinikum Charité, Medizinische Fakultät der Humboldt Universität zu Berlin, Lindenberger Weg 80, D-13125 Berlin, Germany (e-mail: schlag{at}rrk-berlin.de).
From the Klinik für Chirurgie und Chirurgische Onkologie (Drs Schlag and Moesta) and the Surgical Research Unit OP 2000 (Drs Rakowsky and Graschew), Robert-Rössle-Klinik am Max-Delbrück-Centrum für Molekulare Medizin, Universitätsklinikum Charité, Medizinische Fakultät der Humboldt Universität zu Berlin, Berlin, Germany.
REFERENCES
1. Satava RM. Virtual reality, telesurgery, and the new world order of medicine. J Image Guide Surg. 1995;1:12-16.
FULL TEXT
| PUBMED
2. Satava RM. Cybersurgery: a new vision for general surgery. In: Satava RM, ed. Cybersurgery: Advanced Technologies for Surgical Praxis. New York, NY: Wiley-Liss; 1998:3-14.
3. Jutra A. Teleroentgen diagnosis by means of videotape recording. AJR Am J Roentgenol. 1959;82:1099-1102.
4. Bellaire G, Steines D, Graschew G, et al. OP 2000 and MedSeC: a concept to extend the DICOM standard to include digital stereoscopic video sequences. In: Lehmann et al, ed. Bildverarbeitung für die Medizin. Berlin, Germany: Springer-Verlag; 1998:234-238.
5. Zrimec T. A knowledge-based approach to 3-D reconstruction of human cerebral vasculature. Comput Biol Med. 1998;28:405-413.
FULL TEXT
|
WEB OF SCIENCE
| PUBMED
6. van Leeuwen MS, Obertop H, Hennipman AH, Fernandez MA. 3-D reconstruction of hepatic neoplasms: a preoperative planning procedure. Baillieres Clin Gastroenterol. 1995;9:121-133.
FULL TEXT
|
WEB OF SCIENCE
| PUBMED
7. Schlag PM, Graschew G, Bellaire G, Engel Murke F. Virtual operations on real patients. Langenbecks Arch Surg. 1998;115(suppl):101-104.
8. Remy Jardin M, Remy J, Artaud D, Fribourg M, Duhamel A. Volume rendering of the tracheobronchial tree: clinical evaluation of bronchographic images. Radiology. 1998;208:761-770.
FREE FULL TEXT
9. Ogata I, Komohara Y, Yamashita Y, Mitsuzaki K, Takahashi M, Ogawa M. CT evaluation of gastric lesions with three-dimensional display and interactive virtual endoscopy: comparison with conventional barium study and endoscopy. AJR Am J Roentgenol. 1999;172:1263-1270.
FREE FULL TEXT
10. Yuh EL, Jeffrey RB Jr, Birdwell RL, Chen BH, Napel S. Virtual endoscopy using perspective volume-rendered three-dimensional sonographic data: technique and clinical applications. AJR Am J Roentgenol. 1999;172:1193-1197.
FREE FULL TEXT
11. Meyer CR, Boes JL, Kim B, et al. Demonstration of accuracy and clinical versatility of mutual information for automatic multimodality image fusion using affine and thin-plate spline warped geometric deformations. Med Image Analysis. 1997;1:195-206.
12. Hosten N, Wust P, Beier J, Lemke AJ, Felix R. MRI-assisted specification/localization of target volumes: aspects of quality control. Strahlenther Onkol. 1998;174(suppl 2):13-18.
13. Vahl CF, De Simone R, Albers J, et al. Hyperreal 3-D perioperative visualization of the heart: a prerequisite for virtual operations. In: Schlag PM, Graschew G, eds. Tele- und computergestützte Chirurgie. Berlin, Germany: Springer-Verlag; 1999.
14. Bellaire G, Graschew G, Engel Murke F, Krauss M, Neumann P, Schlag PM. Interactive telemedicine in surgery: fast 3D visualization of medical volume data. Minim Invasive Med. 1999;8:3-6.
15. Scott WW Jr, Bluemke DA, Mysko WK, et al. Interpretation of emergency department radiographs by radiologists and emergency medicine physicians: teleradiology workstation versus radiograph readings. Radiology. 1995;195:223-229.
FREE FULL TEXT
16. Scott WW Jr, Rosenbaum JE, Ackerman SJ, et al. Subtle orthopedic fractures: teleradiology workstation versus film interpretation. Radiology. 1993;187:811-815.
FREE FULL TEXT
17. Rosen MP, Levine D, Carpenter JM, et al. Diagnostic accuracy with US: remote radiologists' versus on-site radiologists' interpretations. Radiology. 1999;210:733-736.
FREE FULL TEXT
18. Mun SK, Levine B, Cleary K, Dai H. Deployable teleradiology and telemedicine for the US military. Comput Methods Programs Biomed. 1998;57:21-27.
FULL TEXT
|
WEB OF SCIENCE
| PUBMED
19. Chandrashekhar AK. Telemedicine in the Gulf War [letter]. Aviat Space Environ Med. 1998;69:918.
20. Berlin L. Malpractice issues in radiology: teleradiology. AJR Am J Roentgenol. 1998;170:1417-1422.
FREE FULL TEXT
21. Whelan LJ. Teleradiology legal issues. J Digit Imaging. 1997;10:17-18.
WEB OF SCIENCE
| PUBMED
22. Shimosato Y, Yagi Y, Yamagishi K, et al. Experience and present status of telepathology in the National Cancer Center Hospital, Tokyo. Zentralbl Pathol. 1992;138:413-417.
PUBMED
23. Weinberg DS. How is telepathology being used to improve patient care? Clin Chem. 1996;42:831-835.
FREE FULL TEXT
24. Weinberg DS, Allaert FA, Dusserre P, et al. Telepathology diagnosis by means of digital still images: an international validation study. Hum Pathol. 1996;27:111-118.
FULL TEXT
|
WEB OF SCIENCE
| PUBMED
25. Winokur TS, McClellan S, Siegal GP, et al. An initial trial of a prototype telepathology system featuring static imaging with discrete control of the remote microscope. Am J Clin Pathol. 1998;110:43-49.
WEB OF SCIENCE
| PUBMED
26. Mairinger T, Netzer TT, Schoner W, Gschwendtner A. Pathologists' attitudes to implementing telepathology. J Telemed Telecare. 1998;4:41-46.
27. Felten CL, Strauss JS, Okada DH, Marchevsky AM. Virtual microscopy: high resolution digital photomicrography as a tool for light microscopy simulation. Hum Pathol. 1999;30:477-483.
FULL TEXT
|
WEB OF SCIENCE
| PUBMED
28. Afework A, Beynon MD, Bustamante F, et al. Digital dynamic telepathology: the virtual microscope. Proc AMIA Annu Fall Symp. 1998:912-916.
29. Schlag PM, Graschew G. A vision of surgery: the concept OP 2000. Langenbecks Arch Surg. 1998;383:194-197.
WEB OF SCIENCE
| PUBMED
30. Bellaire G, Steines D, Graschew G, et al. Standardized access, display and retrieval of medical video. In: Mun SK, Kim Y, eds. SPIE Medical Imaging 1999: Image Display. San Diego, Calif: SPIE; 1999:180-190.
31. Satava RM. Virtual reality and telepresence for military medicine. Ann Acad Med Singapore. 1997;26:118-120.
PUBMED
 CiteULike  Connotea  Delicious  Digg  Facebook  Reddit  Technorati  Twitter
What's this?
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
An Experimental Operating Room Project for Advanced Laparoscopic Surgery
Marvik et al.
SURG INNOV 2004;11:211-216.
ABSTRACT
Robotically Assisted Laparoscopic Cholecystectomy: A Pilot Study
Lomanto et al.
Arch Surg 2001;136:1106-1108.
ABSTRACT
| FULL TEXT
|
