c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 5 ( 2 0 0 7 ) 273–283
journal homepage: www.intl.elsevierhealth.com/journals/cmpb
SINERGIA laparoscopic virtual reality simulator: Didactic design and technical development a,∗ b ´ ´ Pablo Lamata a , Enrique J. Gomez , Francisco M. Sanchez-Margallo , c c d d ´ ´ ´ , Carlos Monserrat , Veronica Garc´ıa , Carlos Alberola , Oscar Lopez e ´ ´ Uson ´ b Miguel Angel Rodr´ıguez Florido , Juan Ruiz e , Jesus a
´ Universidad Polit´ecnica de Madrid (UPM), Grupo de Bioingenier´ıa y Telemedicina (GBT), ETSI Telecomunicacion, Ciudad Universitaria s/n, 28040 Madrid, Spain b Minimally Invasive Surgery Centre, Av. de la Universidad s/n, 10071 Caceres, ´ Spain c Medical Image Computer Laboratory (MedICLab), Universidad Polit´ecnica de Valencia, Camino de Vera s/n, 46022 Valencia, Spain d Laboratorio de Procesado de Imagen, E.T.S.I. Telecomunicacion, ´ Universidad de Valladolid, Edificio de las Nuevas Tecnolog´ıas, Campus Miguel Delibes s/n, 47011 Valladolid, Spain e Centro de Tecnolog´ıa M´edica, Universidad de Las Palmas de Gran Canaria, Pabellon ´ B Edif Telecomunicaciones, Laboratorio 203, Campus de Tafira s/n, 35017 Las Palmas, Spain
a r t i c l e
i n f o
a b s t r a c t
Article history:
VR laparoscopic simulators have demonstrated its validity in recent studies, and research
Received 23 October 2006
should be directed towards a high training effectiveness and efficacy. In this direction, an
Received in revised form
insight into simulators’ didactic design and technical development is provided, by describing
27 November 2006
the methodology followed in the building of the SINERGIA simulator. It departs from a clear
Accepted 12 December 2006
analysis of training needs driven by a surgical training curriculum. Existing solutions and validation studies are an important reference for the definition of specifications, which are
Keywords:
described with a suitable use of simulation technologies. Five new didactic exercises are
Laparoscopy
proposed to train some of the basic laparoscopic skills. Simulator construction has required
Surgical training
existing algorithms and the development of a particle-based biomechanical model, called
Virtual reality
PARSYS, and a collision handling solution based in a multi-point strategy. The resulting VR
Simulation design
laparoscopic simulator includes new exercises and enhanced simulation technologies, and
Biomechanical model
is finding a very good acceptance among surgeons. © 2006 Elsevier Ireland Ltd. All rights reserved.
Collision detection and handling
1.
Introduction
Laparoscopy, the most common minimally invasive surgical technique, emerged 20 years ago and is now firmly embedded in routine surgical practice [1]. Its use has spread to almost all surgical services at hospitals among all over the world. It is already the recommended technique in many procedures, like the cholecystectomy, displacing open surgery. Laparoscopy is
∗
also becoming the standard technique for other pathologies, like those associated with colon and rectum. The birth of this technology with its new concepts and skills has caught unaware many surgical practitioners from the services of hospitals. Whereas some surgeons have made an additional effort to adapt to this new technology, others have rejected it and have lost the opportunity. This change from open to minimally invasive surgical techniques is spreading
Corresponding autor. Tel.: +34 91 550 04 35x30; fax: +34 91 336 68 28. ´ E-mail address:
[email protected] (E.J. Gomez). 0169-2607/$ – see front matter © 2006 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.cmpb.2006.12.002
274
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 5 ( 2 0 0 7 ) 273–283
over surgical procedures and specialities. Today there is no doubt that laparoscopic skills are a principal component of the education of the new surgical residents, and that minimal access is the present and, together with robotics, the future of surgery. Moreover, there is a crescent pressure to have transparent training programs, with objective metrics of surgical skill and alternatives that might be used at any time [2]. Virtual reality (VR) simulators are a valuable tool for training and skills’ assessment [3]. Skills learnt with simple VR laparoscopic simulators can be transferred to the operating room (OR) environment [4,5]. One recent meta-analysis has concluded that VR training reduces time and errors, and that it is a valid tool to differentiate expertise levels [6]. The main goal of every surgeon is to improve patient’s safety, and surgical simulators can play a main role for it [7]. Nevertheless, there lack clinical trials that demonstrate the added value of VR training over traditional alternatives. Nowadays new VR simulation technologies are developed, and serious efforts are being made towards the construction of high fidelity systems for surgical training and skills assessment [8]. But little is known about the actual requirements of simulators in order to be effective and efficient training tools [9]. There are even doubts about the better efficiency of VR compared to physical trainers [10]. The development of the first VR simulator in 1997 [11] had to wait 5 years to see the first results about the transfer of skills to the operating theatre, which have been considered a landmark [5]. Now that we have the “proof of concept” clear it’s a time to reconsider the optimal design of simulators looking for the best training effectiveness. Within this context, the approach of this work is to analyse the training needs and the key simulation requirements in order to illuminate the design of surgical simulators and its technical development. Two main contributions arise from this global aim: (1) a specification of didactic exercises which has reported the creation of a new training environment; (2) an appropriate use of simulation technologies that has significantly improved the whole simulation. Thus, this is the aim of the SINERGIA Spanish Collaborative Network (G03/135): to develop new laparoscopic training solutions based in VR technologies by means of the design of new exercises and the enhancement of simulation. The article is therefore divided in two aspects: Section 2 describes the definition of simulation requirements and specifications which conform the didactic design of a basic skills laparoscopic simulator; Section 3 addresses the construction of the simulator, the development of biomechanical models and collision detection and handling strategies in order to implement the specifications. Finally, a discussion section precedes some concluding remarks.
2. Didactic design of a laparoscopic simulator The design process of a VR didactic laparoscopic simulator could be defined as a biomedical engineering task of finding a solution that covers certain surgical training needs by specifying didactic exercises with suitable fidelity using available simulation
resources. Therefore it begins with a clear definition of the training objectives and needs. And these requirements have to be translated into simulation specifications of different exercises regarding the capabilities of simulation technologies. The definition of required simulation fidelity is a very controversial and crucial issue in this process.
2.1.
Insight into the simulation design process
There is little specific literature about how to develop an efficient didactic design of a simulator. It can be found that an ergonomic task analysis [12] was used for the design of the ¨ MIST-VR (Mentice Inc., Goteborg, Sweden), but without any further detail. In a previous work, simulation specifications were divided into perceptual motor skills, spatial skills, and critical steps of surgical procedures [13]. There are several works like this that address some of the aspects of a design process of a VR surgical simulator. This section provides an insight into this process, structured in its three main pillars: (1) a comprehension of the human factors in the laparoscopic interaction, (2) an understanding of VR capabilities and (3) the knowledge of validation results of current solutions. Human beings have perceptual limitations of the sensory, motor and cognitive system. Laparoscopy is characterised by a loss of sensory stimuli of the surgeon, which leads to the need of developing new skills. An interesting man–machine description of this interaction can be found in [14], and a systematic analysis of the visual perceptual aspects in [15]. Knowing and understanding how surgeons interact in the surgical theatre and develop their skills is an important issue in order to address the design of a surgical simulator. This contributes for the definition of the required degree of simulation fidelity, what is a very controversial issue. For example, it is unknown the role of force feedback in surgical training [2]. Comprehension of the laparoscopic interaction leads also to the definition of objective metrics of surgical skill [16,17]. In this field, our SINERGIA consortium has recently analysed how surgeons perceive tissue consistency, which has led to a better definition of the “visual haptics” concept [18]. Results of this study are also the grounds for one of the new didactic exercises, “Force sensitivity”, as explained later. Simulation technologies enable new interaction paradigms and new possibilities for training. In other words, VR simulators can be conceived as training means built using different didactic resources. These resources are classified in three main categories [19] based upon the extent to which simulators: (1) emulate reality (fidelity resources); (2) exploit computer capabilities such as new ways of interaction and guidance (teaching resources); (3) measure performance and deliver feedback (assessment resources). An important issue is to identify which technical aspects make a simulator a good training tool, to identify the suitable use of these didactic resources. Finally, it has to be regarded current solutions and their validation results. It has been demonstrated how skills acquired with a very basic alternative, MIST-VR, are transferred to the operating room [4,5]. This is a key result that validates the hypothesis that a simulator does not require a high level of fidelity to be an effective training tool. Results can also provide information about which exercises in a simulator have
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 5 ( 2 0 0 7 ) 273–283
Fig. 1 – Training curriculum of the Minimally Invasive ´ Surgery Centre of Caceres (Spain). The SINERGIA simulator is conceived to be introduced at the end of the first training stage.
a more consistent validity like the “Manipulate & Diathermy” task of MIST-VR [20,21]. Therefore, several tasks from “basic skills” packages of commercial simulators have been adopted in the design of the SINERGIA VR laparoscopic simulator.
2.2.
Didactic design of simulation tasks
Training objectives and needs of the SINERGIA laparoscopic simulator have departed from the long-term experience of the ´ Minimally Invasive Surgery Centre of Caceres, Spain. This Centre has a thoroughly validated methodology of training based in a pyramidal structure [22] divided in four levels (see Fig. 1): (1) basic and advanced skills with box trainers, (2) anatomical protocols and advanced skills with animal models, (3) advanced procedural skills with tele-surgical applications and (4) practice in the operation room. This curriculum has therefore driven the development of the SINERGIA laparoscopic simulator, which has been conceived as a training means for the first level in the pyramidal training model. An analysis of the laparoscopic skills acquired at this stage has led to the definition of seven didactic units: hand–eye coordination, camera manipulation, grasping, pulling, cutting, dissection and suture. Several didactic exercises are developed in each of these units as explained next. They are summarised in Table 1. The first didactic unit, hand–eye coordination, has the objective of learning how to orientate tools in the laparoscopic space and to displace them with precision. Two coordination exercises, “Coordination” and “Speedy Coordination”, are defined to practice and meet this objective. These exercises are similar to those offered by other commercial simulators like Lap¨ Sim (Surgical Science, Gotegorg, Sweden). This unit requires a perfect correspondence between physical handles and dis-
275
played tools. No other critical resource is needed, whereas a similitude to an organic environment is desirable. Teaching resources are used to guide the task: a colour code is defined to indicate if user has to use the right tool (dark blue), the left tool (light blue) or both tools simultaneously (yellow). This code will be common in all exercises of the simulator. The orientation unit is designed for learning to manipulate the endoscope and to orientate it. It offers two exercises, “Navigation” and “Navigation & Touch”, to practise with the manipulation of the endoscope, the laparoscopic camera. The first exercise is similar to others offered by commercial simulators, but the second includes a new approach since it is designed for training of the “blind insertion”, a skill that enables surgeons to guide a tool to a right region without seeing it on the monitor. Realistic anatomical models and textures are convenient to make users familiar with how anatomy is visualised in laparoscopy. Once a trainee has learnt to orientate tools practising with former exercises, grasping unit addresses the basic skills involved in the grasping of objects. Two exercises are defined for developing: (1) an accurate grasping (in a precise point in the 3D space) and (2) a skill of performing coordinated grasping manoeuvres in order to transfer an object. First exercise, “Accurate Grasping”, is developed with a virtual thread that has to be grasped and released accurately and delicately. A formative constructive feedback is delivered to the trainee by the result of the interaction with the thread: task is done right when no deformation is caused to the thread after grasping and releasing it. This is a challenging issue for the trainee, who can be then further motivated in order to get it, and also allows defining accuracy as the deformation caused to the thread when grasping. Moreover, difficulty of task can be tuned by setting thread ends fixed or not. In a fixed ends mode the thread will come back to its rest position gradually after being deformed, and in a free ends mode the thread will fold as the user makes mistakes, making the task more difficult. Therefore, strength of this exercise relies in the use of these features, these teaching and assessment resources. Further details of this original task are provided in [23]. Second exercise, “Grasp & Transfer”, addresses the coordination skill of transferring objects, in a similar way to the “Transfer & Place” task from MIST-VR simulator. Pulling unit is addressed to two objectives related to the pulling manoeuvre. The first is learning how to perform a symmetric pulling, what is required when tightening an intracorporeal knot. The second is the acquisition of the sensibility to differentiate tissue consistencies, different resistances of tissues against pulling. “Coordinated pulling” exercise is built using a virtual thread, which provides interesting features for improving training like the incorporation of a coordination control-bar, a rod attached to the thread that delivers formative constructive feedback to user about the symmetry of pulling. This control bar is inclined if user pulls more from one end of thread than form the other. This is also used to define a coordination metric as the percentage of the total path travelled by the coordination control-bar in which the bar’s inclination is smaller than a given threshold. Further details about this original exercise are provided in [23]. On the other hand, “Force sensitivity” exercise offers the novel aspect of training users’ sensibility to pulling forces. VR have the strength of enabling
276
Table 1 – Didactic exercises of the SINERGIA VR laparoscopic simulator Exercise
Task
Evaluation metrics
Parameters of difficulty
Touch a set of static balls that appears sequentially in an “organic scene”. There is a time limit to touch each of the balls
(1) Time; (2) distance by each tool and efficiency; (3) errors (wrong tool, harm to backgr.); (4) fulfilment.
(1) Size of balls; (2) time limit; (3) number of balls; (4) geometry of “organic scene”
Speedy coordination
Touch a set of moving balls that appears all together in an “organic scene”. Black balls must be avoided
(1) Time;(2) distance travelled by each tool;(3) errors (wrong tool, wrong ball, harm to background)
(1) Size of balls; (2) number of balls; (3) geometry of “organic scene”
Navigation
To centre endoscope sight in spheres that sequentially appears
(1) Time;(2) distance travelled by each tool andficiency; (3) errors (collision with anatomy)
(1) Size of spheres; (2) time limit; (3) anatomy complexity and movements
Navigation and touch
Centre endoscope sight in spheres and then lead there a tool held with the other hand
Accurate grasping
Grasp certain points of a thread without causing deformations to it. Grasp area appears sequentially. There is a time limit
(1) Time; (2) distance by each tool and efficiency; (3) mistakes in grasping (out marked area, wrong tool); (4) accuracy
(1) Grasping area size; (2) time limit; (3) fixed or free ends mode
Grasp and transfer
Grasp a cylinder which appears lying on an “organic scene”, transfer to the other tool and release it in a marked area
(1) Time; (2) distance travelled by each tool; (3) mistakes (wrong transfer, wrong release)
(1) Size of objects; (2) time limit; (3) size of release area
Coordinated pulling
Grasp thread at marked points and pull them following the white path until the big spheres. A “coordination-control bar” provides formative feedback
(1) Time; (2) distance travelled by tools; (3) coordination
(1) Different pulling paths; (2) time limit; (3) bar inclination sensibility; (4) bar inclination evaluation threshold
Force sensitivity
Grasp and pull different virtual tissue samples and rank its consistency answering a set of comparative questions
(1) Success rate
(1) Magnitude of difference of stiffness between tissues
Accurate cutting
Cut a set of cylinders in marked lines after pulling them
(1) Time; (2) distance by tools; (3) cut accuracy; (4) errors: lack of tension in exposure, tearing
(1) Tearing sensibility
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 5 ( 2 0 0 7 ) 273–283
Coordination
Screen capture
(1) Tearing sensibility; (2) complexity of drawn pattern
(1) Tearing sensibility
(1) Tearing sensibility
(1) Slippery behaviour of thread
(1) Time; (2) distance travelled by tools; (3) cut accuracy; (4) errors: lack of tension in exposure, tearing
(1) Time; (2) distance by tools; (3) errors: harm colliding objects, tearing in exposure, tearing in dissection
(1) Time; (2) distance by tools; (3) errors: tear a joint, harm cylinder or surface, cautery more than one joint at once
(1) Time; (2) distance travelled by tools
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 5 ( 2 0 0 7 ) 273–283
controlled and repeatable force stimuli. Different tissue samples with unknown simulated stiffness are offered to trainees, who are asked to rank their consistency. The grounds of this exercise is the results found in a recent analysis of tissue consistency perception [18]. Cutting unit addresses training of cutting skills, stressing the issue of making this manoeuvre with a right tissue exposure, a right tension, and accurately in two exercises. “Accurate cutting” requires the coordination of both hands to tighten and cut a set of cylinders. This is similar to the “Cutting” exercise from LapSim simulator. Teaching resources can be used to inform the user when tissue is been tightened too little (bad cut) or too much (risk of tearing). A “Continuous cutting” following drawn patterns in a virtual canvas is the second exercise. The different orientations and shapes in these patterns are used to cover the different spatial orientations for this task as well as to increase the level of difficulty. Two interesting teaching resources can be used to guide a correct tissue exposure: (1) spheres to indicate the spatial point where the tip of the tool grasping the tissue has to move to, and (2) growing semitransparent spheres as a metaphor of pulling forces, which changes to a red colour when there is a risk of tearing. These resources are already present in the suturing task of the SEP (SimSurgery, Oslo, Norway). Dissection unit is designed to train two different ways of dissecting organs: making opening manoeuvres with dissectors or scissors (blunt dissection) and using a cautery hook. “Blunt dissection” offers a controlled virtual environment to practise this skill. There is no anatomical fidelity, whereas geometrical shapes focus trainees’ attention on the task. “Hook dissection” trains the skill of hooking and pulling small portions of tissue to cautery them. It is designed with an abstract approach like the “Precise dissection” of LapSim. Finally, suturing unit addresses suturing training, the most complex skill of the whole package. It is focused into the knotting skill, the learning to make and intracorporeal knot. Further details are provided in [23].
Cut a surface following drawn pattern after exposing it
Separate two structures joined by three conjunctive layers using dissection manoeuvres with a right tissue exposure, and without harming organs
Hook and cautery joints of a cylinder one by one, with a right tension and without harming the cylinder or the surface
Make an intracorporeal knot
Blunt dissection
Hook dissection
Knotting
3.
Continuous cutting
277
Technical development
Simulating an operating scene is very complex, much more complicated than flight simulation. The main difference is the great difficulty of modeling living organs and the interaction with them in real time: visual update rate must be around 25 Hz, and haptic update rate 300 Hz [24,25]. An interesting overview about surgical simulation technologies can be found in [8]. There are basically three elements in a laparoscopic simulator: a haptic interface that behaves like laparoscopic tools (in our case the Laparoscopic Surgical Workstation from Immersion Medical, Gaithersburg, USA), a monitor that shows the surgical scene and a computer that manages these two interfaces and the virtual models of tissues, organs and tools. The software running in the computer has four main modules (see Fig. 2): (1) the biomechanical model that calculates the deformation and the behaviour of the organ in the virtual scene, (2) the collision module, which calculates the interaction between the virtual models and handles this information to other modules, (3) a visual motor that renders the geome-
278
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 5 ( 2 0 0 7 ) 273–283
Fig. 2 – The main generic modules of a laparoscopic VR surgical simulator.
try in the visual device (screen), and (4) a haptic motor, which reads the positions of the haptic device and returns the haptic forces to the user. Sometimes some of these modules are integrated, like a biomechanical which integrates the collision detection [26]. Details about these modules are provided in following sections. Another basic concept in simulation is the simulation loop. A real-time behaviour is emulated by the repetition of a cycle that controls the surgical scene. A generic simulation loop would make four principal steps: (1) read the positions of the tools represented by the haptic device, (2) detect the collisions between the elements in the scene and calculates the response of these collisions, (3) calculate the deformation and topological changes of organs and tissues with the deformable models that represents them and (4) display the new geometry and the reaction forces resulted from the deformation process. The time step is defined as the time taken to complete a simulation loop. Therefore “real-time” means that the simulation loop should be done 300 times/s for having a good haptic interactivity, or at least 25 times for a visual rendering. Due to the big difference between these two update rates, it would not be efficient to build a simulation system running so fast to satisfy both at the same time as presented in [27]. This is the reason why simulation has usually two loops, the haptic one and the visual one [28,29]. Some final technical details are that the simulator has been developed in C++ language, with WTK libraries (WorldToolKit, Engineering Animation Inc., Mill Valley, CA-USA) and in a Windows environment.
3.1.
Biomechanical model
Deformable behaviour of objects and organs are simulated with a T2-Mesh mass-spring model [30]. Nevertheless an interesting alternative, called ParSys [31], has been developed and analysed in order to be included in the simulator. A T2-Mesh model is a surface model that seeks for simplicity and speediness of calculi, in detriment of a realistic behaviour. The model defines a set of nodes in the surface of objects, and a mass is assigned to each of them. These nodes are also linked with linear springs which act as energy storage and react against deformations. The equations’ system is relatively small and, therefore, fast in its resolution. Nevertheless it is an iterative model and consequently mined by the risk of instabilities and oscillations, which depends
on the choice of integration periods and mass and springs parameters. Furthermore, these parameters are defined in an heuristic manner, and it is difficult to find a clear relation with actual biomechanical tissue properties. In order to enhance the behaviour of deformable organs, it is developed a new model: ParSys. This is a link-volume model [26], a different alternative to the traditional mass-springs and finite element methods. These models are composed by a set of interconnected volumetric elements, also called particles. Volumetric behaviour is given and guaranteed by its structure, and volume conservation by the constant number of particles of an object. Internal energy of the model is represented by Lennard–Jones potential fields. ParSys has notably enhanced the stability and volumetric behaviour offered by the T2-Mesh model (see an example in Fig. 3). It is also a fast model, and it allows a simple management of topological changes. Finally, it has the advantage of the direct use of real biomechanical parameters. This is therefore the model chosen for solid objects, and the T2-Mesh for hollow ones. On the other hand, three of the didactic exercises have been implemented with a virtual suturing model [23]: accurate grasping, coordinated pulling and knotting. Chosen model has been a simple but effective alternative described in [32]. The thread of length L is built as a set of n cylindrical segments of length Lseg = L/n and radius R = Lseg /2, which join a pair of consecutive nodes (see Fig. 4a). Those n + 1 nodes propagate the movement by means of the algorithm FTL (Follow the Leader), which translates into the new positions the restrictions originated by: (1) the collisions in the previous cycle of simulation and (2) the nodes that the user grasped. A scheme is shown in Fig. 4b.
3.2.
Collision detection and handling
As explained before, simulating a surgical scene is a complex task not only because of the difficulty to model and reproduce biomechanical behaviour, but also because of the need of detect and handle collisions between organs. Nevertheless, this second aspect seems to have caught much less attention in surgical simulation. A clear distinction must be made between the detection and the handling of collisions, between the determination of overlapping regions and the way these overlap situations are solved. For the former, our first approach to the problem has been the use of available software, a library developed by Sundaraj et al. [8]. However, this software does not deal with topological changes occurred in the deformable model, a situation that arises in cutting or dissection gestures. In order for these operations to be manageable with our simulator without loss of efficiency, a collision detection library has been developed by the consortium, which tests geometrically the interaction between tools (rigid objects) and deformable objects, and does manage topological changes. Therefore, our major effort has been focused on collision handling. Basically, collision handling can be divided into methods that rely on the biomechanical model of the objects to be handled and those which provide a geometric solution. As for the former the new object’s position and shape, as a result of a collision, is controlled by the model equations, with
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 5 ( 2 0 0 7 ) 273–283
279
Fig. 3 – ParSys model: (a) triangular liver model (left) with its internal particle system that governs its deformation (right); (b) example of a liver deformation after a contact with a laparoscopic tool.
Fig. 4 – Modeling of a virtual thread: (a) detailed configuration of the thread; (b) FTL scheme. The old positions and both restrictions (grasped nodes and collision) are the inputs to the FTL algorithm so that the outputs are the new positions of the thread.
restrictions (or penalties) given by the kinematics of the collision [33,34]. As for the latter, and assuming that the objects are modelled by means of a triangular mesh, the problem is posed as finding the new positions of the vertices of the triangles involved in the collision. Then, the biomechanical model will calculate how the deformations that stem from the collision propagate throughout the object. If real time constraints are an issue, collision handling method based on biomechanical models may be somewhat slow, due to the amount of calculations to be carried out. Since this is our case, we have resorted to a handling that finds the new vertices positions. Within this approach, many proposals found in the literature deal with only a single point in the surgical tool [35,36]. This may lead to unrealistic effects as those shown in Fig. 5. A new 3D collision handling strategy is therefore developed and implemented. Evolution and details of its development can be found in [37]. However, it is worth mentioning that it is based on the tool kinematics (the tool velocity vector) and the normals to the triangles involved in the collision. Therefore, each of its vertexes is displaced out of the tool bearing in mind the fuzzy nature of the tool motion, which is modelled as penetration or sliding. Fig. 6 depicts the displacement vectors of each vertex computed by
280
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 5 ( 2 0 0 7 ) 273–283
Fig. 5 – Snapshots of the result of collision handling strategies. (a) A single point handling, which results in the upper part of the tool apparently gone through the simulated tissue. (b) A multipoint handling, with which the whole tool is pushing the tissue down.
means of the proposed approach. Finally, the biomechanical model of the organ recalculates how this deformation affects nearby triangles. It could also deal with possible break-ups of the tissue. Many other situations like the one encounter here may be found during the simulation. Those identified by us are properly dealt with in a similar manner as the one just described.
3.3.
Surgical scenarios graphical design
Surgical scenarios suppose an important contribution in surgical training, and a highly realistic environment that familiarizes novice surgeons with the real surgical scene or environment [38]. Furthermore, the design of simple scenarios to enhance or teach some surgical skills, such as dissection
Fig. 6 – Multipoint collision handling. Vector displacement of each collided vertex.
or suturing, allows to train surgeons to develop some abilities before a surgical intervention. Surgical scenarios are composed of 3D virtual models that represent actors (organs) in a surgical procedure. These models can be generated from real multidimensional medical data (CT, MRI, US, etc.) [39], using 3D software or a combination of both. Even though the first option fixes better the real characteristics, the second one is more flexible and allows to customize 3D models adapted to the clinical requirements. Anyway, both alternatives can be used in our work. Many software packages exist for 3D modeling, in this work we have used an open source and multi-platform software, Blender [40]. This software package presents some points of interest for our project: (1) open source and multi-platform; (2) tools for 3D design and modelling; (3) a command interface under python that allows to include scripts to modify the 3D scene. Based on these points, we have found the following advantages: (1) ability to share 3D scenes from different platforms; (2) feasibility to connect models generated from real medical images to a 3D modelling software. We have used Blender to design scenes (included exercises) and add the effects of texture and realism. Additionally, we have done a 3D models library of pig’s organs. In Fig. 7, an example of these surgical scenes is shown. It displays a step of
Fig. 7 – Example of a virtual surgical scene which represents a step of a Nissen fundoplication.
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 5 ( 2 0 0 7 ) 273–283
the Nissen fundoplication, specifically used in the navigation unit.
4.
Discussion
This article brings forward the design and development process followed in the construction of the SINERGIA VR simulator. Literature about this approach is scarce, with vague information about existing solutions like the reference to the ergonomic task analysis of MIST-VR [12]. We believe that this is a key aspect in this field that has not been properly addressed before. Definition of design specifications of a surgical simulator can be a creative and attractive process. Requirements, training objectives, can be translated into a wide variety of didactic exercises making use of different didactic resources of VR technologies. But this process has to regard the limits of these technologies; it is not possible to simulate everything. A good designer is an expert in understanding both the clinical needs and the VR capabilities.
4.1.
Designing a simulator
The first crucial point in the design of a laparoscopic simulator is to realise that it is not necessary to emulate a perfect virtual patient. The aim is to build a training means, or even a skills’ assessment tool, and these goals do not require a completely realistic environment. The design must therefore depart from a clear statement of the needs of training curricula. Nevertheless, definition of each training objective to be met is not as precise as what would be desirable for simulation design. For example, the specification of what is a “good and delicate dissection” is not straightforward with measurable and objective parameters, and this is an important information to be translated into simulation specifications. Unfortunately, there seems not to be a means to address a more objective definition of skills and their relevant metrics. Probably, an a priori solution is not possible, and only validation studies and clinical trials can make objective this knowledge. In fact, the development and validation of surgical VR simulators is providing a better understanding of the different components of a surgical skill, and a means to characterise and objectively define them. Therefore, one important source of information is the review of existing solutions for training basic skills and their validation results. There are very interesting solutions in VR simulators already in market. Many of them have been included to improve proposed didactic design, like the “Manipulate & Diathermy” task of MIST-VR which has shown the most consistent construct validity results [20,21]. The proposed design uses the background of current simulators and incorporates new ideas and aspects. These are the training of “blind insertion”, the use of a virtual thread to train an “accurate grasping” and a “coordinate pulling” with interesting ways of delivering constructive feedback, the incorporation of a “continuous cutting” following a pattern and a simple model to practice a blunt dissection. Finally, the training of the force sensory capabilities (see the “Force sensitivity” exercise) explores new a facet of the perceptual motor skills [13].
4.2.
281
Implementing a surgical simulator
VR technologies offer limited capabilities in simulating the interaction with living organs [8]. Therefore, the question is: “is it possible to build a surgical simulator as it has been specified?” The answer is yes, but with some difficulties partially solved such as simulation of a continuous cutting and simulation of a blunt dissection. A continuous cutting involves a precise collision detection and handling between scissors and tissue. Some surgeons even claim that it is crucial to stroke a tissue before each cut, and a good realism in this manoeuvre would require high resolution models and efficient methods. On the other hand blunt dissection requires both the simulation of conjunctive tissue, those little and feeble joints between organs, and the simulation of how these joints are broken with the opening of a dissector. Realism of harm thresholds for the development of delicateness in surgical manoeuvres is an issue that is regarded only as a challenge. And, what about the simulation of interaction forces? Proposed simulation specifications only use them in the “force sensitivity” basic task, and a simple fuzzy model is considered to be enough regarding conclusions of prior experiences. An issue to be enhanced is the mechanical design of haptic interfaces in order to make them mechanically transparent and reduce its cost. Nevertheless, it must be said that simulation technologies have perfectly solved other important aspects. Tracking of tools’ movements offers enough resolution, and enables all the surgical evaluation issues related with them. This is one of the main added values of a computer-enhanced simulator. Interaction with basic geometries is quite well managed, and this is enough for a VR surgical simulator to offer a training value like validation results of MIST-VR has demonstrated [4,5]. These two fidelity resources might be enough for training all motor skills involved in laparoscopic surgery. Finally, it is important to regard the issue of the required fidelity in a simulator in order to be a useful training tool. Implementation of dissection with a cautery hook can be abstracted like the “Manipulate & Diathermy” task of MIST-VR, or it can be simplified with a button joint to the ground with several threads like the “Precision dissection” task of LapSim, or it can even be simulated with a high level of realism in a cholecystectomy like the “Lap Chole” task of LapMentor. Are there differences between these training alternatives? Initial motivation of trainees is higher with a realistic environment, is this motivation kept? Future research should be focused not only in the efficient use of simulation technologies, but also in the extension of this teaching means to a wider scope of educative goals and the improvement of the technology itself.
4.3.
Validation approach
There are several validation strategies defined in the literature, and many results of different current surgical simulators. A two-step validation strategy is proposed as the better alternative to address this issue for a new didactic design. It consists of small content validity studies with selected surgeons and of extensive studies for defining proficiency levels.
282
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 5 ( 2 0 0 7 ) 273–283
Design process and construction of a surgical simulator is long, and it is convenient to have a close communication with physicians in order to orientate it in the right direction. Validation has to be taken into account in this process since the very beginning, and to assure that the simulator makes sense. This is reached with content validity study sessions in which experts in surgical training and teaching reviews the didactic design. This constitutes the first progressive stage in the proposed validation strategy. Proposed design of a “basic skills” laparoscopic VR simulator has passed several content validities studies, which have been performed in individualised interviews with experts in surgical training. The result of these studies is the final version of the didactic design, which has been described in Section 2. Once the surgical simulator is built, studies can be performed to assess its construct validity or its transfer of skills to the operating room. But proposed second step is to go directly for what could be the last validation result: the proficiency levels definition. Once stated, simulator scores will indicate where trainees are in the learning curve, and the “remaining amount of learning” required finalising the training program and getting the degree. This value is desired currently, and it is starting to be obtained in recent works [41–43]. It would be desirable to contribute for this issue, what is another reason for facing proficiency levels characterization directly in this two-step validation approach, what we are currently addressing. Specifically, surgeons of different degrees of expertise (novel, intermediate and experts) are performing controlled sessions with the simulator, and their scores are being stored and analysed. Experiments are being done in different hospitals and in training workshops in laparoscopic surgery organized by the MISC (Minimally Invasive Surgery ´ Centre) of Caceres (Spain). Preliminary results are promising, and the simulator is finding a very good acceptance among surgeons and residents.
5.
Conclusion
An insight into the processes of didactic design and technical development of VR simulators has been provided. The two key points are: (1) the specification of a didactic design which departs from the clinical need, and (2) the rational use of simulation technical resources. The resulting SINERGIA simulator represents a new alternative for basic laparoscopic training, with original VR exercises. It also incorporates new collision and biomechanical models, and it is finding a very good acceptance among surgeons.
Acknowledgements This research has been partially funded by the SINERGIA Thematic Collaborative Network (G03/135) of the Spanish Ministry of Health and the SIMILAR NoE (IST-2002-507609). Our spe´ cial thanks for their contributions to M. Antol´ın, C. Rincon, ´ S. Rodr´ıguez, P.J. Figueras, A. Cano and P. Sanchez (Grupo de Bioingenier´ıa y Telemedicina, Madrid), J.B. Pagador and all the staff of the Minimally Invasive Surgical Centre (CCMI), ˜ (MedICLab of Valencia), E.Munoz ˜ M. Alcaniz (Laboratorio de
Procesamiento de la Imagen, Valladolid), M. Bernal and D. ´ ´ Suarez (Centro de Tecnolog´ıa Medica de Las Palmas de Gran Canaria) and to F. Lamata (Hospital Universitario de Zaragoza).
references
[1] A. Cuschieri, Laparoscopic surgery: current status, issues and future developments, Surgeon 3 (2005), 125–30, 132–3, 135–8. [2] R. Kneebone, Simulation in surgical training: educational issues and practical implications, Med. Educ. 37 (2003) 267–277. [3] A.G. Gallagher, C.D. Smith, S.P. Bowers, N.E. Seymour, A. Pearson, S. McNatt, D. Hananel, R.M. Satava, Psychomotor skills assessment in practicing surgeons experienced in performing advanced laparoscopic procedures, J. Am. Coll. Surg. 197 (2003) 479–488. [4] T.P. Grantcharov, V.B. Kristiansen, J. Bendix, L. Bardram, J. Rosenberg, P. Funch-Jensen, Randomized clinical trial of virtual reality simulation for laparoscopic skills training, Br. J. Surg. 91 (2004) 146–150. [5] N.E. Seymour, A.G. Gallagher, S.A. Roman, M.K. O’Brien, V.K. Bansal, D.K. Andersen, R.M. Satava, Virtual reality training improves operating room performance: results of a randomized, double-blinded study, Ann. Surg. 236 (2002) 458–463. [6] S. Haque, S. Srinivasan, A meta-analysis of the training effectiveness of virtual reality surgical simulators, IEEE Trans. Inf. Technol. Biomed. 10 (2006) 51–58. [7] L.S. Feldman, V. Sherman, G.M. Fried, Using simulators to assess laparoscopic competence: ready for widespread use? Surgery 135 (2004) 28–42. [8] A. Liu, F. Tendick, K. Cleary, C. Kaufmann, A survey of surgical simulation: applications, technology, and education, Presence 12 (2003) 599–614. [9] J. Dankelman, M.K. Chmarra, E.G.G. Verdaasdonk, L.P.S. Stassen, C.A. Grimbergen, Fundamental aspects of learning minimally invasive surgical skills, Minimal. Invas. Therap. Allied Technol. 14 (2005) 247–256. [10] Y. Munz, B.D. Kumar, K. Moorthy, S. Bann, A. Darzi, Laparoscopic virtual reality and box trainers: is one superior to the other? Surg. Endosc. 18 (2004) 485–494. [11] M.S. Wilson, A. Middlebrook, C. Sutton, R. Stone, R.F. Mccloy, MIST VR: a virtual reality trainer for laparoscopic surgery assesses performance, Ann. Royal College Surgeons Engl. 79 (1997) 403–404. [12] R. Stone, R. McCloy, Ergonomics in medicine and surgery, BMJ 328 (2004) 1115–1118. [13] F. Tendick, M. Downes, T. Goktekin, M.C. Cavusoglu, D. Feygin, X. Wu, R. Eyal, M. Hegarty, L.W. Way, A virtual environment testbed for training laparoscopic surgical skills, Presence 9 (2000) 236–255. [14] H.G. Stassen, J. Dankelman, K.A. Grimbergen, D.W. Meijer, Man–machine aspects of minimally invasive surgery, Annu. Rev. Contr. 25 (2001) 111–122. [15] J. Hofmeister, T.G. Frank, A. Cuschieri, N.J. Wade, Perceptual aspects of two-dimensional and stereoscopic display techniques in endoscopic surgery: review and current problems, Semin. Laparosc. Surg. 8 (2001) 12–24. [16] J. Rosen, M. Solazzo, B. Hannaford, M. Sinanan, Task decomposition of laparoscopic surgery for objective evaluation of surgical residents’ learning curve using hidden Markov model, Comput. Aid. Surg. 7 (2002) 49–61. [17] J. Rosen, B. Hannaford, C.G. Richards, M.N. Sinanan, Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating
c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 8 5 ( 2 0 0 7 ) 273–283
[18]
[19]
[20]
[21]
[22]
[23]
[24] [25] [26]
[27]
[28]
[29]
[30]
[31]
surgical skills, IEEE Trans. Biomed. Eng. 48 (2001) 579– 591. ´ ´ P. Lamata, E.J. Gomez, F.M. Sanchez-Margallo, F. Lamata ´ ´ Gargallo, Study of consistency Hernandez, F. del Pozo, J. Uson perception in laparoscopy for defining the level of fidelity in virtual reality simulation, Surg. Endosc. 20 (2006) 1368–1375. P. Lamata, F. Bello, R.L. Kneebone, R. Aggarwal, F. Lamata ´ Hernandez, E.J. Gomez, Conceptual framework for the analysis, design and evaluation of laparoscopic virtual reality simulators, IEEE Comput. Graph. 26 (2006) 29–39. T.P. Grantcharov, L. Bardram, P. Funch-Jensen, J. Rosenberg, Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills, Am. J. Surg. 185 (2003) 146–149. T.P. Grantcharov, J. Rosenberg, E. Pahle, P. Funch-Jensen, Virtual reality computer simulation—an objective method for the evaluation of laparoscopic surgical skills, Surg. Endosc. 15 (2001) 242–244. ´ ´ S. J. Uson, F.M. Sanchez-Margallo, S. Pascual Sanchez-Gij on, ´ en Cirug´ıa Laparoscopica ´ Climent, Formacion Paso a Paso. 2005. P.J. Figueras Sola, B.S. Rodriguez, P. Lamata, J.B. Pagador, F.M. Sanchez-Margallo, E.J. Gomez, Virtual reality thread simulation for laparoscopic suturing training, Stud. Health Technol. Inform. 119 (2006) 144–149. G. Burdea, Force and Touch Feedback for Virtual Reality, John Wiley & Sons, New York, 1996. H. Delingette, Toward realistic soft-tissue modeling in medical simulation, Proc. IEEE 86 (1998) 512–523. S. Gibson, C. Fyock, E. Grimson, T. Kanade, R. Kikinis, H. Lauer, N. McKenzie, A. Mor, S. Nakajima, H. Ohkami, Volumetric object modeling for surgical simulation, Med. Image Anal. 2 (1998) 121–132. S. Cotin, Real-time elastic deformations of soft tissues for surgery simulation, IEEE Transact. Visual. Comput. Graph. 5 (1999) 62–73. M.C. Cavusoglu, F. Tendick, Multirate simulation for high fidelity haptic interaction with deformable objects in virtual environments, IEEE Int. Conf. Robot. Automat. (ICRA 2000) (2000) 2458–2464. G. Picinbono, J.C. Lombardo, H. Delingette, N. Ayache, Improving realism of a surgery simulator: linear anisotropic elasticity, complex interactions and force extrapolation, J. Visual. Comput. Anim. 13 (2002) 147–167. U. Meier, O. Lopez, C. Monserrat, M.C. Juan, M. Alcaniz, Real-time deformable models for surgery simulation: a survey, Comput. Meth. Progr. Biomed. 77 (2005) 183–197. M. Pithioux, O. Lopez, U. Meier, C. Monserrat, M.C. Juan, M. Alcaniz, ParSys: a new particle system for the introduction of on-line physical behaviour to three-dimensional synthetic objects, Comput. Graphics—UK 29 (2005) 135–144.
283
[32] J. Brown, S. Sorkin, J.C. Latombe, K. Montgomery, M. Stephanides, Algorithmic tools for real-time microsurgery simulation, Med. Image Anal. 6 (2002) 289–300. [33] J. Platt, A. Barr, Constraints methods for flexible models, SIGGRAPH ’88, in: Proceedings of the 15th Annual Conference on Computer Graphics and Interactive Techniques, ACM Press, New York, NY, USA, 1988, pp. 279–288. [34] A. Deguet, A. Joukhadar, C. Laugier, Models and algorithms for the collision of rigid and deformable bodies, WAFR ’98, in: Proceedings of the Third Workshop on the Algorithmic Foundations of Robotics on Robotics: the Algorithmic Perspective, A.K. Peters, Ltd., Natick, MA, USA, 1998, pp. 327–338. [35] W. Chou, T. Wang, Human-computer interactive simulation for the training of minimally invasive neurosurgery, in: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, vol. 2, 2003, pp. 1110– 1115. [36] J. Kim, S. De, Computationally efficient techniques for real time surgical simulation with force feedback, in: Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2002, pp. 51–57. ˜ [37] V. Garc´ıa-Perez, E. Munoz-Moreno, R. de Luis-Garc´ıa, C. ´ Alberola-Lopez, A 3D collision handling algorithm for surgery simulation based on feedback fuzzy logic, in: Proceedings of the IEEE EMBS International Conference on Information Technology Applications in Biomedicine (ITAB’06), “Best Student Paper Award”, Ioannina, Greece, 2006. [38] L. Dom´ınquez-Quintana, M.A. Rodr´ıguez-Florido, J. Ruiz-Alzola, D. Sosa, Modelado 3D de escenarios virtuales ´ realistas para simuladores quirurgicos aplicados a la ´ funduplicatura de Nissen, Informatica y Salud (I + S) 48 (2004) 14–20. [39] W.E. Lorensen, H.E. Cline, Marching cubes: a high resolution 3D surface construction algorithm, SIGGRAPH Comput. Graph. 21 (1987) 163–169. [40] Blender, 2006. www.blender.org. [41] R.M. Satava, A. Cuschieri, J. Hamdorf, Metrics for objective assessment, Surg. Endosc. 17 (2003) 220–226. [42] D. Stefanidis, J.R. Korndorffer Jr., R. Sierra, C. Touchard, J.B. Dunne, D.J. Scott, Skill retention following proficiency-based laparoscopic simulator training, Surgery 138 (2005) 165– 170. [43] W.C. Brunner, J.R. Korndorffer Jr., R. Sierra, J.B. Dunne, C.L. Yau, R.L. Corsetti, D.P. Slakey, M.C. Townsend, D.J. Scott, Determining standards for laparoscopic proficiency using virtual reality, Am. Surg. 71 (2005) 29–35.
