Real-time mandibular angle reduction surgical simulation with haptic rendering.

PubMed

Wang, Qiong; Chen, Hui; Wu, Wen; Jin, Hai-Yang; Heng, Pheng-Ann

2012-11-01

Mandibular angle reduction is a popular and efficient procedure widely used to alter the facial contour. The primary surgical instruments, the reciprocating saw and the round burr, employed in the surgery have a common feature: operating at a high-speed. Generally, inexperienced surgeons need a long-time practice to learn how to minimize the risks caused by the uncontrolled contacts and cutting motions in manipulation of instruments with high-speed reciprocation or rotation. A virtual reality-based surgical simulator for the mandibular angle reduction was designed and implemented on a CUDA-based platform in this paper. High-fidelity visual and haptic feedbacks are provided to enhance the perception in a realistic virtual surgical environment. The impulse-based haptic models were employed to simulate the contact forces and torques on the instruments. It provides convincing haptic sensation for surgeons to control the instruments under different reciprocation or rotation velocities. The real-time methods for bone removal and reconstruction during surgical procedures have been proposed to support realistic visual feedbacks. The simulated contact forces were verified by comparing against the actual force data measured through the constructed mechanical platform. An empirical study based on the patient-specific data was conducted to evaluate the ability of the proposed system in training surgeons with various experiences. The results confirm the validity of our simulator.

Haptic feedback in OP:Sense - augmented reality in telemanipulated robotic surgery.

PubMed

Beyl, T; Nicolai, P; Mönnich, H; Raczkowksy, J; Wörn, H

2012-01-01

In current research, haptic feedback in robot assisted interventions plays an important role. However most approaches to haptic feedback only regard the mapping of the current forces at the surgical instrument to the haptic input devices, whereas surgeons demand a combination of medical imaging and telemanipulated robotic setups. In this paper we describe how this feature is integrated in our robotic research platform OP:Sense. The proposed method allows the automatic transfer of segmented imaging data to the haptic renderer and therefore allows enriching the haptic feedback with virtual fixtures based on imaging data. Anatomical structures are extracted from pre-operative generated medical images or virtual walls are defined by the surgeon inside the imaging data. Combining real forces with virtual fixtures can guide the surgeon to the regions of interest as well as helps to prevent the risk of damage to critical structures inside the patient. We believe that the combination of medical imaging and telemanipulation is a crucial step for the next generation of MIRS-systems.

Co-located haptic and 3D graphic interface for medical simulations.

PubMed

Berkelman, Peter; Miyasaka, Muneaki; Bozlee, Sebastian

2013-01-01

We describe a system which provides high-fidelity haptic feedback in the same physical location as a 3D graphical display, in order to enable realistic physical interaction with virtual anatomical tissue during modelled procedures such as needle driving, palpation, and other interventions performed using handheld instruments. The haptic feedback is produced by the interaction between an array of coils located behind a thin flat LCD screen, and permanent magnets embedded in the instrument held by the user. The coil and magnet configuration permits arbitrary forces and torques to be generated on the instrument in real time according to the dynamics of the simulated tissue by activating the coils in combination. A rigid-body motion tracker provides position and orientation feedback of the handheld instrument to the computer simulation, and the 3D display is produced using LCD shutter glasses and a head-tracking system for the user.

Perception-based 3D tactile rendering from a single image for human skin examinations by dynamic touch.

PubMed

Kim, K; Lee, S

2015-05-01

Diagnosis of skin conditions is dependent on the assessment of skin surface properties that are represented by more tactile properties such as stiffness, roughness, and friction than visual information. Due to this reason, adding tactile feedback to existing vision based diagnosis systems can help dermatologists diagnose skin diseases or disorders more accurately. The goal of our research was therefore to develop a tactile rendering system for skin examinations by dynamic touch. Our development consists of two stages: converting a single image to a 3D haptic surface and rendering the generated haptic surface in real-time. Converting to 3D surfaces from 2D single images was implemented with concerning human perception data collected by a psychophysical experiment that measured human visual and haptic sensibility to 3D skin surface changes. For the second stage, we utilized real skin biomechanical properties found by prior studies. Our tactile rendering system is a standalone system that can be used with any single cameras and haptic feedback devices. We evaluated the performance of our system by conducting an identification experiment with three different skin images with five subjects. The participants had to identify one of the three skin surfaces by using a haptic device (Falcon) only. No visual cue was provided for the experiment. The results indicate that our system provides sufficient performance to render discernable tactile rendering with different skin surfaces. Our system uses only a single skin image and automatically generates a 3D haptic surface based on human haptic perception. Realistic skin interactions can be provided in real-time for the purpose of skin diagnosis, simulations, or training. Our system can also be used for other applications like virtual reality and cosmetic applications. © 2014 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Graphic and haptic simulation system for virtual laparoscopic rectum surgery.

PubMed

Pan, Jun J; Chang, Jian; Yang, Xiaosong; Zhang, Jian J; Qureshi, Tahseen; Howell, Robert; Hickish, Tamas

2011-09-01

Medical simulators with vision and haptic feedback techniques offer a cost-effective and efficient alternative to the traditional medical trainings. They have been used to train doctors in many specialties of medicine, allowing tasks to be practised in a safe and repetitive manner. This paper describes a virtual-reality (VR) system which will help to influence surgeons' learning curves in the technically challenging field of laparoscopic surgery of the rectum. Data from MRI of the rectum and real operation videos are used to construct the virtual models. A haptic force filter based on radial basis functions is designed to offer realistic and smooth force feedback. To handle collision detection efficiently, a hybrid model is presented to compute the deformation of intestines. Finally, a real-time cutting technique based on mesh is employed to represent the incision operation. Despite numerous research efforts, fast and realistic solutions of soft tissues with large deformation, such as intestines, prove extremely challenging. This paper introduces our latest contribution to this endeavour. With this system, the user can haptically operate with the virtual rectum and simultaneously watch the soft tissue deformation. Our system has been tested by colorectal surgeons who believe that the simulated tactile and visual feedbacks are realistic. It could replace the traditional training process and effectively transfer surgical skills to novices. Copyright © 2011 John Wiley & Sons, Ltd.

GPU-based real-time soft tissue deformation with cutting and haptic feedback.

PubMed

Courtecuisse, Hadrien; Jung, Hoeryong; Allard, Jérémie; Duriez, Christian; Lee, Doo Yong; Cotin, Stéphane

2010-12-01

This article describes a series of contributions in the field of real-time simulation of soft tissue biomechanics. These contributions address various requirements for interactive simulation of complex surgical procedures. In particular, this article presents results in the areas of soft tissue deformation, contact modelling, simulation of cutting, and haptic rendering, which are all relevant to a variety of medical interventions. The contributions described in this article share a common underlying model of deformation and rely on GPU implementations to significantly improve computation times. This consistency in the modelling technique and computational approach ensures coherent results as well as efficient, robust and flexible solutions. Copyright © 2010 Elsevier Ltd. All rights reserved.

OzBot and haptics: remote surveillance to physical presence

NASA Astrophysics Data System (ADS)

Mullins, James; Fielding, Mick; Nahavandi, Saeid

2009-05-01

This paper reports on robotic and haptic technologies and capabilities developed for the law enforcement and defence community within Australia by the Centre for Intelligent Systems Research (CISR). The OzBot series of small and medium surveillance robots have been designed in Australia and evaluated by law enforcement and defence personnel to determine suitability and ruggedness in a variety of environments. Using custom developed digital electronics and featuring expandable data busses including RS485, I2C, RS232, video and Ethernet, the robots can be directly connected to many off the shelf payloads such as gas sensors, x-ray sources and camera systems including thermal and night vision. Differentiating the OzBot platform from its peers is its ability to be integrated directly with haptic technology or the 'haptic bubble' developed by CISR. Haptic interfaces allow an operator to physically 'feel' remote environments through position-force control and experience realistic force feedback. By adding the capability to remotely grasp an object, feel its weight, texture and other physical properties in real-time from the remote ground control unit, an operator's situational awareness is greatly improved through Haptic augmentation in an environment where remote-system feedback is often limited.

The effect of haptic guidance and visual feedback on learning a complex tennis task.

PubMed

Marchal-Crespo, Laura; van Raai, Mark; Rauter, Georg; Wolf, Peter; Riener, Robert

2013-11-01

While haptic guidance can improve ongoing performance of a motor task, several studies have found that it ultimately impairs motor learning. However, some recent studies suggest that the haptic demonstration of optimal timing, rather than movement magnitude, enhances learning in subjects trained with haptic guidance. Timing of an action plays a crucial role in the proper accomplishment of many motor skills, such as hitting a moving object (discrete timing task) or learning a velocity profile (time-critical tracking task). The aim of the present study is to evaluate which feedback conditions-visual or haptic guidance-optimize learning of the discrete and continuous elements of a timing task. The experiment consisted in performing a fast tennis forehand stroke in a virtual environment. A tendon-based parallel robot connected to the end of a racket was used to apply haptic guidance during training. In two different experiments, we evaluated which feedback condition was more adequate for learning: (1) a time-dependent discrete task-learning to start a tennis stroke and (2) a tracking task-learning to follow a velocity profile. The effect that the task difficulty and subject's initial skill level have on the selection of the optimal training condition was further evaluated. Results showed that the training condition that maximizes learning of the discrete time-dependent motor task depends on the subjects' initial skill level. Haptic guidance was especially suitable for less-skilled subjects and in especially difficult discrete tasks, while visual feedback seems to benefit more skilled subjects. Additionally, haptic guidance seemed to promote learning in a time-critical tracking task, while visual feedback tended to deteriorate the performance independently of the task difficulty and subjects' initial skill level. Haptic guidance outperformed visual feedback, although additional studies are needed to further analyze the effect of other types of feedback visualization on motor learning of time-critical tasks.

A real-time haptic interface for interventional radiology procedures.

PubMed

Moix, Thomas; Ilic, Dejan; Fracheboud, Blaise; Zoethout, Jurjen; Bleuler, Hannes

2005-01-01

Interventional Radiology (IR) is a minimally-invasive surgery technique (MIS) where guidewires and catheters are steered in the vascular system under X-ray imaging. In order to perform these procedures, a radiologist has to be correctly trained to master hand-eye coordination, instrument manipulation and procedure protocols. This paper proposes a computer-assisted training environment dedicated to IR. The system is composed of a virtual reality (VR) simulation of the anatomy of the patient linked to a robotic interface providing haptic force feedback.The paper focuses on the requirements, design and prototyping of a specific part of the haptic interface dedicated to catheters. Translational tracking and force feedback on the catheter is provided by two cylinders forming a friction drive arrangement. The whole friction can be set in rotation with an additional motor providing torque feedback. A force and a torque sensor are integrated in the cylinders for direct measurement on the catheter enabling disturbance cancellation with a close-loop force control strategy.

Haptic Feedback in Robot-Assisted Minimally Invasive Surgery

PubMed Central

Okamura, Allison M.

2009-01-01

Purpose of Review Robot-assisted minimally invasive surgery (RMIS) holds great promise for improving the accuracy and dexterity of a surgeon while minimizing trauma to the patient. However, widespread clinical success with RMIS has been marginal. It is hypothesized that the lack of haptic (force and tactile) feedback presented to the surgeon is a limiting factor. This review explains the technical challenges of creating haptic feedback for robot-assisted surgery and provides recent results that evaluate the effectiveness of haptic feedback in mock surgical tasks. Recent Findings Haptic feedback systems for RMIS are still under development and evaluation. Most provide only force feedback, with limited fidelity. The major challenge at this time is sensing forces applied to the patient. A few tactile feedback systems for RMIS have been created, but their practicality for clinical implementation needs to be shown. It is particularly difficult to sense and display spatially distributed tactile information. The cost-benefit ratio for haptic feedback in RMIS has not been established. Summary The designs of existing commercial RMIS systems are not conducive for force feedback, and creative solutions are needed to create compelling tactile feedback systems. Surgeons, engineers, and neuroscientists should work together to develop effective solutions for haptic feedback in RMIS. PMID:19057225

Real-time surgery simulation of intracranial aneurysm clipping with patient-specific geometries and haptic feedback

NASA Astrophysics Data System (ADS)

Fenz, Wolfgang; Dirnberger, Johannes

2015-03-01

Providing suitable training for aspiring neurosurgeons is becoming more and more problematic. The increasing popularity of the endovascular treatment of intracranial aneurysms leads to a lack of simple surgical situations for clipping operations, leaving mainly the complex cases, which present even experienced surgeons with a challenge. To alleviate this situation, we have developed a training simulator with haptic interaction allowing trainees to practice virtual clipping surgeries on real patient-specific vessel geometries. By using specialized finite element (FEM) algorithms (fast finite element method, matrix condensation) combined with GPU acceleration, we can achieve the necessary frame rate for smooth real-time interaction with the detailed models needed for a realistic simulation of the vessel wall deformation caused by the clamping with surgical clips. Vessel wall geometries for typical training scenarios were obtained from 3D-reconstructed medical image data, while for the instruments (clipping forceps, various types of clips, suction tubes) we use models provided by manufacturer Aesculap AG. Collisions between vessel and instruments have to be continuously detected and transformed into corresponding boundary conditions and feedback forces, calculated using a contact plane method. After a training, the achieved result can be assessed based on various criteria, including a simulation of the residual blood flow into the aneurysm. Rigid models of the surgical access and surrounding brain tissue, plus coupling a real forceps to the haptic input device further increase the realism of the simulation.

Evaluation of stiffness feedback for hard nodule identification on a phantom silicone model

PubMed Central

Konstantinova, Jelizaveta; Xu, Guanghua; He, Bo; Aminzadeh, Vahid; Xie, Jun; Wurdemann, Helge; Althoefer, Kaspar

2017-01-01

Haptic information in robotic surgery can significantly improve clinical outcomes and help detect hard soft-tissue inclusions that indicate potential abnormalities. Visual representation of tissue stiffness information is a cost-effective technique. Meanwhile, direct force feedback, although considerably more expensive than visual representation, is an intuitive method of conveying information regarding tissue stiffness to surgeons. In this study, real-time visual stiffness feedback by sliding indentation palpation is proposed, validated, and compared with force feedback involving human subjects. In an experimental tele-manipulation environment, a dynamically updated color map depicting the stiffness of probed soft tissue is presented via a graphical interface. The force feedback is provided, aided by a master haptic device. The haptic device uses data acquired from an F/T sensor attached to the end-effector of a tele-manipulated robot. Hard nodule detection performance is evaluated for 2 modes (force feedback and visual stiffness feedback) of stiffness feedback on an artificial organ containing buried stiff nodules. From this artificial organ, a virtual-environment tissue model is generated based on sliding indentation measurements. Employing this virtual-environment tissue model, we compare the performance of human participants in distinguishing differently sized hard nodules by force feedback and visual stiffness feedback. Results indicate that the proposed distributed visual representation of tissue stiffness can be used effectively for hard nodule identification. The representation can also be used as a sufficient substitute for force feedback in tissue palpation. PMID:28248996

Evaluation of stiffness feedback for hard nodule identification on a phantom silicone model.

PubMed

Li, Min; Konstantinova, Jelizaveta; Xu, Guanghua; He, Bo; Aminzadeh, Vahid; Xie, Jun; Wurdemann, Helge; Althoefer, Kaspar

2017-01-01

Haptic information in robotic surgery can significantly improve clinical outcomes and help detect hard soft-tissue inclusions that indicate potential abnormalities. Visual representation of tissue stiffness information is a cost-effective technique. Meanwhile, direct force feedback, although considerably more expensive than visual representation, is an intuitive method of conveying information regarding tissue stiffness to surgeons. In this study, real-time visual stiffness feedback by sliding indentation palpation is proposed, validated, and compared with force feedback involving human subjects. In an experimental tele-manipulation environment, a dynamically updated color map depicting the stiffness of probed soft tissue is presented via a graphical interface. The force feedback is provided, aided by a master haptic device. The haptic device uses data acquired from an F/T sensor attached to the end-effector of a tele-manipulated robot. Hard nodule detection performance is evaluated for 2 modes (force feedback and visual stiffness feedback) of stiffness feedback on an artificial organ containing buried stiff nodules. From this artificial organ, a virtual-environment tissue model is generated based on sliding indentation measurements. Employing this virtual-environment tissue model, we compare the performance of human participants in distinguishing differently sized hard nodules by force feedback and visual stiffness feedback. Results indicate that the proposed distributed visual representation of tissue stiffness can be used effectively for hard nodule identification. The representation can also be used as a sufficient substitute for force feedback in tissue palpation.

Real-time haptic cutting of high-resolution soft tissues.

PubMed

Wu, Jun; Westermann, Rüdiger; Dick, Christian

2014-01-01

We present our systematic efforts in advancing the computational performance of physically accurate soft tissue cutting simulation, which is at the core of surgery simulators in general. We demonstrate a real-time performance of 15 simulation frames per second for haptic soft tissue cutting of a deformable body at an effective resolution of 170,000 finite elements. This is achieved by the following innovative components: (1) a linked octree discretization of the deformable body, which allows for fast and robust topological modifications of the simulation domain, (2) a composite finite element formulation, which thoroughly reduces the number of simulation degrees of freedom and thus enables to carefully balance simulation performance and accuracy, (3) a highly efficient geometric multigrid solver for solving the linear systems of equations arising from implicit time integration, (4) an efficient collision detection algorithm that effectively exploits the composition structure, and (5) a stable haptic rendering algorithm for computing the feedback forces. Considering that our method increases the finite element resolution for physically accurate real-time soft tissue cutting simulation by an order of magnitude, our technique has a high potential to significantly advance the realism of surgery simulators.

Evaluation of Pseudo-Haptic Interactions with Soft Objects in Virtual Environments.

PubMed

Li, Min; Sareh, Sina; Xu, Guanghua; Ridzuan, Maisarah Binti; Luo, Shan; Xie, Jun; Wurdemann, Helge; Althoefer, Kaspar

2016-01-01

This paper proposes a pseudo-haptic feedback method conveying simulated soft surface stiffness information through a visual interface. The method exploits a combination of two feedback techniques, namely visual feedback of soft surface deformation and control of the indenter avatar speed, to convey stiffness information of a simulated surface of a soft object in virtual environments. The proposed method was effective in distinguishing different sizes of virtual hard nodules integrated into the simulated soft bodies. To further improve the interactive experience, the approach was extended creating a multi-point pseudo-haptic feedback system. A comparison with regards to (a) nodule detection sensitivity and (b) elapsed time as performance indicators in hard nodule detection experiments to a tablet computer incorporating vibration feedback was conducted. The multi-point pseudo-haptic interaction is shown to be more time-efficient than the single-point pseudo-haptic interaction. It is noted that multi-point pseudo-haptic feedback performs similarly well when compared to a vibration-based feedback method based on both performance measures elapsed time and nodule detection sensitivity. This proves that the proposed method can be used to convey detailed haptic information for virtual environmental tasks, even subtle ones, using either a computer mouse or a pressure sensitive device as an input device. This pseudo-haptic feedback method provides an opportunity for low-cost simulation of objects with soft surfaces and hard inclusions, as, for example, occurring in ever more realistic video games with increasing emphasis on interaction with the physical environment and minimally invasive surgery in the form of soft tissue organs with embedded cancer nodules. Hence, the method can be used in many low-budget applications where haptic sensation is required, such as surgeon training or video games, either using desktop computers or portable devices, showing reasonably high fidelity in conveying stiffness perception to the user.

Effects of Visual Force Feedback on Robot-Assisted Surgical Task Performance

PubMed Central

Reiley, Carol E.; Akinbiyi, Takintope; Burschka, Darius; Chang, David C.; Okamura, Allison M.; Yuh, David D.

2009-01-01

Background Direct haptic (force or tactile) feedback is negligible in current surgical robotic systems. The relevance of haptic feedback in robot-assisted performances of surgical tasks is controversial. We studied the effects of visual force feedback (VFF), a haptic feedback surrogate, on tying surgical knots with fine sutures similar to those used in cardiovascular surgery. Methods Using a modified da Vinci robotic system (Intuitive Surgical, Inc.) equipped with force-sensing instrument tips and real-time VFF overlays in the console image, ten surgeons each tied 10 knots with and 10 knots without VFF. Four surgeons had significant prior da Vinci experience while the remaining six surgeons did not. Performance parameters, including suture breakage and secure knots, peak and standard deviation of applied forces, and completion times using 5-0 silk sutures were recorded. Chi-square and Student’s t-test analyses determined differences between groups. Results Among surgeon subjects with robotic experience, no differences in measured performance parameters were found between robot-assisted knot ties executed with and without VFF. Among surgeons without robotic experience, however, VFF was associated with lower suture breakage rates, peak applied forces, and standard deviations of applied forces. VFF did not impart differences in knot completion times or loose knots for either surgeon group. Conclusions VFF resulted in reduced suture breakage, lower forces, and decreased force inconsistencies among novice robotic surgeons, although elapsed time and knot quality were unaffected. In contrast, VFF did not affect these metrics among experienced da Vinci surgeons. These results suggest that VFF primarily benefits novice robot-assisted surgeons, with diminishing benefits among experienced surgeons. PMID:18179942

Input and output for surgical simulation: devices to measure tissue properties in vivo and a haptic interface for laparoscopy simulators.

PubMed

Ottensmeyer, M P; Ben-Ur, E; Salisbury, J K

2000-01-01

Current efforts in surgical simulation very often focus on creating realistic graphical feedback, but neglect some or all tactile and force (haptic) feedback that a surgeon would normally receive. Simulations that do include haptic feedback do not typically use real tissue compliance properties, favoring estimates and user feedback to determine realism. When tissue compliance data are used, there are virtually no in vivo property measurements to draw upon. Together with the Center for Innovative Minimally Invasive Therapy at the Massachusetts General Hospital, the Haptics Group is developing tools to introduce more comprehensive haptic feedback in laparoscopy simulators and to provide biological tissue material property data for our software simulation. The platform for providing haptic feedback is a PHANToM Haptic Interface, produced by SensAble Technologies, Inc. Our devices supplement the PHANToM to provide for grasping and optionally, for the roll axis of the tool. Together with feedback from the PHANToM, which provides the pitch, yaw and thrust axes of a typical laparoscopy tool, we can recreate all of the haptic sensations experienced during laparoscopy. The devices integrate real laparoscopy toolhandles and a compliant torso model to complete the set of visual and tactile sensations. Biological tissues are known to exhibit non-linear mechanical properties, and change their properties dramatically when removed from a living organism. To measure the properties in vivo, two devices are being developed. The first is a small displacement, 1-D indenter. It will measure the linear tissue compliance (stiffness and damping) over a wide range of frequencies. These data will be used as inputs to a finite element or other model. The second device will be able to deflect tissues in 3-D over a larger range, so that the non-linearities due to changes in the tissue geometry will be measured. This will allow us to validate the performance of the model on large tissue deformations. Both devices are designed to pass through standard 12 mm laparoscopy trocars, and will be suitable for use during open or minimally invasive procedures. We plan to acquire data from pigs used by surgeons for training purposes, but conceivably, the tools could be refined for use on humans undergoing surgery. Our work will provide the necessary data input for surgical simulations to accurately model the force interactions that a surgeon would have with tissue, and will provide the force output to create a truly realistic simulation of minimally invasive surgery.

Collision detection and modeling of rigid and deformable objects in laparoscopic simulator

NASA Astrophysics Data System (ADS)

Dy, Mary-Clare; Tagawa, Kazuyoshi; Tanaka, Hiromi T.; Komori, Masaru

2015-03-01

Laparoscopic simulators are viable alternatives for surgical training and rehearsal. Haptic devices can also be incorporated with virtual reality simulators to provide additional cues to the users. However, to provide realistic feedback, the haptic device must be updated by 1kHz. On the other hand, realistic visual cues, that is, the collision detection and deformation between interacting objects must be rendered at least 30 fps. Our current laparoscopic simulator detects the collision between a point on the tool tip, and on the organ surfaces, in which haptic devices are attached on actual tool tips for realistic tool manipulation. The triangular-mesh organ model is rendered using a mass spring deformation model, or finite element method-based models. In this paper, we investigated multi-point-based collision detection on the rigid tool rods. Based on the preliminary results, we propose a method to improve the collision detection scheme, and speed up the organ deformation reaction. We discuss our proposal for an efficient method to compute simultaneous multiple collision between rigid (laparoscopic tools) and deformable (organs) objects, and perform the subsequent collision response, with haptic feedback, in real-time.

A new visual feedback-based magnetorheological haptic master for robot-assisted minimally invasive surgery

NASA Astrophysics Data System (ADS)

Choi, Seung-Hyun; Kim, Soomin; Kim, Pyunghwa; Park, Jinhyuk; Choi, Seung-Bok

2015-06-01

In this study, we developed a novel four-degrees-of-freedom haptic master using controllable magnetorheological (MR) fluid. We also integrated the haptic master with a vision device with image processing for robot-assisted minimally invasive surgery (RMIS). The proposed master can be used in RMIS as a haptic interface to provide the surgeon with a sense of touch by using both kinetic and kinesthetic information. The slave robot, which is manipulated with a proportional-integrative-derivative controller, uses a force sensor to obtain the desired forces from tissue contact, and these desired repulsive forces are then embodied through the MR haptic master. To verify the effectiveness of the haptic master, the desired force and actual force are compared in the time domain. In addition, a visual feedback system is implemented in the RMIS experiment to distinguish between the tumor and organ more clearly and provide better visibility to the operator. The hue-saturation-value color space is adopted for the image processing since it is often more intuitive than other color spaces. The image processing and haptic feedback are realized on surgery performance. In this work, tumor-cutting experiments are conducted under four different operating conditions: haptic feedback on, haptic feedback off, image processing on, and image processing off. The experimental realization shows that the performance index, which is a function of pixels, is different in the four operating conditions.

Telerobotic Haptic Exploration in Art Galleries and Museums for Individuals with Visual Impairments.

PubMed

Park, Chung Hyuk; Ryu, Eun-Seok; Howard, Ayanna M

2015-01-01

This paper presents a haptic telepresence system that enables visually impaired users to explore locations with rich visual observation such as art galleries and museums by using a telepresence robot, a RGB-D sensor (color and depth camera), and a haptic interface. The recent improvement on RGB-D sensors has enabled real-time access to 3D spatial information in the form of point clouds. However, the real-time representation of this data in the form of tangible haptic experience has not been challenged enough, especially in the case of telepresence for individuals with visual impairments. Thus, the proposed system addresses the real-time haptic exploration of remote 3D information through video encoding and real-time 3D haptic rendering of the remote real-world environment. This paper investigates two scenarios in haptic telepresence, i.e., mobile navigation and object exploration in a remote environment. Participants with and without visual impairments participated in our experiments based on the two scenarios, and the system performance was validated. In conclusion, the proposed framework provides a new methodology of haptic telepresence for individuals with visual impairments by providing an enhanced interactive experience where they can remotely access public places (art galleries and museums) with the aid of haptic modality and robotic telepresence.

Absence of modulatory action on haptic height perception with musical pitch

PubMed Central

Geronazzo, Michele; Avanzini, Federico; Grassi, Massimo

2015-01-01

Although acoustic frequency is not a spatial property of physical objects, in common language, pitch, i.e., the psychological correlated of frequency, is often labeled spatially (i.e., “high in pitch” or “low in pitch”). Pitch-height is known to modulate (and interact with) the response of participants when they are asked to judge spatial properties of non-auditory stimuli (e.g., visual) in a variety of behavioral tasks. In the current study we investigated whether the modulatory action of pitch-height extended to the haptic estimation of height of a virtual step. We implemented a HW/SW setup which is able to render virtual 3D objects (stair-steps) haptically through a PHANTOM device, and to provide real-time continuous auditory feedback depending on the user interaction with the object. The haptic exploration was associated with a sinusoidal tone whose pitch varied as a function of the interaction point's height within (i) a narrower and (ii) a wider pitch range, or (iii) a random pitch variation acting as a control audio condition. Explorations were also performed with no sound (haptic only). Participants were instructed to explore the virtual step freely, and to communicate height estimation by opening their thumb and index finger to mimic the step riser height, or verbally by reporting the height in centimeters of the step riser. We analyzed the role of musical expertise by dividing participants into non-musicians and musicians. Results showed no effects of musical pitch on high-realistic haptic feedback. Overall there is no difference between the two groups in the proposed multimodal conditions. Additionally, we observed a different haptic response distribution between musicians and non-musicians when estimations of the auditory conditions are matched with estimations in the no sound condition. PMID:26441745

Role of combined tactile and kinesthetic feedback in minimally invasive surgery.

PubMed

Lim, Soo-Chul; Lee, Hyung-Kew; Park, Joonah

2014-10-18

Haptic feedback is of critical importance in surgical tasks. However, conventional surgical robots do not provide haptic feedback to surgeons during surgery. Thus, in this study, a combined tactile and kinesthetic feedback system was developed to provide haptic feedback to surgeons during robotic surgery. To assess haptic feasibility, the effects of two types of haptic feedback were examined empirically - kinesthetic and tactile feedback - to measure object-pulling force with a telesurgery robotics system at two desired pulling forces (1 N and 2 N). Participants answered a set of questionnaires after experiments. The experimental results reveal reductions in force error (39.1% and 40.9%) when using haptic feedback during 1 N and 2 N pulling tasks. Moreover, survey analyses show the effectiveness of the haptic feedback during teleoperation. The combined tactile and kinesthetic feedback of the master device in robotic surgery improves the surgeon's ability to control the interaction force applied to the tissue. Copyright © 2014 John Wiley & Sons, Ltd. Copyright © 2014 John Wiley & Sons, Ltd.

Haptics-based dynamic implicit solid modeling.

PubMed

Hua, Jing; Qin, Hong

2004-01-01

This paper systematically presents a novel, interactive solid modeling framework, Haptics-based Dynamic Implicit Solid Modeling, which is founded upon volumetric implicit functions and powerful physics-based modeling. In particular, we augment our modeling framework with a haptic mechanism in order to take advantage of additional realism associated with a 3D haptic interface. Our dynamic implicit solids are semi-algebraic sets of volumetric implicit functions and are governed by the principles of dynamics, hence responding to sculpting forces in a natural and predictable manner. In order to directly manipulate existing volumetric data sets as well as point clouds, we develop a hierarchical fitting algorithm to reconstruct and represent discrete data sets using our continuous implicit functions, which permit users to further design and edit those existing 3D models in real-time using a large variety of haptic and geometric toolkits, and visualize their interactive deformation at arbitrary resolution. The additional geometric and physical constraints afford more sophisticated control of the dynamic implicit solids. The versatility of our dynamic implicit modeling enables the user to easily modify both the geometry and the topology of modeled objects, while the inherent physical properties can offer an intuitive haptic interface for direct manipulation with force feedback.

Prototyping a Hybrid Cooperative and Tele-robotic Surgical System for Retinal Microsurgery.

PubMed

Balicki, Marcin; Xia, Tian; Jung, Min Yang; Deguet, Anton; Vagvolgyi, Balazs; Kazanzides, Peter; Taylor, Russell

2011-06-01

This paper presents the design of a tele-robotic microsurgical platform designed for development of cooperative and tele-operative control schemes, sensor based smart instruments, user interfaces and new surgical techniques with eye surgery as the driving application. The system is built using the distributed component-based cisst libraries and the Surgical Assistant Workstation framework. It includes a cooperatively controlled EyeRobot2, a da Vinci Master manipulator, and a remote stereo visualization system. We use constrained optimization based virtual fixture control to provide Virtual Remote-Center-of-Motion (vRCM) and haptic feedback. Such system can be used in a hybrid setup, combining local cooperative control with remote tele-operation, where an experienced surgeon can provide hand-over-hand tutoring to a novice user. In another scheme, the system can provide haptic feedback based on virtual fixtures constructed from real-time force and proximity sensor information.

Prototyping a Hybrid Cooperative and Tele-robotic Surgical System for Retinal Microsurgery

PubMed Central

Balicki, Marcin; Xia, Tian; Jung, Min Yang; Deguet, Anton; Vagvolgyi, Balazs; Kazanzides, Peter; Taylor, Russell

2013-01-01

This paper presents the design of a tele-robotic microsurgical platform designed for development of cooperative and tele-operative control schemes, sensor based smart instruments, user interfaces and new surgical techniques with eye surgery as the driving application. The system is built using the distributed component-based cisst libraries and the Surgical Assistant Workstation framework. It includes a cooperatively controlled EyeRobot2, a da Vinci Master manipulator, and a remote stereo visualization system. We use constrained optimization based virtual fixture control to provide Virtual Remote-Center-of-Motion (vRCM) and haptic feedback. Such system can be used in a hybrid setup, combining local cooperative control with remote tele-operation, where an experienced surgeon can provide hand-over-hand tutoring to a novice user. In another scheme, the system can provide haptic feedback based on virtual fixtures constructed from real-time force and proximity sensor information. PMID:24398557

Haptic Foot Pedal: Influence of Shoe Type, Age, and Gender on Subjective Pulse Perception.

PubMed

Geitner, Claudia; Birrell, Stewart; Krehl, Claudia; Jennings, Paul

2018-06-01

This study investigates the influence of shoe type (sneakers and safety boots), age, and gender on the perception of haptic pulse feedback provided by a prototype accelerator pedal in a running stationary vehicle. Haptic feedback can be a less distracting alternative to traditionally visual and auditory in-vehicle feedback. However, to be effective, the device delivering the haptic feedback needs to be in contact with the person. Factors such as shoe type vary naturally over the season and could render feedback that is perceived well in one situation, unnoticeable in another. In this study, we evaluate factors that can influence the subjective perception of haptic feedback in a stationary but running car: shoe type, age, and gender. Thirty-six drivers within three age groups (≤39, 40-59, and ≥60) took part. For each haptic feedback, participants rated intensity, urgency, and comfort via a questionnaire. The perception of the haptic feedback is significantly influenced by the interaction between the pulse's duration and force amplitude and the participant's age and gender but not shoe type. The results indicate that it is important to consider different age groups and gender in the evaluation of haptic feedback. Future research might also look into approaches to adapt haptic feedback to the individual driver's preferences. Findings from this study can be applied to the design of an accelerator pedal in a car, for example, for a nonvisual in-vehicle warning, but also to plan user studies with a haptic pedal in general.

Enhancing the Performance of Passive Teleoperation Systems via Cutaneous Feedback.

PubMed

Pacchierotti, Claudio; Tirmizi, Asad; Bianchini, Gianni; Prattichizzo, Domenico

2015-01-01

We introduce a novel method to improve the performance of passive teleoperation systems with force reflection. It consists of integrating kinesthetic haptic feedback provided by common grounded haptic interfaces with cutaneous haptic feedback. The proposed approach can be used on top of any time-domain control technique that ensures a stable interaction by scaling down kinesthetic feedback when this is required to satisfy stability conditions (e.g., passivity) at the expense of transparency. Performance is recovered by providing a suitable amount of cutaneous force through custom wearable cutaneous devices. The viability of the proposed approach is demonstrated through an experiment of perceived stiffness and an experiment of teleoperated needle insertion in soft tissue.

Integration of soft tissue model and open haptic device for medical training simulator

NASA Astrophysics Data System (ADS)

Akasum, G. F.; Ramdhania, L. N.; Suprijanto; Widyotriatmo, A.

2016-03-01

Minimally Invasive Surgery (MIS) has been widely used to perform any surgical procedures nowadays. Currently, MIS has been applied in some cases in Indonesia. Needle insertion is one of simple MIS procedure that can be used for some purposes. Before the needle insertion technique used in the real situation, it essential to train this type of medical student skills. The research has developed an open platform of needle insertion simulator with haptic feedback that providing the medical student a realistic feel encountered during the actual procedures. There are three main steps in build the training simulator, which are configure hardware system, develop a program to create soft tissue model and the integration of hardware and software. For evaluating its performance, haptic simulator was tested by 24 volunteers on a scenario of soft tissue model. Each volunteer must insert the needle on simulator until rearch the target point with visual feedback that visualized on the monitor. From the result it can concluded that the soft tissue model can bring the sensation of touch through the perceived force feedback on haptic actuator by looking at the different force in accordance with different stiffness in each layer.

Robot-assisted microsurgical forceps with haptic feedback for transoral laser microsurgery.

PubMed

Deshpande, Nikhil; Chauhan, Manish; Pacchierotti, Claudio; Prattichizzo, Domenico; Caldwell, Darwin G; Mattos, Leonardo S

2016-08-01

In this paper, a novel, motorized, multi-degrees-of-freedom (DoF), microsurgical forceps tool is presented, which is based on a master-slave teleoperation architecture. The slave device is a 7-DoF manipulator with: (i) 6-DoF positioning and orientation, (ii) 1 open/close gripper DoF; and (iii) an integrated force/torque sensor for tissue grip-force measurement. The master device is a 7-DoF haptic interface which teleoperates the slave device, and provides haptic feedback in its gripper interface. The combination of the device and the surgeon interface replaces the manual, hand-held device providing easy-to-use and ergonomic tissue control, simplifying the surgical tasks. This makes the system suitable to real surgical scenarios in the operating room (OR). The performance of the system was analysed through the evaluation of teleoperation control and characterization of gripping force. The new system offers an overall positioning error of less than 400 μm demonstrating its safety and accuracy. Improved system precision, usability, and ergonomics point to the potential suitability of the device for the OR and its ability to advance haptic-feedback-enhanced transoral laser microsurgeries.

Functional Contour-following via Haptic Perception and Reinforcement Learning.

PubMed

Hellman, Randall B; Tekin, Cem; van der Schaar, Mihaela; Santos, Veronica J

2018-01-01

Many tasks involve the fine manipulation of objects despite limited visual feedback. In such scenarios, tactile and proprioceptive feedback can be leveraged for task completion. We present an approach for real-time haptic perception and decision-making for a haptics-driven, functional contour-following task: the closure of a ziplock bag. This task is challenging for robots because the bag is deformable, transparent, and visually occluded by artificial fingertip sensors that are also compliant. A deep neural net classifier was trained to estimate the state of a zipper within a robot's pinch grasp. A Contextual Multi-Armed Bandit (C-MAB) reinforcement learning algorithm was implemented to maximize cumulative rewards by balancing exploration versus exploitation of the state-action space. The C-MAB learner outperformed a benchmark Q-learner by more efficiently exploring the state-action space while learning a hard-to-code task. The learned C-MAB policy was tested with novel ziplock bag scenarios and contours (wire, rope). Importantly, this work contributes to the development of reinforcement learning approaches that account for limited resources such as hardware life and researcher time. As robots are used to perform complex, physically interactive tasks in unstructured or unmodeled environments, it becomes important to develop methods that enable efficient and effective learning with physical testbeds.

Design of a haptic device with grasp and push-pull force feedback for a master-slave surgical robot.

PubMed

Hu, Zhenkai; Yoon, Chae-Hyun; Park, Samuel Byeongjun; Jo, Yung-Ho

2016-07-01

We propose a portable haptic device providing grasp (kinesthetic) and push-pull (cutaneous) sensations for optical-motion-capture master interfaces. Although optical-motion-capture master interfaces for surgical robot systems can overcome the stiffness, friction, and coupling problems of mechanical master interfaces, it is difficult to add haptic feedback to an optical-motion-capture master interface without constraining the free motion of the operator's hands. Therefore, we utilized a Bowden cable-driven mechanism to provide the grasp and push-pull sensation while retaining the free hand motion of the optical-motion capture master interface. To evaluate the haptic device, we construct a 2-DOF force sensing/force feedback system. We compare the sensed force and the reproduced force of the haptic device. Finally, a needle insertion test was done to evaluate the performance of the haptic interface in the master-slave system. The results demonstrate that both the grasp force feedback and the push-pull force feedback provided by the haptic interface closely matched with the sensed forces of the slave robot. We successfully apply our haptic interface in the optical-motion-capture master-slave system. The results of the needle insertion test showed that our haptic feedback can provide more safety than merely visual observation. We develop a suitable haptic device to produce both kinesthetic grasp force feedback and cutaneous push-pull force feedback. Our future research will include further objective performance evaluations of the optical-motion-capture master-slave robot system with our haptic interface in surgical scenarios.

A study on haptic collaborative game in shared virtual environment

NASA Astrophysics Data System (ADS)

Lu, Keke; Liu, Guanyang; Liu, Lingzhi

2013-03-01

A study on collaborative game in shared virtual environment with haptic feedback over computer networks is introduced in this paper. A collaborative task was used where the players located at remote sites and played the game together. The player can feel visual and haptic feedback in virtual environment compared to traditional networked multiplayer games. The experiment was desired in two conditions: visual feedback only and visual-haptic feedback. The goal of the experiment is to assess the impact of force feedback on collaborative task performance. Results indicate that haptic feedback is beneficial for performance enhancement for collaborative game in shared virtual environment. The outcomes of this research can have a powerful impact on the networked computer games.

A kinesthetic washout filter for force-feedback rendering.

PubMed

Danieau, Fabien; Lecuyer, Anatole; Guillotel, Philippe; Fleureau, Julien; Mollet, Nicolas; Christie, Marc

2015-01-01

Today haptic feedback can be designed and associated to audiovisual content (haptic-audiovisuals or HAV). Although there are multiple means to create individual haptic effects, the issue of how to properly adapt such effects on force-feedback devices has not been addressed and is mostly a manual endeavor. We propose a new approach for the haptic rendering of HAV, based on a washout filter for force-feedback devices. A body model and an inverse kinematics algorithm simulate the user's kinesthetic perception. Then, the haptic rendering is adapted in order to handle transitions between haptic effects and to optimize the amplitude of effects regarding the device capabilities. Results of a user study show that this new haptic rendering can successfully improve the HAV experience.

Experimental Study on the Perception Characteristics of Haptic Texture by Multidimensional Scaling.

PubMed

Wu, Juan; Li, Na; Liu, Wei; Song, Guangming; Zhang, Jun

2015-01-01

Recent works regarding real texture perception demonstrate that physical factors such as stiffness and spatial period play a fundamental role in texture perception. This research used a multidimensional scaling (MDS) analysis to further characterize and quantify the effects of the simulation parameters on haptic texture rendering and perception. In a pilot experiment, 12 haptic texture samples were generated by using a 3-degrees-of-freedom (3-DOF) force-feedback device with varying spatial period, height, and stiffness coefficient parameter values. The subjects' perceptions of the virtual textures indicate that roughness, denseness, flatness and hardness are distinguishing characteristics of texture. In the main experiment, 19 participants rated the dissimilarities of the textures and estimated the magnitudes of their characteristics. The MDS method was used to recover the underlying perceptual space and reveal the significance of the space from the recorded data. The physical parameters and their combinations have significant effects on the perceptual characteristics. A regression model was used to quantitatively analyze the parameters and their effects on the perceptual characteristics. This paper is to illustrate that haptic texture perception based on force feedback can be modeled in two- or three-dimensional space and provide suggestions on improving perception-based haptic texture rendering.

The value of haptic feedback in conventional and robot-assisted minimal invasive surgery and virtual reality training: a current review.

PubMed

van der Meijden, O A J; Schijven, M P

2009-06-01

Virtual reality (VR) as surgical training tool has become a state-of-the-art technique in training and teaching skills for minimally invasive surgery (MIS). Although intuitively appealing, the true benefits of haptic (VR training) platforms are unknown. Many questions about haptic feedback in the different areas of surgical skills (training) need to be answered before adding costly haptic feedback in VR simulation for MIS training. This study was designed to review the current status and value of haptic feedback in conventional and robot-assisted MIS and training by using virtual reality simulation. A systematic review of the literature was undertaken using PubMed and MEDLINE. The following search terms were used: Haptic feedback OR Haptics OR Force feedback AND/OR Minimal Invasive Surgery AND/OR Minimal Access Surgery AND/OR Robotics AND/OR Robotic Surgery AND/OR Endoscopic Surgery AND/OR Virtual Reality AND/OR Simulation OR Surgical Training/Education. The results were assessed according to level of evidence as reflected by the Oxford Centre of Evidence-based Medicine Levels of Evidence. In the current literature, no firm consensus exists on the importance of haptic feedback in performing minimally invasive surgery. Although the majority of the results show positive assessment of the benefits of force feedback, results are ambivalent and not unanimous on the subject. Benefits are least disputed when related to surgery using robotics, because there is no haptic feedback in currently used robotics. The addition of haptics is believed to reduce surgical errors resulting from a lack of it, especially in knot tying. Little research has been performed in the area of robot-assisted endoscopic surgical training, but results seem promising. Concerning VR training, results indicate that haptic feedback is important during the early phase of psychomotor skill acquisition.

EMG-based visual-haptic biofeedback: a tool to improve motor control in children with primary dystonia.

PubMed

Casellato, Claudia; Pedrocchi, Alessandra; Zorzi, Giovanna; Vernisse, Lea; Ferrigno, Giancarlo; Nardocci, Nardo

2013-05-01

New insights suggest that dystonic motor impairments could also involve a deficit of sensory processing. In this framework, biofeedback, making covert physiological processes more overt, could be useful. The present work proposes an innovative integrated setup which provides the user with an electromyogram (EMG)-based visual-haptic biofeedback during upper limb movements (spiral tracking tasks), to test if augmented sensory feedbacks can induce motor control improvement in patients with primary dystonia. The ad hoc developed real-time control algorithm synchronizes the haptic loop with the EMG reading; the brachioradialis EMG values were used to modify visual and haptic features of the interface: the higher was the EMG level, the higher was the virtual table friction and the background color proportionally moved from green to red. From recordings on dystonic and healthy subjects, statistical results showed that biofeedback has a significant impact, correlated with the local impairment, on the dystonic muscular control. These tests pointed out the effectiveness of biofeedback paradigms in gaining a better specific-muscle voluntary motor control. The flexible tool developed here shows promising prospects of clinical applications and sensorimotor rehabilitation.

Prevailing Trends in Haptic Feedback Simulation for Minimally Invasive Surgery.

PubMed

Pinzon, David; Byrns, Simon; Zheng, Bin

2016-08-01

Background The amount of direct hand-tool-tissue interaction and feedback in minimally invasive surgery varies from being attenuated in laparoscopy to being completely absent in robotic minimally invasive surgery. The role of haptic feedback during surgical skill acquisition and its emphasis in training have been a constant source of controversy. This review discusses the major developments in haptic simulation as they relate to surgical performance and the current research questions that remain unanswered. Search Strategy An in-depth review of the literature was performed using PubMed. Results A total of 198 abstracts were returned based on our search criteria. Three major areas of research were identified, including advancements in 1 of the 4 components of haptic systems, evaluating the effectiveness of haptic integration in simulators, and improvements to haptic feedback in robotic surgery. Conclusions Force feedback is the best method for tissue identification in minimally invasive surgery and haptic feedback provides the greatest benefit to surgical novices in the early stages of their training. New technology has improved our ability to capture, playback and enhance to utility of haptic cues in simulated surgery. Future research should focus on deciphering how haptic training in surgical education can increase performance, safety, and improve training efficiency. © The Author(s) 2016.

Performance evaluation of a robot-assisted catheter operating system with haptic feedback.

PubMed

Song, Yu; Guo, Shuxiang; Yin, Xuanchun; Zhang, Linshuai; Hirata, Hideyuki; Ishihara, Hidenori; Tamiya, Takashi

2018-06-20

In this paper, a novel robot-assisted catheter operating system (RCOS) has been proposed as a method to reduce physical stress and X-ray exposure time to physicians during endovascular procedures. The unique design of this system allows the physician to apply conventional bedside catheterization skills (advance, retreat and rotate) to an input catheter, which is placed at the master side to control another patient catheter placed at the slave side. For this purpose, a magnetorheological (MR) fluids-based master haptic interface has been developed to measure the axial and radial motions of an input catheter, as well as to provide the haptic feedback to the physician during the operation. In order to achieve a quick response of the haptic force in the master haptic interface, a hall sensor-based closed-loop control strategy is employed. In slave side, a catheter manipulator is presented to deliver the patient catheter, according to position commands received from the master haptic interface. The contact forces between the patient catheter and blood vessel system can be measured by designed force sensor unit of catheter manipulator. Four levels of haptic force are provided to make the operator aware of the resistance encountered by the patient catheter during the insertion procedure. The catheter manipulator was evaluated for precision positioning. The time lag from the sensed motion to replicated motion is tested. To verify the efficacy of the proposed haptic feedback method, the evaluation experiments in vitro are carried out. The results demonstrate that the proposed system has the ability to enable decreasing the contact forces between the catheter and vasculature.

The relative importance of real-time in-cab and external feedback in managing fatigue in real-world commercial transport operations.

PubMed

Fitzharris, Michael; Liu, Sara; Stephens, Amanda N; Lenné, Michael G

2017-05-29

Real-time driver monitoring systems represent a solution to address key behavioral risks as they occur, particularly distraction and fatigue. The efficacy of these systems in real-world settings is largely unknown. This article has three objectives: (1) to document the incidence and duration of fatigue in real-world commercial truck-driving operations, (2) to determine the reduction, if any, in the incidence of fatigue episodes associated with providing feedback, and (3) to tease apart the relative contribution of in-cab warnings from 24/7 monitoring and feedback to employers. Data collected from a commercially available in-vehicle camera-based driver monitoring system installed in a commercial truck fleet operating in Australia were analyzed. The real-time driver monitoring system makes continuous assessments of driver drowsiness based on eyelid position and other factors. Data were collected in a baseline period where no feedback was provided to drivers. Real-time feedback to drivers then occurred via in-cab auditory and haptic warnings, which were further enhanced by direct feedback by company management when fatigue events were detected by external 24/7 monitors. Fatigue incidence rates and their timing of occurrence across the three time periods were compared. Relative to no feedback being provided to drivers when fatigue events were detected, in-cab warnings resulted in a 66% reduction in fatigue events, with a 95% reduction achieved by the real-time provision of direct feedback in addition to in-cab warnings (p < 0.01). With feedback, fatigue events were shorter in duration a d occurred later in the trip, and fewer drivers had more than one verified fatigue event per trip. That the provision of feedback to the company on driver fatigue events in real time provides greater benefit than feedback to the driver alone has implications for companies seeking to mitigate risks associated with fatigue. Having fewer fatigue events is likely a reflection of the device itself and the accompanying safety culture of the company in terms of how the information is used. Data were analysed on a per-truck trip basis, and the findings are indicative of fatigue events in a large-scale commercial transport fleet. Future research ought to account for individual driver performance, which was not possible with the available data in this retrospective analysis. Evidence that real-time driver monitoring feedback is effective in reducing fatigue events is invaluable in the development of fleet safety policies, and of future national policy and vehicle safety regulations. Implications for automotive driver monitoring are discussed.

Virtual Cerebral Aneurysm Clipping with Real-Time Haptic Force Feedback in Neurosurgical Education.

PubMed

Gmeiner, Matthias; Dirnberger, Johannes; Fenz, Wolfgang; Gollwitzer, Maria; Wurm, Gabriele; Trenkler, Johannes; Gruber, Andreas

2018-04-01

Realistic, safe, and efficient modalities for simulation-based training are highly warranted to enhance the quality of surgical education, and they should be incorporated in resident training. The aim of this study was to develop a patient-specific virtual cerebral aneurysm-clipping simulator with haptic force feedback and real-time deformation of the aneurysm and vessels. A prototype simulator was developed from 2012 to 2016. Evaluation of virtual clipping by blood flow simulation was integrated in this software, and the prototype was evaluated by 18 neurosurgeons. In 4 patients with different medial cerebral artery aneurysms, virtual clipping was performed after real-life surgery, and surgical results were compared regarding clip application, surgical trajectory, and blood flow. After head positioning and craniotomy, bimanual virtual aneurysm clipping with an original forceps was performed. Blood flow simulation demonstrated residual aneurysm filling or branch stenosis. The simulator improved anatomic understanding for 89% of neurosurgeons. Simulation of head positioning and craniotomy was considered realistic by 89% and 94% of users, respectively. Most participants agreed that this simulator should be integrated into neurosurgical education (94%). Our illustrative cases demonstrated that virtual aneurysm surgery was possible using the same trajectory as in real-life cases. Both virtual clipping and blood flow simulation were realistic in broad-based but not calcified aneurysms. Virtual clipping of a calcified aneurysm could be performed using the same surgical trajectory, but not the same clip type. We have successfully developed a virtual aneurysm-clipping simulator. Next, we will prospectively evaluate this device for surgical procedure planning and education. Copyright © 2018 Elsevier Inc. All rights reserved.

A virtual reality based simulator for learning nasogastric tube placement.

PubMed

Choi, Kup-Sze; He, Xuejian; Chiang, Vico Chung-Lim; Deng, Zhaohong

2015-02-01

Nasogastric tube (NGT) placement is a common clinical procedure where a plastic tube is inserted into the stomach through the nostril for feeding or drainage. However, the placement is a blind process in which the tube may be mistakenly inserted into other locations, leading to unexpected complications or fatal incidents. The placement techniques are conventionally acquired by practising on unrealistic rubber mannequins or on humans. In this paper, a virtual reality based training simulation system is proposed to facilitate the training of NGT placement. It focuses on the simulation of tube insertion and the rendering of the feedback forces with a haptic device. A hybrid force model is developed to compute the forces analytically or numerically under different conditions, including the situations when the patient is swallowing or when the tube is buckled at the nostril. To ensure real-time interactive simulations, an offline simulation approach is adopted to obtain the relationship between the insertion depth and insertion force using a non-linear finite element method. The offline dataset is then used to generate real-time feedback forces by interpolation. The virtual training process is logged quantitatively with metrics that can be used for assessing objective performance and tracking progress. The system has been evaluated by nursing professionals. They found that the haptic feeling produced by the simulated forces is similar to their experience during real NGT insertion. The proposed system provides a new educational tool to enhance conventional training in NGT placement. Copyright © 2014 Elsevier Ltd. All rights reserved.

Providing haptic feedback in robot-assisted minimally invasive surgery: a direct optical force-sensing solution for haptic rendering of deformable bodies.

PubMed

Ehrampoosh, Shervin; Dave, Mohit; Kia, Michael A; Rablau, Corneliu; Zadeh, Mehrdad H

2013-01-01

This paper presents an enhanced haptic-enabled master-slave teleoperation system which can be used to provide force feedback to surgeons in minimally invasive surgery (MIS). One of the research goals was to develop a combined-control architecture framework that included both direct force reflection (DFR) and position-error-based (PEB) control strategies. To achieve this goal, it was essential to measure accurately the direct contact forces between deformable bodies and a robotic tool tip. To measure the forces at a surgical tool tip and enhance the performance of the teleoperation system, an optical force sensor was designed, prototyped, and added to a robot manipulator. The enhanced teleoperation architecture was formulated by developing mathematical models for the optical force sensor, the extended slave robot manipulator, and the combined-control strategy. Human factor studies were also conducted to (a) examine experimentally the performance of the enhanced teleoperation system with the optical force sensor, and (b) study human haptic perception during the identification of remote object deformability. The first experiment was carried out to discriminate deformability of objects when human subjects were in direct contact with deformable objects by means of a laparoscopic tool. The control parameters were then tuned based on the results of this experiment using a gain-scheduling method. The second experiment was conducted to study the effectiveness of the force feedback provided through the enhanced teleoperation system. The results show that the force feedback increased the ability of subjects to correctly identify materials of different deformable types. In addition, the virtual force feedback provided by the teleoperation system comes close to the real force feedback experienced in direct MIS. The experimental results provide design guidelines for choosing and validating the control architecture and the optical force sensor.

End-to-End Flow Control for Visual-Haptic Communication under Bandwidth Change

NASA Astrophysics Data System (ADS)

Yashiro, Daisuke; Tian, Dapeng; Yakoh, Takahiro

This paper proposes an end-to-end flow controller for visual-haptic communication. A visual-haptic communication system transmits non-real-time packets, which contain large-size visual data, and real-time packets, which contain small-size haptic data. When the transmission rate of visual data exceeds the communication bandwidth, the visual-haptic communication system becomes unstable owing to buffer overflow. To solve this problem, an end-to-end flow controller is proposed. This controller determines the optimal transmission rate of visual data on the basis of the traffic conditions, which are estimated by the packets for haptic communication. Experimental results confirm that in the proposed method, a short packet-sending interval and a short delay are achieved under bandwidth change, and thus, high-precision visual-haptic communication is realized.

Augmented reality and haptic interfaces for robot-assisted surgery.

PubMed

Yamamoto, Tomonori; Abolhassani, Niki; Jung, Sung; Okamura, Allison M; Judkins, Timothy N

2012-03-01

Current teleoperated robot-assisted minimally invasive surgical systems do not take full advantage of the potential performance enhancements offered by various forms of haptic feedback to the surgeon. Direct and graphical haptic feedback systems can be integrated with vision and robot control systems in order to provide haptic feedback to improve safety and tissue mechanical property identification. An interoperable interface for teleoperated robot-assisted minimally invasive surgery was developed to provide haptic feedback and augmented visual feedback using three-dimensional (3D) graphical overlays. The software framework consists of control and command software, robot plug-ins, image processing plug-ins and 3D surface reconstructions. The feasibility of the interface was demonstrated in two tasks performed with artificial tissue: palpation to detect hard lumps and surface tracing, using vision-based forbidden-region virtual fixtures to prevent the patient-side manipulator from entering unwanted regions of the workspace. The interoperable interface enables fast development and successful implementation of effective haptic feedback methods in teleoperation. Copyright © 2011 John Wiley & Sons, Ltd.

Using mixed reality, force feedback and tactile augmentation to improve the realism of medical simulation.

PubMed

Fisher, J Brian; Porter, Susan M

2002-01-01

This paper describes an application of a display approach which uses chromakey techniques to composite real and computer-generated images allowing a user to see his hands and medical instruments collocated with the display of virtual objects during a medical training simulation. Haptic feedback is provided through the use of a PHANTOM force feedback device in addition to tactile augmentation, which allows the user to touch virtual objects by introducing corresponding real objects in the workspace. A simplified catheter introducer insertion simulation was developed to demonstrate the capabilities of this approach.

Haptic augmentation of science instruction: Does touch matter?

NASA Astrophysics Data System (ADS)

Jones, M. Gail; Minogue, James; Tretter, Thomas R.; Negishi, Atsuko; Taylor, Russell

2006-01-01

This study investigated the impact of haptic augmentation of a science inquiry program on students' learning about viruses and nanoscale science. The study assessed how the addition of different types of haptic feedback (active touch and kinesthetic feedback) combined with computer visualizations influenced middle and high school students' experiences. The influences of a PHANToM (a sophisticated haptic desktop device), a Sidewinder (a haptic gaming joystick), and a mouse (no haptic feedback) interface were compared. The levels of engagement in the instruction and students' attitudes about the instructional program were assessed using a combination of constructed response and Likert scale items. Potential cognitive differences were examined through an analysis of spontaneously generated analogies that appeared during student discourse. Results showed that the addition of haptic feedback from the haptic-gaming joystick and the PHANToM provided a more immersive learning environment that not only made the instruction more engaging but may also influence the way in which the students construct their understandings about abstract science concepts.

A 3-RSR Haptic Wearable Device for Rendering Fingertip Contact Forces.

PubMed

Leonardis, Daniele; Solazzi, Massimiliano; Bortone, Ilaria; Frisoli, Antonio

2017-01-01

A novel wearable haptic device for modulating contact forces at the fingertip is presented. Rendering of forces by skin deformation in three degrees of freedom (DoF), with contact-no contact capabilities, was implemented through rigid parallel kinematics. The novel asymmetrical three revolute-spherical-revolute (3-RSR) configuration allowed compact dimensions with minimum encumbrance of the hand workspace. The device was designed to render constant to low frequency deformation of the fingerpad in three DoF, combining light weight with relatively high output forces. A differential method for solving the non-trivial inverse kinematics is proposed and implemented in real time for controlling the device. The first experimental activity evaluated discrimination of different fingerpad stretch directions in a group of five subjects. The second experiment, enrolling 19 subjects, evaluated cutaneous feedback provided in a virtual pick-and-place manipulation task. Stiffness of the fingerpad plus device was measured and used to calibrate the physics of the virtual environment. The third experiment with 10 subjects evaluated interaction forces in a virtual lift-and-hold task. Although with different performance in the two manipulation experiments, overall results show that participants better controlled interaction forces when the cutaneous feedback was active, with significant differences between the visual and visuo-haptic experimental conditions.

Direct Visuo-Haptic 4D Volume Rendering Using Respiratory Motion Models.

PubMed

Fortmeier, Dirk; Wilms, Matthias; Mastmeyer, Andre; Handels, Heinz

2015-01-01

This article presents methods for direct visuo-haptic 4D volume rendering of virtual patient models under respiratory motion. Breathing models are computed based on patient-specific 4D CT image data sequences. Virtual patient models are visualized in real-time by ray casting based rendering of a reference CT image warped by a time-variant displacement field, which is computed using the motion models at run-time. Furthermore, haptic interaction with the animated virtual patient models is provided by using the displacements computed at high rendering rates to translate the position of the haptic device into the space of the reference CT image. This concept is applied to virtual palpation and the haptic simulation of insertion of a virtual bendable needle. To this aim, different motion models that are applicable in real-time are presented and the methods are integrated into a needle puncture training simulation framework, which can be used for simulated biopsy or vessel puncture in the liver. To confirm real-time applicability, a performance analysis of the resulting framework is given. It is shown that the presented methods achieve mean update rates around 2,000 Hz for haptic simulation and interactive frame rates for volume rendering and thus are well suited for visuo-haptic rendering of virtual patients under respiratory motion.

Assisting Movement Training and Execution With Visual and Haptic Feedback.

PubMed

Ewerton, Marco; Rother, David; Weimar, Jakob; Kollegger, Gerrit; Wiemeyer, Josef; Peters, Jan; Maeda, Guilherme

2018-01-01

In the practice of motor skills in general, errors in the execution of movements may go unnoticed when a human instructor is not available. In this case, a computer system or robotic device able to detect movement errors and propose corrections would be of great help. This paper addresses the problem of how to detect such execution errors and how to provide feedback to the human to correct his/her motor skill using a general, principled methodology based on imitation learning. The core idea is to compare the observed skill with a probabilistic model learned from expert demonstrations. The intensity of the feedback is regulated by the likelihood of the model given the observed skill. Based on demonstrations, our system can, for example, detect errors in the writing of characters with multiple strokes. Moreover, by using a haptic device, the Haption Virtuose 6D, we demonstrate a method to generate haptic feedback based on a distribution over trajectories, which could be used as an auxiliary means of communication between an instructor and an apprentice. Additionally, given a performance measurement, the haptic device can help the human discover and perform better movements to solve a given task. In this case, the human first tries a few times to solve the task without assistance. Our framework, in turn, uses a reinforcement learning algorithm to compute haptic feedback, which guides the human toward better solutions.

Enhanced operator perception through 3D vision and haptic feedback

NASA Astrophysics Data System (ADS)

Edmondson, Richard; Light, Kenneth; Bodenhamer, Andrew; Bosscher, Paul; Wilkinson, Loren

2012-06-01

Polaris Sensor Technologies (PST) has developed a stereo vision upgrade kit for TALON® robot systems comprised of a replacement gripper camera and a replacement mast zoom camera on the robot, and a replacement display in the Operator Control Unit (OCU). Harris Corporation has developed a haptic manipulation upgrade for TALON® robot systems comprised of a replacement arm and gripper and an OCU that provides haptic (force) feedback. PST and Harris have recently collaborated to integrate the 3D vision system with the haptic manipulation system. In multiple studies done at Fort Leonard Wood, Missouri it has been shown that 3D vision and haptics provide more intuitive perception of complicated scenery and improved robot arm control, allowing for improved mission performance and the potential for reduced time on target. This paper discusses the potential benefits of these enhancements to robotic systems used for the domestic homeland security mission.

A design of hardware haptic interface for gastrointestinal endoscopy simulation.

PubMed

Gu, Yunjin; Lee, Doo Yong

2011-01-01

Gastrointestinal endoscopy simulations have been developed to train endoscopic procedures which require hundreds of practices to be competent in the skills. Even though realistic haptic feedback is important to provide realistic sensation to the user, most of previous simulations including commercialized simulation have mainly focused on providing realistic visual feedback. In this paper, we propose a novel design of portable haptic interface, which provides 2DOF force feedback, for the gastrointestinal endoscopy simulation. The haptic interface consists of translational and rotational force feedback mechanism which are completely decoupled, and gripping mechanism for controlling connection between the endoscope and the force feedback mechanism.

Non-contact monitoring during laser surgery by measuring the incision depth with air-coupled transducers

NASA Astrophysics Data System (ADS)

Oyaga Landa, Francisco Javier; Deán-Ben, Xosé Luís.; Montero de Espinosa, Francisco; Razansky, Daniel

2017-03-01

Lack of haptic feedback during laser surgery hampers controlling the incision depth, leading to a high risk of undesired tissue damage. Here we present a new feedback sensing method that accomplishes non-contact realtime monitoring of laser ablation procedures by detecting shock waves emanating from the ablation spot with air-coupled transducers. Experiments in soft and hard tissue samples attained high reproducibity in real-time depth estimation of the laser-induced cuts. The advantages derived from the non-contact nature of the suggested monitoring approach are expected to greatly promote the general applicability of laser-based surgeries.

Performance Evaluation of Passive Haptic Feedback for Tactile HMI Design in CAVEs.

PubMed

Lassagne, Antoine; Kemeny, Andras; Posselt, Javier; Merienne, Frederic

2018-01-01

This article presents a comparison of different haptic systems, which are designed to simulate flat Human Machine Interfaces (HMIs) like touchscreens in virtual environments (VEs) such as CAVEs, and their respective performance. We compare a tangible passive transparent slate to a classic tablet and a sensory substitution system. These systems were tested during a controlled experiment. The performance and impressions from 20 subjects were collected to understand more about the modalities in the given context. The results show that the preferences of the subjects are strongly related to the use-cases and needs. In terms of performance, passive haptics proved to be significantly useful, acting as a space reference and a real-time continuous calibration system, allowing subjects to have lower execution durations and relative errors. Sensory substitution induced perception drifts during the experiment, causing significant performance disparities, demonstrating the low robustness of perception when spatial cues are insufficiently available. Our findings offer a better understanding on the nature of perception drifts and the need of strong multisensory spatial markers for such use-cases in CAVEs. The importance of a relevant haptic modality specifically designed to match a precise use-case is also emphasized.

Validation of the PASSPORT V2 training environment for arthroscopic skills.

PubMed

Stunt, J J; Kerkhoffs, G M M J; Horeman, T; van Dijk, C N; Tuijthof, G J M

2016-06-01

Virtual reality simulators used in the education of orthopaedic residents often lack realistic haptic feedback. To solve this, the (Practice Arthroscopic Surgical Skills for Perfect Operative Real-life Treatment) PASSPORT simulator was developed, which was subjected to fundamental changes: improved realism and user interface. The purpose was to demonstrate its face and construct validity. Thirty-one participants were divided into three groups having different levels of arthroscopic experience. Participants answered questions regarding general information and the outer appearance of the simulator for face validity. Construct validity was assessed with one standardized navigation task, which was timed. Face validity, educational value and user-friendliness were determined with two representative exercises and by asking participants to fill out the questionnaire. A value of 7 or greater was considered sufficient. Construct validity was demonstrated between experts and novices. Median task time for the fifth trial was 55 s (range 17-139 s) for the novices, 33 s (range 17-59 s) for the intermediates, and 26 s (range 14-52 s) for the experts. Median task times of three trials were not significantly different between the novices and intermediates, and none of the trials between intermediates and experts. Face validity, educational value and user-friendliness were perceived as sufficient (median >7). The presence of realistic tactile feedback was considered the biggest asset of the simulator. Proper preparation for arthroscopic operations will increase the quality of real-life surgery and patients' safety. The PASSPORT simulator can assist in achieving this, as it showed construct and face validity, and its physical nature offered adequate haptic feedback during training. This indicates that PASSPORT has potential to evolve as a valuable training modality.

Development of optical FBG force measurement system for the medical application

NASA Astrophysics Data System (ADS)

Song, Hoseok; Kim, Kiyoung; Suh, Jungwook; Lee, Jungju

2010-03-01

Haptic feedback plays a very important role in medical surgery. In minimally invasive surgery (MIS), however, very long and stiff bar of instruments take haptic feeling away from the surgeon. In minimally invasive robotic surgery (MIRS), moreover, haptic feelings are totally eliminated. Previous researchers have reported that the absence of force feedback increased the average force magnitude applied to the tissue by at least 50%, and increased the peakforce magnitude by at least a factor of two. Therefore, it is very important to provide haptic information in MIRS. Recently, many sensors are being developed for MIS or MIRS, but they have some obstacles in their application to real situations of medical surgery. The most critical problems are size limit and sterilizability. Optical fiber sensors are one of the most suitable sensors for this environment. Especially, optical fiber Bragg grating (FBG) sensor has one additional advantage than the other optical fiber sensors. FBG sensor is not influenced by intensity of light source. In this paper, we would like to present the initial results of study on the application of the FBG sensor to measure reflected forces in MIRS environments and then suggest the possibility of successful application to the MIRS systems.

Development of optical FBG force measurement system for the medical application

NASA Astrophysics Data System (ADS)

Song, Hoseok; Kim, Kiyoung; Suh, Jungwook; Lee, Jungju

2009-12-01

Haptic feedback plays a very important role in medical surgery. In minimally invasive surgery (MIS), however, very long and stiff bar of instruments take haptic feeling away from the surgeon. In minimally invasive robotic surgery (MIRS), moreover, haptic feelings are totally eliminated. Previous researchers have reported that the absence of force feedback increased the average force magnitude applied to the tissue by at least 50%, and increased the peakforce magnitude by at least a factor of two. Therefore, it is very important to provide haptic information in MIRS. Recently, many sensors are being developed for MIS or MIRS, but they have some obstacles in their application to real situations of medical surgery. The most critical problems are size limit and sterilizability. Optical fiber sensors are one of the most suitable sensors for this environment. Especially, optical fiber Bragg grating (FBG) sensor has one additional advantage than the other optical fiber sensors. FBG sensor is not influenced by intensity of light source. In this paper, we would like to present the initial results of study on the application of the FBG sensor to measure reflected forces in MIRS environments and then suggest the possibility of successful application to the MIRS systems.

Sensory subtraction in robot-assisted surgery: fingertip skin deformation feedback to ensure safety and improve transparency in bimanual haptic interaction.

PubMed

Meli, Leonardo; Pacchierotti, Claudio; Prattichizzo, Domenico

2014-04-01

This study presents a novel approach to force feedback in robot-assisted surgery. It consists of substituting haptic stimuli, composed of a kinesthetic component and a skin deformation, with cutaneous stimuli only. The force generated can then be thought as a subtraction between the complete haptic interaction, cutaneous, and kinesthetic, and the kinesthetic part of it. For this reason, we refer to this approach as sensory subtraction. Sensory subtraction aims at outperforming other nonkinesthetic feedback techniques in teleoperation (e.g., sensory substitution) while guaranteeing the stability and safety of the system. We tested the proposed approach in a challenging 7-DoF bimanual teleoperation task, similar to the Pegboard experiment of the da Vinci Skills Simulator. Sensory subtraction showed improved performance in terms of completion time, force exerted, and total displacement of the rings with respect to two popular sensory substitution techniques. Moreover, it guaranteed a stable interaction in the presence of a communication delay in the haptic loop.

Precise Haptic Device Co-Location for Visuo-Haptic Augmented Reality.

PubMed

Eck, Ulrich; Pankratz, Frieder; Sandor, Christian; Klinker, Gudrun; Laga, Hamid

2015-12-01

Visuo-haptic augmented reality systems enable users to see and touch digital information that is embedded in the real world. PHANToM haptic devices are often employed to provide haptic feedback. Precise co-location of computer-generated graphics and the haptic stylus is necessary to provide a realistic user experience. Previous work has focused on calibration procedures that compensate the non-linear position error caused by inaccuracies in the joint angle sensors. In this article we present a more complete procedure that additionally compensates for errors in the gimbal sensors and improves position calibration. The proposed procedure further includes software-based temporal alignment of sensor data and a method for the estimation of a reference for position calibration, resulting in increased robustness against haptic device initialization and external tracker noise. We designed our procedure to require minimal user input to maximize usability. We conducted an extensive evaluation with two different PHANToMs, two different optical trackers, and a mechanical tracker. Compared to state-of-the-art calibration procedures, our approach significantly improves the co-location of the haptic stylus. This results in higher fidelity visual and haptic augmentations, which are crucial for fine-motor tasks in areas such as medical training simulators, assembly planning tools, or rapid prototyping applications.

Effects of 3D virtual haptics force feedback on brand personality perception: the mediating role of physical presence in advergames.

PubMed

Jin, Seung-A Annie

2010-06-01

This study gauged the effects of force feedback in the Novint Falcon haptics system on the sensory and cognitive dimensions of a virtual test-driving experience. First, in order to explore the effects of tactile stimuli with force feedback on users' sensory experience, feelings of physical presence (the extent to which virtual physical objects are experienced as actual physical objects) were measured after participants used the haptics interface. Second, to evaluate the effects of force feedback on the cognitive dimension of consumers' virtual experience, this study investigated brand personality perception. The experiment utilized the Novint Falcon haptics controller to induce immersive virtual test-driving through tactile stimuli. The author designed a two-group (haptics stimuli with force feedback versus no force feedback) comparison experiment (N = 238) by manipulating the level of force feedback. Users in the force feedback condition were exposed to tactile stimuli involving various force feedback effects (e.g., terrain effects, acceleration, and lateral forces) while test-driving a rally car. In contrast, users in the control condition test-drove the rally car using the Novint Falcon but were not given any force feedback. Results of ANOVAs indicated that (a) users exposed to force feedback felt stronger physical presence than those in the no force feedback condition, and (b) users exposed to haptics stimuli with force feedback perceived the brand personality of the car to be more rugged than those in the control condition. Managerial implications of the study for product trial in the business world are discussed.

Real-time dual-band haptic music player for mobile devices.

PubMed

Hwang, Inwook; Lee, Hyeseon; Choi, Seungmoon

2013-01-01

We introduce a novel dual-band haptic music player for real-time simultaneous vibrotactile playback with music in mobile devices. Our haptic music player features a new miniature dual-mode actuator that can produce vibrations consisting of two principal frequencies and a real-time vibration generation algorithm that can extract vibration commands from a music file for dual-band playback (bass and treble). The algorithm uses a "haptic equalizer" and provides plausible sound-to-touch modality conversion based on human perceptual data. In addition, we present a user study carried out to evaluate the subjective performance (precision, harmony, fun, and preference) of the haptic music player, in comparison with the current practice of bass-band-only vibrotactile playback via a single-frequency voice-coil actuator. The evaluation results indicated that the new dual-band playback outperforms the bass-only rendering, also providing several insights for further improvements. The developed system and experimental findings have implications for improving the multimedia experience with mobile devices.

Simulation and training of lumbar punctures using haptic volume rendering and a 6DOF haptic device

NASA Astrophysics Data System (ADS)

Färber, Matthias; Heller, Julika; Handels, Heinz

2007-03-01

The lumbar puncture is performed by inserting a needle into the spinal chord of the patient to inject medicaments or to extract liquor. The training of this procedure is usually done on the patient guided by experienced supervisors. A virtual reality lumbar puncture simulator has been developed in order to minimize the training costs and the patient's risk. We use a haptic device with six degrees of freedom (6DOF) to feedback forces that resist needle insertion and rotation. An improved haptic volume rendering approach is used to calculate the forces. This approach makes use of label data of relevant structures like skin, bone, muscles or fat and original CT data that contributes information about image structures that can not be segmented. A real-time 3D visualization with optional stereo view shows the punctured region. 2D visualizations of orthogonal slices enable a detailed impression of the anatomical context. The input data consisting of CT and label data and surface models of relevant structures is defined in an XML file together with haptic rendering and visualization parameters. In a first evaluation the visible human male data has been used to generate a virtual training body. Several users with different medical experience tested the lumbar puncture trainer. The simulator gives a good haptic and visual impression of the needle insertion and the haptic volume rendering technique enables the feeling of unsegmented structures. Especially, the restriction of transversal needle movement together with rotation constraints enabled by the 6DOF device facilitate a realistic puncture simulation.

Development of a Robotic Colonoscopic Manipulation System, Using Haptic Feedback Algorithm.

PubMed

Woo, Jaehong; Choi, Jae Hyuk; Seo, Jong Tae; Kim, Tae Il; Yi, Byung Ju

2017-01-01

Colonoscopy is one of the most effective diagnostic and therapeutic tools for colorectal diseases. We aim to propose a master-slave robotic colonoscopy that is controllable in remote site using conventional colonoscopy. The master and slave robot were developed to use conventional flexible colonoscopy. The robotic colonoscopic procedure was performed using a colonoscope training model by one expert endoscopist and two unexperienced engineers. To provide the haptic sensation, the insertion force and the rotating torque were measured and sent to the master robot. A slave robot was developed to hold the colonoscopy and its knob, and perform insertion, rotation, and two tilting motions of colonoscope. A master robot was designed to teach motions of the slave robot. These measured force and torque were scaled down by one tenth to provide the operator with some reflection force and torque at the haptic device. The haptic sensation and feedback system was successful and helpful to feel the constrained force or torque in colon. The insertion time using robotic system decreased with repeated procedures. This work proposed a robotic approach for colonoscopy using haptic feedback algorithm, and this robotic device would effectively perform colonoscopy with reduced burden and comparable safety for patients in remote site.

Characteristic analysis and simulation for polysilicon comb micro-accelerometer

NASA Astrophysics Data System (ADS)

Liu, Fengli; Hao, Yongping

2008-10-01

High force update rate is a key factor for achieving high performance haptic rendering, which imposes a stringent real time requirement upon the execution environment of the haptic system. This requirement confines the haptic system to simplified environment for reducing the computation cost of haptic rendering algorithms. In this paper, we present a novel "hyper-threading" architecture consisting of several threads for haptic rendering. The high force update rate is achieved with relatively large computation time interval for each haptic loop. The proposed method was testified and proved to be effective with experiments on virtual wall prototype haptic system via Delta Haptic Device.

Haptic Stylus and Empirical Studies on Braille, Button, and Texture Display

PubMed Central

Kyung, Ki-Uk; Lee, Jun-Young; Park, Junseok

2008-01-01

This paper presents a haptic stylus interface with a built-in compact tactile display module and an impact module as well as empirical studies on Braille, button, and texture display. We describe preliminary evaluations verifying the tactile display's performance indicating that it can satisfactorily represent Braille numbers for both the normal and the blind. In order to prove haptic feedback capability of the stylus, an experiment providing impact feedback mimicking the click of a button has been conducted. Since the developed device is small enough to be attached to a force feedback device, its applicability to combined force and tactile feedback display in a pen-held haptic device is also investigated. The handle of pen-held haptic interface was replaced by the pen-like interface to add tactile feedback capability to the device. Since the system provides combination of force, tactile and impact feedback, three haptic representation methods for texture display have been compared on surface with 3 texture groups which differ in direction, groove width, and shape. In addition, we evaluate its capacity to support touch screen operations by providing tactile sensations when a user rubs against an image displayed on a monitor. PMID:18317520

Haptic stylus and empirical studies on braille, button, and texture display.

PubMed

Kyung, Ki-Uk; Lee, Jun-Young; Park, Junseok

2008-01-01

This paper presents a haptic stylus interface with a built-in compact tactile display module and an impact module as well as empirical studies on Braille, button, and texture display. We describe preliminary evaluations verifying the tactile display's performance indicating that it can satisfactorily represent Braille numbers for both the normal and the blind. In order to prove haptic feedback capability of the stylus, an experiment providing impact feedback mimicking the click of a button has been conducted. Since the developed device is small enough to be attached to a force feedback device, its applicability to combined force and tactile feedback display in a pen-held haptic device is also investigated. The handle of pen-held haptic interface was replaced by the pen-like interface to add tactile feedback capability to the device. Since the system provides combination of force, tactile and impact feedback, three haptic representation methods for texture display have been compared on surface with 3 texture groups which differ in direction, groove width, and shape. In addition, we evaluate its capacity to support touch screen operations by providing tactile sensations when a user rubs against an image displayed on a monitor.

Learning of Temporal and Spatial Movement Aspects: A Comparison of Four Types of Haptic Control and Concurrent Visual Feedback.

PubMed

Rauter, Georg; Sigrist, Roland; Riener, Robert; Wolf, Peter

2015-01-01

In literature, the effectiveness of haptics for motor learning is controversially discussed. Haptics is believed to be effective for motor learning in general; however, different types of haptic control enhance different movement aspects. Thus, in dependence on the movement aspects of interest, one type of haptic control may be effective whereas another one is not. Therefore, in the current work, it was investigated if and how different types of haptic controllers affect learning of spatial and temporal movement aspects. In particular, haptic controllers that enforce active participation of the participants were expected to improve spatial aspects. Only haptic controllers that provide feedback about the task's velocity profile were expected to improve temporal aspects. In a study on learning a complex trunk-arm rowing task, the effect of training with four different types of haptic control was investigated: position control, path control, adaptive path control, and reactive path control. A fifth group (control) trained with visual concurrent augmented feedback. As hypothesized, the position controller was most effective for learning of temporal movement aspects, while the path controller was most effective in teaching spatial movement aspects of the rowing task. Visual feedback was also effective for learning temporal and spatial movement aspects.

Shared virtual environments for telerehabilitation.

PubMed

Popescu, George V; Burdea, Grigore; Boian, Rares

2002-01-01

Current VR telerehabilitation systems use offline remote monitoring from the clinic and patient-therapist videoconferencing. Such "store and forward" and video-based systems cannot implement medical services involving patient therapist direct interaction. Real-time telerehabilitation applications (including remote therapy) can be developed using a shared Virtual Environment (VE) architecture. We developed a two-user shared VE for hand telerehabilitation. Each site has a telerehabilitation workstation with a videocamera and a Rutgers Master II (RMII) force feedback glove. Each user can control a virtual hand and interact hapticly with virtual objects. Simulated physical interactions between therapist and patient are implemented using hand force feedback. The therapist's graphic interface contains several virtual panels, which allow control over the rehabilitation process. These controls start a videoconferencing session, collect patient data, or apply therapy. Several experimental telerehabilitation scenarios were successfully tested on a LAN. A Web-based approach to "real-time" patient telemonitoring--the monitoring portal for hand telerehabilitation--was also developed. The therapist interface is implemented as a Java3D applet that monitors patient hand movement. The monitoring portal gives real-time performance on off-the-shelf desktop workstations.

Investigating Students' Ideas About Buoyancy and the Influence of Haptic Feedback

NASA Astrophysics Data System (ADS)

Minogue, James; Borland, David

2016-04-01

While haptics (simulated touch) represents a potential breakthrough technology for science teaching and learning, there is relatively little research into its differential impact in the context of teaching and learning. This paper describes the testing of a haptically enhanced simulation (HES) for learning about buoyancy. Despite a lifetime of everyday experiences, a scientifically sound explanation of buoyancy remains difficult to construct for many. It requires the integration of domain-specific knowledge regarding density, fluid, force, gravity, mass, weight, and buoyancy. Prior studies suggest that novices often focus on only one dimension of the sinking and floating phenomenon. Our HES was designed to promote the integration of the subconcepts of density and buoyant forces and stresses the relationship between the object itself and the surrounding fluid. The study employed a randomized pretest-posttest control group research design and a suite of measures including an open-ended prompt and objective content questions to provide insights into the influence of haptic feedback on undergraduate students' thinking about buoyancy. A convenience sample (n = 40) was drawn from a university's population of undergraduate elementary education majors. Two groups were formed from haptic feedback (n = 22) and no haptic feedback (n = 18). Through content analysis, discernible differences were seen in the posttest explanations sinking and floating across treatment groups. Learners that experienced the haptic feedback made more frequent use of "haptically grounded" terms (e.g., mass, gravity, buoyant force, pushing), leading us to begin to build a local theory of language-mediated haptic cognition.

Effects of kinesthetic haptic feedback on standing stability of young healthy subjects and stroke patients.

PubMed

Afzal, Muhammad Raheel; Byun, Ha-Young; Oh, Min-Kyun; Yoon, Jungwon

2015-03-13

Haptic control is a useful therapeutic option in rehabilitation featuring virtual reality interaction. As with visual and vibrotactile biofeedback, kinesthetic haptic feedback may assist in postural control, and can achieve balance control. Kinesthetic haptic feedback in terms of body sway can be delivered via a commercially available haptic device and can enhance the balance stability of both young healthy subjects and stroke patients. Our system features a waist-attached smartphone, software running on a computer (PC), and a dedicated Phantom Omni® device. Young healthy participants performed balance tasks after assumption of each of four distinct postures for 30 s (one foot on the ground; the Tandem Romberg stance; one foot on foam; and the Tandem Romberg stance on foam) with eyes closed. Patient eyes were not closed and assumption of the Romberg stance (only) was tested during a balance task 25 s in duration. An Android application running continuously on the smartphone sent mediolateral (ML) and anteroposterior (AP) tilt angles to a PC, which generated kinesthetic haptic feedback via Phantom Omni®. A total of 16 subjects, 8 of whom were young healthy and 8 of whom had suffered stroke, participated in the study. Post-experiment data analysis was performed using MATLAB®. Mean Velocity Displacement (MVD), Planar Deviation (PD), Mediolateral Trajectory (MLT) and Anteroposterior Trajectory (APT) parameters were analyzed to measure reduction in body sway. Our kinesthetic haptic feedback system was effective to reduce postural sway in young healthy subjects regardless of posture and the condition of the substrate (the ground) and to improve MVD and PD in stroke patients who assumed the Romberg stance. Analysis of Variance (ANOVA) revealed that kinesthetic haptic feedback significantly reduced body sway in both categories of subjects. Kinesthetic haptic feedback can be implemented using a commercial haptic device and a smartphone. Intuitive balance cues were created using the handle of a haptic device, rendering the approach very simple yet efficient in practice. This novel form of biofeedback will be a useful rehabilitation tool improving the balance of stroke patients.

Control of repulsive force in a virtual environment using an electrorheological haptic master for a surgical robot application

NASA Astrophysics Data System (ADS)

Oh, Jong-Seok; Choi, Seung-Hyun; Choi, Seung-Bok

2014-01-01

This paper presents control performances of a new type of four-degrees-of-freedom (4-DOF) haptic master that can be used for robot-assisted minimally invasive surgery (RMIS). By adopting a controllable electrorheological (ER) fluid, the function of the proposed master is realized as a haptic feedback as well as remote manipulation. In order to verify the efficacy of the proposed master and method, an experiment is conducted with deformable objects featuring human organs. Since the use of real human organs is difficult for control due to high cost and moral hazard, an excellent alternative method, the virtual reality environment, is used for control in this work. In order to embody a human organ in the virtual space, the experiment adopts a volumetric deformable object represented by a shape-retaining chain linked (S-chain) model which has salient properties such as fast and realistic deformation of elastic objects. In haptic architecture for RMIS, the desired torque/force and desired position originating from the object of the virtual slave and operator of the haptic master are transferred to each other. In order to achieve the desired torque/force trajectories, a sliding mode controller (SMC) which is known to be robust to uncertainties is designed and empirically implemented. Tracking control performances for various torque/force trajectories from the virtual slave are evaluated and presented in the time domain.

Ergonomic evaluation of 3D plane positioning using a mouse and a haptic device.

PubMed

Paul, Laurent; Cartiaux, Olivier; Docquier, Pierre-Louis; Banse, Xavier

2009-12-01

Preoperative planning and intraoperative assistance are needed to improve accuracy in tumour surgery. To be accepted, these processes must be efficient. An experiment was conducted to compare a mouse and a haptic device, with and without force feedback, to perform plan positioning in a 3D space. Ergonomics and performance factors were investigated during the experiment. Positioning strategies were observed. The task completion time, number of 3D orientations and failure rate were analysed. A questionnaire on ergonomics was filled out by each participant. The haptic device showed a significantly lower failure rate and was quicker and more ergonomic than the mouse. The force feedback was not beneficial to the accomplishment of the task. The haptic device is intuitive, ergonomic and more efficient than the mouse for positioning a 3D plane into a 3D space. Useful observations regarding positioning strategies will improve the integration of haptic devices into medical applications. Copyright (c) 2009 John Wiley & Sons, Ltd.

Error amplification to promote motor learning and motivation in therapy robotics.

PubMed

Shirzad, Navid; Van der Loos, H F Machiel

2012-01-01

To study the effects of different feedback error amplification methods on a subject's upper-limb motor learning and affect during a point-to-point reaching exercise, we developed a real-time controller for a robotic manipulandum. The reaching environment was visually distorted by implementing a thirty degrees rotation between the coordinate systems of the robot's end-effector and the visual display. Feedback error amplification was provided to subjects as they trained to learn reaching within the visually rotated environment. Error amplification was provided either visually or through both haptic and visual means, each method with two different amplification gains. Subjects' performance (i.e., trajectory error) and self-reports to a questionnaire were used to study the speed and amount of adaptation promoted by each error amplification method and subjects' emotional changes. We found that providing haptic and visual feedback promotes faster adaptation to the distortion and increases subjects' satisfaction with the task, leading to a higher level of attentiveness during the exercise. This finding can be used to design a novel exercise regimen, where alternating between error amplification methods is used to both increase a subject's motor learning and maintain a minimum level of motivational engagement in the exercise. In future experiments, we will test whether such exercise methods will lead to a faster learning time and greater motivation to pursue a therapy exercise regimen.

A survey of telerobotic surface finishing

NASA Astrophysics Data System (ADS)

Höglund, Thomas; Alander, Jarmo; Mantere, Timo

2018-05-01

This is a survey of research published on the subjects of telerobotics, haptic feedback, and mixed reality applied to surface finishing. The survey especially focuses on how visuo-haptic feedback can be used to improve a grinding process using a remote manipulator or robot. The benefits of teleoperation and reasons for using haptic feedback are presented. The use of genetic algorithms for optimizing haptic sensing is briefly discussed. Ways of augmenting the operator's vision are described. Visual feedback can be used to find defects and analyze the quality of the surface resulting from the surface finishing process. Visual cues can also be used to aid a human operator in manipulating a robot precisely and avoiding collisions.

Shifty: A Weight-Shifting Dynamic Passive Haptic Proxy to Enhance Object Perception in Virtual Reality.

PubMed

Zenner, Andre; Kruger, Antonio

2017-04-01

We define the concept of Dynamic Passive Haptic Feedback (DPHF) for virtual reality by introducing the weight-shifting physical DPHF proxy object Shifty. This concept combines actuators known from active haptics and physical proxies known from passive haptics to construct proxies that automatically adapt their passive haptic feedback. We describe the concept behind our ungrounded weight-shifting DPHF proxy Shifty and the implementation of our prototype. We then investigate how Shifty can, by automatically changing its internal weight distribution, enhance the user's perception of virtual objects interacted with in two experiments. In a first experiment, we show that Shifty can enhance the perception of virtual objects changing in shape, especially in length and thickness. Here, Shifty was shown to increase the user's fun and perceived realism significantly, compared to an equivalent passive haptic proxy. In a second experiment, Shifty is used to pick up virtual objects of different virtual weights. The results show that Shifty enhances the perception of weight and thus the perceived realism by adapting its kinesthetic feedback to the picked-up virtual object. In the same experiment, we additionally show that specific combinations of haptic, visual and auditory feedback during the pick-up interaction help to compensate for visual-haptic mismatch perceived during the shifting process.

Development of a Robotic Colonoscopic Manipulation System, Using Haptic Feedback Algorithm

PubMed Central

Woo, Jaehong; Choi, Jae Hyuk; Seo, Jong Tae

2017-01-01

Purpose Colonoscopy is one of the most effective diagnostic and therapeutic tools for colorectal diseases. We aim to propose a master-slave robotic colonoscopy that is controllable in remote site using conventional colonoscopy. Materials and Methods The master and slave robot were developed to use conventional flexible colonoscopy. The robotic colonoscopic procedure was performed using a colonoscope training model by one expert endoscopist and two unexperienced engineers. To provide the haptic sensation, the insertion force and the rotating torque were measured and sent to the master robot. Results A slave robot was developed to hold the colonoscopy and its knob, and perform insertion, rotation, and two tilting motions of colonoscope. A master robot was designed to teach motions of the slave robot. These measured force and torque were scaled down by one tenth to provide the operator with some reflection force and torque at the haptic device. The haptic sensation and feedback system was successful and helpful to feel the constrained force or torque in colon. The insertion time using robotic system decreased with repeated procedures. Conclusion This work proposed a robotic approach for colonoscopy using haptic feedback algorithm, and this robotic device would effectively perform colonoscopy with reduced burden and comparable safety for patients in remote site. PMID:27873506

Effects of Grip-Force, Contact, and Acceleration Feedback on a Teleoperated Pick-and-Place Task.

PubMed

Khurshid, Rebecca P; Fitter, Naomi T; Fedalei, Elizabeth A; Kuchenbecker, Katherine J

2017-01-01

The multifaceted human sense of touch is fundamental to direct manipulation, but technical challenges prevent most teleoperation systems from providing even a single modality of haptic feedback, such as force feedback. This paper postulates that ungrounded grip-force, fingertip-contact-and-pressure, and high-frequency acceleration haptic feedback will improve human performance of a teleoperated pick-and-place task. Thirty subjects used a teleoperation system consisting of a haptic device worn on the subject's right hand, a remote PR2 humanoid robot, and a Vicon motion capture system to move an object to a target location. Each subject completed the pick-and-place task 10 times under each of the eight haptic conditions obtained by turning on and off grip-force feedback, contact feedback, and acceleration feedback. To understand how object stiffness affects the utility of the feedback, half of the subjects completed the task with a flexible plastic cup, and the others used a rigid plastic block. The results indicate that the addition of grip-force feedback with gain switching enables subjects to hold both the flexible and rigid objects more stably, and it also allowed subjects who manipulated the rigid block to hold the object more delicately and to better control the motion of the remote robot's hand. Contact feedback improved the ability of subjects who manipulated the flexible cup to move the robot's arm in space, but it deteriorated this ability for subjects who manipulated the rigid block. Contact feedback also caused subjects to hold the flexible cup less stably, but the rigid block more securely. Finally, adding acceleration feedback slightly improved the subject's performance when setting the object down, as originally hypothesized; interestingly, it also allowed subjects to feel vibrations produced by the robot's motion, causing them to be more careful when completing the task. This study supports the utility of grip-force and high-frequency acceleration feedback in teleoperation systems and motivates further improvements to fingertip-contact-and-pressure feedback.

Faster acquisition of laparoscopic skills in virtual reality with haptic feedback and 3D vision.

PubMed

Hagelsteen, Kristine; Langegård, Anders; Lantz, Adam; Ekelund, Mikael; Anderberg, Magnus; Bergenfelz, Anders

2017-10-01

The study investigated whether 3D vision and haptic feedback in combination in a virtual reality environment leads to more efficient learning of laparoscopic skills in novices. Twenty novices were allocated to two groups. All completed a training course in the LapSim ® virtual reality trainer consisting of four tasks: 'instrument navigation', 'grasping', 'fine dissection' and 'suturing'. The study group performed with haptic feedback and 3D vision and the control group without. Before and after the LapSim ® course, the participants' metrics were recorded when tying a laparoscopic knot in the 2D video box trainer Simball ® Box. The study group completed the training course in 146 (100-291) minutes compared to 215 (175-489) minutes in the control group (p = .002). The number of attempts to reach proficiency was significantly lower. The study group had significantly faster learning of skills in three out of four individual tasks; instrument navigation, grasping and suturing. Using the Simball ® Box, no difference in laparoscopic knot tying after the LapSim ® course was noted when comparing the groups. Laparoscopic training in virtual reality with 3D vision and haptic feedback made training more time efficient and did not negatively affect later video box-performance in 2D. [Formula: see text].

Vision-Based Haptic Feedback for Remote Micromanipulation in-SEM Environment

NASA Astrophysics Data System (ADS)

Bolopion, Aude; Dahmen, Christian; Stolle, Christian; Haliyo, Sinan; Régnier, Stéphane; Fatikow, Sergej

2012-07-01

This article presents an intuitive environment for remote micromanipulation composed of both haptic feedback and virtual reconstruction of the scene. To enable nonexpert users to perform complex teleoperated micromanipulation tasks, it is of utmost importance to provide them with information about the 3-D relative positions of the objects and the tools. Haptic feedback is an intuitive way to transmit such information. Since position sensors are not available at this scale, visual feedback is used to derive information about the scene. In this work, three different techniques are implemented, evaluated, and compared to derive the object positions from scanning electron microscope images. The modified correlation matching with generated template algorithm is accurate and provides reliable detection of objects. To track the tool, a marker-based approach is chosen since fast detection is required for stable haptic feedback. Information derived from these algorithms is used to propose an intuitive remote manipulation system that enables users situated in geographically distant sites to benefit from specific equipments, such as SEMs. Stability of the haptic feedback is ensured by the minimization of the delays, the computational efficiency of vision algorithms, and the proper tuning of the haptic coupling. Virtual guides are proposed to avoid any involuntary collisions between the tool and the objects. This approach is validated by a teleoperation involving melamine microspheres with a diameter of less than 2 μ m between Paris, France and Oldenburg, Germany.

Mechatronic design of haptic forceps for robotic surgery.

PubMed

Rizun, P; Gunn, D; Cox, B; Sutherland, G

2006-12-01

Haptic feedback increases operator performance and comfort during telerobotic manipulation. Feedback of grasping pressure is critical in many microsurgical tasks, yet no haptic interface for surgical tools is commercially available. Literature on the psychophysics of touch was reviewed to define the spectrum of human touch perception and the fidelity requirements of an ideal haptic interface. Mechanical design and control literature was reviewed to translate the psychophysical requirements to engineering specification. High-fidelity haptic forceps were then developed through an iterative process between engineering and surgery. The forceps are a modular device that integrate with a haptic hand controller to add force feedback for tool actuation in telerobotic or virtual surgery. Their overall length is 153 mm and their mass is 125 g. A contact-free voice coil actuator generates force feedback at frequencies up to 800 Hz. Maximum force output is 6 N (2N continuous) and the force resolution is 4 mN. The forceps employ a contact-free magnetic position sensor as well as micro-machined accelerometers to measure opening/closing acceleration. Position resolution is 0.6 microm with 1.3 microm RMS noise. The forceps can simulate stiffness greater than 20N/mm or impedances smaller than 15 g with no noticeable haptic artifacts or friction. As telerobotic surgery evolves, haptics will play an increasingly important role. Copyright 2006 John Wiley & Sons, Ltd.

Neurosurgery simulation using non-linear finite element modeling and haptic interaction

NASA Astrophysics Data System (ADS)

Lee, Huai-Ping; Audette, Michel; Joldes, Grand R.; Enquobahrie, Andinet

2012-02-01

Real-time surgical simulation is becoming an important component of surgical training. To meet the realtime requirement, however, the accuracy of the biomechancial modeling of soft tissue is often compromised due to computing resource constraints. Furthermore, haptic integration presents an additional challenge with its requirement for a high update rate. As a result, most real-time surgical simulation systems employ a linear elasticity model, simplified numerical methods such as the boundary element method or spring-particle systems, and coarse volumetric meshes. However, these systems are not clinically realistic. We present here an ongoing work aimed at developing an efficient and physically realistic neurosurgery simulator using a non-linear finite element method (FEM) with haptic interaction. Real-time finite element analysis is achieved by utilizing the total Lagrangian explicit dynamic (TLED) formulation and GPU acceleration of per-node and per-element operations. We employ a virtual coupling method for separating deformable body simulation and collision detection from haptic rendering, which needs to be updated at a much higher rate than the visual simulation. The system provides accurate biomechancial modeling of soft tissue while retaining a real-time performance with haptic interaction. However, our experiments showed that the stability of the simulator depends heavily on the material property of the tissue and the speed of colliding objects. Hence, additional efforts including dynamic relaxation are required to improve the stability of the system.

Objective Assessment of Laparoscopic Force and Psychomotor Skills in a Novel Virtual Reality-Based Haptic Simulator.

PubMed

Prasad, M S Raghu; Manivannan, Muniyandi; Manoharan, Govindan; Chandramohan, S M

2016-01-01

Most of the commercially available virtual reality-based laparoscopic simulators do not effectively evaluate combined psychomotor and force-based laparoscopic skills. Consequently, the lack of training on these critical skills leads to intraoperative errors. To assess the effectiveness of the novel virtual reality-based simulator, this study analyzed the combined psychomotor (i.e., motion or movement) and force skills of residents and expert surgeons. The study also examined the effectiveness of real-time visual force feedback and tool motion during training. Bimanual fundamental (i.e., probing, pulling, sweeping, grasping, and twisting) and complex tasks (i.e., tissue dissection) were evaluated. In both tasks, visual feedback on applied force and tool motion were provided. The skills of the participants while performing the early tasks were assessed with and without visual feedback. Participants performed 5 repetitions of fundamental and complex tasks. Reaction force and instrument acceleration were used as metrics. Surgical Gastroenterology, Government Stanley Medical College and Hospital; Institute of Surgical Gastroenterology, Madras Medical College and Rajiv Gandhi Government General Hospital. Residents (N = 25; postgraduates and surgeons with <2 years of laparoscopic surgery) and expert surgeons (N = 25; surgeons with >4 and ≤10 years of laparoscopic surgery). Residents applied large forces compared with expert surgeons and performed abrupt tool movements (p < 0.001). However, visual + haptic feedback improved the performance of residents (p < 0.001). In complex tasks, visual + haptic feedback did not influence the applied force of expert surgeons, but influenced their tool motion (p < 0.001). Furthermore, in complex tissue sweeping task, expert surgeons applied more force, but were within the tissue damage limits. In both groups, exertion of large forces and abrupt tool motion were observed during grasping, probing or pulling, and tissue sweeping maneuvers (p < 0.001). Modern day curriculum-based training should evaluate the skills of residents with robust force and psychomotor-based exercises for proficient laparoscopy. Visual feedback on force and motion during training has the potential to enhance the learning curve of residents. Copyright © 2016 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Palpation simulator with stable haptic feedback.

PubMed

Kim, Sang-Youn; Ryu, Jee-Hwan; Lee, WooJeong

2015-01-01

The main difficulty in constructing palpation simulators is to compute and to generate stable and realistic haptic feedback without vibration. When a user haptically interacts with highly non-homogeneous soft tissues through a palpation simulator, a sudden change of stiffness in target tissues causes unstable interaction with the object. We propose a model consisting of a virtual adjustable damper and an energy measuring element. The energy measuring element gauges energy which is stored in a palpation simulator and the virtual adjustable damper dissipates the energy to achieve stable haptic interaction. To investigate the haptic behavior of the proposed method, impulse and continuous inputs are provided to target tissues. If a haptic interface point meets with the hardest portion in the target tissues modeled with a conventional method, we observe unstable motion and feedback force. However, when the target tissues are modeled with the proposed method, a palpation simulator provides stable interaction without vibration. The proposed method overcomes a problem in conventional haptic palpation simulators where unstable force or vibration can be generated if there is a big discrepancy in material property between an element and its neighboring elements in target tissues.

Sensing and Force-Feedback Exoskeleton (SAFE) Robotic Glove.

PubMed

Ben-Tzvi, Pinhas; Ma, Zhou

2015-11-01

This paper presents the design, implementation and experimental validation of a novel robotic haptic exoskeleton device to measure the user's hand motion and assist hand motion while remaining portable and lightweight. The device consists of a five-finger mechanism actuated with miniature DC motors through antagonistically routed cables at each finger, which act as both active and passive force actuators. The SAFE Glove is a wireless and self-contained mechatronic system that mounts over the dorsum of a bare hand and provides haptic force feedback to each finger. The glove is adaptable to a wide variety of finger sizes without constraining the range of motion. This makes it possible to accurately and comfortably track the complex motion of the finger and thumb joints associated with common movements of hand functions, including grip and release patterns. The glove can be wirelessly linked to a computer for displaying and recording the hand status through 3D Graphical User Interface (GUI) in real-time. The experimental results demonstrate that the SAFE Glove is capable of reliably modeling hand kinematics, measuring finger motion and assisting hand grasping motion. Simulation and experimental results show the potential of the proposed system in rehabilitation therapy and virtual reality applications.

Soft tissue deformation modelling through neural dynamics-based reaction-diffusion mechanics.

PubMed

Zhang, Jinao; Zhong, Yongmin; Gu, Chengfan

2018-05-30

Soft tissue deformation modelling forms the basis of development of surgical simulation, surgical planning and robotic-assisted minimally invasive surgery. This paper presents a new methodology for modelling of soft tissue deformation based on reaction-diffusion mechanics via neural dynamics. The potential energy stored in soft tissues due to a mechanical load to deform tissues away from their rest state is treated as the equivalent transmembrane potential energy, and it is distributed in the tissue masses in the manner of reaction-diffusion propagation of nonlinear electrical waves. The reaction-diffusion propagation of mechanical potential energy and nonrigid mechanics of motion are combined to model soft tissue deformation and its dynamics, both of which are further formulated as the dynamics of cellular neural networks to achieve real-time computational performance. The proposed methodology is implemented with a haptic device for interactive soft tissue deformation with force feedback. Experimental results demonstrate that the proposed methodology exhibits nonlinear force-displacement relationship for nonlinear soft tissue deformation. Homogeneous, anisotropic and heterogeneous soft tissue material properties can be modelled through the inherent physical properties of mass points. Graphical abstract Soft tissue deformation modelling with haptic feedback via neural dynamics-based reaction-diffusion mechanics.

Stereo camera based virtual cane system with identifiable distance tactile feedback for the blind.

PubMed

Kim, Donghun; Kim, Kwangtaek; Lee, Sangyoun

2014-06-13

In this paper, we propose a new haptic-assisted virtual cane system operated by a simple finger pointing gesture. The system is developed by two stages: development of visual information delivery assistant (VIDA) with a stereo camera and adding a tactile feedback interface with dual actuators for guidance and distance feedbacks. In the first stage, user's pointing finger is automatically detected using color and disparity data from stereo images and then a 3D pointing direction of the finger is estimated with its geometric and textural features. Finally, any object within the estimated pointing trajectory in 3D space is detected and the distance is then estimated in real time. For the second stage, identifiable tactile signals are designed through a series of identification experiments, and an identifiable tactile feedback interface is developed and integrated into the VIDA system. Our approach differs in that navigation guidance is provided by a simple finger pointing gesture and tactile distance feedbacks are perfectly identifiable to the blind.

Stereo Camera Based Virtual Cane System with Identifiable Distance Tactile Feedback for the Blind

PubMed Central

Kim, Donghun; Kim, Kwangtaek; Lee, Sangyoun

2014-01-01

In this paper, we propose a new haptic-assisted virtual cane system operated by a simple finger pointing gesture. The system is developed by two stages: development of visual information delivery assistant (VIDA) with a stereo camera and adding a tactile feedback interface with dual actuators for guidance and distance feedbacks. In the first stage, user's pointing finger is automatically detected using color and disparity data from stereo images and then a 3D pointing direction of the finger is estimated with its geometric and textural features. Finally, any object within the estimated pointing trajectory in 3D space is detected and the distance is then estimated in real time. For the second stage, identifiable tactile signals are designed through a series of identification experiments, and an identifiable tactile feedback interface is developed and integrated into the VIDA system. Our approach differs in that navigation guidance is provided by a simple finger pointing gesture and tactile distance feedbacks are perfectly identifiable to the blind. PMID:24932864

Passive haptics in a knee arthroscopy simulator: is it valid for core skills training?

PubMed

McCarthy, Avril D; Moody, Louise; Waterworth, Alan R; Bickerstaff, Derek R

2006-01-01

Previous investigation of a cost-effective virtual reality arthroscopic training system, the Sheffield Knee Arthroscopy Training System (SKATS), indicated the desirability of including haptic feedback. A formal task analysis confirmed the importance of knee positioning as a core skill for trainees learning to navigate the knee arthroscopically. The system cost and existing limb interface, which permits knee positioning, would be compromised by the addition of commercial active haptic devices available currently. The validation results obtained when passive haptic feedback (resistance provided by physical structures) is provided indicate that SKATS has construct, predictive and face validity for navigation and triangulation training. When tested using SKATS, experienced surgeons (n = 11) performed significantly faster, located significantly more pathologies, and showed significantly shorter arthroscope path lengths than a less experienced surgeon cohort (n = 12). After SKATS training sessions, novices (n = 3) showed significant improvements in: task completion time, shorter arthroscope path lengths, shorter probe path lengths, and fewer arthroscope tip contacts. Main improvements occurred after the first two practice sessions, indicating rapid familiarization and a training effect. Feedback from questionnaires completed by orthopaedic surgeons indicates that the system has face validity for its remit of basic arthroscopic training.

Overview Electrotactile Feedback for Enhancing Human Computer Interface

NASA Astrophysics Data System (ADS)

Pamungkas, Daniel S.; Caesarendra, Wahyu

2018-04-01

To achieve effective interaction between a human and a computing device or machine, adequate feedback from the computing device or machine is required. Recently, haptic feedback is increasingly being utilised to improve the interactivity of the Human Computer Interface (HCI). Most existing haptic feedback enhancements aim at producing forces or vibrations to enrich the user’s interactive experience. However, these force and/or vibration actuated haptic feedback systems can be bulky and uncomfortable to wear and only capable of delivering a limited amount of information to the user which can limit both their effectiveness and the applications they can be applied to. To address this deficiency, electrotactile feedback is used. This involves delivering haptic sensations to the user by electrically stimulating nerves in the skin via electrodes placed on the surface of the skin. This paper presents a review and explores the capability of electrotactile feedback for HCI applications. In addition, a description of the sensory receptors within the skin for sensing tactile stimulus and electric currents alsoseveral factors which influenced electric signal to transmit to the brain via human skinare explained.

Toward a comprehensive hybrid physical-virtual reality simulator of peripheral anesthesia with ultrasound and neurostimulator guidance.

PubMed

Samosky, Joseph T; Allen, Pete; Boronyak, Steve; Branstetter, Barton; Hein, Steven; Juhas, Mark; Nelson, Douglas A; Orebaugh, Steven; Pinto, Rohan; Smelko, Adam; Thompson, Mitch; Weaver, Robert A

2011-01-01

We are developing a simulator of peripheral nerve block utilizing a mixed-reality approach: the combination of a physical model, an MRI-derived virtual model, mechatronics and spatial tracking. Our design uses tangible (physical) interfaces to simulate surface anatomy, haptic feedback during needle insertion, mechatronic display of muscle twitch corresponding to the specific nerve stimulated, and visual and haptic feedback for the injection syringe. The twitch response is calculated incorporating the sensed output of a real neurostimulator. The virtual model is isomorphic with the physical model and is derived from segmented MRI data. This model provides the subsurface anatomy and, combined with electromagnetic tracking of a sham ultrasound probe and a standard nerve block needle, supports simulated ultrasound display and measurement of needle location and proximity to nerves and vessels. The needle tracking and virtual model also support objective performance metrics of needle targeting technique.

Real-time simulation of biological soft tissues: a PGD approach.

PubMed

Niroomandi, S; González, D; Alfaro, I; Bordeu, F; Leygue, A; Cueto, E; Chinesta, F

2013-05-01

We introduce here a novel approach for the numerical simulation of nonlinear, hyperelastic soft tissues at kilohertz feedback rates necessary for haptic rendering. This approach is based upon the use of proper generalized decomposition techniques, a generalization of PODs. Proper generalized decomposition techniques can be considered as a means of a priori model order reduction and provides a physics-based meta-model without the need for prior computer experiments. The suggested strategy is thus composed of an offline phase, in which a general meta-model is computed, and an online evaluation phase in which the results are obtained at real time. Results are provided that show the potential of the proposed technique, together with some benchmark test that shows the accuracy of the method. Copyright © 2013 John Wiley & Sons, Ltd.

Development of a StandAlone Surgical Haptic Arm.

PubMed

Jones, Daniel; Lewis, Andrew; Fischer, Gregory S

2011-01-01

When performing telesurgery with current commercially available Minimally Invasive Robotic Surgery (MIRS) systems, a surgeon cannot feel the tool interactions that are inherent in traditional laparoscopy. It is proposed that haptic feedback in the control of MIRS systems could improve the speed, safety and learning curve of robotic surgery. To test this hypothesis, a standalone surgical haptic arm (SASHA) capable of manipulating da Vinci tools has been designed and fabricated with the additional ability of providing information for haptic feedback. This arm was developed as a research platform for developing and evaluating approaches to telesurgery, including various haptic mappings between master and slave and evaluating the effects of latency.

The simulation of the half-dry stroke based on the force feedback technology

NASA Astrophysics Data System (ADS)

Guo, Chao; Hou, Zeng-xuan; Zheng, Shuan-zhu; Yang, Guang-qing

2017-02-01

A novel stroke simulation method of the Half-dry style of Chinese calligraphy based on the force feedback technology is proposed for the virtual painting. Firstly, according to the deformation of the brush when the force is exerted on it, the brush footprint between the brush and paper is calculated. The complete brush stroke is obtained by superimposing brush footprints along the painting direction, and the dynamic painting of the brush stroke is implemented. Then, we establish the half-dry texture databases and propose the concept of half-dry value by researching the main factors that affect the effects of the half-dry stroke. In the virtual painting, the half-dry texture is mapped into the stroke in real time according to the half-dry value and painting technique. A technique of texture blending based on the KM model is applied to avoid the seams while texture mapping. The proposed method has been successfully applied to the virtual painting system based on the force feedback technology. In this system, users can implement the painting in real time with a Phantom Desktop haptic device, which can effectively enhance reality to users.

Evaluation of haptic interfaces for simulation of drill vibration in virtual temporal bone surgery.

PubMed

Ghasemloonia, Ahmad; Baxandall, Shalese; Zareinia, Kourosh; Lui, Justin T; Dort, Joseph C; Sutherland, Garnette R; Chan, Sonny

2016-11-01

Surgical training is evolving from an observership model towards a new paradigm that includes virtual-reality (VR) simulation. In otolaryngology, temporal bone dissection has become intimately linked with VR simulation as the complexity of anatomy demands a high level of surgeon aptitude and confidence. While an adequate 3D visualization of the surgical site is available in current simulators, the force feedback rendered during haptic interaction does not convey vibrations. This lack of vibration rendering limits the simulation fidelity of a surgical drill such as that used in temporal bone dissection. In order to develop an immersive simulation platform capable of haptic force and vibration feedback, the efficacy of hand controllers for rendering vibration in different drilling circumstances needs to be investigated. In this study, the vibration rendering ability of four different haptic hand controllers were analyzed and compared to find the best commercial haptic hand controller. A test-rig was developed to record vibrations encountered during temporal bone dissection and a software was written to render the recorded signals without adding hardware to the system. An accelerometer mounted on the end-effector of each device recorded the rendered vibration signals. The newly recorded vibration signal was compared with the input signal in both time and frequency domains by coherence and cross correlation analyses to quantitatively measure the fidelity of these devices in terms of rendering vibrotactile drilling feedback in different drilling conditions. This method can be used to assess the vibration rendering ability in VR simulation systems and selection of ideal haptic devices. Copyright © 2016 Elsevier Ltd. All rights reserved.

Advanced Maintenance Simulation by Means of Hand-Based Haptic Interfaces

NASA Astrophysics Data System (ADS)

Nappi, Michele; Paolino, Luca; Ricciardi, Stefano; Sebillo, Monica; Vitiello, Giuliana

Aerospace industry has been involved in virtual simulation for design and testing since the birth of virtual reality. Today this industry is showing a growing interest in the development of haptic-based maintenance training applications, which represent the most advanced way to simulate maintenance and repair tasks within a virtual environment by means of a visual-haptic approach. The goal is to allow the trainee to experiment the service procedures not only as a workflow reproduced at a visual level but also in terms of the kinaesthetic feedback involved with the manipulation of tools and components. This study, conducted in collaboration with aerospace industry specialists, is aimed to the development of an immersive virtual capable of immerging the trainees into a virtual environment where mechanics and technicians can perform maintenance simulation or training tasks by directly manipulating 3D virtual models of aircraft parts while perceiving force feedback through the haptic interface. The proposed system is based on ViRstperson, a virtual reality engine under development at the Italian Center for Aerospace Research (CIRA) to support engineering and technical activities such as design-time maintenance procedure validation, and maintenance training. This engine has been extended to support haptic-based interaction, enabling a more complete level of interaction, also in terms of impedance control, and thus fostering the development of haptic knowledge in the user. The user’s “sense of touch” within the immersive virtual environment is simulated through an Immersion CyberForce® hand-based force-feedback device. Preliminary testing of the proposed system seems encouraging.

Graphic and haptic simulation for transvaginal cholecystectomy training in NOTES.

PubMed

Pan, Jun J; Ahn, Woojin; Dargar, Saurabh; Halic, Tansel; Li, Bai C; Sankaranarayanan, Ganesh; Roberts, Kurt; Schwaitzberg, Steven; De, Suvranu

2016-04-01

Natural Orifice Transluminal Endoscopic Surgery (NOTES) provides an emerging surgical technique which usually needs a long learning curve for surgeons. Virtual reality (VR) medical simulators with vision and haptic feedback can usually offer an efficient and cost-effective alternative without risk to the traditional training approaches. Under this motivation, we developed the first virtual reality simulator for transvaginal cholecystectomy in NOTES (VTEST™). This VR-based surgical simulator aims to simulate the hybrid NOTES of cholecystectomy. We use a 6DOF haptic device and a tracking sensor to construct the core hardware component of simulator. For software, an innovative approach based on the inner-spheres is presented to deform the organs in real time. To handle the frequent collision between soft tissue and surgical instruments, an adaptive collision detection method based on GPU is designed and implemented. To give a realistic visual performance of gallbladder fat tissue removal by cautery hook, a multi-layer hexahedral model is presented to simulate the electric dissection of fat tissue. From the experimental results, trainees can operate in real time with high degree of stability and fidelity. A preliminary study was also performed to evaluate the realism and the usefulness of this hybrid NOTES simulator. This prototyped simulation system has been verified by surgeons through a pilot study. Some items of its visual performance and the utility were rated fairly high by the participants during testing. It exhibits the potential to improve the surgical skills of trainee and effectively shorten their learning curve. Copyright © 2016 Elsevier Inc. All rights reserved.

Multi-fingered haptic palpation utilizing granular jamming stiffness feedback actuators

NASA Astrophysics Data System (ADS)

Li, Min; Ranzani, Tommaso; Sareh, Sina; Seneviratne, Lakmal D.; Dasgupta, Prokar; Wurdemann, Helge A.; Althoefer, Kaspar

2014-09-01

This paper describes a multi-fingered haptic palpation method using stiffness feedback actuators for simulating tissue palpation procedures in traditional and in robot-assisted minimally invasive surgery. Soft tissue stiffness is simulated by changing the stiffness property of the actuator during palpation. For the first time, granular jamming and pneumatic air actuation are combined to realize stiffness modulation. The stiffness feedback actuator is validated by stiffness measurements in indentation tests and through stiffness discrimination based on a user study. According to the indentation test results, the introduction of a pneumatic chamber to granular jamming can amplify the stiffness variation range and reduce hysteresis of the actuator. The advantage of multi-fingered palpation using the proposed actuators is proven by the comparison of the results of the stiffness discrimination performance using two-fingered (sensitivity: 82.2%, specificity: 88.9%, positive predicative value: 80.0%, accuracy: 85.4%, time: 4.84 s) and single-fingered (sensitivity: 76.4%, specificity: 85.7%, positive predicative value: 75.3%, accuracy: 81.8%, time: 7.48 s) stiffness feedback.

Experimental evaluation of a miniature MR device for a wide range of human perceivable haptic sensations

NASA Astrophysics Data System (ADS)

Yang, Tae-Heon; Koo, Jeong-Hoi

2017-12-01

Humans can experience a realistic and vivid haptic sensations by the sense of touch. In order to have a fully immersive haptic experience, both kinaesthetic and vibrotactile information must be presented to human users. Currently, little haptic research has been performed on small haptic actuators that can covey both vibrotactile feedback based on the frequency of vibrations up to the human-perceivable limit and multiple levels of kinaesthetic feedback rapidly. Therefore, this study intends to design a miniature haptic device based on MR fluid and experimentally evaluate its ability to convey vibrotactile feedback up to 300 Hz along with kinaesthetic feedback. After constructing a prototype device, a series of testing was performed to evaluate its performance of the prototype using an experimental setup, consisting of a precision dynamic mechanical analyzer and an accelerometer. The kinaesthetic testing results show that the prototype device can provide the force rate up to 89% at 5 V (360 mA), which can be discretized into multiple levels of ‘just noticeable difference’ force rate, indicating that the device can convey a wide range of kinaesthetic sensations. To evaluate the high frequency vibrotactile feedback performance of the device, its acceleration responses were measured and processed using the FFT analysis. The results indicate that the device can convey high frequency vibrotactile sensations up to 300 Hz with the sufficiently large intensity of accelerations that human can feel.

It Pays to Go Off-Track: Practicing with Error-Augmenting Haptic Feedback Facilitates Learning of a Curve-Tracing Task

PubMed Central

Williams, Camille K.; Tremblay, Luc; Carnahan, Heather

2016-01-01

Researchers in the domain of haptic training are now entering the long-standing debate regarding whether or not it is best to learn a skill by experiencing errors. Haptic training paradigms provide fertile ground for exploring how various theories about feedback, errors and physical guidance intersect during motor learning. Our objective was to determine how error minimizing, error augmenting and no haptic feedback while learning a self-paced curve-tracing task impact performance on delayed (1 day) retention and transfer tests, which indicate learning. We assessed performance using movement time and tracing error to calculate a measure of overall performance – the speed accuracy cost function. Our results showed that despite exhibiting the worst performance during skill acquisition, the error augmentation group had significantly better accuracy (but not overall performance) than the error minimization group on delayed retention and transfer tests. The control group’s performance fell between that of the two experimental groups but was not significantly different from either on the delayed retention test. We propose that the nature of the task (requiring online feedback to guide performance) coupled with the error augmentation group’s frequent off-target experience and rich experience of error-correction promoted information processing related to error-detection and error-correction that are essential for motor learning. PMID:28082937

KinoHaptics: An Automated, Wearable, Haptic Assisted, Physio-therapeutic System for Post-surgery Rehabilitation and Self-care.

PubMed

Rajanna, Vijay; Vo, Patrick; Barth, Jerry; Mjelde, Matthew; Grey, Trevor; Oduola, Cassandra; Hammond, Tracy

2016-03-01

A carefully planned, structured, and supervised physiotherapy program, following a surgery, is crucial for the successful diagnosis of physical injuries. Nearly 50 % of the surgeries fail due to unsupervised, and erroneous physiotherapy. The demand for a physiotherapist for an extended period is expensive to afford, and sometimes inaccessible. Researchers have tried to leverage the advancements in wearable sensors and motion tracking by building affordable, automated, physio-therapeutic systems that direct a physiotherapy session by providing audio-visual feedback on patient's performance. There are many aspects of automated physiotherapy program which are yet to be addressed by the existing systems: a wide classification of patients' physiological conditions to be diagnosed, multiple demographics of the patients (blind, deaf, etc.), and the need to pursue patients to adopt the system for an extended period for self-care. In our research, we have tried to address these aspects by building a health behavior change support system called KinoHaptics, for post-surgery rehabilitation. KinoHaptics is an automated, wearable, haptic assisted, physio-therapeutic system that can be used by a wide variety of demographics and for various physiological conditions of the patients. The system provides rich and accurate vibro-haptic feedback that can be felt by the user, irrespective of the physiological limitations. KinoHaptics is built to ensure that no injuries are induced during the rehabilitation period. The persuasive nature of the system allows for personal goal-setting, progress tracking, and most importantly life-style compatibility. The system was evaluated under laboratory conditions, involving 14 users. Results show that KinoHaptics is highly convenient to use, and the vibro-haptic feedback is intuitive, accurate, and has shown to prevent accidental injuries. Also, results show that KinoHaptics is persuasive in nature as it supports behavior change and habit building. The successful acceptance of KinoHaptics, an automated, wearable, haptic assisted, physio-therapeutic system proves the need and future-scope of automated physio-therapeutic systems for self-care and behavior change. It also proves that such systems incorporated with vibro-haptic feedback encourage strong adherence to the physiotherapy program; can have profound impact on the physiotherapy experience resulting in higher acceptance rate.

Safe Local Navigation for Visually Impaired Users With a Time-of-Flight and Haptic Feedback Device.

PubMed

Katzschmann, Robert K; Araki, Brandon; Rus, Daniela

2018-03-01

This paper presents ALVU (Array of Lidars and Vibrotactile Units), a contactless, intuitive, hands-free, and discreet wearable device that allows visually impaired users to detect low- and high-hanging obstacles, as well as physical boundaries in their immediate environment. The solution allows for safe local navigation in both confined and open spaces by enabling the user to distinguish free space from obstacles. The device presented is composed of two parts: a sensor belt and a haptic strap. The sensor belt is an array of time-of-flight distance sensors worn around the front of a user's waist, and the pulses of infrared light provide reliable and accurate measurements of the distances between the user and surrounding obstacles or surfaces. The haptic strap communicates the measured distances through an array of vibratory motors worn around the user's upper abdomen, providing haptic feedback. The linear vibration motors are combined with a point-loaded pretensioned applicator to transmit isolated vibrations to the user. We validated the device's capability in an extensive user study entailing 162 trials with 12 blind users. Users wearing the device successfully walked through hallways, avoided obstacles, and detected staircases.

Robot-Assisted Proprioceptive Training with Added Vibro-Tactile Feedback Enhances Somatosensory and Motor Performance.

PubMed

Cuppone, Anna Vera; Squeri, Valentina; Semprini, Marianna; Masia, Lorenzo; Konczak, Jürgen

2016-01-01

This study examined the trainability of the proprioceptive sense and explored the relationship between proprioception and motor learning. With vision blocked, human learners had to perform goal-directed wrist movements relying solely on proprioceptive/haptic cues to reach several haptically specified targets. One group received additional somatosensory movement error feedback in form of vibro-tactile cues applied to the skin of the forearm. We used a haptic robotic device for the wrist and implemented a 3-day training regimen that required learners to make spatially precise goal-directed wrist reaching movements without vision. We assessed whether training improved the acuity of the wrist joint position sense. In addition, we checked if sensory learning generalized to the motor domain and improved spatial precision of wrist tracking movements that were not trained. The main findings of the study are: First, proprioceptive acuity of the wrist joint position sense improved after training for the group that received the combined proprioceptive/haptic and vibro-tactile feedback (VTF). Second, training had no impact on the spatial accuracy of the untrained tracking task. However, learners who had received VTF significantly reduced their reliance on haptic guidance feedback when performing the untrained motor task. That is, concurrent VTF was highly salient movement feedback and obviated the need for haptic feedback. Third, VTF can be also provided by the limb not involved in the task. Learners who received VTF to the contralateral limb equally benefitted. In conclusion, somatosensory training can significantly enhance proprioceptive acuity within days when learning is coupled with vibro-tactile sensory cues that provide feedback about movement errors. The observable sensory improvements in proprioception facilitates motor learning and such learning may generalize to the sensorimotor control of the untrained motor tasks. The implications of these findings for neurorehabilitation are discussed.

Patient DF's visual brain in action: Visual feedforward control in visual form agnosia.

PubMed

Whitwell, Robert L; Milner, A David; Cavina-Pratesi, Cristiana; Barat, Masihullah; Goodale, Melvyn A

2015-05-01

Patient DF, who developed visual form agnosia following ventral-stream damage, is unable to discriminate the width of objects, performing at chance, for example, when asked to open her thumb and forefinger a matching amount. Remarkably, however, DF adjusts her hand aperture to accommodate the width of objects when reaching out to pick them up (grip scaling). While this spared ability to grasp objects is presumed to be mediated by visuomotor modules in her relatively intact dorsal stream, it is possible that it may rely abnormally on online visual or haptic feedback. We report here that DF's grip scaling remained intact when her vision was completely suppressed during grasp movements, and it still dissociated sharply from her poor perceptual estimates of target size. We then tested whether providing trial-by-trial haptic feedback after making such perceptual estimates might improve DF's performance, but found that they remained significantly impaired. In a final experiment, we re-examined whether DF's grip scaling depends on receiving veridical haptic feedback during grasping. In one condition, the haptic feedback was identical to the visual targets. In a second condition, the haptic feedback was of a constant intermediate width while the visual target varied trial by trial. Despite this incongruent feedback, DF still scaled her grip aperture to the visual widths of the target blocks, showing only normal adaptation to the false haptically-experienced width. Taken together, these results strengthen the view that DF's spared grasping relies on a normal mode of dorsal-stream functioning, based chiefly on visual feedforward processing. Copyright © 2014 Elsevier B.V. All rights reserved.

Ultraino: An Open Phased-Array System for Narrowband Airborne Ultrasound Transmission.

PubMed

Marzo, Asier; Corkett, Tom; Drinkwater, Bruce W

2018-01-01

Modern ultrasonic phased-array controllers are electronic systems capable of delaying the transmitted or received signals of multiple transducers. Configurable transmit-receive array systems, capable of electronic steering and shaping of the beam in near real-time, are available commercially, for example, for medical imaging. However, emerging applications, such as ultrasonic haptics, parametric audio, or ultrasonic levitation, require only a small subset of the capabilities provided by the existing controllers. To meet this need, we present Ultraino, a modular, inexpensive, and open platform that provides hardware, software, and example applications specifically aimed at controlling the transmission of narrowband airborne ultrasound. Our system is composed of software, driver boards, and arrays that enable users to quickly and efficiently perform research in various emerging applications. The software can be used to define array geometries, simulate the acoustic field in real time, and control the connected driver boards. The driver board design is based on an Arduino Mega and can control 64 channels with a square wave of up to 17 Vpp and /5 phase resolution. Multiple boards can be chained together to increase the number of channels. The 40-kHz arrays with flat and spherical geometries are demonstrated for parametric audio generation, acoustic levitation, and haptic feedback.

Self-Control of Haptic Assistance for Motor Learning: Influences of Frequency and Opinion of Utility

PubMed Central

Williams, Camille K.; Tseung, Victrine; Carnahan, Heather

2017-01-01

Studies of self-controlled practice have shown benefits when learners controlled feedback schedule, use of assistive devices and task difficulty, with benefits attributed to information processing and motivational advantages of self-control. Although haptic assistance serves as feedback, aids task performance and modifies task difficulty, researchers have yet to explore whether self-control over haptic assistance could be beneficial for learning. We explored whether self-control of haptic assistance would be beneficial for learning a tracing task. Self-controlled participants selected practice blocks on which they would receive haptic assistance, while participants in a yoked group received haptic assistance on blocks determined by a matched self-controlled participant. We inferred learning from performance on retention tests without haptic assistance. From qualitative analysis of open-ended questions related to rationales for/experiences of the haptic assistance that was chosen/provided, themes emerged regarding participants’ views of the utility of haptic assistance for performance and learning. Results showed that learning was directly impacted by the frequency of haptic assistance for self-controlled participants only and view of haptic assistance. Furthermore, self-controlled participants’ views were significantly associated with their requested haptic assistance frequency. We discuss these findings as further support for the beneficial role of self-controlled practice for motor learning. PMID:29255438

Exploring the simulation requirements for virtual regional anesthesia training

NASA Astrophysics Data System (ADS)

Charissis, V.; Zimmer, C. R.; Sakellariou, S.; Chan, W.

2010-01-01

This paper presents an investigation towards the simulation requirements for virtual regional anaesthesia training. To this end we have developed a prototype human-computer interface designed to facilitate Virtual Reality (VR) augmenting educational tactics for regional anaesthesia training. The proposed interface system, aims to compliment nerve blocking techniques methods. The system is designed to operate in real-time 3D environment presenting anatomical information and enabling the user to explore the spatial relation of different human parts without any physical constrains. Furthermore the proposed system aims to assist the trainee anaesthetists so as to build a mental, three-dimensional map of the anatomical elements and their depictive relationship to the Ultra-Sound imaging which is used for navigation of the anaesthetic needle. Opting for a sophisticated approach of interaction, the interface elements are based on simplified visual representation of real objects, and can be operated through haptic devices and surround auditory cues. This paper discusses the challenges involved in the HCI design, introduces the visual components of the interface and presents a tentative plan of future work which involves the development of realistic haptic feedback and various regional anaesthesia training scenarios.

Metal Sounds Stiffer than Drums for Ears, but Not Always for Hands: Low-Level Auditory Features Affect Multisensory Stiffness Perception More than High-Level Categorical Information

PubMed Central

Liu, Juan; Ando, Hiroshi

2016-01-01

Most real-world events stimulate multiple sensory modalities simultaneously. Usually, the stiffness of an object is perceived haptically. However, auditory signals also contain stiffness-related information, and people can form impressions of stiffness from the different impact sounds of metal, wood, or glass. To understand whether there is any interaction between auditory and haptic stiffness perception, and if so, whether the inferred material category is the most relevant auditory information, we conducted experiments using a force-feedback device and the modal synthesis method to present haptic stimuli and impact sound in accordance with participants’ actions, and to modulate low-level acoustic parameters, i.e., frequency and damping, without changing the inferred material categories of sound sources. We found that metal sounds consistently induced an impression of stiffer surfaces than did drum sounds in the audio-only condition, but participants haptically perceived surfaces with modulated metal sounds as significantly softer than the same surfaces with modulated drum sounds, which directly opposes the impression induced by these sounds alone. This result indicates that, although the inferred material category is strongly associated with audio-only stiffness perception, low-level acoustic parameters, especially damping, are more tightly integrated with haptic signals than the material category is. Frequency played an important role in both audio-only and audio-haptic conditions. Our study provides evidence that auditory information influences stiffness perception differently in unisensory and multisensory tasks. Furthermore, the data demonstrated that sounds with higher frequency and/or shorter decay time tended to be judged as stiffer, and contact sounds of stiff objects had no effect on the haptic perception of soft surfaces. We argue that the intrinsic physical relationship between object stiffness and acoustic parameters may be applied as prior knowledge to achieve robust estimation of stiffness in multisensory perception. PMID:27902718

Enhancing audiovisual experience with haptic feedback: a survey on HAV.

PubMed

Danieau, F; Lecuyer, A; Guillotel, P; Fleureau, J; Mollet, N; Christie, M

2013-01-01

Haptic technology has been widely employed in applications ranging from teleoperation and medical simulation to art and design, including entertainment, flight simulation, and virtual reality. Today there is a growing interest among researchers in integrating haptic feedback into audiovisual systems. A new medium emerges from this effort: haptic-audiovisual (HAV) content. This paper presents the techniques, formalisms, and key results pertinent to this medium. We first review the three main stages of the HAV workflow: the production, distribution, and rendering of haptic effects. We then highlight the pressing necessity for evaluation techniques in this context and discuss the key challenges in the field. By building on existing technologies and tackling the specific challenges of the enhancement of audiovisual experience with haptics, we believe the field presents exciting research perspectives whose financial and societal stakes are significant.

The Efficacy of Surface Haptics and Force Feedback in Education

ERIC Educational Resources Information Center

Gorlewicz, Jenna Lynn

2013-01-01

This dissertation bridges the fields of haptics, engineering, and education to realize some of the potential benefits haptic devices may have in Science, Technology, Engineering, and Math (STEM) education. Specifically, this dissertation demonstrates the development, implementation, and assessment of two haptic devices in engineering and math…

Incorporating Haptic Feedback in Simulation for Learning Physics

ERIC Educational Resources Information Center

Han, Insook; Black, John B.

2011-01-01

The purpose of this study was to investigate the effectiveness of a haptic augmented simulation in learning physics. The results indicate that haptic augmented simulations, both the force and kinesthetic and the purely kinesthetic simulations, were more effective than the equivalent non-haptic simulation in providing perceptual experiences and…

Virtual Reality simulator for dental anesthesia training in the inferior alveolar nerve block.

PubMed

Corrêa, Cléber Gimenez; Machado, Maria Aparecida de Andrade Moreira; Ranzini, Edith; Tori, Romero; Nunes, Fátima de Lourdes Santos

2017-01-01

This study shows the development and validation of a dental anesthesia-training simulator, specifically for the inferior alveolar nerve block (IANB). The system developed provides the tactile sensation of inserting a real needle in a human patient, using Virtual Reality (VR) techniques and a haptic device that can provide a perceived force feedback in the needle insertion task during the anesthesia procedure. To simulate a realistic anesthesia procedure, a Carpule syringe was coupled to a haptic device. The Volere method was used to elicit requirements from users in the Dentistry area; Repeated Measures Two-Way ANOVA (Analysis of Variance), Tukey post-hoc test and averages for the results' analysis. A questionnaire-based subjective evaluation method was applied to collect information about the simulator, and 26 people participated in the experiments (12 beginners, 12 at intermediate level, and 2 experts). The questionnaire included profile, preferences (number of viewpoints, texture of the objects, and haptic device handler), as well as visual (appearance, scale, and position of objects) and haptic aspects (motion space, tactile sensation, and motion reproduction). The visual aspect was considered appropriate and the haptic feedback must be improved, which the users can do by calibrating the virtual tissues' resistance. The evaluation of visual aspects was influenced by the participants' experience, according to ANOVA test (F=15.6, p=0.0002, with p<0.01). The user preferences were the simulator with two viewpoints, objects with texture based on images and the device with a syringe coupled to it. The simulation was considered thoroughly satisfactory for the anesthesia training, considering the needle insertion task, which includes the correct insertion point and depth, as well as the perception of tissues resistances during the insertion.

Invited Article: A review of haptic optical tweezers for an interactive microworld exploration

NASA Astrophysics Data System (ADS)

Pacoret, Cécile; Régnier, Stéphane

2013-08-01

This paper is the first review of haptic optical tweezers, a new technique which associates force feedback teleoperation with optical tweezers. This technique allows users to explore the microworld by sensing and exerting picoNewton-scale forces with trapped microspheres. Haptic optical tweezers also allow improved dexterity of micromanipulation and micro-assembly. One of the challenges of this technique is to sense and magnify picoNewton-scale forces by a factor of 1012 to enable human operators to perceive interactions that they have never experienced before, such as adhesion phenomena, extremely low inertia, and high frequency dynamics of extremely small objects. The design of optical tweezers for high quality haptic feedback is challenging, given the requirements for very high sensitivity and dynamic stability. The concept, design process, and specification of optical tweezers reviewed here are focused on those intended for haptic teleoperation. In this paper, two new specific designs as well as the current state-of-the-art are presented. Moreover, the remaining important issues are identified for further developments. The initial results obtained are promising and demonstrate that optical tweezers have a significant potential for haptic exploration of the microworld. Haptic optical tweezers will become an invaluable tool for force feedback micromanipulation of biological samples and nano- and micro-assembly parts.

Computer-aided trauma simulation system with haptic feedback is easy and fast for oral-maxillofacial surgeons to learn and use.

PubMed

Schvartzman, Sara C; Silva, Rebeka; Salisbury, Ken; Gaudilliere, Dyani; Girod, Sabine

2014-10-01

Computer-assisted surgical (CAS) planning tools have become widely available in craniomaxillofacial surgery, but are time consuming and often require professional technical assistance to simulate a case. An initial oral and maxillofacial (OM) surgical user experience was evaluated with a newly developed CAS system featuring a bimanual sense of touch (haptic). Three volunteer OM surgeons received a 5-minute verbal introduction to the use of a newly developed haptic-enabled planning system. The surgeons were instructed to simulate mandibular fracture reductions of 3 clinical cases, within a 15-minute time limit and without a time limit, and complete a questionnaire to assess their subjective experience with the system. Standard landmarks and linear and angular measurements between the simulated results and the actual surgical outcome were compared. After the 5-minute instruction, all 3 surgeons were able to use the system independently. The analysis of standardized anatomic measurements showed that the simulation results within a 15-minute time limit were not significantly different from those without a time limit. Mean differences between measurements of surgical and simulated fracture reductions were within current resolution limitations in collision detection, segmentation of computed tomographic scans, and haptic devices. All 3 surgeons reported that the system was easy to learn and use and that they would be comfortable integrating it into their daily clinical practice for trauma cases. A CAS system with a haptic interface that capitalizes on touch and force feedback experience similar to operative procedures is fast and easy for OM surgeons to learn and use. Copyright © 2014 American Association of Oral and Maxillofacial Surgeons. All rights reserved.

Force modeling for incisions into various tissues with MRF haptic master

NASA Astrophysics Data System (ADS)

Kim, Pyunghwa; Kim, Soomin; Park, Young-Dai; Choi, Seung-Bok

2016-03-01

This study proposes a new model to predict the reaction force that occurs in incisions during robot-assisted minimally invasive surgery. The reaction force is fed back to the manipulator by a magneto-rheological fluid (MRF) haptic master, which is featured by a bi-directional clutch actuator. The reaction force feedback provides similar sensations to laparotomy that cannot be provided by a conventional master for surgery. This advantage shortens the training period for robot-assisted minimally invasive surgery and can improve the accuracy of operations. The reaction force modeling of incisions can be utilized in a surgical simulator that provides a virtual reaction force. In this work, in order to model the reaction force during incisions, the energy aspect of the incision process is adopted and analyzed. Each mode of the incision process is classified by the tendency of the energy change, and modeled for realistic real-time application. The reaction force model uses actual reaction force information with three types of actual tissues: hard tissue, medium tissue, and soft tissue. This modeled force is realized by the MRF haptic master through an algorithm based on the position and velocity of a scalpel using two different control methods: an open-loop algorithm and a closed-loop algorithm. The reaction forces obtained from the proposed model are compared with a desired force in time domain.

Synergistic Effects on the Elderly People's Motor Control by Wearable Skin-Stretch Device Combined with Haptic Joystick

PubMed Central

Yoon, Han U.; Anil Kumar, Namita; Hur, Pilwon

2017-01-01

Cutaneous sensory feedback can be used to provide additional sensory cues to a person performing a motor task where vision is a dominant feedback signal. A haptic joystick has been widely used to guide a user by providing force feedback. However, the benefit of providing force feedback is still debatable due to performance dependency on factors such as the user's skill-level, task difficulty. Meanwhile, recent studies have shown the feasibility of improving a motor task performance by providing skin-stretch feedback. Therefore, a combination of two aforementioned feedback types is deemed to be promising to promote synergistic effects to consistently improve the person's motor performance. In this study, we aimed at identifying the effect of the combined haptic and skin-stretch feedbacks on the aged person's driving motor performance. For the experiment, 15 healthy elderly subjects (age 72.8 ± 6.6 years) were recruited and were instructed to drive a virtual power-wheelchair through four different courses with obstacles. Four augmented sensory feedback conditions were tested: no feedback, force feedback, skin-stretch feedback, and a combination of both force and skin-stretch feedbacks. While the haptic force was provided to the hand by the joystick, the skin-stretch was provided to the steering forearm by a custom-designed wearable skin-stretch device. We tested two hypotheses: (i) an elderly individual's motor control would benefit from receiving information about a desired trajectory from multiple sensory feedback sources, and (ii) the benefit does not depend on task difficulty. Various metrics related to skills and safety were used to evaluate the control performance. Repeated measure ANOVA was performed for those metrics with two factors: task scenario and the type of the augmented sensory feedback. The results revealed that elderly subjects' control performance significantly improved when the combined feedback of both haptic force and skin-stretch feedback was applied. The proposed approach suggest the feasibility to improve people's task performance by the synergistic effects of multiple augmented sensory feedback modalities. PMID:28690514

Using haptic feedback to increase seat belt use of service vehicle drivers.

DOT National Transportation Integrated Search

2011-01-01

This study pilot-tested a new application of a technology-based intervention to increase seat belt use. The technology was based on a : contingency in which unbelted drivers experienced sustained haptic feedback to the gas pedal when they exceeded 25...

Towards a Teleoperated Needle Driver Robot with Haptic Feedback for RFA of Breast Tumors under Continuous MRI1

PubMed Central

Kokes, Rebecca; Lister, Kevin; Gullapalli, Rao; Zhang, Bao; MacMillan, Alan; Richard, Howard; Desai, Jaydev P.

2009-01-01

Objective The purpose of this paper is to explore the feasibility of developing a MRI-compatible needle driver system for radiofrequency ablation (RFA) of breast tumors under continuous MRI imaging while being teleoperated by a haptic feedback device from outside the scanning room. The developed needle driver prototype was designed and tested for both tumor targeting capability as well as RFA. Methods The single degree-of-freedom (DOF) prototype was interfaced with a PHANToM haptic device controlled from outside the scanning room. Experiments were performed to demonstrate MRI-compatibility and position control accuracy with hydraulic actuation, along with an experiment to determine the PHANToM’s ability to guide the RFA tool to a tumor nodule within a phantom breast tissue model while continuously imaging within the MRI and receiving force feedback from the RFA tool. Results Hydraulic actuation is shown to be a feasible actuation technique for operation in an MRI environment. The design is MRI-compatible in all aspects except for force sensing in the directions perpendicular to the direction of motion. Experiments confirm that the user is able to detect healthy vs. cancerous tissue in a phantom model when provided with both visual (imaging) feedback and haptic feedback. Conclusion The teleoperated 1-DOF needle driver system presented in this paper demonstrates the feasibility of implementing a MRI-compatible robot for RFA of breast tumors with haptic feedback capability. PMID:19303805

Algorithms for Haptic Rendering of 3D Objects

NASA Technical Reports Server (NTRS)

Basdogan, Cagatay; Ho, Chih-Hao; Srinavasan, Mandayam

2003-01-01

Algorithms have been developed to provide haptic rendering of three-dimensional (3D) objects in virtual (that is, computationally simulated) environments. The goal of haptic rendering is to generate tactual displays of the shapes, hardnesses, surface textures, and frictional properties of 3D objects in real time. Haptic rendering is a major element of the emerging field of computer haptics, which invites comparison with computer graphics. We have already seen various applications of computer haptics in the areas of medicine (surgical simulation, telemedicine, haptic user interfaces for blind people, and rehabilitation of patients with neurological disorders), entertainment (3D painting, character animation, morphing, and sculpting), mechanical design (path planning and assembly sequencing), and scientific visualization (geophysical data analysis and molecular manipulation).

A pervasive visual-haptic framework for virtual delivery training.

PubMed

Abate, Andrea F; Acampora, Giovanni; Loia, Vincenzo; Ricciardi, Stefano; Vasilakos, Athanasios V

2010-03-01

Thanks to the advances of voltage regulator (VR) technologies and haptic systems, virtual simulators are increasingly becoming a viable alternative to physical simulators in medicine and surgery, though many challenges still remain. In this study, a pervasive visual-haptic framework aimed to the training of obstetricians and midwives to vaginal delivery is described. The haptic feedback is provided by means of two hand-based haptic devices able to reproduce force-feedbacks on fingers and arms, thus enabling a much more realistic manipulation respect to stylus-based solutions. The interactive simulation is not solely driven by an approximated model of complex forces and physical constraints but, instead, is approached by a formal modeling of the whole labor and of the assistance/intervention procedures performed by means of a timed automata network and applied to a parametrical 3-D model of the anatomy, able to mimic a wide range of configurations. This novel methodology is able to represent not only the sequence of the main events associated to either a spontaneous or to an operative childbirth process, but also to help in validating the manual intervention as the actions performed by the user during the simulation are evaluated according to established medical guidelines. A discussion on the first results as well as on the challenges still unaddressed is included.

Neuromorphic Vibrotactile Stimulation of Fingertips for Encoding Object Stiffness in Telepresence Sensory Substitution and Augmentation Applications

PubMed Central

Sorgini, Francesca; Massari, Luca; D’Abbraccio, Jessica; Petrovic, Petar B.; Carrozza, Maria Chiara; Newell, Fiona N.

2018-01-01

We present a tactile telepresence system for real-time transmission of information about object stiffness to the human fingertips. Experimental tests were performed across two laboratories (Italy and Ireland). In the Italian laboratory, a mechatronic sensing platform indented different rubber samples. Information about rubber stiffness was converted into on-off events using a neuronal spiking model and sent to a vibrotactile glove in the Irish laboratory. Participants discriminated the variation of the stiffness of stimuli according to a two-alternative forced choice protocol. Stiffness discrimination was based on the variation of the temporal pattern of spikes generated during the indentation of the rubber samples. The results suggest that vibrotactile stimulation can effectively simulate surface stiffness when using neuronal spiking models to trigger vibrations in the haptic interface. Specifically, fractional variations of stiffness down to 0.67 were significantly discriminated with the developed neuromorphic haptic interface. This is a performance comparable, though slightly worse, to the threshold obtained in a benchmark experiment evaluating the same set of stimuli naturally with the own hand. Our paper presents a bioinspired method for delivering sensory feedback about object properties to human skin based on contingency–mimetic neuronal models, and can be useful for the design of high performance haptic devices. PMID:29342076

Haptic Technologies for MEMS Design

NASA Astrophysics Data System (ADS)

Calis, Mustafa; Desmulliez, Marc P. Y.

2006-04-01

This paper presents for the first time a design methodology for MEMS/NEMS based on haptic sensing technologies. The software tool created as a result of this methodology will enable designers to model and interact in real time with their virtual prototype. One of the main advantages of haptic sensing is the ability to bring unusual microscopic forces back to the designer's world. Other significant benefits for developing such a methodology include gain productivity and the capability to include manufacturing costs within the design cycle.

Object discrimination using optimized multi-frequency auditory cross-modal haptic feedback.

PubMed

Gibson, Alison; Artemiadis, Panagiotis

2014-01-01

As the field of brain-machine interfaces and neuro-prosthetics continues to grow, there is a high need for sensor and actuation mechanisms that can provide haptic feedback to the user. Current technologies employ expensive, invasive and often inefficient force feedback methods, resulting in an unrealistic solution for individuals who rely on these devices. This paper responds through the development, integration and analysis of a novel feedback architecture where haptic information during the neural control of a prosthetic hand is perceived through multi-frequency auditory signals. Through representing force magnitude with volume and force location with frequency, the feedback architecture can translate the haptic experiences of a robotic end effector into the alternative sensory modality of sound. Previous research with the proposed cross-modal feedback method confirmed its learnability, so the current work aimed to investigate which frequency map (i.e. frequency-specific locations on the hand) is optimal in helping users distinguish between hand-held objects and tasks associated with them. After short use with the cross-modal feedback during the electromyographic (EMG) control of a prosthetic hand, testing results show that users are able to use audial feedback alone to discriminate between everyday objects. While users showed adaptation to three different frequency maps, the simplest map containing only two frequencies was found to be the most useful in discriminating between objects. This outcome provides support for the feasibility and practicality of the cross-modal feedback method during the neural control of prosthetics.

Virtual reality neurosurgery: a simulator blueprint.

PubMed

Spicer, Mark A; van Velsen, Martin; Caffrey, John P; Apuzzo, Michael L J

2004-04-01

This article details preliminary studies undertaken to integrate the most relevant advancements across multiple disciplines in an effort to construct a highly realistic neurosurgical simulator based on a distributed computer architecture. Techniques based on modified computational modeling paradigms incorporating finite element analysis are presented, as are current and projected efforts directed toward the implementation of a novel bidirectional haptic device. Patient-specific data derived from noninvasive magnetic resonance imaging sequences are used to construct a computational model of the surgical region of interest. Magnetic resonance images of the brain may be coregistered with those obtained from magnetic resonance angiography, magnetic resonance venography, and diffusion tensor imaging to formulate models of varying anatomic complexity. The majority of the computational burden is encountered in the presimulation reduction of the computational model and allows realization of the required threshold rates for the accurate and realistic representation of real-time visual animations. Intracranial neurosurgical procedures offer an ideal testing site for the development of a totally immersive virtual reality surgical simulator when compared with the simulations required in other surgical subspecialties. The material properties of the brain as well as the typically small volumes of tissue exposed in the surgical field, coupled with techniques and strategies to minimize computational demands, provide unique opportunities for the development of such a simulator. Incorporation of real-time haptic and visual feedback is approached here and likely will be accomplished soon.

Assessment of Haptic Interaction for Home-Based Physical Tele-Therapy using Wearable Devices and Depth Sensors.

PubMed

Barmpoutis, Angelos; Alzate, Jose; Beekhuizen, Samantha; Delgado, Horacio; Donaldson, Preston; Hall, Andrew; Lago, Charlie; Vidal, Kevin; Fox, Emily J

2016-01-01

In this paper a prototype system is presented for home-based physical tele-therapy using a wearable device for haptic feedback. The haptic feedback is generated as a sequence of vibratory cues from 8 vibrator motors equally spaced along an elastic wearable band. The motors guide the patients' movement as they perform a prescribed exercise routine in a way that replaces the physical therapists' haptic guidance in an unsupervised or remotely supervised home-based therapy session. A pilot study of 25 human subjects was performed that focused on: a) testing the capability of the system to guide the users in arbitrary motion paths in the space and b) comparing the motion of the users during typical physical therapy exercises with and without haptic-based guidance. The results demonstrate the efficacy of the proposed system.

Haptic Paddle Enhancements and a Formal Assessment of Student Learning in System Dynamics

ERIC Educational Resources Information Center

Gorlewicz, Jenna L.; Kratchman, Louis B.; Webster, Robert J., III

2014-01-01

The haptic paddle is a force-feedback joystick used at several universities in teaching System Dynamics, a core mechanical engineering undergraduate course where students learn to model dynamic systems in several domains. A second goal of the haptic paddle is to increase the accessibility of robotics and haptics by providing a low-cost device for…

Deformation of Soft Tissue and Force Feedback Using the Smoothed Particle Hydrodynamics

PubMed Central

Liu, Xuemei; Wang, Ruiyi; Li, Yunhua; Song, Dongdong

2015-01-01

We study the deformation and haptic feedback of soft tissue in virtual surgery based on a liver model by using a force feedback device named PHANTOM OMNI developed by SensAble Company in USA. Although a significant amount of research efforts have been dedicated to simulating the behaviors of soft tissue and implementing force feedback, it is still a challenging problem. This paper introduces a kind of meshfree method for deformation simulation of soft tissue and force computation based on viscoelastic mechanical model and smoothed particle hydrodynamics (SPH). Firstly, viscoelastic model can present the mechanical characteristics of soft tissue which greatly promotes the realism. Secondly, SPH has features of meshless technique and self-adaption, which supply higher precision than methods based on meshes for force feedback computation. Finally, a SPH method based on dynamic interaction area is proposed to improve the real time performance of simulation. The results reveal that SPH methodology is suitable for simulating soft tissue deformation and force feedback calculation, and SPH based on dynamic local interaction area has a higher computational efficiency significantly compared with usual SPH. Our algorithm has a bright prospect in the area of virtual surgery. PMID:26417380

A haptics-assisted cranio-maxillofacial surgery planning system for restoring skeletal anatomy in complex trauma cases.

PubMed

Olsson, Pontus; Nysjö, Fredrik; Hirsch, Jan-Michaél; Carlbom, Ingrid B

2013-11-01

   Cranio-maxillofacial (CMF) surgery to restore normal skeletal anatomy in patients with serious trauma to the face can be both complex and time-consuming. But it is generally accepted that careful pre-operative planning leads to a better outcome with a higher degree of function and reduced morbidity in addition to reduced time in the operating room. However, today's surgery planning systems are primitive, relying mostly on the user's ability to plan complex tasks with a two-dimensional graphical interface.    A system for planning the restoration of skeletal anatomy in facial trauma patients using a virtual model derived from patient-specific CT data. The system combines stereo visualization with six degrees-of-freedom, high-fidelity haptic feedback that enables analysis, planning, and preoperative testing of alternative solutions for restoring bone fragments to their proper positions. The stereo display provides accurate visual spatial perception, and the haptics system provides intuitive haptic feedback when bone fragments are in contact as well as six degrees-of-freedom attraction forces for precise bone fragment alignment.    A senior surgeon without prior experience of the system received 45 min of system training. Following the training session, he completed a virtual reconstruction in 22 min of a complex mandibular fracture with an adequately reduced result.    Preliminary testing with one surgeon indicates that our surgery planning system, which combines stereo visualization with sophisticated haptics, has the potential to become a powerful tool for CMF surgery planning. With little training, it allows a surgeon to complete a complex plan in a short amount of time.

Design and performance evaluation of collision protection-based safety operation for a haptic robot-assisted catheter operating system.

PubMed

Zhang, Linshuai; Guo, Shuxiang; Yu, Huadong; Song, Yu; Tamiya, Takashi; Hirata, Hideyuki; Ishihara, Hidenori

2018-02-23

The robot-assisted catheter system can increase operating distance thus preventing the exposure radiation of the surgeon to X-ray for endovascular catheterization. However, few designs have considered the collision protection between the catheter tip and the vessel wall. This paper presents a novel catheter operating system based on tissue protection to prevent vessel puncture caused by collision. The integrated haptic interface not only allows the operator to feel the real force feedback, but also combines with the newly proposed collision protection mechanism (CPM) to mitigate the collision trauma. The CPM can release the catheter quickly when the measured force exceeds a certain threshold, so as to avoid the vessel puncture. A significant advantage is that the proposed mechanism can adjust the protection threshold in real time by the current according to the actual characteristics of the blood vessel. To verify the effectiveness of the tissue protection by the system, the evaluation experiments in vitro were carried out. The results show that the further collision damage can be effectively prevented by the CPM, which implies the realization of relative safe catheterization. This research provides some insights into the functional improvements of safe and reliable robot-assisted catheter systems.

Shared control of a medical robot with haptic guidance.

PubMed

Xiong, Linfei; Chng, Chin Boon; Chui, Chee Kong; Yu, Peiwu; Li, Yao

2017-01-01

Tele-operation of robotic surgery reduces the radiation exposure during the interventional radiological operations. However, endoscope vision without force feedback on the surgical tool increases the difficulty for precise manipulation and the risk of tissue damage. The shared control of vision and force provides a novel approach of enhanced control with haptic guidance, which could lead to subtle dexterity and better maneuvrability during MIS surgery. The paper provides an innovative shared control method for robotic minimally invasive surgery system, in which vision and haptic feedback are incorporated to provide guidance cues to the clinician during surgery. The incremental potential field (IPF) method is utilized to generate a guidance path based on the anatomy of tissue and surgical tool interaction. Haptic guidance is provided at the master end to assist the clinician during tele-operative surgical robotic task. The approach has been validated with path following and virtual tumor targeting experiments. The experiment results demonstrate that comparing with vision only guidance, the shared control with vision and haptics improved the accuracy and efficiency of surgical robotic manipulation, where the tool-position error distance and execution time are reduced. The validation experiment demonstrates that the shared control approach could help the surgical robot system provide stable assistance and precise performance to execute the designated surgical task. The methodology could also be implemented with other surgical robot with different surgical tools and applications.

The Role of Direct and Visual Force Feedback in Suturing Using a 7-DOF Dual-Arm Teleoperated System.

PubMed

Talasaz, Ali; Trejos, Ana Luisa; Patel, Rajni V

2017-01-01

The lack of haptic feedback in robotics-assisted surgery can result in tissue damage or accidental tool-tissue hits. This paper focuses on exploring the effect of haptic feedback via direct force reflection and visual presentation of force magnitudes on performance during suturing in robotics-assisted minimally invasive surgery (RAMIS). For this purpose, a haptics-enabled dual-arm master-slave teleoperation system capable of measuring tool-tissue interaction forces in all seven Degrees-of-Freedom (DOFs) was used. Two suturing tasks, tissue puncturing and knot-tightening, were chosen to assess user skills when suturing on phantom tissue. Sixteen subjects participated in the trials and their performance was evaluated from various points of view: force consistency, number of accidental hits with tissue, amount of tissue damage, quality of the suture knot, and the time required to accomplish the task. According to the results, visual force feedback was not very useful during the tissue puncturing task as different users needed different amounts of force depending on the penetration of the needle into the tissue. Direct force feedback, however, was more useful for this task to apply less force and to minimize the amount of damage to the tissue. Statistical results also reveal that both visual and direct force feedback were required for effective knot tightening: direct force feedback could reduce the number of accidental hits with the tissue and also the amount of tissue damage, while visual force feedback could help to securely tighten the suture knots and maintain force consistency among different trials/users. These results provide evidence of the importance of 7-DOF force reflection when performing complex tasks in a RAMIS setting.

Human-computer interface including haptically controlled interactions

DOEpatents

Anderson, Thomas G.

2005-10-11

The present invention provides a method of human-computer interfacing that provides haptic feedback to control interface interactions such as scrolling or zooming within an application. Haptic feedback in the present method allows the user more intuitive control of the interface interactions, and allows the user's visual focus to remain on the application. The method comprises providing a control domain within which the user can control interactions. For example, a haptic boundary can be provided corresponding to scrollable or scalable portions of the application domain. The user can position a cursor near such a boundary, feeling its presence haptically (reducing the requirement for visual attention for control of scrolling of the display). The user can then apply force relative to the boundary, causing the interface to scroll the domain. The rate of scrolling can be related to the magnitude of applied force, providing the user with additional intuitive, non-visual control of scrolling.

Prediction of Knee Joint Contact Forces From External Measures Using Principal Component Prediction and Reconstruction.

PubMed

Saliba, Christopher M; Clouthier, Allison L; Brandon, Scott C E; Rainbow, Michael J; Deluzio, Kevin J

2018-05-29

Abnormal loading of the knee joint contributes to the pathogenesis of knee osteoarthritis. Gait retraining is a non-invasive intervention that aims to reduce knee loads by providing audible, visual, or haptic feedback of gait parameters. The computational expense of joint contact force prediction has limited real-time feedback to surrogate measures of the contact force, such as the knee adduction moment. We developed a method to predict knee joint contact forces using motion analysis and a statistical regression model that can be implemented in near real-time. Gait waveform variables were deconstructed using principal component analysis and a linear regression was used to predict the principal component scores of the contact force waveforms. Knee joint contact force waveforms were reconstructed using the predicted scores. We tested our method using a heterogenous population of asymptomatic controls and subjects with knee osteoarthritis. The reconstructed contact force waveforms had mean (SD) RMS differences of 0.17 (0.05) bodyweight compared to the contact forces predicted by a musculoskeletal model. Our method successfully predicted subject-specific shape features of contact force waveforms and is a potentially powerful tool in biofeedback and clinical gait analysis.

Multimodal Interaction with Speech, Gestures and Haptic Feedback in a Media Center Application

NASA Astrophysics Data System (ADS)

Turunen, Markku; Hakulinen, Jaakko; Hella, Juho; Rajaniemi, Juha-Pekka; Melto, Aleksi; Mäkinen, Erno; Rantala, Jussi; Heimonen, Tomi; Laivo, Tuuli; Soronen, Hannu; Hansen, Mervi; Valkama, Pellervo; Miettinen, Toni; Raisamo, Roope

We demonstrate interaction with a multimodal media center application. Mobile phone-based interface includes speech and gesture input and haptic feedback. The setup resembles our long-term public pilot study, where a living room environment containing the application was constructed inside a local media museum allowing visitors to freely test the system.

Haptic seat for fuel economy feedback

DOEpatents

Bobbitt, III, John Thomas

2016-08-30

A process of providing driver fuel economy feedback is disclosed in which vehicle sensors provide for haptic feedback on fuel usage. Such sensors may include one or more of a speed sensors, global position satellite units, vehicle pitch/roll angle sensors, suspension displacement sensors, longitudinal accelerometer sensors, throttle position in sensors, steering angle sensors, break pressure sensors, and lateral accelerometer sensors. Sensors used singlely or collectively can provide enhanced feedback as to various environmental conditions and operating conditions such that a more accurate assessment of fuel economy information can be provided to the driver.

A novel shape-changing haptic table-top display

NASA Astrophysics Data System (ADS)

Wang, Jiabin; Zhao, Lu; Liu, Yue; Wang, Yongtian; Cai, Yi

2018-01-01

A shape-changing table-top display with haptic feedback allows its users to perceive 3D visual and texture displays interactively. Since few existing devices are developed as accurate displays with regulatory haptic feedback, a novel attentive and immersive shape changing mechanical interface (SCMI) consisting of image processing unit and transformation unit was proposed in this paper. In order to support a precise 3D table-top display with an offset of less than 2 mm, a custommade mechanism was developed to form precise surface and regulate the feedback force. The proposed image processing unit was capable of extracting texture data from 2D picture for rendering shape-changing surface and realizing 3D modeling. The preliminary evaluation result proved the feasibility of the proposed system.

[Experimental study of angiography using vascular interventional robot-2(VIR-2)].

PubMed

Tian, Zeng-min; Lu, Wang-sheng; Liu, Da; Wang, Da-ming; Guo, Shu-xiang; Xu, Wu-yi; Jia, Bo; Zhao, De-peng; Liu, Bo; Gao, Bao-feng

2012-06-01

To verify the feasibility and safety of new vascular interventional robot system used in vascular interventional procedures. Vascular interventional robot type-2 (VIR-2) included master-slave parts of body propulsion system, image navigation systems and force feedback system, the catheter movement could achieve under automatic control and navigation, force feedback was integrated real-time, followed by in vitro pre-test in vascular model and cerebral angiography in dog. Surgeon controlled vascular interventional robot remotely, the catheter was inserted into the intended target, the catheter positioning error and the operation time would be evaluated. In vitro pre-test and animal experiment went well; the catheter can enter any branch of vascular. Catheter positioning error was less than 1 mm. The angiography operation in animal was carried out smoothly without complication; the success rate of the operation was 100% and the entire experiment took 26 and 30 minutes, efficiency was slightly improved compared with the VIR-1, and the time what staff exposed to the DSA machine was 0 minute. The resistance of force sensor can be displayed to the operator to provide a security guarantee for the operation. No surgical complications. VIR-2 is safe and feasible, and can achieve the catheter remote operation and angiography; the master-slave system meets the characteristics of traditional procedure. The three-dimensional image can guide the operation more smoothly; force feedback device provides remote real-time haptic information to provide security for the operation.

Visuo-Haptic Mixed Reality with Unobstructed Tool-Hand Integration.

PubMed

Cosco, Francesco; Garre, Carlos; Bruno, Fabio; Muzzupappa, Maurizio; Otaduy, Miguel A

2013-01-01

Visuo-haptic mixed reality consists of adding to a real scene the ability to see and touch virtual objects. It requires the use of see-through display technology for visually mixing real and virtual objects, and haptic devices for adding haptic interaction with the virtual objects. Unfortunately, the use of commodity haptic devices poses obstruction and misalignment issues that complicate the correct integration of a virtual tool and the user's real hand in the mixed reality scene. In this work, we propose a novel mixed reality paradigm where it is possible to touch and see virtual objects in combination with a real scene, using commodity haptic devices, and with a visually consistent integration of the user's hand and the virtual tool. We discuss the visual obstruction and misalignment issues introduced by commodity haptic devices, and then propose a solution that relies on four simple technical steps: color-based segmentation of the hand, tracking-based segmentation of the haptic device, background repainting using image-based models, and misalignment-free compositing of the user's hand. We have developed a successful proof-of-concept implementation, where a user can touch virtual objects and interact with them in the context of a real scene, and we have evaluated the impact on user performance of obstruction and misalignment correction.

On the design of a miniature haptic ring for cutaneous force feedback using shape memory alloy actuators

NASA Astrophysics Data System (ADS)

Hwang, Donghyun; Lee, Jaemin; Kim, Keehoon

2017-10-01

This paper proposes a miniature haptic ring that can display touch/pressure and shearing force to the user’s fingerpad. For practical use and wider application of the device, it is developed with the aim of achieving high wearability and mobility/portability as well as cutaneous force feedback functionality. A main body of the device is designed as a ring-shaped lightweight structure with a simple driving mechanism, and thin shape memory alloy (SMA) wires having high energy density are applied as actuating elements. Also, based on a band-type wireless control unit including a wireless data communication module, the whole device could be realized as a wearable mobile haptic device system. These features enable the device to take diverse advantages on functional performances and to provide users with significant usability. In this work, the proposed miniature haptic ring is systematically designed, and its working performances are experimentally evaluated with a fabricated functional prototype. The experimental results obviously demonstrate that the proposed device exhibits higher force-to-weight ratio than conventional finger-wearable haptic devices for cutaneous force feedback. Also, it is investigated that operational performances of the device are strongly influenced by electro-thermomechanical behaviors of the SMA actuator. In addition to the experiments for performance evaluation, we conduct a preliminary user test to assess practical feasibility and usability based on user’s qualitative feedback.

Eye Tracking of Occluded Self-Moved Targets: Role of Haptic Feedback and Hand-Target Dynamics.

PubMed

Danion, Frederic; Mathew, James; Flanagan, J Randall

2017-01-01

Previous studies on smooth pursuit eye movements have shown that humans can continue to track the position of their hand, or a target controlled by the hand, after it is occluded, thereby demonstrating that arm motor commands contribute to the prediction of target motion driving pursuit eye movements. Here, we investigated this predictive mechanism by manipulating both the complexity of the hand-target mapping and the provision of haptic feedback. Two hand-target mappings were used, either a rigid (simple) one in which hand and target motion matched perfectly or a nonrigid (complex) one in which the target behaved as a mass attached to the hand by means of a spring. Target animation was obtained by asking participants to oscillate a lightweight robotic device that provided (or not) haptic feedback consistent with the target dynamics. Results showed that as long as 7 s after target occlusion, smooth pursuit continued to be the main contributor to total eye displacement (∼60%). However, the accuracy of eye-tracking varied substantially across experimental conditions. In general, eye-tracking was less accurate under the nonrigid mapping, as reflected by higher positional and velocity errors. Interestingly, haptic feedback helped to reduce the detrimental effects of target occlusion when participants used the nonrigid mapping, but not when they used the rigid one. Overall, we conclude that the ability to maintain smooth pursuit in the absence of visual information can extend to complex hand-target mappings, but the provision of haptic feedback is critical for the maintenance of accurate eye-tracking performance.

Eye Tracking of Occluded Self-Moved Targets: Role of Haptic Feedback and Hand-Target Dynamics

PubMed Central

Mathew, James

2017-01-01

Abstract Previous studies on smooth pursuit eye movements have shown that humans can continue to track the position of their hand, or a target controlled by the hand, after it is occluded, thereby demonstrating that arm motor commands contribute to the prediction of target motion driving pursuit eye movements. Here, we investigated this predictive mechanism by manipulating both the complexity of the hand-target mapping and the provision of haptic feedback. Two hand-target mappings were used, either a rigid (simple) one in which hand and target motion matched perfectly or a nonrigid (complex) one in which the target behaved as a mass attached to the hand by means of a spring. Target animation was obtained by asking participants to oscillate a lightweight robotic device that provided (or not) haptic feedback consistent with the target dynamics. Results showed that as long as 7 s after target occlusion, smooth pursuit continued to be the main contributor to total eye displacement (∼60%). However, the accuracy of eye-tracking varied substantially across experimental conditions. In general, eye-tracking was less accurate under the nonrigid mapping, as reflected by higher positional and velocity errors. Interestingly, haptic feedback helped to reduce the detrimental effects of target occlusion when participants used the nonrigid mapping, but not when they used the rigid one. Overall, we conclude that the ability to maintain smooth pursuit in the absence of visual information can extend to complex hand-target mappings, but the provision of haptic feedback is critical for the maintenance of accurate eye-tracking performance. PMID:28680964

Lack of transfer of skills after virtual reality simulator training with haptic feedback.

PubMed

Våpenstad, Cecilie; Hofstad, Erlend Fagertun; Bø, Lars Eirik; Kuhry, Esther; Johnsen, Gjermund; Mårvik, Ronald; Langø, Thomas; Hernes, Toril Nagelhus

2017-12-01

Virtual reality (VR) simulators enrich surgical training and offer training possibilities outside of the operating room (OR). In this study, we created a criterion-based training program on a VR simulator with haptic feedback and tested it by comparing the performances of a simulator group against a control group. Medical students with no experience in laparoscopy were randomly assigned to a simulator group or a control group. In the simulator group, the candidates trained until they reached predefined criteria on the LapSim ® VR simulator (Surgical Science AB, Göteborg, Sweden) with haptic feedback (Xitact TM IHP, Mentice AB, Göteborg, Sweden). All candidates performed a cholecystectomy on a porcine organ model in a box trainer (the clinical setting). The performances were video rated by two surgeons blinded to subject training status. In total, 30 students performed the cholecystectomy and had their videos rated (N = 16 simulator group, N = 14 control group). The control group achieved better video rating scores than the simulator group (p < .05). The criterion-based training program did not transfer skills to the clinical setting. Poor mechanical performance of the simulated haptic feedback is believed to have resulted in a negative training effect.

Haptics-based immersive telerobotic system for improvised explosive device disposal: Are two hands better than one?

NASA Astrophysics Data System (ADS)

Erickson, David; Lacheray, Hervé; Lambert, Jason Michel; Mantegh, Iraj; Crymble, Derry; Daly, John; Zhao, Yan

2012-06-01

State-of-the-art robotic explosive ordnance disposal robotics have not, in general, adopted recent advances in control technology and man-machine interfaces and lag many years behind academia. This paper describes the Haptics-based Immersive Telerobotic System project investigating an immersive telepresence envrionment incorporating advanced vehicle control systems, Augmented immersive sensory feedback, dynamic 3D visual information, and haptic feedback for explosive ordnance disposal operators. The project aim is to provide operatiors a more sophisticated interface and expand sensory input to perform complex tasks to defeat improvised explosive devices successfully. The introduction of haptics and immersive teleprescence has the potential to shift the way teleprescence systems work for explosive ordnance disposal tasks or more widely for first responders scenarios involving remote unmanned ground vehicles.

[Visual cuing effect for haptic angle judgment].

PubMed

Era, Ataru; Yokosawa, Kazuhiko

2009-08-01

We investigated whether visual cues are useful for judging haptic angles. Participants explored three-dimensional angles with a virtual haptic feedback device. For visual cues, we use a location cue, which synchronizes haptic exploration, and a space cue, which specifies the haptic space. In Experiment 1, angles were judged more correctly with both cues, but were overestimated with a location cue only. In Experiment 2, the visual cues emphasized depth, and overestimation with location cues occurred, but space cues had no influence. The results showed that (a) when both cues are presented, haptic angles are judged more correctly. (b) Location cues facilitate only motion information, and not depth information. (c) Haptic angles are apt to be overestimated when there is both haptic and visual information.

Investigating Students' Ideas about Buoyancy and the Influence of Haptic Feedback

ERIC Educational Resources Information Center

Minogue, James; Borland, David

2016-01-01

While haptics (simulated touch) represents a potential breakthrough technology for science teaching and learning, there is relatively little research into its differential impact in the context of teaching and learning. This paper describes the testing of a haptically enhanced simulation (HES) for learning about buoyancy. Despite a lifetime of…

Haptic force-feedback devices for the office computer: performance and musculoskeletal loading issues.

PubMed

Dennerlein, J T; Yang, M C

2001-01-01

Pointing devices, essential input tools for the graphical user interface (GUI) of desktop computers, require precise motor control and dexterity to use. Haptic force-feedback devices provide the human operator with tactile cues, adding the sense of touch to existing visual and auditory interfaces. However, the performance enhancements, comfort, and possible musculoskeletal loading of using a force-feedback device in an office environment are unknown. Hypothesizing that the time to perform a task and the self-reported pain and discomfort of the task improve with the addition of force feedback, 26 people ranging in age from 22 to 44 years performed a point-and-click task 540 times with and without an attractive force field surrounding the desired target. The point-and-click movements were approximately 25% faster with the addition of force feedback (paired t-tests, p < 0.001). Perceived user discomfort and pain, as measured through a questionnaire, were also smaller with the addition of force feedback (p < 0.001). However, this difference decreased as additional distracting force fields were added to the task environment, simulating a more realistic work situation. These results suggest that for a given task, use of a force-feedback device improves performance, and potentially reduces musculoskeletal loading during mouse use. Actual or potential applications of this research include human-computer interface design, specifically that of the pointing device extensively used for the graphical user interface.

Haptic fMRI: combining functional neuroimaging with haptics for studying the brain's motor control representation.

PubMed

Menon, Samir; Brantner, Gerald; Aholt, Chris; Kay, Kendrick; Khatib, Oussama

2013-01-01

A challenging problem in motor control neuroimaging studies is the inability to perform complex human motor tasks given the Magnetic Resonance Imaging (MRI) scanner's disruptive magnetic fields and confined workspace. In this paper, we propose a novel experimental platform that combines Functional MRI (fMRI) neuroimaging, haptic virtual simulation environments, and an fMRI-compatible haptic device for real-time haptic interaction across the scanner workspace (above torso ∼ .65×.40×.20m(3)). We implement this Haptic fMRI platform with a novel haptic device, the Haptic fMRI Interface (HFI), and demonstrate its suitability for motor neuroimaging studies. HFI has three degrees-of-freedom (DOF), uses electromagnetic motors to enable high-fidelity haptic rendering (>350Hz), integrates radio frequency (RF) shields to prevent electromagnetic interference with fMRI (temporal SNR >100), and is kinematically designed to minimize currents induced by the MRI scanner's magnetic field during motor displacement (<2cm). HFI possesses uniform inertial and force transmission properties across the workspace, and has low friction (.05-.30N). HFI's RF noise levels, in addition, are within a 3 Tesla fMRI scanner's baseline noise variation (∼.85±.1%). Finally, HFI is haptically transparent and does not interfere with human motor tasks (tested for .4m reaches). By allowing fMRI experiments involving complex three-dimensional manipulation with haptic interaction, Haptic fMRI enables-for the first time-non-invasive neuroscience experiments involving interactive motor tasks, object manipulation, tactile perception, and visuo-motor integration.

A one degree of freedom haptic system to investigate issues in human perception with particular application to probing tissue.

PubMed

Dibble, Edward; Zivanovic, Aleksandar; Davies, Brian

2004-01-01

This paper presents the results of several early studies relating to human haptic perception sensitivity when probing a virtual object. A 1 degree of freedom (DoF) rotary haptic system, that was designed and built for this purpose, is also presented. The experiments were to assess the maximum forces applied in a minimally invasive surgery (MIS) procedure, quantify the compliance sensitivity threshold when probing virtual tissue and identify the haptic system loop rate necessary for haptic feedback to feel realistic.

Robot-assisted training of the kinesthetic sense: enhancing proprioception after stroke.

PubMed

De Santis, Dalia; Zenzeri, Jacopo; Casadio, Maura; Masia, Lorenzo; Riva, Assunta; Morasso, Pietro; Squeri, Valentina

2014-01-01

Proprioception has a crucial role in promoting or hindering motor learning. In particular, an intact position sense strongly correlates with the chances of recovery after stroke. A great majority of neurological patients present both motor dysfunctions and impairments in kinesthesia, but traditional robot and virtual reality training techniques focus either in recovering motor functions or in assessing proprioceptive deficits. An open challenge is to implement effective and reliable tests and training protocols for proprioception that go beyond the mere position sense evaluation and exploit the intrinsic bidirectionality of the kinesthetic sense, which refers to both sense of position and sense of movement. Modulated haptic interaction has a leading role in promoting sensorimotor integration, and it is a natural way to enhance volitional effort. Therefore, we designed a preliminary clinical study to test a new proprioception-based motor training technique for augmenting kinesthetic awareness via haptic feedback. The feedback was provided by a robotic manipulandum and the test involved seven chronic hemiparetic subjects over 3 weeks. The protocol included evaluation sessions that consisted of a psychometric estimate of the subject's kinesthetic sensation, and training sessions, in which the subject executed planar reaching movements in the absence of vision and under a minimally assistive haptic guidance made by sequences of graded force pulses. The bidirectional haptic interaction between the subject and the robot was optimally adapted to each participant in order to achieve a uniform task difficulty over the workspace. All the subjects consistently improved in the perceptual scores as a consequence of training. Moreover, they could minimize the level of haptic guidance in time. Results suggest that the proposed method is effective in enhancing kinesthetic acuity, but the level of impairment may affect the ability of subjects to retain their improvement in time.

Robot-Assisted Training of the Kinesthetic Sense: Enhancing Proprioception after Stroke

PubMed Central

De Santis, Dalia; Zenzeri, Jacopo; Casadio, Maura; Masia, Lorenzo; Riva, Assunta; Morasso, Pietro; Squeri, Valentina

2015-01-01

Proprioception has a crucial role in promoting or hindering motor learning. In particular, an intact position sense strongly correlates with the chances of recovery after stroke. A great majority of neurological patients present both motor dysfunctions and impairments in kinesthesia, but traditional robot and virtual reality training techniques focus either in recovering motor functions or in assessing proprioceptive deficits. An open challenge is to implement effective and reliable tests and training protocols for proprioception that go beyond the mere position sense evaluation and exploit the intrinsic bidirectionality of the kinesthetic sense, which refers to both sense of position and sense of movement. Modulated haptic interaction has a leading role in promoting sensorimotor integration, and it is a natural way to enhance volitional effort. Therefore, we designed a preliminary clinical study to test a new proprioception-based motor training technique for augmenting kinesthetic awareness via haptic feedback. The feedback was provided by a robotic manipulandum and the test involved seven chronic hemiparetic subjects over 3 weeks. The protocol included evaluation sessions that consisted of a psychometric estimate of the subject’s kinesthetic sensation, and training sessions, in which the subject executed planar reaching movements in the absence of vision and under a minimally assistive haptic guidance made by sequences of graded force pulses. The bidirectional haptic interaction between the subject and the robot was optimally adapted to each participant in order to achieve a uniform task difficulty over the workspace. All the subjects consistently improved in the perceptual scores as a consequence of training. Moreover, they could minimize the level of haptic guidance in time. Results suggest that the proposed method is effective in enhancing kinesthetic acuity, but the level of impairment may affect the ability of subjects to retain their improvement in time. PMID:25601833

Perception of synchronization errors in haptic and visual communications

NASA Astrophysics Data System (ADS)

Kameyama, Seiji; Ishibashi, Yutaka

2006-10-01

This paper deals with a system which conveys the haptic sensation experimented by a user to a remote user. In the system, the user controls a haptic interface device with another remote haptic interface device while watching video. Haptic media and video of a real object which the user is touching are transmitted to another user. By subjective assessment, we investigate the allowable range and imperceptible range of synchronization error between haptic media and video. We employ four real objects and ask each subject whether the synchronization error is perceived or not for each object in the assessment. Assessment results show that we can more easily perceive the synchronization error in the case of haptic media ahead of video than in the case of the haptic media behind the video.

Characterization of a smartphone size haptic rendering system based on thin-film AlN actuators on glass substrates

NASA Astrophysics Data System (ADS)

Bernard, F.; Casset, F.; Danel, J. S.; Chappaz, C.; Basrour, S.

2016-08-01

This paper presents for the first time the characterization of a smartphone-size haptic rendering system based on the friction modulation effect. According to previous work and finite element modeling, the homogeneous flexural modes are needed to get the haptic feedback effect. The device studied consists of a thin film AlN transducers deposited on an 110  ×  65 mm2 glass substrate. The transducer’s localization on the glass plate allows a transparent central area of 90  ×  49 mm2. Electrical and mechanical parameters of the system are extracted from measurement. From this extraction, the electrical impedance matching reduced the applied voltage to 17.5 V AC and the power consumption to 1.53 W at the resonance frequency of the vibrating system to reach the haptic rendering specification. Transient characterizations of the actuation highlight a delay under the dynamic tactile detection. The characterization of the AlN transducers used as sensors, including the noise rejection, the delay or the output charge amplitude allows detections with high accuracy of any variation due to external influences. Those specifications are the first step to a low-power-consumption feedback-looped system.

Palpation imaging using a haptic system for virtual reality applications in medicine.

PubMed

Khaled, W; Reichling, S; Bruhns, O T; Boese, H; Baumann, M; Monkman, G; Egersdoerfer, S; Klein, D; Tunayar, A; Freimuth, H; Lorenz, A; Pessavento, A; Ermert, H

2004-01-01

In the field of medical diagnosis, there is a strong need to determine mechanical properties of biological tissue, which are of histological and pathological relevance. Malignant tumors are significantly stiffer than surrounding healthy tissue. One of the established diagnosis procedures is the palpation of body organs and tissue. Palpation is used to measure swelling, detect bone fracture, find and measure pulse, or to locate changes in the pathological state of tissue and organs. Current medical practice routinely uses sophisticated diagnostic tests through magnetic resonance imaging (MRI), computed tomography (CT) and ultrasound (US) imaging. However, they cannot provide direct measure of tissue elasticity. Last year we presented the concept of the first haptic sensor actuator system to visualize and reconstruct mechanical properties of tissue using ultrasonic elastography and a haptic display with electrorheological fluids. We developed a real time strain imaging system for tumor diagnosis. It allows biopsies simultaneously to conventional ultrasound B-Mode and strain imaging investigations. We deduce the relative mechanical properties by using finite element simulations and numerical solution models solving the inverse problem. Various modifications on the haptic sensor actuator system have been investigated. This haptic system has the potential of inducing real time substantial forces, using a compact lightweight mechanism which can be applied to numerous areas including intraoperative navigation, telemedicine, teaching and telecommunication.

Physical Student-Robot Interaction with the ETHZ Haptic Paddle

ERIC Educational Resources Information Center

Gassert, R.; Metzger, J.; Leuenberger, K.; Popp, W. L.; Tucker, M. R.; Vigaru, B.; Zimmermann, R.; Lambercy, O.

2013-01-01

Haptic paddles--low-cost one-degree-of-freedom force feedback devices--have been used with great success at several universities throughout the US to teach the basic concepts of dynamic systems and physical human-robot interaction (pHRI) to students. The ETHZ haptic paddle was developed for a new pHRI course offered in the undergraduate…

A “virtually minimal” visuo-haptic training of attention in severe traumatic brain injury

PubMed Central

2013-01-01

Background Although common during the early stages of recovery from severe traumatic brain injury (TBI), attention deficits have been scarcely investigated. Encouraging evidence suggests beneficial effects of attention training in more chronic and higher functioning patients. Interactive technology may provide new opportunities for rehabilitation in inpatients who are earlier in their recovery. Methods We designed a “virtually minimal” approach using robot-rendered haptics in a virtual environment to train severely injured inpatients in the early stages of recovery to sustain attention to a visuo-motor task. 21 inpatients with severe TBI completed repetitive reaching toward targets that were both seen and felt. Patients were tested over two consecutive days, experiencing 3 conditions (no haptic feedback, a break-through force, and haptic nudge) in 12 successive, 4-minute blocks. Results The interactive visuo-haptic environments were well-tolerated and engaging. Patients typically remained attentive to the task. However, patients exhibited attention loss both before (prolonged initiation) and during (pauses during motion) a movement. Compared to no haptic feedback, patients benefited from haptic nudge cues but not break-through forces. As training progressed, patients increased the number of targets acquired and spontaneously improved from one day to the next. Conclusions Interactive visuo-haptic environments could be beneficial for attention training for severe TBI patients in the early stages of recovery and warrants further and more prolonged clinical testing. PMID:23938101

A "virtually minimal" visuo-haptic training of attention in severe traumatic brain injury.

PubMed

Dvorkin, Assaf Y; Ramaiya, Milan; Larson, Eric B; Zollman, Felise S; Hsu, Nancy; Pacini, Sonia; Shah, Amit; Patton, James L

2013-08-09

Although common during the early stages of recovery from severe traumatic brain injury (TBI), attention deficits have been scarcely investigated. Encouraging evidence suggests beneficial effects of attention training in more chronic and higher functioning patients. Interactive technology may provide new opportunities for rehabilitation in inpatients who are earlier in their recovery. We designed a "virtually minimal" approach using robot-rendered haptics in a virtual environment to train severely injured inpatients in the early stages of recovery to sustain attention to a visuo-motor task. 21 inpatients with severe TBI completed repetitive reaching toward targets that were both seen and felt. Patients were tested over two consecutive days, experiencing 3 conditions (no haptic feedback, a break-through force, and haptic nudge) in 12 successive, 4-minute blocks. The interactive visuo-haptic environments were well-tolerated and engaging. Patients typically remained attentive to the task. However, patients exhibited attention loss both before (prolonged initiation) and during (pauses during motion) a movement. Compared to no haptic feedback, patients benefited from haptic nudge cues but not break-through forces. As training progressed, patients increased the number of targets acquired and spontaneously improved from one day to the next. Interactive visuo-haptic environments could be beneficial for attention training for severe TBI patients in the early stages of recovery and warrants further and more prolonged clinical testing.

Detection of Nuclear Sources by UAV Teleoperation Using a Visuo-Haptic Augmented Reality Interface

PubMed Central

Micconi, Giorgio; Caselli, Stefano; Benassi, Giacomo; Zambelli, Nicola; Bettelli, Manuele

2017-01-01

A visuo-haptic augmented reality (VHAR) interface is presented enabling an operator to teleoperate an unmanned aerial vehicle (UAV) equipped with a custom CdZnTe-based spectroscopic gamma-ray detector in outdoor environments. The task is to localize nuclear radiation sources, whose location is unknown to the user, without the close exposure of the operator. The developed detector also enables identification of the localized nuclear sources. The aim of the VHAR interface is to increase the situation awareness of the operator. The user teleoperates the UAV using a 3DOF haptic device that provides an attractive force feedback around the location of the most intense detected radiation source. Moreover, a fixed camera on the ground observes the environment where the UAV is flying. A 3D augmented reality scene is displayed on a computer screen accessible to the operator. Multiple types of graphical overlays are shown, including sensor data acquired by the nuclear radiation detector, a virtual cursor that tracks the UAV and geographical information, such as buildings. Experiments performed in a real environment are reported using an intense nuclear source. PMID:28961198

Detection of Nuclear Sources by UAV Teleoperation Using a Visuo-Haptic Augmented Reality Interface.

PubMed

Aleotti, Jacopo; Micconi, Giorgio; Caselli, Stefano; Benassi, Giacomo; Zambelli, Nicola; Bettelli, Manuele; Zappettini, Andrea

2017-09-29

A visuo-haptic augmented reality (VHAR) interface is presented enabling an operator to teleoperate an unmanned aerial vehicle (UAV) equipped with a custom CdZnTe-based spectroscopic gamma-ray detector in outdoor environments. The task is to localize nuclear radiation sources, whose location is unknown to the user, without the close exposure of the operator. The developed detector also enables identification of the localized nuclear sources. The aim of the VHAR interface is to increase the situation awareness of the operator. The user teleoperates the UAV using a 3DOF haptic device that provides an attractive force feedback around the location of the most intense detected radiation source. Moreover, a fixed camera on the ground observes the environment where the UAV is flying. A 3D augmented reality scene is displayed on a computer screen accessible to the operator. Multiple types of graphical overlays are shown, including sensor data acquired by the nuclear radiation detector, a virtual cursor that tracks the UAV and geographical information, such as buildings. Experiments performed in a real environment are reported using an intense nuclear source.

Virtual reality in neurosurgical education: part-task ventriculostomy simulation with dynamic visual and haptic feedback.

PubMed

Lemole, G Michael; Banerjee, P Pat; Luciano, Cristian; Neckrysh, Sergey; Charbel, Fady T

2007-07-01

Mastery of the neurosurgical skill set involves many hours of supervised intraoperative training. Convergence of political, economic, and social forces has limited neurosurgical resident operative exposure. There is need to develop realistic neurosurgical simulations that reproduce the operative experience, unrestricted by time and patient safety constraints. Computer-based, virtual reality platforms offer just such a possibility. The combination of virtual reality with dynamic, three-dimensional stereoscopic visualization, and haptic feedback technologies makes realistic procedural simulation possible. Most neurosurgical procedures can be conceptualized and segmented into critical task components, which can be simulated independently or in conjunction with other modules to recreate the experience of a complex neurosurgical procedure. We use the ImmersiveTouch (ImmersiveTouch, Inc., Chicago, IL) virtual reality platform, developed at the University of Illinois at Chicago, to simulate the task of ventriculostomy catheter placement as a proof-of-concept. Computed tomographic data are used to create a virtual anatomic volume. Haptic feedback offers simulated resistance and relaxation with passage of a virtual three-dimensional ventriculostomy catheter through the brain parenchyma into the ventricle. A dynamic three-dimensional graphical interface renders changing visual perspective as the user's head moves. The simulation platform was found to have realistic visual, tactile, and handling characteristics, as assessed by neurosurgical faculty, residents, and medical students. We have developed a realistic, haptics-based virtual reality simulator for neurosurgical education. Our first module recreates a critical component of the ventriculostomy placement task. This approach to task simulation can be assembled in a modular manner to reproduce entire neurosurgical procedures.

AR Feels "Softer" than VR: Haptic Perception of Stiffness in Augmented versus Virtual Reality.

PubMed

Gaffary, Yoren; Le Gouis, Benoit; Marchal, Maud; Argelaguet, Ferran; Arnaldi, Bruno; Lecuyer, Anatole

2017-11-01

Does it feel the same when you touch an object in Augmented Reality (AR) or in Virtual Reality (VR)? In this paper we study and compare the haptic perception of stiffness of a virtual object in two situations: (1) a purely virtual environment versus (2) a real and augmented environment. We have designed an experimental setup based on a Microsoft HoloLens and a haptic force-feedback device, enabling to press a virtual piston, and compare its stiffness successively in either Augmented Reality (the virtual piston is surrounded by several real objects all located inside a cardboard box) or in Virtual Reality (the same virtual piston is displayed in a fully virtual scene composed of the same other objects). We have conducted a psychophysical experiment with 12 participants. Our results show a surprising bias in perception between the two conditions. The virtual piston is on average perceived stiffer in the VR condition compared to the AR condition. For instance, when the piston had the same stiffness in AR and VR, participants would select the VR piston as the stiffer one in 60% of cases. This suggests a psychological effect as if objects in AR would feel "softer" than in pure VR. Taken together, our results open new perspectives on perception in AR versus VR, and pave the way to future studies aiming at characterizing potential perceptual biases.

Haptic Guidance Improves the Visuo-Manual Tracking of Trajectories

PubMed Central

Bluteau, Jérémy; Coquillart, Sabine; Payan, Yohan; Gentaz, Edouard

2008-01-01

Background Learning to perform new movements is usually achieved by following visual demonstrations. Haptic guidance by a force feedback device is a recent and original technology which provides additional proprioceptive cues during visuo-motor learning tasks. The effects of two types of haptic guidances-control in position (HGP) or in force (HGF)–on visuo-manual tracking (“following”) of trajectories are still under debate. Methodology/Principals Findings Three training techniques of haptic guidance (HGP, HGF or control condition, NHG, without haptic guidance) were evaluated in two experiments. Movements produced by adults were assessed in terms of shapes (dynamic time warping) and kinematics criteria (number of velocity peaks and mean velocity) before and after the training sessions. Trajectories consisted of two Arabic and two Japanese-inspired letters in Experiment 1 and ellipses in Experiment 2. We observed that the use of HGF globally improves the fluency of the visuo-manual tracking of trajectories while no significant improvement was found for HGP or NHG. Conclusion/Significance These results show that the addition of haptic information, probably encoded in force coordinates, play a crucial role on the visuo-manual tracking of new trajectories. PMID:18335049

Augmented kinematic feedback from haptic virtual reality for dental skill acquisition.

PubMed

Suebnukarn, Siriwan; Haddawy, Peter; Rhienmora, Phattanapon; Jittimanee, Pannapa; Viratket, Piyanuch

2010-12-01

We have developed a haptic virtual reality system for dental skill training. In this study we examined several kinds of kinematic information about the movement provided by the system supplement knowledge of results (KR) in dental skill acquisition. The kinematic variables examined involved force utilization (F) and mirror view (M). This created three experimental conditions that received augmented kinematic feedback (F, M, FM) and one control condition that did not (KR-only). Thirty-two dental students were randomly assigned to four groups. Their task was to perform access opening on the upper first molar with the haptic virtual reality system. An acquisition session consisted of two days of ten trials of practice in which augmented kinematic feedback was provided for the appropriate experimental conditions after each trial. One week after, a retention test consisting of two trials without augmented feedback was completed. The results showed that the augmented kinematic feedback groups had larger mean performance scores than the KR-only group in Day 1 of the acquisition and retention sessions (ANOVA, p<0.05). The apparent differences among feedback groups were not significant in Day 2 of the acquisition session (ANOVA, p>0.05). The trends in acquisition and retention sessions suggest that the augmented kinematic feedback can enhance the performance earlier in the skill acquisition and retention sessions.

Remote ultrasound palpation for robotic interventions using absolute elastography.

PubMed

Schneider, Caitlin; Baghani, Ali; Rohling, Robert; Salcudean, Septimiu

2012-01-01

Although robotic surgery has addressed many of the challenges presented by minimally invasive surgery, haptic feedback and the lack of knowledge of tissue stiffness is an unsolved problem. This paper presents a system for finding the absolute elastic properties of tissue using a freehand ultrasound scanning technique, which utilizes the da Vinci Surgical robot and a custom 2D ultrasound transducer for intraoperative use. An external exciter creates shear waves in the tissue, and a local frequency estimation method computes the shear modulus. Results are reported for both phantom and in vivo models. This system can be extended to any 6 degree-of-freedom tracking method and any 2D transducer to provide real-time absolute elastic properties of tissue.

A haptic sensor-actor-system based on ultrasound elastography and electrorheological fluids for virtual reality applications in medicine.

PubMed

Khaled, W; Ermert, H; Bruhns, O; Boese, H; Baumann, M; Monkman, G J; Egersdoerfer, S; Meier, A; Klein, D; Freimuth, H

2003-01-01

Mechanical properties of biological tissue represent important diagnostic information and are of histological relevance (hard lesions, "nodes" in organs: tumors; calcifications in vessels: arteriosclerosis). The problem is, that such information is usually obtained by digital palpation only, which is limited with respect to sensitivity. It requires intuitive assessment and does not allow quantitative documentation. A suitable sensor is required for quantitative detection of mechanical tissue properties. On the other hand, there is also some need for a realistic mechanical display of those tissue properties. Suitable actuator arrays with high spatial resolution and real-time capabilities are required operating in a haptic sensor actuator system with different applications. The sensor system uses real time ultrasonic elastography whereas the tactile actuator is based on electrorheological fluids. Due to their small size the actuator array elements have to be manufactured by micro-mechanical production methods. In order to supply the actuator elements with individual high voltages a sophisticated switching and control concept have been designed. This haptic system has the potential of inducing real time substantial forces, using a compact lightweight mechanism which can be applied to numerous areas including intraoperative navigation, telemedicine, teaching, space and telecommunication.

Portable haptic interface with omni-directional movement and force capability.

PubMed

Avizzano, Carlo Alberto; Satler, Massimo; Ruffaldi, Emanuele

2014-01-01

We describe the design of a new mobile haptic interface that employs wheels for force rendering. The interface, consisting of an omni-directional Killough type platform, provides 2DOF force feedback with different control modalities. The system autonomously performs sensor fusion for localization and force rendering. This paper explains the relevant choices concerning the functional aspects, the control design, the mechanical and electronic solution. Experimental results for force feedback characterization are reported.

Ascending and Descending in Virtual Reality: Simple and Safe System Using Passive Haptics.

PubMed

Nagao, Ryohei; Matsumoto, Keigo; Narumi, Takuji; Tanikawa, Tomohiro; Hirose, Michitaka

2018-04-01

This paper presents a novel interactive system that provides users with virtual reality (VR) experiences, wherein users feel as if they are ascending/descending stairs through passive haptic feedback. The passive haptic stimuli are provided by small bumps under the feet of users; these stimuli are provided to represent the edges of the stairs in the virtual environment. The visual stimuli of the stairs and shoes, provided by head-mounted displays, evoke a visuo-haptic interaction that modifies a user's perception of the floor shape. Our system enables users to experience all types of stairs, such as half-turn and spiral stairs, in a VR setting. We conducted a preliminary user study and two experiments to evaluate the proposed technique. The preliminary user study investigated the effectiveness of the basic idea associated with the proposed technique for the case of a user ascending stairs. The results demonstrated that the passive haptic feedback produced by the small bumps enhanced the user's feeling of presence and sense of ascending. We subsequently performed an experiment to investigate an improved viewpoint manipulation method and the interaction of the manipulation and haptics for both the ascending and descending cases. The experimental results demonstrated that the participants had a feeling of presence and felt a steep stair gradient under the condition of haptic feedback and viewpoint manipulation based on the characteristics of actual stair walking data. However, these results also indicated that the proposed system may not be as effective in providing a sense of descending stairs without an optimization of the haptic stimuli. We then redesigned the shape of the small bumps, and evaluated the design in a second experiment. The results indicated that the best shape to present haptic stimuli is a right triangle cross section in both the ascending and descending cases. Although it is necessary to install small protrusions in the determined direction, by using this optimized shape the users feeling of presence of the stairs and the sensation of walking up and down was enhanced.

Grasping trajectories in a virtual environment adhere to Weber's law.

PubMed

Ozana, Aviad; Berman, Sigal; Ganel, Tzvi

2018-06-01

Virtual-reality and telerobotic devices simulate local motor control of virtual objects within computerized environments. Here, we explored grasping kinematics within a virtual environment and tested whether, as in normal 3D grasping, trajectories in the virtual environment are performed analytically, violating Weber's law with respect to object's size. Participants were asked to grasp a series of 2D objects using a haptic system, which projected their movements to a virtual space presented on a computer screen. The apparatus also provided object-specific haptic information upon "touching" the edges of the virtual targets. The results showed that grasping movements performed within the virtual environment did not produce the typical analytical trajectory pattern obtained during 3D grasping. Unlike as in 3D grasping, grasping trajectories in the virtual environment adhered to Weber's law, which indicates relative resolution in size processing. In addition, the trajectory patterns differed from typical trajectories obtained during 3D grasping, with longer times to complete the movement, and with maximum grip apertures appearing relatively early in the movement. The results suggest that grasping movements within a virtual environment could differ from those performed in real space, and are subjected to irrelevant effects of perceptual information. Such atypical pattern of visuomotor control may be mediated by the lack of complete transparency between the interface and the virtual environment in terms of the provided visual and haptic feedback. Possible implications of the findings to movement control within robotic and virtual environments are further discussed.

The impact of haptic feedback on students' conceptions of the cell

NASA Astrophysics Data System (ADS)

Minogue, James

2005-07-01

The purpose of this study was to investigate the efficacy of adding haptic (sense of touch) feedback to computer generated visualizations for use in middle school science instruction. Current technology allows for the simulation of tactile and kinesthetic sensations via haptic devices and a computer interface. This study, conducted with middle school students (n = 80), explored the cognitive and affective impacts of this innovative technology on students' conceptions of the cell and the process of passive transport. A pretest-posttest control group design was used and participants were randomly assigned to one of two treatment groups (n = 40 for each). Both groups experienced the same core computer-mediated instructional program. This Cell Exploration program engaged students in a 3-D immersive environment that allowed them to actively investigate the form and function of a typical animal cell including its major organelles. The program also engaged students in a study of the structure and function of the cell membrane as it pertains to the process of passive transport and the mechanisms behind the membrane's selective permeability. As they conducted their investigations, students in the experimental group received bi-modal visual and haptic (simulated tactile and kinesthetic) feedback whereas the control group students experienced the program with only visual stimuli. A battery of assessments, including objective and open-ended written response items as well as a haptic performance assessment, were used to gather quantitative and qualitative data regarding changes in students' understandings of the cell concepts prior to and following their completion of the instructional program. Additionally, the impact of haptics on the affective domain of students' learning was assessed using a post-experience semi-structured interview and an attitudinal survey. Results showed that students from both conditions (Visual-Only and Visual + Haptic) found the instructional program interesting and engaging. Additionally, the vast majority of the students reported that they learned a lot about and were more interested in the topic due to their participation. Moreover, students who received the bi-modal (Visual + Haptic) feedback indicated that they experienced lower levels of frustration and spatial disorientation as they conducted their investigations when compared to individuals that relied solely on vision. There were no significant differences measured across the treatment groups on the cognitive assessment items. Despite this finding, the study provided valuable insight into the theoretical and practical considerations involved in the development of multimodal instructional programs.

Stereoscopic visualization and haptic technology used to create a virtual environment for remote surgery - biomed 2011.

PubMed

Bornhoft, J M; Strabala, K W; Wortman, T D; Lehman, A C; Oleynikov, D; Farritor, S M

2011-01-01

The objective of this research is to study the effectiveness of using a stereoscopic visualization system for performing remote surgery. The use of stereoscopic vision has become common with the advent of the da Vinci® system (Intuitive, Sunnyvale CA). This system creates a virtual environment that consists of a 3-D display for visual feedback and haptic tactile feedback, together providing an intuitive environment for remote surgical applications. This study will use simple in vivo robotic surgical devices and compare the performance of surgeons using the stereoscopic interfacing system to the performance of surgeons using one dimensional monitors. The stereoscopic viewing system consists of two cameras, two monitors, and four mirrors. The cameras are mounted to a multi-functional miniature in vivo robot; and mimic the depth perception of the actual human eyes. This is done by placing the cameras at a calculated angle and distance apart. Live video streams from the left and right cameras are displayed on the left and right monitors, respectively. A system of angled mirrors allows the left and right eyes to see the video stream from the left and right monitor, respectively, creating the illusion of depth. The haptic interface consists of two PHANTOM Omni® (SensAble, Woburn Ma) controllers. These controllers measure the position and orientation of a pen-like end effector with three degrees of freedom. As the surgeon uses this interface, they see a 3-D image and feel force feedback for collision and workspace limits. The stereoscopic viewing system has been used in several surgical training tests and shows a potential improvement in depth perception and 3-D vision. The haptic system accurately gives force feedback that aids in surgery. Both have been used in non-survival animal surgeries, and have successfully been used in suturing and gallbladder removal. Bench top experiments using the interfacing system have also been conducted. A group of participants completed two different surgical training tasks using both a two dimensional visual system and the stereoscopic visual system. Results suggest that the stereoscopic visual system decreased the amount of time taken to complete the tasks. All participants also reported that the stereoscopic system was easier to utilize than the two dimensional system. Haptic controllers combined with stereoscopic vision provides for a more intuitive virtual environment. This system provides the surgeon with 3-D vision, depth perception, and the ability to receive feedback through forces applied in the haptic controller while performing surgery. These capabilities potentially enable the performance of more complex surgeries with a higher level of precision.

Validation of the updated ArthroS simulator: face and construct validity of a passive haptic virtual reality simulator with novel performance metrics.

PubMed

Garfjeld Roberts, Patrick; Guyver, Paul; Baldwin, Mathew; Akhtar, Kash; Alvand, Abtin; Price, Andrew J; Rees, Jonathan L

2017-02-01

To assess the construct and face validity of ArthroS, a passive haptic VR simulator. A secondary aim was to evaluate the novel performance metrics produced by this simulator. Two groups of 30 participants, each divided into novice, intermediate or expert based on arthroscopic experience, completed three separate tasks on either the knee or shoulder module of the simulator. Performance was recorded using 12 automatically generated performance metrics and video footage of the arthroscopic procedures. The videos were blindly assessed using a validated global rating scale (GRS). Participants completed a survey about the simulator's realism and training utility. This new simulator demonstrated construct validity of its tasks when evaluated against a GRS (p ≤ 0.003 in all cases). Regarding it's automatically generated performance metrics, established outputs such as time taken (p ≤ 0.001) and instrument path length (p ≤ 0.007) also demonstrated good construct validity. However, two-thirds of the proposed 'novel metrics' the simulator reports could not distinguish participants based on arthroscopic experience. Face validity assessment rated the simulator as a realistic and useful tool for trainees, but the passive haptic feedback (a key feature of this simulator) is rated as less realistic. The ArthroS simulator has good task construct validity based on established objective outputs, but some of the novel performance metrics could not distinguish between surgical experience. The passive haptic feedback of the simulator also needs improvement. If simulators could offer automated and validated performance feedback, this would facilitate improvements in the delivery of training by allowing trainees to practise and self-assess.

Human-Centered Design and Evaluation of Haptic Cueing for Teleoperation of Multiple Mobile Robots.

PubMed

Son, Hyoung Il; Franchi, Antonio; Chuang, Lewis L; Kim, Junsuk; Bulthoff, Heinrich H; Giordano, Paolo Robuffo

2013-04-01

In this paper, we investigate the effect of haptic cueing on a human operator's performance in the field of bilateral teleoperation of multiple mobile robots, particularly multiple unmanned aerial vehicles (UAVs). Two aspects of human performance are deemed important in this area, namely, the maneuverability of mobile robots and the perceptual sensitivity of the remote environment. We introduce metrics that allow us to address these aspects in two psychophysical studies, which are reported here. Three fundamental haptic cue types were evaluated. The Force cue conveys information on the proximity of the commanded trajectory to obstacles in the remote environment. The Velocity cue represents the mismatch between the commanded and actual velocities of the UAVs and can implicitly provide a rich amount of information regarding the actual behavior of the UAVs. Finally, the Velocity+Force cue is a linear combination of the two. Our experimental results show that, while maneuverability is best supported by the Force cue feedback, perceptual sensitivity is best served by the Velocity cue feedback. In addition, we show that large gains in the haptic feedbacks do not always guarantee an enhancement in the teleoperator's performance.

Virtual reality training and assessment in laparoscopic rectum surgery.

PubMed

Pan, Jun J; Chang, Jian; Yang, Xiaosong; Liang, Hui; Zhang, Jian J; Qureshi, Tahseen; Howell, Robert; Hickish, Tamas

2015-06-01

Virtual-reality (VR) based simulation techniques offer an efficient and low cost alternative to conventional surgery training. This article describes a VR training and assessment system in laparoscopic rectum surgery. To give a realistic visual performance of interaction between membrane tissue and surgery tools, a generalized cylinder based collision detection and a multi-layer mass-spring model are presented. A dynamic assessment model is also designed for hierarchy training evaluation. With this simulator, trainees can operate on the virtual rectum with both visual and haptic sensation feedback simultaneously. The system also offers surgeons instructions in real time when improper manipulation happens. The simulator has been tested and evaluated by ten subjects. This prototype system has been verified by colorectal surgeons through a pilot study. They believe the visual performance and the tactile feedback are realistic. It exhibits the potential to effectively improve the surgical skills of trainee surgeons and significantly shorten their learning curve. Copyright © 2014 John Wiley & Sons, Ltd.

Surgical scissors extension adds the 7th axis of force feedback to the Freedom 6S.

PubMed

Powers, Marilyn J; Sinclair, Ian P W; Brouwer, Iman; Laroche, Denis

2007-01-01

A virtual reality surgical simulator ideally allows seamless transition between the real and virtual world. In that respect, all of a surgeon's motions and tools must be simulated. Until now researchers have been limited to using a pen-like tool in six degrees-of-freedom. This paper presents the addition of haptically enabled scissors to the end effector of a 6-DOF haptic device, the Freedom 6S. The scissors are capable of pinching a maximum torque of 460 mN.m with low inertia and low back-drive friction. The device is a balanced design so that the user feels like they are holding no more than actual scissors, although with some added inertia on the load end. The system is interchangeable between the 6-DOF and 7-DOF configurations to allow switching tools quickly.

Haptic feedback for virtual assembly

NASA Astrophysics Data System (ADS)

Luecke, Greg R.; Zafer, Naci

1998-12-01

Assembly operations require high speed and precision with low cost. The manufacturing industry has recently turned attenuation to the possibility of investigating assembly procedures using graphical display of CAD parts. For these tasks, some sort of feedback to the person is invaluable in providing a real sense of interaction with virtual parts. This research develops the use of a commercial assembly robot as the haptic display in such tasks. For demonstration, a peg-hole insertion task is studied. Kane's Method is employed to derive the dynamics of the peg and the contact motions between the peg and the hole. A handle modeled as a cylindrical peg is attached to the end effector of a PUMA 560 robotic arm. The arm is handle modeled as a cylindrical peg is attached to the end effector of a PUMA 560 robotic arm. The arm is equipped with a six axis force/torque transducer. The use grabs the handle and the user-applied forces are recorded. A 300 MHz Pentium computer is used to simulate the dynamics of the virtual peg and its interactions as it is inserted in the virtual hole. The computed torque control is then employed to exert the full dynamics of the task to the user hand. Visual feedback is also incorporated to help the user in the process of inserting the peg into the hole. Experimental results are presented to show several contact configurations for this virtually simulated task.

Visual-perceptual mismatch in robotic surgery.

PubMed

Abiri, Ahmad; Tao, Anna; LaRocca, Meg; Guan, Xingmin; Askari, Syed J; Bisley, James W; Dutson, Erik P; Grundfest, Warren S

2017-08-01

The principal objective of the experiment was to analyze the effects of the clutch operation of robotic surgical systems on the performance of the operator. The relative coordinate system introduced by the clutch operation can introduce a visual-perceptual mismatch which can potentially have negative impact on a surgeon's performance. We also assess the impact of the introduction of additional tactile sensory information on reducing the impact of visual-perceptual mismatch on the performance of the operator. We asked 45 novice subjects to complete peg transfers using the da Vinci IS 1200 system with grasper-mounted, normal force sensors. The task involves picking up a peg with one of the robotic arms, passing it to the other arm, and then placing it on the opposite side of the view. Subjects were divided into three groups: aligned group (no mismatch), the misaligned group (10 cm z axis mismatch), and the haptics-misaligned group (haptic feedback and z axis mismatch). Each subject performed the task five times, during which the grip force, time of completion, and number of faults were recorded. Compared to the subjects that performed the tasks using a properly aligned controller/arm configuration, subjects with a single-axis misalignment showed significantly more peg drops (p = 0.011) and longer time to completion (p < 0.001). Additionally, it was observed that addition of tactile feedback helps reduce the negative effects of visual-perceptual mismatch in some cases. Grip force data recorded from grasper-mounted sensors showed no difference between the different groups. The visual-perceptual mismatch created by the misalignment of the robotic controls relative to the robotic arms has a negative impact on the operator of a robotic surgical system. Introduction of other sensory information and haptic feedback systems can help in potentially reducing this effect.

Haptograph Representation of Real-World Haptic Information by Wideband Force Control

NASA Astrophysics Data System (ADS)

Katsura, Seiichiro; Irie, Kouhei; Ohishi, Kiyoshi

Artificial acquisition and reproduction of human sensations are basic technologies of communication engineering. For example, auditory information is obtained by a microphone, and a speaker reproduces it by artificial means. Furthermore, a video camera and a television make it possible to transmit visual sensation by broadcasting. On the contrary, since tactile or haptic information is subject to the Newton's “law of action and reaction” in the real world, a device which acquires, transmits, and reproduces the information has not been established. From the point of view, real-world haptics is the key technology for future haptic communication engineering. This paper proposes a novel acquisition method of haptic information named “haptograph”. The haptograph visualizes the haptic information like photograph. The proposed haptograph is applied to haptic recognition of the contact environment. A linear motor contacts to the surface of the environment and its reaction force is used to make a haptograph. A robust contact motion and sensor-less sensing of the reaction force are attained by using a disturbance observer. As a result, an encyclopedia of contact environment is attained. Since temporal and spatial analyses are conducted to represent haptic information as the haptograph, it is possible to be recognized and to be evaluated intuitively.

Comparative study on collaborative interaction in non-immersive and immersive systems

NASA Astrophysics Data System (ADS)

Shahab, Qonita M.; Kwon, Yong-Moo; Ko, Heedong; Mayangsari, Maria N.; Yamasaki, Shoko; Nishino, Hiroaki

2007-09-01

This research studies the Virtual Reality simulation for collaborative interaction so that different people from different places can interact with one object concurrently. Our focus is the real-time handling of inputs from multiple users, where object's behavior is determined by the combination of the multiple inputs. Issues addressed in this research are: 1) The effects of using haptics on a collaborative interaction, 2) The possibilities of collaboration between users from different environments. We conducted user tests on our system in several cases: 1) Comparison between non-haptics and haptics collaborative interaction over LAN, 2) Comparison between non-haptics and haptics collaborative interaction over Internet, and 3) Analysis of collaborative interaction between non-immersive and immersive display environments. The case studies are the interaction of users in two cases: collaborative authoring of a 3D model by two users, and collaborative haptic interaction by multiple users. In Virtual Dollhouse, users can observe physics law while constructing a dollhouse using existing building blocks, under gravity effects. In Virtual Stretcher, multiple users can collaborate on moving a stretcher together while feeling each other's haptic motions.

Haptic feedback improves surgeons' user experience and fracture reduction in facial trauma simulation.

PubMed

Girod, Sabine; Schvartzman, Sara C; Gaudilliere, Dyani; Salisbury, Kenneth; Silva, Rebeka

2016-01-01

Computer-assisted surgical (CAS) planning tools are available for craniofacial surgery, but are usually based on computer-aided design (CAD) tools that lack the ability to detect the collision of virtual objects (i.e., fractured bone segments). We developed a CAS system featuring a sense of touch (haptic) that enables surgeons to physically interact with individual, patient-specific anatomy and immerse in a three-dimensional virtual environment. In this study, we evaluated initial user experience with our novel system compared to an existing CAD system. Ten surgery resident trainees received a brief verbal introduction to both the haptic and CAD systems. Users simulated mandibular fracture reduction in three clinical cases within a 15 min time limit for each system and completed a questionnaire to assess their subjective experience. We compared standard landmarks and linear and angular measurements between the simulated results and the actual surgical outcome and found that haptic simulation results were not significantly different from actual postoperative outcomes. In contrast, CAD results significantly differed from both the haptic simulation and actual postoperative results. In addition to enabling a more accurate fracture repair, the haptic system provided a better user experience than the CAD system in terms of intuitiveness and self-reported quality of repair.

Learning in a Virtual Environment Using Haptic Systems for Movement Re-Education: Can This Medium Be Used for Remodeling Other Behaviors and Actions?

PubMed Central

Merians, Alma S; Fluet, Gerard G; Qiu, Qinyin; Lafond, Ian; Adamovich, Sergei V

2011-01-01

Robotic systems that are interfaced with virtual reality gaming and task simulations are increasingly being developed to provide repetitive intensive practice to promote increased compliance and facilitate better outcomes in rehabilitation post-stroke. A major development in the use of virtual environments (VEs) has been to incorporate tactile information and interaction forces into what was previously an essentially visual experience. Robots of varying complexity are being interfaced with more traditional virtual presentations to provide haptic feedback that enriches the sensory experience and adds physical task parameters. This provides forces that produce biomechanical and neuromuscular interactions with the VE that approximate real-world movement more accurately than visual-only VEs, simulating the weight and force found in upper extremity tasks. The purpose of this article is to present an overview of several systems that are commercially available for ambulation training and for training movement of the upper extremity. We will also report on the system that we have developed (NJIT-RAVR system) that incorporates motivating and challenging haptic feedback effects into VE simulations to facilitate motor recovery of the upper extremity post-stroke. The NJIT-RAVR system trains both the upper arm and the hand. The robotic arm acts as an interface between the participants and the VEs, enabling multiplanar movements against gravity in a three-dimensional workspace. The ultimate question is whether this medium can provide a motivating, challenging, gaming experience with dramatically decreased physical difficulty levels, which would allow for participation by an obese person and facilitate greater adherence to exercise regimes. PMID:21527097

Haptic exploration of fingertip-sized geometric features using a multimodal tactile sensor

NASA Astrophysics Data System (ADS)

Ponce Wong, Ruben D.; Hellman, Randall B.; Santos, Veronica J.

2014-06-01

Haptic perception remains a grand challenge for artificial hands. Dexterous manipulators could be enhanced by "haptic intelligence" that enables identification of objects and their features via touch alone. Haptic perception of local shape would be useful when vision is obstructed or when proprioceptive feedback is inadequate, as observed in this study. In this work, a robot hand outfitted with a deformable, bladder-type, multimodal tactile sensor was used to replay four human-inspired haptic "exploratory procedures" on fingertip-sized geometric features. The geometric features varied by type (bump, pit), curvature (planar, conical, spherical), and footprint dimension (1.25 - 20 mm). Tactile signals generated by active fingertip motions were used to extract key parameters for use as inputs to supervised learning models. A support vector classifier estimated order of curvature while support vector regression models estimated footprint dimension once curvature had been estimated. A distal-proximal stroke (along the long axis of the finger) enabled estimation of order of curvature with an accuracy of 97%. Best-performing, curvature-specific, support vector regression models yielded R2 values of at least 0.95. While a radial-ulnar stroke (along the short axis of the finger) was most helpful for estimating feature type and size for planar features, a rolling motion was most helpful for conical and spherical features. The ability to haptically perceive local shape could be used to advance robot autonomy and provide haptic feedback to human teleoperators of devices ranging from bomb defusal robots to neuroprostheses.

Development of a novel haptic glove for improving finger dexterity in poststroke rehabilitation.

PubMed

Lin, Chi-Ying; Tsai, Chia-Min; Shih, Pei-Cheng; Wu, Hsiao-Ching

2015-01-01

Almost all stroke patients experience a certain degree of fine motor impairment, and impeded finger movement may limit activities in daily life. Thus, to improve the quality of life of stroke patients, designing an efficient training device for fine motor rehabilitation is crucial. This study aimed to develop a novel fine motor training glove that integrates a virtual-reality based interactive environment with vibrotactile feedback for more effective post stroke hand rehabilitation. The proposed haptic rehabilitation device is equipped with small DC vibration motors for vibrotactile feedback stimulation and piezoresistive thin-film force sensors for motor function evaluation. Two virtual-reality based games ``gopher hitting'' and ``musical note hitting'' were developed as a haptic interface. According to the designed rehabilitation program, patients intuitively push and practice their fingers to improve the finger isolation function. Preliminary tests were conducted to assess the feasibility of the developed haptic rehabilitation system and to identify design concerns regarding the practical use in future clinical testing.

A Review of Simulators with Haptic Devices for Medical Training.

PubMed

Escobar-Castillejos, David; Noguez, Julieta; Neri, Luis; Magana, Alejandra; Benes, Bedrich

2016-04-01

Medical procedures often involve the use of the tactile sense to manipulate organs or tissues by using special tools. Doctors require extensive preparation in order to perform them successfully; for example, research shows that a minimum of 750 operations are needed to acquire sufficient experience to perform medical procedures correctly. Haptic devices have become an important training alternative and they have been considered to improve medical training because they let users interact with virtual environments by adding the sense of touch to the simulation. Previous articles in the field state that haptic devices enhance the learning of surgeons compared to current training environments used in medical schools (corpses, animals, or synthetic skin and organs). Consequently, virtual environments use haptic devices to improve realism. The goal of this paper is to provide a state of the art review of recent medical simulators that use haptic devices. In particular we focus on stitching, palpation, dental procedures, endoscopy, laparoscopy, and orthopaedics. These simulators are reviewed and compared from the viewpoint of used technology, the number of degrees of freedom, degrees of force feedback, perceived realism, immersion, and feedback provided to the user. In the conclusion, several observations per area and suggestions for future work are provided.

Soft Somatosensitive Actuators via Embedded 3D Printing.

PubMed

Truby, Ryan L; Wehner, Michael; Grosskopf, Abigail K; Vogt, Daniel M; Uzel, Sebastien G M; Wood, Robert J; Lewis, Jennifer A

2018-04-01

Humans possess manual dexterity, motor skills, and other physical abilities that rely on feedback provided by the somatosensory system. Herein, a method is reported for creating soft somatosensitive actuators (SSAs) via embedded 3D printing, which are innervated with multiple conductive features that simultaneously enable haptic, proprioceptive, and thermoceptive sensing. This novel manufacturing approach enables the seamless integration of multiple ionically conductive and fluidic features within elastomeric matrices to produce SSAs with the desired bioinspired sensing and actuation capabilities. Each printed sensor is composed of an ionically conductive gel that exhibits both long-term stability and hysteresis-free performance. As an exemplar, multiple SSAs are combined into a soft robotic gripper that provides proprioceptive and haptic feedback via embedded curvature, inflation, and contact sensors, including deep and fine touch contact sensors. The multimaterial manufacturing platform enables complex sensing motifs to be easily integrated into soft actuating systems, which is a necessary step toward closed-loop feedback control of soft robots, machines, and haptic devices. © 2018 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

Cutaneous Feedback of Fingertip Deformation and Vibration for Palpation in Robotic Surgery.

PubMed

Pacchierotti, Claudio; Prattichizzo, Domenico; Kuchenbecker, Katherine J

2016-02-01

Despite its expected clinical benefits, current teleoperated surgical robots do not provide the surgeon with haptic feedback largely because grounded forces can destabilize the system's closed-loop controller. This paper presents an alternative approach that enables the surgeon to feel fingertip contact deformations and vibrations while guaranteeing the teleoperator's stability. We implemented our cutaneous feedback solution on an Intuitive Surgical da Vinci Standard robot by mounting a SynTouch BioTac tactile sensor to the distal end of a surgical instrument and a custom cutaneous display to the corresponding master controller. As the user probes the remote environment, the contact deformations, dc pressure, and ac pressure (vibrations) sensed by the BioTac are directly mapped to input commands for the cutaneous device's motors using a model-free algorithm based on look-up tables. The cutaneous display continually moves, tilts, and vibrates a flat plate at the operator's fingertip to optimally reproduce the tactile sensations experienced by the BioTac. We tested the proposed approach by having eighteen subjects use the augmented da Vinci robot to palpate a heart model with no haptic feedback, only deformation feedback, and deformation plus vibration feedback. Fingertip deformation feedback significantly improved palpation performance by reducing the task completion time, the pressure exerted on the heart model, and the subject's absolute error in detecting the orientation of the embedded plastic stick. Vibration feedback significantly improved palpation performance only for the seven subjects who dragged the BioTac across the model, rather than pressing straight into it.

Human eye haptics-based multimedia.

PubMed

Velandia, David; Uribe-Quevedo, Alvaro; Perez-Gutierrez, Byron

2014-01-01

Immersive and interactive multimedia applications offer complementary study tools in anatomy as users can explore 3D models while obtaining information about the organ, tissue or part being explored. Haptics increases the sense of interaction with virtual objects improving user experience in a more realistic manner. Common eye studying tools are books, illustrations, assembly models, and more recently these are being complemented with mobile apps whose 3D capabilities, computing power and customers are increasing. The goal of this project is to develop a complementary eye anatomy and pathology study tool using deformable models within a multimedia application, offering the students the opportunity for exploring the eye from up close and within with relevant information. Validation of the tool provided feedback on the potential of the development, along with suggestions on improving haptic feedback and navigation.

Haptic-Based Perception-Empathy Biofeedback Enhances Postural Motor Learning During High-Cognitive Load Task in Healthy Older Adults

PubMed Central

Yasuda, Kazuhiro; Saichi, Kenta; Iwata, Hiroyasu

2018-01-01

Falls and fall-induced injuries are major global public health problems, and sensory input impairment in older adults results in significant limitations in feedback-type postural control. A haptic-based biofeedback (BF) system can be used for augmenting somatosensory input in older adults, and the application of this BF system can increase the objectivity of the feedback and encourage comparison with that provided by a trainer. Nevertheless, an optimal BF system that focuses on interpersonal feedback for balance training in older adults has not been proposed. Thus, we proposed a haptic-based perception-empathy BF system that provides information regarding the older adult's center-of-foot pressure pattern to the trainee and trainer for refining the motor learning effect. The first objective of this study was to examine the effect of this balance training regimen in healthy older adults performing a postural learning task. Second, this study aimed to determine whether BF training required high cognitive load to clarify its practicability in real-life settings. Twenty older adults were assigned to two groups: BF and control groups. Participants in both groups tried balance training in the single-leg stance while performing a cognitive task (i.e., serial subtraction task). Retention was tested 24 h later. Testing comprised balance performance measures (i.e., 95% confidence ellipse area and mean velocity of sway) and dual-task performance (number of responses and correct answers). Measurements of postural control using a force plate revealed that the stability of the single-leg stance was significantly lower in the BF group than in the control group during the balance task. The BF group retained the improvement in the 95% confidence ellipse area 24 h after the retention test. Results of dual-task performance during the balance task were not different between the two groups. These results confirmed the potential benefit of the proposed balance training regimen in designing successful motor learning programs for preventing falls in older adults. PMID:29868597

Haptic-Based Perception-Empathy Biofeedback Enhances Postural Motor Learning During High-Cognitive Load Task in Healthy Older Adults.

PubMed

Yasuda, Kazuhiro; Saichi, Kenta; Iwata, Hiroyasu

2018-01-01

Falls and fall-induced injuries are major global public health problems, and sensory input impairment in older adults results in significant limitations in feedback-type postural control. A haptic-based biofeedback (BF) system can be used for augmenting somatosensory input in older adults, and the application of this BF system can increase the objectivity of the feedback and encourage comparison with that provided by a trainer. Nevertheless, an optimal BF system that focuses on interpersonal feedback for balance training in older adults has not been proposed. Thus, we proposed a haptic-based perception-empathy BF system that provides information regarding the older adult's center-of-foot pressure pattern to the trainee and trainer for refining the motor learning effect. The first objective of this study was to examine the effect of this balance training regimen in healthy older adults performing a postural learning task. Second, this study aimed to determine whether BF training required high cognitive load to clarify its practicability in real-life settings. Twenty older adults were assigned to two groups: BF and control groups. Participants in both groups tried balance training in the single-leg stance while performing a cognitive task (i.e., serial subtraction task). Retention was tested 24 h later. Testing comprised balance performance measures (i.e., 95% confidence ellipse area and mean velocity of sway) and dual-task performance (number of responses and correct answers). Measurements of postural control using a force plate revealed that the stability of the single-leg stance was significantly lower in the BF group than in the control group during the balance task. The BF group retained the improvement in the 95% confidence ellipse area 24 h after the retention test. Results of dual-task performance during the balance task were not different between the two groups. These results confirmed the potential benefit of the proposed balance training regimen in designing successful motor learning programs for preventing falls in older adults.

The Use of Haptic Display Technology in Education

ERIC Educational Resources Information Center

Barfield, Woodrow

2009-01-01

The experience of "virtual reality" can consist of head-tracked and stereoscopic virtual worlds, spatialized sound, haptic feedback, and to a lesser extent olfactory cues. Although virtual reality systems have been proposed for numerous applications, the field of education is one particular application that seems well-suited for virtual…

Do Haptic Representations Help Complex Molecular Learning?

ERIC Educational Resources Information Center

Bivall, Petter; Ainsworth, Shaaron; Tibell, Lena A. E.

2011-01-01

This study explored whether adding a haptic interface (that provides users with somatosensory information about virtual objects by force and tactile feedback) to a three-dimensional (3D) chemical model enhanced students' understanding of complex molecular interactions. Two modes of the model were compared in a between-groups pre- and posttest…

Generalized interactions using virtual tools within the spring framework: probing, piercing, cauterizing and ablating

NASA Technical Reports Server (NTRS)

Montgomery, Kevin; Bruyns, Cynthia D.

2002-01-01

We present schemes for real-time generalized interactions such as probing, piercing, cauterizing and ablating virtual tissues. These methods have been implemented in a robust, real-time (haptic rate) surgical simulation environment allowing us to model procedures including animal dissection, microsurgery, hysteroscopy, and cleft lip repair.

Multilateral haptics-based immersive teleoperation for improvised explosive device disposal

NASA Astrophysics Data System (ADS)

Erickson, David; Lacheray, Hervé; Daly, John

2013-05-01

Of great interest to police and military organizations is the development of effective improvised explosive device (IED) disposal (IEDD) technology to aid in activities such as mine field clearing, and bomb disposal. At the same time minimizing risk to personnel. This paper presents new results in the research and development of a next generation mobile immersive teleoperated explosive ordnance disposal system. This system incorporates elements of 3D vision, multilateral teleoperation for high transparency haptic feedback, immersive augmented reality operator control interfaces, and a realistic hardware-in-the-loop (HIL) 3D simulation environment incorporating vehicle and manipulator dynamics for both operator training and algorithm development. In the past year, new algorithms have been developed to facilitate incorporating commercial off-the-shelf (COTS) robotic hardware into the teleoperation system. In particular, a real-time numerical inverse position kinematics algorithm that can be applied to a wide range of manipulators has been implemented, an inertial measurement unit (IMU) attitude stabilization system for manipulators has been developed and experimentally validated, and a voice­operated manipulator control system has been developed and integrated into the operator control station. The integration of these components into a vehicle simulation environment with half-car vehicle dynamics has also been successfully carried out. A physical half-car plant is currently being constructed for HIL integration with the simulation environment.

Haptic/graphic rehabilitation: integrating a robot into a virtual environment library and applying it to stroke therapy.

PubMed

Sharp, Ian; Patton, James; Listenberger, Molly; Case, Emily

2011-08-08

Recent research that tests interactive devices for prolonged therapy practice has revealed new prospects for robotics combined with graphical and other forms of biofeedback. Previous human-robot interactive systems have required different software commands to be implemented for each robot leading to unnecessary developmental overhead time each time a new system becomes available. For example, when a haptic/graphic virtual reality environment has been coded for one specific robot to provide haptic feedback, that specific robot would not be able to be traded for another robot without recoding the program. However, recent efforts in the open source community have proposed a wrapper class approach that can elicit nearly identical responses regardless of the robot used. The result can lead researchers across the globe to perform similar experiments using shared code. Therefore modular "switching out"of one robot for another would not affect development time. In this paper, we outline the successful creation and implementation of a wrapper class for one robot into the open-source H3DAPI, which integrates the software commands most commonly used by all robots.

A predictive bone drilling force model for haptic rendering with experimental validation using fresh cadaveric bone.

PubMed

Lin, Yanping; Chen, Huajiang; Yu, Dedong; Zhang, Ying; Yuan, Wen

2017-01-01

Bone drilling simulators with virtual and haptic feedback provide a safe, cost-effective and repeatable alternative to traditional surgical training methods. To develop such a simulator, accurate haptic rendering based on a force model is required to feedback bone drilling forces based on user input. Current predictive bone drilling force models based on bovine bones with various drilling conditions and parameters are not representative of the bone drilling process in bone surgery. The objective of this study was to provide a bone drilling force model for haptic rendering based on calibration and validation experiments in fresh cadaveric bones with different bone densities. Using a commonly used drill bit geometry (2 mm diameter), feed rates (20-60 mm/min) and spindle speeds (4000-6000 rpm) in orthognathic surgeries, the bone drilling forces of specimens from two groups were measured and the calibration coefficients of the specific normal and frictional pressures were determined. The comparison of the predicted forces and the measured forces from validation experiments with a large range of feed rates and spindle speeds demonstrates that the proposed bone drilling forces can predict the trends and average forces well. The presented bone drilling force model can be used for haptic rendering in surgical simulators.

Discriminating Tissue Stiffness with a Haptic Catheter: Feeling the Inside of the Beating Heart.

PubMed

Kesner, Samuel B; Howe, Robert D

2011-01-01

Catheter devices allow physicians to access the inside of the human body easily and painlessly through natural orifices and vessels. Although catheters allow for the delivery of fluids and drugs, the deployment of devices, and the acquisition of the measurements, they do not allow clinicians to assess the physical properties of tissue inside the body due to the tissue motion and transmission limitations of the catheter devices, including compliance, friction, and backlash. The goal of this research is to increase the tactile information available to physicians during catheter procedures by providing haptic feedback during palpation procedures. To accomplish this goal, we have developed the first motion compensated actuated catheter system that enables haptic perception of fast moving tissue structures. The actuated catheter is instrumented with a distal tip force sensor and a force feedback interface that allows users to adjust the position of the catheter while experiencing the forces on the catheter tip. The efficacy of this device and interface is evaluated through a psychophyisical study comparing how accurately users can differentiate various materials attached to a cardiac motion simulator using the haptic device and a conventional manual catheter. The results demonstrate that haptics improves a user's ability to differentiate material properties and decreases the total number of errors by 50% over the manual catheter system.

Haptic interface of the KAIST-Ewha colonoscopy simulator II.

PubMed

Woo, Hyun Soo; Kim, Woo Seok; Ahn, Woojin; Lee, Doo Yong; Yi, Sun Young

2008-11-01

This paper presents an improved haptic interface for the Korea Advanced Institute of Science and Technology Ewha Colonoscopy Simulator II. The haptic interface enables the distal portion of the colonoscope to be freely bent while guaranteeing sufficient workspace and reflective forces for colonoscopy simulation. Its force-torque sensor measures the profiles of the user. Manipulation of the colonoscope tip is monitored by four deflection sensors and triggers computations to render accurate graphic images corresponding to the rotation of the angle knob. Tack sensors are attached to the valve-actuation buttons of the colonoscope to simulate air injection or suction as well as the corresponding deformation of the colon. A survey study for face validation was conducted, and the result shows that the developed haptic interface provides realistic haptic feedback for colonoscopy simulations.

Factors Influencing Undergraduate Students' Acceptance of a Haptic Interface for Learning Gross Anatomy

ERIC Educational Resources Information Center

Yeom, Soonja; Choi-Lundberg, Derek L.; Fluck, Andrew Edward; Sale, Arthur

2017-01-01

Purpose: This study aims to evaluate factors influencing undergraduate students' acceptance of a computer-aided learning resource using the Phantom Omni haptic stylus to enable rotation, touch and kinaesthetic feedback and display of names of three-dimensional (3D) human anatomical structures on a visual display. Design/methodology/approach: The…

Unpacking Students' Conceptualizations through Haptic Feedback

ERIC Educational Resources Information Center

Magana, A. J.; Balachandran, S.

2017-01-01

While it is clear that the use of computer simulations has a beneficial effect on learning when compared to instruction without computer simulations, there is still room for improvement to fully realize their benefits for learning. Haptic technologies can fulfill the educational potential of computer simulations by adding the sense of touch.…

User Acceptance of a Haptic Interface for Learning Anatomy

ERIC Educational Resources Information Center

Yeom, Soonja; Choi-Lundberg, Derek; Fluck, Andrew; Sale, Arthur

2013-01-01

Visualizing the structure and relationships in three dimensions (3D) of organs is a challenge for students of anatomy. To provide an alternative way of learning anatomy engaging multiple senses, we are developing a force-feedback (haptic) interface for manipulation of 3D virtual organs, using design research methodology, with iterations of system…

Open Touch/Sound Maps: A system to convey street data through haptic and auditory feedback

NASA Astrophysics Data System (ADS)

Kaklanis, Nikolaos; Votis, Konstantinos; Tzovaras, Dimitrios

2013-08-01

The use of spatial (geographic) information is becoming ever more central and pervasive in today's internet society but the most of it is currently inaccessible to visually impaired users. However, access in visual maps is severely restricted to visually impaired and people with blindness, due to their inability to interpret graphical information. Thus, alternative ways of a map's presentation have to be explored, in order to enforce the accessibility of maps. Multiple types of sensory perception like touch and hearing may work as a substitute of vision for the exploration of maps. The use of multimodal virtual environments seems to be a promising alternative for people with visual impairments. The present paper introduces a tool for automatic multimodal map generation having haptic and audio feedback using OpenStreetMap data. For a desired map area, an elevation map is being automatically generated and can be explored by touch, using a haptic device. A sonification and a text-to-speech (TTS) mechanism provide also audio navigation information during the haptic exploration of the map.

Learning in a virtual environment using haptic systems for movement re-education: can this medium be used for remodeling other behaviors and actions?

PubMed

Merians, Alma S; Fluet, Gerard G; Qiu, Qinyin; Lafond, Ian; Adamovich, Sergei V

2011-03-01

Robotic systems that are interfaced with virtual reality gaming and task simulations are increasingly being developed to provide repetitive intensive practice to promote increased compliance and facilitate better outcomes in rehabilitation post-stroke. A major development in the use of virtual environments (VEs) has been to incorporate tactile information and interaction forces into what was previously an essentially visual experience. Robots of varying complexity are being interfaced with more traditional virtual presentations to provide haptic feedback that enriches the sensory experience and adds physical task parameters. This provides forces that produce biomechanical and neuromuscular interactions with the VE that approximate real-world movement more accurately than visual-only VEs, simulating the weight and force found in upper extremity tasks. The purpose of this article is to present an overview of several systems that are commercially available for ambulation training and for training movement of the upper extremity. We will also report on the system that we have developed (NJIT-RAVR system) that incorporates motivating and challenging haptic feedback effects into VE simulations to facilitate motor recovery of the upper extremity post-stroke. The NJIT-RAVR system trains both the upper arm and the hand. The robotic arm acts as an interface between the participants and the VEs, enabling multiplanar movements against gravity in a three-dimensional workspace. The ultimate question is whether this medium can provide a motivating, challenging, gaming experience with dramatically decreased physical difficulty levels, which would allow for participation by an obese person and facilitate greater adherence to exercise regimes. © 2011 Diabetes Technology Society.

Should drivers be operating within an automation-free bandwidth? Evaluating haptic steering support systems with different levels of authority.

PubMed

Petermeijer, Sebastiaan M; Abbink, David A; de Winter, Joost C F

2015-02-01

The aim of this study was to compare continuous versus bandwidth haptic steering guidance in terms of lane-keeping behavior, aftereffects, and satisfaction. An important human factors question is whether operators should be supported continuously or only when tolerance limits are exceeded. We aimed to clarify this issue for haptic steering guidance by investigating costs and benefits of both approaches in a driving simulator. Thirty-two participants drove five trials, each with a different level of haptic support: no guidance (Manual); guidance outside a 0.5-m bandwidth (Band1); a hysteresis version of Band1, which guided back to the lane center once triggered (Band2); continuous guidance (Cont); and Cont with double feedback gain (ContS). Participants performed a reaction time task while driving. Toward the end of each trial, the guidance was unexpectedly disabled to investigate aftereffects. All four guidance systems prevented large lateral errors (>0.7 m). Cont and especially ContS yielded smaller lateral errors and higher time to line crossing than Manual, Band1, and Band2. Cont and ContS yielded short-lasting aftereffects, whereas Band1 and Band2 did not. Cont yielded higher self-reported satisfaction and faster reaction times than Band1. Continuous and bandwidth guidance both prevent large driver errors. Continuous guidance yields improved performance and satisfaction over bandwidth guidance at the cost of aftereffects and variability in driver torque (indicating human-automation conflicts). The presented results are useful for designers of haptic guidance systems and support critical thinking about the costs and benefits of automation support systems.

Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery

NASA Astrophysics Data System (ADS)

Gomez-Rodriguez, M.; Peters, J.; Hill, J.; Schölkopf, B.; Gharabaghi, A.; Grosse-Wentrup, M.

2011-06-01

The combination of brain-computer interfaces (BCIs) with robot-assisted physical therapy constitutes a promising approach to neurorehabilitation of patients with severe hemiparetic syndromes caused by cerebrovascular brain damage (e.g. stroke) and other neurological conditions. In such a scenario, a key aspect is how to reestablish the disrupted sensorimotor feedback loop. However, to date it is an open question how artificially closing the sensorimotor feedback loop influences the decoding performance of a BCI. In this paper, we answer this issue by studying six healthy subjects and two stroke patients. We present empirical evidence that haptic feedback, provided by a seven degrees of freedom robotic arm, facilitates online decoding of arm movement intention. The results support the feasibility of future rehabilitative treatments based on the combination of robot-assisted physical therapy with BCIs.

A simulator for surgery training: optimal sensory stimuli in a bone pinning simulation

NASA Astrophysics Data System (ADS)

Daenzer, Stefan; Fritzsche, Klaus

2008-03-01

Currently available low cost haptic devices allow inexpensive surgical training with no risk to patients. Major drawbacks of lower cost devices include limited maximum feedback force and the incapability to expose occurring moments. Aim of this work was the design and implementation of a surgical simulator that allows the evaluation of multi-sensory stimuli in order to overcome the occurring drawbacks. The simulator was built following a modular architecture to allow flexible combinations and thorough evaluation of different multi-sensory feedback modules. A Kirschner-Wire (K-Wire) tibial fracture fixation procedure was defined and implemented as a first test scenario. A set of computational metrics has been derived from the clinical requirements of the task to objectively assess the trainees performance during simulation. Sensory feedback modules for haptic and visual feedback have been developed, each in a basic and additionally in an enhanced form. First tests have shown that specific visual concepts can overcome some of the drawbacks coming along with low cost haptic devices. The simulator, the metrics and the surgery scenario together represent an important step towards a better understanding of the perception of multi-sensory feedback in complex surgical training tasks. Field studies on top of the architecture can open the way to risk-less and inexpensive surgical simulations that can keep up with traditional surgical training.

Effects of kinesthetic and cutaneous stimulation during the learning of a viscous force field.

PubMed

Rosati, Giulio; Oscari, Fabio; Pacchierotti, Claudio; Prattichizzo, Domenico

2014-01-01

Haptic stimulation can help humans learn perceptual motor skills, but the precise way in which it influences the learning process has not yet been clarified. This study investigates the role of the kinesthetic and cutaneous components of haptic feedback during the learning of a viscous curl field, taking also into account the influence of visual feedback. We present the results of an experiment in which 17 subjects were asked to make reaching movements while grasping a joystick and wearing a pair of cutaneous devices. Each device was able to provide cutaneous contact forces through a moving platform. The subjects received visual feedback about joystick's position. During the experiment, the system delivered a perturbation through (1) full haptic stimulation, (2) kinesthetic stimulation alone, (3) cutaneous stimulation alone, (4) altered visual feedback, or (5) altered visual feedback plus cutaneous stimulation. Conditions 1, 2, and 3 were also tested with the cancellation of the visual feedback of position error. Results indicate that kinesthetic stimuli played a primary role during motor adaptation to the viscous field, which is a fundamental premise to motor learning and rehabilitation. On the other hand, cutaneous stimulation alone appeared not to bring significant direct or adaptation effects, although it helped in reducing direct effects when used in addition to kinesthetic stimulation. The experimental conditions with visual cancellation of position error showed slower adaptation rates, indicating that visual feedback actively contributes to the formation of internal models. However, modest learning effects were detected when the visual information was used to render the viscous field.

Development of Velocity Guidance Assistance System by Haptic Accelerator Pedal Reaction Force Control

NASA Astrophysics Data System (ADS)

Yin, Feilong; Hayashi, Ryuzo; Raksincharoensak, Pongsathorn; Nagai, Masao

This research proposes a haptic velocity guidance assistance system for realizing eco-driving as well as enhancing traffic capacity by cooperating with ITS (Intelligent Transportation Systems). The proposed guidance system generates the desired accelerator pedal (abbreviated as pedal) stroke with respect to the desired velocity obtained from ITS considering vehicle dynamics, and provides the desired pedal stroke to the driver via a haptic pedal whose reaction force is controllable and guides the driver in order to trace the desired velocity in real time. The main purpose of this paper is to discuss the feasibility of the haptic velocity guidance. A haptic velocity guidance system for research is developed on the Driving Simulator of TUAT (DS), by attaching a low-inertia, low-friction motor to the pedal, which does not change the original characteristics of the original pedal when it is not operated, implementing an algorithm regarding the desired pedal stroke calculation and the reaction force controller. The haptic guidance maneuver is designed based on human pedal stepping experiments. A simple velocity profile with acceleration, deceleration and cruising is synthesized according to naturalistic driving for testing the proposed system. The experiment result of 9 drivers shows that the haptic guidance provides high accuracy and quick response in velocity tracking. These results prove that the haptic guidance is a promising velocity guidance method from the viewpoint of HMI (Human Machine Interface).

Development of visuo-haptic transfer for object recognition in typical preschool and school-aged children.

PubMed

Purpura, Giulia; Cioni, Giovanni; Tinelli, Francesca

2018-07-01

Object recognition is a long and complex adaptive process and its full maturation requires combination of many different sensory experiences as well as cognitive abilities to manipulate previous experiences in order to develop new percepts and subsequently to learn from the environment. It is well recognized that the transfer of visual and haptic information facilitates object recognition in adults, but less is known about development of this ability. In this study, we explored the developmental course of object recognition capacity in children using unimodal visual information, unimodal haptic information, and visuo-haptic information transfer in children from 4 years to 10 years and 11 months of age. Participants were tested through a clinical protocol, involving visual exploration of black-and-white photographs of common objects, haptic exploration of real objects, and visuo-haptic transfer of these two types of information. Results show an age-dependent development of object recognition abilities for visual, haptic, and visuo-haptic modalities. A significant effect of time on development of unimodal and crossmodal recognition skills was found. Moreover, our data suggest that multisensory processes for common object recognition are active at 4 years of age. They facilitate recognition of common objects, and, although not fully mature, are significant in adaptive behavior from the first years of age. The study of typical development of visuo-haptic processes in childhood is a starting point for future studies regarding object recognition in impaired populations.

Optimal haptic feedback control of artificial muscles

NASA Astrophysics Data System (ADS)

Chen, Daniel; Besier, Thor; Anderson, Iain; McKay, Thomas

2014-03-01

As our population ages, and trends in obesity continue to grow, joint degenerative diseases like osteoarthritis (OA) are becoming increasingly prevalent. With no cure currently in sight, the only effective treatments for OA are orthopaedic surgery and prolonged rehabilitation, neither of which is guaranteed to succeed. Gait retraining has tremendous potential to alter the contact forces in the joints due to walking, reducing the risk of one developing hip and knee OA. Dielectric Elastomer Actuators (DEAs) are being explored as a potential way of applying intuitive haptic feedback to alter a patient's walking gait. The main challenge with the use of DEAs in this application is producing large enough forces and strains to induce sensation when coupled to a patient's skin. A novel controller has been proposed to solve this issue. The controller uses simultaneous capacitive self-sensing and actuation which will optimally apply a haptic sensation to the patient's skin independent of variability in DEAs and patient geometries.

Development of a virtual reality haptic Veress needle insertion simulator for surgical skills training.

PubMed

Okrainec, A; Farcas, M; Henao, O; Choy, I; Green, J; Fotoohi, M; Leslie, R; Wight, D; Karam, P; Gonzalez, N; Apkarian, J

2009-01-01

The Veress needle is the most commonly used technique for creating the pneumoperitoneum at the start of a laparoscopic surgical procedure. Inserting the Veress needle correctly is crucial since errors can cause significant harm to patients. Unfortunately, this technique can be difficult to teach since surgeons rely heavily on tactile feedback while advancing the needle through the various layers of the abdominal wall. This critical step in laparoscopy, therefore, can be challenging for novice trainees to learn without adequate opportunities to practice in a safe environment with no risk of injury to patients. To address this issue, we have successfully developed a prototype of a virtual reality haptic needle insertion simulator using the tactile feedback of 22 surgeons to set realistic haptic parameters. A survey of these surgeons concluded that our device appeared and felt realistic, and could potentially be a useful tool for teaching the proper technique of Veress needle insertion.

Adaptive displays and controllers using alternative feedback.

PubMed

Repperger, D W

2004-12-01

Investigations on the design of haptic (force reflecting joystick or force display) controllers were conducted by viewing the display of force information within the context of several different paradigms. First, using analogies from electrical and mechanical systems, certain schemes of the haptic interface were hypothesized which may improve the human-machine interaction with respect to various criteria. A discussion is given on how this interaction benefits the electrical and mechanical system. To generalize this concept to the design of human-machine interfaces, three studies with haptic mechanisms were then synthesized and analyzed.

Control of a Robot Dancer for Enhancing Haptic Human-Robot Interaction in Waltz.

PubMed

Hongbo Wang; Kosuge, K

2012-01-01

Haptic interaction between a human leader and a robot follower in waltz is studied in this paper. An inverted pendulum model is used to approximate the human's body dynamics. With the feedbacks from the force sensor and laser range finders, the robot is able to estimate the human leader's state by using an extended Kalman filter (EKF). To reduce interaction force, two robot controllers, namely, admittance with virtual force controller, and inverted pendulum controller, are proposed and evaluated in experiments. The former controller failed the experiment; reasons for the failure are explained. At the same time, the use of the latter controller is validated by experiment results.

Enhancing Mediated Interpersonal Communication through Affective Haptics

NASA Astrophysics Data System (ADS)

Tsetserukou, Dzmitry; Neviarouskaya, Alena; Prendinger, Helmut; Kawakami, Naoki; Ishizuka, Mitsuru; Tachi, Susumu

Driven by the motivation to enhance emotionally immersive experience of real-time messaging in 3D virtual world Second Life, we are proposing a conceptually novel approach to reinforcing (intensifying) own feelings and reproducing (simulating) the emotions felt by the partner through specially designed system, iFeel_IM!. In the paper we are describing the development of novel haptic devices (HaptiHeart, HaptiHug, HaptiTickler, HaptiCooler, and HaptiWarmer) integrated into iFeel_IM! system, which architecture is presented in detail.

Review of Designs for Haptic Data Visualization.

PubMed

Paneels, Sabrina; Roberts, Jonathan C

2010-01-01

There are many different uses for haptics, such as training medical practitioners, teleoperation, or navigation of virtual environments. This review focuses on haptic methods that display data. The hypothesis is that haptic devices can be used to present information, and consequently, the user gains quantitative, qualitative, or holistic knowledge about the presented data. Not only is this useful for users who are blind or partially sighted (who can feel line graphs, for instance), but also the haptic modality can be used alongside other modalities, to increase the amount of variables being presented, or to duplicate some variables to reinforce the presentation. Over the last 20 years, a significant amount of research has been done in haptic data presentation; e.g., researchers have developed force feedback line graphs, bar charts, and other forms of haptic representations. However, previous research is published in different conferences and journals, with different application emphases. This paper gathers and collates these various designs to provide a comprehensive review of designs for haptic data visualization. The designs are classified by their representation: Charts, Maps, Signs, Networks, Diagrams, Images, and Tables. This review provides a comprehensive reference for researchers and learners, and highlights areas for further research.

Remote atomic force microscopy of microscopic organisms: Technological innovations for hands-on science with middle and high school students

NASA Astrophysics Data System (ADS)

Jones, M. G.; Andre, T.; Kubasko, D.; Bokinsky, A.; Tretter, T.; Negishi, A.; Taylor, R.; Superfine, R.

2004-01-01

This study examined hands-on experiences in the context of an investigation of viruses and explored how and why hands-on experiences may be effective. We sought to understand whether or not touching and manipulating materials and objects could lead to a deeper, more effective type of knowing than that we obtain from sight or sound alone. Four classes of high school biology students and four classes of seventh graders participated in the study that examined students' use of remote microscopy with a new scientific tool called the nanoManipulator, which enabled them to reach out and touch live viruses inside an atomic force microscope. Half of the students received full haptic (tactile and kinesthetic) feedback from a haptic joystick, whereas half of the students were able to use the haptic joystick to manipulate viruses but the tactile feedback was blocked. Results showed that there were significant gains from pre- to postinstruction across treatment groups for knowledge and attitudes. Students in both treatment groups developed conceptual models of viruses that were more consistent with current scientific research, including a move from a two-dimensional to a three-dimensional understanding of virus morphology. There were significant changes in students' understandings of scale; after instruction, students were more likely to identify examples of nanosized objects and be able to describe the degree to which a human would have to be shrunk to reach the size of a virus. Students who received full-haptic feedback had significantly better attitudes suggesting that the increased sensory feedback and stimulation may have made the experience more engaging and motivating to students.

Discriminating Tissue Stiffness with a Haptic Catheter: Feeling the Inside of the Beating Heart

PubMed Central

Kesner, Samuel B.; Howe, Robert D.

2011-01-01

Catheter devices allow physicians to access the inside of the human body easily and painlessly through natural orifices and vessels. Although catheters allow for the delivery of fluids and drugs, the deployment of devices, and the acquisition of the measurements, they do not allow clinicians to assess the physical properties of tissue inside the body due to the tissue motion and transmission limitations of the catheter devices, including compliance, friction, and backlash. The goal of this research is to increase the tactile information available to physicians during catheter procedures by providing haptic feedback during palpation procedures. To accomplish this goal, we have developed the first motion compensated actuated catheter system that enables haptic perception of fast moving tissue structures. The actuated catheter is instrumented with a distal tip force sensor and a force feedback interface that allows users to adjust the position of the catheter while experiencing the forces on the catheter tip. The efficacy of this device and interface is evaluated through a psychophyisical study comparing how accurately users can differentiate various materials attached to a cardiac motion simulator using the haptic device and a conventional manual catheter. The results demonstrate that haptics improves a user's ability to differentiate material properties and decreases the total number of errors by 50% over the manual catheter system. PMID:25285321

Robotic guidance benefits the learning of dynamic, but not of spatial movement characteristics.

PubMed

Lüttgen, Jenna; Heuer, Herbert

2012-10-01

Robotic guidance is an engineered form of haptic-guidance training and intended to enhance motor learning in rehabilitation, surgery, and sports. However, its benefits (and pitfalls) are still debated. Here, we investigate the effects of different presentation modes on the reproduction of a spatiotemporal movement pattern. In three different groups of participants, the movement was demonstrated in three different modalities, namely visual, haptic, and visuo-haptic. After demonstration, participants had to reproduce the movement in two alternating recall conditions: haptic and visuo-haptic. Performance of the three groups during recall was compared with regard to spatial and dynamic movement characteristics. After haptic presentation, participants showed superior dynamic accuracy, whereas after visual presentation, participants performed better with regard to spatial accuracy. Added visual feedback during recall always led to enhanced performance, independent of the movement characteristic and the presentation modality. These findings substantiate the different benefits of different presentation modes for different movement characteristics. In particular, robotic guidance is beneficial for the learning of dynamic, but not of spatial movement characteristics.

Design of a 7-DOF haptic master using a magneto-rheological devices for robot surgery

NASA Astrophysics Data System (ADS)

Kang, Seok-Rae; Choi, Seung-Bok; Hwang, Yong-Hoon; Cha, Seung-Woo

2017-04-01

This paper presents a 7 degrees-of-freedom (7-DOF) haptic master which is applicable to the robot-assisted minimally invasive surgery (RMIS). By utilizing a controllable magneto-rheological (MR) fluid, the haptic master can provide force information to the surgeon during surgery. The proposed haptic master consists of three degrees motions of X, Y, Z and four degrees motions of the pitch, yaw, roll and grasping. All of them have force feedback capability. The proposed haptic master can generate the repulsive forces or torques by activating MR clutch and MR brake. Both MR clutch and MR brake are designed and manufactured with consideration of the size and output torque which is usable to the robotic surgery. A proportional-integral-derivative (PID) controller is then designed and implemented to achieve torque/force tracking trajectories. It is verified that the proposed haptic master can track well the desired torque and force occurred in the surgical place by controlling the input current applied to MR clutch and brake.

High-fidelity bilateral teleoperation systems and the effect of multimodal haptics.

PubMed

Tavakoli, Mahdi; Aziminejad, Arash; Patel, Rajni V; Moallem, Mehrdad

2007-12-01

In master-slave teleoperation applications that deal with a delicate and sensitive environment, it is important to provide haptic feedback of slave/environment interactions to the user's hand as it improves task performance and teleoperation transparency (fidelity), which is the extent of telepresence of the remote environment available to the user through the master-slave system. For haptic teleoperation, in addition to a haptics-capable master interface, often one or more force sensors are also used, which warrant new bilateral control architectures while increasing the cost and the complexity of the teleoperation system. In this paper, we investigate the added benefits of using force sensors that measure hand/master and slave/environment interactions and of utilizing local feedback loops on the teleoperation transparency. We compare the two-channel and the four-channel bilateral control systems in terms of stability and transparency, and study the stability and performance robustness of the four-channel method against nonidealities that arise during bilateral control implementation, which include master-slave communication latency and changes in the environment dynamics. The next issue addressed in the paper deals with the case where the master interface is not haptics capable, but the slave is equipped with a force sensor. In the context of robotics-assisted soft-tissue surgical applications, we explore through human factors experiments whether slave/environment force measurements can be of any help with regard to improving task performance. The last problem we study is whether slave/environment force information, with and without haptic capability in the master interface, can help improve outcomes under degraded visual conditions.

EMG and EPP-integrated human-machine interface between the paralyzed and rehabilitation exoskeleton.

PubMed

Yin, Yue H; Fan, Yuan J; Xu, Li D

2012-07-01

Although a lower extremity exoskeleton shows great prospect in the rehabilitation of the lower limb, it has not yet been widely applied to the clinical rehabilitation of the paralyzed. This is partly caused by insufficient information interactions between the paralyzed and existing exoskeleton that cannot meet the requirements of harmonious control. In this research, a bidirectional human-machine interface including a neurofuzzy controller and an extended physiological proprioception (EPP) feedback system is developed by imitating the biological closed-loop control system of human body. The neurofuzzy controller is built to decode human motion in advance by the fusion of the fuzzy electromyographic signals reflecting human motion intention and the precise proprioception providing joint angular feedback information. It transmits control information from human to exoskeleton, while the EPP feedback system based on haptic stimuli transmits motion information of the exoskeleton back to the human. Joint angle and torque information are transmitted in the form of air pressure to the human body. The real-time bidirectional human-machine interface can help a patient with lower limb paralysis to control the exoskeleton with his/her healthy side and simultaneously perceive motion on the paralyzed side by EPP. The interface rebuilds a closed-loop motion control system for paralyzed patients and realizes harmonious control of the human-machine system.

A New Hybrid Viscoelastic Soft Tissue Model based on Meshless Method for Haptic Surgical Simulation

PubMed Central

Bao, Yidong; Wu, Dongmei; Yan, Zhiyuan; Du, Zhijiang

2013-01-01

This paper proposes a hybrid soft tissue model that consists of a multilayer structure and many spheres for surgical simulation system based on meshless. To improve accuracy of the model, tension is added to the three-parameter viscoelastic structure that connects the two spheres. By using haptic device, the three-parameter viscoelastic model (TPM) produces accurate deformationand also has better stress-strain, stress relaxation and creep properties. Stress relaxation and creep formulas have been obtained by mathematical formula derivation. Comparing with the experimental results of the real pig liver which were reported by Evren et al. and Amy et al., the curve lines of stress-strain, stress relaxation and creep of TPM are close to the experimental data of the real liver. Simulated results show that TPM has better real-time, stability and accuracy. PMID:24339837

Visualizing Without Vision at the Microscale: Students With Visual Impairments Explore Cells With Touch

NASA Astrophysics Data System (ADS)

Jones, M. Gail; Minogue, James; Oppewal, Tom; Cook, Michelle P.; Broadwell, Bethany

2006-12-01

Science instruction is typically highly dependent on visual representations of scientific concepts that are communicated through textbooks, teacher presentations, and computer-based multimedia materials. Little is known about how students with visual impairments access and interpret these types of visually-dependent instructional materials. This study explored the efficacy of new haptic (simulated tactile feedback and kinesthetics) instructional technology for teaching cell morphology and function to middle and high school students with visual impairments. The study examined students' prior experiences learning about the cell and cell functions in classroom instruction, as well as how haptic feedback technology impacted students' awareness of the 3-D nature of an animal cell, the morphology and function of cell organelles, and students' interest in the haptic technology as an instructional tool. Twenty-one students with visual impairment participated in the study. Students explored a tactile model of the cell with a haptic point probe that allowed them to feel the cell and its organelles. Results showed that students made significant gains in their ability to identify cell organelles and found the technology to be highly interesting as an instructional tool. The need for additional adaptive technology for students with visual impairments is discussed.

Control of an ER haptic master in a virtual slave environment for minimally invasive surgery applications

NASA Astrophysics Data System (ADS)

Han, Young-Min; Choi, Seung-Bok

2008-12-01

This paper presents the control performance of an electrorheological (ER) fluid-based haptic master device connected to a virtual slave environment that can be used for minimally invasive surgery (MIS). An already developed haptic joint featuring controllable ER fluid and a spherical joint mechanism is adopted for the master system. Medical forceps and an angular position measuring device are devised and integrated with the joint to establish the MIS master system. In order to embody a human organ in virtual space, a volumetric deformable object is used. The virtual object is then mathematically formulated by a shape-retaining chain-linked (S-chain) model. After evaluating the reflection force, computation time and compatibility with real-time control, the haptic architecture for MIS is established by incorporating the virtual slave with the master device so that the reflection force for the object of the virtual slave and the desired position for the master operator are transferred to each other. In order to achieve the desired force trajectories, a sliding mode controller is formulated and then experimentally realized. Tracking control performances for various force trajectories are evaluated and presented in the time domain.

Teaching bovine abdominal anatomy: use of a haptic simulator.

PubMed

Kinnison, Tierney; Forrest, Neil David; Frean, Stephen Philip; Baillie, Sarah

2009-01-01

Traditional methods of teaching anatomy to undergraduate medical and veterinary students are being challenged and need to adapt to modern concerns and requirements. There is a move away from the use of cadavers to new technologies as a way of complementing the traditional approaches and addressing resource and ethical problems. Haptic (touch) technology, which allows the student to feel a 3D computer-generated virtual environment, provides a novel way to address some of these challenges. To evaluate the practicalities and usefulness of a haptic simulator, first year veterinary students at the Royal Veterinary College, University of London, were taught basic bovine abdominal anatomy using a rectal palpation simulator: "The Haptic Cow." Over two days, 186 students were taught in small groups and 184 provided feedback via a questionnaire. The results were positive; the majority of students considered that the simulator had been useful for appreciating both the feel and location of key internal anatomical structures, had helped with their understanding of bovine abdominal anatomy and 3D visualization, and the tutorial had been enjoyable. The students were mostly in favor of the small group tutorial format, but some requested more time on the simulator. The findings indicate that the haptic simulator is an engaging way of teaching bovine abdominal anatomy to a large number of students in an efficient manner without using cadavers, thereby addressing some of the current challenges in anatomy teaching.

3D skin surface reconstruction from a single image by merging global curvature and local texture using the guided filtering for 3D haptic palpation.

PubMed

Lee, K; Kim, M; Kim, K

2018-05-11

Skin surface evaluation has been studied using various imaging techniques. However, all these studies had limited impact because they were performed using visual exam only. To improve on this scenario with haptic feedback, we propose 3D reconstruction of the skin surface using a single image. Unlike extant 3D skin surface reconstruction algorithms, we utilize the local texture and global curvature regions, combining the results for reconstruction. The first entails the reconstruction of global curvature, achieved by bilateral filtering that removes noise on the surface while maintaining the edge (ie, furrow) to obtain the overall curvature. The second entails the reconstruction of local texture, representing the fine wrinkles of the skin, using an advanced form of bilateral filtering. The final image is then composed by merging the two reconstructed images. We tested the curvature reconstruction part by comparing the resulting curvatures with measured values from real phantom objects while local texture reconstruction was verified by measuring skin surface roughness. Then, we showed the reconstructed result of our proposed algorithm via the reconstruction of various real skin surfaces. The experimental results demonstrate that our approach is a promising technology to reconstruct an accurate skin surface with a single skin image. We proposed 3D skin surface reconstruction using only a single camera. We highlighted the utility of global curvature, which has not been considered important in the past. Thus, we proposed a new method for 3D reconstruction that can be used for 3D haptic palpation, dividing the concepts of local and global regions. © 2018 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Multi-arm multilateral haptics-based immersive tele-robotic system (HITS) for improvised explosive device disposal

NASA Astrophysics Data System (ADS)

Erickson, David; Lacheray, Hervé; Lai, Gilbert; Haddadi, Amir

2014-06-01

This paper presents the latest advancements of the Haptics-based Immersive Tele-robotic System (HITS) project, a next generation Improvised Explosive Device (IED) disposal (IEDD) robotic interface containing an immersive telepresence environment for a remotely-controlled three-articulated-robotic-arm system. While the haptic feedback enhances the operator's perception of the remote environment, a third teleoperated dexterous arm, equipped with multiple vision sensors and cameras, provides stereo vision with proper visual cues, and a 3D photo-realistic model of the potential IED. This decentralized system combines various capabilities including stable and scaled motion, singularity avoidance, cross-coupled hybrid control, active collision detection and avoidance, compliance control and constrained motion to provide a safe and intuitive control environment for the operators. Experimental results and validation of the current system are presented through various essential IEDD tasks. This project demonstrates that a two-armed anthropomorphic Explosive Ordnance Disposal (EOD) robot interface can achieve complex neutralization techniques against realistic IEDs without the operator approaching at any time.

Perception of force and stiffness in the presence of low-frequency haptic noise

PubMed Central

Gurari, Netta; Okamura, Allison M.; Kuchenbecker, Katherine J.

2017-01-01

Objective This work lays the foundation for future research on quantitative modeling of human stiffness perception. Our goal was to develop a method by which a human’s ability to perceive suprathreshold haptic force stimuli and haptic stiffness stimuli can be affected by adding haptic noise. Methods Five human participants performed a same-different task with a one-degree-of-freedom force-feedback device. Participants used the right index finger to actively interact with variations of force (∼5 and ∼8 N) and stiffness (∼290 N/m) stimuli that included one of four scaled amounts of haptically rendered noise (None, Low, Medium, High). The haptic noise was zero-mean Gaussian white noise that was low-pass filtered with a 2 Hz cut-off frequency; the resulting low-frequency signal was added to the force rendered while the participant interacted with the force and stiffness stimuli. Results We found that the precision with which participants could identify the magnitude of both the force and stiffness stimuli was affected by the magnitude of the low-frequency haptically rendered noise added to the haptic stimulus, as well as the magnitude of the haptic stimulus itself. The Weber fraction strongly correlated with the standard deviation of the low-frequency haptic noise with a Pearson product-moment correlation coefficient of ρ > 0.83. The mean standard deviation of the low-frequency haptic noise in the haptic stimuli ranged from 0.184 N to 1.111 N across the four haptically rendered noise levels, and the corresponding mean Weber fractions spanned between 0.042 and 0.101. Conclusions The human ability to perceive both suprathreshold haptic force and stiffness stimuli degrades in the presence of added low-frequency haptic noise. Future work can use the reported methods to investigate how force perception and stiffness perception may relate, with possible applications in haptic watermarking and in the assessment of the functionality of peripheral pathways in individuals with haptic impairments. PMID:28575068

Equipment and technology in surgical robotics.

PubMed

Sim, Hong Gee; Yip, Sidney Kam Hung; Cheng, Christopher Wai Sam

2006-06-01

Contemporary medical robotic systems used in urologic surgery usually consist of a computer and a mechanical device to carry out the designated task with an image acquisition module. These systems are typically from one of the two categories: offline or online robots. Offline robots, also known as fixed path robots, are completely automated with pre-programmed motion planning based on pre-operative imaging studies where precise movements within set confines are carried out. Online robotic systems rely on continuous input from the surgeons and change their movements and actions according to the input in real time. This class of robots is further divided into endoscopic manipulators and master-slave robotic systems. Current robotic surgical systems have resulted in a paradigm shift in the minimally invasive approach to complex laparoscopic urological procedures. Future developments will focus on refining haptic feedback, system miniaturization and improved augmented reality and telesurgical capabilities.

Haptic identification of objects and their depictions.

PubMed

Klatzky, R L; Loomis, J M; Lederman, S J; Wake, H; Fujita, N

1993-08-01

Haptic identification of real objects is superior to that of raised two-dimensional (2-D) depictions. Three explanations of real-object superiority were investigated: contribution of material information, contribution of 3-D shape and size, and greater potential for integration across the fingers. In Experiment 1, subjects, while wearing gloves that gently attenuated material information, haptically identified real objects that provided reduced cues to compliance, mass, and part motion. The gloves permitted exploration with free hand movement, a single outstretched finger, or five outstretched fingers. Performance decreased over these three conditions but was superior to identification of pictures of the same objects in all cases, indicating the contribution of 3-D structure and integration across the fingers. Picture performance was also better with five fingers than with one. In Experiment 2, the subjects wore open-fingered gloves, which provided them with material information. Consequently, the effect of type of exploration was substantially reduced but not eliminated. Material compensates somewhat for limited access to object structure but is not the primary basis for haptic object identification.

An electromechanical based deformable model for soft tissue simulation.

PubMed

Zhong, Yongmin; Shirinzadeh, Bijan; Smith, Julian; Gu, Chengfan

2009-11-01

Soft tissue deformation is of great importance to surgery simulation. Although a significant amount of research efforts have been dedicated to simulating the behaviours of soft tissues, modelling of soft tissue deformation is still a challenging problem. This paper presents a new deformable model for simulation of soft tissue deformation from the electromechanical viewpoint of soft tissues. Soft tissue deformation is formulated as a reaction-diffusion process coupled with a mechanical load. The mechanical load applied to a soft tissue to cause a deformation is incorporated into the reaction-diffusion system, and consequently distributed among mass points of the soft tissue. Reaction-diffusion of mechanical load and non-rigid mechanics of motion are combined to govern the simulation dynamics of soft tissue deformation. An improved reaction-diffusion model is developed to describe the distribution of the mechanical load in soft tissues. A three-layer artificial cellular neural network is constructed to solve the reaction-diffusion model for real-time simulation of soft tissue deformation. A gradient based method is established to derive internal forces from the distribution of the mechanical load. Integration with a haptic device has also been achieved to simulate soft tissue deformation with haptic feedback. The proposed methodology does not only predict the typical behaviours of living tissues, but it also accepts both local and large-range deformations. It also accommodates isotropic, anisotropic and inhomogeneous deformations by simple modification of diffusion coefficients.

Virtual wall-based haptic-guided teleoperated surgical robotic system for single-port brain tumor removal surgery.

PubMed

Seung, Sungmin; Choi, Hongseok; Jang, Jongseong; Kim, Young Soo; Park, Jong-Oh; Park, Sukho; Ko, Seong Young

2017-01-01

This article presents a haptic-guided teleoperation for a tumor removal surgical robotic system, so-called a SIROMAN system. The system was developed in our previous work to make it possible to access tumor tissue, even those that seat deeply inside the brain, and to remove the tissue with full maneuverability. For a safe and accurate operation to remove only tumor tissue completely while minimizing damage to the normal tissue, a virtual wall-based haptic guidance together with a medical image-guided control is proposed and developed. The virtual wall is extracted from preoperative medical images, and the robot is controlled to restrict its motion within the virtual wall using haptic feedback. Coordinate transformation between sub-systems, a collision detection algorithm, and a haptic-guided teleoperation using a virtual wall are described in the context of using SIROMAN. A series of experiments using a simplified virtual wall are performed to evaluate the performance of virtual wall-based haptic-guided teleoperation. With haptic guidance, the accuracy of the robotic manipulator's trajectory is improved by 57% compared to one without. The tissue removal performance is also improved by 21% ( p < 0.05). The experiments show that virtual wall-based haptic guidance provides safer and more accurate tissue removal for single-port brain surgery.

Modeling the behavior of human body tissues on penetration

NASA Astrophysics Data System (ADS)

Conci, A.; Brazil, A. L.; Popovici, D.; Jiga, G.; Lebon, F.

2018-02-01

Several procedures in medicine (such as anesthesia, injections, biopsies and percutaneous treatments) involve a needle insertion. Such procedures operate without vision of the internal involved areas. Physicians and anesthetists rely on manual (force and tactile) feedback to guide their movements, so a number of medical practice is strongly based on manual skill. In order to be expert in the execution of such procedures the medical students must practice a number of times, but before practice in a real patient they must be trained in some place and a virtual environment, using Virtual Reality (VR) or Augmented Reality (AR) is the best possible solution for such training. In a virtual environment the success of user practices is improved by the addition of force output using haptic device to improve the manual sensations in the interactions between user and computer. Haptic devices enable simulate the physical restriction of the diverse tissues and force reactions to movements of operator hands. The trainees can effectively "feel" the reactions to theirs movements and receive immediate feedback from the actions executed by them in the implemented environment. However, in order to implement such systems, the tissue reaction to penetration and cutting must be modeled. A proper model must emulate the physical sensations of the needle action in the skin, fat, muscle, and so one, as if it really done in a patient that is as they are holding a real needle and feeling each tissue resistance when inserting it through the body. For example an average force value for human skin puncture is 6.0 N, it is 2.0 N for subcutaneous fat tissue and 4.4 N for muscles: this difference of sensations to penetration of each layers trespassed by the needle makes possible to suppose the correct position inside the body. This work presents a model for tissues before and after the cutting that with proper assumptions of proprieties can model any part of human body. It was based on experiments and used in embryonic system for epidural anesthesia having good evaluation as presented in the last section "Preliminary Results".

Haptic-STM: a human-in-the-loop interface to a scanning tunneling microscope.

PubMed

Perdigão, Luís M A; Saywell, Alex

2011-07-01

The operation of a haptic device interfaced with a scanning tunneling microscope (STM) is presented here. The user moves the STM tip in three dimensions by means of a stylus attached to the haptic instrument. The tunneling current measured by the STM is converted to a vertical force, applied to the stylus and felt by the user, with the user being incorporated into the feedback loop that controls the tip-surface distance. A haptic-STM interface of this nature allows the user to feel atomic features on the surface and facilitates the tactile manipulation of the adsorbate/substrate system. The operation of this device is demonstrated via the room temperature STM imaging of C(60) molecules adsorbed on an Au(111) surface in ultra-high vacuum.

A haptic pedal for surgery assistance.

PubMed

Díaz, Iñaki; Gil, Jorge Juan; Louredo, Marcos

2014-09-01

The research and development of mechatronic aids for surgery is a persistent challenge in the field of robotic surgery. This paper presents a new haptic pedal conceived to assist surgeons in the operating room by transmitting real-time surgical information through the foot. An effective human-robot interaction system for medical practice must exchange appropriate information with the operator as quickly and accurately as possible. Moreover, information must flow through the appropriate sensory modalities for a natural and simple interaction. However, users of current robotic systems might experience cognitive overload and be increasingly overwhelmed by data streams from multiple modalities. A new haptic channel is thus explored to complement and improve existing systems. A preliminary set of experiments has been carried out to evaluate the performance of the proposed system in a virtual surgical drilling task. The results of the experiments show the effectiveness of the haptic pedal in providing surgical information through the foot. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

Feel, imagine and learn! - Haptic augmented simulation and embodied instruction in physics learning

NASA Astrophysics Data System (ADS)

Han, In Sook

The purpose of this study was to investigate the potentials and effects of an embodied instructional model in abstract concept learning. This embodied instructional process included haptic augmented educational simulation as an instructional tool to provide perceptual experiences as well as further instruction to activate those previous experiences with perceptual simulation. In order to verify the effectiveness of this instructional model, haptic augmented simulation with three different haptic levels (force and kinesthetic, kinesthetic, and non-haptic) and instructional materials (narrative and expository) were developed and their effectiveness tested. 220 fifth grade students were recruited to participate in the study from three elementary schools located in lower SES neighborhoods in Bronx, New York. The study was conducted for three consecutive weeks in regular class periods. The data was analyzed using ANCOVA, ANOVA, and MANOVA. The result indicates that haptic augmented simulations, both the force and kinesthetic and the kinesthetic simulations, was more effective than the non-haptic simulation in providing perceptual experiences and helping elementary students to create multimodal representations about machines' movements. However, in most cases, force feedback was needed to construct a fully loaded multimodal representation that could be activated when the instruction with less sensory modalities was being given. In addition, the force and kinesthetic simulation was effective in providing cognitive grounding to comprehend a new learning content based on the multimodal representation created with enhanced force feedback. Regarding the instruction type, it was found that the narrative and the expository instructions did not make any difference in activating previous perceptual experiences. These findings suggest that it is important to help students to make a solid cognitive ground with perceptual anchor. Also, sequential abstraction process would deepen students' understanding by providing an opportunity to practice their mental simulation by removing sensory modalities used one by one and to gradually reach abstract level of understanding where students can imagine the machine's movements and working mechanisms with only abstract language without any perceptual supports.

Acquisition and Visualization Techniques of Human Motion Using Master-Slave System and Haptograph

NASA Astrophysics Data System (ADS)

Katsura, Seiichiro; Ohishi, Kiyoshi

Artificial acquisition and reproduction of human sensations are basic technologies of communication engineering. For example, auditory information is obtained by a microphone, and a speaker reproduces it by artificial means. Furthermore, a video camera and a television make it possible to transmit visual sensation by broadcasting. On the contrary, since tactile or haptic information is subject to the Newton's “law of action and reaction” in the real world, a device which acquires, transmits, and reproduces the information has not been established. From the point of view, real-world haptics is the key technology for future haptic communication engineering. This paper proposes a novel acquisition method of haptic information named “haptograph”. The haptograph visualizes the haptic information like photograph. Since temporal and spatial analyses are conducted to represent haptic information as the haptograph, it is possible to be recognized and to be evaluated intuitively. In this paper, the proposed haptograph is applied to visualization of human motion. It is possible to represent the motion characteristics, the expert's skill and the personal habit, and so on. In other words, a personal encyclopedia is attained. Once such a personal encyclopedia is stored in ubiquitous environment, the future human support technology will be developed.

Haptic feedback can provide an objective assessment of arthroscopic skills.

PubMed

Chami, George; Ward, James W; Phillips, Roger; Sherman, Kevin P

2008-04-01

The outcome of arthroscopic procedures is related to the surgeon's skills in arthroscopy. Currently, evaluation of such skills relies on direct observation by a surgeon trainer. This type of assessment, by its nature, is subjective and time-consuming. The aim of our study was to identify whether haptic information generated from arthroscopic tools could distinguish between skilled and less skilled surgeons. A standard arthroscopic probe was fitted with a force/torque sensor. The probe was used by five surgeons with different levels of experience in knee arthroscopy performing 11 different tasks in 10 standard knee arthroscopies. The force/torque data from the hand and tool interface were recorded and synchronized with a video recording of the procedure. The torque magnitude and patterns generated were analyzed and compared. A computerized system was used to analyze the force/torque signature based on general principles for quality of performance using such measures as economy in movement, time efficiency, and consistency in performance. The results showed a considerable correlation between three haptic parameters and the surgeon's experience, which could be used in an automated objective assessment system for arthroscopic surgery. Level II, diagnostic study. See the Guidelines for Authors for a complete description of levels of evidence.

Prototype tactile feedback system for examination by skin touch.

PubMed

Lee, O; Lee, K; Oh, C; Kim, K; Kim, M

2014-08-01

Diagnosis of conditions such as psoriasis and atopic dermatitis, in the case of induration, involves palpating the infected area via hands and then selecting a ratings score. However, the score is determined based on the tester's experience and standards, making it subjective. To provide tactile feedback on the skin, we developed a prototype tactile feedback system to simulate skin wrinkles with PHANToM OMNI. To provide the user with tactile feedback on skin wrinkles, a visual and haptic Augmented Reality system was developed. First, a pair of stereo skin images obtained by a stereo camera generates a disparity map of skin wrinkles. Second, the generated disparity map is sent to an implemented tactile rendering algorithm that computes a reaction force according to the user's interaction with the skin image. We first obtained a stereo image of skin wrinkles from the in vivo stereo imaging system, which has a baseline of 50.8 μm, and obtained the disparity map with a graph cuts algorithm. The left image is displayed on the monitor to enable the user to recognize the location visually. The disparity map of the skin wrinkle image sends skin wrinkle information as a tactile response to the user through a haptic device. We successfully developed a tactile feedback system for virtual skin wrinkle simulation by means of a commercialized haptic device that provides the user with a single point of contact to feel the surface roughness of a virtual skin sample. © 2013 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

A haptic sensing upgrade for the current EOD robotic fleet

NASA Astrophysics Data System (ADS)

Rowe, Patrick

2014-06-01

The past decade and a half has seen a tremendous rise in the use of mobile manipulator robotic platforms for bomb inspection and disposal, explosive ordnance disposal, and other extremely hazardous tasks in both military and civilian settings. Skilled operators are able to control these robotic vehicles in amazing ways given the very limited situational awareness obtained from a few on-board camera views. Future generations of robotic platforms will, no doubt, provide some sort of additional force or haptic sensor feedback to further enhance the operator's interaction with the robot, especially when dealing with fragile, unstable, and explosive objects. Unfortunately, the robot operators need this capability today. This paper discusses an approach to provide existing (and future) robotic mobile manipulator platforms, with which trained operators are already familiar and highly proficient, this desired haptic and force feedback capability. The goals of this technology are to be rugged, reliable, and affordable. It should also be able to be applied to a wide range of existing robots with a wide variety of manipulator/gripper sizes and styles. Finally, the presentation of the haptic information to the operator is discussed, given the fact that control devices that physically interact with the operators are not widely available and still in the research stages.

Fiber optical sensor system for shape and haptics for flexible instruments in minimally invasive surgery: overview and status quo

NASA Astrophysics Data System (ADS)

Ledermann, Christoph; Pauer, Hendrikje; Woern, Heinz

2014-05-01

In minimally invasive surgery, exible mechatronic instruments promise to improve the overall performance of surgical interventions. However, those instruments require highly developed sensors in order to provide haptic feedback to the surgeon or to enable (semi-)autonomous tasks. Precisely, haptic sensors and a shape sensor are required. In this paper, we present our ber optical sensor system of Fiber Bragg Gratings, which consists of a shape sensor, a kinesthetic sensor and a tactile sensor. The status quo of each of the three sensors is described, as well as the concept to integrate them into one ber optical sensor system.

Sensorimotor enhancement with a mixed reality system for balance and mobility rehabilitation.

PubMed

Fung, Joyce; Perez, Claire F

2011-01-01

We have developed a mixed reality system incorporating virtual reality (VR), surface perturbations and light touch for gait rehabilitation. Haptic touch has emerged as a novel and efficient technique to improve postural control and dynamic stability. Our system combines visual display with the manipulation of physical environments and addition of haptic feedback to enhance balance and mobility post stroke. A research study involving 9 participants with stroke and 9 age-matched healthy individuals show that the haptic cue provided while walking is an effective means of improving gait stability in people post stroke, especially during challenging environmental conditions such as downslope walking.

Differential effects of delay upon visually and haptically guided grasping and perceptual judgments.

PubMed

Pettypiece, Charles E; Culham, Jody C; Goodale, Melvyn A

2009-05-01

Experiments with visual illusions have revealed a dissociation between the systems that mediate object perception and those responsible for object-directed action. More recently, an experiment on a haptic version of the visual size-contrast illusion has provided evidence for the notion that the haptic modality shows a similar dissociation when grasping and estimating the size of objects in real-time. Here we present evidence suggesting that the similarities between the two modalities begin to break down once a delay is introduced between when people feel the target object and when they perform the grasp or estimation. In particular, when grasping after a delay in a haptic paradigm, people scale their grasps differently when the target is presented with a flanking object of a different size (although the difference does not reflect a size-contrast effect). When estimating after a delay, however, it appears that people ignore the size of the flanking objects entirely. This does not fit well with the results commonly found in visual experiments. Thus, introducing a delay reveals important differences in the way in which haptic and visual memories are stored and accessed.

H-Man: a planar, H-shape cabled differential robotic manipulandum for experiments on human motor control.

PubMed

Campolo, Domenico; Tommasino, Paolo; Gamage, Kumudu; Klein, Julius; Hughes, Charmayne M L; Masia, Lorenzo

2014-09-30

In the last decades more robotic manipulanda have been employed to investigate the effect of haptic environments on motor learning and rehabilitation. However, implementing complex haptic renderings can be challenging from technological and control perspectives. We propose a novel robot (H-Man) characterized by a mechanical design based on cabled differential transmission providing advantages over current robotic technology. The H-Man transmission translates to extremely simplified kinematics and homogenous dynamic properties, offering the possibility to generate haptic channels by passively blocking the mechanics, and eliminating stability concerns. We report results of experiments characterizing the performance of the device (haptic bandwidth, Z-width, and perceived impedance). We also present the results of a study investigating the influence of haptic channel compliance on motor learning in healthy individuals, which highlights the effects of channel compliance in enhancing proprioceptive information. The generation of haptic channels to study motor redundancy is not easy for actual robots because of the needs of powerful actuation and complex real-time control implementation. The mechanical design of H-Man affords the possibility to promptly create haptic channels by mechanical stoppers (on one of the motors) without compromising the superior backdriveability and high isotropic manipulability. This paper presents a novel robotic device for motor control studies and robotic rehabilitation. The hardware was designed with specific emphasis on the mechanics that result in a system that is easy to control, homogeneous, and is intrinsically safe for use. Copyright © 2014 Elsevier B.V. All rights reserved.

A haptic-inspired audio approach for structural health monitoring decision-making

NASA Astrophysics Data System (ADS)

Mao, Zhu; Todd, Michael; Mascareñas, David

2015-03-01

Haptics is the field at the interface of human touch (tactile sensation) and classification, whereby tactile feedback is used to train and inform a decision-making process. In structural health monitoring (SHM) applications, haptic devices have been introduced and applied in a simplified laboratory scale scenario, in which nonlinearity, representing the presence of damage, was encoded into a vibratory manual interface. In this paper, the "spirit" of haptics is adopted, but here ultrasonic guided wave scattering information is transformed into audio (rather than tactile) range signals. After sufficient training, the structural damage condition, including occurrence and location, can be identified through the encoded audio waveforms. Different algorithms are employed in this paper to generate the transformed audio signals and the performance of each encoding algorithms is compared, and also compared with standard machine learning classifiers. In the long run, the haptic decision-making is aiming to detect and classify structural damages in a more rigorous environment, and approaching a baseline-free fashion with embedded temperature compensation.

Spatial asymmetry in tactile sensor skin deformation aids perception of edge orientation during haptic exploration.

PubMed

Ponce Wong, Ruben D; Hellman, Randall B; Santos, Veronica J

2014-01-01

Upper-limb amputees rely primarily on visual feedback when using their prostheses to interact with others or objects in their environment. A constant reliance upon visual feedback can be mentally exhausting and does not suffice for many activities when line-of-sight is unavailable. Upper-limb amputees could greatly benefit from the ability to perceive edges, one of the most salient features of 3D shape, through touch alone. We present an approach for estimating edge orientation with respect to an artificial fingertip through haptic exploration using a multimodal tactile sensor on a robot hand. Key parameters from the tactile signals for each of four exploratory procedures were used as inputs to a support vector regression model. Edge orientation angles ranging from -90 to 90 degrees were estimated with an 85-input model having an R (2) of 0.99 and RMS error of 5.08 degrees. Electrode impedance signals provided the most useful inputs by encoding spatially asymmetric skin deformation across the entire fingertip. Interestingly, sensor regions that were not in direct contact with the stimulus provided particularly useful information. Methods described here could pave the way for semi-autonomous capabilities in prosthetic or robotic hands during haptic exploration, especially when visual feedback is unavailable.

A haptic device for guide wire in interventional radiology procedures.

PubMed

Moix, Thomas; Ilic, Dejan; Bleuler, Hannes; Zoethout, Jurjen

2006-01-01

Interventional Radiology (IR) is a minimally invasive procedure where thin tubular instruments, guide wires and catheters, are steered through the patient's vascular system under X-ray imaging. In order to perform these procedures, a radiologist has to be trained to master hand-eye coordination, instrument manipulation and procedure protocols. The existing simulation systems all have major drawbacks: the use of modified instruments, unrealistic insertion lengths, high inertia of the haptic device that creates a noticeably degraded dynamic behavior or excessive friction that is not properly compensated for. In this paper we propose a quality training environment dedicated to IR. The system is composed of a virtual reality (VR) simulation of the patient's anatomy linked to a robotic interface providing haptic force feedback. This paper focuses on the requirements, design and prototyping of a specific haptic interface for guide wires.

Training Surgical Residents With a Haptic Robotic Central Venous Catheterization Simulator.

PubMed

Pepley, David F; Gordon, Adam B; Yovanoff, Mary A; Mirkin, Katelin A; Miller, Scarlett R; Han, David C; Moore, Jason Z

Ultrasound guided central venous catheterization (CVC) is a common surgical procedure with complication rates ranging from 5 to 21 percent. Training is typically performed using manikins that do not simulate anatomical variations such as obesity and abnormal vessel positioning. The goal of this study was to develop and validate the effectiveness of a new virtual reality and force haptic based simulation platform for CVC of the right internal jugular vein. A CVC simulation platform was developed using a haptic robotic arm, 3D position tracker, and computer visualization. The haptic robotic arm simulated needle insertion force that was based on cadaver experiments. The 3D position tracker was used as a mock ultrasound device with realistic visualization on a computer screen. Upon completion of a practice simulation, performance feedback is given to the user through a graphical user interface including scoring factors based on good CVC practice. The effectiveness of the system was evaluated by training 13 first year surgical residents using the virtual reality haptic based training system over a 3 month period. The participants' performance increased from 52% to 96% on the baseline training scenario, approaching the average score of an expert surgeon: 98%. This also resulted in improvement in positive CVC practices including a 61% decrease between final needle tip position and vein center, a decrease in mean insertion attempts from 1.92 to 1.23, and a 12% increase in time spent aspirating the syringe throughout the procedure. A virtual reality haptic robotic simulator for CVC was successfully developed. Surgical residents training on the simulation improved to near expert levels after three robotic training sessions. This suggests that this system could act as an effective training device for CVC. Copyright © 2017 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Hand-eye calibration for rigid laparoscopes using an invariant point.

PubMed

Thompson, Stephen; Stoyanov, Danail; Schneider, Crispin; Gurusamy, Kurinchi; Ourselin, Sébastien; Davidson, Brian; Hawkes, David; Clarkson, Matthew J

2016-06-01

Laparoscopic liver resection has significant advantages over open surgery due to less patient trauma and faster recovery times, yet it can be difficult due to the restricted field of view and lack of haptic feedback. Image guidance provides a potential solution but one current challenge is in accurate "hand-eye" calibration, which determines the position and orientation of the laparoscope camera relative to the tracking markers. In this paper, we propose a simple and clinically feasible calibration method based on a single invariant point. The method requires no additional hardware, can be constructed by theatre staff during surgical setup, requires minimal image processing and can be visualised in real time. Real-time visualisation allows the surgical team to assess the calibration accuracy before use in surgery. In addition, in the laboratory, we have developed a laparoscope with an electromagnetic tracking sensor attached to the camera end and an optical tracking marker attached to the distal end. This enables a comparison of tracking performance. We have evaluated our method in the laboratory and compared it to two widely used methods, "Tsai's method" and "direct" calibration. The new method is of comparable accuracy to existing methods, and we show RMS projected error due to calibration of 1.95 mm for optical tracking and 0.85 mm for EM tracking, versus 4.13 and 1.00 mm respectively, using existing methods. The new method has also been shown to be workable under sterile conditions in the operating room. We have proposed a new method of hand-eye calibration, based on a single invariant point. Initial experience has shown that the method provides visual feedback, satisfactory accuracy and can be performed during surgery. We also show that an EM sensor placed near the camera would provide significantly improved image overlay accuracy.

Improving manual skills in persons with disabilities (PWD) through a multimodal assistance system.

PubMed

Covarrubias, Mario; Gatti, Elia; Bordegoni, Monica; Cugini, Umberto; Mansutti, Alessandro

2014-07-01

In this research work, we present a Multimodal Guidance System (MGS) whose aim is to provide dynamic assistance to persons with disabilities (PWD) while performing manual activities such as drawing, coloring in and foam-cutting tasks. The MGS provides robotic assistance in the execution of 2D tasks through haptic and sound interactions. Haptic technology provides the virtual path of 2D shapes through the point-based approach, while sound technology provides audio feedback inputs related to the hand's velocity while sketching and filling or cutting operations. By combining this Multimodal System with the haptic assistance, we have created a new approach with possible applications to such diverse fields as physical rehabilitation, scientific investigation of sensorimotor learning and assessment of hand movements in PWD. The MGS has been tested by people with specific disorders affecting coordination, such as Down syndrome and developmental disabilities, under the supervision of their teachers and care assistants inside their learning environment. A Graphic User Interface has been designed for teachers and care assistants in order to provide training during the test sessions. Our results provide conclusive evidence that the effect of using the MGS increases the accuracy in the tasks operations. The Multimodal Guidance System (MGS) is an interface that offers haptic and sound feedback while performing manual tasks. Several studies demonstrated that the haptic guidance systems can help people in recovering cognitive function at different levels of complexity and impairment. The applications supported by our device could also have an important role in supporting physical therapist and cognitive psychologist in helping patients to recover motor and visuo-spatial abilities.

Capturing patient experience: a qualitative study of implementing real-time feedback in primary care

PubMed Central

Carter, Mary; Davey, Antoinette; Wright, Christine; Elmore, Natasha; Newbould, Jenny; Roland, Martin; Campbell, John; Burt, Jenni

2016-01-01

Background In recent years, hospitals have made use of new technologies, such as real-time feedback, to collect patient experience information. This approach is currently rarely used in primary care settings, but may provide practices with a useful tool that enables them to take prompt, focused action to improve their services. Aim To identify the factors inhibiting and enabling the implementation of real-time feedback in general practices. Design and setting Qualitative study embedded within an exploratory trial (July 2014 to February 2015) of a real-time feedback intervention targeting patient experience in general practices in south-west England and Cambridgeshire. Method Semi-structured interviews (n = 22) and focus groups (n = 4, total of 28 attendees) with practice staff were audiorecorded, transcribed, and analysed thematically, using a framework based on constructs from normalisation process theory. Results Staff engagement with real-time feedback varied considerably, and staff made sense of real-time feedback by comparing it with more familiar feedback modalities. Effective within-team communication was associated with positive attitudes towards real-time feedback. Timing of requests for feedback was important in relation to patient engagement. Real-time feedback may offer potential as a means of informing practice development, perhaps as a component of a wider programme of capturing and responding to patients’ comments. Conclusion Successful implementation of real-time feedback requires effective communication across the practice team to engender thorough engagement. Feedback processes should be carefully introduced to fit with existing patient and practice routines. Future studies should consider making real-time feedback content relevant to specific practice needs, and support participation by all patient groups. PMID:27621292

Haptic biofeedback for improving compliance with lower-extremity partial weight bearing.

PubMed

Fu, Michael C; DeLuke, Levi; Buerba, Rafael A; Fan, Richard E; Zheng, Ying Jean; Leslie, Michael P; Baumgaertner, Michael R; Grauer, Jonathan N

2014-11-01

After lower-extremity orthopedic trauma and surgery, patients are often advised to restrict weight bearing on the affected limb. Conventional training methods are not effective at enabling patients to comply with recommendations for partial weight bearing. The current study assessed a novel method of using real-time haptic (vibratory/vibrotactile) biofeedback to improve compliance with instructions for partial weight bearing. Thirty healthy, asymptomatic participants were randomized into 1 of 3 groups: verbal instruction, bathroom scale training, and haptic biofeedback. Participants were instructed to restrict lower-extremity weight bearing in a walking boot with crutches to 25 lb, with an acceptable range of 15 to 35 lb. A custom weight bearing sensor and biofeedback system was attached to all participants, but only those in the haptic biofeedback group were given a vibrotactile signal if they exceeded the acceptable range. Weight bearing in all groups was measured with a separate validated commercial system. The verbal instruction group bore an average of 60.3±30.5 lb (mean±standard deviation). The bathroom scale group averaged 43.8±17.2 lb, whereas the haptic biofeedback group averaged 22.4±9.1 lb (P<.05). As a percentage of body weight, the verbal instruction group averaged 40.2±19.3%, the bathroom scale group averaged 32.5±16.9%, and the haptic biofeedback group averaged 14.5±6.3% (P<.05). In this initial evaluation of the use of haptic biofeedback to improve compliance with lower-extremity partial weight bearing, haptic biofeedback was superior to conventional physical therapy methods. Further studies in patients with clinical orthopedic trauma are warranted. Copyright 2014, SLACK Incorporated.

Haptic simulation framework for determining virtual dental occlusion.

PubMed

Wu, Wen; Chen, Hui; Cen, Yuhai; Hong, Yang; Khambay, Balvinder; Heng, Pheng Ann

2017-04-01

The surgical treatment of many dentofacial deformities is often complex due to its three-dimensional nature. To determine the dental occlusion in the most stable position is essential for the success of the treatment. Computer-aided virtual planning on individualized patient-specific 3D model can help formulate the surgical plan and predict the surgical change. However, in current computer-aided planning systems, it is not possible to determine the dental occlusion of the digital models in the intuitive way during virtual surgical planning because of absence of haptic feedback. In this paper, a physically based haptic simulation framework is proposed, which can provide surgeons with the intuitive haptic feedback to determine the dental occlusion of the digital models in their most stable position. To provide the physically realistic force feedback when the dental models contact each other during the searching process, the contact model is proposed to describe the dynamic and collision properties of the dental models during the alignment. The simulated impulse/contact-based forces are integrated into the unified simulation framework. A validation study has been conducted on fifteen sets of virtual dental models chosen at random and covering a wide range of the dental relationships found clinically. The dental occlusions obtained by an expert were employed as a benchmark to compare the virtual occlusion results. The mean translational and angular deviations of the virtual occlusion results from the benchmark were small. The experimental results show the validity of our method. The simulated forces can provide valuable insights to determine the virtual dental occlusion. The findings of this work and the validation of proposed concept lead the way for full virtual surgical planning on patient-specific virtual models allowing fully customized treatment plans for the surgical correction of dentofacial deformities.

A Physics-driven Neural Networks-based Simulation System (PhyNNeSS) for multimodal interactive virtual environments involving nonlinear deformable objects

PubMed Central

De, Suvranu; Deo, Dhannanjay; Sankaranarayanan, Ganesh; Arikatla, Venkata S.

2012-01-01

Background While an update rate of 30 Hz is considered adequate for real time graphics, a much higher update rate of about 1 kHz is necessary for haptics. Physics-based modeling of deformable objects, especially when large nonlinear deformations and complex nonlinear material properties are involved, at these very high rates is one of the most challenging tasks in the development of real time simulation systems. While some specialized solutions exist, there is no general solution for arbitrary nonlinearities. Methods In this work we present PhyNNeSS - a Physics-driven Neural Networks-based Simulation System - to address this long-standing technical challenge. The first step is an off-line pre-computation step in which a database is generated by applying carefully prescribed displacements to each node of the finite element models of the deformable objects. In the next step, the data is condensed into a set of coefficients describing neurons of a Radial Basis Function network (RBFN). During real-time computation, these neural networks are used to reconstruct the deformation fields as well as the interaction forces. Results We present realistic simulation examples from interactive surgical simulation with real time force feedback. As an example, we have developed a deformable human stomach model and a Penrose-drain model used in the Fundamentals of Laparoscopic Surgery (FLS) training tool box. Conclusions A unique computational modeling system has been developed that is capable of simulating the response of nonlinear deformable objects in real time. The method distinguishes itself from previous efforts in that a systematic physics-based pre-computational step allows training of neural networks which may be used in real time simulations. We show, through careful error analysis, that the scheme is scalable, with the accuracy being controlled by the number of neurons used in the simulation. PhyNNeSS has been integrated into SoFMIS (Software Framework for Multimodal Interactive Simulation) for general use. PMID:22629108

A review of haptic simulator for oral and maxillofacial surgery based on virtual reality.

PubMed

Chen, Xiaojun; Hu, Junlei

2018-06-01

Traditional medical training in oral and maxillofacial surgery (OMFS) may be limited by its low efficiency and high price due to the shortage of cadaver resources. With the combination of visual rendering and feedback force, surgery simulators become increasingly popular in hospitals and medical schools as an alternative to the traditional training. Areas covered: The major goal of this review is to provide a comprehensive reference source of current and future developments of haptic OMFS simulators based on virtual reality (VR) for relevant researchers. Expert commentary: Visual rendering, haptic rendering, tissue deformation, and evaluation are key components of haptic surgery simulator based on VR. Compared with traditional medical training, virtual and tactical fusion of virtual environment in surgery simulator enables considerably vivid sensation, and the operators have more opportunities to practice surgical skills and receive objective evaluation as reference.

NASA's Hybrid Reality Lab: One Giant Leap for Full Dive

NASA Technical Reports Server (NTRS)

Delgado, Francisco J.; Noyes, Matthew

2017-01-01

This presentation demonstrates how NASA is using consumer VR headsets, game engine technology and NVIDIA's GPUs to create highly immersive future training systems augmented with extremely realistic haptic feedback, sound, additional sensory information, and how these can be used to improve the engineering workflow. Include in this presentation is an environment simulation of the ISS, where users can interact with virtual objects, handrails, and tracked physical objects while inside VR, integration of consumer VR headsets with the Active Response Gravity Offload System, and a space habitat architectural evaluation tool. Attendees will learn how the best elements of real and virtual worlds can be combined into a hybrid reality environment with tangible engineering and scientific applications.

Comparison of grasping movements made by healthy subjects in a 3-dimensional immersive virtual versus physical environment.

PubMed

Magdalon, Eliane C; Michaelsen, Stella M; Quevedo, Antonio A; Levin, Mindy F

2011-09-01

Virtual reality (VR) technology is being used with increasing frequency as a training medium for motor rehabilitation. However, before addressing training effectiveness in virtual environments (VEs), it is necessary to identify if movements made in such environments are kinematically similar to those made in physical environments (PEs) and the effect of provision of haptic feedback on these movement patterns. These questions are important since reach-to-grasp movements may be inaccurate when visual or haptic feedback is altered or absent. Our goal was to compare kinematics of reaching and grasping movements to three objects performed in an immersive three-dimensional (3D) VE with haptic feedback (cyberglove/grasp system) viewed through a head-mounted display to those made in an equivalent physical environment (PE). We also compared movements in PE made with and without wearing the cyberglove/grasp haptic feedback system. Ten healthy subjects (8 women, 62.1±8.8years) reached and grasped objects requiring 3 different grasp types (can, diameter 65.6mm, cylindrical grasp; screwdriver, diameter 31.6mm, power grasp; pen, diameter 7.5mm, precision grasp) in PE and visually similar virtual objects in VE. Temporal and spatial arm and trunk kinematics were analyzed. Movements were slower and grip apertures were wider when wearing the glove in both the PE and the VE compared to movements made in the PE without the glove. When wearing the glove, subjects used similar reaching trajectories in both environments, preserved the coordination between reaching and grasping and scaled grip aperture to object size for the larger object (cylindrical grasp). However, in VE compared to PE, movements were slower and had longer deceleration times, elbow extension was greater when reaching to the smallest object and apertures were wider for the power and precision grip tasks. Overall, the differences in spatial and temporal kinematics of movements between environments were greater than those due only to wearing the cyberglove/grasp system. Differences in movement kinematics due to the viewing environment were likely due to a lack of prior experience with the virtual environment, an uncertainty of object location and the restricted field-of-view when wearing the head-mounted display. The results can be used to inform the design and disposition of objects within 3D VEs for the study of the control of prehension and for upper limb rehabilitation. Copyright © 2011 Elsevier B.V. All rights reserved.

Using postural synergies to animate a low-dimensional hand avatar in haptic simulation.

PubMed

Mulatto, Sara; Formaglio, Alessandro; Malvezzi, Monica; Prattichizzo, Domenico

2013-01-01

A technique to animate a realistic hand avatar with 20 DoFs based on the biomechanics of the human hand is presented. The animation does not use any sensor glove or advanced tracker with markers. The proposed approach is based on the knowledge of a set of kinematic constraints on the model of the hand, referred to as postural synergies, which allows to represent the hand posture using a number of variables lower than the number of joints of the hand model. This low-dimensional set of parameters is estimated from direct measurement of the motion of thumb and index finger tracked using two haptic devices. A kinematic inversion algorithm has been developed, which takes synergies into account and estimates the kinematic configuration of the whole hand, i.e., also of the fingers whose end tips are not directly tracked by the two haptic devices. The hand skin is deformable and its deformation is computed using a linear vertex blending technique. The proposed synergy-based animation of the hand avatar involves only algebraic computations and is suitable for real-time implementation as required in haptics.

Adaptation of a haptic robot in a 3T fMRI.

PubMed

Snider, Joseph; Plank, Markus; May, Larry; Liu, Thomas T; Poizner, Howard

2011-10-04

Functional magnetic resonance imaging (fMRI) provides excellent functional brain imaging via the BOLD signal with advantages including non-ionizing radiation, millimeter spatial accuracy of anatomical and functional data, and nearly real-time analyses. Haptic robots provide precise measurement and control of position and force of a cursor in a reasonably confined space. Here we combine these two technologies to allow precision experiments involving motor control with haptic/tactile environment interaction such as reaching or grasping. The basic idea is to attach an 8 foot end effecter supported in the center to the robot allowing the subject to use the robot, but shielding it and keeping it out of the most extreme part of the magnetic field from the fMRI machine (Figure 1). The Phantom Premium 3.0, 6DoF, high-force robot (SensAble Technologies, Inc.) is an excellent choice for providing force-feedback in virtual reality experiments, but it is inherently non-MR safe, introduces significant noise to the sensitive fMRI equipment, and its electric motors may be affected by the fMRI's strongly varying magnetic field. We have constructed a table and shielding system that allows the robot to be safely introduced into the fMRI environment and limits both the degradation of the fMRI signal by the electrically noisy motors and the degradation of the electric motor performance by the strongly varying magnetic field of the fMRI. With the shield, the signal to noise ratio (SNR: mean signal/noise standard deviation) of the fMRI goes from a baseline of ~380 to ~330, and ~250 without the shielding. The remaining noise appears to be uncorrelated and does not add artifacts to the fMRI of a test sphere (Figure 2). The long, stiff handle allows placement of the robot out of range of the most strongly varying parts of the magnetic field so there is no significant effect of the fMRI on the robot. The effect of the handle on the robot's kinematics is minimal since it is lightweight (~2.6 lbs) but extremely stiff 3/4" graphite and well balanced on the 3DoF joint in the middle. The end result is an fMRI compatible, haptic system with about 1 cubic foot of working space, and, when combined with virtual reality, it allows for a new set of experiments to be performed in the fMRI environment including naturalistic reaching, passive displacement of the limb and haptic perception, adaptation learning in varying force fields, or texture identification.

Haptic-Based Neurorehabilitation in Poststroke Patients: A Feasibility Prospective Multicentre Trial for Robotics Hand Rehabilitation

PubMed Central

Daud Albasini, Omar A.; Oboe, Roberto; Tonin, Paolo; Paolucci, Stefano; Sandrini, Giorgio; Piron, Lamberto

2013-01-01

Background. Haptic robots allow the exploitation of known motor learning mechanisms, representing a valuable option for motor treatment after stroke. The aim of this feasibility multicentre study was to test the clinical efficacy of a haptic prototype, for the recovery of hand function after stroke. Methods. A prospective pilot clinical trial was planned on 15 consecutive patients enrolled in 3 rehabilitation centre in Italy. All the framework features of the haptic robot (e.g., control loop, external communication, and graphic rendering for virtual reality) were implemented into a real-time MATLAB/Simulink environment, controlling a five-bar linkage able to provide forces up to 20 [N] at the end effector, used for finger and hand rehabilitation therapies. Clinical (i.e., Fugl-Meyer upper extremity scale; nine hold pegboard test) and kinematics (i.e., time; velocity; jerk metric; normalized jerk of standard movements) outcomes were assessed before and after treatment to detect changes in patients' motor performance. Reorganization of cortical activation was detected in one patient by fMRI. Results and Conclusions. All patients showed significant improvements in both clinical and kinematic outcomes. Additionally, fMRI results suggest that the proposed approach may promote a better cortical activation in the brain. PMID:24319496

Haptic-based neurorehabilitation in poststroke patients: a feasibility prospective multicentre trial for robotics hand rehabilitation.

PubMed

Turolla, Andrea; Daud Albasini, Omar A; Oboe, Roberto; Agostini, Michela; Tonin, Paolo; Paolucci, Stefano; Sandrini, Giorgio; Venneri, Annalena; Piron, Lamberto

2013-01-01

Background. Haptic robots allow the exploitation of known motor learning mechanisms, representing a valuable option for motor treatment after stroke. The aim of this feasibility multicentre study was to test the clinical efficacy of a haptic prototype, for the recovery of hand function after stroke. Methods. A prospective pilot clinical trial was planned on 15 consecutive patients enrolled in 3 rehabilitation centre in Italy. All the framework features of the haptic robot (e.g., control loop, external communication, and graphic rendering for virtual reality) were implemented into a real-time MATLAB/Simulink environment, controlling a five-bar linkage able to provide forces up to 20 [N] at the end effector, used for finger and hand rehabilitation therapies. Clinical (i.e., Fugl-Meyer upper extremity scale; nine hold pegboard test) and kinematics (i.e., time; velocity; jerk metric; normalized jerk of standard movements) outcomes were assessed before and after treatment to detect changes in patients' motor performance. Reorganization of cortical activation was detected in one patient by fMRI. Results and Conclusions. All patients showed significant improvements in both clinical and kinematic outcomes. Additionally, fMRI results suggest that the proposed approach may promote a better cortical activation in the brain.

Haptic perception and body representation in lateral and medial occipito-temporal cortices.

PubMed

Costantini, Marcello; Urgesi, Cosimo; Galati, Gaspare; Romani, Gian Luca; Aglioti, Salvatore M

2011-04-01

Although vision is the primary sensory modality that humans and other primates use to identify objects in the environment, we can recognize crucial object features (e.g., shape, size) using the somatic modality. Previous studies have shown that the occipito-temporal areas dedicated to the visual processing of object forms, faces and bodies also show category-selective responses when the preferred stimuli are haptically explored out of view. Visual processing of human bodies engages specific areas in lateral (extrastriate body area, EBA) and medial (fusiform body area, FBA) occipito-temporal cortex. This study aimed at exploring the relative involvement of EBA and FBA in the haptic exploration of body parts. During fMRI scanning, participants were asked to haptically explore either real-size fake body parts or objects. We found a selective activation of right and left EBA, but not of right FBA, while participants haptically explored body parts as compared to real objects. This suggests that EBA may integrate visual body representations with somatosensory information regarding body parts and form a multimodal representation of the body. Furthermore, both left and right EBA showed a comparable level of body selectivity during haptic perception and visual imagery. However, right but not left EBA was more activated during haptic exploration than visual imagery of body parts, ruling out that the response to haptic body exploration was entirely due to the use of visual imagery. Overall, the results point to the existence of different multimodal body representations in the occipito-temporal cortex which are activated during perception and imagery of human body parts. Copyright © 2011 Elsevier Ltd. All rights reserved.

Improving Challenge/Skill Ratio in a Multimodal Interface by Simultaneously Adapting Game Difficulty and Haptic Assistance through Psychophysiological and Performance Feedback

PubMed Central

Rodriguez-Guerrero, Carlos; Knaepen, Kristel; Fraile-Marinero, Juan C.; Perez-Turiel, Javier; Gonzalez-de-Garibay, Valentin; Lefeber, Dirk

2017-01-01

In order to harmonize robotic devices with human beings, the robots should be able to perceive important psychosomatic impact triggered by emotional states such as frustration or boredom. This paper presents a new type of biocooperative control architecture, which acts toward improving the challenge/skill relation perceived by the user when interacting with a robotic multimodal interface in a cooperative scenario. In the first part of the paper, open-loop experiments revealed which physiological signals were optimal for inclusion in the feedback loop. These were heart rate, skin conductance level, and skin conductance response frequency. In the second part of the paper, the proposed controller, consisting of a biocooperative architecture with two degrees of freedom, simultaneously modulating game difficulty and haptic assistance through performance and psychophysiological feedback, is presented. With this setup, the perceived challenge can be modulated by means of the game difficulty and the perceived skill by means of the haptic assistance. A new metric (FlowIndex) is proposed to numerically quantify and visualize the challenge/skill relation. The results are contrasted with comparable previously published work and show that the new method afforded a higher FlowIndex (i.e., a superior challenge/skill relation) and an improved balance between augmented performance and user satisfaction (higher level of valence, i.e., a more enjoyable and satisfactory experience). PMID:28507503

A Study on Immersion and Presence of a Portable Hand Haptic System for Immersive Virtual Reality

PubMed Central

Kim, Mingyu; Jeon, Changyu; Kim, Jinmo

2017-01-01

This paper proposes a portable hand haptic system using Leap Motion as a haptic interface that can be used in various virtual reality (VR) applications. The proposed hand haptic system was designed as an Arduino-based sensor architecture to enable a variety of tactile senses at low cost, and is also equipped with a portable wristband. As a haptic system designed for tactile feedback, the proposed system first identifies the left and right hands and then sends tactile senses (vibration and heat) to each fingertip (thumb and index finger). It is incorporated into a wearable band-type system, making its use easy and convenient. Next, hand motion is accurately captured using the sensor of the hand tracking system and is used for virtual object control, thus achieving interaction that enhances immersion. A VR application was designed with the purpose of testing the immersion and presence aspects of the proposed system. Lastly, technical and statistical tests were carried out to assess whether the proposed haptic system can provide a new immersive presence to users. According to the results of the presence questionnaire and the simulator sickness questionnaire, we confirmed that the proposed hand haptic system, in comparison to the existing interaction that uses only the hand tracking system, provided greater presence and a more immersive environment in the virtual reality. PMID:28513545

A Study on Immersion and Presence of a Portable Hand Haptic System for Immersive Virtual Reality.

PubMed

Kim, Mingyu; Jeon, Changyu; Kim, Jinmo

2017-05-17

This paper proposes a portable hand haptic system using Leap Motion as a haptic interface that can be used in various virtual reality (VR) applications. The proposed hand haptic system was designed as an Arduino-based sensor architecture to enable a variety of tactile senses at low cost, and is also equipped with a portable wristband. As a haptic system designed for tactile feedback, the proposed system first identifies the left and right hands and then sends tactile senses (vibration and heat) to each fingertip (thumb and index finger). It is incorporated into a wearable band-type system, making its use easy and convenient. Next, hand motion is accurately captured using the sensor of the hand tracking system and is used for virtual object control, thus achieving interaction that enhances immersion. A VR application was designed with the purpose of testing the immersion and presence aspects of the proposed system. Lastly, technical and statistical tests were carried out to assess whether the proposed haptic system can provide a new immersive presence to users. According to the results of the presence questionnaire and the simulator sickness questionnaire, we confirmed that the proposed hand haptic system, in comparison to the existing interaction that uses only the hand tracking system, provided greater presence and a more immersive environment in the virtual reality.

Human Haptic Interaction with Soft Objects: Discriminability, Force Control, and Contact Visualization

DTIC Science & Technology

1998-01-01

consisted of a videomicroscopy system and a tactile stimulator system. By using this setup, real-time images from the contact region as wvell as the... Videomicroscopy system . 4.3.2 Tactile stimulator svsteln . 4.3.3 Real-time imaging setup. 4.3.4 Active and passive touch experiments. 4.3.5...contact process is an important step. In this study, therefore, a videomicroscopy system was built’to visualize the contact re- gion of the fingerpad

Human haptic perception is interrupted by explorative stops of milliseconds

PubMed Central

Grunwald, Martin; Muniyandi, Manivannan; Kim, Hyun; Kim, Jung; Krause, Frank; Mueller, Stephanie; Srinivasan, Mandayam A.

2014-01-01

Introduction: The explorative scanning movements of the hands have been compared to those of the eyes. The visual process is known to be composed of alternating phases of saccadic eye movements and fixation pauses. Descriptive results suggest that during the haptic exploration of objects short movement pauses occur as well. The goal of the present study was to detect these “explorative stops” (ES) during one-handed and two-handed haptic explorations of various objects and patterns, and to measure their duration. Additionally, the associations between the following variables were analyzed: (a) between mean exploration time and duration of ES, (b) between certain stimulus features and ES frequency, and (c) the duration of ES during the course of exploration. Methods: Five different Experiments were used. The first two Experiments were classical recognition tasks of unknown haptic stimuli (A) and of common objects (B). In Experiment C space-position information of angle legs had to be perceived and reproduced. For Experiments D and E the PHANToM haptic device was used for the exploration of virtual (D) and real (E) sunken reliefs. Results: In each Experiment we observed explorative stops of different average durations. For Experiment A: 329.50 ms, Experiment B: 67.47 ms, Experiment C: 189.92 ms, Experiment D: 186.17 ms and Experiment E: 140.02 ms. Significant correlations were observed between exploration time and the duration of the ES. Also, ES occurred more frequently, but not exclusively, at defined stimulus features like corners, curves and the endpoints of lines. However, explorative stops do not occur every time a stimulus feature is explored. Conclusions: We assume that ES are a general aspect of human haptic exploration processes. We have tried to interpret the occurrence and duration of ES with respect to the Hypotheses-Rebuild-Model and the Limited Capacity Control System theory. PMID:24782797

Haptic-based perception-empathy biofeedback system for balance rehabilitation in patients with chronic stroke: Concepts and initial feasibility study.

PubMed

Yasuda, Kazuhiro; Saichi, Kenta; Kaibuki, Naomi; Harashima, Hiroaki; Iwata, Hiroyasu

2018-05-01

Most individuals have sensory disturbances post stroke, and these deficits contribute to post-stroke balance impairment. The haptic-based biofeedback (BF) system appears to be one of the promising tools for balance rehabilitation in patients with stroke, and the BF system can increase the objectivity of feedback and encouragement than that provided by a therapist. Studies in skill science indicated that feedback or encouragement from a coach or trainer enhances motor learning effect. Nevertheless, the optimal BF system (or its concept) which would refine the interpersonal feedback between patients and therapist has not been proposed. Thus, the purpose of this study was to propose a haptic-based perception-empathy BF system which provides information regarding the patient's center-of-foot pressure (CoP) pattern to the patient and the physical therapist to enhance the motor learning effect and validate the feasibility of this balance-training regimen in patients with chronic stroke. This study used a pre-post design without control group. Nine chronic stroke patients (mean age: 64.4 ± 9.2 years) received a balance-training regimen using this BF system twice a week for 4 weeks. Testing comprised quantitative measures (i.e., CoP) and clinical balance scale (Berg Balance Scale, BBS; Functional Reach Test, FRT; and Timed-Up and Go test, TUG). Post training, patients demonstrated marginally reduced postural spatial variability (i.e., 95% confidence elliptical area), and clinical balance performance significantly improved at post-training. Although the changes in FRT and TUG exceeded the minimal detectable change (MDC), changes in BBS did not reach clinical significance (i.e., smaller than MDC). These results may provide initial knowledge (i.e., beneficial effects, utility and its limitation) of the proposed BF system in designing effective motor learning strategies for stroke rehabilitation. More studies are required addressing limitations due to research design and training method for future clinical use. Copyright © 2018 Elsevier B.V. All rights reserved.

Validation and learning in the Procedicus KSA virtual reality surgical simulator.

PubMed

Ström, P; Kjellin, A; Hedman, L; Johnson, E; Wredmark, T; Felländer-Tsai, L

2003-02-01

Advanced simulator training within medicine is a rapidly growing field. Virtual reality simulators are being introduced as cost-saving educational tools, which also lead to increased patient safety. Fifteen medical students were included in the study. For 10 medical students performance was monitored, before and after 1 h of training, in two endoscopic simulators (the Procedicus KSA with haptic feedback and anatomical graphics and the established MIST simulator without this haptic feedback and graphics). Five medical students performed 50 tests in the Procedicus KSA in order to analyze learning curves. One of these five medical students performed multiple training sessions during 2 weeks and performed more than 300 tests. There was a significant improvement after 1 h of training regarding time, movement economy, and total score. The results in the two simulators were highly correlated. Our results show that the use of surgical simulators as a pedagogical tool in medical student training is encouraging. It shows rapid learning curves and our suggestion is to introduce endoscopic simulator training in undergraduate medical education during the course in surgery when motivation is high and before the development of "negative stereotypes" and incorrect practices.

An augmented reality haptic training simulator for spinal needle procedures.

PubMed

Sutherland, Colin; Hashtrudi-Zaad, Keyvan; Sellens, Rick; Abolmaesumi, Purang; Mousavi, Parvin

2013-11-01

This paper presents the prototype for an augmented reality haptic simulation system with potential for spinal needle insertion training. The proposed system is composed of a torso mannequin, a MicronTracker2 optical tracking system, a PHANToM haptic device, and a graphical user interface to provide visual feedback. The system allows users to perform simulated needle insertions on a physical mannequin overlaid with an augmented reality cutaway of patient anatomy. A tissue model based on a finite-element model provides force during the insertion. The system allows for training without the need for the presence of a trained clinician or access to live patients or cadavers. A pilot user study demonstrates the potential and functionality of the system.

Visual-haptic integration with pliers and tongs: signal “weights” take account of changes in haptic sensitivity caused by different tools

PubMed Central

Takahashi, Chie; Watt, Simon J.

2014-01-01

When we hold an object while looking at it, estimates from visual and haptic cues to size are combined in a statistically optimal fashion, whereby the “weight” given to each signal reflects their relative reliabilities. This allows object properties to be estimated more precisely than would otherwise be possible. Tools such as pliers and tongs systematically perturb the mapping between object size and the hand opening. This could complicate visual-haptic integration because it may alter the reliability of the haptic signal, thereby disrupting the determination of appropriate signal weights. To investigate this we first measured the reliability of haptic size estimates made with virtual pliers-like tools (created using a stereoscopic display and force-feedback robots) with different “gains” between hand opening and object size. Haptic reliability in tool use was straightforwardly determined by a combination of sensitivity to changes in hand opening and the effects of tool geometry. The precise pattern of sensitivity to hand opening, which violated Weber's law, meant that haptic reliability changed with tool gain. We then examined whether the visuo-motor system accounts for these reliability changes. We measured the weight given to visual and haptic stimuli when both were available, again with different tool gains, by measuring the perceived size of stimuli in which visual and haptic sizes were varied independently. The weight given to each sensory cue changed with tool gain in a manner that closely resembled the predictions of optimal sensory integration. The results are consistent with the idea that different tool geometries are modeled by the brain, allowing it to calculate not only the distal properties of objects felt with tools, but also the certainty with which those properties are known. These findings highlight the flexibility of human sensory integration and tool-use, and potentially provide an approach for optimizing the design of visual-haptic devices. PMID:24592245

Virtual reality simulation: basic concepts and use in endoscopic neurosurgery training.

PubMed

Cohen, Alan R; Lohani, Subash; Manjila, Sunil; Natsupakpong, Suriya; Brown, Nathan; Cavusoglu, M Cenk

2013-08-01

Virtual reality simulation is a promising alternative to training surgical residents outside the operating room. It is also a useful aide to anatomic study, residency training, surgical rehearsal, credentialing, and recertification. Surgical simulation is based on a virtual reality with varying degrees of immersion and realism. Simulators provide a no-risk environment for harmless and repeatable practice. Virtual reality has three main components of simulation: graphics/volume rendering, model behavior/tissue deformation, and haptic feedback. The challenge of accurately simulating the forces and tactile sensations experienced in neurosurgery limits the sophistication of a virtual simulator. The limited haptic feedback available in minimally invasive neurosurgery makes it a favorable subject for simulation. Virtual simulators with realistic graphics and force feedback have been developed for ventriculostomy, intraventricular surgery, and transsphenoidal pituitary surgery, thus allowing preoperative study of the individual anatomy and increasing the safety of the procedure. The authors also present experiences with their own virtual simulation of endoscopic third ventriculostomy.

Adaptive space warping to enhance passive haptics in an arthroscopy surgical simulator.

PubMed

Spillmann, Jonas; Tuchschmid, Stefan; Harders, Matthias

2013-04-01

Passive haptics, also known as tactile augmentation, denotes the use of a physical counterpart to a virtual environment to provide tactile feedback. Employing passive haptics can result in more realistic touch sensations than those from active force feedback, especially for rigid contacts. However, changes in the virtual environment would necessitate modifications of the physical counterparts. In recent work space warping has been proposed as one solution to overcome this limitation. In this technique virtual space is distorted such that a variety of virtual models can be mapped onto one single physical object. In this paper, we propose as an extension adaptive space warping; we show how this technique can be employed in a mixed-reality surgical training simulator in order to map different virtual patients onto one physical anatomical model. We developed methods to warp different organ geometries onto one physical mock-up, to handle different mechanical behaviors of the virtual patients, and to allow interactive modifications of the virtual structures, while the physical counterparts remain unchanged. Various practical examples underline the wide applicability of our approach. To the best of our knowledge this is the first practical usage of such a technique in the specific context of interactive medical training.

A haptic unit designed for magnetic-resonance-guided biopsy.

PubMed

Tse, Z T H; Elhawary, H; Rea, M; Young, I; Davis, B L; Lamperth, M

2009-02-01

The magnetic fields present in the magnetic resonance (MR) environment impose severe constraints on any mechatronic device present in its midst, requiring alternative actuators, sensors, and materials to those conventionally used in traditional system engineering. In addition the spatial constraints of closed-bore scanners require a physical separation between the radiologist and the imaged region of the patient. This configuration produces a loss of the sense of touch from the target anatomy for the clinician, which often provides useful information. To recover the force feedback from the tissue, an MR-compatible haptic unit, designed to be integrated with a five-degrees-of-freedom mechatronic system for MR-guided prostate biopsy, has been developed which incorporates position control and force feedback to the operator. The haptic unit is designed to be located inside the scanner isocentre with the master console in the control room. MR compatibility of the device has been demonstrated, showing a negligible degradation of the signal-to-noise ratio and virtually no geometric distortion. By combining information from the position encoder and force sensor, tissue stiffness measurement along the needle trajectory is demonstrated in a lamb liver to aid diagnosis of suspected cancerous tissue.

Virtual-Reality Simulator System for Double Interventional Cardiac Catheterization Using Fractional-Order Vascular Access Tracker and Haptic Force Producer

PubMed Central

Chen, Guan-Chun; Lin, Chia-Hung; Hsieh, Kai-Sheng; Du, Yi-Chun; Chen, Tainsong

2015-01-01

This study proposes virtual-reality (VR) simulator system for double interventional cardiac catheterization (ICC) using fractional-order vascular access tracker and haptic force producer. An endoscope or a catheter for diagnosis and surgery of cardiovascular disease has been commonly used in minimally invasive surgery. It needs specific skills and experiences for young surgeons or postgraduate year (PGY) students to operate a Berman catheter and a pigtail catheter in the inside of the human body and requires avoiding damaging vessels. To improve the training in inserting catheters, a double-catheter mechanism is designed for the ICC procedures. A fractional-order vascular access tracker is used to trace the senior surgeons' consoled trajectories and transmit the frictional feedback and visual feedback during the insertion of catheters. Based on the clinical feeling through the aortic arch, vein into the ventricle, or tortuous blood vessels, haptic force producer is used to mock the elasticity of the vessel wall using voice coil motors (VCMs). The VR establishment with surgeons' consoled vessel trajectories and hand feeling is achieved, and the experimental results show the effectiveness for the double ICC procedures. PMID:26171419

Training haptic stiffness discrimination: time course of learning with or without visual information and knowledge of results.

PubMed

Teodorescu, Kinneret; Bouchigny, Sylvain; Korman, Maria

2013-08-01

In this study, we explored the time course of haptic stiffness discrimination learning and how it was affected by two experimental factors, the addition of visual information and/or knowledge of results (KR) during training. Stiffness perception may integrate both haptic and visual modalities. However, in many tasks, the visual field is typically occluded, forcing stiffness perception to be dependent exclusively on haptic information. No studies to date addressed the time course of haptic stiffness perceptual learning. Using a virtual environment (VE) haptic interface and a two-alternative forced-choice discrimination task, the haptic stiffness discrimination ability of 48 participants was tested across 2 days. Each day included two haptic test blocks separated by a training block Additional visual information and/or KR were manipulated between participants during training blocks. Practice repetitions alone induced significant improvement in haptic stiffness discrimination. Between days, accuracy was slightly improved, but decision time performance was deteriorated. The addition of visual information and/or KR had only temporary effects on decision time, without affecting the time course of haptic discrimination learning. Learning in haptic stiffness discrimination appears to evolve through at least two distinctive phases: A single training session resulted in both immediate and latent learning. This learning was not affected by the training manipulations inspected. Training skills in VE in spaced sessions can be beneficial for tasks in which haptic perception is critical, such as surgery procedures, when the visual field is occluded. However, training protocols for such tasks should account for low impact of multisensory information and KR.

Innovative approaches to the rehabilitation of upper extremity hemiparesis using virtual environments

PubMed Central

MERIANS, A. S.; TUNIK, E.; FLUET, G. G.; QIU, Q.; ADAMOVICH, S. V.

2017-01-01

Aim Upper-extremity interventions for hemiparesis are a challenging aspect of stroke rehabilitation. Purpose of this paper is to report the feasibility of using virtual environments (VEs) in combination with robotics to assist recovery of hand-arm function and to present preliminary data demonstrating the potential of using sensory manipulations in VE to drive activation in targeted neural regions. Methods We trained 8 subjects for 8 three hour sessions using a library of complex VE’s integrated with robots, comparing training arm and hand separately to training arm and hand together. Instrumented gloves and hand exoskeleton were used for hand tracking and haptic effects. Haptic Master robotic arm was used for arm tracking and generating three-dimensional haptic VEs. To investigate the use of manipulations in VE to drive neural activations, we created a “virtual mirror” that subjects used while performing a unimanual task. Cortical activation was measured with functional MRI (fMRI) and transcranial magnetic stimulation. Results Both groups showed improvement in kinematics and measures of real-world function. The group trained using their arm and hand together showed greater improvement. In a stroke subject, fMRI data suggested virtual mirror feedback could activate the sensorimotor cortex contralateral to the reflected hand (ipsilateral to the moving hand) thus recruiting the lesioned hemisphere. Conclusion Gaming simulations interfaced with robotic devices provide a training medium that can modify movement patterns. In addition to showing that our VE therapies can optimize behavioral performance, we show preliminary evidence to support the potential of using specific sensory manipulations to selectively recruit targeted neural circuits. PMID:19158659

Robotics in Colorectal Surgery

PubMed Central

Weaver, Allison; Steele, Scott

2016-01-01

Over the past few decades, robotic surgery has developed from a futuristic dream to a real, widely used technology. Today, robotic platforms are used for a range of procedures and have added a new facet to the development and implementation of minimally invasive surgeries. The potential advantages are enormous, but the current progress is impeded by high costs and limited technology. However, recent advances in haptic feedback systems and single-port surgical techniques demonstrate a clear role for robotics and are likely to improve surgical outcomes. Although robotic surgeries have become the gold standard for a number of procedures, the research in colorectal surgery is not definitive and more work needs to be done to prove its safety and efficacy to both surgeons and patients. PMID:27746895

Design and Calibration of a New 6 DOF Haptic Device

PubMed Central

Qin, Huanhuan; Song, Aiguo; Liu, Yuqing; Jiang, Guohua; Zhou, Bohe

2015-01-01

For many applications such as tele-operational robots and interactions with virtual environments, it is better to have performance with force feedback than without. Haptic devices are force reflecting interfaces. They can also track human hand positions simultaneously. A new 6 DOF (degree-of-freedom) haptic device was designed and calibrated in this study. It mainly contains a double parallel linkage, a rhombus linkage, a rotating mechanical structure and a grasping interface. Benefited from the unique design, it is a hybrid structure device with a large workspace and high output capability. Therefore, it is capable of multi-finger interactions. Moreover, with an adjustable base, operators can change different postures without interrupting haptic tasks. To investigate the performance regarding position tracking accuracy and static output forces, we conducted experiments on a three-dimensional electric sliding platform and a digital force gauge, respectively. Displacement errors and force errors are calculated and analyzed. To identify the capability and potential of the device, four application examples were programmed. PMID:26690449

Development of haptic system for surgical robot

NASA Astrophysics Data System (ADS)

Gang, Han Gyeol; Park, Jiong Min; Choi, Seung-Bok; Sohn, Jung Woo

2017-04-01

In this paper, a new type of haptic system for surgical robot application is proposed and its performances are evaluated experimentally. The proposed haptic system consists of an effective master device and a precision slave robot. The master device has 3-DOF rotational motion as same as human wrist motion. It has lightweight structure with a gyro sensor and three small-sized MR brakes for position measurement and repulsive torque generation, respectively. The slave robot has 3-DOF rotational motion using servomotors, five bar linkage and a torque sensor is used to measure resistive torque. It has been experimentally demonstrated that the proposed haptic system has good performances on tracking control of desired position and repulsive torque. It can be concluded that the proposed haptic system can be effectively applied to the surgical robot system in real field.

A brain-computer interface with vibrotactile biofeedback for haptic information.

PubMed

Chatterjee, Aniruddha; Aggarwal, Vikram; Ramos, Ander; Acharya, Soumyadipta; Thakor, Nitish V

2007-10-17

It has been suggested that Brain-Computer Interfaces (BCI) may one day be suitable for controlling a neuroprosthesis. For closed-loop operation of BCI, a tactile feedback channel that is compatible with neuroprosthetic applications is desired. Operation of an EEG-based BCI using only vibrotactile feedback, a commonly used method to convey haptic senses of contact and pressure, is demonstrated with a high level of accuracy. A Mu-rhythm based BCI using a motor imagery paradigm was used to control the position of a virtual cursor. The cursor position was shown visually as well as transmitted haptically by modulating the intensity of a vibrotactile stimulus to the upper limb. A total of six subjects operated the BCI in a two-stage targeting task, receiving only vibrotactile biofeedback of performance. The location of the vibration was also systematically varied between the left and right arms to investigate location-dependent effects on performance. Subjects are able to control the BCI using only vibrotactile feedback with an average accuracy of 56% and as high as 72%. These accuracies are significantly higher than the 15% predicted by random chance if the subject had no voluntary control of their Mu-rhythm. The results of this study demonstrate that vibrotactile feedback is an effective biofeedback modality to operate a BCI using motor imagery. In addition, the study shows that placement of the vibrotactile stimulation on the biceps ipsilateral or contralateral to the motor imagery introduces a significant bias in the BCI accuracy. This bias is consistent with a drop in performance generated by stimulation of the contralateral limb. Users demonstrated the capability to overcome this bias with training.

New tools for sculpting cranial implants in a shared haptic augmented reality environment.

PubMed

Ai, Zhuming; Evenhouse, Ray; Leigh, Jason; Charbel, Fady; Rasmussen, Mary

2006-01-01

New volumetric tools were developed for the design and fabrication of high quality cranial implants from patient CT data. These virtual tools replace time consuming physical sculpting, mold making and casting steps. The implant is designed by medical professionals in tele-immersive collaboration. Virtual clay is added in the virtual defect area on the CT data using the adding tool. With force feedback the modeler can feel the edge of the defect and fill only the space where no bone is present. A carving tool and a smoothing tool are then used to sculpt and refine the implant. To make a physical evaluation, the skull with simulated defect and the implant are fabricated via stereolithography to allow neurosurgeons to evaluate the quality of the implant. Initial tests demonstrate a very high quality fit. These new haptic volumetric sculpting tools are a critical component of a comprehensive tele-immersive system.

Importance of Matching Physical Friction, Hardness, and Texture in Creating Realistic Haptic Virtual Surfaces.

PubMed

Culbertson, Heather; Kuchenbecker, Katherine J

2017-01-01

Interacting with physical objects through a tool elicits tactile and kinesthetic sensations that comprise your haptic impression of the object. These cues, however, are largely missing from interactions with virtual objects, yielding an unrealistic user experience. This article evaluates the realism of virtual surfaces rendered using haptic models constructed from data recorded during interactions with real surfaces. The models include three components: surface friction, tapping transients, and texture vibrations. We render the virtual surfaces on a SensAble Phantom Omni haptic interface augmented with a Tactile Labs Haptuator for vibration output. We conducted a human-subject study to assess the realism of these virtual surfaces and the importance of the three model components. Following a perceptual discrepancy paradigm, subjects compared each of 15 real surfaces to a full rendering of the same surface plus versions missing each model component. The realism improvement achieved by including friction, tapping, or texture in the rendering was found to directly relate to the intensity of the surface's property in that domain (slipperiness, hardness, or roughness). A subsequent analysis of forces and vibrations measured during interactions with virtual surfaces indicated that the Omni's inherent mechanical properties corrupted the user's haptic experience, decreasing realism of the virtual surface.

Identification of Vibrotactile Patterns Encoding Obstacle Distance Information.

PubMed

Kim, Yeongmi; Harders, Matthias; Gassert, Roger

2015-01-01

Delivering distance information of nearby obstacles from sensors embedded in a white cane-in addition to the intrinsic mechanical feedback from the cane-can aid the visually impaired in ambulating independently. Haptics is a common modality for conveying such information to cane users, typically in the form of vibrotactile signals. In this context, we investigated the effect of tactile rendering methods, tactile feedback configurations and directions of tactile flow on the identification of obstacle distance. Three tactile rendering methods with temporal variation only, spatio-temporal variation and spatial/temporal/intensity variation were investigated for two vibration feedback configurations. Results showed a significant interaction between tactile rendering method and feedback configuration. Spatio-temporal variation generally resulted in high correct identification rates for both feedback configurations. In the case of the four-finger vibration, tactile rendering with spatial/temporal/intensity variation also resulted in high distance identification rate. Further, participants expressed their preference for the four-finger vibration over the single-finger vibration in a survey. Both preferred rendering methods with spatio-temporal variation and spatial/temporal/intensity variation for the four-finger vibration could convey obstacle distance information with low workload. Overall, the presented findings provide valuable insights and guidance for the design of haptic displays for electronic travel aids for the visually impaired.

Assimilation of virtual legs and perception of floor texture by complete paraplegic patients receiving artificial tactile feedback.

PubMed

Shokur, Solaiman; Gallo, Simone; Moioli, Renan C; Donati, Ana Rita C; Morya, Edgard; Bleuler, Hannes; Nicolelis, Miguel A L

2016-09-19

Spinal cord injuries disrupt bidirectional communication between the patient's brain and body. Here, we demonstrate a new approach for reproducing lower limb somatosensory feedback in paraplegics by remapping missing leg/foot tactile sensations onto the skin of patients' forearms. A portable haptic display was tested in eight patients in a setup where the lower limbs were simulated using immersive virtual reality (VR). For six out of eight patients, the haptic display induced the realistic illusion of walking on three different types of floor surfaces: beach sand, a paved street or grass. Additionally, patients experienced the movements of the virtual legs during the swing phase or the sensation of the foot rolling on the floor while walking. Relying solely on this tactile feedback, patients reported the position of the avatar leg during virtual walking. Crossmodal interference between vision of the virtual legs and tactile feedback revealed that patients assimilated the virtual lower limbs as if they were their own legs. We propose that the addition of tactile feedback to neuroprosthetic devices is essential to restore a full lower limb perceptual experience in spinal cord injury (SCI) patients, and will ultimately, lead to a higher rate of prosthetic acceptance/use and a better level of motor proficiency.

Assimilation of virtual legs and perception of floor texture by complete paraplegic patients receiving artificial tactile feedback

PubMed Central

Shokur, Solaiman; Gallo, Simone; Moioli, Renan C.; Donati, Ana Rita C.; Morya, Edgard; Bleuler, Hannes; Nicolelis, Miguel A.L.

2016-01-01

Spinal cord injuries disrupt bidirectional communication between the patient’s brain and body. Here, we demonstrate a new approach for reproducing lower limb somatosensory feedback in paraplegics by remapping missing leg/foot tactile sensations onto the skin of patients’ forearms. A portable haptic display was tested in eight patients in a setup where the lower limbs were simulated using immersive virtual reality (VR). For six out of eight patients, the haptic display induced the realistic illusion of walking on three different types of floor surfaces: beach sand, a paved street or grass. Additionally, patients experienced the movements of the virtual legs during the swing phase or the sensation of the foot rolling on the floor while walking. Relying solely on this tactile feedback, patients reported the position of the avatar leg during virtual walking. Crossmodal interference between vision of the virtual legs and tactile feedback revealed that patients assimilated the virtual lower limbs as if they were their own legs. We propose that the addition of tactile feedback to neuroprosthetic devices is essential to restore a full lower limb perceptual experience in spinal cord injury (SCI) patients, and will ultimately, lead to a higher rate of prosthetic acceptance/use and a better level of motor proficiency. PMID:27640345

Force Sensitive Handles and Capacitive Touch Sensor for Driving a Flexible Haptic-Based Immersive System

PubMed Central

Covarrubias, Mario; Bordegoni, Monica; Cugini, Umberto

2013-01-01

In this article, we present an approach that uses both two force sensitive handles (FSH) and a flexible capacitive touch sensor (FCTS) to drive a haptic-based immersive system. The immersive system has been developed as part of a multimodal interface for product design. The haptic interface consists of a strip that can be used by product designers to evaluate the quality of a 3D virtual shape by using touch, vision and hearing and, also, to interactively change the shape of the virtual object. Specifically, the user interacts with the FSH to move the virtual object and to appropriately position the haptic interface for retrieving the six degrees of freedom required for both manipulation and modification modalities. The FCTS allows the system to track the movement and position of the user's fingers on the strip, which is used for rendering visual and sound feedback. Two evaluation experiments are described, which involve both the evaluation and the modification of a 3D shape. Results show that the use of the haptic strip for the evaluation of aesthetic shapes is effective and supports product designers in the appreciation of the aesthetic qualities of the shape. PMID:24113680

Force sensitive handles and capacitive touch sensor for driving a flexible haptic-based immersive system.

PubMed

Covarrubias, Mario; Bordegoni, Monica; Cugini, Umberto

2013-10-09

In this article, we present an approach that uses both two force sensitive handles (FSH) and a flexible capacitive touch sensor (FCTS) to drive a haptic-based immersive system. The immersive system has been developed as part of a multimodal interface for product design. The haptic interface consists of a strip that can be used by product designers to evaluate the quality of a 3D virtual shape by using touch, vision and hearing and, also, to interactively change the shape of the virtual object. Specifically, the user interacts with the FSH to move the virtual object and to appropriately position the haptic interface for retrieving the six degrees of freedom required for both manipulation and modification modalities. The FCTS allows the system to track the movement and position of the user's fingers on the strip, which is used for rendering visual and sound feedback. Two evaluation experiments are described, which involve both the evaluation and the modification of a 3D shape. Results show that the use of the haptic strip for the evaluation of aesthetic shapes is effective and supports product designers in the appreciation of the aesthetic qualities of the shape.

Limited value of haptics in virtual reality laparoscopic cholecystectomy training.

PubMed

Thompson, Jonathan R; Leonard, Anthony C; Doarn, Charles R; Roesch, Matt J; Broderick, Timothy J

2011-04-01

Haptics is an expensive addition to virtual reality (VR) simulators, and the added value to training has not been proven. This study evaluated the benefit of haptics in VR laparoscopic surgery training for novices. The Simbionix LapMentor II haptic VR simulator was used in the study. Randomly, 33 laparoscopic novice students were placed in one of three groups: control, haptics-trained, or nonhaptics-trained group. The control group performed nine basic laparoscopy tasks and four cholecystectomy procedural tasks one time with haptics engaged at the default setting. The haptics group was trained to proficiency in the basic tasks and then performed each of the procedural tasks one time with haptics engaged. The nonhaptics group used the same training protocol except that haptics was disengaged. The proficiency values used were previously published expert values. Each group was assessed in the performance of 10 laparoscopic cholecystectomies (alternating with and without haptics). Performance was measured via automatically collected simulator data. The three groups exhibited no differences in terms of sex, education level, hand dominance, video game experience, surgical experience, and nonsurgical simulator experience. The number of attempts required to reach proficiency did not differ between the haptics- and nonhaptics-training groups. The haptics and nonhaptics groups exhibited no difference in performance. Both training groups outperformed the control group in number of movements as well as path length of the left instrument. In addition, the nonhaptics group outperformed the control group in total time. Haptics does not improve the efficiency or effectiveness of LapMentor II VR laparoscopic surgery training. The limited benefit and the significant cost of haptics suggest that haptics should not be included routinely in VR laparoscopic surgery training.

A flexible tactile-feedback touch screen using transparent ferroelectric polymer film vibrators

NASA Astrophysics Data System (ADS)

Ju, Woo-Eon; Moon, Yong-Ju; Park, Cheon-Ho; Choi, Seung Tae

2014-07-01

To provide tactile feedback on flexible touch screens, transparent relaxor ferroelectric polymer film vibrators were designed and fabricated in this study. The film vibrator can be integrated underneath a transparent cover film or glass, and can also produce acoustic waves that cause a tactile sensation on human fingertips. Poly(vinylidene fluoride-trifluoroethylene-chlorotrifluoroethylene) [P(VDF-TrFE-CTFE)] polymer was used as the relaxor ferroelectric polymer because it produces a large strain under applied electric fields, shows a fast response, and has excellent optical transparency. The natural frequency of this tactile-feedback touch screen was designed to be around 200-240 Hz, at which the haptic perception of human fingertips is the most sensitive; therefore, the resonance of the touch screen at its natural frequency provides maximum haptic sensation. A multilayered relaxor ferroelectric polymer film vibrator was also demonstrated to provide the same vibration power at reduced voltage. The flexible P(VDF-TrFE-CTFE) film vibrators developed in this study are expected to provide tactile sensation not only in large-area flat panel displays, but also in flexible displays and touch screens.

Using haptic feedback to increase seat belt use : traffic tech.

DOT National Transportation Integrated Search

2011-07-01

The legacy of research on increasing seat belt use has : focused on enactment of seat belt legislation, public education, : high-visibility police enforcement, and seat belt : reminder systems. Several behavioral programs have : produced large, susta...

Emotion Telepresence: Emotion Augmentation through Affective Haptics and Visual Stimuli

NASA Astrophysics Data System (ADS)

Tsetserukou, D.; Neviarouskaya, A.

2012-03-01

The paper focuses on a novel concept of emotional telepresence. The iFeel_IM! system which is in the vanguard of this technology integrates 3D virtual world Second Life, intelligent component for automatic emotion recognition from text messages, and innovative affective haptic interfaces providing additional nonverbal communication channels through simulation of emotional feedback and social touch (physical co-presence). Users can not only exchange messages but also emotionally and physically feel the presence of the communication partner (e.g., family member, friend, or beloved person). The next prototype of the system will include the tablet computer. The user can realize haptic interaction with avatar, and thus influence its mood and emotion of the partner. The finger gesture language will be designed for communication with avatar. This will bring new level of immersion of on-line communication.

Smart glove: hand master using magnetorheological fluid actuators

NASA Astrophysics Data System (ADS)

Nam, Y. J.; Park, M. K.; Yamane, R.

2007-12-01

In this study, a hand master using five miniature magneto-rheological (MR) actuators, which is called 'the smart glove', is introduced. This hand master is intended to display haptic feedback to the fingertip of the human user interacting with any virtual objects in virtual environment. For the smart glove, two effective approaches are proposed: (i) by using the MR actuator which can be considered as a passive actuator, the smart glove is made simple in structure, high in power, low in inertia, safe in interface and stable in haptic feedback, and (ii) with a novel flexible link mechanism designed for the position-force transmission between the fingertips and the actuators, the number of the actuator and the weight of the smart glove can be reduced. These features lead to the improvement in the manipulability and portability of the smart glove. The feasibility of the constructed smart glove is verified through basic performance evaluation.

Feedback as Real-Time Constructions

ERIC Educational Resources Information Center

Keiding, Tina Bering; Qvortrup, Ane

2014-01-01

This article offers a re-description of feedback and the significance of time in feedback constructions based on systems theory. It describes feedback as internal, real-time constructions in a learning system. From this perspective, feedback is neither immediate nor delayed, but occurs in the very moment it takes place. This article argues for a…

The role of haptic feedback in laparoscopic training using the LapMentor II.

PubMed

Salkini, Mohamad W; Doarn, Charles R; Kiehl, Nicholai; Broderick, Timothy J; Donovan, James F; Gaitonde, Krishnanath

2010-01-01

Laparoscopic surgery has become the standard of care for many surgical diseases. Haptic (tactile) feedback (HFB) is considered an important component of laparoscopic surgery. Virtual reality simulation (VRS) is an alternative method to teach surgical skills to surgeons in training. Newer VRS trainers such as the Simbionix Lap Mentor II provide significantly improved tactile feedback. However, VRSs are expensive and adding HFB software adds an estimated cost of $30,000 to the commercial price. The HFB provided by the Lap Mentor II has not been validated by an independent party. We used the Simbionix Lap Mentor II in this study to demonstrate the effect of adding an HFB mechanism in the VRS trainer. The study was approved by the University of Cincinnati Institutional Review Board. Twenty laparoscopically novice medical students were enrolled. Each student was asked to perform three different tasks on the Lap Mentor II and repeat each one five times. The chosen tasks demanded significant amount of traction and counter traction. The first task was to pull leaking tubes enough and clip them. The second task was stretching a jelly plate enough to see its attachments to the floor and cut these attachments. In the third task, the trainee had to separate the gallbladder from its bed on the liver. The students were randomized into two groups to perform the tasks with and without HFB. We used accuracy, speed, and economy of movement as scales to compare the performance between the two groups. The participants also completed a simple questionnaire that highlighted age, sex, and experiences in videogame usage. The two groups were comparable in age, sex, and videogame playing. No differences in the accuracy, the economy, and the speed of hand movement were noticed. In fact, adding HFB to the Lap Mentor II simulator did not contribute to any improvement in the performance of the trainees. Interestingly, we found that videogame expert players tend to have faster and more economic motion in their dominant hands. However, the performance accuracy was not significantly affected. The presence of HFB has less effect than it thought to be on the performance of the novice trainees. This may suggest that better HFB is still needed. However, there may be visual compensation for the lack of haptics. Playing videogames has a positive impact on economy, and the speed of the dominant had motion without affecting its accuracy. Further research is needed to clarify the value of haptics to the expert surgeon and compare it to the new trainees.

Preliminary assessment of faculty and student perception of a haptic virtual reality simulator for training dental manual dexterity.

PubMed

Gal, Gilad Ben; Weiss, Ervin I; Gafni, Naomi; Ziv, Amitai

2011-04-01

Virtual reality force feedback simulators provide a haptic (sense of touch) feedback through the device being held by the user. The simulator's goal is to provide a learning experience resembling reality. A newly developed haptic simulator (IDEA Dental, Las Vegas, NV, USA) was assessed in this study. Our objectives were to assess the simulator's ability to serve as a tool for dental instruction, self-practice, and student evaluation, as well as to evaluate the sensation it provides. A total of thirty-three evaluators were divided into two groups. The first group consisted of twenty-one experienced dental educators; the second consisted of twelve fifth-year dental students. Each participant performed drilling tasks using the simulator and filled out a questionnaire regarding the simulator and potential ways of using it in dental education. The results show that experienced dental faculty members as well as advanced dental students found that the simulator could provide significant potential benefits in the teaching and self-learning of manual dental skills. Development of the simulator's tactile sensation is needed to attune it to genuine sensation. Further studies relating to aspects of the simulator's structure and its predictive validity, its scoring system, and the nature of the performed tasks should be conducted.

Nonlinear friction modelling and compensation control of hysteresis phenomena for a pair of tendon-sheath actuated surgical robots

NASA Astrophysics Data System (ADS)

Do, T. N.; Tjahjowidodo, T.; Lau, M. W. S.; Phee, S. J.

2015-08-01

Natural Orifice Transluminal Endoscopic Surgery (NOTES) is a special method that allows surgical operations via natural orifices like mouth, anus, and vagina, without leaving visible scars. The use of flexible tendon-sheath mechanism (TSM) is common in these systems because of its light weight in structure, flexibility, and easy transmission of power. However, nonlinear friction and backlash hysteresis pose many challenges to control of such systems; in addition, they do not provide haptic feedback to assist the surgeon in the operation of the systems. In this paper, we propose a new dynamic friction model and backlash hysteresis nonlinearity for a pair of TSM to deal with these problems. The proposed friction model, unlike current approaches in the literature, is smooth and able to capture the force at near zero velocity when the system is stationary or operates at small motion. This model can be used to estimate the friction force for haptic feedback purpose. To improve the system tracking performances, a backlash hysteresis model will be introduced, which can be used in a feedforward controller scheme. The controller involves a simple computation of the inverse hysteresis model. The proposed models are configuration independent and able to capture the nonlinearities for arbitrary tendon-sheath shapes. A representative experimental setup is used to validate the proposed models and to demonstrate the improvement in position tracking accuracy and the possibility of providing desired force information at the distal end of a pair of TSM slave manipulator for haptic feedback to the surgeons.

A better understanding of ambulance personnel's attitude towards real-time resuscitation feedback.

PubMed

Brinkrolf, Peter; Lukas, Roman; Harding, Ulf; Thies, Sebastian; Gerss, Joachim; Van Aken, Hugo; Lemke, Hans; Schniedermeier, Udo; Bohn, Andreas

2018-03-01

High-quality chest compressions during cardiopulmonary resuscitation (CPR) play a significant role in surviving cardiac arrest. Chest-compression quality can be measured and corrected by real-time CPR feedback devices, which are not yet commonly used. This article looks at the acceptance of such systems in comparison of equipped and unequipped personnel. Two groups of emergency medical services' (EMS) personnel were interviewed using standardized questionnaires. The survey was conducted in the German cities Dortmund and Münster. Overall, 205 persons participated in the survey: 103 paramedics and emergency physicians from the Dortmund fire service and 102 personnel from the Münster service. The staff of the Dortmund service were not equipped with real-time feedback systems. The test group of equipped personnel of the ambulance service of Münster Fire brigade uses real-time feedback systems since 2007. What is the acceptance level of real-time feedback systems? Are there differences between equipped and unequipped personnel? The total sample is receptive towards real-time feedback systems. More than 80% deem the system useful. However, this study revealed concerns and prejudices by unequipped personnel. Negative ratings are significantly lower at the Münster site that is experienced with the use of the real-time feedback system in contrast to the Dortmund site where no such experience exists-the system's use in daily routine results in better evaluation than the expectations of unequipped personnel. Real-time feedback systems receive overall positive ratings. Prejudices and concerns seem to decrease with continued use of the system.

The Organization of Exploratory Behaviors in Infant Locomotor Planning

ERIC Educational Resources Information Center

Kretch, Kari S.; Adolph, Karen E.

2017-01-01

How do infants plan and guide locomotion under challenging conditions? This experiment investigated the real-time process of visual and haptic exploration in 14-month-old infants as they decided whether and how to walk over challenging terrain--a series of bridges varying in width. Infants' direction of gaze was recorded with a head-mounted eye…

Haptic display for the VR arthroscopy training simulator

NASA Astrophysics Data System (ADS)

Ziegler, Rolf; Brandt, Christoph; Kunstmann, Christian; Mueller, Wolfgang; Werkhaeuser, Holger

1997-05-01

A specific desire to find new training methods arose from the new fields called 'minimal invasive surgery.' With the technical advance modern video arthroscopy became the standard procedure in the ORs. Holding the optical system with the video camera in one hand, watching the operation field on the monitor, the other hand was free to guide, e.g., a probe. As arthroscopy became a more common procedure it became obvious that some sort of special training was necessary to guarantee a certain level of qualification of the surgeons. Therefore, a hospital in Frankfurt, Germany approached the Fraunhofer Institute for Computer Graphics to develop a training system for arthroscopy based on VR techniques. At least the main drawback of the developed simulator is the missing of haptic perception, especially of force feedback. In cooperation with the Department of Electro-Mechanical Construction at the Darmstadt Technical University we have designed and built a haptic display for the VR arthroscopy training simulator. In parallel we developed a concept for the integration of the haptic display in a configurable way.

Roughness Perception of Haptically Displayed Fractal Surfaces

NASA Technical Reports Server (NTRS)

Costa, Michael A.; Cutkosky, Mark R.; Lau, Sonie (Technical Monitor)

2000-01-01

Surface profiles were generated by a fractal algorithm and haptically rendered on a force feedback joystick, Subjects were asked to use the joystick to explore pairs of surfaces and report to the experimenter which of the surfaces they felt was rougher. Surfaces were characterized by their root mean square (RMS) amplitude and their fractal dimension. The most important factor affecting the perceived roughness of the fractal surfaces was the RMS amplitude of the surface. When comparing surfaces of fractal dimension 1.2-1.35 it was found that the fractal dimension was negatively correlated with perceived roughness.

Real-time evaluation and visualization of learner performance in a mixed-reality environment for clinical breast examination.

PubMed

Kotranza, Aaron; Lind, D Scott; Lok, Benjamin

2012-07-01

We investigate the efficacy of incorporating real-time feedback of user performance within mixed-reality environments (MREs) for training real-world tasks with tightly coupled cognitive and psychomotor components. This paper presents an approach to providing real-time evaluation and visual feedback of learner performance in an MRE for training clinical breast examination (CBE). In a user study of experienced and novice CBE practitioners (n = 69), novices receiving real-time feedback performed equivalently or better than more experienced practitioners in the completeness and correctness of the exam. A second user study (n = 8) followed novices through repeated practice of CBE in the MRE. Results indicate that skills improvement in the MRE transfers to the real-world task of CBE of human patients. This initial case study demonstrates the efficacy of MREs incorporating real-time feedback for training real-world cognitive-psychomotor tasks.

Real-time image guidance in laparoscopic liver surgery: first clinical experience with a guidance system based on intraoperative CT imaging.

PubMed

Kenngott, Hannes G; Wagner, Martin; Gondan, Matthias; Nickel, Felix; Nolden, Marco; Fetzer, Andreas; Weitz, Jürgen; Fischer, Lars; Speidel, Stefanie; Meinzer, Hans-Peter; Böckler, Dittmar; Büchler, Markus W; Müller-Stich, Beat P

2014-03-01

Laparoscopic liver surgery is particularly challenging owing to restricted access, risk of bleeding, and lack of haptic feedback. Navigation systems have the potential to improve information on the exact position of intrahepatic tumors, and thus facilitate oncological resection. This study aims to evaluate the feasibility of a commercially available augmented reality (AR) guidance system employing intraoperative robotic C-arm cone-beam computed tomography (CBCT) for laparoscopic liver surgery. A human liver-like phantom with 16 target fiducials was used to evaluate the Syngo iPilot(®) AR system. Subsequently, the system was used for the laparoscopic resection of a hepatocellular carcinoma in segment 7 of a 50-year-old male patient. In the phantom experiment, the AR system showed a mean target registration error of 0.96 ± 0.52 mm, with a maximum error of 2.49 mm. The patient successfully underwent the operation and showed no postoperative complications. The use of intraoperative CBCT and AR for laparoscopic liver resection is feasible and could be considered an option for future liver surgery in complex cases.

Mobile tele-echography: user interface design.

PubMed

Cañero, Cristina; Thomos, Nikolaos; Triantafyllidis, George A; Litos, George C; Strintzis, Michael Gerassimos

2005-03-01

Ultrasound imaging allows the evaluation of the degree of emergency of a patient. However, in some instances, a well-trained sonographer is unavailable to perform such echography. To cope with this issue, the Mobile Tele-Echography Using an Ultralight Robot (OTELO) project aims to develop a fully integrated end-to-end mobile tele-echography system using an ultralight remote-controlled robot for population groups that are not served locally by medical experts. This paper focuses on the user interface of the OTELO system, consisting of the following parts: an ultrasound video transmission system providing real-time images of the scanned area, an audio/video conference to communicate with the paramedical assistant and with the patient, and a virtual-reality environment, providing visual and haptic feedback to the expert, while capturing the expert's hand movements. These movements are reproduced by the robot at the patient site while holding the ultrasound probe against the patient skin. In addition, the user interface includes an image processing facility for enhancing the received images and the possibility to include them into a database.

Effects of Real-Time Visual Feedback on Pre-Service Teachers' Singing

ERIC Educational Resources Information Center

Leong, S.; Cheng, L.

2014-01-01

This pilot study focuses on the use real-time visual feedback technology (VFT) in vocal training. The empirical research has two aims: to ascertain the effectiveness of the real-time visual feedback software "Sing & See" in the vocal training of pre-service music teachers and the teachers' perspective on their experience with…

In vivo biomechanical measurement and haptic simulation of portal placement procedure in shoulder arthroscopic surgery

PubMed Central

Chae, Sanghoon; Jung, Sung-Weon

2018-01-01

A survey of 67 experienced orthopedic surgeons indicated that precise portal placement was the most important skill in arthroscopic surgery. However, none of the currently available virtual reality simulators include simulation / training in portal placement, including haptic feedback of the necessary puncture force. This study aimed to: (1) measure the in vivo force and stiffness during a portal placement procedure in an actual operating room and (2) implement active haptic simulation of a portal placement procedure using the measured in vivo data. We measured the force required for port placement and the stiffness of the joint capsule during portal placement procedures performed by an experienced arthroscopic surgeon. Based on the acquired mechanical property values, we developed a cable-driven active haptic simulator designed to train the portal placement skill and evaluated the validity of the simulated haptics. Ten patients diagnosed with rotator cuff tears were enrolled in this experiment. The maximum peak force and joint capsule stiffness during posterior portal placement procedures were 66.46 (±10.76N) and 2560.82(±252.92) N/m, respectively. We then designed an active haptic simulator using the acquired data. Our cable-driven mechanism structure had a friction force of 3.763 ± 0.341 N, less than 6% of the mean puncture force. Simulator performance was evaluated by comparing the target stiffness and force with the stiffness and force reproduced by the device. R-squared values were 0.998 for puncture force replication and 0.902 for stiffness replication, indicating that the in vivo data can be used to implement a realistic haptic simulator. PMID:29494691

Clinical usefulness of augmented reality using infrared camera based real-time feedback on gait function in cerebral palsy: a case study.

PubMed

Lee, Byoung-Hee

2016-04-01

[Purpose] This study investigated the effects of real-time feedback using infrared camera recognition technology-based augmented reality in gait training for children with cerebral palsy. [Subjects] Two subjects with cerebral palsy were recruited. [Methods] In this study, augmented reality based real-time feedback training was conducted for the subjects in two 30-minute sessions per week for four weeks. Spatiotemporal gait parameters were used to measure the effect of augmented reality-based real-time feedback training. [Results] Velocity, cadence, bilateral step and stride length, and functional ambulation improved after the intervention in both cases. [Conclusion] Although additional follow-up studies of the augmented reality based real-time feedback training are required, the results of this study demonstrate that it improved the gait ability of two children with cerebral palsy. These findings suggest a variety of applications of conservative therapeutic methods which require future clinical trials.

In vivo soft tissue differentiation by diffuse reflectance spectroscopy: preliminary results

NASA Astrophysics Data System (ADS)

Zam, Azhar; Stelzle, Florian; Tangermann-Gerk, Katja; Adler, Werner; Nkenke, Emeka; Neukam, Friedrich Wilhelm; Schmidt, Michael; Douplik, Alexandre

Remote laser surgery does not provide haptic feedback to operate layer by layer and preserve vulnerable anatomical structures like nerve tissue or blood vessels. The aim of this study is identification of soft tissue in vivo by diffuse reflectance spectroscopy to set the base for a feedback control system to enhance nerve preservation in oral and maxillofacial laser surgery. Various soft tissues can be identified by diffuse reflectance spectroscopy in vivo. The results may set the base for a feedback system to prevent nerve damage during oral and maxillofacial laser surgery.

Can You Help Me with My Pitch? Studying a Tool for Real-Time Automated Feedback

ERIC Educational Resources Information Center

Schneider, Jan; Borner, Dirk; van Rosmalen, Peter; Specht, Marcus

2016-01-01

In our pursue to study effective real-time feedback in Technology Enhanced Learning, we developed the Presentation Trainer, a tool designed to support the practice of nonverbal communication skills for public speaking. The tool tracks the user's voice and body to analyze her performance, and selects the type of real-time feedback to be presented.…

Data-Driven Haptic Modeling and Rendering of Viscoelastic and Frictional Responses of Deformable Objects.

PubMed

Yim, Sunghoon; Jeon, Seokhee; Choi, Seungmoon

2016-01-01

In this paper, we present an extended data-driven haptic rendering method capable of reproducing force responses during pushing and sliding interaction on a large surface area. The main part of the approach is a novel input variable set for the training of an interpolation model, which incorporates the position of a proxy - an imaginary contact point on the undeformed surface. This allows us to estimate friction in both sliding and sticking states in a unified framework. Estimating the proxy position is done in real-time based on simulation using a sliding yield surface - a surface defining a border between the sliding and sticking regions in the external force space. During modeling, the sliding yield surface is first identified via an automated palpation procedure. Then, through manual palpation on a target surface, input data and resultant force data are acquired. The data are used to build a radial basis interpolation model. During rendering, this input-output mapping interpolation model is used to estimate force responses in real-time in accordance with the interaction input. Physical performance evaluation demonstrates that our approach achieves reasonably high estimation accuracy. A user study also shows plausible perceptual realism under diverse and extensive exploration.

Augmented versus Virtual Reality Laparoscopic Simulation: What Is the Difference?

PubMed Central

Botden, Sanne M.B.I.; Buzink, Sonja N.; Schijven, Marlies P.

2007-01-01

Background Virtual reality (VR) is an emerging new modality for laparoscopic skills training; however, most simulators lack realistic haptic feedback. Augmented reality (AR) is a new laparoscopic simulation system offering a combination of physical objects and VR simulation. Laparoscopic instruments are used within an hybrid mannequin on tissue or objects while using video tracking. This study was designed to assess the difference in realism, haptic feedback, and didactic value between AR and VR laparoscopic simulation. Methods The ProMIS AR and LapSim VR simulators were used in this study. The participants performed a basic skills task and a suturing task on both simulators, after which they filled out a questionnaire about their demographics and their opinion of both simulators scored on a 5-point Likert scale. The participants were allotted to 3 groups depending on their experience: experts, intermediates and novices. Significant differences were calculated with the paired t-test. Results There was general consensus in all groups that the ProMIS AR laparoscopic simulator is more realistic than the LapSim VR laparoscopic simulator in both the basic skills task (mean 4.22 resp. 2.18, P < 0.000) as well as the suturing task (mean 4.15 resp. 1.85, P < 0.000). The ProMIS is regarded as having better haptic feedback (mean 3.92 resp. 1.92, P < 0.000) and as being more useful for training surgical residents (mean 4.51 resp. 2.94, P < 0.000). Conclusions In comparison with the VR simulator, the AR laparoscopic simulator was regarded by all participants as a better simulator for laparoscopic skills training on all tested features. PMID:17361356

The role of visuohaptic experience in visually perceived depth.

PubMed

Ho, Yun-Xian; Serwe, Sascha; Trommershäuser, Julia; Maloney, Laurence T; Landy, Michael S

2009-06-01

Berkeley suggested that "touch educates vision," that is, haptic input may be used to calibrate visual cues to improve visual estimation of properties of the world. Here, we test whether haptic input may be used to "miseducate" vision, causing observers to rely more heavily on misleading visual cues. Human subjects compared the depth of two cylindrical bumps illuminated by light sources located at different positions relative to the surface. As in previous work using judgments of surface roughness, we find that observers judge bumps to have greater depth when the light source is located eccentric to the surface normal (i.e., when shadows are more salient). Following several sessions of visual judgments of depth, subjects then underwent visuohaptic training in which haptic feedback was artificially correlated with the "pseudocue" of shadow size and artificially decorrelated with disparity and texture. Although there were large individual differences, almost all observers demonstrated integration of haptic cues during visuohaptic training. For some observers, subsequent visual judgments of bump depth were unaffected by the training. However, for 5 of 12 observers, training significantly increased the weight given to pseudocues, causing subsequent visual estimates of shape to be less veridical. We conclude that haptic information can be used to reweight visual cues, putting more weight on misleading pseudocues, even when more trustworthy visual cues are available in the scene.

Force Control and Nonlinear Master-Slave Force Profile to Manage an Admittance Type Multi-Fingered Haptic User Interface

DOE Office of Scientific and Technical Information (OSTI.GOV)

Anthony L. Crawford

2012-08-01

Natural movements and force feedback are important elements in using teleoperated equipment if complex and speedy manipulation tasks are to be accomplished in remote and/or hazardous environments, such as hot cells, glove boxes, decommissioning, explosives disarmament, and space to name a few. In order to achieve this end the research presented in this paper has developed an admittance type exoskeleton like multi-fingered haptic hand user interface that secures the user’s palm and provides 3-dimensional force feedback to the user’s fingertips. Atypical to conventional haptic hand user interfaces that limit themselves to integrating the human hand’s characteristics just into the system’smore » mechanical design this system also perpetuates that inspiration into the designed user interface’s controller. This is achieved by manifesting the property differences of manipulation and grasping activities as they pertain to the human hand into a nonlinear master-slave force relationship. The results presented in this paper show that the admittance-type system has sufficient bandwidth that it appears nearly transparent to the user when the user is in free motion and when the system is subjected to a manipulation task, increased performance is achieved using the nonlinear force relationship compared to the traditional linear scaling techniques implemented in the vast majority of systems.« less

A mixed reality simulator for feline abdominal palpation training in veterinary medicine.

PubMed

Parkes, Rebecca; Forrest, Neil; Baillie, Sarah

2009-01-01

The opportunities for veterinary students to practice feline abdominal palpation are limited as cats have a low tolerance to being examined. Therefore, a mixed reality simulator was developed to complement clinical training. Two PHANToM premium haptic devices were positioned either side of a modified toy cat. Virtual models of the chest and some abdominal contents were superimposed on the physical model. The haptic properties of the virtual models were set by seven veterinarians; values were adjusted while the simulation was being palpated until the representation was satisfactory. Feedback from the veterinarians was encouraging suggesting that the simulator has a potential role in student training.

A temporal bone surgery simulator with real-time feedback for surgical training.

PubMed

Wijewickrema, Sudanthi; Ioannou, Ioanna; Zhou, Yun; Piromchai, Patorn; Bailey, James; Kennedy, Gregor; O'Leary, Stephen

2014-01-01

Timely feedback on surgical technique is an important aspect of surgical skill training in any learning environment, be it virtual or otherwise. Feedback on technique should be provided in real-time to allow trainees to recognize and amend their errors as they occur. Expert surgeons have typically carried out this task, but they have limited time available to spend with trainees. Virtual reality surgical simulators offer effective, repeatable training at relatively low cost, but their benefits may not be fully realized while they still require the presence of experts to provide feedback. We attempt to overcome this limitation by introducing a real-time feedback system for surgical technique within a temporal bone surgical simulator. Our evaluation study shows that this feedback system performs exceptionally well with respect to accuracy and effectiveness.

Multimodal information Management: Evaluation of Auditory and Haptic Cues for NextGen Communication Displays

NASA Technical Reports Server (NTRS)

Begault, Durand R.; Bittner, Rachel M.; Anderson, Mark R.

2012-01-01

Auditory communication displays within the NextGen data link system may use multiple synthetic speech messages replacing traditional ATC and company communications. The design of an interface for selecting amongst multiple incoming messages can impact both performance (time to select, audit and release a message) and preference. Two design factors were evaluated: physical pressure-sensitive switches versus flat panel "virtual switches", and the presence or absence of auditory feedback from switch contact. Performance with stimuli using physical switches was 1.2 s faster than virtual switches (2.0 s vs. 3.2 s); auditory feedback provided a 0.54 s performance advantage (2.33 s vs. 2.87 s). There was no interaction between these variables. Preference data were highly correlated with performance.

A Virtual Reality System for PTCD Simulation Using Direct Visuo-Haptic Rendering of Partially Segmented Image Data.

PubMed

Fortmeier, Dirk; Mastmeyer, Andre; Schröder, Julian; Handels, Heinz

2016-01-01

This study presents a new visuo-haptic virtual reality (VR) training and planning system for percutaneous transhepatic cholangio-drainage (PTCD) based on partially segmented virtual patient models. We only use partially segmented image data instead of a full segmentation and circumvent the necessity of surface or volume mesh models. Haptic interaction with the virtual patient during virtual palpation, ultrasound probing and needle insertion is provided. Furthermore, the VR simulator includes X-ray and ultrasound simulation for image-guided training. The visualization techniques are GPU-accelerated by implementation in Cuda and include real-time volume deformations computed on the grid of the image data. Computation on the image grid enables straightforward integration of the deformed image data into the visualization components. To provide shorter rendering times, the performance of the volume deformation algorithm is improved by a multigrid approach. To evaluate the VR training system, a user evaluation has been performed and deformation algorithms are analyzed in terms of convergence speed with respect to a fully converged solution. The user evaluation shows positive results with increased user confidence after a training session. It is shown that using partially segmented patient data and direct volume rendering is suitable for the simulation of needle insertion procedures such as PTCD.

Design and Evaluation of Shape-Changing Haptic Interfaces for Pedestrian Navigation Assistance.

PubMed

Spiers, Adam J; Dollar, Aaron M

2017-01-01

Shape-changing interfaces are a category of device capable of altering their form in order to facilitate communication of information. In this work, we present a shape-changing device that has been designed for navigation assistance. 'The Animotus' (previously, 'The Haptic Sandwich' ), resembles a cube with an articulated upper half that is able to rotate and extend (translate) relative to the bottom half, which is fixed in the user's grasp. This rotation and extension, generally felt via the user's fingers, is used to represent heading and proximity to navigational targets. The device is intended to provide an alternative to screen or audio based interfaces for visually impaired, hearing impaired, deafblind, and sighted pedestrians. The motivation and design of the haptic device is presented, followed by the results of a navigation experiment that aimed to determine the role of each device DOF, in terms of facilitating guidance. An additional device, 'The Haptic Taco', which modulated its volume in response to target proximity (negating directional feedback), was also compared. Results indicate that while the heading (rotational) DOF benefited motion efficiency, the proximity (translational) DOF benefited velocity. Combination of the two DOF improved overall performance. The volumetric Taco performed comparably to the Animotus' extension DOF.

Mechanical model of orthopaedic drilling for augmented-haptics-based training.

PubMed

Pourkand, Ashkan; Zamani, Naghmeh; Grow, David

2017-10-01

In this study, augmented-haptic feedback is used to combine a physical object with virtual elements in order to simulate anatomic variability in bone. This requires generating levels of force/torque consistent with clinical bone drilling, which exceed the capabilities of commercially available haptic devices. Accurate total force generation is facilitated by a predictive model of axial force during simulated orthopaedic drilling. This model is informed by kinematic data collected while drilling into synthetic bone samples using an instrumented linkage attached to the orthopaedic drill. Axial force is measured using a force sensor incorporated into the bone fixture. A nonlinear function, relating force to axial position and velocity, was used to fit the data. The normalized root-mean-square error (RMSE) of forces predicted by the model compared to those measured experimentally was 0.11 N across various bones with significant differences in geometry and density. This suggests that a predictive model can be used to capture relevant variations in the thickness and hardness of cortical and cancellous bone. The practical performance of this approach is measured using the Phantom Premium haptic device, with some required customizations. Copyright © 2017 Elsevier Ltd. All rights reserved.

The contribution of cutaneous and kinesthetic sensory modalities in haptic perception of orientation.

PubMed

Frisoli, Antonio; Solazzi, Massimiliano; Reiner, Miriam; Bergamasco, Massimo

2011-06-30

The aim of this study was to understand the integration of cutaneous and kinesthetic sensory modalities in haptic perception of shape orientation. A specific robotic apparatus was employed to simulate the exploration of virtual surfaces by active touch with two fingers, with kinesthetic only, cutaneous only and combined sensory feedback. The cutaneous feedback was capable of displaying the local surface orientation at the contact point, through a small plate indenting the fingerpad at contact. A psychophysics test was conducted with SDT methodology on 6 subjects to assess the discrimination threshold of angle perception between two parallel surfaces, with three sensory modalities and two shape sizes. Results show that the cutaneous sensor modality is not affected by size of shape, but kinesthetic performance is decreasing with smaller size. Cutaneous and kinesthetic sensory cues are integrated according to a Bayesian model, so that the combined sensory stimulation always performs better than single modalities alone. Copyright © 2010 Elsevier Inc. All rights reserved.

Clinical usefulness of augmented reality using infrared camera based real-time feedback on gait function in cerebral palsy: a case study

PubMed Central

Lee, Byoung-Hee

2016-01-01

[Purpose] This study investigated the effects of real-time feedback using infrared camera recognition technology-based augmented reality in gait training for children with cerebral palsy. [Subjects] Two subjects with cerebral palsy were recruited. [Methods] In this study, augmented reality based real-time feedback training was conducted for the subjects in two 30-minute sessions per week for four weeks. Spatiotemporal gait parameters were used to measure the effect of augmented reality-based real-time feedback training. [Results] Velocity, cadence, bilateral step and stride length, and functional ambulation improved after the intervention in both cases. [Conclusion] Although additional follow-up studies of the augmented reality based real-time feedback training are required, the results of this study demonstrate that it improved the gait ability of two children with cerebral palsy. These findings suggest a variety of applications of conservative therapeutic methods which require future clinical trials. PMID:27190489

Haptic augmented skin surface generation toward telepalpation from a mobile skin image.

PubMed

Kim, K

2018-05-01

Very little is known about the methods of integrating palpation techniques to existing mobile teleskin imaging that delivers low quality tactile information (roughness) for telepalpation. However, no study has been reported yet regarding telehaptic palpation using mobile phone images for teledermatology or teleconsultations of skincare. This study is therefore aimed at introducing a new algorithm accurately reconstructing a haptic augmented skin surface for telehaptic palpation using a low-cost clip-on microscope simply attached to a mobile phone. Multiple algorithms such as gradient-based image enhancement, roughness-adaptive tactile mask generation, roughness-enhanced 3D tactile map building, and visual and haptic rendering with a three-degrees-of-freedom (DOF) haptic device were developed and integrated as one system. Evaluation experiments have been conducted to test the performance of 3D roughness reconstruction with/without the tactile mask. The results confirm that reconstructed haptic roughness with the tactile mask is superior to the reconstructed haptic roughness without the tactile mask. Additional experiments demonstrate that the proposed algorithm is robust against varying lighting conditions and blurring. In last, a user study has been designed to see the effect of the haptic modality to the existing visual only interface and the results attest that the haptic skin palpation can significantly improve the skin exam performance. Mobile image-based telehaptic palpation technology was proposed, and an initial version was developed. The developed technology was tested with several skin images and the experimental results showed the superiority of the proposed scheme in terms of the performance of haptic augmentation of real skin images. © 2017 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Vibration Influences Haptic Perception of Surface Compliance During Walking

PubMed Central

Visell, Yon; Giordano, Bruno L.; Millet, Guillaume; Cooperstock, Jeremy R.

2011-01-01

Background The haptic perception of ground compliance is used for stable regulation of dynamic posture and the control of locomotion in diverse natural environments. Although rarely investigated in relation to walking, vibrotactile sensory channels are known to be active in the discrimination of material properties of objects and surfaces through touch. This study investigated how the perception of ground surface compliance is altered by plantar vibration feedback. Methodology/Principal Findings Subjects walked in shoes over a rigid floor plate that provided plantar vibration feedback, and responded indicating how compliant it felt, either in subjective magnitude or via pairwise comparisons. In one experiment, the compliance of the floor plate was also varied. Results showed that perceived compliance of the plate increased monotonically with vibration feedback intensity, and depended to a lesser extent on the temporal or frequency distribution of the feedback. When both plate stiffness (inverse compliance) and vibration amplitude were manipulated, the effect persisted, with both factors contributing to compliance perception. A significant influence of vibration was observed even for amplitudes close to psychophysical detection thresholds. Conclusions/Significance These findings reveal that vibrotactile sensory channels are highly salient to the perception of surface compliance, and suggest that correlations between vibrotactile sensory information and motor activity may be of broader significance for the control of human locomotion than has been previously acknowledged. PMID:21464979

Object Recognition and Localization: The Role of Tactile Sensors

PubMed Central

Aggarwal, Achint; Kirchner, Frank

2014-01-01

Tactile sensors, because of their intrinsic insensitivity to lighting conditions and water turbidity, provide promising opportunities for augmenting the capabilities of vision sensors in applications involving object recognition and localization. This paper presents two approaches for haptic object recognition and localization for ground and underwater environments. The first approach called Batch Ransac and Iterative Closest Point augmented Particle Filter (BRICPPF) is based on an innovative combination of particle filters, Iterative-Closest-Point algorithm, and a feature-based Random Sampling and Consensus (RANSAC) algorithm for database matching. It can handle a large database of 3D-objects of complex shapes and performs a complete six-degree-of-freedom localization of static objects. The algorithms are validated by experimentation in ground and underwater environments using real hardware. To our knowledge this is the first instance of haptic object recognition and localization in underwater environments. The second approach is biologically inspired, and provides a close integration between exploration and recognition. An edge following exploration strategy is developed that receives feedback from the current state of recognition. A recognition by parts approach is developed which uses the BRICPPF for object sub-part recognition. Object exploration is either directed to explore a part until it is successfully recognized, or is directed towards new parts to endorse the current recognition belief. This approach is validated by simulation experiments. PMID:24553087

A magnetorheological haptic cue accelerator for manual transmission vehicles

NASA Astrophysics Data System (ADS)

Han, Young-Min; Noh, Kyung-Wook; Lee, Yang-Sub; Choi, Seung-Bok

2010-07-01

This paper proposes a new haptic cue function for manual transmission vehicles to achieve optimal gear shifting. This function is implemented on the accelerator pedal by utilizing a magnetorheological (MR) brake mechanism. By combining the haptic cue function with the accelerator pedal, the proposed haptic cue device can transmit the optimal moment of gear shifting for manual transmission to a driver without requiring the driver's visual attention. As a first step to achieve this goal, a MR fluid-based haptic device is devised to enable rotary motion of the accelerator pedal. Taking into account spatial limitations, the design parameters are optimally determined using finite element analysis to maximize the relative control torque. The proposed haptic cue device is then manufactured and its field-dependent torque and time response are experimentally evaluated. Then the manufactured MR haptic cue device is integrated with the accelerator pedal. A simple virtual vehicle emulating the operation of the engine of a passenger vehicle is constructed and put into communication with the haptic cue device. A feed-forward torque control algorithm for the haptic cue is formulated and control performances are experimentally evaluated and presented in the time domain.

Face and Construct Validity of a Novel Virtual Reality-Based Bimanual Laparoscopic Force-Skills Trainer With Haptics Feedback.

PubMed

Prasad, Raghu; Muniyandi, Manivannan; Manoharan, Govindan; Chandramohan, Servarayan M

2018-05-01

The purpose of this study was to examine the face and construct validity of a custom-developed bimanual laparoscopic force-skills trainer with haptics feedback. The study also examined the effect of handedness on fundamental and complex tasks. Residents (n = 25) and surgeons (n = 25) performed virtual reality-based bimanual fundamental and complex tasks. Tool-tissue reaction forces were summed, recorded, and analysed. Seven different force-based measures and a 1-time measure were used as metrics. Subsequently, participants filled out face validity and demographic questionnaires. Residents and surgeons were positive on the design, workspace, and usefulness of the simulator. Construct validity results showed significant differences between residents and experts during the execution of fundamental and complex tasks. In both tasks, residents applied large forces with higher coefficient of variation and force jerks (P < .001). Experts, with their dominant hand, applied lower forces in complex tasks and higher forces in fundamental tasks (P < .001). The coefficients of force variation (CoV) of residents and experts were higher in complex tasks (P < .001). Strong correlations were observed between CoV and task time for fundamental (r = 0.70) and complex tasks (r = 0.85). Range of smoothness of force was higher for the non-dominant hand in both fundamental and complex tasks. The simulator was able to differentiate the force-skills of residents and surgeons, and objectively evaluate the effects of handedness on laparoscopic force-skills. Competency-based laparoscopic skills assessment curriculum should be updated to meet the requirements of bimanual force-based training.

Calibrating Reach Distance to Visual Targets

ERIC Educational Resources Information Center

Mon-Williams, Mark; Bingham, Geoffrey P.

2007-01-01

The authors investigated the calibration of reach distance by gradually distorting the haptic feedback obtained when participants grasped visible target objects. The authors found that the modified relationship between visually specified distance and reach distance could be captured by a straight-line mapping function. Thus, the relation could be…

The effect of force feedback on student reasoning about gravity, mass, force and motion

NASA Astrophysics Data System (ADS)

Bussell, Linda

The purpose of this study was to examine whether force feedback within a computer simulation had an effect on reasoning by fifth grade students about gravity, mass, force, and motion, concepts which can be difficult for learners to grasp. Few studies have been done on cognitive learning and haptic feedback, particularly with young learners, but there is an extensive base of literature on children's conceptions of science and a number of studies focus specifically on children's conceptions of force and motion. This case study used a computer-based paddleball simulation with guided inquiry as the primary stimulus. Within the simulation, the learner could adjust the mass of the ball and the gravitational force. The experimental group used the simulation with visual and force feedback; the control group used the simulation with visual feedback but without force feedback. The proposition was that there would be differences in reasoning between the experimental and control groups, with force feedback being helpful with concepts that are more obvious when felt. Participants were 34 fifth-grade students from three schools. Students completed a modal (visual, auditory, and haptic) learning preference assessment and a pretest. The sessions, including participant experimentation and interviews, were audio recorded and observed. The interviews were followed by a written posttest. These data were analyzed to determine whether there were differences based on treatment, learning style, demographics, prior gaming experience, force feedback experience, or prior knowledge. Work with the simulation, regardless of group, was found to increase students' understanding of key concepts. The experimental group appeared to benefit from the supplementary help that force feedback provided. Those in the experimental group scored higher on the posttest than those in the control group. The greatest difference between mean group scores was on a question concerning the effects of increased gravitational force.

Development of a force-reflecting robotic platform for cardiac catheter navigation.

PubMed

Park, Jun Woo; Choi, Jaesoon; Pak, Hui-Nam; Song, Seung Joon; Lee, Jung Chan; Park, Yongdoo; Shin, Seung Min; Sun, Kyung

2010-11-01

Electrophysiological catheters are used for both diagnostics and clinical intervention. To facilitate more accurate and precise catheter navigation, robotic cardiac catheter navigation systems have been developed and commercialized. The authors have developed a novel force-reflecting robotic catheter navigation system. The system is a network-based master-slave configuration having a 3-degree of freedom robotic manipulator for operation with a conventional cardiac ablation catheter. The master manipulator implements a haptic user interface device with force feedback using a force or torque signal either measured with a sensor or estimated from the motor current signal in the slave manipulator. The slave manipulator is a robotic motion control platform on which the cardiac ablation catheter is mounted. The catheter motions-forward and backward movements, rolling, and catheter tip bending-are controlled by electromechanical actuators located in the slave manipulator. The control software runs on a real-time operating system-based workstation and implements the master/slave motion synchronization control of the robot system. The master/slave motion synchronization response was assessed with step, sinusoidal, and arbitrarily varying motion commands, and showed satisfactory performance with insignificant steady-state motion error. The current system successfully implemented the motion control function and will undergo safety and performance evaluation by means of animal experiments. Further studies on the force feedback control algorithm and on an active motion catheter with an embedded actuation mechanism are underway. © 2010, Copyright the Authors. Artificial Organs © 2010, International Center for Artificial Organs and Transplantation and Wiley Periodicals, Inc.

Friction Compensation for Enhancing Transparency of a Teleoperator with Compliant Transmission

PubMed Central

Mahvash, Mohsen; Okamura, Allison

2009-01-01

This article presents a model-based compensator for canceling friction in the tendon-driven joints of a haptic-feedback teleoperator. Unlike position-tracking systems, a teleoperator involves an unknown environment force that prevents the use of tracking position error as a feedback to the compensator. Thus, we use a model-based feedforward friction compensator to cancel the friction forces. We provide conditions for selecting compensator parameters to ensure passivity of the teleoperator and demonstrate performance experimentally. PMID:20514151

Mastoidectomy simulation with combined visual and haptic feedback.

PubMed

Agus, Marco; Giachetti, Andrea; Gobbetti, Enrico; Zanetti, Gianluigi; Zorcolo, Antonio; John, Nigel W; Stone, Robert J

2002-01-01

Mastoidectomy is one of the most common surgical procedures relating to the petrous bone. In this paper we describe our preliminary results in the realization of a virtual reality mastoidectomy simulator. Our system is designed to work on patient-specific volumetric object models directly derived from 3D CT and MRI images. The paper summarizes the detailed task analysis performed in order to define the system requirements, introduces the architecture of the prototype simulator, and discusses the initial feedback received from selected end users.

Mathematical model of bone drilling for virtual surgery system

NASA Astrophysics Data System (ADS)

Alaytsev, Innokentiy K.; Danilova, Tatyana V.; Manturov, Alexey O.; Mareev, Gleb O.; Mareev, Oleg V.

2018-04-01

The bone drilling is an essential part of surgeries in ENT and Dentistry. A proper training of drilling machine handling skills is impossible without proper modelling of the drilling process. Utilization of high precision methods like FEM is limited due to the requirement of 1000 Hz update rate for haptic feedback. The study presents a mathematical model of the drilling process that accounts the properties of materials, the geometry and the rotation rate of a burr to compute the removed material volume. The simplicity of the model allows for integrating it in the high-frequency haptic thread. The precision of the model is enough for a virtual surgery system targeted on the training of the basic surgery skills.

Touch-screen technology for the dynamic display of -2D spatial information without vision: promise and progress.

PubMed

Klatzky, Roberta L; Giudice, Nicholas A; Bennett, Christopher R; Loomis, Jack M

2014-01-01

Many developers wish to capitalize on touch-screen technology for developing aids for the blind, particularly by incorporating vibrotactile stimulation to convey patterns on their surfaces, which otherwise are featureless. Our belief is that they will need to take into account basic research on haptic perception in designing these graphics interfaces. We point out constraints and limitations in haptic processing that affect the use of these devices. We also suggest ways to use sound to augment basic information from touch, and we include evaluation data from users of a touch-screen device with vibrotactile and auditory feedback that we have been developing, called a vibro-audio interface.

Reviewing the technological challenges associated with the development of a laparoscopic palpation device.

PubMed

Culmer, Peter; Barrie, Jenifer; Hewson, Rob; Levesley, Martin; Mon-Williams, Mark; Jayne, David; Neville, Anne

2012-06-01

Minimally invasive surgery (MIS) has heralded a revolution in surgical practice, with numerous advantages over open surgery. Nevertheless, it prevents the surgeon from directly touching and manipulating tissue and therefore severely restricts the use of valuable techniques such as palpation. Accordingly a key challenge in MIS is to restore haptic feedback to the surgeon. This paper reviews the state-of-the-art in laparoscopic palpation devices (LPDs) with particular focus on device mechanisms, sensors and data analysis. It concludes by examining the challenges that must be overcome to create effective LPD systems that measure and display haptic information to the surgeon for improved intraoperative assessment. Copyright © 2012 John Wiley & Sons, Ltd.

Does haptic steering guidance instigate speeding? A driving simulator study into causes and remedies.

PubMed

Melman, T; de Winter, J C F; Abbink, D A

2017-01-01

An important issue in road traffic safety is that drivers show adverse behavioral adaptation (BA) to driver assistance systems. Haptic steering guidance is an upcoming assistance system which facilitates lane-keeping performance while keeping drivers in the loop, and which may be particularly prone to BA. Thus far, experiments on haptic steering guidance have measured driver performance while the vehicle speed was kept constant. The aim of the present driving simulator study was to examine whether haptic steering guidance causes BA in the form of speeding, and to evaluate two types of haptic steering guidance designed not to suffer from BA. Twenty-four participants drove a 1.8m wide car for 13.9km on a curved road, with cones demarcating a single 2.2m narrow lane. Participants completed four conditions in a counterbalanced design: no guidance (Manual), continuous haptic guidance (Cont), continuous guidance that linearly reduced feedback gains from full guidance at 125km/h towards manual control at 130km/h and above (ContRF), and haptic guidance provided only when the predicted lateral position was outside a lateral bandwidth (Band). Participants were familiarized with each condition prior to the experimental runs and were instructed to drive as they normally would while minimizing the number of cone hits. Compared to Manual, the Cont condition yielded a significantly higher driving speed (on average by 7km/h), whereas ContRF and Band did not. All three guidance conditions yielded better lane-keeping performance than Manual, whereas Cont and ContRF yielded lower self-reported workload than Manual. In conclusion, continuous steering guidance entices drivers to increase their speed, thereby diminishing its potential safety benefits. It is possible to prevent BA while retaining safety benefits by making a design adjustment either in lateral (Band) or in longitudinal (ContRF) direction. Copyright © 2016. Published by Elsevier Ltd.

A pseudo-haptic knot diagram interface

NASA Astrophysics Data System (ADS)

Zhang, Hui; Weng, Jianguang; Hanson, Andrew J.

2011-01-01

To make progress in understanding knot theory, we will need to interact with the projected representations of mathematical knots which are of course continuous in 3D but significantly interrupted in the projective images. One way to achieve such a goal would be to design an interactive system that allows us to sketch 2D knot diagrams by taking advantage of a collision-sensing controller and explore their underlying smooth structures through a continuous motion. Recent advances of interaction techniques have been made that allow progress to be made in this direction. Pseudo-haptics that simulates haptic effects using pure visual feedback can be used to develop such an interactive system. This paper outlines one such pseudo-haptic knot diagram interface. Our interface derives from the familiar pencil-and-paper process of drawing 2D knot diagrams and provides haptic-like sensations to facilitate the creation and exploration of knot diagrams. A centerpiece of the interaction model simulates a "physically" reactive mouse cursor, which is exploited to resolve the apparent conflict between the continuous structure of the actual smooth knot and the visual discontinuities in the knot diagram representation. Another value in exploiting pseudo-haptics is that an acceleration (or deceleration) of the mouse cursor (or surface locator) can be used to indicate the slope of the curve (or surface) of whom the projective image is being explored. By exploiting these additional visual cues, we proceed to a full-featured extension to a pseudo-haptic 4D visualization system that simulates the continuous navigation on 4D objects and allows us to sense the bumps and holes in the fourth dimension. Preliminary tests of the software show that main features of the interface overcome some expected perceptual limitations in our interaction with 2D knot diagrams of 3D knots and 3D projective images of 4D mathematical objects.

Inertial sensors as real-time feedback improve learning posterior-anterior thoracic manipulation: a randomized controlled trial.

PubMed

Cuesta-Vargas, Antonio I; González-Sánchez, Manuel; Lenfant, Yves

2015-01-01

The purpose of this study was to analyze the effect of real-time feedback on the learning process for posterior-anterior thoracic manipulation (PATM) comparing 2 undergraduate physiotherapy student groups. The study design was a randomized controlled trial in an educational setting. Sixty-one undergraduate physiotherapy students were divided randomly into 2 groups, G1 (n = 31; group without feedback in real time) and G2 (n = 30; group with real-time feedback) participated in this randomized controlled trial. Two groups of physiotherapy students learned PATM, one using a traditional method and the other using real-time feedback (inertial sensor). Measures were obtained preintervention and postintervention. Intragroup preintervention and postintervention and intergroup postintervention scores were calculated. An analysis of the measures' stability was developed through an interclass correlation index. Time, displacement and velocity, and improvement (only between groups) to reach maximum peak and to reach minimum peak from maximum peak, total manipulation time, and stability of all outcome measures were the outcome measures. Statistically significant differences were found in all variables analyzed (intragroup and intergroup) in favor of G2. The values of interclass correlation ranged from 0.627 to 0.706 (G1) and between 0.881 and 0.997 (G2). This study found that the learning process for PATM is facilitated when the student receives real-time feedback. Copyright © 2015 National University of Health Sciences. Published by Elsevier Inc. All rights reserved.

Safety implications of providing real-time feedback to distracted drivers.

PubMed

Donmez, Birsen; Boyle, Linda Ng; Lee, John D

2007-05-01

A driving simulator study was conducted to assess whether real-time feedback on a driver's state can influence the driver's interaction with in-vehicle information systems (IVIS). Previous studies have shown that IVIS tasks can undermine driver safety by increasing driver distraction. Thus, mitigating driver distraction using a feedback mechanism appears promising. This study was designed to test real-time feedback that alerts drivers based on their off-road eye glances. Feedback was displayed in two display locations (vehicle-centered, and IVIS-centered) to 16 young and 13 middle-aged drivers. Distraction was observed as problematic for both age groups with delayed responses to a lead vehicle-braking event as indicated by delayed accelerator releases. Significant benefits were not observed for braking and steering behavior for this experiment, but there was a significant change in drivers' interaction with IVIS. When given feedback on their distracted state, drivers looked at the in-vehicle display less frequently regardless of where feedback was displayed in the vehicle. This indicates that real-time feedback based on the driver state can positively alter driver's engagement in distracting activities, helping them attend better to the roadway.

Mixed reality ventriculostomy simulation: experience in neurosurgical residency.

PubMed

Hooten, Kristopher G; Lister, J Richard; Lombard, Gwen; Lizdas, David E; Lampotang, Samsun; Rajon, Didier A; Bova, Frank; Murad, Gregory J A

2014-12-01

Medicine and surgery are turning toward simulation to improve on limited patient interaction during residency training. Many simulators today use virtual reality with augmented haptic feedback with little to no physical elements. In a collaborative effort, the University of Florida Department of Neurosurgery and the Center for Safety, Simulation & Advanced Learning Technologies created a novel "mixed" physical and virtual simulator to mimic the ventriculostomy procedure. The simulator contains all the physical components encountered for the procedure with superimposed 3-D virtual elements for the neuroanatomical structures. To introduce the ventriculostomy simulator and its validation as a necessary training tool in neurosurgical residency. We tested the simulator in more than 260 residents. An algorithm combining time and accuracy was used to grade performance. Voluntary postperformance surveys were used to evaluate the experience. Results demonstrate that more experienced residents have statistically significant better scores and completed the procedure in less time than inexperienced residents. Survey results revealed that most residents agreed that practice on the simulator would help with future ventriculostomies. This mixed reality simulator provides a real-life experience, and will be an instrumental tool in training the next generation of neurosurgeons. We have now implemented a standard where incoming residents must prove efficiency and skill on the simulator before their first interaction with a patient.

Evaluation of Augmented Reality Feedback in Surgical Training Environment.

PubMed

Zahiri, Mohsen; Nelson, Carl A; Oleynikov, Dmitry; Siu, Ka-Chun

2018-02-01

Providing computer-based laparoscopic surgical training has several advantages that enhance the training process. Self-evaluation and real-time performance feedback are 2 of these advantages, which avoid dependency of trainees on expert feedback. The goal of this study was to investigate the use of a visual time indicator as real-time feedback correlated with the laparoscopic surgical training. Twenty novices participated in this study working with (and without) different presentations of time indicators. They performed a standard peg transfer task, and their completion times and muscle activity were recorded and compared. Also of interest was whether the use of this type of feedback induced any side effect in terms of motivation or muscle fatigue. Of the 20 participants, 15 (75%) preferred using a time indicator in the training process rather than having no feedback. However, time to task completion showed no significant difference in performance with the time indicator; furthermore, no significant differences in muscle activity or muscle fatigue were detected with/without time feedback. The absence of significant difference between task performance with/without time feedback shows that using visual real-time feedback can be included in surgical training based on user preference. Trainees may benefit from this type of feedback in the form of increased motivation. The extent to which this can influence training frequency leading to performance improvement is a question for further study.

Effect of Real-Time Feedback on Screw Placement Into Synthetic Cancellous Bone.

PubMed

Gustafson, Peter A; Geeslin, Andrew G; Prior, David M; Chess, Joseph L

2016-08-01

The objective of this study is to evaluate whether real-time torque feedback may reduce the occurrence of stripping when inserting nonlocking screws through fracture plates into synthetic cancellous bone. Five attending orthopaedic surgeons and 5 senior level orthopaedic residents inserted 8 screws in each phase. In phase I, screws were inserted without feedback simulating conventional techniques. In phase II, screws were driven with visual torque feedback. In phase III, screws were again inserted with conventional techniques. Comparison of these 3 phases with respect to screw insertion torque, surgeon rank, and perception of stripping was used to establish the effects of feedback. Seventy-three of 239 screws resulted in stripping. During the first phase, no feedback was provided and the overall strip rate was 41.8%; this decreased to 15% with visual feedback (P < 0.001) and returned to 35% when repeated without feedback. With feedback, a lower average torque was applied over a narrower torque distribution. Residents stripped 40.8% of screws compared with 20.2% for attending surgeons. Surgeons were poor at perceiving whether they stripped. Prevention and identification of stripping is influenced by surgeon perception of tactile sensation. This is significantly improved with utilization of real-time visual feedback of a torque versus roll curve. This concept of real-time feedback seems beneficial toward performance in synthetic cancellous bone and may lead to improved fixation in cancellous bone in a surgical setting.

Simulation System for Training in Laparoscopic Surgery

NASA Technical Reports Server (NTRS)

Basdogan, Cagatay; Ho, Chih-Hao

2003-01-01

A computer-based simulation system creates a visual and haptic virtual environment for training a medical practitioner in laparoscopic surgery. Heretofore, it has been common practice to perform training in partial laparoscopic surgical procedures by use of a laparoscopic training box that encloses a pair of laparoscopic tools, objects to be manipulated by the tools, and an endoscopic video camera. However, the surgical procedures simulated by use of a training box are usually poor imitations of the actual ones. The present computer-based system improves training by presenting a more realistic simulated environment to the trainee. The system includes a computer monitor that displays a real-time image of the affected interior region of the patient, showing laparoscopic instruments interacting with organs and tissues, as would be viewed by use of an endoscopic video camera and displayed to a surgeon during a laparoscopic operation. The system also includes laparoscopic tools that the trainee manipulates while observing the image on the computer monitor (see figure). The instrumentation on the tools consists of (1) position and orientation sensors that provide input data for the simulation and (2) actuators that provide force feedback to simulate the contact forces between the tools and tissues. The simulation software includes components that model the geometries of surgical tools, components that model the geometries and physical behaviors of soft tissues, and components that detect collisions between them. Using the measured positions and orientations of the tools, the software detects whether they are in contact with tissues. In the event of contact, the deformations of the tissues and contact forces are computed by use of the geometric and physical models. The image on the computer screen shows tissues deformed accordingly, while the actuators apply the corresponding forces to the distal ends of the tools. For the purpose of demonstration, the system has been set up to simulate the insertion of a flexible catheter in a bile duct. [As thus configured, the system can also be used to simulate other endoscopic procedures (e.g., bronchoscopy and colonoscopy) that include the insertion of flexible tubes into flexible ducts.] A hybrid approach has been followed in developing the software for real-time simulation of the visual and haptic interactions (1) between forceps and the catheter, (2) between the forceps and the duct, and (3) between the catheter and the duct. The deformations of the duct are simulated by finite-element and modalanalysis procedures, using only the most significant vibration modes of the duct for computing deformations and interaction forces. The catheter is modeled as a set of virtual particles uniformly distributed along the center line of the catheter and connected to each other via linear and torsional springs and damping elements. The interactions between the forceps and the duct as well as the catheter are simulated by use of a ray-based haptic-interaction- simulating technique in which the forceps are modeled as connected line segments.

Surgical model-view-controller simulation software framework for local and collaborative applications

PubMed Central

Sankaranarayanan, Ganesh; Halic, Tansel; Arikatla, Venkata Sreekanth; Lu, Zhonghua; De, Suvranu

2010-01-01

Purpose Surgical simulations require haptic interactions and collaboration in a shared virtual environment. A software framework for decoupled surgical simulation based on a multi-controller and multi-viewer model-view-controller (MVC) pattern was developed and tested. Methods A software framework for multimodal virtual environments was designed, supporting both visual interactions and haptic feedback while providing developers with an integration tool for heterogeneous architectures maintaining high performance, simplicity of implementation, and straightforward extension. The framework uses decoupled simulation with updates of over 1,000 Hz for haptics and accommodates networked simulation with delays of over 1,000 ms without performance penalty. Results The simulation software framework was implemented and was used to support the design of virtual reality-based surgery simulation systems. The framework supports the high level of complexity of such applications and the fast response required for interaction with haptics. The efficacy of the framework was tested by implementation of a minimally invasive surgery simulator. Conclusion A decoupled simulation approach can be implemented as a framework to handle simultaneous processes of the system at the various frame rates each process requires. The framework was successfully used to develop collaborative virtual environments (VEs) involving geographically distributed users connected through a network, with the results comparable to VEs for local users. PMID:20714933

Surgical model-view-controller simulation software framework for local and collaborative applications.

PubMed

Maciel, Anderson; Sankaranarayanan, Ganesh; Halic, Tansel; Arikatla, Venkata Sreekanth; Lu, Zhonghua; De, Suvranu

2011-07-01

Surgical simulations require haptic interactions and collaboration in a shared virtual environment. A software framework for decoupled surgical simulation based on a multi-controller and multi-viewer model-view-controller (MVC) pattern was developed and tested. A software framework for multimodal virtual environments was designed, supporting both visual interactions and haptic feedback while providing developers with an integration tool for heterogeneous architectures maintaining high performance, simplicity of implementation, and straightforward extension. The framework uses decoupled simulation with updates of over 1,000 Hz for haptics and accommodates networked simulation with delays of over 1,000 ms without performance penalty. The simulation software framework was implemented and was used to support the design of virtual reality-based surgery simulation systems. The framework supports the high level of complexity of such applications and the fast response required for interaction with haptics. The efficacy of the framework was tested by implementation of a minimally invasive surgery simulator. A decoupled simulation approach can be implemented as a framework to handle simultaneous processes of the system at the various frame rates each process requires. The framework was successfully used to develop collaborative virtual environments (VEs) involving geographically distributed users connected through a network, with the results comparable to VEs for local users.

Rehabilitation Program Integrating Virtual Environment to Improve Orientation and Mobility Skills for People Who Are Blind

PubMed Central

Lahav, Orly; Schloerb, David W.; Srinivasan, Mandayam A.

2014-01-01

This paper presents the integration of a virtual environment (BlindAid) in an orientation and mobility rehabilitation program as a training aid for people who are blind. BlindAid allows the users to interact with different virtual structures and objects through auditory and haptic feedback. This research explores if and how use of the BlindAid in conjunction with a rehabilitation program can help people who are blind train themselves in familiar and unfamiliar spaces. The study, focused on nine participants who were congenitally, adventitiously, and newly blind, during their orientation and mobility rehabilitation program at the Carroll Center for the Blind (Newton, Massachusetts, USA). The research was implemented using virtual environment (VE) exploration tasks and orientation tasks in virtual environments and real spaces. The methodology encompassed both qualitative and quantitative methods, including interviews, a questionnaire, videotape recording, and user computer logs. The results demonstrated that the BlindAid training gave participants additional time to explore the virtual environment systematically. Secondly, it helped elucidate several issues concerning the potential strengths of the BlindAid system as a training aid for orientation and mobility for both adults and teenagers who are congenitally, adventitiously, and newly blind. PMID:25284952

A Novel Haptic Interactive Approach to Simulation of Surgery Cutting Based on Mesh and Meshless Models

PubMed Central

Liu, Peter X.; Lai, Pinhua; Xu, Shaoping; Zou, Yanni

2018-01-01

In the present work, the majority of implemented virtual surgery simulation systems have been based on either a mesh or meshless strategy with regard to soft tissue modelling. To take full advantage of the mesh and meshless models, a novel coupled soft tissue cutting model is proposed. Specifically, the reconstructed virtual soft tissue consists of two essential components. One is associated with surface mesh that is convenient for surface rendering and the other with internal meshless point elements that is used to calculate the force feedback during cutting. To combine two components in a seamless way, virtual points are introduced. During the simulation of cutting, the Bezier curve is used to characterize smooth and vivid incision on the surface mesh. At the same time, the deformation of internal soft tissue caused by cutting operation can be treated as displacements of the internal point elements. Furthermore, we discussed and proved the stability and convergence of the proposed approach theoretically. The real biomechanical tests verified the validity of the introduced model. And the simulation experiments show that the proposed approach offers high computational efficiency and good visual effect, enabling cutting of soft tissue with high stability. PMID:29850006

Surgeon Training in Telerobotic Surgery via a Hardware-in-the-Loop Simulator

PubMed Central

Alemzadeh, Homa; Chen, Daniel; Kalbarczyk, Zbigniew; Iyer, Ravishankar K.; Kesavadas, Thenkurussi

2017-01-01

This work presents a software and hardware framework for a telerobotic surgery safety and motor skill training simulator. The aims are at providing trainees a comprehensive simulator for acquiring essential skills to perform telerobotic surgery. Existing commercial robotic surgery simulators lack features for safety training and optimal motion planning, which are critical factors in ensuring patient safety and efficiency in operation. In this work, we propose a hardware-in-the-loop simulator directly introducing these two features. The proposed simulator is built upon the Raven-II™ open source surgical robot, integrated with a physics engine and a safety hazard injection engine. Also, a Fast Marching Tree-based motion planning algorithm is used to help trainee learn the optimal instrument motion patterns. The main contributions of this work are (1) reproducing safety hazards events, related to da Vinci™ system, reported to the FDA MAUDE database, with a novel haptic feedback strategy to provide feedback to the operator when the underlying dynamics differ from the real robot's states so that the operator will be aware and can mitigate the negative impact of the safety-critical events, and (2) using motion planner to generate semioptimal path in an interactive robotic surgery training environment. PMID:29065635

What is going on in augmented reality simulation in laparoscopic surgery?

PubMed

Botden, Sanne M B I; Jakimowicz, Jack J

2009-08-01

To prevent unnecessary errors and adverse results of laparoscopic surgery, proper training is of paramount importance. A safe way to train surgeons for laparoscopic skills is simulation. For this purpose traditional box trainers are often used, however they lack objective assessment of performance. Virtual reality laparoscopic simulators assess performance, but lack realistic haptic feedback. Augmented reality (AR) combines a virtual reality (VR) setting with real physical materials, instruments, and feedback. This article presents the current developments in augmented reality laparoscopic simulation. Pubmed searches were performed to identify articles regarding surgical simulation and augmented reality. Identified companies manufacturing an AR laparoscopic simulator received the same questionnaire referring to the features of the simulator. Seven simulators that fitted the definition of augmented reality were identified during the literature search. Five of the approached manufacturers returned a completed questionnaire, of which one simulator appeared to be VR and was therefore not applicable for this review. Several augmented reality simulators have been developed over the past few years and they are improving rapidly. We recommend the development of AR laparoscopic simulators for component tasks of procedural training. AR simulators should be implemented in current laparoscopic training curricula, in particular for laparoscopic suturing training.

Quality of closed chest compression on a manikin in ambulance vehicles and flying helicopters with a real time automated feedback.

PubMed

Havel, Christof; Schreiber, Wolfgang; Trimmel, Helmut; Malzer, Reinhard; Haugk, Moritz; Richling, Nina; Riedmüller, Eva; Sterz, Fritz; Herkner, Harald

2010-01-01

Automated verbal and visual feedback improves quality of resuscitation in out-of-hospital cardiac arrest and was proven to increase short-term survival. Quality of resuscitation may be hampered in more difficult situations like emergency transportation. Currently there is no evidence if feedback devices can improve resuscitation quality during different modes of transportation. To assess the effect of real time automated feedback on the quality of resuscitation in an emergency transportation setting. Randomised cross-over trial. Medical University of Vienna, Vienna Municipal Ambulance Service and Helicopter Emergency Medical Service Unit (Christophorus Flugrettungsverein) in September 2007. European Resuscitation Council (ERC) certified health care professionals performing CPR in a flying helicopter and in a moving ambulance vehicle on a manikin with human-like chest properties. CPR sessions, with real time automated feedback as the intervention and standard CPR without feedback as control. Quality of chest compression during resuscitation. Feedback resulted in less deviation from ideal compression rate 100 min(-1) (9+/-9 min(-1), p<0.0001) with this effect becoming steadily larger over time. Applied work was less in the feedback group compared to controls (373+/-448 cm x compression; p<0.001). Feedback did not influence ideal compression depth significantly. There was some indication of a learning effect of the feedback device. Real time automated feedback improves certain aspects of CPR quality in flying helicopters and moving ambulance vehicles. The effect of feedback guidance was most pronounced for chest compression rate. Copyright 2009 Elsevier Ireland Ltd. All rights reserved.

Designing feedback to mitigate teen distracted driving: A social norms approach.

PubMed

Merrikhpour, Maryam; Donmez, Birsen

2017-07-01

The purpose of this research is to investigate teens' perceived social norms and whether providing normative information can reduce distracted driving behaviors among them. Parents are among the most important social referents for teens; they have significant influences on teens' driving behaviors, including distracted driving which significantly contributes to teens' crash risks. Social norms interventions have been successfully applied in various domains including driving; however, this approach is yet to be explored for mitigating driver distraction among teens. Forty teens completed a driving simulator experiment while performing a self-paced visual-manual secondary task in four between-subject conditions: a) social norms feedback that provided a report at the end of each drive on teens' distracted driving behavior, comparing their distraction engagement to their parent's, b) post-drive feedback that provided just the report on teens' distracted driving behavior without information on their parents, c) real-time feedback in the form of auditory warnings based on eyes of road-time, and d) no feedback as control. Questionnaires were administered to collect data on these teens' and their parents' self-reported engagement in driver distractions and the associated social norms. Social norms and real-time feedback conditions resulted in significantly smaller average off-road glance duration, rate of long (>2s) off-road glances, and standard deviation of lane position compared to no feedback. Further, social norms feedback decreased brake response time and percentage of time not looking at the road compared to no feedback. No major effect was observed for post-drive feedback. Questionnaire results suggest that teens appeared to overestimate parental norms, but no effect of feedback was found on their perceptions. Feedback systems that leverage social norms can help mitigate driver distraction among teens. Overall, both social norms and real-time feedback induced positive driving behaviors, with social norms feedback outperforming real-time feedback. Copyright © 2017 Elsevier Ltd. All rights reserved.

Electrorheological Fluid Based Force Feedback Device

NASA Technical Reports Server (NTRS)

Pfeiffer, Charles; Bar-Cohen, Yoseph; Mavroidis, Constantinos; Dolgin, Benjamin

1999-01-01

Parallel to the efforts to develop fully autonomous robots, it is increasingly being realized that there are applications where it is essential to have a fully controlled robot and "feel" its operating conditions, i.e. telepresence. This trend is a result of the increasing efforts to address tasks where humans can perform significantly better but, due to associated hazards, distance, physical limitations and other causes, only robots can be employed to perform these tasks. Such robots need to be assisted by a human that remotely controls the operation. To address the goal of operating robots as human surrogates, the authors launched a study of mechanisms that provide mechanical feedback. For this purpose, electrorheological fluids (ERF) are being investigated for the potential application as miniature haptic devices. This family of electroactive fluids has the property of changing the viscosity during electrical stimulation. Consequently, ERF can be used to produce force feedback haptic devices for tele-operated control of medical and space robotic systems. Forces applied at the robot end-effector due to a compliant environment are reflected to the user using an ERF device where a change in the system viscosity will occur proportionally to the transmitted force. Analytical model and control algorithms are being developed taking into account the non-linearities of these type of devices. This paper will describe the concept and the developed mechanism of ERF based force feedback. The test process and the physical properties of this device will be described and the results of preliminary tests will be presented.

Poor shape perception is the reason reaches-to-grasp are visually guided online.

PubMed

Lee, Young-Lim; Crabtree, Charles E; Norman, J Farley; Bingham, Geoffrey P

2008-08-01

Both judgment studies and studies of feedforward reaching have shown that the visual perception of object distance, size, and shape are inaccurate. However, feedback has been shown to calibrate feedfoward reaches-to-grasp to make them accurate with respect to object distance and size. We now investigate whether shape perception (in particular, the aspect ratio of object depth to width) can be calibrated in the context of reaches-to-grasp. We used cylindrical objects with elliptical cross-sections of varying eccentricity. Our participants reached to grasp the width or the depth of these objects with the index finger and thumb. The maximum grasp aperture and the terminal grasp aperture were used to evaluate perception. Both occur before the hand has contacted an object. In Experiments 1 and 2, we investigated whether perceived shape is recalibrated by distorted haptic feedback. Although somewhat equivocal, the results suggest that it is not. In Experiment 3, we tested the accuracy of feedforward grasping with respect to shape with haptic feedback to allow calibration. Grasping was inaccurate in ways comparable to findings in shape perception judgment studies. In Experiment 4, we hypothesized that online guidance is needed for accurate grasping. Participants reached to grasp either with or without vision of the hand. The result was that the former was accurate, whereas the latter was not. We conclude that shape perception is not calibrated by feedback from reaches-to-grasp and that online visual guidance is required for accurate grasping because shape perception is poor.

Virtual Proprioception for eccentric training.

PubMed

LeMoyne, Robert; Mastroianni, Timothy

2017-07-01

Wireless inertial sensors enable quantified feedback, which can be applied to evaluate the efficacy of therapy and rehabilitation. In particular eccentric training promotes a beneficial rehabilitation and strength training strategy. Virtual Proprioception for eccentric training applies real-time feedback from a wireless gyroscope platform enabled through a software application for a smartphone. Virtual Proprioception for eccentric training is applied to the eccentric phase of a biceps brachii strength training and contrasted to a biceps brachii strength training scenario without feedback. During the operation of Virtual Proprioception for eccentric training the intent is to not exceed a prescribed gyroscope signal threshold based on the real-time presentation of the gyroscope signal, in order to promote the eccentric aspect of the strength training endeavor. The experimental trial data is transmitted wireless through connectivity to the Internet as an email attachment for remote post-processing. A feature set is derived from the gyroscope signal for machine learning classification of the two scenarios of Virtual Proprioception real-time feedback for eccentric training and eccentric training without feedback. Considerable classification accuracy is achieved through the application of a multilayer perceptron neural network for distinguishing between the Virtual Proprioception real-time feedback for eccentric training and eccentric training without feedback.

Pseudo-Haptic Feedback for Promoting Narrative Comprehension

ERIC Educational Resources Information Center

Umetsu, Kazuaki; Kashihara, Akihiro

2017-01-01

Skill in reading comprehension requires reading sentences to understand an intention embedded between the lines. In the case of narrative, it is particularly necessary to read a narrative and find essential concepts such as emotions of the characters embedded between the lines for comprehending an intention of the narrative. In this work, we focus…

Have 3D, Will Travel

ERIC Educational Resources Information Center

Duncan, Mike R.; Birrell, Bob; Williams, Toni

2005-01-01

Virtual Reality (VR) is primarily a visual technology. Elements such as haptics (touch feedback) and sound can augment an experience, but the visual cues are the prime driver of what an audience will experience from a VR presentation. At its inception in 2001 the Centre for Advanced Visualization (CFAV) at Niagara College of Arts and Technology…

An REU Experience with Micro Assembly Workcell Research

ERIC Educational Resources Information Center

Stapleton, William; Asiabanpour, Bahram; Jimenez, Jesus; Um, Dugan

2010-01-01

Under an NSF REU center grant REU-0755355 entitled "Micro/Nano Assembly Workcell Via Micro Visual Sensing and Haptic Feedback", Texas A&M University-Corpus Christi and Texas State University-San Marcos collaboratively hosted two groups of 10 students from different backgrounds for 10 weeks each in Summer 2008 and 2009 respectively.…

PROJECT HEAVEN: Preoperative Training in Virtual Reality

PubMed Central

Iamsakul, Kiratipath; Pavlovcik, Alexander V.; Calderon, Jesus I.; Sanderson, Lance M.

2017-01-01

A cephalosomatic anastomosis (CSA; also called HEAVEN: head anastomosis venture) has been proposed as an option for patients with neurological impairments, such as spinal cord injury (SCI), and terminal medical illnesses, for which medicine is currently powerless. Protocols to prepare a patient for life after CSA do not currently exist. However, methods used in conventional neurorehabilitation can be used as a reference for developing preparatory training. Studies on virtual reality (VR) technologies have documented VR's ability to enhance rehabilitation and improve the quality of recovery in patients with neurological disabilities. VR-augmented rehabilitation resulted in increased motivation towards performing functional training and improved the biopsychosocial state of patients. In addition, VR experiences coupled with haptic feedback promote neuroplasticity, resulting in the recovery of motor functions in neurologically-impaired individuals. To prepare the recipient psychologically for life after CSA, the development of VR experiences paired with haptic feedback is proposed. This proposal aims to innovate techniques in conventional neurorehabilitation to implement preoperative psychological training for the recipient of HEAVEN. Recipient's familiarity to body movements will prevent unexpected psychological reactions from occurring after the HEAVEN procedure. PMID:28540125

PROJECT HEAVEN: Preoperative Training in Virtual Reality.

PubMed

Iamsakul, Kiratipath; Pavlovcik, Alexander V; Calderon, Jesus I; Sanderson, Lance M

2017-01-01

A cephalosomatic anastomosis (CSA; also called HEAVEN: head anastomosis venture) has been proposed as an option for patients with neurological impairments, such as spinal cord injury (SCI), and terminal medical illnesses, for which medicine is currently powerless. Protocols to prepare a patient for life after CSA do not currently exist. However, methods used in conventional neurorehabilitation can be used as a reference for developing preparatory training. Studies on virtual reality (VR) technologies have documented VR's ability to enhance rehabilitation and improve the quality of recovery in patients with neurological disabilities. VR-augmented rehabilitation resulted in increased motivation towards performing functional training and improved the biopsychosocial state of patients. In addition, VR experiences coupled with haptic feedback promote neuroplasticity, resulting in the recovery of motor functions in neurologically-impaired individuals. To prepare the recipient psychologically for life after CSA, the development of VR experiences paired with haptic feedback is proposed. This proposal aims to innovate techniques in conventional neurorehabilitation to implement preoperative psychological training for the recipient of HEAVEN. Recipient's familiarity to body movements will prevent unexpected psychological reactions from occurring after the HEAVEN procedure.

SeaTouch: A Haptic and Auditory Maritime Environment for Non Visual Cognitive Mapping of Blind Sailors

NASA Astrophysics Data System (ADS)

Simonnet, Mathieu; Jacobson, Dan; Vieilledent, Stephane; Tisseau, Jacques

Navigating consists of coordinating egocentric and allocentric spatial frames of reference. Virtual environments have afforded researchers in the spatial community with tools to investigate the learning of space. The issue of the transfer between virtual and real situations is not trivial. A central question is the role of frames of reference in mediating spatial knowledge transfer to external surroundings, as is the effect of different sensory modalities accessed in simulated and real worlds. This challenges the capacity of blind people to use virtual reality to explore a scene without graphics. The present experiment involves a haptic and auditory maritime virtual environment. In triangulation tasks, we measure systematic errors and preliminary results show an ability to learn configurational knowledge and to navigate through it without vision. Subjects appeared to take advantage of getting lost in an egocentric “haptic” view in the virtual environment to improve performances in the real environment.

Real-time recognition of feedback error-related potentials during a time-estimation task.

PubMed

Lopez-Larraz, Eduardo; Iturrate, Iñaki; Montesano, Luis; Minguez, Javier

2010-01-01

Feedback error-related potentials are a promising brain process in the field of rehabilitation since they are related to human learning. Due to the fact that many therapeutic strategies rely on the presentation of feedback stimuli, potentials generated by these stimuli could be used to ameliorate the patient's progress. In this paper we propose a method that can identify, in real-time, feedback evoked potentials in a time-estimation task. We have tested our system with five participants in two different days with a separation of three weeks between them, achieving a mean single-trial detection performance of 71.62% for real-time recognition, and 78.08% in offline classification. Additionally, an analysis of the stability of the signal between the two days is performed, suggesting that the feedback responses are stable enough to be used without the needing of training again the user.

Sensewheel: an adjunct to wheelchair skills training

PubMed Central

Taylor, Stephen J.G.; Holloway, Catherine

2016-01-01

The purpose of this Letter was to investigate the influence of real-time verbal feedback to optimise push arc during over ground manual wheelchair propulsion. Ten healthy non-wheelchair users pushed a manual wheelchair for a distance of 25 m on level paving, initially with no feedback and then with real-time verbal feedback aimed at controlling push arc within a range of 85°–100°. The real-time feedback was provided by a physiotherapist walking behind the wheelchair, viewing real-time data on a tablet personal computer received from the Sensewheel, a lightweight instrumented wheelchair wheel. The real-time verbal feedback enabled the participants to significantly increase their push arc. This increase in push arc resulted in a non-significant reduction in push rate and a significant increase in peak force application. The intervention enabled participants to complete the task at a higher mean velocity using significantly fewer pushes. This was achieved via a significant increase in the power generated during the push phase. This Letter identifies that a lightweight instrumented wheelchair wheel such as the Sensewheel is a useful adjunct to wheelchair skills training. Targeting the optimisation of push arc resulted in beneficial changes in propulsion technique. PMID:28008362

Eye-in-Hand Manipulation for Remote Handling: Experimental Setup

NASA Astrophysics Data System (ADS)

Niu, Longchuan; Suominen, Olli; Aref, Mohammad M.; Mattila, Jouni; Ruiz, Emilio; Esque, Salvador

2018-03-01

A prototype for eye-in-hand manipulation in the context of remote handling in the International Thermonuclear Experimental Reactor (ITER)1 is presented in this paper. The setup consists of an industrial robot manipulator with a modified open control architecture and equipped with a pair of stereoscopic cameras, a force/torque sensor, and pneumatic tools. It is controlled through a haptic device in a mock-up environment. The industrial robot controller has been replaced by a single industrial PC running Xenomai that has a real-time connection to both the robot controller and another Linux PC running as the controller for the haptic device. The new remote handling control environment enables further development of advanced control schemes for autonomous and semi-autonomous manipulation tasks. This setup benefits from a stereovision system for accurate tracking of the target objects with irregular shapes. The overall environmental setup successfully demonstrates the required robustness and precision that remote handling tasks need.

Virtual environment navigation with look-around mode to explore new real spaces by people who are blind.

PubMed

Lahav, Orly; Gedalevitz, Hadas; Battersby, Steven; Brown, David; Evett, Lindsay; Merritt, Patrick

2018-05-01

This paper examines the ability of people who are blind to construct a mental map and perform orientation tasks in real space by using Nintendo Wii technologies to explore virtual environments. The participant explores new spaces through haptic and auditory feedback triggered by pointing or walking in the virtual environments and later constructs a mental map, which can be used to navigate in real space. The study included 10 participants who were congenitally or adventitiously blind, divided into experimental and control groups. The research was implemented by using virtual environments exploration and orientation tasks in real spaces, using both qualitative and quantitative methods in its methodology. The results show that the mode of exploration afforded to the experimental group is radically new in orientation and mobility training; as a result 60% of the experimental participants constructed mental maps that were based on map model, compared with only 30% of the control group participants. Using technology that enabled them to explore and to collect spatial information in a way that does not exist in real space influenced the ability of the experimental group to construct a mental map based on the map model. Implications for rehabilitation The virtual cane system for the first time enables people who are blind to explore and collect spatial information via the look-around mode in addition to the walk-around mode. People who are blind prefer to use look-around mode to explore new spaces, as opposed to the walking mode. Although the look-around mode requires users to establish a complex collecting and processing procedure for the spatial data, people who are blind using this mode are able to construct a mental map as a map model. For people who are blind (as for the sighted) construction of a mental map based on map model offers more flexibility in choosing a walking path in a real space, accounting for changes that occur in the space.

The use of real-time ultrasound feedback in teaching abdominal hollowing exercises to healthy subjects.

PubMed

Henry, Sharon M; Westervelt, Karen C

2005-06-01

Randomized controlled trial. To determine if supplementing typical clinical instruction with real-time ultrasound feedback facilitates performance and retention of the abdominal hollowing exercise (AHE). Increasingly clinicians are using real-time ultrasound imaging as a form of feedback when teaching patients trunk stabilization exercises; however, there has been no justification for this practice. Forty-eight subjects were divided randomly into 3 groups that received different types of feedback: group 1 received minimal verbal feedback, group 2 received verbal and palpatory feedback, and group 3 received real-time ultrasound, verbal, and palpatory feedback. If the subject performed 3 consecutive correct AHEs during the initial session, she/he returned for a retention test. The performance of 3 consecutive, correct AHEs was the criterion measure; the number of trials to criterion was also recorded during the initial and retention test sessions. The ability to perform the AHE differed among groups (P<.001). During the initial session, 12.5% of subjects in group 1, 50.0% of subjects in group 2, and 87.5% of subjects in group 3 were able to perform 3 consecutive AHEs. Group 3 subjects achieved the criterion in fewer trials than the other 2 groups (P = .0006). No differences among groups were found for the retention testing; however, low power due to fewer subjects precluded a strong interpretation of this finding. Real-time ultrasound feedback can decrease the number of trials needed to consistently perform the AHE; however, the data are inconclusive with regard to retention of this skill.

Shadow-driven 4D haptic visualization.

PubMed

Zhang, Hui; Hanson, Andrew

2007-01-01

Just as we can work with two-dimensional floor plans to communicate 3D architectural design, we can exploit reduced-dimension shadows to manipulate the higher-dimensional objects generating the shadows. In particular, by taking advantage of physically reactive 3D shadow-space controllers, we can transform the task of interacting with 4D objects to a new level of physical reality. We begin with a teaching tool that uses 2D knot diagrams to manipulate the geometry of 3D mathematical knots via their projections; our unique 2D haptic interface allows the user to become familiar with sketching, editing, exploration, and manipulation of 3D knots rendered as projected imageson a 2D shadow space. By combining graphics and collision-sensing haptics, we can enhance the 2D shadow-driven editing protocol to successfully leverage 2D pen-and-paper or blackboard skills. Building on the reduced-dimension 2D editing tool for manipulating 3D shapes, we develop the natural analogy to produce a reduced-dimension 3D tool for manipulating 4D shapes. By physically modeling the correct properties of 4D surfaces, their bending forces, and their collisions in the 3D haptic controller interface, we can support full-featured physical exploration of 4D mathematical objects in a manner that is otherwise far beyond the experience accessible to human beings. As far as we are aware, this paper reports the first interactive system with force-feedback that provides "4D haptic visualization" permitting the user to model and interact with 4D cloth-like objects.

Real-time augmented feedback benefits robotic laparoscopic training.

PubMed

Judkins, Timothy N; Oleynikov, Dmitry; Stergiou, Nick

2006-01-01

Robotic laparoscopic surgery has revolutionized minimally invasive surgery for treatment of abdominal pathologies. However, current training techniques rely on subjective evaluation. There is a lack of research on the type of tasks that should be used for training. Robotic surgical systems also do not currently have the ability to provide feedback to the surgeon regarding success of performing tasks. We trained medical students on three laparoscopic tasks and provided real-time feedback of performance during training. We found that real-time feedback can benefit training if the feedback provides information that is not available through other means (grip force). Subjects that received grip force feedback applied less force when the feedback was removed. Other forms of feedback (speed and relative phase) did not aid or impede training. Secondly, a relatively short training period (10 trials for each task) significantly improved most objective measures of performance. We also showed that robotic surgical performance can be quantitatively measured and evaluated. Providing grip force feedback can make the surgeon more aware of the forces being applied to delicate tissue during surgery.

Exploring laterality and memory effects in the haptic discrimination of verbal and non-verbal shapes.

PubMed

Stoycheva, Polina; Tiippana, Kaisa

2018-03-14

The brain's left hemisphere often displays advantages in processing verbal information, while the right hemisphere favours processing non-verbal information. In the haptic domain due to contra-lateral innervations, this functional lateralization is reflected in a hand advantage during certain functions. Findings regarding the hand-hemisphere advantage for haptic information remain contradictory, however. This study addressed these laterality effects and their interaction with memory retention times in the haptic modality. Participants performed haptic discrimination of letters, geometric shapes and nonsense shapes at memory retention times of 5, 15 and 30 s with the left and right hand separately, and we measured the discriminability index d'. The d' values were significantly higher for letters and geometric shapes than for nonsense shapes. This might result from dual coding (naming + spatial) or/and from a low stimulus complexity. There was no stimulus-specific laterality effect. However, we found a time-dependent laterality effect, which revealed that the performance of the left hand-right hemisphere was sustained up to 15 s, while the performance of the right-hand-left hemisphere decreased progressively throughout all retention times. This suggests that haptic memory traces are more robust to decay when they are processed by the left hand-right hemisphere.

A wearable biofeedback control system based body area network for freestyle swimming.

PubMed

Rui Li; Zibo Cai; WeeSit Lee; Lai, Daniel T H

2016-08-01

Wearable posture measurement units are capable of enabling real-time performance evaluation and providing feedback to end users. This paper presents a wearable feedback prototype designed for freestyle swimming with focus on trunk rotation measurement. The system consists of a nine-degree-of-freedom inertial sensor, which is built in a central data collection and processing unit, and two vibration motors for delivering real-time feedback. Theses devices form a fundamental body area network (BAN). In the experiment setup, four recreational swimmers were asked to do two sets of 4 x 25m freestyle swimming without and with feedback provided respectively. Results showed that real-time biofeedback mechanism improves swimmers kinematic performance by an average of 4.5% reduction in session time. Swimmers can gradually adapt to feedback signals, and the biofeedback control system can be employed in swimmers daily training for fitness maintenance.

Measuring the Impact of Haptic Feedback Using the SOLO Taxonomy

ERIC Educational Resources Information Center

Minogue, James; Jones, Gail

2009-01-01

The application of Biggs' and Collis' Structure of Observed Learning Outcomes taxonomy in the evaluation of student learning about cell membrane transport via a computer-based learning environment is described in this study. Pre-test-post-test comparisons of student outcome data (n = 80) were made across two groups of randomly assigned students:…

Description of European Space Agency (ESA) Remote Manipulator (RM) System Breadboard Currently Under Development for Demonstration of Critical Technology Foreseen to be Used in the Mars Sample Receiving Facility (MSRF)

NASA Astrophysics Data System (ADS)

Vrublevskis, J.; Duncan, S.; Berthoud, L.; Bowman, P.; Hills, R.; McCulloch, Y.; Pisla, D.; Vaida, C.; Gherman, B.; Hofbaur, M.; Dieber, B.; Neythalath, N.; Smith, C.; van Winnendael, M.; Duvet, L.

2018-04-01

In order to avoid the use of 'double walled' gloves, a haptic feedback Remote Manipulation (RM) system rather than a gloved isolator is needed inside a Double Walled Isolator (DWI) to handle a sample returned from Mars.

Review of surgical robotics user interface: what is the best way to control robotic surgery?

PubMed

Simorov, Anton; Otte, R Stephen; Kopietz, Courtni M; Oleynikov, Dmitry

2012-08-01

As surgical robots begin to occupy a larger place in operating rooms around the world, continued innovation is necessary to improve our outcomes. A comprehensive review of current surgical robotic user interfaces was performed to describe the modern surgical platforms, identify the benefits, and address the issues of feedback and limitations of visualization. Most robots currently used in surgery employ a master/slave relationship, with the surgeon seated at a work-console, manipulating the master system and visualizing the operation on a video screen. Although enormous strides have been made to advance current technology to the point of clinical use, limitations still exist. A lack of haptic feedback to the surgeon and the inability of the surgeon to be stationed at the operating table are the most notable examples. The future of robotic surgery sees a marked increase in the visualization technologies used in the operating room, as well as in the robots' abilities to convey haptic feedback to the surgeon. This will allow unparalleled sensation for the surgeon and almost eliminate inadvertent tissue contact and injury. A novel design for a user interface will allow the surgeon to have access to the patient bedside, remaining sterile throughout the procedure, employ a head-mounted three-dimensional visualization system, and allow the most intuitive master manipulation of the slave robot to date.

Haptically facilitated bimanual training combined with augmented visual feedback in moderate to severe hemiplegia.

PubMed

Boos, Amy; Qiu, Qinyin; Fluet, Gerard G; Adamovich, Sergei V

2011-01-01

This study describes the design and feasibility testing of a hand rehabilitation system that provides haptic assistance for hand opening in moderate to severe hemiplegia while subjects attempt to perform bilateral hand movements. A cable-actuated exoskeleton robot assists the subjects in performing impaired finger movements but is controlled by movement of the unimpaired hand. In an attempt to combine the neurophysiological stimuli of bilateral movement and action observation during training, visual feedback of the impaired hand is replaced by feedback of the unimpaired hand, either by using a sagittaly oriented mirror or a virtual reality setup with a pair of virtual hands presented on a flat screen controlled with movement of the unimpaired hand, providing a visual image of their paretic hand moving normally. Joint angles for both hands are measured using data gloves. The system is programmed to maintain a symmetrical relationship between the two hands as they respond to commands to open and close simultaneously. Three persons with moderate to severe hemiplegia secondary to stroke trained with the system for eight, 30 to 60 minute sessions without adverse events. Each demonstrated positive motor adaptations to training. The system was well tolerated by persons with moderate to severe upper extremity hemiplegia. Further testing of its effects on motor ability with a broader range of clinical presentations is indicated.

Fusion of Haptic and Gesture Sensors for Rehabilitation of Bimanual Coordination and Dexterous Manipulation.

PubMed

Yu, Ningbo; Xu, Chang; Li, Huanshuai; Wang, Kui; Wang, Liancheng; Liu, Jingtai

2016-03-18

Disabilities after neural injury, such as stroke, bring tremendous burden to patients, families and society. Besides the conventional constrained-induced training with a paretic arm, bilateral rehabilitation training involves both the ipsilateral and contralateral sides of the neural injury, fitting well with the fact that both arms are needed in common activities of daily living (ADLs), and can promote good functional recovery. In this work, the fusion of a gesture sensor and a haptic sensor with force feedback capabilities has enabled a bilateral rehabilitation training therapy. The Leap Motion gesture sensor detects the motion of the healthy hand, and the omega.7 device can detect and assist the paretic hand, according to the designed cooperative task paradigm, as much as needed, with active force feedback to accomplish the manipulation task. A virtual scenario has been built up, and the motion and force data facilitate instantaneous visual and audio feedback, as well as further analysis of the functional capabilities of the patient. This task-oriented bimanual training paradigm recruits the sensory, motor and cognitive aspects of the patient into one loop, encourages the active involvement of the patients into rehabilitation training, strengthens the cooperation of both the healthy and impaired hands, challenges the dexterous manipulation capability of the paretic hand, suits easy of use at home or centralized institutions and, thus, promises effective potentials for rehabilitation training.

Fusion of Haptic and Gesture Sensors for Rehabilitation of Bimanual Coordination and Dexterous Manipulation

PubMed Central

Yu, Ningbo; Xu, Chang; Li, Huanshuai; Wang, Kui; Wang, Liancheng; Liu, Jingtai

2016-01-01

Disabilities after neural injury, such as stroke, bring tremendous burden to patients, families and society. Besides the conventional constrained-induced training with a paretic arm, bilateral rehabilitation training involves both the ipsilateral and contralateral sides of the neural injury, fitting well with the fact that both arms are needed in common activities of daily living (ADLs), and can promote good functional recovery. In this work, the fusion of a gesture sensor and a haptic sensor with force feedback capabilities has enabled a bilateral rehabilitation training therapy. The Leap Motion gesture sensor detects the motion of the healthy hand, and the omega.7 device can detect and assist the paretic hand, according to the designed cooperative task paradigm, as much as needed, with active force feedback to accomplish the manipulation task. A virtual scenario has been built up, and the motion and force data facilitate instantaneous visual and audio feedback, as well as further analysis of the functional capabilities of the patient. This task-oriented bimanual training paradigm recruits the sensory, motor and cognitive aspects of the patient into one loop, encourages the active involvement of the patients into rehabilitation training, strengthens the cooperation of both the healthy and impaired hands, challenges the dexterous manipulation capability of the paretic hand, suits easy of use at home or centralized institutions and, thus, promises effective potentials for rehabilitation training. PMID:26999149

Human detection and discrimination of tactile repeatability, mechanical backlash, and temporal delay in a combined tactile-kinesthetic haptic display system.

PubMed

Doxon, Andrew J; Johnson, David E; Tan, Hong Z; Provancher, William R

2013-01-01

Many of the devices used in haptics research are over-engineered for the task and are designed with capabilities that go far beyond human perception levels. Designing devices that more closely match the limits of human perception will make them smaller, less expensive, and more useful. However, many device-centric perception thresholds have yet to be evaluated. To this end, three experiments were conducted, using one degree-of-freedom contact location feedback device in combination with a kinesthetic display, to provide a more explicit set of specifications for similar tactile-kinesthetic haptic devices. The first of these experiments evaluated the ability of humans to repeatedly localize tactile cues across the fingerpad. Subjects could localize cues to within 1.3 mm and showed bias toward the center of the fingerpad. The second experiment evaluated the minimum perceptible difference of backlash at the tactile element. Subjects were able to discriminate device backlash in excess of 0.46 mm on low-curvature models and 0.93 mm on high-curvature models. The last experiment evaluated the minimum perceptible difference of system delay between user action and device reaction. Subjects were able to discriminate delays in excess of 61 ms. The results from these studies can serve as the maximum (i.e., most demanding) device specifications for most tactile-kinesthetic haptic systems.

Modeling the forces of cutting with scissors.

PubMed

Mahvash, Mohsen; Voo, Liming M; Kim, Diana; Jeung, Kristin; Wainer, Joshua; Okamura, Allison M

2008-03-01

Modeling forces applied to scissors during cutting of biological materials is useful for surgical simulation. Previous approaches to haptic display of scissor cutting are based on recording and replaying measured data. This paper presents an analytical model based on the concepts of contact mechanics and fracture mechanics to calculate forces applied to scissors during cutting of a slab of material. The model considers the process of cutting as a sequence of deformation and fracture phases. During deformation phases, forces applied to the scissors are calculated from a torque-angle response model synthesized from measurement data multiplied by a ratio that depends on the position of the cutting crack edge and the curve of the blades. Using the principle of conservation of energy, the forces of fracture are related to the fracture toughness of the material and the geometry of the blades of the scissors. The forces applied to scissors generally include high-frequency fluctuations. We show that the analytical model accurately predicts the average applied force. The cutting model is computationally efficient, so it can be used for real-time computations such as haptic rendering. Experimental results from cutting samples of paper, plastic, cloth, and chicken skin confirm the model, and the model is rendered in a haptic virtual environment.

Realistic soft tissue deformation strategies for real time surgery simulation.

PubMed

Shen, Yunhe; Zhou, Xiangmin; Zhang, Nan; Tamma, Kumar; Sweet, Robert

2008-01-01

A volume-preserving deformation method (VPDM) is developed in complement with the mass-spring method (MSM) to improve the deformation quality of the MSM to model soft tissue in surgical simulation. This method can also be implemented as a stand-alone model. The proposed VPDM satisfies the Newton's laws of motion by obtaining the resultant vectors form an equilibrium condition. The proposed method has been tested in virtual surgery systems with haptic rendering demands.

An experimental study on CHVE's performance evaluation.

PubMed

Paiva, Paulo V F; Machado, Liliane S; Oliveira, Jauvane C

2012-01-01

Virtual reality-based training simulators, with collaborative capabilities, are known to improve the way users interact with one another while learning or improving skills on a given medical procedure. Performance evaluation of Collaborative Haptic Virtual Environments (CHVE) allows us to understand how such systems can work in the Internet, as well as the requirements for multisensorial and real-time data. This work discloses new performance evaluation results for the collaborative module of the CyberMed VR framework.

Man, mind, and machine: the past and future of virtual reality simulation in neurologic surgery.

PubMed

Robison, R Aaron; Liu, Charles Y; Apuzzo, Michael L J

2011-11-01

To review virtual reality in neurosurgery, including the history of simulation and virtual reality and some of the current implementations; to examine some of the technical challenges involved; and to propose a potential paradigm for the development of virtual reality in neurosurgery going forward. A search was made on PubMed using key words surgical simulation, virtual reality, haptics, collision detection, and volumetric modeling to assess the current status of virtual reality in neurosurgery. Based on previous results, investigators extrapolated the possible integration of existing efforts and potential future directions. Simulation has a rich history in surgical training, and there are numerous currently existing applications and systems that involve virtual reality. All existing applications are limited to specific task-oriented functions and typically sacrifice visual realism for real-time interactivity or vice versa, owing to numerous technical challenges in rendering a virtual space in real time, including graphic and tissue modeling, collision detection, and direction of the haptic interface. With ongoing technical advancements in computer hardware and graphic and physical rendering, incremental or modular development of a fully immersive, multipurpose virtual reality neurosurgical simulator is feasible. The use of virtual reality in neurosurgery is predicted to change the nature of neurosurgical education, and to play an increased role in surgical rehearsal and the continuing education and credentialing of surgical practitioners. Copyright © 2011 Elsevier Inc. All rights reserved.

Haptic-assistive technologies for audition and vision sensory disabilities.

PubMed

Sorgini, Francesca; Caliò, Renato; Carrozza, Maria Chiara; Oddo, Calogero Maria

2018-05-01

The aim of this review is to analyze haptic sensory substitution technologies for deaf, blind and deaf-blind individuals. The literature search has been performed in Scopus, PubMed and Google Scholar databases using selected keywords, analyzing studies from 1960s to present. Search on databases for scientific publications has been accompanied by web search for commercial devices. Results have been classified by sensory disability and functionality, and analyzed by assistive technology. Complementary analyses have also been carried out on websites of public international agencies, such as the World Health Organization (WHO), and of associations representing sensory disabled persons. The reviewed literature provides evidences that sensory substitution aids are able to mitigate in part the deficits in language learning, communication and navigation for deaf, blind and deaf-blind individuals, and that the tactile sense can be a means of communication to provide some kind of information to sensory disabled individuals. A lack of acceptance emerged from the discussion of capabilities and limitations of haptic assistive technologies. Future researches shall go towards miniaturized, custom-designed and low-cost haptic interfaces and integration with personal devices such as smartphones for a major diffusion of sensory aids among disabled. Implications for rehabilitation Systematic review of state of the art of haptic assistive technologies for vision and audition sensory disabilities. Sensory substitution systems for visual and hearing disabilities have a central role in the transmission of information for patients with sensory impairments, enabling users to interact with the not disabled community in daily activities. Visual and auditory inputs are converted in haptic feedback via different actuation technologies. The information is presented in the form of static or dynamic stimulation of the skin. Their effectiveness and ease of use make haptic sensory substitution systems suitable for patients with different levels of disabilities. They constitute a cheaper and less invasive alternative to implantable partial sensory restitution systems. Future researches are oriented towards the optimization of the stimulation parameters together with the development of miniaturized, custom-designed and low-cost aids operating in synergy in networks, aiming to increase patients' acceptability of these technologies.

Mixed reality temporal bone surgical dissector: mechanical design.

PubMed

Hochman, Jordan Brent; Sepehri, Nariman; Rampersad, Vivek; Kraut, Jay; Khazraee, Milad; Pisa, Justyn; Unger, Bertram

2014-08-08

The Development of a Novel Mixed Reality (MR) Simulation. An evolving training environment emphasizes the importance of simulation. Current haptic temporal bone simulators have difficulty representing realistic contact forces and while 3D printed models convincingly represent vibrational properties of bone, they cannot reproduce soft tissue. This paper introduces a mixed reality model, where the effective elements of both simulations are combined; haptic rendering of soft tissue directly interacts with a printed bone model. This paper addresses one aspect in a series of challenges, specifically the mechanical merger of a haptic device with an otic drill. This further necessitates gravity cancelation of the work assembly gripper mechanism. In this system, the haptic end-effector is replaced by a high-speed drill and the virtual contact forces need to be repositioned to the drill tip from the mid wand. Previous publications detail generation of both the requisite printed and haptic simulations. Custom software was developed to reposition the haptic interaction point to the drill tip. A custom fitting, to hold the otic drill, was developed and its weight was offset using the haptic device. The robustness of the system to disturbances and its stable performance during drilling were tested. The experiments were performed on a mixed reality model consisting of two drillable rapid-prototyped layers separated by a free-space. Within the free-space, a linear virtual force model is applied to simulate drill contact with soft tissue. Testing illustrated the effectiveness of gravity cancellation. Additionally, the system exhibited excellent performance given random inputs and during the drill's passage between real and virtual components of the model. No issues with registration at model boundaries were encountered. These tests provide a proof of concept for the initial stages in the development of a novel mixed-reality temporal bone simulator.

Roughness based perceptual analysis towards digital skin imaging system with haptic feedback.

PubMed

Kim, K

2016-08-01

To examine psoriasis or atopic eczema, analyzing skin roughness by palpation is essential to precisely diagnose skin diseases. However, optical sensor based skin imaging systems do not allow dermatologists to touch skin images. To solve the problem, a new haptic rendering technology that can accurately display skin roughness must be developed. In addition, the rendering algorithm must be able to filter spatial noises created during 2D to 3D image conversion without losing the original roughness on the skin image. In this study, a perceptual way to design a noise filter that will remove spatial noises and in the meantime recover maximized roughness is introduced by understanding human sensitivity on surface roughness. A visuohaptic rendering system that can provide a user with seeing and touching digital skin surface roughness has been developed including a geometric roughness estimation method from a meshed surface. In following, a psychophysical experiment was designed and conducted with 12 human subjects to measure human perception with the developed visual and haptic interfaces to examine surface roughness. From the psychophysical experiment, it was found that touch is more sensitive at lower surface roughness, and vice versa. Human perception with both senses, vision and touch, becomes less sensitive to surface distortions as roughness increases. When interact with both channels, visual and haptic interfaces, the performance to detect abnormalities on roughness is greatly improved by sensory integration with the developed visuohaptic rendering system. The result can be used as a guideline to design a noise filter that can perceptually remove spatial noises while recover maximized roughness values from a digital skin image obtained by optical sensors. In addition, the result also confirms that the developed visuohaptic rendering system can help dermatologists or skin care professionals examine skin conditions by using vision and touch at the same time. © 2015 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Modulation of Visually Evoked Postural Responses by Contextual Visual, Haptic and Auditory Information: A ‘Virtual Reality Check’

PubMed Central

Meyer, Georg F.; Shao, Fei; White, Mark D.; Hopkins, Carl; Robotham, Antony J.

2013-01-01

Externally generated visual motion signals can cause the illusion of self-motion in space (vection) and corresponding visually evoked postural responses (VEPR). These VEPRs are not simple responses to optokinetic stimulation, but are modulated by the configuration of the environment. The aim of this paper is to explore what factors modulate VEPRs in a high quality virtual reality (VR) environment where real and virtual foreground objects served as static visual, auditory and haptic reference points. Data from four experiments on visually evoked postural responses show that: 1) visually evoked postural sway in the lateral direction is modulated by the presence of static anchor points that can be haptic, visual and auditory reference signals; 2) real objects and their matching virtual reality representations as visual anchors have different effects on postural sway; 3) visual motion in the anterior-posterior plane induces robust postural responses that are not modulated by the presence of reference signals or the reality of objects that can serve as visual anchors in the scene. We conclude that automatic postural responses for laterally moving visual stimuli are strongly influenced by the configuration and interpretation of the environment and draw on multisensory representations. Different postural responses were observed for real and virtual visual reference objects. On the basis that automatic visually evoked postural responses in high fidelity virtual environments should mimic those seen in real situations we propose to use the observed effect as a robust objective test for presence and fidelity in VR. PMID:23840760

Real-time feedback to improve gait in children with cerebral palsy.

PubMed

van Gelder, Linda; Booth, Adam T C; van de Port, Ingrid; Buizer, Annemieke I; Harlaar, Jaap; van der Krogt, Marjolein M

2017-02-01

Real-time feedback may be useful for enhancing information gained from clinical gait analysis of children with cerebral palsy (CP). It may also be effective in functional gait training, however, it is not known if children with CP can adapt gait in response to real-time feedback of kinematic parameters. Sixteen children with cerebral palsy (age 6-16; GMFCS I-III), walking with a flexed-knee gait pattern, walked on an instrumented treadmill with virtual reality in three conditions: regular walking without feedback (NF), feedback on hip angle (FH) and feedback on knee angle (FK). Clinically relevant gait parameters were calculated and the gait profile score (GPS) was used as a measure of overall gait changes between conditions. All children, except one, were able to improve hip and/or knee extension during gait in response to feedback, with nine achieving a clinically relevant improvement. Peak hip extension improved significantly by 5.1±5.9° (NF: 8.9±12.8°, FH: 3.8±10.4°, p=0.01). Peak knee extension improved significantly by 7.7±7.1° (NF: 22.2±12.0°, FK: 14.5±12.7°, p<0.01). GPS did not change between conditions due to increased deviations in other gait parameters. Responders to feedback were shown to have worse initial gait as measured by GPS (p=0.005) and functional selectivity score (p=0.049). In conclusion, ambulatory children with CP show adaptability in gait and are able to respond to real-time feedback, resulting in significant and clinically relevant improvements in peak hip and knee extension. These findings show the potential of real-time feedback as a tool for functional gait training and advanced gait analysis in CP. Copyright © 2016 Elsevier B.V. All rights reserved.

Auditory Proprioceptive Integration: Effects of Real-Time Kinematic Auditory Feedback on Knee Proprioception

PubMed Central

Ghai, Shashank; Schmitz, Gerd; Hwang, Tong-Hun; Effenberg, Alfred O.

2018-01-01

The purpose of the study was to assess the influence of real-time auditory feedback on knee proprioception. Thirty healthy participants were randomly allocated to control (n = 15), and experimental group I (15). The participants performed an active knee-repositioning task using their dominant leg, with/without additional real-time auditory feedback where the frequency was mapped in a convergent manner to two different target angles (40 and 75°). Statistical analysis revealed significant enhancement in knee re-positioning accuracy for the constant and absolute error with real-time auditory feedback, within and across the groups. Besides this convergent condition, we established a second divergent condition. Here, a step-wise transposition of frequency was performed to explore whether a systematic tuning between auditory-proprioceptive repositioning exists. No significant effects were identified in this divergent auditory feedback condition. An additional experimental group II (n = 20) was further included. Here, we investigated the influence of a larger magnitude and directional change of step-wise transposition of the frequency. In a first step, results confirm the findings of experiment I. Moreover, significant effects on knee auditory-proprioception repositioning were evident when divergent auditory feedback was applied. During the step-wise transposition participants showed systematic modulation of knee movements in the opposite direction of transposition. We confirm that knee re-positioning accuracy can be enhanced with concurrent application of real-time auditory feedback and that knee re-positioning can modulated in a goal-directed manner with step-wise transposition of frequency. Clinical implications are discussed with respect to joint position sense in rehabilitation settings. PMID:29568259

Virtual reality haptic dissection.

PubMed

Erolin, Caroline; Wilkinson, Caroline; Soames, Roger

2011-12-01

This project aims to create a three-dimensional digital model of the human hand and wrist which can be virtually 'dissected' through a haptic interface. Tissue properties will be added to the various anatomical structures to replicate a realistic look and feel. The project will explore the role of the medical artist, and investigate cross-discipline collaborations in the field of virtual anatomy. The software will be used to train anatomy students in dissection skills, before experience on a real cadaver. The effectiveness of the software will be evaluated and assessed both quantitatively as well as qualitatively.

Design and Construction of a Bilateral Haptic System for the Remote Assessment of the Stiffness and Range of Motion of the Hand

PubMed Central

Oscari, Fabio; Oboe, Roberto; Daud Albasini, Omar Andres; Masiero, Stefano; Rosati, Giulio

2016-01-01

The use of haptic devices in the rehabilitation of impaired limbs has become rather popular, given the proven effectiveness in promoting recovery. In a standard framework, such devices are used in rehabilitation centers, where patients interact with virtual tasks, presented on a screen. To track their sessions, kinematic/dynamic parameters or performance scores are recorded. However, as Internet access is now available at almost every home and in order to reduce the hospitalization time of the patient, the idea of doing rehabilitation at home is gaining wide consent. Medical care programs can be synchronized with the home rehabilitation device; patient data can be sent to the central server that could redirect to the therapist laptop (tele-healthcare). The controversial issue is that the recorded data do not actually represent the clinical conditions of the patients according to the medical assessment scales, forcing them to frequently undergo clinical tests at the hospital. To respond to this demand, we propose the use of a bilateral master/slave haptic system that could allow the clinician, who interacts with the master, to assess remotely and in real time the clinical conditions of the patient that uses the home rehabilitation device as the slave. In this paper, we describe a proof of concept to highlight the main issues of such an application, limited to one degree of freedom, and to the measure of the stiffness and range of motion of the hand. PMID:27706085

Can Force Feedback and Science Learning Enhance the Effectiveness of Neuro-Rehabilitation? An Experimental Study on Using a Low-Cost 3D Joystick and a Virtual Visit to a Zoo

PubMed Central

Cappa, Paolo; Clerico, Andrea; Nov, Oded; Porfiri, Maurizio

2013-01-01

In this paper, we demonstrate that healthy adults respond differentially to the administration of force feedback and the presentation of scientific content in a virtual environment, where they interact with a low-cost haptic device. Subjects are tasked with controlling the movement of a cursor on a predefined trajectory that is superimposed on a map of New York City’s Bronx Zoo. The system is characterized in terms of a suite of objective indices quantifying the subjects’ dexterity in planning and generating the multijoint visuomotor tasks. We find that force feedback regulates the smoothness, accuracy, and duration of the subject’s movement, whereby converging or diverging force fields influence the range of variations of the hand speed. Finally, our findings provide preliminary evidence that using educational content increases subjects’ satisfaction. Improving the level of interest through the inclusion of learning elements can increase the time spent performing rehabilitation tasks and promote learning in a new context. PMID:24349562

Precision displacement reference system

DOEpatents

Bieg, Lothar F.; Dubois, Robert R.; Strother, Jerry D.

2000-02-22

A precision displacement reference system is described, which enables real time accountability over the applied displacement feedback system to precision machine tools, positioning mechanisms, motion devices, and related operations. As independent measurements of tool location is taken by a displacement feedback system, a rotating reference disk compares feedback counts with performed motion. These measurements are compared to characterize and analyze real time mechanical and control performance during operation.

The Milestones Passport: A Learner-Centered Application of the Milestone Framework to Prompt Real-Time Feedback in the Emergency Department.

PubMed

Yarris, Lalena M; Jones, David; Kornegay, Joshua G; Hansen, Matthew

2014-09-01

In July 2013, emergency medicine residency programs implemented the Milestone assessment as part of the Next Accreditation System. We hypothesized that applying the Milestone framework to real-time feedback in the emergency department (ED) could affect current feedback processes and culture. We describe the development and implementation of a Milestone-based, learner-centered intervention designed to prompt real-time feedback in the ED. We developed and implemented the Milestones Passport, a feedback intervention incorporating subcompetencies, in our residency program in July 2013. Our primary outcomes were feasibility, including faculty and staff time and costs, number of documented feedback encounters in the first 2 months of implementation, and user-reported time required to complete the intervention. We also assessed learner and faculty acceptability. Development and implementation of the Milestones Passport required 10 hours of program coordinator time, 120 hours of software developer time, and 20 hours of faculty time. Twenty-eight residents and 34 faculty members generated 257 Milestones Passport feedback encounters. Most residents and faculty reported that the encounters required fewer than 5 minutes to complete, and 48% (12 of 25) of the residents and 68% (19 of 28) of faculty reported satisfaction with the Milestones Passport intervention. Faculty satisfaction with overall feedback in the ED improved after the intervention (93% versus 54%, P  =  .003), whereas resident satisfaction with feedback did not change significantly. The Milestones Passport feedback intervention was feasible and acceptable to users; however, learner satisfaction with the Milestone assessment in the ED was modest.

Real-time visual biofeedback during weight bearing improves therapy compliance in patients following lower extremity fractures.

PubMed

Raaben, Marco; Holtslag, Herman R; Leenen, Luke P H; Augustine, Robin; Blokhuis, Taco J

2018-01-01

Individuals with lower extremity fractures are often instructed on how much weight to bear on the affected extremity. Previous studies have shown limited therapy compliance in weight bearing during rehabilitation. In this study we investigated the effect of real-time visual biofeedback on weight bearing in individuals with lower extremity fractures in two conditions: full weight bearing and touch-down weight bearing. 11 participants with full weight bearing and 12 participants with touch-down weight bearing after lower extremity fractures have been measured with an ambulatory biofeedback system. The participants first walked 15m and the biofeedback system was only used to register the weight bearing. The same protocol was then repeated with real-time visual feedback during weight bearing. The participants could thereby adapt their loading to the desired level and improve therapy compliance. In participants with full weight bearing, real-time visual biofeedback resulted in a significant increase in loading from 50.9±7.51% bodyweight (BW) without feedback to 63.2±6.74%BW with feedback (P=0.0016). In participants with touch-down weight bearing, the exerted lower extremity load decreased from 16.7±9.77kg without feedback to 10.27±4.56kg with feedback (P=0.0718). More important, the variance between individual steps significantly decreased after feedback (P=0.018). Ambulatory monitoring weight bearing after lower extremity fractures showed that therapy compliance is low, both in full and touch-down weight bearing. Real-time visual biofeedback resulted in significantly higher peak loads in full weight bearing and increased accuracy of individual steps in touch-down weight bearing. Real-time visual biofeedback therefore results in improved therapy compliance after lower extremity fractures. Copyright © 2017 Elsevier B.V. All rights reserved.

Customization, control, and characterization of a commercial haptic device for high-fidelity rendering of weak forces.

PubMed

Gurari, Netta; Baud-Bovy, Gabriel

2014-09-30

The emergence of commercial haptic devices offers new research opportunities to enhance our understanding of the human sensory-motor system. Yet, commercial device capabilities have limitations which need to be addressed. This paper describes the customization of a commercial force feedback device for displaying forces with a precision that exceeds the human force perception threshold. The device was outfitted with a multi-axis force sensor and closed-loop controlled to improve its transparency. Additionally, two force sensing resistors were attached to the device to measure grip force. Force errors were modeled in the frequency- and time-domain to identify contributions from the mass, viscous friction, and Coulomb friction during open- and closed-loop control. The effect of user interaction on system stability was assessed in the context of a user study which aimed to measure force perceptual thresholds. Findings based on 15 participants demonstrate that the system maintains stability when rendering forces ranging from 0-0.20 N, with an average maximum absolute force error of 0.041 ± 0.013 N. Modeling the force errors revealed that Coulomb friction and inertia were the main contributors to force distortions during respectively slow and fast motions. Existing commercial force feedback devices cannot render forces with the required precision for certain testing scenarios. Building on existing robotics work, this paper shows how a device can be customized to make it reliable for studying the perception of weak forces. The customized and closed-loop controlled device is suitable for measuring force perceptual thresholds. Copyright © 2014 Elsevier B.V. All rights reserved.

Bionic Control of Cheetah Bounding with a Segmented Spine.

PubMed

Wang, Chunlei; Wang, Shigang

2016-01-01

A cheetah model is built to mimic real cheetah and its mechanical and dimensional parameters are derived from the real cheetah. In particular, two joints in spine and four joints in a leg are used to realize the motion of segmented spine and segmented legs which are the key properties of the cheetah bounding. For actuating and stabilizing the bounding gait of cheetah, we present a bioinspired controller based on the state-machine. The controller mainly mimics the function of the cerebellum to plan the locomotion and keep the body balance. The haptic sensor and proprioception system are used to detect the trigger of the phase transition. Besides, the vestibular modulation could perceive the pitching angle of the trunk. At last, the cerebellum acts as the CPU to operate the information from the biological sensors. In addition, the calculated results are transmitted to the low-level controller to actuate and stabilize the cheetah bounding. Moreover, the delay feedback control method is employed to plan the motion of the leg joints to stabilize the pitching motion of trunk with the stability criterion. Finally, the cyclic cheetah bounding with biological properties is realized. Meanwhile, the stability and dynamic properties of the cheetah bounding gait are analyzed elaborately.

Modeling and Compensation of the Internal Friction Torque of a Travelling Wave Ultrasonic Motor.

PubMed

Giraud, F; Sandulescu, P; Amberg, M; Lemaire-Semail, B; Ionescu, F

2011-01-01

This paper deals with the control and experimentation of a one-degree-of-freedom haptic stick, actuated by a travelling wave ultrasonic motor. This type of actuator has many interesting properties such as low-speed operation capabilities and a high torque-to-weight ratio, making it appropriate for haptic applications. However, the motor used in this application displays nonlinear behavior due to the necessary contact between its rotor and stator. Moreover, due to its energy conversion process, the torque applied to the end-effector is not a straightforward function of the supply current or voltage. This is why a force-feedback control strategy is presented, which includes an online parameter estimator. Experimental runs are then presented to examine the fidelity of the interface.

Research of the master-slave robot surgical system with the function of force feedback.

PubMed

Shi, Yunyong; Zhou, Chaozheng; Xie, Le; Chen, Yongjun; Jiang, Jun; Zhang, Zhenfeng; Deng, Ze

2017-12-01

Surgical robots lack force feedback, which may lead to operation errors. In order to improve surgical outcomes, this research developed a new master-slave surgical robot, which was designed with an integrated force sensor. The new structure designed for the master-slave robot employs a force feedback mechanism. A six-dimensional force sensor was mounted on the tip of the slave robot's actuator. Sliding model control was adopted to control the slave robot. According to the movement of the master system manipulated by the surgeon, the slave's movement and the force feedback function were validated. The motion was completed, the standard deviation was calculated, and the force data were detected. Hence, force feedback was realized in the experiment. The surgical robot can help surgeons to complete trajectory motions with haptic sensation. Copyright © 2017 John Wiley & Sons, Ltd.

The System-Wide Effect of Real-Time Audiovisual Feedback and Postevent Debriefing for In-Hospital Cardiac Arrest: The Cardiopulmonary Resuscitation Quality Improvement Initiative.

PubMed

Couper, Keith; Kimani, Peter K; Abella, Benjamin S; Chilwan, Mehboob; Cooke, Matthew W; Davies, Robin P; Field, Richard A; Gao, Fang; Quinton, Sarah; Stallard, Nigel; Woolley, Sarah; Perkins, Gavin D

2015-11-01

To evaluate the effect of implementing real-time audiovisual feedback with and without postevent debriefing on survival and quality of cardiopulmonary resuscitation quality at in-hospital cardiac arrest. A two-phase, multicentre prospective cohort study. Three UK hospitals, all part of one National Health Service Acute Trust. One thousand three hundred and ninety-five adult patients who sustained an in-hospital cardiac arrest at the study hospitals and were treated by hospital emergency teams between November 2009 and May 2013. During phase 1, quality of cardiopulmonary resuscitation and patient outcomes were measured with no intervention implemented. During phase 2, staff at hospital 1 received real-time audiovisual feedback, whereas staff at hospital 2 received real-time audiovisual feedback supplemented by postevent debriefing. No intervention was implemented at hospital 3 during phase 2. The primary outcome was return of spontaneous circulation. Secondary endpoints included other patient-focused outcomes, such as survival to hospital discharge, and process-focused outcomes, such as chest compression depth. Random-effect logistic and linear regression models, adjusted for baseline patient characteristics, were used to analyze the effect of the interventions on study outcomes. In comparison with no intervention, neither real-time audiovisual feedback (adjusted odds ratio, 0.62; 95% CI, 0.31-1.22; p=0.17) nor real-time audiovisual feedback supplemented by postevent debriefing (adjusted odds ratio, 0.65; 95% CI, 0.35-1.21; p=0.17) was associated with a statistically significant improvement in return of spontaneous circulation or any process-focused outcome. Despite this, there was evidence of a system-wide improvement in phase 2, leading to improvements in return of spontaneous circulation (adjusted odds ratio, 1.87; 95% CI, 1.06-3.30; p=0.03) and process-focused outcomes. Implementation of real-time audiovisual feedback with or without postevent debriefing did not lead to a measured improvement in patient or process-focused outcomes at individual hospital sites. However, there was an unexplained system-wide improvement in return of spontaneous circulation and process-focused outcomes during the second phase of the study.

Real-Time Performance Feedback for the Manual Control of Spacecraft

NASA Astrophysics Data System (ADS)

Karasinski, John Austin

Real-time performance metrics were developed to quantify workload, situational awareness, and manual task performance for use as visual feedback to pilots of aerospace vehicles. Results from prior lunar lander experiments with variable levels of automation were replicated and extended to provide insights for the development of real-time metrics. Increased levels of automation resulted in increased flight performance, lower workload, and increased situational awareness. Automated Speech Recognition (ASR) was employed to detect verbal callouts as a limited measure of subjects' situational awareness. A one-dimensional manual tracking task and simple instructor-model visual feedback scheme was developed. This feedback was indicated to the operator by changing the color of a guidance element on the primary flight display, similar to how a flight instructor points out elements of a display to a student pilot. Experiments showed that for this low-complexity task, visual feedback did not change subject performance, but did increase the subjects' measured workload. Insights gained from these experiments were applied to a Simplified Aid for EVA Rescue (SAFER) inspection task. The effects of variations of an instructor-model performance-feedback strategy on human performance in a novel SAFER inspection task were investigated. Real-time feedback was found to have a statistically significant effect of improving subject performance and decreasing workload in this complicated four degree of freedom manual control task with two secondary tasks.

[Real-time feedback systems for improvement of resuscitation quality].

PubMed

Lukas, R P; Van Aken, H; Engel, P; Bohn, A

2011-07-01

The quality of chest compression is a determinant of survival after cardiac arrest. Therefore, the European Resuscitation Council (ERC) 2010 guidelines on resuscitation strongly focus on compression quality. Despite its impact on survival, observational studies have shown that chest compression quality is not reached by professional rescue teams. Real-time feedback devices for resuscitation are able to measure chest compression during an ongoing resuscitation attempt through a sternal sensor equipped with a motion and pressure detection system. In addition to the electrocardiograph (ECG) ventilation can be detected by transthoracic impedance monitoring. In cases of quality deviation, such as shallow chest compression depth or hyperventilation, feedback systems produce visual or acoustic alarms. Rescuers can thereby be supported and guided to the requested quality in chest compression and ventilation. Feedback technology is currently available both as a so-called stand-alone device and as an integrated feature in a monitor/defibrillator unit. Multiple studies have demonstrated sustainable enhancement in the education of resuscitation due to the use of real-time feedback technology. There is evidence that real-time feedback for resuscitation combined with training and debriefing strategies can improve both resuscitation quality and patient survival. Chest compression quality is an independent predictor for survival in resuscitation and should therefore be measured and documented in further clinical multicenter trials.

Anthro-Centric Multisensory Interface for Sensory Augmentation of Tele-Surgery (ACMI-SATS)

DTIC Science & Technology

2010-09-01

surgeon from perceiving useful kinesthetic feedback from direct interaction with the tissues present in traditional “open” procedures. Additionally... Kinesthetic and haptic signals in surgical applications are critical, and prior work with VEs has shown that errors increase without realistic...telepresence related kinesthetic sensory interactions while tactile will refer to more general or abstract tactual interactions. Figure 2: (left

Virtual Environments for People Who Are Visually Impaired Integrated into an Orientation and Mobility Program

ERIC Educational Resources Information Center

Lahav, Orly; Schloerb, David W.; Srinivasan, Mandayam A.

2015-01-01

Introduction: The BlindAid, a virtual system developed for orientation and mobility (O&M) training of people who are blind or have low vision, allows interaction with different virtual components (structures and objects) via auditory and haptic feedback. This research examined if and how the BlindAid that was integrated within an O&M…

Low cost heads-up virtual reality (HUVR) with optical tracking and haptic feedback

NASA Astrophysics Data System (ADS)

Margolis, Todd; DeFanti, Thomas A.; Dawe, Greg; Prudhomme, Andrew; Schulze, Jurgen P.; Cutchin, Steve

2011-03-01

Researchers at the University of California, San Diego, have created a new, relatively low-cost augmented reality system that enables users to touch the virtual environment they are immersed in. The Heads-Up Virtual Reality device (HUVR) couples a consumer 3D HD flat screen TV with a half-silvered mirror to project any graphic image onto the user's hands and into the space surrounding them. With his or her head position optically tracked to generate the correct perspective view, the user maneuvers a force-feedback (haptic) device to interact with the 3D image, literally 'touching' the object's angles and contours as if it was a tangible physical object. HUVR can be used for training and education in structural and mechanical engineering, archaeology and medicine as well as other tasks that require hand-eye coordination. One of the most unique characteristics of HUVR is that a user can place their hands inside of the virtual environment without occluding the 3D image. Built using open-source software and consumer level hardware, HUVR offers users a tactile experience in an immersive environment that is functional, affordable and scalable.

A systematic review of phacoemulsification cataract surgery in virtual reality simulators.

PubMed

Lam, Chee Kiang; Sundaraj, Kenneth; Sulaiman, Mohd Nazri

2013-01-01

The aim of this study was to review the capability of virtual reality simulators in the application of phacoemulsification cataract surgery training. Our review included the scientific publications on cataract surgery simulators that had been developed by different groups of researchers along with commercialized surgical training products, such as EYESI® and PhacoVision®. The review covers the simulation of the main cataract surgery procedures, i.e., corneal incision, capsulorrhexis, phacosculpting, and intraocular lens implantation in various virtual reality surgery simulators. Haptics realism and visual realism of the procedures are the main elements in imitating the actual surgical environment. The involvement of ophthalmology in research on virtual reality since the early 1990s has made a great impact on the development of surgical simulators. Most of the latest cataract surgery training systems are able to offer high fidelity in visual feedback and haptics feedback, but visual realism, such as the rotational movements of an eyeball with response to the force applied by surgical instruments, is still lacking in some of them. The assessment of the surgical tasks carried out on the simulators showed a significant difference in the performance before and after the training.

Integration of serious games and wearable haptic interfaces for Neuro Rehabilitation of children with movement disorders: A feasibility study.

PubMed

Bortone, Ilaria; Leonardis, Daniele; Solazzi, Massimiliano; Procopio, Caterina; Crecchi, Alessandra; Bonfiglio, Luca; Frisoli, Antonio

2017-07-01

The past decade has seen the emergence of rehabilitation treatments using virtual reality environments. One of the advantages in using this technology is the potential to create positive motivation, by means of engaging environments and tasks shaped in the form of serious games. In this work, we propose a novel Neuro Rehabilitation System for children with movement disorders, that is based on serious games in immersive virtual reality with haptic feedback. The system design aims to enhance involvement and engagement of patients, to provide congruent multi-sensory afferent feedback during motor exercises, and to benefit from the flexibility of virtual reality in adapting exercises to the patient's needs. We present a feasibility study of the method conducted through an experimental rehabilitation session in a group of 4 children with Cerebral Palsy and Developmental Dyspraxia, 4 Typically Developing children and 4 healthy adults. Subjects and patients were able to accomplish the proposed rehabilitation session and average performance of the motor exercises in patients were lower, although comparable, to healthy subjects. Together with positive comments reported by children after the rehabilitation session, results are encouraging for application of the method in a prolonged rehabilitation treatment.

Haptic contents of a movie dynamically engage the spectator's sensorimotor cortex.

PubMed

Lankinen, Kaisu; Smeds, Eero; Tikka, Pia; Pihko, Elina; Hari, Riitta; Koskinen, Miika

2016-11-01

Observation of another person's actions and feelings activates brain areas that support similar functions in the observer, thereby facilitating inferences about the other's mental and bodily states. In real life, events eliciting this kind of vicarious brain activations are intermingled with other complex, ever-changing stimuli in the environment. One practical approach to study the neural underpinnings of real-life vicarious perception is to image brain activity during movie viewing. Here the goal was to find out how observed haptic events in a silent movie would affect the spectator's sensorimotor cortex. The functional state of the sensorimotor cortex was monitored by analyzing, in 16 healthy subjects, magnetoencephalographic (MEG) responses to tactile finger stimuli that were presented once per second throughout the session. Using canonical correlation analysis and spatial filtering, consistent single-trial responses across subjects were uncovered, and their waveform changes throughout the movie were quantified. The long-latency (85-175 ms) parts of the responses were modulated in concordance with the participants' average moment-by-moment ratings of own engagement in the haptic content of the movie (correlation r = 0.49; ratings collected after the MEG session). The results, obtained by using novel signal-analysis approaches, demonstrate that the functional state of the human sensorimotor cortex fluctuates in a fine-grained manner even during passive observation of temporally varying haptic events. Hum Brain Mapp 37:4061-4068, 2016. © 2016 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc. © 2016 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc.

Visualizing Syllables: Real-Time Computerized Feedback within a Speech-Language Intervention

ERIC Educational Resources Information Center

DeThorne, Laura; Aparicio Betancourt, Mariana; Karahalios, Karrie; Halle, Jim; Bogue, Ellen

2015-01-01

Computerized technologies now offer unprecedented opportunities to provide real-time visual feedback to facilitate children's speech-language development. We employed a mixed-method design to examine the effectiveness of two speech-language interventions aimed at facilitating children's multisyllabic productions: one incorporated a novel…

Magnetorheological fluid based automotive steer-by-wire systems

NASA Astrophysics Data System (ADS)

Ahmadkhanlou, Farzad; Washington, Gregory N.; Bechtel, Stephen E.; Wang, Yingru

2006-03-01

The idea of this paper is to design a Magnetorheological (MR) fluid based damper for steer-by-wire systems to provide sensory feedback to the driver. The advantages of using MR fluids in haptic devices stem from the increase in transparency gained from the lightweight semiactive system and controller implementation. The performance of MR fluid based steer-by wire system depends on MR fluid model and specifications, MR damper geometry, and the control algorithm. All of these factors are addressed in this study. The experimental results show the improvements in steer-by-wire by adding force feedback to the system.

Enabling multi-level relevance feedback on PubMed by integrating rank learning into DBMS.

PubMed

Yu, Hwanjo; Kim, Taehoon; Oh, Jinoh; Ko, Ilhwan; Kim, Sungchul; Han, Wook-Shin

2010-04-16

Finding relevant articles from PubMed is challenging because it is hard to express the user's specific intention in the given query interface, and a keyword query typically retrieves a large number of results. Researchers have applied machine learning techniques to find relevant articles by ranking the articles according to the learned relevance function. However, the process of learning and ranking is usually done offline without integrated with the keyword queries, and the users have to provide a large amount of training documents to get a reasonable learning accuracy. This paper proposes a novel multi-level relevance feedback system for PubMed, called RefMed, which supports both ad-hoc keyword queries and a multi-level relevance feedback in real time on PubMed. RefMed supports a multi-level relevance feedback by using the RankSVM as the learning method, and thus it achieves higher accuracy with less feedback. RefMed "tightly" integrates the RankSVM into RDBMS to support both keyword queries and the multi-level relevance feedback in real time; the tight coupling of the RankSVM and DBMS substantially improves the processing time. An efficient parameter selection method for the RankSVM is also proposed, which tunes the RankSVM parameter without performing validation. Thereby, RefMed achieves a high learning accuracy in real time without performing a validation process. RefMed is accessible at http://dm.postech.ac.kr/refmed. RefMed is the first multi-level relevance feedback system for PubMed, which achieves a high accuracy with less feedback. It effectively learns an accurate relevance function from the user's feedback and efficiently processes the function to return relevant articles in real time.

Enabling multi-level relevance feedback on PubMed by integrating rank learning into DBMS

PubMed Central

2010-01-01

Background Finding relevant articles from PubMed is challenging because it is hard to express the user's specific intention in the given query interface, and a keyword query typically retrieves a large number of results. Researchers have applied machine learning techniques to find relevant articles by ranking the articles according to the learned relevance function. However, the process of learning and ranking is usually done offline without integrated with the keyword queries, and the users have to provide a large amount of training documents to get a reasonable learning accuracy. This paper proposes a novel multi-level relevance feedback system for PubMed, called RefMed, which supports both ad-hoc keyword queries and a multi-level relevance feedback in real time on PubMed. Results RefMed supports a multi-level relevance feedback by using the RankSVM as the learning method, and thus it achieves higher accuracy with less feedback. RefMed "tightly" integrates the RankSVM into RDBMS to support both keyword queries and the multi-level relevance feedback in real time; the tight coupling of the RankSVM and DBMS substantially improves the processing time. An efficient parameter selection method for the RankSVM is also proposed, which tunes the RankSVM parameter without performing validation. Thereby, RefMed achieves a high learning accuracy in real time without performing a validation process. RefMed is accessible at http://dm.postech.ac.kr/refmed. Conclusions RefMed is the first multi-level relevance feedback system for PubMed, which achieves a high accuracy with less feedback. It effectively learns an accurate relevance function from the user’s feedback and efficiently processes the function to return relevant articles in real time. PMID:20406504

Inertial sensor real-time feedback enhances the learning of cervical spine manipulation: a prospective study

PubMed Central

2014-01-01

Background Cervical Spinal Manipulation (CSM) is considered a high-level skill of the central nervous system because it requires bimanual coordinated rhythmical movements therefore necessitating training to achieve proficiency. The objective of the present study was to investigate the effect of real-time feedback on the performance of CSM. Methods Six postgraduate physiotherapy students attending a training workshop on Cervical Spine Manipulation Technique (CSMT) using inertial sensor derived real-time feedback participated in this study. The key variables were pre-manipulative position, angular displacement of the thrust and angular velocity of the thrust. Differences between variables before and after training were investigated using t-tests. Results There were no significant differences after training for the pre-manipulative position (rotation p = 0.549; side bending p = 0.312) or for thrust displacement (rotation p = 0.247; side bending p = 0.314). Thrust angular velocity demonstrated a significant difference following training for rotation (pre-training mean (sd) 48.9°/s (35.1); post-training mean (sd) 96.9°/s (53.9); p = 0.027) but not for side bending (p = 0.521). Conclusion Real-time feedback using an inertial sensor may be valuable in the development of specific manipulative skill. Future studies investigating manipulation could consider a randomized controlled trial using inertial sensor real time feedback compared to traditional training. PMID:24942483

Real-time feedback on knee abduction moment does not improve frontal-plane knee mechanics during jump landings.

PubMed

Beaulieu, M L; Palmieri-Smith, R M

2014-08-01

Excessive knee abduction loading is a contributing factor to anterior cruciate ligament (ACL) injury risk. The purpose of this study was to determine whether a double-leg landing training program with real-time visual feedback improves frontal-plane mechanics during double- and single-leg landings. Knee abduction angles and moments and vertical ground reaction forces (GRF) of 21 recreationally active women were quantified for double- and single-leg landings before and after the training program. This program consisted of two sessions of double-leg jump landings with real-time visual feedback on knee abduction moments for the experimental group and without real-time feedback for the control group. No significant differences were found between training groups. In comparison with pre-training data, peak knee abduction moments decreased 12% post-training for both double- and single-leg landings; whereas peak vertical GRF decreased 8% post-training for double-leg landings only, irrespective of training group. Real-time feedback on knee abduction moments, therefore, did not significantly improve frontal-plane knee mechanics during landings. The effect of the training program on knee abduction moments, however, transferred from the double-leg landings (simple task) to single-leg landings (more complex task). Consequently, ACL injury prevention efforts may not need to focus on complex tasks during which injury occurs. © 2013 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Deterministic generation of remote entanglement with active quantum feedback

DOE PAGES

Martin, Leigh; Motzoi, Felix; Li, Hanhan; ...

2015-12-10

We develop and study protocols for deterministic remote entanglement generation using quantum feedback, without relying on an entangling Hamiltonian. In order to formulate the most effective experimentally feasible protocol, we introduce the notion of average-sense locally optimal feedback protocols, which do not require real-time quantum state estimation, a difficult component of real-time quantum feedback control. We use this notion of optimality to construct two protocols that can deterministically create maximal entanglement: a semiclassical feedback protocol for low-efficiency measurements and a quantum feedback protocol for high-efficiency measurements. The latter reduces to direct feedback in the continuous-time limit, whose dynamics can bemore » modeled by a Wiseman-Milburn feedback master equation, which yields an analytic solution in the limit of unit measurement efficiency. Our formalism can smoothly interpolate between continuous-time and discrete-time descriptions of feedback dynamics and we exploit this feature to derive a superior hybrid protocol for arbitrary nonunit measurement efficiency that switches between quantum and semiclassical protocols. Lastly, we show using simulations incorporating experimental imperfections that deterministic entanglement of remote superconducting qubits may be achieved with current technology using the continuous-time feedback protocol alone.« less

A Model for Steering with Haptic-Force Guidance

NASA Astrophysics Data System (ADS)

Yang, Xing-Dong; Irani, Pourang; Boulanger, Pierre; Bischof, Walter F.

Trajectory-based tasks are common in many applications and have been widely studied. Recently, researchers have shown that even very simple tasks, such as selecting items from cascading menus, can benefit from haptic-force guidance. Haptic guidance is also of significant value in many applications such as medical training, handwriting learning, and in applications requiring precise manipulations. There are, however, only very few guiding principles for selecting parameters that are best suited for proper force guiding. In this paper, we present a model, derived from the steering law that relates movement time to the essential components of a tunneling task in the presence of haptic-force guidance. Results of an experiment show that our model is highly accurate for predicting performance times in force-enhanced tunneling tasks.

Perception of CPR quality: Influence of CPR feedback, Just-in-Time CPR training and provider role.

PubMed

Cheng, Adam; Overly, Frank; Kessler, David; Nadkarni, Vinay M; Lin, Yiqun; Doan, Quynh; Duff, Jonathan P; Tofil, Nancy M; Bhanji, Farhan; Adler, Mark; Charnovich, Alex; Hunt, Elizabeth A; Brown, Linda L

2015-02-01

Many healthcare providers rely on visual perception to guide cardiopulmonary resuscitation (CPR), but little is known about the accuracy of provider perceptions of CPR quality. We aimed to describe the difference between perceived versus measured CPR quality, and to determine the impact of provider role, real-time visual CPR feedback and Just-in-Time (JIT) CPR training on provider perceptions. We conducted secondary analyses of data collected from a prospective, multicenter, randomized trial of 324 healthcare providers who participated in a simulated cardiac arrest scenario between July 2012 and April 2014. Participants were randomized to one of four permutations of: JIT CPR training and real-time visual CPR feedback. We calculated the difference between perceived and measured quality of CPR and reported the proportion of subjects accurately estimating the quality of CPR within each study arm. Participants overestimated achieving adequate chest compression depth (mean difference range: 16.1-60.6%) and rate (range: 0.2-51%), and underestimated chest compression fraction (0.2-2.9%) across all arms. Compared to no intervention, the use of real-time feedback and JIT CPR training (alone or in combination) improved perception of depth (p<0.001). Accurate estimation of CPR quality was poor for chest compression depth (0-13%), rate (5-46%) and chest compression fraction (60-63%). Perception of depth is more accurate in CPR providers versus team leaders (27.8% vs. 7.4%; p=0.043) when using real-time feedback. Healthcare providers' visual perception of CPR quality is poor. Perceptions of CPR depth are improved by using real-time visual feedback and with prior JIT CPR training. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

Feedback mechanisms including real-time electronic alerts to achieve near 100% timely prophylactic antibiotic administration in surgical cases.

PubMed

Nair, Bala G; Newman, Shu-Fang; Peterson, Gene N; Wu, Wei-Ying; Schwid, Howard A

2010-11-01

Administration of prophylactic antibiotics during surgery is generally performed by the anesthesia providers. Timely antibiotic administration within the optimal time window before incision is critical for prevention of surgical site infections. However, this often becomes a difficult task for the anesthesia team during the busy part of a case when the patient is being anesthetized. Starting with the implementation of an anesthesia information management system (AIMS), we designed and implemented several feedback mechanisms to improve compliance of proper antibiotic delivery and documentation. This included generating e-mail feedback of missed documentation, distributing monthly summary reports, and generating real-time electronic alerts with a decision support system. In 20,974 surgical cases for the period, June 2008 to January 2010, the interventions of AIMS install, e-mail feedback, summary reports, and real-time alerts changed antibiotic compliance by -1.5%, 2.3%, 4.9%, and 9.3%, respectively, when compared with the baseline value of 90.0% ± 2.9% when paper anesthesia records were used. Highest antibiotic compliance was achieved when using real-time alerts. With real-time alerts, monthly compliance was >99% for every month between June 2009 and January 2010. Installation of AIMS itself did not improve antibiotic compliance over that achieved with paper anesthesia records. However, real-time guidance and reminders through electronic messages generated by a computerized decision support system (Smart Anesthesia Messenger, or SAM) significantly improved compliance. With such a system a consistent compliance of >99% was achieved.

The Milestones Passport: A Learner-Centered Application of the Milestone Framework to Prompt Real-Time Feedback in the Emergency Department

PubMed Central

Yarris, Lalena M.; Jones, David; Kornegay, Joshua G.; Hansen, Matthew

2014-01-01

Background In July 2013, emergency medicine residency programs implemented the Milestone assessment as part of the Next Accreditation System. Objective We hypothesized that applying the Milestone framework to real-time feedback in the emergency department (ED) could affect current feedback processes and culture. We describe the development and implementation of a Milestone-based, learner-centered intervention designed to prompt real-time feedback in the ED. Methods We developed and implemented the Milestones Passport, a feedback intervention incorporating subcompetencies, in our residency program in July 2013. Our primary outcomes were feasibility, including faculty and staff time and costs, number of documented feedback encounters in the first 2 months of implementation, and user-reported time required to complete the intervention. We also assessed learner and faculty acceptability. Results Development and implementation of the Milestones Passport required 10 hours of program coordinator time, 120 hours of software developer time, and 20 hours of faculty time. Twenty-eight residents and 34 faculty members generated 257 Milestones Passport feedback encounters. Most residents and faculty reported that the encounters required fewer than 5 minutes to complete, and 48% (12 of 25) of the residents and 68% (19 of 28) of faculty reported satisfaction with the Milestones Passport intervention. Faculty satisfaction with overall feedback in the ED improved after the intervention (93% versus 54%, P  =  .003), whereas resident satisfaction with feedback did not change significantly. Conclusions The Milestones Passport feedback intervention was feasible and acceptable to users; however, learner satisfaction with the Milestone assessment in the ED was modest. PMID:26279784

Estimating Tool–Tissue Forces Using a 3-Degree-of-Freedom Robotic Surgical Tool

PubMed Central

Zhao, Baoliang; Nelson, Carl A.

2016-01-01

Robot-assisted minimally invasive surgery (MIS) has gained popularity due to its high dexterity and reduced invasiveness to the patient; however, due to the loss of direct touch of the surgical site, surgeons may be prone to exert larger forces and cause tissue damage. To quantify tool–tissue interaction forces, researchers have tried to attach different kinds of sensors on the surgical tools. This sensor attachment generally makes the tools bulky and/or unduly expensive and may hinder the normal function of the tools; it is also unlikely that these sensors can survive harsh sterilization processes. This paper investigates an alternative method by estimating tool–tissue interaction forces using driving motors' current, and validates this sensorless force estimation method on a 3-degree-of-freedom (DOF) robotic surgical grasper prototype. The results show that the performance of this method is acceptable with regard to latency and accuracy. With this tool–tissue interaction force estimation method, it is possible to implement force feedback on existing robotic surgical systems without any sensors. This may allow a haptic surgical robot which is compatible with existing sterilization methods and surgical procedures, so that the surgeon can obtain tool–tissue interaction forces in real time, thereby increasing surgical efficiency and safety. PMID:27303591

Estimating Tool-Tissue Forces Using a 3-Degree-of-Freedom Robotic Surgical Tool.

PubMed

Zhao, Baoliang; Nelson, Carl A

2016-10-01

Robot-assisted minimally invasive surgery (MIS) has gained popularity due to its high dexterity and reduced invasiveness to the patient; however, due to the loss of direct touch of the surgical site, surgeons may be prone to exert larger forces and cause tissue damage. To quantify tool-tissue interaction forces, researchers have tried to attach different kinds of sensors on the surgical tools. This sensor attachment generally makes the tools bulky and/or unduly expensive and may hinder the normal function of the tools; it is also unlikely that these sensors can survive harsh sterilization processes. This paper investigates an alternative method by estimating tool-tissue interaction forces using driving motors' current, and validates this sensorless force estimation method on a 3-degree-of-freedom (DOF) robotic surgical grasper prototype. The results show that the performance of this method is acceptable with regard to latency and accuracy. With this tool-tissue interaction force estimation method, it is possible to implement force feedback on existing robotic surgical systems without any sensors. This may allow a haptic surgical robot which is compatible with existing sterilization methods and surgical procedures, so that the surgeon can obtain tool-tissue interaction forces in real time, thereby increasing surgical efficiency and safety.

[Fabrication and Performance Study of Polydimethylsiloxane Intraocular Lens].

PubMed

Du, Qiuyue; Yu, Yueqing

2016-10-01

To simplify the production process of intraocular lens(IOL)and to solve the problem of lacking adjustable ability,we proposed a novel soft IOL with large-scale adjustable ability and rigid haptics based on heat-assisted method,and gravity-assisted method.Polydimethylsiloxane(PDMS)and rigid material--polymethyl methacrylate(PMMA)were used as the materials for fabricating optical lens(PDMS)and haptics(PDMS and PMMA)through changing the weight ratio of the solution.A lens-smartphone microscopy system was established to replace the traditional digital microscopy to measure the tiny displacements and shape changes.The PDMS lens has excellent optical property through an experiment in which the maximum optical power was around 273.2D.Experimental results indicated that the maximum optical power of PMMA IOL was 129.3D,and that in PDMS IOL,however,was only 56.0D.Thus,the rigid PMMA-IOL has a larger adjustable range.The production process of PDMS was mold-free,rapid,real-time,and highly repeatable and there was no need for a rigorous experimental environment either.This creative processing technology reduced the manufacturing steps from which an optical lens with high transmittance and high resolution,as well as hatics with accurate dimensions,were obtained.The rigidity of haptics affected more intensely than other factors did for improving the pre-displacement and changing the appearance of PDMS lens.Even though the rigidity of PMMA haptics is large enough for accommodating,there are many drawbacks during manufacturing.It is unavailable to rapid fabricate IOL using PMMA.Thus,further work will be required to alter the weight ratio of PDMS material,increase the rigidity,and enhance the adjustable ability of PDMS IOL.

Remote video auditing with real-time feedback in an academic surgical suite improves safety and efficiency metrics: a cluster randomised study.

PubMed

Overdyk, Frank J; Dowling, Oonagh; Newman, Sheldon; Glatt, David; Chester, Michelle; Armellino, Donna; Cole, Brandon; Landis, Gregg S; Schoenfeld, David; DiCapua, John F

2016-12-01

Compliance with the surgical safety checklist during operative procedures has been shown to reduce inhospital mortality and complications but proper execution by the surgical team remains elusive. We evaluated the impact of remote video auditing with real-time provider feedback on checklist compliance during sign-in, time-out and sign-out and case turnover times. Prospective, cluster randomised study in a 23-operating room (OR) suite. Surgeons, anaesthesia providers, nurses and support staff. ORs were randomised to receive, or not receive, real-time feedback on safety checklist compliance and efficiency metrics via display boards and text messages, followed by a period during which all ORs received feedback. Checklist compliance (Pass/Fail) during sign-in, time-out and sign-out demonstrated by (1) use of checklist, (2) team attentiveness, (3) required duration, (4) proper sequence and duration of case turnover times. Sign-in, time-out and sign-out PASS rates increased from 25%, 16% and 32% during baseline phase (n=1886) to 64%, 84% and 68% for feedback ORs versus 40%, 77% and 51% for no-feedback ORs (p<0.004) during the intervention phase (n=2693). Pass rates were 91%, 95% and 84% during the all-feedback phase (n=2001). For scheduled cases (n=1406, 71%), feedback reduced mean turnover times by 14% (41.4 min vs 48.1 min, p<0.004), and the improvement was sustained during the all-feedback period. Feedback had no effect on turnover time for unscheduled cases (n=587, 29%). Our data indicate that remote video auditing with feedback improves surgical safety checklist compliance for all cases, and turnover time for scheduled cases, but not for unscheduled cases. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Remote video auditing with real-time feedback in an academic surgical suite improves safety and efficiency metrics: a cluster randomised study

PubMed Central

Overdyk, Frank J; Dowling, Oonagh; Newman, Sheldon; Glatt, David; Chester, Michelle; Armellino, Donna; Cole, Brandon; Landis, Gregg S; Schoenfeld, David; DiCapua, John F

2016-01-01

Importance Compliance with the surgical safety checklist during operative procedures has been shown to reduce inhospital mortality and complications but proper execution by the surgical team remains elusive. Objective We evaluated the impact of remote video auditing with real-time provider feedback on checklist compliance during sign-in, time-out and sign-out and case turnover times. Design, setting Prospective, cluster randomised study in a 23-operating room (OR) suite. Participants Surgeons, anaesthesia providers, nurses and support staff. Exposure ORs were randomised to receive, or not receive, real-time feedback on safety checklist compliance and efficiency metrics via display boards and text messages, followed by a period during which all ORs received feedback. Main outcome(s) and measure(s) Checklist compliance (Pass/Fail) during sign-in, time-out and sign-out demonstrated by (1) use of checklist, (2) team attentiveness, (3) required duration, (4) proper sequence and duration of case turnover times. Results Sign-in, time-out and sign-out PASS rates increased from 25%, 16% and 32% during baseline phase (n=1886) to 64%, 84% and 68% for feedback ORs versus 40%, 77% and 51% for no-feedback ORs (p<0.004) during the intervention phase (n=2693). Pass rates were 91%, 95% and 84% during the all-feedback phase (n=2001). For scheduled cases (n=1406, 71%), feedback reduced mean turnover times by 14% (41.4 min vs 48.1 min, p<0.004), and the improvement was sustained during the all-feedback period. Feedback had no effect on turnover time for unscheduled cases (n=587, 29%). Conclusions and relevance Our data indicate that remote video auditing with feedback improves surgical safety checklist compliance for all cases, and turnover time for scheduled cases, but not for unscheduled cases. PMID:26658775

Percutaneous spinal fixation simulation with virtual reality and haptics.

PubMed

Luciano, Cristian J; Banerjee, P Pat; Sorenson, Jeffery M; Foley, Kevin T; Ansari, Sameer A; Rizzi, Silvio; Germanwala, Anand V; Kranzler, Leonard; Chittiboina, Prashant; Roitberg, Ben Z

2013-01-01

In this study, we evaluated the use of a part-task simulator with 3-dimensional and haptic feedback as a training tool for percutaneous spinal needle placement. To evaluate the learning effectiveness in terms of entry point/target point accuracy of percutaneous spinal needle placement on a high-performance augmented-reality and haptic technology workstation with the ability to control the duration of computer-simulated fluoroscopic exposure, thereby simulating an actual situation. Sixty-three fellows and residents performed needle placement on the simulator. A virtual needle was percutaneously inserted into a virtual patient's thoracic spine derived from an actual patient computed tomography data set. Ten of 126 needle placement attempts by 63 participants ended in failure for a failure rate of 7.93%. From all 126 needle insertions, the average error (15.69 vs 13.91), average fluoroscopy exposure (4.6 vs 3.92), and average individual performance score (32.39 vs 30.71) improved from the first to the second attempt. Performance accuracy yielded P = .04 from a 2-sample t test in which the rejected null hypothesis assumes no improvement in performance accuracy from the first to second attempt in the test session. The experiments showed evidence (P = .04) of performance accuracy improvement from the first to the second percutaneous needle placement attempt. This result, combined with previous learning retention and/or face validity results of using the simulator for open thoracic pedicle screw placement and ventriculostomy catheter placement, supports the efficacy of augmented reality and haptics simulation as a learning tool.

Haptics using a smart material for eyes-free interaction in personal devices

NASA Astrophysics Data System (ADS)

Wang, Huihui; Lane, William Brian; Pappas, Devin; Duque, Bryam; Leong, John

2014-03-01

In this paper we present a prototype using a dry ionic polymer metal composite (IPMC) in interactive personal devices such as bracelet, necklace, pocket key chain or mobile devices for haptic interaction when audio or visual feedback is not possible or practical. This prototype interface is an electro-mechanical system that realizes a shape-changing haptic display for information communication. A dry IPMC will change its dimensions due to the electrostatic effect when an electrical potential is provided to them. The IPMC can operate at a lower voltage (less than 2.5V) which is compatible with requirements for personal electrical devices or mobile devices. The prototype consists of the addressable arrays of the IPMCs with different dimensions which are deformable to different shapes with proper handling or customization. 3D printing technology will be used to form supporting parts. Microcontrollers (about 3cm square) from DigiKey will be imbedded into this personal device. An Android based mobile APP will be developed to talk with microcontrollers to control IPMCs. When personal devices receive information signals, the original shape of the prototype will change to another shape related to the specific sender or types of information sources. This interactive prototype can simultaneously realize multiple methods for conveying haptic information such as dimension, force, and texture due to the flexible array design. We conduct several studies of user experience to explore how users' respond to shape change information.

Virtual reality haptic human dissection.

PubMed

Needham, Caroline; Wilkinson, Caroline; Soames, Roger

2011-01-01

This project aims to create a three-dimensional digital model of the human hand and wrist which can be virtually 'dissected' through a haptic interface. Tissue properties will be added to the various anatomical structures to replicate a realistic look and feel. The project will explore the role of the medical artist and investigate the cross-discipline collaborations required in the field of virtual anatomy. The software will be used to train anatomy students in dissection skills before experience on a real cadaver. The effectiveness of the software will be evaluated and assessed both quantitatively as well as qualitatively.

Method and apparatus for adaptive force and position control of manipulators

NASA Technical Reports Server (NTRS)

Seraji, Homayoun (Inventor)

1989-01-01

The present invention discloses systematic methods and apparatus for the design of real time controllers. Real-time control employs adaptive force/position by use of feedforward and feedback controllers, with the feedforward controller being the inverse of the linearized model of robot dynamics and containing only proportional-double-derivative terms is disclosed. The feedback controller, of the proportional-integral-derivative type, ensures that manipulator joints follow reference trajectories and the feedback controller achieves robust tracking of step-plus-exponential trajectories, all in real time. The adaptive controller includes adaptive force and position control within a hybrid control architecture. The adaptive controller, for force control, achieves tracking of desired force setpoints, and the adaptive position controller accomplishes tracking of desired position trajectories. Circuits in the adaptive feedback and feedforward controllers are varied by adaptation laws.

Analysis of hand contact areas and interaction capabilities during manipulation and exploration.

PubMed

Gonzalez, Franck; Gosselin, Florian; Bachta, Wael

2014-01-01

Manual human-computer interfaces for virtual reality are designed to allow an operator interacting with a computer simulation as naturally as possible. Dexterous haptic interfaces are the best suited for this goal. They give intuitive and efficient control on the environment with haptic and tactile feedback. This paper is aimed at helping in the choice of the interaction areas to be taken into account in the design of such interfaces. The literature dealing with hand interactions is first reviewed in order to point out the contact areas involved in exploration and manipulation tasks. Their frequencies of use are then extracted from existing recordings. The results are gathered in an original graphical interaction map allowing for a simple visualization of the way the hand is used, and compared with a map of mechanoreceptors densities. Then an interaction tree, mapping the relative amount of actions made available through the use of a given contact area, is built and correlated with the losses of hand function induced by amputations. A rating of some existing haptic interfaces and guidelines for their design are finally achieved to illustrate a possible use of the developed graphical tools.

Heterogeneous Deformable Modeling of Bio-Tissues and Haptic Force Rendering for Bio-Object Modeling

NASA Astrophysics Data System (ADS)

Lin, Shiyong; Lee, Yuan-Shin; Narayan, Roger J.

This paper presents a novel technique for modeling soft biological tissues as well as the development of an innovative interface for bio-manufacturing and medical applications. Heterogeneous deformable models may be used to represent the actual internal structures of deformable biological objects, which possess multiple components and nonuniform material properties. Both heterogeneous deformable object modeling and accurate haptic rendering can greatly enhance the realism and fidelity of virtual reality environments. In this paper, a tri-ray node snapping algorithm is proposed to generate a volumetric heterogeneous deformable model from a set of object interface surfaces between different materials. A constrained local static integration method is presented for simulating deformation and accurate force feedback based on the material properties of a heterogeneous structure. Biological soft tissue modeling is used as an example to demonstrate the proposed techniques. By integrating the heterogeneous deformable model into a virtual environment, users can both observe different materials inside a deformable object as well as interact with it by touching the deformable object using a haptic device. The presented techniques can be used for surgical simulation, bio-product design, bio-manufacturing, and medical applications.

Configurable, wearable sensing and vibrotactile feedback system for real-time postural balance and gait training: proof-of-concept.

PubMed

Xu, Junkai; Bao, Tian; Lee, Ung Hee; Kinnaird, Catherine; Carender, Wendy; Huang, Yangjian; Sienko, Kathleen H; Shull, Peter B

2017-10-11

Postural balance and gait training is important for treating persons with functional impairments, however current systems are generally not portable and are unable to train different types of movements. This paper describes a proof-of-concept design of a configurable, wearable sensing and feedback system for real-time postural balance and gait training targeted for home-based treatments and other portable usage. Sensing and vibrotactile feedback are performed via eight distributed, wireless nodes or "Dots" (size: 22.5 × 20.5 × 15.0 mm, weight: 12.0 g) that can each be configured for sensing and/or feedback according to movement training requirements. In the first experiment, four healthy older adults were trained to reduce medial-lateral (M/L) trunk tilt while performing balance exercises. When trunk tilt deviated too far from vertical (estimated via a sensing Dot on the lower spine), vibrotactile feedback (via feedback Dots placed on the left and right sides of the lower torso) cued participants to move away from the vibration and back toward the vertical no feedback zone to correct their posture. A second experiment was conducted with the same wearable system to train six healthy older adults to alter their foot progression angle in real-time by internally or externally rotating their feet while walking. Foot progression angle was estimated via a sensing Dot adhered to the dorsal side of the foot, and vibrotactile feedback was provided via feedback Dots placed on the medial and lateral sides of the mid-shank cued participants to internally or externally rotate their foot away from vibration. In the first experiment, the wearable system enabled participants to significantly reduce trunk tilt and increase the amount of time inside the no feedback zone. In the second experiment, all participants were able to adopt new gait patterns of internal and external foot rotation within two minutes of real-time training with the wearable system. These results suggest that the configurable, wearable sensing and feedback system is portable and effective for different types of real-time human movement training and thus may be suitable for home-based or clinic-based rehabilitation applications.

A Real-Time Brain-Machine Interface Combining Motor Target and Trajectory Intent Using an Optimal Feedback Control Design

PubMed Central

Shanechi, Maryam M.; Williams, Ziv M.; Wornell, Gregory W.; Hu, Rollin C.; Powers, Marissa; Brown, Emery N.

2013-01-01

Real-time brain-machine interfaces (BMI) have focused on either estimating the continuous movement trajectory or target intent. However, natural movement often incorporates both. Additionally, BMIs can be modeled as a feedback control system in which the subject modulates the neural activity to move the prosthetic device towards a desired target while receiving real-time sensory feedback of the state of the movement. We develop a novel real-time BMI using an optimal feedback control design that jointly estimates the movement target and trajectory of monkeys in two stages. First, the target is decoded from neural spiking activity before movement initiation. Second, the trajectory is decoded by combining the decoded target with the peri-movement spiking activity using an optimal feedback control design. This design exploits a recursive Bayesian decoder that uses an optimal feedback control model of the sensorimotor system to take into account the intended target location and the sensory feedback in its trajectory estimation from spiking activity. The real-time BMI processes the spiking activity directly using point process modeling. We implement the BMI in experiments consisting of an instructed-delay center-out task in which monkeys are presented with a target location on the screen during a delay period and then have to move a cursor to it without touching the incorrect targets. We show that the two-stage BMI performs more accurately than either stage alone. Correct target prediction can compensate for inaccurate trajectory estimation and vice versa. The optimal feedback control design also results in trajectories that are smoother and have lower estimation error. The two-stage decoder also performs better than linear regression approaches in offline cross-validation analyses. Our results demonstrate the advantage of a BMI design that jointly estimates the target and trajectory of movement and more closely mimics the sensorimotor control system. PMID:23593130

The Critical Role of Self-Contact for Embodiment in Virtual Reality.

PubMed

Bovet, Sidney; Debarba, Henrique Galvan; Herbelin, Bruno; Molla, Eray; Boulic, Ronan

2018-04-01

With the broad range of motion capture devices available on the market, it is now commonplace to directly control the limb movement of an avatar during immersion in a virtual environment. Here, we study how the subjective experience of embodying a full-body controlled avatar is influenced by motor alteration and self-contact mismatches. Self-contact is in particular a strong source of passive haptic feedback and we assume it to bring a clear benefit in terms of embodiment. For evaluating this hypothesis, we experimentally manipulate self-contacts and the virtual hand displacement relatively to the body. We introduce these body posture transformations to experimentally reproduce the imperfect or incorrect mapping between real and virtual bodies, with the goal of quantifying the limits of acceptance for distorted mapping on the reported body ownership and agency. We first describe how we exploit egocentric coordinate representations to perform a motion capture ensuring that real and virtual hands coincide whenever the real hand is in contact with the body. Then, we present a pilot study that focuses on quantifying our sensitivity to visuo-tactile mismatches. The results are then used to design our main study with two factors, offset (for self-contact) and amplitude (for movement amplification). Our main result shows that subjects' embodiment remains important, even when an artificially amplified movement of the hand was performed, but provided that correct self-contacts are ensured.

Three Dimensional Projection Environment for Molecular Design and Surgical Simulation

DTIC Science & Technology

2011-08-01

bypasses the cumbersome meshing process . The deformation model is only comprised of mass nodes, which are generated by sampling the object volume before...force should minimize the penetration volume, the haptic feedback force is derived directly. Additionally, a post- processing technique is developed to...render distinct physi-cal tissue properties across different interaction areas. The proposed approach does not require any pre- processing and is

Using Real-Time Visual Feedback to Improve Posture at Computer Workstations

ERIC Educational Resources Information Center

Sigurdsson, Sigurdur O.; Austin, John

2008-01-01

The purpose of the current study was to examine the effects of a multicomponent intervention that included discrimination training, real-time visual feedback, and self-monitoring on postural behavior at a computer workstation in a simulated office environment. Using a nonconcurrent multiple baseline design across 8 participants, the study assessed…

Real-time optical signal processors employing optical feedback: amplitude and phase control.

PubMed

Gallagher, N C

1976-04-01

The development of real-time coherent optical signal processors has increased the appeal of optical computing techniques in signal processing applications. A major limitation of these real-time systems is the. fact that the optical processing material is generally of a phase-only type. The result is that the spatial filters synthesized with these systems must be either phase-only filters or amplitude-only filters. The main concern of this paper is the application of optical feedback techniques to obtain simultaneous and independent amplitude and phase control of the light passing through the system. It is shown that optical feedback techniques may be employed with phase-only spatial filters to obtain this amplitude and phase control. The feedback system with phase-only filters is compared with other feedback systems that employ combinations of phase-only and amplitude-only filters; it is found that the phase-only system is substantially more flexible than the other two systems investigated.

Aging and solid shape recognition: Vision and haptics.

PubMed

Norman, J Farley; Cheeseman, Jacob R; Adkins, Olivia C; Cox, Andrea G; Rogers, Connor E; Dowell, Catherine J; Baxter, Michael W; Norman, Hideko F; Reyes, Cecia M

2015-10-01

The ability of 114 younger and older adults to recognize naturally-shaped objects was evaluated in three experiments. The participants viewed or haptically explored six randomly-chosen bell peppers (Capsicum annuum) in a study session and were later required to judge whether each of twelve bell peppers was "old" (previously presented during the study session) or "new" (not presented during the study session). When recognition memory was tested immediately after study, the younger adults' (Experiment 1) performance for vision and haptics was identical when the individual study objects were presented once. Vision became superior to haptics, however, when the individual study objects were presented multiple times. When 10- and 20-min delays (Experiment 2) were inserted in between study and test sessions, no significant differences occurred between vision and haptics: recognition performance in both modalities was comparable. When the recognition performance of older adults was evaluated (Experiment 3), a negative effect of age was found for visual shape recognition (younger adults' overall recognition performance was 60% higher). There was no age effect, however, for haptic shape recognition. The results of the present experiments indicate that the visual recognition of natural object shape is different from haptic recognition in multiple ways: visual shape recognition can be superior to that of haptics and is affected by aging, while haptic shape recognition is less accurate and unaffected by aging. Copyright © 2015 Elsevier Ltd. All rights reserved.

Mixed reality temporal bone surgical dissector: mechanical design

PubMed Central

2014-01-01

Objective The Development of a Novel Mixed Reality (MR) Simulation. An evolving training environment emphasizes the importance of simulation. Current haptic temporal bone simulators have difficulty representing realistic contact forces and while 3D printed models convincingly represent vibrational properties of bone, they cannot reproduce soft tissue. This paper introduces a mixed reality model, where the effective elements of both simulations are combined; haptic rendering of soft tissue directly interacts with a printed bone model. This paper addresses one aspect in a series of challenges, specifically the mechanical merger of a haptic device with an otic drill. This further necessitates gravity cancelation of the work assembly gripper mechanism. In this system, the haptic end-effector is replaced by a high-speed drill and the virtual contact forces need to be repositioned to the drill tip from the mid wand. Previous publications detail generation of both the requisite printed and haptic simulations. Method Custom software was developed to reposition the haptic interaction point to the drill tip. A custom fitting, to hold the otic drill, was developed and its weight was offset using the haptic device. The robustness of the system to disturbances and its stable performance during drilling were tested. The experiments were performed on a mixed reality model consisting of two drillable rapid-prototyped layers separated by a free-space. Within the free-space, a linear virtual force model is applied to simulate drill contact with soft tissue. Results Testing illustrated the effectiveness of gravity cancellation. Additionally, the system exhibited excellent performance given random inputs and during the drill’s passage between real and virtual components of the model. No issues with registration at model boundaries were encountered. Conclusion These tests provide a proof of concept for the initial stages in the development of a novel mixed-reality temporal bone simulator. PMID:25927300

Rehabilitation of activities of daily living in virtual environments with intuitive user interface and force feedback.

PubMed

Chiang, Vico Chung-Lim; Lo, King-Hung; Choi, Kup-Sze

2017-10-01

To investigate the feasibility of using a virtual rehabilitation system with intuitive user interface and force feedback to improve the skills in activities of daily living (ADL). A virtual training system equipped with haptic devices was developed for the rehabilitation of three ADL tasks - door unlocking, water pouring and meat cutting. Twenty subjects with upper limb disabilities, supervised by two occupational therapists, received a four-session training using the system. The task completion time and the amount of water poured into a virtual glass were recorded. The performance of the three tasks in reality was assessed before and after the virtual training. Feedback of the participants was collected with questionnaires after the study. The completion time of the virtual tasks decreased during the training (p < 0.01) while the percentage of water successfully poured increased (p = 0.051). The score of the Borg scale of perceived exertion was 1.05 (SD = 1.85; 95% CI =  0.18-1.92) and that of the task specific feedback questionnaire was 31 (SD =  4.85; 95% CI =  28.66-33.34). The feedback of the therapists suggested a positive rehabilitation effect. The participants had positive perception towards the system. The system can potentially be used as a tool to complement conventional rehabilitation approaches of ADL. Implications for rehabilitation Rehabilitation of activities of daily living can be facilitated using computer-assisted approaches. The existing approaches focus on cognitive training rather than the manual skills. A virtual training system with intuitive user interface and force feedback was designed to improve the learning of the manual skills. The study shows that system could be used as a training tool to complement conventional rehabilitation approaches.

The System-Wide Effect of Real-Time Audiovisual Feedback and Postevent Debriefing for In-Hospital Cardiac Arrest: The Cardiopulmonary Resuscitation Quality Improvement Initiative*

PubMed Central

Couper, Keith; Kimani, Peter K.; Abella, Benjamin S.; Chilwan, Mehboob; Cooke, Matthew W.; Davies, Robin P.; Field, Richard A.; Gao, Fang; Quinton, Sarah; Stallard, Nigel; Woolley, Sarah

2015-01-01

Objective: To evaluate the effect of implementing real-time audiovisual feedback with and without postevent debriefing on survival and quality of cardiopulmonary resuscitation quality at in-hospital cardiac arrest. Design: A two-phase, multicentre prospective cohort study. Setting: Three UK hospitals, all part of one National Health Service Acute Trust. Patients: One thousand three hundred and ninety-five adult patients who sustained an in-hospital cardiac arrest at the study hospitals and were treated by hospital emergency teams between November 2009 and May 2013. Interventions: During phase 1, quality of cardiopulmonary resuscitation and patient outcomes were measured with no intervention implemented. During phase 2, staff at hospital 1 received real-time audiovisual feedback, whereas staff at hospital 2 received real-time audiovisual feedback supplemented by postevent debriefing. No intervention was implemented at hospital 3 during phase 2. Measurements and Main Results: The primary outcome was return of spontaneous circulation. Secondary endpoints included other patient-focused outcomes, such as survival to hospital discharge, and process-focused outcomes, such as chest compression depth. Random-effect logistic and linear regression models, adjusted for baseline patient characteristics, were used to analyze the effect of the interventions on study outcomes. In comparison with no intervention, neither real-time audiovisual feedback (adjusted odds ratio, 0.62; 95% CI, 0.31–1.22; p = 0.17) nor real-time audiovisual feedback supplemented by postevent debriefing (adjusted odds ratio, 0.65; 95% CI, 0.35–1.21; p = 0.17) was associated with a statistically significant improvement in return of spontaneous circulation or any process-focused outcome. Despite this, there was evidence of a system-wide improvement in phase 2, leading to improvements in return of spontaneous circulation (adjusted odds ratio, 1.87; 95% CI, 1.06–3.30; p = 0.03) and process-focused outcomes. Conclusions: Implementation of real-time audiovisual feedback with or without postevent debriefing did not lead to a measured improvement in patient or process-focused outcomes at individual hospital sites. However, there was an unexplained system-wide improvement in return of spontaneous circulation and process-focused outcomes during the second phase of the study. PMID:26186567

Automated near-real-time clinical performance feedback for anesthesiology residents: one piece of the milestones puzzle.

PubMed

Ehrenfeld, Jesse M; McEvoy, Matthew D; Furman, William R; Snyder, Dylan; Sandberg, Warren S

2014-01-01

Anesthesiology residencies are developing trainee assessment tools to evaluate 25 milestones that map to the six core competencies. The effort will be facilitated by development of automated methods to capture, assess, and report trainee performance to program directors, the Accreditation Council for Graduate Medical Education and the trainees themselves. The authors leveraged a perioperative information management system to develop an automated, near-real-time performance capture and feedback tool that provides objective data on clinical performance and requires minimal administrative effort. Before development, the authors surveyed trainees about satisfaction with clinical performance feedback and about preferences for future feedback. Resident performance on 24,154 completed cases has been incorporated into the authors' automated dashboard, and trainees now have access to their own performance data. Eighty percent (48 of 60) of the residents responded to the feedback survey. Overall, residents "agreed/strongly agreed" that they desire frequent updates on their clinical performance on defined quality metrics and that they desired to see how they compared with the residency as a whole. Before deployment of the new tool, they "disagreed" that they were receiving feedback in a timely manner. Survey results were used to guide the format of the feedback tool that has been implemented. The authors demonstrate the implementation of a system that provides near-real-time feedback concerning resident performance on an extensible series of quality metrics, and which is responsive to requests arising from resident feedback about desired reporting mechanisms.

Automated Near Real-Time Clinical Performance Feedback for Anesthesiology Residents: One Piece of the Milestones Puzzle

PubMed Central

Ehrenfeld, Jesse M.; McEvoy, Matthew D.; Furman, William R.; Snyder, Dylan; Sandberg, Warren S.

2014-01-01

Background Anesthesiology residencies are developing trainee assessment tools to evaluate 25 milestones that map to the 6 core competencies. The effort will be facilitated by development of automated methods to capture, assess, and report trainee performance to program directors, the Accreditation Council for Graduate Medical Education and the trainees themselves. Methods We leveraged a perioperative information management system to develop an automated, near-real-time performance capture and feedback tool that provides objective data on clinical performance and requires minimal administrative effort. Prior to development, we surveyed trainees about satisfaction with clinical performance feedback and about preferences for future feedback. Results Resident performance on 24,154 completed cases has been incorporated into our automated dashboard, and trainees now have access to their own performance data. Eighty percent (48 of 60) of our residents responded to the feedback survey. Overall, residents ‘agreed/strongly agreed’ that they desire frequent updates on their clinical performance on defined quality metrics and that they desired to see how they compared to the residency as a whole. Prior to deployment of the new tool, they ‘disagreed’ that they were receiving feedback in a timely manner. Survey results were used to guide the format of the feedback tool that has been implemented. Conclusions We demonstrate the implementation of a system that provides near real-time feedback concerning resident performance on an extensible series of quality metrics, and which is responsive to requests arising from resident feedback about desired reporting mechanisms. PMID:24398735

Evaluation of a virtual-reality-based simulator using passive haptic feedback for knee arthroscopy.

PubMed

Fucentese, Sandro F; Rahm, Stefan; Wieser, Karl; Spillmann, Jonas; Harders, Matthias; Koch, Peter P

2015-04-01

The aim of this work is to determine face validity and construct validity of a new virtual-reality-based simulator for diagnostic and therapeutic knee arthroscopy. The study tests a novel arthroscopic simulator based on passive haptics. Sixty-eight participants were grouped into novices, intermediates, and experts. All participants completed two exercises. In order to establish face validity, all participants filled out a questionnaire concerning different aspects of simulator realism, training capacity, and different statements using a seven-point Likert scale (range 1-7). Construct validity was tested by comparing various simulator metric values between novices and experts. Face validity could be established: overall realism was rated with a mean value of 5.5 points. Global training capacity scored a mean value of 5.9. Participants considered the simulator as useful for procedural training of diagnostic and therapeutic arthroscopy. In the foreign body removal exercise, experts were overall significantly faster in the whole procedure (6 min 24 s vs. 8 min 24 s, p < 0.001), took less time to complete the diagnostic tour (2 min 49 s vs. 3 min 32 s, p = 0.027), and had a shorter camera path length (186 vs. 246 cm, p = 0.006). The simulator achieved high scores in terms of realism. It was regarded as a useful training tool, which is also capable of differentiating between varying levels of arthroscopic experience. Nevertheless, further improvements of the simulator especially in the field of therapeutic arthroscopy are desirable. In general, the findings support that virtual-reality-based simulation using passive haptics has the potential to complement conventional training of knee arthroscopy skills. II.

Tactile Evaluation Feedback System for Multi-Layered Structure Inspired by Human Tactile Perception Mechanism.

PubMed

Hashim, Iza Husna Mohamad; Kumamoto, Shogo; Takemura, Kenjiro; Maeno, Takashi; Okuda, Shin; Mori, Yukio

2017-11-11

Tactile sensation is one type of valuable feedback in evaluating a product. Conventionally, sensory evaluation is used to get direct subjective responses from the consumers, in order to improve the product's quality. However, this method is a time-consuming and costly process. Therefore, this paper proposes a novel tactile evaluation system that can give tactile feedback from a sensor's output. The main concept of this system is hierarchically layering the tactile sensation, which is inspired by the flow of human perception. The tactile sensation is classified from low-order of tactile sensation (LTS) to high-order of tactile sensation (HTS), and also to preference. Here, LTS will be correlated with physical measures. Furthermore, the physical measures that are used to correlate with LTS are selected based on four main aspects of haptic information (roughness, compliance, coldness, and slipperiness), which are perceived through human tactile sensors. By using statistical analysis, the correlation between each hierarchy was obtained, and the preference was derived in terms of physical measures. A verification test was conducted by using unknown samples to determine the reliability of the system. The results showed that the system developed was capable of estimating preference with an accuracy of approximately 80%.

Depth Camera-Based 3D Hand Gesture Controls with Immersive Tactile Feedback for Natural Mid-Air Gesture Interactions

PubMed Central

Kim, Kwangtaek; Kim, Joongrock; Choi, Jaesung; Kim, Junghyun; Lee, Sangyoun

2015-01-01

Vision-based hand gesture interactions are natural and intuitive when interacting with computers, since we naturally exploit gestures to communicate with other people. However, it is agreed that users suffer from discomfort and fatigue when using gesture-controlled interfaces, due to the lack of physical feedback. To solve the problem, we propose a novel complete solution of a hand gesture control system employing immersive tactile feedback to the user's hand. For this goal, we first developed a fast and accurate hand-tracking algorithm with a Kinect sensor using the proposed MLBP (modified local binary pattern) that can efficiently analyze 3D shapes in depth images. The superiority of our tracking method was verified in terms of tracking accuracy and speed by comparing with existing methods, Natural Interaction Technology for End-user (NITE), 3D Hand Tracker and CamShift. As the second step, a new tactile feedback technology with a piezoelectric actuator has been developed and integrated into the developed hand tracking algorithm, including the DTW (dynamic time warping) gesture recognition algorithm for a complete solution of an immersive gesture control system. The quantitative and qualitative evaluations of the integrated system were conducted with human subjects, and the results demonstrate that our gesture control with tactile feedback is a promising technology compared to a vision-based gesture control system that has typically no feedback for the user's gesture inputs. Our study provides researchers and designers with informative guidelines to develop more natural gesture control systems or immersive user interfaces with haptic feedback. PMID:25580901

Depth camera-based 3D hand gesture controls with immersive tactile feedback for natural mid-air gesture interactions.

PubMed

Kim, Kwangtaek; Kim, Joongrock; Choi, Jaesung; Kim, Junghyun; Lee, Sangyoun

2015-01-08

Vision-based hand gesture interactions are natural and intuitive when interacting with computers, since we naturally exploit gestures to communicate with other people. However, it is agreed that users suffer from discomfort and fatigue when using gesture-controlled interfaces, due to the lack of physical feedback. To solve the problem, we propose a novel complete solution of a hand gesture control system employing immersive tactile feedback to the user's hand. For this goal, we first developed a fast and accurate hand-tracking algorithm with a Kinect sensor using the proposed MLBP (modified local binary pattern) that can efficiently analyze 3D shapes in depth images. The superiority of our tracking method was verified in terms of tracking accuracy and speed by comparing with existing methods, Natural Interaction Technology for End-user (NITE), 3D Hand Tracker and CamShift. As the second step, a new tactile feedback technology with a piezoelectric actuator has been developed and integrated into the developed hand tracking algorithm, including the DTW (dynamic time warping) gesture recognition algorithm for a complete solution of an immersive gesture control system. The quantitative and qualitative evaluations of the integrated system were conducted with human subjects, and the results demonstrate that our gesture control with tactile feedback is a promising technology compared to a vision-based gesture control system that has typically no feedback for the user's gesture inputs. Our study provides researchers and designers with informative guidelines to develop more natural gesture control systems or immersive user interfaces with haptic feedback.

Does real-time objective feedback and competition improve performance and quality in manikin CPR training--a prospective observational study from several European EMS.

PubMed

Smart, J R; Kranz, K; Carmona, F; Lindner, T W; Newton, A

2015-10-15

Previous studies have reported that the quality of cardiopulmonary resuscitation (CPR) is important for patient survival. Real time objective feedback during manikin training has been shown to improve CPR performance. Objective measurement could facilitate competition and help motivate participants to improve their CPR performance. The aims of this study were to investigate whether real time objective feedback on manikins helps improve CPR performance and whether competition between separate European Emergency Medical Services (EMS) and between participants at each EMS helps motivation to train. Ten European EMS took part in the study and was carried out in two stages. At Stage 1, each EMS provided 20 pre-hospital professionals. A questionnaire was completed and standardised assessment scenarios were performed for adult and infant out of hospital cardiac arrest (OHCA). CPR performance was objectively measured and recorded but no feedback given. Between Stage 1 and 2, each EMS was given access to manikins for 6 months and instructed on how to use with objective real-time CPR feedback available. Stage 2 was undertaken and was a repeat of Stage 1 with a questionnaire with additional questions relating to usefulness of feedback and the competition nature of the study (using a 10 point Likert score). The EMS that improved the most from Stage 1 to Stage 2 was declared the winner. An independent samples Student t-test was used to analyse the objective CPR metrics with the significance level taken as p < 0.05. Overall mean Improvement of CPR performance from Stage 1 to Stage 2 was significant. The improvement was greater for the infant assessment. The participants thought the real-time feedback very useful (mean score of 8.5) and very easy to use (mean score of 8.2). Competition between EMS organisations recorded a mean score of 5.8 and competition between participants recorded a mean score of 6.0. The results suggest that the use of real time objective feedback can significantly help improve CPR performance. Competition, especially between participants, appeared to encourage staff to practice and this study suggests that competition might have a useful role to help motivate staff to perform CPR training.

Demonstrating real-time feedback that enhances the performance of measurement sequence with cat states in a cavity

NASA Astrophysics Data System (ADS)

Ofek, N.; Petrenko, A.; Liu, Y.; Vlastakis, B.; Sun, L.; Leghtas, Z.; Heeres, R.; Sliwa, K. M.; Mirrahimi, M.; Jiang, L.; Devoret, M. H.; Schoelkopf, R. J.

2015-03-01

Real-time feedback offers not just the convenience of streamlined data acquisition, but is an essential element in any quantum computational architecture that requires branching based on measurement outcomes. State-preparation, mitigating the effects of qubit decoherence, and recording the trajectories of quantum systems are just a few of the many potential applications of real-time feedback. Photon number parity measurements of cat states in superconducting resonators are a particularly useful platform for demonstrating the clear advantages of having sophisticated feedback schemes to enhance the performance a proposed error-correction protocol [Leghtas et.al. PRL 2013]. In a cQED architecture, where a transmon qubit is coupled to two superconducting cavities, we present a field-programmable gate array (FPGA) device capable of making decisions and calculations with latency times far shorter than the lifetimes of any of the system's constituents. This level of performance opens the door to realizing many complex, previously unfeasible, experiments in superconducting qubit systems.

Getting a grip on reality: Grasping movements directed to real objects and images rely on dissociable neural representations.

PubMed

Freud, Erez; Macdonald, Scott N; Chen, Juan; Quinlan, Derek J; Goodale, Melvyn A; Culham, Jody C

2018-01-01

In the current era of touchscreen technology, humans commonly execute visually guided actions directed to two-dimensional (2D) images of objects. Although real, three-dimensional (3D), objects and images of the same objects share high degree of visual similarity, they differ fundamentally in the actions that can be performed on them. Indeed, previous behavioral studies have suggested that simulated grasping of images relies on different representations than actual grasping of real 3D objects. Yet the neural underpinnings of this phenomena have not been investigated. Here we used functional magnetic resonance imaging (fMRI) to investigate how brain activation patterns differed for grasping and reaching actions directed toward real 3D objects compared to images. Multivoxel Pattern Analysis (MVPA) revealed that the left anterior intraparietal sulcus (aIPS), a key region for visually guided grasping, discriminates between both the format in which objects were presented (real/image) and the motor task performed on them (grasping/reaching). Interestingly, during action planning, the representations of real 3D objects versus images differed more for grasping movements than reaching movements, likely because grasping real 3D objects involves fine-grained planning and anticipation of the consequences of a real interaction. Importantly, this dissociation was evident in the planning phase, before movement initiation, and was not found in any other regions, including motor and somatosensory cortices. This suggests that the dissociable representations in the left aIPS were not based on haptic, motor or proprioceptive feedback. Together, these findings provide novel evidence that actions, particularly grasping, are affected by the realness of the target objects during planning, perhaps because real targets require a more elaborate forward model based on visual cues to predict the consequences of real manipulation. Copyright © 2017 Elsevier Ltd. All rights reserved.

Real-time computer-based visual feedback improves visual acuity in downbeat nystagmus - a pilot study.

PubMed

Teufel, Julian; Bardins, S; Spiegel, Rainer; Kremmyda, O; Schneider, E; Strupp, M; Kalla, R

2016-01-04

Patients with downbeat nystagmus syndrome suffer from oscillopsia, which leads to an unstable visual perception and therefore impaired visual acuity. The aim of this study was to use real-time computer-based visual feedback to compensate for the destabilizing slow phase eye movements. The patients were sitting in front of a computer screen with the head fixed on a chin rest. The eye movements were recorded by an eye tracking system (EyeSeeCam®). We tested the visual acuity with a fixed Landolt C (static) and during real-time feedback driven condition (dynamic) in gaze straight ahead and (20°) sideward gaze. In the dynamic condition, the Landolt C moved according to the slow phase eye velocity of the downbeat nystagmus. The Shapiro-Wilk test was used to test for normal distribution and one-way ANOVA for comparison. Ten patients with downbeat nystagmus were included in the study. Median age was 76 years and the median duration of symptoms was 6.3 years (SD +/- 3.1y). The mean slow phase velocity was moderate during gaze straight ahead (1.44°/s, SD +/- 1.18°/s) and increased significantly in sideward gaze (mean left 3.36°/s; right 3.58°/s). In gaze straight ahead, we found no difference between the static and feedback driven condition. In sideward gaze, visual acuity improved in five out of ten subjects during the feedback-driven condition (p = 0.043). This study provides proof of concept that non-invasive real-time computer-based visual feedback compensates for the SPV in DBN. Therefore, real-time visual feedback may be a promising aid for patients suffering from oscillopsia and impaired text reading on screen. Recent technological advances in the area of virtual reality displays might soon render this approach feasible in fully mobile settings.

Real-time feedback control of twin-screw wet granulation based on image analysis.

PubMed

Madarász, Lajos; Nagy, Zsombor Kristóf; Hoffer, István; Szabó, Barnabás; Csontos, István; Pataki, Hajnalka; Démuth, Balázs; Szabó, Bence; Csorba, Kristóf; Marosi, György

2018-06-04

The present paper reports the first dynamic image analysis-based feedback control of continuous twin-screw wet granulation process. Granulation of the blend of lactose and starch was selected as a model process. The size and size distribution of the obtained particles were successfully monitored by a process camera coupled with an image analysis software developed by the authors. The validation of the developed system showed that the particle size analysis tool can determine the size of the granules with an error of less than 5 µm. The next step was to implement real-time feedback control of the process by controlling the liquid feeding rate of the pump through a PC, based on the real-time determined particle size results. After the establishment of the feedback control, the system could correct different real-life disturbances, creating a Process Analytically Controlled Technology (PACT), which guarantees the real-time monitoring and controlling of the quality of the granules. In the event of changes or bad tendencies in the particle size, the system can automatically compensate the effect of disturbances, ensuring proper product quality. This kind of quality assurance approach is especially important in the case of continuous pharmaceutical technologies. Copyright © 2018 Elsevier B.V. All rights reserved.

An implementation of sensor-based force feedback in a compact laparoscopic surgery robot.

PubMed

Lee, Duk-Hee; Choi, Jaesoon; Park, Jun-Woo; Bach, Du-Jin; Song, Seung-Jun; Kim, Yoon-Ho; Jo, Yungho; Sun, Kyung

2009-01-01

Despite the rapid progress in the clinical application of laparoscopic surgery robots, many shortcomings have not yet been fully overcome, one of which is the lack of reliable haptic feedback. This study implemented a force-feedback structure in our compact laparoscopic surgery robot. The surgery robot is a master-slave configuration robot with 5 DOF (degree of freedom corresponding laparoscopic surgical motion. The force-feedback implementation was made in the robot with torque sensors and controllers installed in the pitch joint of the master and slave robots. A simple dynamic model of action-reaction force in the slave robot was used, through which the reflective force was estimated and fed back to the master robot. The results showed the system model could be identified with significant fidelity and the force feedback at the master robot was feasible. However, the qualitative human assessment of the fed-back force showed only limited level of object discrimination ability. Further developments are underway with this result as a framework.

Telerobotic system concept for real-time soft-tissue imaging during radiotherapy beam delivery.

PubMed

Schlosser, Jeffrey; Salisbury, Kenneth; Hristov, Dimitre

2010-12-01

The curative potential of external beam radiation therapy is critically dependent on having the ability to accurately aim radiation beams at intended targets while avoiding surrounding healthy tissues. However, existing technologies are incapable of real-time, volumetric, soft-tissue imaging during radiation beam delivery, when accurate target tracking is most critical. The authors address this challenge in the development and evaluation of a novel, minimally interfering, telerobotic ultrasound (U.S.) imaging system that can be integrated with existing medical linear accelerators (LINACs) for therapy guidance. A customized human-safe robotic manipulator was designed and built to control the pressure and pitch of an abdominal U.S. transducer while avoiding LINAC gantry collisions. A haptic device was integrated to remotely control the robotic manipulator motion and U.S. image acquisition outside the LINAC room. The ability of the system to continuously maintain high quality prostate images was evaluated in volunteers over extended time periods. Treatment feasibility was assessed by comparing a clinically deployed prostate treatment plan to an alternative plan in which beam directions were restricted to sectors that did not interfere with the transabdominal U.S. transducer. To demonstrate imaging capability concurrent with delivery, robot performance and U.S. target tracking in a phantom were tested with a 15 MV radiation beam active. Remote image acquisition and maintenance of image quality with the haptic interface was successfully demonstrated over 10 min periods in representative treatment setups of volunteers. Furthermore, the robot's ability to maintain a constant probe force and desired pitch angle was unaffected by the LINAC beam. For a representative prostate patient, the dose-volume histogram (DVH) for a plan with restricted sectors remained virtually identical to the DVH of a clinically deployed plan. With reduced margins, as would be enabled by real-time imaging, gross tumor volume coverage was identical while notable reductions of bladder and rectal volumes exposed to large doses were possible. The quality of U.S. images obtained during beam operation was not appreciably degraded by radiofrequency interference and 2D tracking of a phantom object in U.S. images obtained with the beam on/off yielded no significant differences. Remotely controlled robotic U.S. imaging is feasible in the radiotherapy environment and for the first time may offer real-time volumetric soft-tissue guidance concurrent with radiotherapy delivery.

Immersion of virtual reality for rehabilitation - Review.

PubMed

Rose, Tyler; Nam, Chang S; Chen, Karen B

2018-05-01

Virtual reality (VR) shows promise in the application of healthcare and because it presents patients an immersive, often entertaining, approach to accomplish the goal of improvement in performance. Eighteen studies were reviewed to understand human performance and health outcomes after utilizing VR rehabilitation systems. We aimed to understand: (1) the influence of immersion in VR performance and health outcomes; (2) the relationship between enjoyment and potential patient adherence to VR rehabilitation routine; and (3) the influence of haptic feedback on performance in VR. Performance measures including postural stability, navigation task performance, and joint mobility showed varying relations to immersion. Limited data did not allow a solid conclusion between enjoyment and adherence, but patient enjoyment and willingness to participate were reported in care plans that incorporates VR. Finally, different haptic devices such as gloves and controllers provided both strengths and weakness in areas such movement velocity, movement accuracy, and path efficiency. Copyright © 2018 Elsevier Ltd. All rights reserved.

Force, Torque and Stiffness: Interactions in Perceptual Discrimination

PubMed Central

Wu, Bing; Klatzky, Roberta L.; Hollis, Ralph L.

2011-01-01

Three experiments investigated whether force and torque cues interact in haptic discrimination of force, torque and stiffness, and if so, how. The statistical relation between force and torque was manipulated across four experimental conditions: Either one type of cue varied while the other was constant, or both varied so as to be positively correlated, negatively correlated, or uncorrelated. Experiment 1 showed that the subjects’ ability to discriminate force was improved by positively correlated torque but impaired with uncorrelated torque, as compared to the constant torque condition. Corresponding effects were found in Experiment 2 for the influence of force on torque discrimination. These findings indicate that force and torque are integrated in perception, rather than being processed as separate dimensions. A further experiment demonstrated facilitation of stiffness discrimination by correlated force and torque, whether the correlation was positive or negative. The findings suggest new means of augmenting haptic feedback to facilitate perception of the properties of soft objects. PMID:21359137

Programmable prostate palpation simulator using property-changing pneumatic bladder.

PubMed

Talhan, Aishwari; Jeon, Seokhee

2018-05-01

The currently available prostate palpation simulators are based on either a physical mock-up or pure virtual simulation. Both cases have their inherent limitations. The former lacks flexibility in presenting abnormalities and scenarios because of the static nature of the mock-up and has usability issues because the prostate model must be replaced in different scenarios. The latter has realism issues, particularly in haptic feedback, because of the very limited performance of haptic hardware and inaccurate haptic simulation. This paper presents a highly flexible and programmable simulator with high haptic fidelity. Our new approach is based on a pneumatic-driven, property-changing, silicone prostate mock-up that can be embedded in a human torso mannequin. The mock-up has seven pneumatically controlled, multi-layered bladder cells to mimic the stiffness, size, and location changes of nodules in the prostate. The size is controlled by inflating the bladder with positive pressure in the chamber, and a hard nodule can be generated using the particle jamming technique; the fine sand in the bladder becomes stiff when it is vacuumed. The programmable valves and system identification process enable us to precisely control the size and stiffness, which results in a simulator that can realistically generate many different diseases without replacing anything. The three most common abnormalities in a prostate are selected for demonstration, and multiple progressive stages of each abnormality are carefully designed based on medical data. A human perception experiment is performed by actual medical professionals and confirms that our simulator exhibits higher realism and usability than do the conventional simulators. Copyright © 2018 Elsevier Ltd. All rights reserved.

Virtual Reality and Computer-Enhanced Training Devices Equally Improve Laparoscopic Surgical Skill in Novices

PubMed Central

Kanumuri, Prathima; Ganai, Sabha; Wohaibi, Eyad M.; Bush, Ronald W.; Grow, Daniel R.

2008-01-01

Background: The study aim was to compare the effectiveness of virtual reality and computer-enhanced video-scopic training devices for training novice surgeons in complex laparoscopic skills. Methods: Third-year medical students received instruction on laparoscopic intracorporeal suturing and knot tying and then underwent a pretraining assessment of the task using a live porcine model. Students were then randomized to objectives-based training on either the virtual reality (n=8) or computer-enhanced (n=8) training devices for 4 weeks, after which the assessment was repeated. Results: Posttraining performance had improved compared with pretraining performance in both task completion rate (94% versus 18%; P<0.001*) and time [181±58 (SD) versus 292±24*]. Performance of the 2 groups was comparable before and after training. Of the subjects, 88% thought that haptic cues were important in simulators. Both groups agreed that their respective training systems were effective teaching tools, but computer-enhanced device trainees were more likely to rate their training as representative of reality (P<0.01). Conclusions: Training on virtual reality and computer-enhanced devices had equivalent effects on skills improvement in novices. Despite the perception that haptic feedback is important in laparoscopic simulation training, its absence in the virtual reality device did not impede acquisition of skill. PMID:18765042

Vibrotactile stimulation promotes embodiment of an alien hand in amputees with phantom sensations.

PubMed

D'Alonzo, Marco; Clemente, Francesco; Cipriani, Christian

2015-05-01

Tactile feedback is essential to intuitive control and to promote the sense of self-attribution of a prosthetic limb. Recent findings showed that amputees can be tricked to experience this embodiment, when synchronous and modality-matched stimuli are delivered to biological afferent structures and to an alien rubber hand. Hence, it was suggested to exploit this effect by coupling touch sensors in a prosthesis to an array of haptic tactile stimulators in the prosthetic socket. However, this approach is not clinically viable due to physical limits of current haptic devices. To address this issue we have proposed modality-mismatched stimulation and demonstrated that this promotes self-attribution of an alien hand on normally limbed subjects. In this work we investigated whether similar effects could be induced in transradial amputees with referred phantom sensations in a series of experiments fashioned after the Rubber Hand Illusion using vibrotactile stimulators. Results from three independent measures of embodiment demonstrated that vibrotactile sensory substitution elicits body-ownership of a rubber hand in transradial amputees. These results open up promising possibilities in this field; indeed miniature, safe and inexpensive vibrators could be fitted into commercially available prostheses and sockets to induce the illusion every time the prosthesis manipulates an object.

Real-time gas sensing based on optical feedback in a terahertz quantum-cascade laser.

PubMed

Hagelschuer, Till; Wienold, Martin; Richter, Heiko; Schrottke, Lutz; Grahn, Holger T; Hübers, Heinz-Wilhelm

2017-11-27

We report on real-time gas sensing with a terahertz quantum-cascade laser (QCL). The method is solely based on the modulation of the external cavity length, exploiting the intermediate optical feedback regime. While the QCL is operated in continuous-wave mode, optical feedback results in a change of the QCL frequency as well as its terminal voltage. The first effect is exploited to tune the lasing frequency across a molecular absorption line. The second effect is used for the detection of the self-mixing signal. This allows for fast measurement times on the order of 10 ms per spectrum and for real-time measurements of gas concentrations with a rate of 100 Hz. This technique is demonstrated with a mixture of D 2 O and CH 3 OD in an absorption cell.

Real-time feedback can improve infant manikin cardiopulmonary resuscitation by up to 79%--a randomised controlled trial.

PubMed

Martin, Philip; Theobald, Peter; Kemp, Alison; Maguire, Sabine; Maconochie, Ian; Jones, Michael

2013-08-01

European and Advanced Paediatric Life Support training courses. Sixty-nine certified CPR providers. CPR providers were randomly allocated to a 'no-feedback' or 'feedback' group, performing two-thumb and two-finger chest compressions on a "physiological", instrumented resuscitation manikin. Baseline data was recorded without feedback, before chest compressions were repeated with one group receiving feedback. Indices were calculated that defined chest compression quality, based upon comparison of the chest wall displacement to the targets of four, internationally recommended parameters: chest compression depth, release force, chest compression rate and compression duty cycle. Baseline data were consistent with other studies, with <1% of chest compressions performed by providers simultaneously achieving the target of the four internationally recommended parameters. During the 'experimental' phase, 34 CPR providers benefitted from the provision of 'real-time' feedback which, on analysis, coincided with a statistical improvement in compression rate, depth and duty cycle quality across both compression techniques (all measures: p<0.001). Feedback enabled providers to simultaneously achieve the four targets in 75% (two-finger) and 80% (two-thumb) of chest compressions. Real-time feedback produced a dramatic increase in the quality of chest compression (i.e. from <1% to 75-80%). If these results transfer to a clinical scenario this technology could, for the first time, support providers in consistently performing accurate chest compressions during infant CPR and thus potentially improving clinical outcomes. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

a New ER Fluid Based Haptic Actuator System for Virtual Reality

NASA Astrophysics Data System (ADS)

Böse, H.; Baumann, M.; Monkman, G. J.; Egersdörfer, S.; Tunayar, A.; Freimuth, H.; Ermert, H.; Khaled, W.

The concept and some steps in the development of a new actuator system which enables the haptic perception of mechanically inhomogeneous virtual objects are introduced. The system consists of a two-dimensional planar array of actuator elements containing an electrorheological (ER) fluid. When a user presses his fingers onto the surface of the actuator array, he perceives locally variable resistance forces generated by vertical pistons which slide in the ER fluid through the gaps between electrode pairs. The voltage in each actuator element can be individually controlled by a novel sophisticated switching technology based on optoelectric gallium arsenide elements. The haptic information which is represented at the actuator array can be transferred from a corresponding sensor system based on ultrasonic elastography. The combined sensor-actuator system may serve as a technology platform for various applications in virtual reality, like telemedicine where the information on the consistency of tissue of a real patient is detected by the sensor part and recorded by the actuator part at a remote location.

Training voluntary motor suppression with real-time feedback of motor evoked potentials.

PubMed

Majid, D S Adnan; Lewis, Christina; Aron, Adam R

2015-05-01

Training people to suppress motor representations voluntarily could improve response control. We evaluated a novel training procedure of real-time feedback of motor evoked potentials (MEPs) generated by transcranial magnetic stimulation (TMS) over motor cortex. On each trial, a cue instructed participants to use a mental strategy to suppress a particular finger representation without overt movement. A single pulse of TMS was delivered over motor cortex, and an MEP-derived measure of hand motor excitability was delivered visually to the participant within 500 ms. In experiment 1, we showed that participants learned to reduce the excitability of a particular finger beneath baseline (selective motor suppression) within 30 min of practice. In experiment 2, we performed a double-blind study with 2 training groups (1 with veridical feedback and 1 with matched sham feedback) to show that selective motor suppression depends on the veridical feedback itself. Experiment 3 further demonstrated the importance of veridical feedback by showing that selective motor suppression did not arise from mere mental imagery, even when incentivized with reward. Thus participants can use real-time feedback of TMS-induced MEPs to discover an effective mental strategy for selective motor suppression. This high-temporal-resolution, trial-by-trial-feedback training method could be used to help people better control response tendencies and may serve as a potential therapy for motor disorders such as Tourette's and dystonia. Copyright © 2015 the American Physiological Society.

Verticality perception during and after galvanic vestibular stimulation.

PubMed

Volkening, Katharina; Bergmann, Jeannine; Keller, Ingo; Wuehr, Max; Müller, Friedemann; Jahn, Klaus

2014-10-03

The human brain constructs verticality perception by integrating vestibular, somatosensory, and visual information. Here we investigated whether galvanic vestibular stimulation (GVS) has an effect on verticality perception both during and after application, by assessing the subjective verticals (visual, haptic and postural) in healthy subjects at those times. During stimulation the subjective visual vertical and the subjective haptic vertical shifted towards the anode, whereas this shift was reversed towards the cathode in all modalities once stimulation was turned off. Overall, the effects were strongest for the haptic modality. Additional investigation of the time course of GVS-induced changes in the haptic vertical revealed that anodal shifts persisted for the entire 20-min stimulation interval in the majority of subjects. Aftereffects exhibited different types of decay, with a preponderance for an exponential decay. The existence of such reverse effects after stimulation could have implications for GVS-based therapy. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

A review of invasive and non-invasive sensory feedback in upper limb prostheses.

PubMed

Svensson, Pamela; Wijk, Ulrika; Björkman, Anders; Antfolk, Christian

2017-06-01

The constant challenge to restore sensory feedback in prosthetic hands has provided several research solutions, but virtually none has reached clinical fruition. A prosthetic hand with sensory feedback that closely imitates an intact hand and provides a natural feeling may induce the prosthetic hand to be included in the body image and also reinforces the control of the prosthesis. Areas covered: This review presents non-invasive sensory feedback systems such as mechanotactile, vibrotactile, electrotactile and combinational systems which combine the modalities; multi-haptic feedback. Invasive sensory feedback has been tried less, because of the inherent risk, but it has successfully shown to restore some afferent channels. In this review, invasive methods are also discussed, both extraneural and intraneural electrodes, such as cuff electrodes and transverse intrafascicular multichannel electrodes. The focus of the review is on non-invasive methods of providing sensory feedback to upper-limb amputees. Expert commentary: Invoking embodiment has shown to be of importance for the control of prosthesis and acceptance by the prosthetic wearers. It is a challenge to provide conscious feedback to cover the lost sensibility of a hand, not be overwhelming and confusing for the user, and to integrate technology within the constraint of a wearable prosthesis.

Development of a Haptic Interface for Natural Orifice Translumenal Endoscopic Surgery Simulation

PubMed Central

Dargar, Saurabh; Sankaranarayanan, Ganesh

2016-01-01

Natural orifice translumenal endoscopic surgery (NOTES) is a minimally invasive procedure, which utilizes the body’s natural orifices to gain access to the peritoneal cavity. The NOTES procedure is designed to minimize external scarring and patient trauma, however flexible endoscopy based pure NOTES procedures require critical scope handling skills. The delicate nature of the NOTES procedure requires extensive training, thus to improve access to training while reducing risk to patients we have designed and developed the VTEST©, a virtual reality NOTES simulator. As part of the simulator, a novel decoupled 2-DOF haptic device was developed to provide realistic force feedback to the user in training. A series of experiments were performed to determine the behavioral characteristics of the device. The device was found capable of rendering up to 5.62N and 0.190Nm of continuous force and torque in the translational and rotational DOF, respectively. The device possesses 18.1Hz and 5.7Hz of force bandwidth in the translational and rotational DOF, respectively. A feedforward friction compensator was also successfully implemented to minimize the negative impact of friction during the interaction with the device. In this work we have presented the detailed development and evaluation of the haptic device for the VTEST©. PMID:27008674

Opportunities of hydrostatically coupled dielectric elastomer actuators for haptic interfaces

NASA Astrophysics Data System (ADS)

Carpi, Federico; Frediani, Gabriele; De Rossi, Danilo

2011-04-01

As a means to improve versatility and safety of dielectric elastomer actuators (DEAs) for several fields of application, so-called 'hydrostatically coupled' DEAs (HC-DEAs) have recently been described. HC-DEAs are based on an incompressible fluid that mechanically couples a DE-based active part to a passive part interfaced to the load, so as to enable hydrostatic transmission. This paper presents ongoing developments of HC-DEAs and potential applications in the field of haptics. Three specific examples are considered. The first deals with a wearable tactile display used to provide users with tactile feedback during electronic navigation in virtual environments. The display consists of HCDEAs arranged in contact with finger tips. As a second example, an up-scaled prototype version of an 8-dots refreshable cell for dynamic Braille displays is shown. Each Braille dot consists of a miniature HC-DEA, with a diameter lower than 2 mm. The third example refers to a device for finger rehabilitation, conceived to work as a sort of active version of a rehabilitation squeezing ball. The device is designed to dynamically change its compliance according to an electric control. The three examples of applications intend to show the potential of the new technology and the prospective opportunities for haptic interfaces.

ProMIS augmented reality training of laparoscopic procedures face validity.

PubMed

Botden, Sanne M B I; Buzink, Sonja N; Schijven, Marlies P; Jakimowicz, Jack J

2008-01-01

Conventional video trainers lack the ability to assess the trainee objectively, but offer modalities that are often missing in virtual reality simulation, such as realistic haptic feedback. The ProMIS augmented reality laparoscopic simulator retains the benefit of a traditional box trainer, by using original laparoscopic instruments and tactile tasks, but additionally generates objective measures of performance. Fifty-five participants performed a "basic skills" and "suturing and knot-tying" task on ProMIS, after which they filled out a questionnaire regarding realism, haptics, and didactic value of the simulator, on a 5-point-Likert scale. The participants were allotted to 2 experience groups: "experienced" (>50 procedures and >5 sutures; N = 27), and "moderately experienced" (<50 procedures and <5 sutures; N = 28). General consensus among all participants, particularly the experienced, was that ProMIS is a useful tool for training (mean: 4.67, SD: 0.48). It was considered very realistic (mean: 4.44, SD: 0.66), with good haptics (mean: 4.10, SD: 0.97) and didactic value (mean 4.10, SD: 0.65). This study established the face validity of the ProMIS augmented reality simulator for "basic skills" and "suturing and knot-tying" tasks. ProMIS was considered a good tool for training in laparoscopic skills for surgical residents and surgeons.

Feasibility of Measuring Tobacco Smoke Air Pollution in Homes: Report from a Pilot Study

PubMed Central

Rosen, Laura; Zucker, David; Hovell, Melbourne; Brown, Nili; Ram, Amit; Myers, Vicki

2015-01-01

Tobacco smoke air pollution (TSAP) measurement may persuade parents to adopt smoke-free homes and thereby reduce harm to children from tobacco smoke in the home. In a pilot study involving 29 smoking families, a Sidepak was used to continuously monitor home PM2.5 during an 8-h period, Sidepak and/or Dylos monitors provided real-time feedback, and passive nicotine monitors were used to measure home air nicotine for one week. Feedback was provided to participants in the context of motivational interviews. Home PM2.5 levels recorded by continuous monitoring were not well-accepted by participants because of the noise level. Also, graphs from continuous monitoring showed unexplained peaks, often associated with sources unrelated to indoor smoking, such as cooking, construction, or outdoor sources. This hampered delivery of a persuasive message about the relationship between home smoking and TSAP. By contrast, immediate real-time PM2.5 feedback (with Sidepak or Dylos monitor) was feasible and provided unambiguous information; the Dylos had the additional advantages of being more economical and quieter. Air nicotine sampling was complicated by the time-lag for feedback and questions regarding shelf-life. Improvement in the science of TSAP measurement in the home environment is needed to encourage and help maintain smoke-free homes and protect vulnerable children. Recent advances in the use of mobile devices for real-time feedback are promising and warrant further development, as do accurate methods for real-time air nicotine air monitoring. PMID:26633440

Using Tele-Coaching to Increase Behavior-Specific Praise Delivered by Secondary Teachers in an Augmented Reality Learning Environment

ERIC Educational Resources Information Center

Elford, Martha Denton

2013-01-01

This study analyzes the effects of real-time feedback on teacher behavior in an augmented reality simulation environment. Real-time feedback prompts teachers to deliver behavior-specific praise to students in the TeachLivE KU Lab as an evidence-based practice known to decrease disruptive behavior in inclusive classrooms. All educators face the…

A study of the application of singular perturbation theory. [development of a real time algorithm for optimal three dimensional aircraft maneuvers

NASA Technical Reports Server (NTRS)

Mehra, R. K.; Washburn, R. B.; Sajan, S.; Carroll, J. V.

1979-01-01

A hierarchical real time algorithm for optimal three dimensional control of aircraft is described. Systematic methods are developed for real time computation of nonlinear feedback controls by means of singular perturbation theory. The results are applied to a six state, three control variable, point mass model of an F-4 aircraft. Nonlinear feedback laws are presented for computing the optimal control of throttle, bank angle, and angle of attack. Real Time capability is assessed on a TI 9900 microcomputer. The breakdown of the singular perturbation approximation near the terminal point is examined Continuation methods are examined to obtain exact optimal trajectories starting from the singular perturbation solutions.

Real-time control systems: feedback, scheduling and robustness

NASA Astrophysics Data System (ADS)

Simon, Daniel; Seuret, Alexandre; Sename, Olivier

2017-08-01

The efficient control of real-time distributed systems, where continuous components are governed through digital devices and communication networks, needs a careful examination of the constraints arising from the different involved domains inside co-design approaches. Thanks to the robustness of feedback control, both new control methodologies and slackened real-time scheduling schemes are proposed beyond the frontiers between these traditionally separated fields. A methodology to design robust aperiodic controllers is provided, where the sampling interval is considered as a control variable of the system. Promising experimental results are provided to show the feasibility and robustness of the approach.

Fusion interfaces for tactical environments: An application of virtual reality technology

NASA Technical Reports Server (NTRS)

Haas, Michael W.

1994-01-01

The term Fusion Interface is defined as a class of interface which integrally incorporates both virtual and nonvirtual concepts and devices across the visual, auditory, and haptic sensory modalities. A fusion interface is a multisensory virtually-augmented synthetic environment. A new facility has been developed within the Human Engineering Division of the Armstrong Laboratory dedicated to exploratory development of fusion interface concepts. This new facility, the Fusion Interfaces for Tactical Environments (FITE) Facility is a specialized flight simulator enabling efficient concept development through rapid prototyping and direct experience of new fusion concepts. The FITE Facility also supports evaluation of fusion concepts by operation fighter pilots in an air combat environment. The facility is utilized by a multidisciplinary design team composed of human factors engineers, electronics engineers, computer scientists, experimental psychologists, and oeprational pilots. The FITE computational architecture is composed of twenty-five 80486-based microcomputers operating in real-time. The microcomputers generate out-the-window visuals, in-cockpit and head-mounted visuals, localized auditory presentations, haptic displays on the stick and rudder pedals, as well as executing weapons models, aerodynamic models, and threat models.

Real-time simulation of the nonlinear visco-elastic deformations of soft tissues.

PubMed

Basafa, Ehsan; Farahmand, Farzam

2011-05-01

Mass-spring-damper (MSD) models are often used for real-time surgery simulation due to their fast response and fairly realistic deformation replication. An improved real time simulation model of soft tissue deformation due to a laparoscopic surgical indenter was developed and tested. The mechanical realization of conventional MSD models was improved using nonlinear springs and nodal dampers, while their high computational efficiency was maintained using an adapted implicit integration algorithm. New practical algorithms for model parameter tuning, collision detection, and simulation were incorporated. The model was able to replicate complex biological soft tissue mechanical properties under large deformations, i.e., the nonlinear and viscoelastic behaviors. The simulated response of the model after tuning of its parameters to the experimental data of a deer liver sample, closely tracked the reference data with high correlation and maximum relative differences of less than 5 and 10%, for the tuning and testing data sets respectively. Finally, implementation of the proposed model and algorithms in a graphical environment resulted in a real-time simulation with update rates of 150 Hz for interactive deformation and haptic manipulation, and 30 Hz for visual rendering. The proposed real time simulation model of soft tissue deformation due to a laparoscopic surgical indenter was efficient, realistic, and accurate in ex vivo testing. This model is a suitable candidate for testing in vivo during laparoscopic surgery.

Low-Cost Robotic Assessment of Visuo-Motor Deficits in Alzheimer's Disease.

PubMed

Bartoli, Eleonora; Caso, Francesca; Magnani, Giuseppe; Baud-Bovy, Gabriel

2017-07-01

A low-cost robotic interface was used to assess the visuo-motor performance of patients with Alzheimer's disease (AD). Twenty AD patients and twenty age-matched controls participated in this work. The battery of tests included simple reaction times, position tracking, and stabilization tasks performed with both hands. The regularity, velocity, visual and haptic feedback were manipulated to vary movement complexity. Reaction times and movement tracking error were analyzed. Results show a marked group effect on a subset of conditions, in particular when the patients could not rely on the visual feedback of hand movement. The visuo-motor performance correlated with the measures of global cognitive functioning and with different memory-related abilities. Our results support the hypothesis that the ability to recall and use visuo-spatial associations might underlie the impairment in complex motor behavior that has been reported in AD patients. Importantly, the patients had preserved learning effects across sessions, which might relate to visuo-motor deficits being less evident in every-day life and clinical assessments. This robotic assessment, lasting less than 1 h, provides detailed information about the integrity of visuo-motor abilities. The data can aid the understanding of the complex pattern of deficits that characterizes this pervasive disease.

Sensing interactions in the microworld with optical tweezers

NASA Astrophysics Data System (ADS)

Pacoret, Cécile; Bowman, Richard; Gibson, Graham; Sinan, Haliyo D.; Bergander, Arvid; Carberry, David; Régnier, Stéphane; Padgett, Miles

2009-08-01

Optical Tweezers have become a widespread tool in Cell Biology, microengineering and other fields requiring delicate micromanipulation. But for those sensitive tasks, it remains difficult to handle objects without damaging them. As the precision in position and force measurement increase, the richness of information cannot be fully exploited with simple interfaces such as a mouse or a common joystick. For this reason, we propose a haptic force-feedback optical tweezer command and a force-feedback system controlled by one hand. The system combines accurate force measurement using a fast camera and the coupling of these measured forces with a human operator. The overall transparency allows even the feeling of the Brownian motion.

Development and control of a magnetorheological haptic device for robot assisted surgery.

PubMed

Shokrollahi, Elnaz; Goldenberg, Andrew A; Drake, James M; Eastwood, Kyle W; Kang, Matthew

2017-07-01

A prototype magnetorheological (MR) fluid-based actuator has been designed for tele-robotic surgical applications. This device is capable of generating forces up to 47 N, with input currents ranging from 0 to 1.5 A. We begin by outlining the physical design of the device, and then discuss a novel nonlinear model of the device's behavior. The model was developed using the Hammerstein-Wiener (H-W) nonlinear black-box technique and is intended to accurately capture the hysteresis behavior of the MR-fluid. Several experiments were conducted on the device to collect estimation and validation datasets to construct the model and assess its performance. Different estimating functions were used to construct the model, and their effectiveness is assessed based on goodness-of-fit and final-prediction-error measurements. A sigmoid network was found to have a goodness-of-fit of 95%. The model estimate was then used to tune a PID controller. Two control schemes were proposed to eliminate the hysteresis behavior present in the MR fluid device. One method uses a traditional force feedback control loop and the other is based on measuring the magnetic field using a Hall-effect sensor embedded within the device. The Hall-effect sensor scheme was found to be superior in terms of cost, simplicity and real-time control performance compared to the force control strategy.

Haptic Cues for Balance: Use of a Cane Provides Immediate Body Stabilization

PubMed Central

Sozzi, Stefania; Crisafulli, Oscar; Schieppati, Marco

2017-01-01

Haptic cues are important for balance. Knowledge of the temporal features of their effect may be crucial for the design of neural prostheses. Touching a stable surface with a fingertip reduces body sway in standing subjects eyes closed (EC), and removal of haptic cue reinstates a large sway pattern. Changes in sway occur rapidly on changing haptic conditions. Here, we describe the effects and time-course of stabilization produced by a haptic cue derived from a walking cane. We intended to confirm that cane use reduces body sway, to evaluate the effect of vision on stabilization by a cane, and to estimate the delay of the changes in body sway after addition and withdrawal of haptic input. Seventeen healthy young subjects stood in tandem position on a force platform, with eyes closed or open (EO). They gently lowered the cane onto and lifted it from a second force platform. Sixty trials per direction of haptic shift (Touch → NoTouch, T-NT; NoTouch → Touch, NT-T) and visual condition (EC-EO) were acquired. Traces of Center of foot Pressure (CoP) and the force exerted by cane were filtered, rectified, and averaged. The position in space of a reflective marker positioned on the cane tip was also acquired by an optoelectronic device. Cross-correlation (CC) analysis was performed between traces of cane tip and CoP displacement. Latencies of changes in CoP oscillation in the frontal plane EC following the T-NT and NT-T haptic shift were statistically estimated. The CoP oscillations were larger in EC than EO under both T and NT (p < 0.001) and larger during NT than T conditions (p < 0.001). Haptic-induced effect under EC (Romberg quotient NT/T ~ 1.2) was less effective than that of vision under NT condition (EC/EO ~ 1.5) (p < 0.001). With EO cane had little effect. Cane displacement lagged CoP displacement under both EC and EO. Latencies to changes in CoP oscillations were longer after addition (NT-T, about 1.6 s) than withdrawal (T-NT, about 0.9 s) of haptic input (p < 0.001). These latencies were similar to those occurring on fingertip touch, as previously shown. Overall, data speak in favor of substantial equivalence of the haptic information derived from both “direct” fingertip contact and “indirect” contact with the floor mediated by the cane. Cane, finger and visual inputs would be similarly integrated in the same neural centers for balance control. Haptic input from a walking aid and its processing time should be considered when designing prostheses for locomotion. PMID:29311785

Investigations into haptic space and haptic perception of shape for active touch

NASA Astrophysics Data System (ADS)

Sanders, A. F. J.

2008-12-01

This thesis presents a number of psychophysical investigations into haptic space and haptic perception of shape. Haptic perception is understood to include the two subsystems of the cutaneous sense and kinesthesis. Chapter 2 provides an extensive quantitative study into haptic perception of curvature. I investigated bimanual curvature discrimination of cylindrically curved, hand-sized surfaces. I found that discrimination thresholds were in the same range as unimanual thresholds reported in previous studies. Moreover, the distance between the surfaces or the position of the setup with respect to the observer had no effect on thresholds. Finally, I found idiosyncratic biases: A number of observers judged two surfaces that had different radii as equally curved. Biases were of the same order of magnitude as thresholds. In Chapter 3, I investigated haptic space. Here, haptic space is understood to be (1) the set of observer’s judgments of spatial relations in physical space, and (2) a set of constraints by which these judgments are internally consistent. I asked blindfolded observers to construct straight lines in a number of different tasks. I show that the shape of the haptically straight line depends on the task used to produce it. I therefore conclude that there is no unique definition of the haptically straight line and that doubts are cast on the usefulness of the concept of haptic space. In Chapter 4, I present a new experiment into haptic length perception. I show that when observers trace curved pathways with their index finger and judge distance traversed, their distance estimates depend on the geometry of the paths: Lengths of convex, cylindrically curved pathways were overestimated and lengths of concave pathways were underestimated. In addition, I show that a kinematic mechanism must underlie this interaction: (1) the geometry of the path traced by the finger affects movement speed and consequently movement time, and (2) movement time is taken as a measure of traversed length. The study presented in Chapter 5 addresses the question of how kinematic properties of exploratory movements affect perceived shape. I identify a kinematic invariant for the case of a single finger moving across cylindrically curved strips under conditions of slip. I found that the rotation angle of the finger increased linearly with the curvature of the stimulus. In addition, I show that observers took rotation angle as their primary measure of perceived curvature: Observers rotated their finger less on a concave curvature by a constant amount, and consequently, they overestimated the radius of the concave strips compared to the convex ones. Finally, in Chapter 6, I investigated the haptic filled-space illusion for dynamic touch: Observers move their fingertip across an unfilled extent or an extent filled with intermediate stimulations. Previous researchers have reported lengths of filled extents to be overestimated, but the parameters affecting the strength of the illusion are still largely unknown. Factors investigated in this chapter include end point effects, filler density and overall average movement speed.

Real-time control of hind limb functional electrical stimulation using feedback from dorsal root ganglia recordings

NASA Astrophysics Data System (ADS)

Bruns, Tim M.; Wagenaar, Joost B.; Bauman, Matthew J.; Gaunt, Robert A.; Weber, Douglas J.

2013-04-01

Objective. Functional electrical stimulation (FES) approaches often utilize an open-loop controller to drive state transitions. The addition of sensory feedback may allow for closed-loop control that can respond effectively to perturbations and muscle fatigue. Approach. We evaluated the use of natural sensory nerve signals obtained with penetrating microelectrode arrays in lumbar dorsal root ganglia (DRG) as real-time feedback for closed-loop control of FES-generated hind limb stepping in anesthetized cats. Main results. Leg position feedback was obtained in near real-time at 50 ms intervals by decoding the firing rates of more than 120 DRG neurons recorded simultaneously. Over 5 m of effective linear distance was traversed during closed-loop stepping trials in each of two cats. The controller compensated effectively for perturbations in the stepping path when DRG sensory feedback was provided. The presence of stimulation artifacts and the quality of DRG unit sorting did not significantly affect the accuracy of leg position feedback obtained from the linear decoding model as long as at least 20 DRG units were included in the model. Significance. This work demonstrates the feasibility and utility of closed-loop FES control based on natural neural sensors. Further work is needed to improve the controller and electrode technologies and to evaluate long-term viability.

Real-time control of hind limb functional electrical stimulation using feedback from dorsal root ganglia recordings

PubMed Central

Bruns, Tim M; Wagenaar, Joost B; Bauman, Matthew J; Gaunt, Robert A; Weber, Douglas J

2013-01-01

Objective Functional electrical stimulation (FES) approaches often utilize an open-loop controller to drive state transitions. The addition of sensory feedback may allow for closed-loop control that can respond effectively to perturbations and muscle fatigue. Approach We evaluated the use of natural sensory nerve signals obtained with penetrating microelectrode arrays in lumbar dorsal root ganglia (DRG) as real-time feedback for closed-loop control of FES-generated hind limb stepping in anesthetized cats. Main results Leg position feedback was obtained in near real-time at 50 ms intervals by decoding the firing rates of more than 120 DRG neurons recorded simultaneously. Over 5 m of effective linear distance was traversed during closed-loop stepping trials in each of two cats. The controller compensated effectively for perturbations in the stepping path when DRG sensory feedback was provided. The presence of stimulation artifacts and the quality of DRG unit sorting did not significantly affect the accuracy of leg position feedback obtained from the linear decoding model as long as at least 20 DRG units were included in the model. Significance This work demonstrates the feasibility and utility of closed-loop FES control based on natural neural sensors. Further work is needed to improve the controller and electrode technologies and to evaluate long-term viability. PMID:23503062

Development of a smart backboard system for real-time feedback during CPR chest compression on a soft back support surface.

PubMed

Gohier, Francis; Dellimore, Kiran; Scheffer, Cornie

2013-01-01

The quality of cardiopulmonary resuscitation (CPR) is often inconsistent and frequently fails to meet recommended guidelines. One promising approach to address this problem is for clinicians to use an active feedback device during CPR. However, one major deficiency of existing feedback systems is that they fail to account for the displacement of the back support surface during chest compression (CC), which can be important when CPR is performed on a soft surface. In this study we present the development of a real-time CPR feedback system based on an algorithm which uses force and dual-accelerometer measurements to provide accurate estimation of the CC depth on a soft surface, without assuming full chest decompression. Based on adult CPR manikin tests it was found that the accuracy of the estimated CC depth for a dual accelerometer feedback system is significantly better (7.3% vs. 24.4%) than for a single accelerometer system on soft back support surfaces, in the absence or presence of a backboard. In conclusion, the algorithm used was found to be suitable for a real-time, dual accelerometer CPR feedback application since it yielded reasonable accuracy in terms of CC depth estimation, even when used on a soft back support surface.

Defense applications of the CAVE (CAVE automatic virtual environment)

NASA Astrophysics Data System (ADS)

Isabelle, Scott K.; Gilkey, Robert H.; Kenyon, Robert V.; Valentino, George; Flach, John M.; Spenny, Curtis H.; Anderson, Timothy R.

1997-07-01

The CAVE is a multi-person, room-sized, high-resolution, 3D video and auditory environment, which can be used to present very immersive virtual environment experiences. This paper describes the CAVE technology and the capability of the CAVE system as originally developed at the Electronics Visualization Laboratory of the University of Illinois- Chicago and as more recently implemented by Wright State University (WSU) in the Armstrong Laboratory at Wright- Patterson Air Force Base (WPAFB). One planned use of the WSU/WPAFB CAVE is research addressing the appropriate design of display and control interfaces for controlling uninhabited aerial vehicles. The WSU/WPAFB CAVE has a number of features that make it well-suited to this work: (1) 360 degrees surround, plus floor, high resolution visual displays, (2) virtual spatialized audio, (3) the ability to integrate real and virtual objects, and (4) rapid and flexible reconfiguration. However, even though the CAVE is likely to have broad utility for military applications, it does have certain limitations that may make it less well- suited to applications that require 'natural' haptic feedback, vestibular stimulation, or an ability to interact with close detailed objects.

Haptic Feedback Manipulation During Botulinum Toxin Injection Therapy for Focal Hand Dystonia Patients: A Possible New Assistive Strategy.

PubMed

Atashzar, Seyed Farokh; Shahbazi, Mahya; Ward, Christopher; Samotus, Olivia; Delrobaei, Mehdi; Rahimi, Fariborz; Lee, Jack; Jackman, Mallory; Jog, Mandar S; Patel, Rajni V

2016-01-01

Abnormality of sensorimotor integration in the basal ganglia and cortex has been reported in the literature for patients with task-specific focal hand dystonia (FHD). In this study, we investigate the effect of manipulation of kinesthetic input in people living with writer's cramp disorder (a major form of FHD). For this purpose, severity of dystonia is studied for 11 participants while the symptoms of seven participants have been tracked during five sessions of assessment and Botulinum toxin injection (BoNT-A) therapy (one of the current suggested therapies for dystonia). BoNT-A therapy is delivered in the first and the third session. The goal is to analyze the effect of haptic manipulation as a potential assistive technique during BoNT-A therapy. The trial includes writing, hovering, and spiral/sinusoidal drawing subtasks. In each session, the subtasks are repeated twice when (a) a participant uses a normal pen, and (b) when the participant uses a robotics-assisted system (supporting the pen) which provides a compliant virtual writing surface and manipulates the kinesthetic sensory input. The results show (p-value using one-sample t-tests) that reducing the writing surface rigidity significantly decreases the severity of dystonia and results in better control of grip pressure (an indicator of dystonic cramping). It is also shown that (p-value based on paired-samples t-test) using the proposed haptic manipulation strategy, it is possible to augment the effectiveness of BoNT-A therapy. The outcome of this study is then used in the design of an actuated pen as a writing-assistance tool that can provide compliant haptic interaction during writing for FHD patients.

Teleoperated master-slave needle insertion.

PubMed

Abolhassani, Niki; Patel, Rajni V

2009-12-01

Accuracy of needle tip placement and needle tracking in soft tissue are of particular importance in many medical procedures. In recent years, developing autonomous and teleoperated systems for needle insertion has become an active area of research. In this study, needle insertion was performed using a master-slave set-up with multi-degrees of freedom. The effect of force feedback on the accuracy of needle insertion was investigated. In addition, this study compared autonomous, teleoperated and semi-autonomous needle insertion. The results of this study show that incorporation of force feedback can improve teleoperated needle insertion. However, autonomous and semi-autonomous needle insertions, which use feedback from a deflection model, provide significantly better performance. Development of a haptic master-slave needle insertion system, which is capable of performing some autonomous tasks based on feedback from tissue deformation and needle deflection models, can improve the performance of autonomous robotics-based insertions as well as non-autonomous teleoperated manual insertions. Copyright (c) 2009 John Wiley & Sons, Ltd.

Utilization of virtual reality for endotracheal intubation training.

PubMed

Mayrose, James; Kesavadas, T; Chugh, Kevin; Joshi, Dhananjay; Ellis, David G

2003-10-01

Tracheal intubation is performed for urgent airway control in injured patients. Current methods of training include working on cadavers and manikins, which lack the realism of a living human being. Work in this field has been limited due to the complex nature of simulating in real-time, the interactive forces and deformations which occur during an actual patient intubation. This study addressed the issue of intubation training in an attempt to bridge the gap between actual and virtual patient scenarios. The haptic device along with the real-time performance of the simulator give it both visual and physical realism. The three-dimensional viewing and interaction available through virtual reality make it possible for physicians, pre-hospital personnel and students to practice many endotracheal intubations without ever touching a patient. The ability for a medical professional to practice a procedure multiple times prior to performing it on a patient will both enhance the skill of the individual while reducing the risk to the patient.

Method for neural network control of motion using real-time environmental feedback

NASA Technical Reports Server (NTRS)

Buckley, Theresa M. (Inventor)

1997-01-01

A method of motion control for robotics and other automatically controlled machinery using a neural network controller with real-time environmental feedback. The method is illustrated with a two-finger robotic hand having proximity sensors and force sensors that provide environmental feedback signals. The neural network controller is taught to control the robotic hand through training sets using back- propagation methods. The training sets are created by recording the control signals and the feedback signal as the robotic hand or a simulation of the robotic hand is moved through a representative grasping motion. The data recorded is divided into discrete increments of time and the feedback data is shifted out of phase with the control signal data so that the feedback signal data lag one time increment behind the control signal data. The modified data is presented to the neural network controller as a training set. The time lag introduced into the data allows the neural network controller to account for the temporal component of the robotic motion. Thus trained, the neural network controlled robotic hand is able to grasp a wide variety of different objects by generalizing from the training sets.

Augmented reality in surgical procedures

NASA Astrophysics Data System (ADS)

Samset, E.; Schmalstieg, D.; Vander Sloten, J.; Freudenthal, A.; Declerck, J.; Casciaro, S.; Rideng, Ø.; Gersak, B.

2008-02-01

Minimally invasive therapy (MIT) is one of the most important trends in modern medicine. It includes a wide range of therapies in videoscopic surgery and interventional radiology and is performed through small incisions. It reduces hospital stay-time by allowing faster recovery and offers substantially improved cost-effectiveness for the hospital and the society. However, the introduction of MIT has also led to new problems. The manipulation of structures within the body through small incisions reduces dexterity and tactile feedback. It requires a different approach than conventional surgical procedures, since eye-hand co-ordination is not based on direct vision, but more predominantly on image guidance via endoscopes or radiological imaging modalities. ARIS*ER is a multidisciplinary consortium developing a new generation of decision support tools for MIT by augmenting visual and sensorial feedback. We will present tools based on novel concepts in visualization, robotics and haptics providing tailored solutions for a range of clinical applications. Examples from radio-frequency ablation of liver-tumors, laparoscopic liver surgery and minimally invasive cardiac surgery will be presented. Demonstrators were developed with the aim to provide a seamless workflow for the clinical user conducting image-guided therapy.

The effect of perceptual grouping on haptic numerosity perception.

PubMed

Verlaers, K; Wagemans, J; Overvliet, K E

2015-01-01

We used a haptic enumeration task to investigate whether enumeration can be facilitated by perceptual grouping in the haptic modality. Eight participants were asked to count tangible dots as quickly and accurately as possible, while moving their finger pad over a tactile display. In Experiment 1, we manipulated the number and organization of the dots, while keeping the total exploration area constant. The dots were either evenly distributed on a horizontal line (baseline condition) or organized into groups based on either proximity (dots placed in closer proximity to each other) or configural cues (dots placed in a geometric configuration). In Experiment 2, we varied the distance between the subsets of dots. We hypothesized that when subsets of dots can be grouped together, the enumeration time will be shorter and accuracy will be higher than in the baseline condition. The results of both experiments showed faster enumeration for the configural condition than for the baseline condition, indicating that configural grouping also facilitates haptic enumeration. In Experiment 2, faster enumeration was also observed for the proximity condition than for the baseline condition. Thus, perceptual grouping speeds up haptic enumeration by both configural and proximity cues, suggesting that similar mechanisms underlie perceptual grouping in both visual and haptic enumeration.

Antimicrobial Stewardship Program Implementation of a Quality Improvement Intervention Using Real-Time Feedback and an Electronic Order Set for the Management of Staphylococcus aureus Bacteremia.

PubMed

Rosa, Rossana; Zavala, Bruno; Cain, Natalie; Anjan, Shweta; Aragon, Laura; Abbo, Lilian M

2018-03-01

Antimicrobial stewardship programs can optimize the management of Staphylococcus aureus bacteremia by integrating information technology and microbiology laboratory resources. This study describes our experience implementing an intervention consisting of real-time feedback and the use of an electronic order set for the management of S. aureus bacteremia. Infect Control Hosp Epidemiol 2018;39:346-349.