multimodality imaging including: Topics by Science.gov

Sample records for multimodality imaging including

An atlas-based multimodal registration method for 2D images with discrepancy structures.

PubMed

Lv, Wenchao; Chen, Houjin; Peng, Yahui; Li, Yanfeng; Li, Jupeng

2018-06-04

An atlas-based multimodal registration method for 2-dimension images with discrepancy structures was proposed in this paper. Atlas was utilized for complementing the discrepancy structure information in multimodal medical images. The scheme includes three steps: floating image to atlas registration, atlas to reference image registration, and field-based deformation. To evaluate the performance, a frame model, a brain model, and clinical images were employed in registration experiments. We measured the registration performance by the squared sum of intensity differences. Results indicate that this method is robust and performs better than the direct registration for multimodal images with discrepancy structures. We conclude that the proposed method is suitable for multimodal images with discrepancy structures. Graphical Abstract An Atlas-based multimodal registration method schematic diagram.
Supervised learning based multimodal MRI brain tumour segmentation using texture features from supervoxels.

PubMed

Soltaninejad, Mohammadreza; Yang, Guang; Lambrou, Tryphon; Allinson, Nigel; Jones, Timothy L; Barrick, Thomas R; Howe, Franklyn A; Ye, Xujiong

2018-04-01

Accurate segmentation of brain tumour in magnetic resonance images (MRI) is a difficult task due to various tumour types. Using information and features from multimodal MRI including structural MRI and isotropic (p) and anisotropic (q) components derived from the diffusion tensor imaging (DTI) may result in a more accurate analysis of brain images. We propose a novel 3D supervoxel based learning method for segmentation of tumour in multimodal MRI brain images (conventional MRI and DTI). Supervoxels are generated using the information across the multimodal MRI dataset. For each supervoxel, a variety of features including histograms of texton descriptor, calculated using a set of Gabor filters with different sizes and orientations, and first order intensity statistical features are extracted. Those features are fed into a random forests (RF) classifier to classify each supervoxel into tumour core, oedema or healthy brain tissue. The method is evaluated on two datasets: 1) Our clinical dataset: 11 multimodal images of patients and 2) BRATS 2013 clinical dataset: 30 multimodal images. For our clinical dataset, the average detection sensitivity of tumour (including tumour core and oedema) using multimodal MRI is 86% with balanced error rate (BER) 7%; while the Dice score for automatic tumour segmentation against ground truth is 0.84. The corresponding results of the BRATS 2013 dataset are 96%, 2% and 0.89, respectively. The method demonstrates promising results in the segmentation of brain tumour. Adding features from multimodal MRI images can largely increase the segmentation accuracy. The method provides a close match to expert delineation across all tumour grades, leading to a faster and more reproducible method of brain tumour detection and delineation to aid patient management. Copyright © 2018 Elsevier B.V. All rights reserved.
NaGdF4:Nd3+/Yb3+ Nanoparticles as Multimodal Imaging Agents

NASA Astrophysics Data System (ADS)

Pedraza, Francisco; Rightsell, Chris; Kumar, Ga; Giuliani, Jason; Monton, Car; Sardar, Dhiraj

Medical imaging is a fundamental tool used for the diagnosis of numerous ailments. Each imaging modality has unique advantages; however, they possess intrinsic limitations. Some of which include low spatial resolution, sensitivity, penetration depth, and radiation damage. To circumvent this problem, the combination of imaging modalities, or multimodal imaging, has been proposed, such as Near Infrared Fluorescence imaging (NIRF) and Magnetic Resonance Imaging (MRI). Combining individual advantages, specificity and selectivity of NIRF with the deep penetration and high spatial resolution of MRI, it is possible to circumvent their shortcomings for a more robust imaging technique. In addition, both imaging modalities are very safe and minimally invasive. Fluorescent nanoparticles, such as NaGdF4:Nd3 +/Yb3 +, are excellent candidates for NIRF/MRI multimodal imaging. The dopants, Nd and Yb, absorb and emit within the biological window; where near infrared light is less attenuated by soft tissue. This results in less tissue damage and deeper tissue penetration making it a viable candidate in biological imaging. In addition, the inclusion of Gd results in paramagnetic properties, allowing their use as contrast agents in multimodal imaging. The work presented will include crystallographic results, as well as full optical and magnetic characterization to determine the nanoparticle's viability in multimodal imaging.
Multimodal Diffuse Optical Imaging

NASA Astrophysics Data System (ADS)

Intes, Xavier; Venugopal, Vivek; Chen, Jin; Azar, Fred S.

Diffuse optical imaging, particularly diffuse optical tomography (DOT), is an emerging clinical modality capable of providing unique functional information, at a relatively low cost, and with nonionizing radiation. Multimodal diffuse optical imaging has enabled a synergistic combination of functional and anatomical information: the quality of DOT reconstructions has been significantly improved by incorporating the structural information derived by the combined anatomical modality. In this chapter, we will review the basic principles of diffuse optical imaging, including instrumentation and reconstruction algorithm design. We will also discuss the approaches for multimodal imaging strategies that integrate DOI with clinically established modalities. The merit of the multimodal imaging approaches is demonstrated in the context of optical mammography, but the techniques described herein can be translated to other clinical scenarios such as brain functional imaging or muscle functional imaging.
Multi-Modality Phantom Development

DOE Office of Scientific and Technical Information (OSTI.GOV)

Huber, Jennifer S.; Peng, Qiyu; Moses, William W.

2009-03-20

Multi-modality imaging has an increasing role in the diagnosis and treatment of a large number of diseases, particularly if both functional and anatomical information are acquired and accurately co-registered. Hence, there is a resulting need for multi modality phantoms in order to validate image co-registration and calibrate the imaging systems. We present our PET-ultrasound phantom development, including PET and ultrasound images of a simple prostate phantom. We use agar and gelatin mixed with a radioactive solution. We also present our development of custom multi-modality phantoms that are compatible with PET, transrectal ultrasound (TRUS), MRI and CT imaging. We describe bothmore » our selection of tissue mimicking materials and phantom construction procedures. These custom PET-TRUS-CT-MRI prostate phantoms use agargelatin radioactive mixtures with additional contrast agents and preservatives. We show multi-modality images of these custom prostate phantoms, as well as discuss phantom construction alternatives. Although we are currently focused on prostate imaging, this phantom development is applicable to many multi-modality imaging applications.« less
Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking.

PubMed

Yu, Jun; Yang, Xiaokang; Gao, Fei; Tao, Dacheng

2017-12-01

How do we retrieve images accurately? Also, how do we rank a group of images precisely and efficiently for specific queries? These problems are critical for researchers and engineers to generate a novel image searching engine. First, it is important to obtain an appropriate description that effectively represent the images. In this paper, multimodal features are considered for describing images. The images unique properties are reflected by visual features, which are correlated to each other. However, semantic gaps always exist between images visual features and semantics. Therefore, we utilize click feature to reduce the semantic gap. The second key issue is learning an appropriate distance metric to combine these multimodal features. This paper develops a novel deep multimodal distance metric learning (Deep-MDML) method. A structured ranking model is adopted to utilize both visual and click features in distance metric learning (DML). Specifically, images and their related ranking results are first collected to form the training set. Multimodal features, including click and visual features, are collected with these images. Next, a group of autoencoders is applied to obtain initially a distance metric in different visual spaces, and an MDML method is used to assign optimal weights for different modalities. Next, we conduct alternating optimization to train the ranking model, which is used for the ranking of new queries with click features. Compared with existing image ranking methods, the proposed method adopts a new ranking model to use multimodal features, including click features and visual features in DML. We operated experiments to analyze the proposed Deep-MDML in two benchmark data sets, and the results validate the effects of the method.
Radioactive Nanomaterials for Multimodality Imaging

PubMed Central

Chen, Daiqin; Dougherty, Casey A.; Yang, Dongzhi; Wu, Hongwei; Hong, Hao

2016-01-01

Nuclear imaging techniques, including primarily positron emission tomography (PET) and single-photon emission computed tomography (SPECT), can provide quantitative information for a biological event in vivo with ultra-high sensitivity, however, the comparatively low spatial resolution is their major limitation in clinical application. By convergence of nuclear imaging with other imaging modalities like computed tomography (CT), magnetic resonance imaging (MRI) and optical imaging, the hybrid imaging platforms can overcome the limitations from each individual imaging technique. Possessing versatile chemical linking ability and good cargo-loading capacity, radioactive nanomaterials can serve as ideal imaging contrast agents. In this review, we provide a brief overview about current state-of-the-art applications of radioactive nanomaterials in the circumstances of multimodality imaging. We present strategies for incorporation of radioisotope(s) into nanomaterials along with applications of radioactive nanomaterials in multimodal imaging. Advantages and limitations of radioactive nanomaterials for multimodal imaging applications are discussed. Finally, a future perspective of possible radioactive nanomaterial utilization is presented for improving diagnosis and patient management in a variety of diseases. PMID:27227167
Fluorescence Imaging Topography Scanning System for intraoperative multimodal imaging

PubMed Central

Quang, Tri T.; Kim, Hye-Yeong; Bao, Forrest Sheng; Papay, Francis A.; Edwards, W. Barry; Liu, Yang

2017-01-01

Fluorescence imaging is a powerful technique with diverse applications in intraoperative settings. Visualization of three dimensional (3D) structures and depth assessment of lesions, however, are oftentimes limited in planar fluorescence imaging systems. In this study, a novel Fluorescence Imaging Topography Scanning (FITS) system has been developed, which offers color reflectance imaging, fluorescence imaging and surface topography scanning capabilities. The system is compact and portable, and thus suitable for deployment in the operating room without disturbing the surgical flow. For system performance, parameters including near infrared fluorescence detection limit, contrast transfer functions and topography depth resolution were characterized. The developed system was tested in chicken tissues ex vivo with simulated tumors for intraoperative imaging. We subsequently conducted in vivo multimodal imaging of sentinel lymph nodes in mice using FITS and PET/CT. The PET/CT/optical multimodal images were co-registered and conveniently presented to users to guide surgeries. Our results show that the developed system can facilitate multimodal intraoperative imaging. PMID:28437441
Recommendations on nuclear and multimodality imaging in IE and CIED infections.

PubMed

Erba, Paola Anna; Lancellotti, Patrizio; Vilacosta, Isidre; Gaemperli, Oliver; Rouzet, Francois; Hacker, Marcus; Signore, Alberto; Slart, Riemer H J A; Habib, Gilbert

2018-05-24

In the latest update of the European Society of Cardiology (ESC) guidelines for the management of infective endocarditis (IE), imaging is positioned at the centre of the diagnostic work-up so that an early and accurate diagnosis can be reached. Besides echocardiography, contrast-enhanced CT (ce-CT), radiolabelled leucocyte (white blood cell, WBC) SPECT/CT and [ 18 F]FDG PET/CT are included as diagnostic tools in the diagnostic flow chart for IE. Following the clinical guidelines that provided a straightforward message on the role of multimodality imaging, we believe that it is highly relevant to produce specific recommendations on nuclear multimodality imaging in IE and cardiac implantable electronic device infections. In these procedural recommendations we therefore describe in detail the technical and practical aspects of WBC SPECT/CT and [ 18 F]FDG PET/CT, including ce-CT acquisition protocols. We also discuss the advantages and limitations of each procedure, specific pitfalls when interpreting images, and the most important results from the literature, and also provide recommendations on the appropriate use of multimodality imaging.
Integrated photoacoustic microscopy, optical coherence tomography, and fluorescence microscopy for multimodal chorioretinal imaging

NASA Astrophysics Data System (ADS)

Tian, Chao; Zhang, Wei; Nguyen, Van Phuc; Huang, Ziyi; Wang, Xueding; Paulus, Yannis M.

2018-02-01

Current clinical available retinal imaging techniques have limitations, including limited depth of penetration or requirement for the invasive injection of exogenous contrast agents. Here, we developed a novel multimodal imaging system for high-speed, high-resolution retinal imaging of larger animals, such as rabbits. The system integrates three state-of-the-art imaging modalities, including photoacoustic microscopy (PAM), optical coherence tomography (OCT), and fluorescence microscopy (FM). In vivo experimental results of rabbit eyes show that the PAM is able to visualize laser-induced retinal burns and distinguish individual eye blood vessels using a laser exposure dose of 80 nJ, which is well below the American National Standards Institute (ANSI) safety limit 160 nJ. The OCT can discern different retinal layers and visualize laser burns and choroidal detachments. The novel multi-modal imaging platform holds great promise in ophthalmic imaging.
Overlapping-image multimode interference couplers with a reduced number of self-images for uniform and nonuniform power splitting

NASA Astrophysics Data System (ADS)

Bachmann, M.; Besse, P. A.; Melchior, H.

1995-10-01

Overlapping-image multimode interference (MMI) couplers, a new class of devices, permit uniform and nonuniform power splitting. A theoretical description directly relates coupler geometry to image intensities, positions, and phases. Among many possibilities of nonuniform power splitting, examples of 1 \\times 2 couplers with ratios of 15:85 and 28:72 are given. An analysis of uniform power splitters includes the well-known 2 \\times N and 1 \\times N MMI couplers. Applications of MMI couplers include mode filters, mode splitters-combiners, and mode converters.
Design and demonstration of multimodal optical scanning microscopy for confocal and two-photon imaging

NASA Astrophysics Data System (ADS)

Chun, Wanhee; Do, Dukho; Gweon, Dae-Gab

2013-01-01

We developed a multimodal microscopy based on an optical scanning system in order to obtain diverse optical information of the same area of a sample. Multimodal imaging researches have mostly depended on a commercial microscope platform, easy to use but restrictive to extend imaging modalities. In this work, the beam scanning optics, especially including a relay lens, was customized to transfer broadband (400-1000 nm) lights to a sample without any optical error or loss. The customized scanning optics guarantees the best performances of imaging techniques utilizing the lights within the design wavelength. Confocal reflection, confocal fluorescence, and two-photon excitation fluorescence images were obtained, through respective implemented imaging channels, to demonstrate imaging feasibility for near-UV, visible, near-IR continuous light, and pulsed light in the scanning optics. The imaging performances for spatial resolution and image contrast were verified experimentally; the results were satisfactory in comparison with theoretical results. The advantages of customization, containing low cost, outstanding combining ability and diverse applications, will contribute to vitalize multimodal imaging researches.
Multimodal fusion of brain imaging data: A key to finding the missing link(s) in complex mental illness

PubMed Central

Calhoun, Vince D; Sui, Jing

2016-01-01

It is becoming increasingly clear that combining multi-modal brain imaging data is able to provide more information for individual subjects by exploiting the rich multimodal information that exists. However, the number of studies that do true multimodal fusion (i.e. capitalizing on joint information among modalities) is still remarkably small given the known benefits. In part, this is because multi-modal studies require broader expertise in collecting, analyzing, and interpreting the results than do unimodal studies. In this paper, we start by introducing the basic reasons why multimodal data fusion is important and what it can do, and importantly how it can help us avoid wrong conclusions and help compensate for imperfect brain imaging studies. We also discuss the challenges that need to be confronted for such approaches to be more widely applied by the community. We then provide a review of the diverse studies that have used multimodal data fusion (primarily focused on psychosis) as well as provide an introduction to some of the existing analytic approaches. Finally, we discuss some up-and-coming approaches to multi-modal fusion including deep learning and multimodal classification which show considerable promise. Our conclusion is that multimodal data fusion is rapidly growing, but it is still underutilized. The complexity of the human brain coupled with the incomplete measurement provided by existing imaging technology makes multimodal fusion essential in order to mitigate against misdirection and hopefully provide a key to finding the missing link(s) in complex mental illness. PMID:27347565
Multimodal fusion of brain imaging data: A key to finding the missing link(s) in complex mental illness.

PubMed

Calhoun, Vince D; Sui, Jing

2016-05-01

It is becoming increasingly clear that combining multi-modal brain imaging data is able to provide more information for individual subjects by exploiting the rich multimodal information that exists. However, the number of studies that do true multimodal fusion (i.e. capitalizing on joint information among modalities) is still remarkably small given the known benefits. In part, this is because multi-modal studies require broader expertise in collecting, analyzing, and interpreting the results than do unimodal studies. In this paper, we start by introducing the basic reasons why multimodal data fusion is important and what it can do, and importantly how it can help us avoid wrong conclusions and help compensate for imperfect brain imaging studies. We also discuss the challenges that need to be confronted for such approaches to be more widely applied by the community. We then provide a review of the diverse studies that have used multimodal data fusion (primarily focused on psychosis) as well as provide an introduction to some of the existing analytic approaches. Finally, we discuss some up-and-coming approaches to multi-modal fusion including deep learning and multimodal classification which show considerable promise. Our conclusion is that multimodal data fusion is rapidly growing, but it is still underutilized. The complexity of the human brain coupled with the incomplete measurement provided by existing imaging technology makes multimodal fusion essential in order to mitigate against misdirection and hopefully provide a key to finding the missing link(s) in complex mental illness.
A novel automated method for doing registration and 3D reconstruction from multi-modal RGB/IR image sequences

NASA Astrophysics Data System (ADS)

Kirby, Richard; Whitaker, Ross

2016-09-01

In recent years, the use of multi-modal camera rigs consisting of an RGB sensor and an infrared (IR) sensor have become increasingly popular for use in surveillance and robotics applications. The advantages of using multi-modal camera rigs include improved foreground/background segmentation, wider range of lighting conditions under which the system works, and richer information (e.g. visible light and heat signature) for target identification. However, the traditional computer vision method of mapping pairs of images using pixel intensities or image features is often not possible with an RGB/IR image pair. We introduce a novel method to overcome the lack of common features in RGB/IR image pairs by using a variational methods optimization algorithm to map the optical flow fields computed from different wavelength images. This results in the alignment of the flow fields, which in turn produce correspondences similar to those found in a stereo RGB/RGB camera rig using pixel intensities or image features. In addition to aligning the different wavelength images, these correspondences are used to generate dense disparity and depth maps. We obtain accuracies similar to other multi-modal image alignment methodologies as long as the scene contains sufficient depth variations, although a direct comparison is not possible because of the lack of standard image sets from moving multi-modal camera rigs. We test our method on synthetic optical flow fields and on real image sequences that we created with a multi-modal binocular stereo RGB/IR camera rig. We determine our method's accuracy by comparing against a ground truth.
MO-DE-202-03: Image-Guided Surgery and Interventions in the Advanced Multimodality Image-Guided Operating (AMIGO) Suite

DOE Office of Scientific and Technical Information (OSTI.GOV)

Kapur, T.

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guidedmore » neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504 Disclosure and CoI: IGI Technologies, small-business partner on the grants.« less
Dye-enhanced multimodal confocal imaging as a novel approach to intraoperative diagnosis of brain tumors.

PubMed

Snuderl, Matija; Wirth, Dennis; Sheth, Sameer A; Bourne, Sarah K; Kwon, Churl-Su; Ancukiewicz, Marek; Curry, William T; Frosch, Matthew P; Yaroslavsky, Anna N

2013-01-01

Intraoperative diagnosis plays an important role in accurate sampling of brain tumors, limiting the number of biopsies required and improving the distinction between brain and tumor. The goal of this study was to evaluate dye-enhanced multimodal confocal imaging for discriminating gliomas from nonglial brain tumors and from normal brain tissue for diagnostic use. We investigated a total of 37 samples including glioma (13), meningioma (7), metastatic tumors (9) and normal brain removed for nontumoral indications (8). Tissue was stained in 0.05 mg/mL aqueous solution of methylene blue (MB) for 2-5 minutes and multimodal confocal images were acquired using a custom-built microscope. After imaging, tissue was formalin fixed and paraffin embedded for standard neuropathologic evaluation. Thirteen pathologists provided diagnoses based on the multimodal confocal images. The investigated tumor types exhibited distinctive and complimentary characteristics in both the reflectance and fluorescence responses. Images showed distinct morphological features similar to standard histology. Pathologists were able to distinguish gliomas from normal brain tissue and nonglial brain tumors, and to render diagnoses from the images in a manner comparable to haematoxylin and eosin (H&E) slides. These results confirm the feasibility of multimodal confocal imaging for intravital intraoperative diagnosis. © 2012 The Authors; Brain Pathology © 2012 International Society of Neuropathology.
MO-DE-202-04: Multimodality Image-Guided Surgery and Intervention: For the Rest of Us

DOE Office of Scientific and Technical Information (OSTI.GOV)

Shekhar, R.

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guidedmore » neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504 Disclosure and CoI: IGI Technologies, small-business partner on the grants.« less
Image-guided plasma therapy of cutaneous wound

NASA Astrophysics Data System (ADS)

Zhang, Zhiwu; Ren, Wenqi; Yu, Zelin; Zhang, Shiwu; Yue, Ting; Xu, Ronald

2014-02-01

The wound healing process involves the reparative phases of inflammation, proliferation, and remodeling. Interrupting any of these phases may result in chronically unhealed wounds, amputation, or even patient death. Despite the clinical significance in chronic wound management, no effective methods have been developed for quantitative image-guided treatment. We integrated a multimodal imaging system with a cold atmospheric plasma probe for image-guided treatment of chronic wound. Multimodal imaging system offers a non-invasive, painless, simultaneous and quantitative assessment of cutaneous wound healing. Cold atmospheric plasma accelerates the wound healing process through many mechanisms including decontamination, coagulation and stimulation of the wound healing. The therapeutic effect of cold atmospheric plasma is studied in vivo under the guidance of a multimodal imaging system. Cutaneous wounds are created on the dorsal skin of the nude mice. During the healing process, the sample wound is treated by cold atmospheric plasma at different controlled dosage, while the control wound is healed naturally. The multimodal imaging system integrating a multispectral imaging module and a laser speckle imaging module is used to collect the information of cutaneous tissue oxygenation (i.e. oxygen saturation, StO2) and blood perfusion simultaneously to assess and guide the plasma therapy. Our preliminary tests show that cold atmospheric plasma in combination with multimodal imaging guidance has the potential to facilitate the healing of chronic wounds.
High-resolution multimodal clinical multiphoton tomography of skin

NASA Astrophysics Data System (ADS)

König, Karsten

2011-03-01

This review focuses on multimodal multiphoton tomography based on near infrared femtosecond lasers. Clinical multiphoton tomographs for 3D high-resolution in vivo imaging have been placed into the market several years ago. The second generation of this Prism-Award winning High-Tech skin imaging tool (MPTflex) was introduced in 2010. The same year, the world's first clinical CARS studies have been performed with a hybrid multimodal multiphoton tomograph. In particular, non-fluorescent lipids and water as well as mitochondrial fluorescent NAD(P)H, fluorescent elastin, keratin, and melanin as well as SHG-active collagen has been imaged with submicron resolution in patients suffering from psoriasis. Further multimodal approaches include the combination of multiphoton tomographs with low-resolution wide-field systems such as ultrasound, optoacoustical, OCT, and dermoscopy systems. Multiphoton tomographs are currently employed in Australia, Japan, the US, and in several European countries for early diagnosis of skin cancer, optimization of treatment strategies, and cosmetic research including long-term testing of sunscreen nanoparticles as well as anti-aging products.

Multimodal optical imaging system for in vivo investigation of cerebral oxygen delivery and energy metabolism

PubMed Central

Yaseen, Mohammad A.; Srinivasan, Vivek J.; Gorczynska, Iwona; Fujimoto, James G.; Boas, David A.; Sakadžić, Sava

2015-01-01

Improving our understanding of brain function requires novel tools to observe multiple physiological parameters with high resolution in vivo. We have developed a multimodal imaging system for investigating multiple facets of cerebral blood flow and metabolism in small animals. The system was custom designed and features multiple optical imaging capabilities, including 2-photon and confocal lifetime microscopy, optical coherence tomography, laser speckle imaging, and optical intrinsic signal imaging. Here, we provide details of the system’s design and present in vivo observations of multiple metrics of cerebral oxygen delivery and energy metabolism, including oxygen partial pressure, microvascular blood flow, and NADH autofluorescence. PMID:26713212
Radiolabeled Nanoparticles for Multimodality Tumor Imaging

PubMed Central

Xing, Yan; Zhao, Jinhua; Conti, Peter S.; Chen, Kai

2014-01-01

Each imaging modality has its own unique strengths. Multimodality imaging, taking advantages of strengths from two or more imaging modalities, can provide overall structural, functional, and molecular information, offering the prospect of improved diagnostic and therapeutic monitoring abilities. The devices of molecular imaging with multimodality and multifunction are of great value for cancer diagnosis and treatment, and greatly accelerate the development of radionuclide-based multimodal molecular imaging. Radiolabeled nanoparticles bearing intrinsic properties have gained great interest in multimodality tumor imaging over the past decade. Significant breakthrough has been made toward the development of various radiolabeled nanoparticles, which can be used as novel cancer diagnostic tools in multimodality imaging systems. It is expected that quantitative multimodality imaging with multifunctional radiolabeled nanoparticles will afford accurate and precise assessment of biological signatures in cancer in a real-time manner and thus, pave the path towards personalized cancer medicine. This review addresses advantages and challenges in developing multimodality imaging probes by using different types of nanoparticles, and summarizes the recent advances in the applications of radiolabeled nanoparticles for multimodal imaging of tumor. The key issues involved in the translation of radiolabeled nanoparticles to the clinic are also discussed. PMID:24505237
The new frontiers of multimodality and multi-isotope imaging

NASA Astrophysics Data System (ADS)

Behnam Azad, Babak; Nimmagadda, Sridhar

2014-06-01

Technological advances in imaging systems and the development of target specific imaging tracers has been rapidly growing over the past two decades. Recent progress in "all-in-one" imaging systems that allow for automated image coregistration has significantly added to the growth of this field. These developments include ultra high resolution PET and SPECT scanners that can be integrated with CT or MR resulting in PET/CT, SPECT/CT, SPECT/PET and PET/MRI scanners for simultaneous high resolution high sensitivity anatomical and functional imaging. These technological developments have also resulted in drastic enhancements in image quality and acquisition time while eliminating cross compatibility issues between modalities. Furthermore, the most cutting edge technology, though mostly preclinical, also allows for simultaneous multimodality multi-isotope image acquisition and image reconstruction based on radioisotope decay characteristics. These scientific advances, in conjunction with the explosion in the development of highly specific multimodality molecular imaging agents, may aid in realizing simultaneous imaging of multiple biological processes and pave the way towards more efficient diagnosis and improved patient care.
New Finger Biometric Method Using Near Infrared Imaging

PubMed Central

Lee, Eui Chul; Jung, Hyunwoo; Kim, Daeyeoul

2011-01-01

In this paper, we propose a new finger biometric method. Infrared finger images are first captured, and then feature extraction is performed using a modified Gaussian high-pass filter through binarization, local binary pattern (LBP), and local derivative pattern (LDP) methods. Infrared finger images include the multimodal features of finger veins and finger geometries. Instead of extracting each feature using different methods, the modified Gaussian high-pass filter is fully convolved. Therefore, the extracted binary patterns of finger images include the multimodal features of veins and finger geometries. Experimental results show that the proposed method has an error rate of 0.13%. PMID:22163741
MO-DE-202-00: Image-Guided Interventions: Advances in Intraoperative Imaging, Guidance, and An Emerging Role for Medical Physics in Surgery

DOE Office of Scientific and Technical Information (OSTI.GOV)

NONE

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guidedmore » neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504 Disclosure and CoI: IGI Technologies, small-business partner on the grants.« less
MO-DE-202-02: Advances in Image Registration and Reconstruction for Image-Guided Neurosurgery

DOE Office of Scientific and Technical Information (OSTI.GOV)

Siewerdsen, J.

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guidedmore » neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504 Disclosure and CoI: IGI Technologies, small-business partner on the grants.« less
MO-DE-202-01: Image-Guided Focused Ultrasound Surgery and Therapy

DOE Office of Scientific and Technical Information (OSTI.GOV)

Farahani, K.

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guidedmore » neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504 Disclosure and CoI: IGI Technologies, small-business partner on the grants.« less
A Multimodal Approach to Counselor Supervision.

ERIC Educational Resources Information Center

Ponterotto, Joseph G.; Zander, Toni A.

1984-01-01

Represents an initial effort to apply Lazarus's multimodal approach to a model of counselor supervision. Includes continuously monitoring the trainee's behavior, affect, sensations, images, cognitions, interpersonal functioning, and when appropriate, biological functioning (diet and drugs) in the supervisory process. (LLL)
Introduction of a standardized multimodality image protocol for navigation-guided surgery of suspected low-grade gliomas.

PubMed

Mert, Aygül; Kiesel, Barbara; Wöhrer, Adelheid; Martínez-Moreno, Mauricio; Minchev, Georgi; Furtner, Julia; Knosp, Engelbert; Wolfsberger, Stefan; Widhalm, Georg

2015-01-01

OBJECT Surgery of suspected low-grade gliomas (LGGs) poses a special challenge for neurosurgeons due to their diffusely infiltrative growth and histopathological heterogeneity. Consequently, neuronavigation with multimodality imaging data, such as structural and metabolic data, fiber tracking, and 3D brain visualization, has been proposed to optimize surgery. However, currently no standardized protocol has been established for multimodality imaging data in modern glioma surgery. The aim of this study was therefore to define a specific protocol for multimodality imaging and navigation for suspected LGG. METHODS Fifty-one patients who underwent surgery for a diffusely infiltrating glioma with nonsignificant contrast enhancement on MRI and available multimodality imaging data were included. In the first 40 patients with glioma, the authors retrospectively reviewed the imaging data, including structural MRI (contrast-enhanced T1-weighted, T2-weighted, and FLAIR sequences), metabolic images derived from PET, or MR spectroscopy chemical shift imaging, fiber tracking, and 3D brain surface/vessel visualization, to define standardized image settings and specific indications for each imaging modality. The feasibility and surgical relevance of this new protocol was subsequently prospectively investigated during surgery with the assistance of an advanced electromagnetic navigation system in the remaining 11 patients. Furthermore, specific surgical outcome parameters, including the extent of resection, histological analysis of the metabolic hotspot, presence of a new postoperative neurological deficit, and intraoperative accuracy of 3D brain visualization models, were assessed in each of these patients. RESULTS After reviewing these first 40 cases of glioma, the authors defined a specific protocol with standardized image settings and specific indications that allows for optimal and simultaneous visualization of structural and metabolic data, fiber tracking, and 3D brain visualization. This new protocol was feasible and was estimated to be surgically relevant during navigation-guided surgery in all 11 patients. According to the authors' predefined surgical outcome parameters, they observed a complete resection in all resectable gliomas (n = 5) by using contour visualization with T2-weighted or FLAIR images. Additionally, tumor tissue derived from the metabolic hotspot showed the presence of malignant tissue in all WHO Grade III or IV gliomas (n = 5). Moreover, no permanent postoperative neurological deficits occurred in any of these patients, and fiber tracking and/or intraoperative monitoring were applied during surgery in the vast majority of cases (n = 10). Furthermore, the authors found a significant intraoperative topographical correlation of 3D brain surface and vessel models with gyral anatomy and superficial vessels. Finally, real-time navigation with multimodality imaging data using the advanced electromagnetic navigation system was found to be useful for precise guidance to surgical targets, such as the tumor margin or the metabolic hotspot. CONCLUSIONS In this study, the authors defined a specific protocol for multimodality imaging data in suspected LGGs, and they propose the application of this new protocol for advanced navigation-guided procedures optimally in conjunction with continuous electromagnetic instrument tracking to optimize glioma surgery.
Multimodal imaging of lung cancer and its microenvironment (Conference Presentation)

NASA Astrophysics Data System (ADS)

Hariri, Lida P.; Niederst, Matthew J.; Mulvey, Hillary; Adams, David C.; Hu, Haichuan; Chico Calero, Isabel; Szabari, Margit V.; Vakoc, Benjamin J.; Hasan, Tayyaba; Bouma, Brett E.; Engelman, Jeffrey A.; Suter, Melissa J.

2016-03-01

Despite significant advances in targeted therapies for lung cancer, nearly all patients develop drug resistance within 6-12 months and prognosis remains poor. Developing drug resistance is a progressive process that involves tumor cells and their microenvironment. We hypothesize that microenvironment factors alter tumor growth and response to targeted therapy. We conducted in vitro studies in human EGFR-mutant lung carcinoma cells, and demonstrated that factors secreted from lung fibroblasts results in increased tumor cell survival during targeted therapy with EGFR inhibitor, gefitinib. We also demonstrated that increased environment stiffness results in increased tumor survival during gefitinib therapy. In order to test our hypothesis in vivo, we developed a multimodal optical imaging protocol for preclinical intravital imaging in mouse models to assess tumor and its microenvironment over time. We have successfully conducted multimodal imaging of dorsal skinfold chamber (DSC) window mice implanted with GFP-labeled human EGFR mutant lung carcinoma cells and visualized changes in tumor development and microenvironment facets over time. Multimodal imaging included structural OCT to assess tumor viability and necrosis, polarization-sensitive OCT to measure tissue birefringence for collagen/fibroblast detection, and Doppler OCT to assess tumor vasculature. Confocal imaging was also performed for high-resolution visualization of EGFR-mutant lung cancer cells labeled with GFP, and was coregistered with OCT. Our results demonstrated that stromal support and vascular growth are essential to tumor progression. Multimodal imaging is a useful tool to assess tumor and its microenvironment over time.
XML-based scripting of multimodality image presentations in multidisciplinary clinical conferences

NASA Astrophysics Data System (ADS)

Ratib, Osman M.; Allada, Vivekanand; Dahlbom, Magdalena; Marcus, Phillip; Fine, Ian; Lapstra, Lorelle

2002-05-01

We developed a multi-modality image presentation software for display and analysis of images and related data from different imaging modalities. The software is part of a cardiac image review and presentation platform that supports integration of digital images and data from digital and analog media such as videotapes, analog x-ray films and 35 mm cine films. The software supports standard DICOM image files as well as AVI and PDF data formats. The system is integrated in a digital conferencing room that includes projections of digital and analog sources, remote videoconferencing capabilities, and an electronic whiteboard. The goal of this pilot project is to: 1) develop a new paradigm for image and data management for presentation in a clinically meaningful sequence adapted to case-specific scenarios, 2) design and implement a multi-modality review and conferencing workstation using component technology and customizable 'plug-in' architecture to support complex review and diagnostic tasks applicable to all cardiac imaging modalities and 3) develop an XML-based scripting model of image and data presentation for clinical review and decision making during routine clinical tasks and multidisciplinary clinical conferences.
Multimodality imaging of ovarian cystic lesions: Review with an imaging based algorithmic approach

PubMed Central

Wasnik, Ashish P; Menias, Christine O; Platt, Joel F; Lalchandani, Usha R; Bedi, Deepak G; Elsayes, Khaled M

2013-01-01

Ovarian cystic masses include a spectrum of benign, borderline and high grade malignant neoplasms. Imaging plays a crucial role in characterization and pretreatment planning of incidentally detected or suspected adnexal masses, as diagnosis of ovarian malignancy at an early stage is correlated with a better prognosis. Knowledge of differential diagnosis, imaging features, management trends and an algorithmic approach of such lesions is important for optimal clinical management. This article illustrates a multi-modality approach in the diagnosis of a spectrum of ovarian cystic masses and also proposes an algorithmic approach for the diagnosis of these lesions. PMID:23671748
A systematic, multimodality approach to emergency elbow imaging.

PubMed

Singer, Adam D; Hanna, Tarek; Jose, Jean; Datir, Abhijit

2016-01-01

The elbow is a complex synovial hinge joint that is frequently involved in both athletic and nonathletic injuries. A thorough understanding of the normal anatomy and various injury patterns is essential when utilizing diagnostic imaging to identify damaged structures and to assist in surgical planning. In this review, the elbow anatomy will be scrutinized in a systematic approach. This will be followed by a comprehensive presentation of elbow injuries that are commonly seen in the emergency department accompanied by multimodality imaging findings. A short discussion regarding pitfalls in elbow imaging is also included. Copyright © 2015 Elsevier Inc. All rights reserved.
Application of Multimodality Imaging Fusion Technology in Diagnosis and Treatment of Malignant Tumors under the Precision Medicine Plan.

PubMed

Wang, Shun-Yi; Chen, Xian-Xia; Li, Yi; Zhang, Yu-Ying

2016-12-20

The arrival of precision medicine plan brings new opportunities and challenges for patients undergoing precision diagnosis and treatment of malignant tumors. With the development of medical imaging, information on different modality imaging can be integrated and comprehensively analyzed by imaging fusion system. This review aimed to update the application of multimodality imaging fusion technology in the precise diagnosis and treatment of malignant tumors under the precision medicine plan. We introduced several multimodality imaging fusion technologies and their application to the diagnosis and treatment of malignant tumors in clinical practice. The data cited in this review were obtained mainly from the PubMed database from 1996 to 2016, using the keywords of "precision medicine", "fusion imaging", "multimodality", and "tumor diagnosis and treatment". Original articles, clinical practice, reviews, and other relevant literatures published in English were reviewed. Papers focusing on precision medicine, fusion imaging, multimodality, and tumor diagnosis and treatment were selected. Duplicated papers were excluded. Multimodality imaging fusion technology plays an important role in tumor diagnosis and treatment under the precision medicine plan, such as accurate location, qualitative diagnosis, tumor staging, treatment plan design, and real-time intraoperative monitoring. Multimodality imaging fusion systems could provide more imaging information of tumors from different dimensions and angles, thereby offing strong technical support for the implementation of precision oncology. Under the precision medicine plan, personalized treatment of tumors is a distinct possibility. We believe that multimodality imaging fusion technology will find an increasingly wide application in clinical practice.
Biological Parametric Mapping: A Statistical Toolbox for Multi-Modality Brain Image Analysis

PubMed Central

Casanova, Ramon; Ryali, Srikanth; Baer, Aaron; Laurienti, Paul J.; Burdette, Jonathan H.; Hayasaka, Satoru; Flowers, Lynn; Wood, Frank; Maldjian, Joseph A.

2006-01-01

In recent years multiple brain MR imaging modalities have emerged; however, analysis methodologies have mainly remained modality specific. In addition, when comparing across imaging modalities, most researchers have been forced to rely on simple region-of-interest type analyses, which do not allow the voxel-by-voxel comparisons necessary to answer more sophisticated neuroscience questions. To overcome these limitations, we developed a toolbox for multimodal image analysis called biological parametric mapping (BPM), based on a voxel-wise use of the general linear model. The BPM toolbox incorporates information obtained from other modalities as regressors in a voxel-wise analysis, thereby permitting investigation of more sophisticated hypotheses. The BPM toolbox has been developed in MATLAB with a user friendly interface for performing analyses, including voxel-wise multimodal correlation, ANCOVA, and multiple regression. It has a high degree of integration with the SPM (statistical parametric mapping) software relying on it for visualization and statistical inference. Furthermore, statistical inference for a correlation field, rather than a widely-used T-field, has been implemented in the correlation analysis for more accurate results. An example with in-vivo data is presented demonstrating the potential of the BPM methodology as a tool for multimodal image analysis. PMID:17070709
CHARACTERIZING PHOTORECEPTOR CHANGES IN ACUTE POSTERIOR MULTIFOCAL PLACOID PIGMENT EPITHELIOPATHY USING ADAPTIVE OPTICS.

PubMed

Roberts, Philipp K; Nesper, Peter L; Onishi, Alex C; Skondra, Dimitra; Jampol, Lee M; Fawzi, Amani A

2018-01-01

To characterize lesions of acute posterior multifocal placoid pigment epitheliopathy (APMPPE) by multimodal imaging including adaptive optics scanning laser ophthalmoscopy (AOSLO). We included patients with APMPPE at different stages of evolution of the placoid lesions. Color fundus photography, spectral domain optical coherence tomography, infrared reflectance, fundus autofluorescence, and AOSLO images were obtained and registered to correlate microstructural changes. Eight eyes of four patients (two women) were included and analyzed by multimodal imaging. Photoreceptor reflectivity within APMPPE lesions was more heterogeneous than in adjacent healthy areas. Hyperpigmentation on color fundus photography appeared hyperreflective on infrared reflectance and on AOSLO. Irregularity of the interdigitation zone and the photoreceptor inner and outer segment junctions (IS/OS) on spectral domain optical coherence tomography was associated with photoreceptor hyporeflectivity on AOSLO. Interruption of the interdigitation zone or IS/OS was associated with loss of photoreceptor reflectivity on AOSLO. Irregularities in the reflectivity of the photoreceptor mosaic are visible on AOSLO even in inactive APMPPE lesions, where the photoreceptor bands on spectral domain optical coherence tomography have recovered. Adaptive optics scanning laser ophthalmoscopy combined with multimodal imaging has the potential to enhance our understanding of photoreceptor involvement in APMPPE.
Multi-Modality Cascaded Convolutional Neural Networks for Alzheimer's Disease Diagnosis.

PubMed

Liu, Manhua; Cheng, Danni; Wang, Kundong; Wang, Yaping

2018-03-23

Accurate and early diagnosis of Alzheimer's disease (AD) plays important role for patient care and development of future treatment. Structural and functional neuroimages, such as magnetic resonance images (MRI) and positron emission tomography (PET), are providing powerful imaging modalities to help understand the anatomical and functional neural changes related to AD. In recent years, machine learning methods have been widely studied on analysis of multi-modality neuroimages for quantitative evaluation and computer-aided-diagnosis (CAD) of AD. Most existing methods extract the hand-craft imaging features after image preprocessing such as registration and segmentation, and then train a classifier to distinguish AD subjects from other groups. This paper proposes to construct cascaded convolutional neural networks (CNNs) to learn the multi-level and multimodal features of MRI and PET brain images for AD classification. First, multiple deep 3D-CNNs are constructed on different local image patches to transform the local brain image into more compact high-level features. Then, an upper high-level 2D-CNN followed by softmax layer is cascaded to ensemble the high-level features learned from the multi-modality and generate the latent multimodal correlation features of the corresponding image patches for classification task. Finally, these learned features are combined by a fully connected layer followed by softmax layer for AD classification. The proposed method can automatically learn the generic multi-level and multimodal features from multiple imaging modalities for classification, which are robust to the scale and rotation variations to some extent. No image segmentation and rigid registration are required in pre-processing the brain images. Our method is evaluated on the baseline MRI and PET images of 397 subjects including 93 AD patients, 204 mild cognitive impairment (MCI, 76 pMCI +128 sMCI) and 100 normal controls (NC) from Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Experimental results show that the proposed method achieves an accuracy of 93.26% for classification of AD vs. NC and 82.95% for classification pMCI vs. NC, demonstrating the promising classification performance.
Machine learning approaches for integrating clinical and imaging features in late-life depression classification and response prediction.

PubMed

Patel, Meenal J; Andreescu, Carmen; Price, Julie C; Edelman, Kathryn L; Reynolds, Charles F; Aizenstein, Howard J

2015-10-01

Currently, depression diagnosis relies primarily on behavioral symptoms and signs, and treatment is guided by trial and error instead of evaluating associated underlying brain characteristics. Unlike past studies, we attempted to estimate accurate prediction models for late-life depression diagnosis and treatment response using multiple machine learning methods with inputs of multi-modal imaging and non-imaging whole brain and network-based features. Late-life depression patients (medicated post-recruitment) (n = 33) and older non-depressed individuals (n = 35) were recruited. Their demographics and cognitive ability scores were recorded, and brain characteristics were acquired using multi-modal magnetic resonance imaging pretreatment. Linear and nonlinear learning methods were tested for estimating accurate prediction models. A learning method called alternating decision trees estimated the most accurate prediction models for late-life depression diagnosis (87.27% accuracy) and treatment response (89.47% accuracy). The diagnosis model included measures of age, Mini-mental state examination score, and structural imaging (e.g. whole brain atrophy and global white mater hyperintensity burden). The treatment response model included measures of structural and functional connectivity. Combinations of multi-modal imaging and/or non-imaging measures may help better predict late-life depression diagnosis and treatment response. As a preliminary observation, we speculate that the results may also suggest that different underlying brain characteristics defined by multi-modal imaging measures-rather than region-based differences-are associated with depression versus depression recovery because to our knowledge this is the first depression study to accurately predict both using the same approach. These findings may help better understand late-life depression and identify preliminary steps toward personalized late-life depression treatment. Copyright © 2015 John Wiley & Sons, Ltd.
Children Creating Multimodal Stories about a Familiar Environment

ERIC Educational Resources Information Center

Kervin, Lisa; Mantei, Jessica

2017-01-01

Storytelling is a practice that enables children to apply their literacy skills. This article shares a collaborative literacy strategy devised to enable children to create multimodal stories about their familiar school environment. The strategy uses resources, including the children's own drawings, images from Google Maps, and the Puppet Pals…
Image-guided thoracic surgery in the hybrid operation room.

PubMed

Ujiie, Hideki; Effat, Andrew; Yasufuku, Kazuhiro

2017-01-01

There has been an increase in the use of image-guided technology to facilitate minimally invasive therapy. The next generation of minimally invasive therapy is focused on advancement and translation of novel image-guided technologies in therapeutic interventions, including surgery, interventional pulmonology, radiation therapy, and interventional laser therapy. To establish the efficacy of different minimally invasive therapies, we have developed a hybrid operating room, known as the guided therapeutics operating room (GTx OR) at the Toronto General Hospital. The GTx OR is equipped with multi-modality image-guidance systems, which features a dual source-dual energy computed tomography (CT) scanner, a robotic cone-beam CT (CBCT)/fluoroscopy, high-performance endobronchial ultrasound system, endoscopic surgery system, near-infrared (NIR) fluorescence imaging system, and navigation tracking systems. The novel multimodality image-guidance systems allow physicians to quickly, and accurately image patients while they are on the operating table. This yield improved outcomes since physicians are able to use image guidance during their procedures, and carry out innovative multi-modality therapeutics. Multiple preclinical translational studies pertaining to innovative minimally invasive technology is being developed in our guided therapeutics laboratory (GTx Lab). The GTx Lab is equipped with similar technology, and multimodality image-guidance systems as the GTx OR, and acts as an appropriate platform for translation of research into human clinical trials. Through the GTx Lab, we are able to perform basic research, such as the development of image-guided technologies, preclinical model testing, as well as preclinical imaging, and then translate that research into the GTx OR. This OR allows for the utilization of new technologies in cancer therapy, including molecular imaging, and other innovative imaging modalities, and therefore enables a better quality of life for patients, both during and after the procedure. In this article, we describe capabilities of the GTx systems, and discuss the first-in-human technologies used, and evaluated in GTx OR.

Molecular brain imaging in the multimodality era

PubMed Central

Price, Julie C

2012-01-01

Multimodality molecular brain imaging encompasses in vivo visualization, evaluation, and measurement of cellular/molecular processes. Instrumentation and software developments over the past 30 years have fueled advancements in multimodality imaging platforms that enable acquisition of multiple complementary imaging outcomes by either combined sequential or simultaneous acquisition. This article provides a general overview of multimodality neuroimaging in the context of positron emission tomography as a molecular imaging tool and magnetic resonance imaging as a structural and functional imaging tool. Several image examples are provided and general challenges are discussed to exemplify complementary features of the modalities, as well as important strengths and weaknesses of combined assessments. Alzheimer's disease is highlighted, as this clinical area has been strongly impacted by multimodality neuroimaging findings that have improved understanding of the natural history of disease progression, early disease detection, and informed therapy evaluation. PMID:22434068
Advanced Contrast Agents for Multimodal Biomedical Imaging Based on Nanotechnology.

PubMed

Calle, Daniel; Ballesteros, Paloma; Cerdán, Sebastián

2018-01-01

Clinical imaging modalities have reached a prominent role in medical diagnosis and patient management in the last decades. Different image methodologies as Positron Emission Tomography, Single Photon Emission Tomography, X-Rays, or Magnetic Resonance Imaging are in continuous evolution to satisfy the increasing demands of current medical diagnosis. Progress in these methodologies has been favored by the parallel development of increasingly more powerful contrast agents. These are molecules that enhance the intrinsic contrast of the images in the tissues where they accumulate, revealing noninvasively the presence of characteristic molecular targets or differential physiopathological microenvironments. The contrast agent field is currently moving to improve the performance of these molecules by incorporating the advantages that modern nanotechnology offers. These include, mainly, the possibilities to combine imaging and therapeutic capabilities over the same theranostic platform or improve the targeting efficiency in vivo by molecular engineering of the nanostructures. In this review, we provide an introduction to multimodal imaging methods in biomedicine, the sub-nanometric imaging agents previously used and the development of advanced multimodal and theranostic imaging agents based in nanotechnology. We conclude providing some illustrative examples from our own laboratories, including recent progress in theranostic formulations of magnetoliposomes containing ω-3 poly-unsaturated fatty acids to treat inflammatory diseases, or the use of stealth liposomes engineered with a pH-sensitive nanovalve to release their cargo specifically in the acidic extracellular pH microenvironment of tumors.
ADMultiImg: a novel missing modality transfer learning based CAD system for diagnosis of MCI due to AD using incomplete multi-modality imaging data

NASA Astrophysics Data System (ADS)

Liu, Xiaonan; Chen, Kewei; Wu, Teresa; Weidman, David; Lure, Fleming; Li, Jing

2018-02-01

Alzheimer's Disease (AD) is the most common cause of dementia and currently has no cure. Treatments targeting early stages of AD such as Mild Cognitive Impairment (MCI) may be most effective to deaccelerate AD, thus attracting increasing attention. However, MCI has substantial heterogeneity in that it can be caused by various underlying conditions, not only AD. To detect MCI due to AD, NIA-AA published updated consensus criteria in 2011, in which the use of multi-modality images was highlighted as one of the most promising methods. It is of great interest to develop a CAD system based on automatic, quantitative analysis of multi-modality images and machine learning algorithms to help physicians more adequately diagnose MCI due to AD. The challenge, however, is that multi-modality images are not universally available for many patients due to cost, access, safety, and lack of consent. We developed a novel Missing Modality Transfer Learning (MMTL) algorithm capable of utilizing whatever imaging modalities are available for an MCI patient to diagnose the patient's likelihood of MCI due to AD. Furthermore, we integrated MMTL with radiomics steps including image processing, feature extraction, and feature screening, and a post-processing for uncertainty quantification (UQ), and developed a CAD system called "ADMultiImg" to assist clinical diagnosis of MCI due to AD using multi-modality images together with patient demographic and genetic information. Tested on ADNI date, our system can generate a diagnosis with high accuracy even for patients with only partially available image modalities (AUC=0.94), and therefore may have broad clinical utility.
Unifying framework for multimodal brain MRI segmentation based on Hidden Markov Chains.

PubMed

Bricq, S; Collet, Ch; Armspach, J P

2008-12-01

In the frame of 3D medical imaging, accurate segmentation of multimodal brain MR images is of interest for many brain disorders. However, due to several factors such as noise, imaging artifacts, intrinsic tissue variation and partial volume effects, tissue classification remains a challenging task. In this paper, we present a unifying framework for unsupervised segmentation of multimodal brain MR images including partial volume effect, bias field correction, and information given by a probabilistic atlas. Here-proposed method takes into account neighborhood information using a Hidden Markov Chain (HMC) model. Due to the limited resolution of imaging devices, voxels may be composed of a mixture of different tissue types, this partial volume effect is included to achieve an accurate segmentation of brain tissues. Instead of assigning each voxel to a single tissue class (i.e., hard classification), we compute the relative amount of each pure tissue class in each voxel (mixture estimation). Further, a bias field estimation step is added to the proposed algorithm to correct intensity inhomogeneities. Furthermore, atlas priors were incorporated using probabilistic brain atlas containing prior expectations about the spatial localization of different tissue classes. This atlas is considered as a complementary sensor and the proposed method is extended to multimodal brain MRI without any user-tunable parameter (unsupervised algorithm). To validate this new unifying framework, we present experimental results on both synthetic and real brain images, for which the ground truth is available. Comparison with other often used techniques demonstrates the accuracy and the robustness of this new Markovian segmentation scheme.
Radionuclide Myocardial Perfusion Imaging for the Evaluation of Patients With Known or Suspected Coronary Artery Disease in the Era of Multimodality Cardiovascular Imaging

PubMed Central

Taqueti, Viviany R.; Di Carli, Marcelo F.

2018-01-01

Over the last several decades, radionuclide myocardial perfusion imaging (MPI) with single photon emission tomography and positron emission tomography has been a mainstay for the evaluation of patients with known or suspected coronary artery disease (CAD). More recently, technical advances in separate and complementary imaging modalities including coronary computed tomography angiography, computed tomography perfusion, cardiac magnetic resonance imaging, and contrast stress echocardiography have expanded the toolbox of diagnostic testing for cardiac patients. While the growth of available technologies has heralded an exciting era of multimodality cardiovascular imaging, coordinated and dispassionate utilization of these techniques is needed to implement the right test for the right patient at the right time, a promise of “precision medicine.” In this article, we review the maturing role of MPI in the current era of multimodality cardiovascular imaging, particularly in the context of recent advances in myocardial blood flow quantitation, and as applied to the evaluation of patients with known or suspected CAD. PMID:25770849
Use of multidimensional, multimodal imaging and PACS to support neurological diagnoses

NASA Astrophysics Data System (ADS)

Wong, Stephen T. C.; Knowlton, Robert C.; Hoo, Kent S.; Huang, H. K.

1995-05-01

Technological advances in brain imaging have revolutionized diagnosis in neurology and neurological surgery. Major imaging techniques include magnetic resonance imaging (MRI) to visualize structural anatomy, positron emission tomography (PET) to image metabolic function and cerebral blood flow, magnetoencephalography (MEG) to visualize the location of physiologic current sources, and magnetic resonance spectroscopy (MRS) to measure specific biochemicals. Each of these techniques studies different biomedical aspects of the brain, but there lacks an effective means to quantify and correlate the disparate imaging datasets in order to improve clinical decision making processes. This paper describes several techniques developed in a UNIX-based neurodiagnostic workstation to aid the noninvasive presurgical evaluation of epilepsy patients. These techniques include online access to the picture archiving and communication systems (PACS) multimedia archive, coregistration of multimodality image datasets, and correlation and quantitation of structural and functional information contained in the registered images. For illustration, we describe the use of these techniques in a patient case of nonlesional neocortical epilepsy. We also present out future work based on preliminary studies.
Multimodal Discourse Analysis of the Movie "Argo"

ERIC Educational Resources Information Center

Bo, Xu

2018-01-01

Based on multimodal discourse theory, this paper makes a multimodal discourse analysis of some shots in the movie "Argo" from the perspective of context of culture, context of situation and meaning of image. Results show that this movie constructs multimodal discourse through particular context, language and image, and successfully…
Multimodal Imaging of Human Brain Activity: Rational, Biophysical Aspects and Modes of Integration

PubMed Central

Blinowska, Katarzyna; Müller-Putz, Gernot; Kaiser, Vera; Astolfi, Laura; Vanderperren, Katrien; Van Huffel, Sabine; Lemieux, Louis

2009-01-01

Until relatively recently the vast majority of imaging and electrophysiological studies of human brain activity have relied on single-modality measurements usually correlated with readily observable or experimentally modified behavioural or brain state patterns. Multi-modal imaging is the concept of bringing together observations or measurements from different instruments. We discuss the aims of multi-modal imaging and the ways in which it can be accomplished using representative applications. Given the importance of haemodynamic and electrophysiological signals in current multi-modal imaging applications, we also review some of the basic physiology relevant to understanding their relationship. PMID:19547657
Nanoparticles in Higher-Order Multimodal Imaging

NASA Astrophysics Data System (ADS)

Rieffel, James Ki

Imaging procedures are a cornerstone in our current medical infrastructure. In everything from screening, diagnostics, and treatment, medical imaging is perhaps our greatest tool in evaluating individual health. Recently, there has been tremendous increase in the development of multimodal systems that combine the strengths of complimentary imaging technologies to overcome their independent weaknesses. Clinically, this has manifested in the virtually universal manufacture of combined PET-CT scanners. With this push toward more integrated imaging, new contrast agents with multimodal functionality are needed. Nanoparticle-based systems are ideal candidates based on their unique size, properties, and diversity. In chapter 1, an extensive background on recent multimodal imaging agents capable of enhancing signal or contrast in three or more modalities is presented. Chapter 2 discusses the development and characterization of a nanoparticulate probe with hexamodal imaging functionality. It is my hope that the information contained in this thesis will demonstrate the many benefits of nanoparticles in multimodal imaging, and provide insight into the potential of fully integrated imaging.
Use of multimodality imaging and artificial intelligence for diagnosis and prognosis of early stages of Alzheimer's disease.

PubMed

Liu, Xiaonan; Chen, Kewei; Wu, Teresa; Weidman, David; Lure, Fleming; Li, Jing

2018-04-01

Alzheimer's disease (AD) is a major neurodegenerative disease and the most common cause of dementia. Currently, no treatment exists to slow down or stop the progression of AD. There is converging belief that disease-modifying treatments should focus on early stages of the disease, that is, the mild cognitive impairment (MCI) and preclinical stages. Making a diagnosis of AD and offering a prognosis (likelihood of converting to AD) at these early stages are challenging tasks but possible with the help of multimodality imaging, such as magnetic resonance imaging (MRI), fluorodeoxyglucose (FDG)-positron emission topography (PET), amyloid-PET, and recently introduced tau-PET, which provides different but complementary information. This article is a focused review of existing research in the recent decade that used statistical machine learning and artificial intelligence methods to perform quantitative analysis of multimodality image data for diagnosis and prognosis of AD at the MCI or preclinical stages. We review the existing work in 3 subareas: diagnosis, prognosis, and methods for handling modality-wise missing data-a commonly encountered problem when using multimodality imaging for prediction or classification. Factors contributing to missing data include lack of imaging equipment, cost, difficulty of obtaining patient consent, and patient drop-off (in longitudinal studies). Finally, we summarize our major findings and provide some recommendations for potential future research directions. Copyright © 2018 Elsevier Inc. All rights reserved.
GMars-T Enabling Multimodal Subdiffraction Structural and Functional Fluorescence Imaging in Live Cells.

PubMed

Wang, Sheng; Chen, Xuanze; Chang, Lei; Ding, Miao; Xue, Ruiying; Duan, Haifeng; Sun, Yujie

2018-06-05

Fluorescent probes with multimodal and multilevel imaging capabilities are highly valuable as imaging with such probes not only can obtain new layers of information but also enable cross-validation of results under different experimental conditions. In recent years, the development of genetically encoded reversibly photoswitchable fluorescent proteins (RSFPs) has greatly promoted the application of various kinds of live-cell nanoscopy approaches, including reversible saturable optical fluorescence transitions (RESOLFT) and stochastic optical fluctuation imaging (SOFI). However, these two classes of live-cell nanoscopy approaches require different optical characteristics of specific RSFPs. In this work, we developed GMars-T, a monomeric bright green RSFP which can satisfy both RESOLFT and photochromic SOFI (pcSOFI) imaging in live cells. We further generated biosensor based on bimolecular fluorescence complementation (BiFC) of GMars-T which offers high specificity and sensitivity in detecting and visualizing various protein-protein interactions (PPIs) in different subcellular compartments under physiological conditions (e.g., 37 °C) in live mammalian cells. Thus, the newly developed GMars-T can serve as both structural imaging probe with multimodal super-resolution imaging capability and functional imaging probe for reporting PPIs with high specificity and sensitivity based on its derived biosensor.
Development of a multi-scale and multi-modality imaging system to characterize tumours and their microenvironment in vivo

NASA Astrophysics Data System (ADS)

Rouffiac, Valérie; Ser-Leroux, Karine; Dugon, Emilie; Leguerney, Ingrid; Polrot, Mélanie; Robin, Sandra; Salomé-Desnoulez, Sophie; Ginefri, Jean-Christophe; Sebrié, Catherine; Laplace-Builhé, Corinne

2015-03-01

In vivo high-resolution imaging of tumor development is possible through dorsal skinfold chamber implantable on mice model. However, current intravital imaging systems are weakly tolerated along time by mice and do not allow multimodality imaging. Our project aims to develop a new chamber for: 1- long-term micro/macroscopic visualization of tumor (vascular and cellular compartments) and tissue microenvironment; and 2- multimodality imaging (photonic, MRI and sonography). Our new experimental device was patented in March 2014 and was primarily assessed on 75 mouse engrafted with 4T1-Luc tumor cell line, and validated in confocal and multiphoton imaging after staining the mice vasculature using Dextran 155KDa-TRITC or Dextran 2000kDa-FITC. Simultaneously, a universal stage was designed for optimal removal of respiratory and cardiac artifacts during microscopy assays. Experimental results from optical, ultrasound (Bmode and pulse subtraction mode) and MRI imaging (anatomic sequences) showed that our patented design, unlike commercial devices, improves longitudinal monitoring over several weeks (35 days on average against 12 for the commercial chamber) and allows for a better characterization of the early and late tissue alterations due to tumour development. We also demonstrated the compatibility for multimodality imaging and the increase of mice survival was by a factor of 2.9, with our new skinfold chamber. Current developments include: 1- defining new procedures for multi-labelling of cells and tissue (screening of fluorescent molecules and imaging protocols); 2- developing ultrasound and MRI imaging procedures with specific probes; 3- correlating optical/ultrasound/MRI data for a complete mapping of tumour development and microenvironment.
Tinnitus Multimodal Imaging

DTIC Science & Technology

2014-10-01

1 AWARD NUMBER: W81XWH-13-1-0494 TITLE: Tinnitus Multimodal Imaging PRINCIPAL INVESTIGATOR...TYPE Annual 3. DATES COVERED 30 Sept 2013 – 29 Oct 2014 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER Tinnitus Multimodal Imaging...AVAILABILITY STATEMENT Approved for Public Release; Distribution Unlimited 13. SUPPLEMENTARY NOTES 14. ABSTRACT Tinnitus is a common auditory
Laser Microdissection and Atmospheric Pressure Chemical Ionization Mass Spectrometry Coupled for Multimodal Imaging

DOE Office of Scientific and Technical Information (OSTI.GOV)

Lorenz, Matthias; Ovchinnikova, Olga S; Kertesz, Vilmos

2013-01-01

This paper describes the coupling of ambient laser ablation surface sampling, accomplished using a laser capture microdissection system, with atmospheric pressure chemical ionization mass spectrometry for high spatial resolution multimodal imaging. A commercial laser capture microdissection system was placed in close proximity to a modified ion source of a mass spectrometer designed to allow for sampling of laser ablated material via a transfer tube directly into the ionization region. Rhodamine 6G dye of red sharpie ink in a laser etched pattern as well as cholesterol and phosphatidylcholine in a cerebellum mouse brain thin tissue section were identified and imaged frommore » full scan mass spectra. A minimal spot diameter of 8 m was achieved using the 10X microscope cutting objective with a lateral oversampling pixel resolution of about 3.7 m. Distinguishing between features approximately 13 m apart in a cerebellum mouse brain thin tissue section was demonstrated in a multimodal fashion including co-registered optical and mass spectral chemical images.« less
New developments in multimodal clinical multiphoton tomography

NASA Astrophysics Data System (ADS)

König, Karsten

2011-03-01

80 years ago, the PhD student Maria Goeppert predicted in her thesis in Goettingen, Germany, two-photon effects. It took 30 years to prove her theory, and another three decades to realize the first two-photon microscope. With the beginning of this millennium, first clinical multiphoton tomographs started operation in research institutions, hospitals, and in the cosmetic industry. The multiphoton tomograph MPTflexTM with its miniaturized flexible scan head became the Prism-Award 2010 winner in the category Life Sciences. Multiphoton tomographs with its superior submicron spatial resolution can be upgraded to 5D imaging tools by adding spectral time-correlated single photon counting units. Furthermore, multimodal hybrid tomographs provide chemical fingerprinting and fast wide-field imaging. The world's first clinical CARS studies have been performed with a hybrid multimodal multiphoton tomograph in spring 2010. In particular, nonfluorescent lipids and water as well as mitochondrial fluorescent NAD(P)H, fluorescent elastin, keratin, and melanin as well as SHG-active collagen have been imaged in patients with dermatological disorders. Further multimodal approaches include the combination of multiphoton tomographs with low-resolution imaging tools such as ultrasound, optoacoustic, OCT, and dermoscopy systems. Multiphoton tomographs are currently employed in Australia, Japan, the US, and in several European countries for early diagnosis of skin cancer (malignant melanoma), optimization of treatment strategies (wound healing, dermatitis), and cosmetic research including long-term biosafety tests of ZnO sunscreen nanoparticles and the measurement of the stimulated biosynthesis of collagen by anti-ageing products.
Multimodality Molecular Imaging-Guided Tumor Border Delineation and Photothermal Therapy Analysis Based on Graphene Oxide-Conjugated Gold Nanoparticles Chelated with Gd.

PubMed

Ma, Xibo; Jin, Yushen; Wang, Yi; Zhang, Shuai; Peng, Dong; Yang, Xin; Wei, Shoushui; Chai, Wei; Li, Xuejun; Tian, Jie

2018-01-01

Tumor cell complete extinction is a crucial measure to evaluate antitumor efficacy. The difficulties in defining tumor margins and finding satellite metastases are the reason for tumor recurrence. A synergistic method based on multimodality molecular imaging needs to be developed so as to achieve the complete extinction of the tumor cells. In this study, graphene oxide conjugated with gold nanostars and chelated with Gd through 1,4,7,10-tetraazacyclododecane-N,N',N,N'-tetraacetic acid (DOTA) (GO-AuNS-DOTA-Gd) were prepared to target HCC-LM3-fLuc cells and used for therapy. For subcutaneous tumor, multimodality molecular imaging including photoacoustic imaging (PAI) and magnetic resonance imaging (MRI) and the related processing techniques were used to monitor the pharmacokinetics process of GO-AuNS-DOTA-Gd in order to determine the optimal time for treatment. For orthotopic tumor, MRI was used to delineate the tumor location and margin in vivo before treatment. Then handheld photoacoustic imaging system was used to determine the tumor location during the surgery and guided the photothermal therapy. The experiment result based on orthotopic tumor demonstrated that this synergistic method could effectively reduce tumor residual and satellite metastases by 85.71% compared with the routine photothermal method without handheld PAI guidance. These results indicate that this multimodality molecular imaging-guided photothermal therapy method is promising with a good prospect in clinical application.
Photoacoustic-Based Multimodal Nanoprobes: from Constructing to Biological Applications.

PubMed

Gao, Duyang; Yuan, Zhen

2017-01-01

Multimodal nanoprobes have attracted intensive attentions since they can integrate various imaging modalities to obtain complementary merits of single modality. Meanwhile, recent interest in laser-induced photoacoustic imaging is rapidly growing due to its unique advantages in visualizing tissue structure and function with high spatial resolution and satisfactory imaging depth. In this review, we summarize multimodal nanoprobes involving photoacoustic imaging. In particular, we focus on the method to construct multimodal nanoprobes. We have divided the synthetic methods into two types. First, we call it "one for all" concept, which involves intrinsic properties of the element in a single particle. Second, "all in one" concept, which means integrating different functional blocks in one particle. Then, we simply introduce the applications of the multifunctional nanoprobes for in vivo imaging and imaging-guided tumor therapy. At last, we discuss the advantages and disadvantages of the present methods to construct the multimodal nanoprobes and share our viewpoints in this area.
Multimodal Image-Based Virtual Reality Presurgical Simulation and Evaluation for Trigeminal Neuralgia and Hemifacial Spasm.

PubMed

Yao, Shujing; Zhang, Jiashu; Zhao, Yining; Hou, Yuanzheng; Xu, Xinghua; Zhang, Zhizhong; Kikinis, Ron; Chen, Xiaolei

2018-05-01

To address the feasibility and predictive value of multimodal image-based virtual reality in detecting and assessing features of neurovascular confliction (NVC), particularly regarding the detection of offending vessels, degree of compression exerted on the nerve root, in patients who underwent microvascular decompression for nonlesional trigeminal neuralgia and hemifacial spasm (HFS). This prospective study includes 42 consecutive patients who underwent microvascular decompression for classic primary trigeminal neuralgia or HFS. All patients underwent preoperative 1.5-T magnetic resonance imaging (MRI) with T2-weighted three-dimensional (3D) sampling perfection with application-optimized contrasts by using different flip angle evolutions, 3D time-of-flight magnetic resonance angiography, and 3D T1-weighted gadolinium-enhanced sequences in combination, whereas 2 patients underwent extra experimental preoperative 7.0-T MRI scans with the same imaging protocol. Multimodal MRIs were then coregistered with open-source software 3D Slicer, followed by 3D image reconstruction to generate virtual reality (VR) images for detection of possible NVC in the cerebellopontine angle. Evaluations were performed by 2 reviewers and compared with the intraoperative findings. For detection of NVC, multimodal image-based VR sensitivity was 97.6% (40/41) and specificity was 100% (1/1). Compared with the intraoperative findings, the κ coefficients for predicting the offending vessel and the degree of compression were >0.75 (P < 0.001). The 7.0-T scans have a clearer view of vessels in the cerebellopontine angle, which may have significant impact on detection of small-caliber offending vessels with relatively slow flow speed in cases of HFS. Multimodal image-based VR using 3D sampling perfection with application-optimized contrasts by using different flip angle evolutions in combination with 3D time-of-flight magnetic resonance angiography sequences proved to be reliable in detecting NVC and in predicting the degree of root compression. The VR image-based simulation correlated well with the real surgical view. Copyright © 2018 Elsevier Inc. All rights reserved.
Prospective Evaluation of Multimodal Optical Imaging with Automated Image Analysis to Detect Oral Neoplasia In Vivo.

PubMed

Quang, Timothy; Tran, Emily Q; Schwarz, Richard A; Williams, Michelle D; Vigneswaran, Nadarajah; Gillenwater, Ann M; Richards-Kortum, Rebecca

2017-10-01

The 5-year survival rate for patients with oral cancer remains low, in part because diagnosis often occurs at a late stage. Early and accurate identification of oral high-grade dysplasia and cancer can help improve patient outcomes. Multimodal optical imaging is an adjunctive diagnostic technique in which autofluorescence imaging is used to identify high-risk regions within the oral cavity, followed by high-resolution microendoscopy to confirm or rule out the presence of neoplasia. Multimodal optical images were obtained from 206 sites in 100 patients. Histologic diagnosis, either from a punch biopsy or an excised surgical specimen, was used as the gold standard for all sites. Histopathologic diagnoses of moderate dysplasia or worse were considered neoplastic. Images from 92 sites in the first 30 patients were used as a training set to develop automated image analysis methods for identification of neoplasia. Diagnostic performance was evaluated prospectively using images from 114 sites in the remaining 70 patients as a test set. In the training set, multimodal optical imaging with automated image analysis correctly classified 95% of nonneoplastic sites and 94% of neoplastic sites. Among the 56 sites in the test set that were biopsied, multimodal optical imaging correctly classified 100% of nonneoplastic sites and 85% of neoplastic sites. Among the 58 sites in the test set that corresponded to a surgical specimen, multimodal imaging correctly classified 100% of nonneoplastic sites and 61% of neoplastic sites. These findings support the potential of multimodal optical imaging to aid in the early detection of oral cancer. Cancer Prev Res; 10(10); 563-70. ©2017 AACR . ©2017 American Association for Cancer Research.
Multimodality Image Fusion-Guided Procedures: Technique, Accuracy, and Applications

DOE Office of Scientific and Technical Information (OSTI.GOV)

Abi-Jaoudeh, Nadine, E-mail: naj@mail.nih.gov; Kruecker, Jochen, E-mail: jochen.kruecker@philips.com; Kadoury, Samuel, E-mail: samuel.kadoury@polymtl.ca

2012-10-15

Personalized therapies play an increasingly critical role in cancer care: Image guidance with multimodality image fusion facilitates the targeting of specific tissue for tissue characterization and plays a role in drug discovery and optimization of tailored therapies. Positron-emission tomography (PET), magnetic resonance imaging (MRI), and contrast-enhanced computed tomography (CT) may offer additional information not otherwise available to the operator during minimally invasive image-guided procedures, such as biopsy and ablation. With use of multimodality image fusion for image-guided interventions, navigation with advanced modalities does not require the physical presence of the PET, MRI, or CT imaging system. Several commercially available methodsmore » of image-fusion and device navigation are reviewed along with an explanation of common tracking hardware and software. An overview of current clinical applications for multimodality navigation is provided.« less

Multimodal quantitative phase and fluorescence imaging of cell apoptosis

NASA Astrophysics Data System (ADS)

Fu, Xinye; Zuo, Chao; Yan, Hao

2017-06-01

Fluorescence microscopy, utilizing fluorescence labeling, has the capability to observe intercellular changes which transmitted and reflected light microscopy techniques cannot resolve. However, the parts without fluorescence labeling are not imaged. Hence, the processes simultaneously happen in these parts cannot be revealed. Meanwhile, fluorescence imaging is 2D imaging where information in the depth is missing. Therefore the information in labeling parts is also not complete. On the other hand, quantitative phase imaging is capable to image cells in 3D in real time through phase calculation. However, its resolution is limited by the optical diffraction and cannot observe intercellular changes below 200 nanometers. In this work, fluorescence imaging and quantitative phase imaging are combined to build a multimodal imaging system. Such system has the capability to simultaneously observe the detailed intercellular phenomenon and 3D cell morphology. In this study the proposed multimodal imaging system is used to observe the cell behavior in the cell apoptosis. The aim is to highlight the limitations of fluorescence microscopy and to point out the advantages of multimodal quantitative phase and fluorescence imaging. The proposed multimodal quantitative phase imaging could be further applied in cell related biomedical research, such as tumor.
Multimodal imaging of ischemic wounds

NASA Astrophysics Data System (ADS)

Zhang, Shiwu; Gnyawali, Surya; Huang, Jiwei; Liu, Peng; Gordillo, Gayle; Sen, Chandan K.; Xu, Ronald

2012-12-01

The wound healing process involves the reparative phases of inflammation, proliferation, and remodeling. Interrupting any of these phases may result in chronically unhealed wounds, amputation, or even patient death. Quantitative assessment of wound tissue ischemia, perfusion, and inflammation provides critical information for appropriate detection, staging, and treatment of chronic wounds. However, no method is available for noninvasive, simultaneous, and quantitative imaging of these tissue parameters. We integrated hyperspectral, laser speckle, and thermographic imaging modalities into a single setup for multimodal assessment of tissue oxygenation, perfusion, and inflammation characteristics. Advanced algorithms were developed for accurate reconstruction of wound oxygenation and appropriate co-registration between different imaging modalities. The multimodal wound imaging system was validated by an ongoing clinical trials approved by OSU IRB. In the clinical trial, a wound of 3mm in diameter was introduced on a healthy subject's lower extremity and the healing process was serially monitored by the multimodal imaging setup. Our experiments demonstrated the clinical usability of multimodal wound imaging.
The pivotal role of multimodality reporter sensors in drug discovery: from cell based assays to real time molecular imaging.

PubMed

Ray, Pritha

2011-04-01

Development and marketing of new drugs require stringent validation that are expensive and time consuming. Non-invasive multimodality molecular imaging using reporter genes holds great potential to expedite these processes at reduced cost. New generations of smarter molecular imaging strategies such as Split reporter, Bioluminescence resonance energy transfer, Multimodality fusion reporter technologies will further assist to streamline and shorten the drug discovery and developmental process. This review illustrates the importance and potential of molecular imaging using multimodality reporter genes in drug development at preclinical phases.
In vivo multimodal nonlinear optical imaging of mucosal tissue

NASA Astrophysics Data System (ADS)

Sun, Ju; Shilagard, Tuya; Bell, Brent; Motamedi, Massoud; Vargas, Gracie

2004-05-01

We present a multimodal nonlinear imaging approach to elucidate microstructures and spectroscopic features of oral mucosa and submucosa in vivo. The hamster buccal pouch was imaged using 3-D high resolution multiphoton and second harmonic generation microscopy. The multimodal imaging approach enables colocalization and differentiation of prominent known spectroscopic and structural features such as keratin, epithelial cells, and submucosal collagen at various depths in tissue. Visualization of cellular morphology and epithelial thickness are in excellent agreement with histological observations. These results suggest that multimodal nonlinear optical microscopy can be an effective tool for studying the physiology and pathology of mucosal tissue.
Multimodality Imaging of Myocardial Injury and Remodeling

PubMed Central

Kramer, Christopher M.; Sinusas, Albert J.; Sosnovik, David E.; French, Brent A.; Bengel, Frank M.

2011-01-01

Advances in cardiovascular molecular imaging have come at a rapid pace over the last several years. Multiple approaches have been taken to better understand the structural, molecular, and cellular events that underlie the progression from myocardial injury to myocardial infarction (MI) and, ultimately, to congestive heart failure. Multimodality molecular imaging including SPECT, PET, cardiac MRI, and optical approaches is offering new insights into the pathophysiology of MI and left ventricular remodeling in small-animal models. Targets that are being probed include, among others, angiotensin receptors, matrix metalloproteinases, integrins, apoptosis, macrophages, and sympathetic innervation. It is only a matter of time before these advances are applied in the clinical setting to improve post-MI prognostication and identify appropriate therapies in patients to prevent the onset of congestive heart failure. PMID:20395347
SU-E-J-110: A Novel Level Set Active Contour Algorithm for Multimodality Joint Segmentation/Registration Using the Jensen-Rényi Divergence.

PubMed

Markel, D; Naqa, I El; Freeman, C; Vallières, M

2012-06-01

To present a novel joint segmentation/registration for multimodality image-guided and adaptive radiotherapy. A major challenge to this framework is the sensitivity of many segmentation or registration algorithms to noise. Presented is a level set active contour based on the Jensen-Renyi (JR) divergence to achieve improved noise robustness in a multi-modality imaging space. To present a novel joint segmentation/registration for multimodality image-guided and adaptive radiotherapy. A major challenge to this framework is the sensitivity of many segmentation or registration algorithms to noise. Presented is a level set active contour based on the Jensen-Renyi (JR) divergence to achieve improved noise robustness in a multi-modality imaging space. It was found that JR divergence when used for segmentation has an improved robustness to noise compared to using mutual information, or other entropy-based metrics. The MI metric failed at around 2/3 the noise power than the JR divergence. The JR divergence metric is useful for the task of joint segmentation/registration of multimodality images and shows improved results compared entropy based metric. The algorithm can be easily modified to incorporate non-intensity based images, which would allow applications into multi-modality and texture analysis. © 2012 American Association of Physicists in Medicine.
NOVEL PRERETINAL HAIR PIN-LIKE VESSEL IN RETINAL ASTROCYTIC HAMARTOMA WITH VITREOUS HEMORRHAGE.

PubMed

Soeta, Megumi; Arai, Yusuke; Takahashi, Hidenori; Fujino, Yujiro; Tanabe, Tatsuro; Inoue, Yuji; Kawashima, Hidetoshi

2018-01-01

To report a case of retinal astrocytic hamartoma with vitreous hemorrhage and a hair pin-like vessel adhering to a posterior vitreous membrane. A 33-year-old man with a retinal astrocytic hamartoma presented with vitreous hemorrhage 5 times. Multimodal imaging, including fundus photography, fluorescein angiography, optical coherence tomography, and B-mode ultrasonography. Multimodal imaging demonstrated a novel hair pin-like vessel that adhered to the posterior vitreous membrane. Some cases of retinal astrocytic hamartoma with vitreous hemorrhage may be related to structure abnormalities of tumor vessels.
MMX-I: A data-processing software for multi-modal X-ray imaging and tomography

NASA Astrophysics Data System (ADS)

Bergamaschi, A.; Medjoubi, K.; Messaoudi, C.; Marco, S.; Somogyi, A.

2017-06-01

Scanning hard X-ray imaging allows simultaneous acquisition of multimodal information, including X-ray fluorescence, absorption, phase and dark-field contrasts, providing structural and chemical details of the samples. Combining these scanning techniques with the infrastructure developed for fast data acquisition at Synchrotron Soleil permits to perform multimodal imaging and tomography during routine user experiments at the Nanoscopium beamline. A main challenge of such imaging techniques is the online processing and analysis of the generated very large volume (several hundreds of Giga Bytes) multimodal data-sets. This is especially important for the wide user community foreseen at the user oriented Nanoscopium beamline (e.g. from the fields of Biology, Life Sciences, Geology, Geobiology), having no experience in such data-handling. MMX-I is a new multi-platform open-source freeware for the processing and reconstruction of scanning multi-technique X-ray imaging and tomographic datasets. The MMX-I project aims to offer, both expert users and beginners, the possibility of processing and analysing raw data, either on-site or off-site. Therefore we have developed a multi-platform (Mac, Windows and Linux 64bit) data processing tool, which is easy to install, comprehensive, intuitive, extendable and user-friendly. MMX-I is now routinely used by the Nanoscopium user community and has demonstrated its performance in treating big data.
Label-aligned Multi-task Feature Learning for Multimodal Classification of Alzheimer’s Disease and Mild Cognitive Impairment

PubMed Central

Zu, Chen; Jie, Biao; Liu, Mingxia; Chen, Songcan

2015-01-01

Multimodal classification methods using different modalities of imaging and non-imaging data have recently shown great advantages over traditional single-modality-based ones for diagnosis and prognosis of Alzheimer’s disease (AD), as well as its prodromal stage, i.e., mild cognitive impairment (MCI). However, to the best of our knowledge, most existing methods focus on mining the relationship across multiple modalities of the same subjects, while ignoring the potentially useful relationship across different subjects. Accordingly, in this paper, we propose a novel learning method for multimodal classification of AD/MCI, by fully exploring the relationships across both modalities and subjects. Specifically, our proposed method includes two subsequent components, i.e., label-aligned multi-task feature selection and multimodal classification. In the first step, the feature selection learning from multiple modalities are treated as different learning tasks and a group sparsity regularizer is imposed to jointly select a subset of relevant features. Furthermore, to utilize the discriminative information among labeled subjects, a new label-aligned regularization term is added into the objective function of standard multi-task feature selection, where label-alignment means that all multi-modality subjects with the same class labels should be closer in the new feature-reduced space. In the second step, a multi-kernel support vector machine (SVM) is adopted to fuse the selected features from multi-modality data for final classification. To validate our method, we perform experiments on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database using baseline MRI and FDG-PET imaging data. The experimental results demonstrate that our proposed method achieves better classification performance compared with several state-of-the-art methods for multimodal classification of AD/MCI. PMID:26572145
Appearance of osteolysis with melorheostosis: redefining the disease or a new disorder? A novel case report with multimodality imaging.

PubMed

Osher, Lawrence S; Blazer, Marie Mantini; Bumpus, Kelly

2013-01-01

We present a case report of melorheostosis with the novel radiographic finding of underlying cortical resorption. A number of radiographic patterns of melorheostosis have been described; however, the combination of new bone formation and resorption of the original cortex appears unique. Although the presence of underlying lysis has been postulated in published studies, direct radiographic evidence of bony resorption in melorheostosis has not been reported. These findings can be subtle and might go unnoticed using standard imaging. An in-depth review of the radiographic features is presented, including multimodality imaging with magnetic resonance imaging and computed tomography. Copyright © 2013 American College of Foot and Ankle Surgeons. Published by Elsevier Inc. All rights reserved.
Use of multidimensional, multimodal imaging and PACS to support neurological diagnoses

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wong, S.T.C.; Knowlton, R.; Hoo, K.S.

1995-12-31

Technological advances in brain imaging have revolutionized diagnosis in neurology and neurological surgery. Major imaging techniques include magnetic resonance imaging (MRI) to visualize structural anatomy, positron emission tomography (PET) to image metabolic function and cerebral blood flow, magnetoencephalography (MEG) to visualize the location of physiologic current sources, and magnetic resonance spectroscopy (MRS) to measure specific biochemicals. Each of these techniques studies different biomedical aspects of the grain, but there lacks an effective means to quantify and correlate the disparate imaging datasets in order to improve clinical decision making processes. This paper describes several techniques developed in a UNIX-based neurodiagnostic workstationmore » to aid the non-invasive presurgical evaluation of epilepsy patients. These techniques include on-line access to the picture archiving and communication systems (PACS) multimedia archive, coregistration of multimodality image datasets, and correlation and quantitative of structural and functional information contained in the registered images. For illustration, the authors describe the use of these techniques in a patient case of non-lesional neocortical epilepsy. They also present the future work based on preliminary studies.« less
Ridge-branch-based blood vessel detection algorithm for multimodal retinal images

NASA Astrophysics Data System (ADS)

Li, Y.; Hutchings, N.; Knighton, R. W.; Gregori, G.; Lujan, B. J.; Flanagan, J. G.

2009-02-01

Automatic detection of retinal blood vessels is important to medical diagnoses and imaging. With the development of imaging technologies, various modals of retinal images are available. Few of currently published algorithms are applied to multimodal retinal images. Besides, the performance of algorithms with pathologies is expected to be improved. The purpose of this paper is to propose an automatic Ridge-Branch-Based (RBB) detection algorithm of blood vessel centerlines and blood vessels for multimodal retinal images (color fundus photographs, fluorescein angiograms, fundus autofluorescence images, SLO fundus images and OCT fundus images, for example). Ridges, which can be considered as centerlines of vessel-like patterns, are first extracted. The method uses the connective branching information of image ridges: if ridge pixels are connected, they are more likely to be in the same class, vessel ridge pixels or non-vessel ridge pixels. Thanks to the good distinguishing ability of the designed "Segment-Based Ridge Features", the classifier and its parameters can be easily adapted to multimodal retinal images without ground truth training. We present thorough experimental results on SLO images, color fundus photograph database and other multimodal retinal images, as well as comparison between other published algorithms. Results showed that the RBB algorithm achieved a good performance.
Developing single-laser sources for multimodal coherent anti-Stokes Raman scattering microscopy

NASA Astrophysics Data System (ADS)

Pegoraro, Adrian Frank

Coherent anti-Stokes Raman scattering (CARS) microscopy has developed rapidly and is opening the door to new types of experiments. This work describes the development of new laser sources for CARS microscopy and their use for different applications. It is specifically focused on multimodal nonlinear optical microscopy—the simultaneous combination of different imaging techniques. This allows us to address a diverse range of applications, such as the study of biomaterials, fluid inclusions, atherosclerosis, hepatitis C infection in cells, and ice formation in cells. For these applications new laser sources are developed that allow for practical multimodal imaging. For example, it is shown that using a single Ti:sapphire oscillator with a photonic crystal fiber, it is possible to develop a versatile multimodal imaging system using optimally chirped laser pulses. This system can perform simultaneous two photon excited fluorescence, second harmonic generation, and CARS microscopy. The versatility of the system is further demonstrated by showing that it is possible to probe different Raman modes using CARS microscopy simply by changing a time delay between the excitation beams. Using optimally chirped pulses also enables further simplification of the laser system required by using a single fiber laser combined with nonlinear optical fibers to perform effective multimodal imaging. While these sources are useful for practical multimodal imaging, it is believed that for further improvements in CARS microscopy sensitivity, new excitation schemes are necessary. This has led to the design of a new, high power, extended cavity oscillator that should be capable of implementing new excitation schemes for CARS microscopy as well as other techniques. Our interest in multimodal imaging has led us to other areas of research as well. For example, a fiber-coupling scheme for signal collection in the forward direction is demonstrated that allows for fluorescence lifetime imaging without significant temporal distortion. Also highlighted is an imaging artifact that is unique to CARS microscopy that can alter image interpretation, especially when using multimodal imaging. By combining expertise in nonlinear optics, laser development, fiber optics, and microscopy, we have developed systems and techniques that will be of benefit for multimodal CARS microscopy.
Mobile, Multi-modal, Label-Free Imaging Probe Analysis of Choroidal Oximetry and Retinal Hypoxia

DTIC Science & Technology

2015-10-01

eyes and image choroidal vessels/capillaries using CARS intravital microscopy Subtask 3: Measure oxy-hemoglobin levels in PBI test and control eyes...AWARD NUMBER: W81XWH-14-1-0537 TITLE: Mobile, Multi-modal, Label-Free Imaging Probe Analysis of Choroidal Oximetry and Retinal Hypoxia...4. TITLE AND SUBTITLE Mobile, Multimodal, Label-Free Imaging Probe Analysis of Choroidal Oximetry and Retinal Hypoxia 5a. CONTRACT NUMBER W81XWH
Robust Multimodal Dictionary Learning

PubMed Central

Cao, Tian; Jojic, Vladimir; Modla, Shannon; Powell, Debbie; Czymmek, Kirk; Niethammer, Marc

2014-01-01

We propose a robust multimodal dictionary learning method for multimodal images. Joint dictionary learning for both modalities may be impaired by lack of correspondence between image modalities in training data, for example due to areas of low quality in one of the modalities. Dictionaries learned with such non-corresponding data will induce uncertainty about image representation. In this paper, we propose a probabilistic model that accounts for image areas that are poorly corresponding between the image modalities. We cast the problem of learning a dictionary in presence of problematic image patches as a likelihood maximization problem and solve it with a variant of the EM algorithm. Our algorithm iterates identification of poorly corresponding patches and re-finements of the dictionary. We tested our method on synthetic and real data. We show improvements in image prediction quality and alignment accuracy when using the method for multimodal image registration. PMID:24505674
Radiation Safety in Children With Congenital and Acquired Heart Disease: A Scientific Position Statement on Multimodality Dose Optimization From the Image Gently Alliance.

PubMed

Hill, Kevin D; Frush, Donald P; Han, B Kelly; Abbott, Brian G; Armstrong, Aimee K; DeKemp, Robert A; Glatz, Andrew C; Greenberg, S Bruce; Herbert, Alexander Sheldon; Justino, Henri; Mah, Douglas; Mahesh, Mahadevappa; Rigsby, Cynthia K; Slesnick, Timothy C; Strauss, Keith J; Trattner, Sigal; Viswanathan, Mohan N; Einstein, Andrew J

2017-07-01

There is a need for consensus recommendations for ionizing radiation dose optimization during multimodality medical imaging in children with congenital and acquired heart disease (CAHD). These children often have complex diseases and may be exposed to a relatively high cumulative burden of ionizing radiation from medical imaging procedures, including cardiac computed tomography, nuclear cardiology studies, and fluoroscopically guided diagnostic and interventional catheterization and electrophysiology procedures. Although these imaging procedures are all essential to the care of children with CAHD and have contributed to meaningfully improved outcomes in these patients, exposure to ionizing radiation is associated with potential risks, including an increased lifetime attributable risk of cancer. The goal of these recommendations is to encourage informed imaging to achieve appropriate study quality at the lowest achievable dose. Other strategies to improve care include a patient-centered approach to imaging, emphasizing education and informed decision making and programmatic approaches to ensure appropriate dose monitoring. Looking ahead, there is a need for standardization of dose metrics across imaging modalities, so as to encourage comparative effectiveness studies across the spectrum of CAHD in children. Copyright © 2017 The Authors. Published by Elsevier Inc. All rights reserved.
Microscopy with multimode fibers

NASA Astrophysics Data System (ADS)

Moser, Christophe; Papadopoulos, Ioannis; Farahi, Salma; Psaltis, Demetri

2013-04-01

Microscopes are usually thought of comprising imaging elements such as objectives and eye-piece lenses. A different type of microscope, used for endoscopy, consists of waveguiding elements such as fiber bundles, where each fiber in the bundle transports the light corresponding to one pixel in the image. Recently a new type of microscope has emerged that exploits the large number of propagating modes in a single multimode fiber. We have successfully produced fluorescence images of neural cells with sub-micrometer resolution via a 200 micrometer core multimode fiber. The method for achieving imaging consists of using digital phase conjugation to reproduce a focal spot at the tip of the multimode fiber. The image is formed by scanning the focal spot digitally and collecting the fluorescence point by point.
Landmark Image Retrieval by Jointing Feature Refinement and Multimodal Classifier Learning.

PubMed

Zhang, Xiaoming; Wang, Senzhang; Li, Zhoujun; Ma, Shuai; Xiaoming Zhang; Senzhang Wang; Zhoujun Li; Shuai Ma; Ma, Shuai; Zhang, Xiaoming; Wang, Senzhang; Li, Zhoujun

2018-06-01

Landmark retrieval is to return a set of images with their landmarks similar to those of the query images. Existing studies on landmark retrieval focus on exploiting the geometries of landmarks for visual similarity matches. However, the visual content of social images is of large diversity in many landmarks, and also some images share common patterns over different landmarks. On the other side, it has been observed that social images usually contain multimodal contents, i.e., visual content and text tags, and each landmark has the unique characteristic of both visual content and text content. Therefore, the approaches based on similarity matching may not be effective in this environment. In this paper, we investigate whether the geographical correlation among the visual content and the text content could be exploited for landmark retrieval. In particular, we propose an effective multimodal landmark classification paradigm to leverage the multimodal contents of social image for landmark retrieval, which integrates feature refinement and landmark classifier with multimodal contents by a joint model. The geo-tagged images are automatically labeled for classifier learning. Visual features are refined based on low rank matrix recovery, and multimodal classification combined with group sparse is learned from the automatically labeled images. Finally, candidate images are ranked by combining classification result and semantic consistence measuring between the visual content and text content. Experiments on real-world datasets demonstrate the superiority of the proposed approach as compared to existing methods.
Colocalization of cellular nanostructure using confocal fluorescence and partial wave spectroscopy.

PubMed

Chandler, John E; Stypula-Cyrus, Yolanda; Almassalha, Luay; Bauer, Greta; Bowen, Leah; Subramanian, Hariharan; Szleifer, Igal; Backman, Vadim

2017-03-01

A new multimodal confocal microscope has been developed, which includes a parallel Partial Wave Spectroscopic (PWS) microscopy path. This combination of modalities allows molecular-specific sensing of nanoscale intracellular structure using fluorescent labels. Combining molecular specificity and sensitivity to nanoscale structure allows localization of nanostructural intracellular changes, which is critical for understanding the mechanisms of diseases such as cancer. To demonstrate the capabilities of this multimodal instrument, we imaged HeLa cells treated with valinomycin, a potassium ionophore that uncouples oxidative phosphorylation. Colocalization of fluorescence images of the nuclei (Hoechst 33342) and mitochondria (anti-mitochondria conjugated to Alexa Fluor 488) with PWS measurements allowed us to detect a significant decrease in nuclear nanoscale heterogeneity (Σ), while no significant change in Σ was observed at mitochondrial sites. In addition, application of the new multimodal imaging approach was demonstrated on human buccal samples prepared using a cancer screening protocol. These images demonstrate that nanoscale intracellular structure can be studied in healthy and diseased cells at molecular-specific sites. © 2017 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim.
Multimodal Imaging in Diabetic Macular Edema.

PubMed

Acón, Dhariana; Wu, Lihteh

2018-01-01

Throughout ophthalmic history it has been shown that progress has gone hand in hand with technological breakthroughs. In the past, fluorescein angiography and fundus photographs were the most commonly used imaging modalities in the management of diabetic macular edema (DME). Today, despite the moderate correlation between macular thickness and functional outcomes, spectral domain optical coherence tomography (SD-OCT) has become the DME workhorse in clinical practice. Several SD-OCT biomarkers have been looked at including presence of epiretinal membrane, vitreomacular adhesion, disorganization of the inner retinal layers, central macular thickness, integrity of the ellipsoid layer, and subretinal fluid, among others. Emerging imaging modalities include fundus autofluorescence, macular pigment optical density, fluorescence lifetime imaging ophthalmoscopy, OCT angiography, and adaptive optics. Technological advances in imaging of the posterior segment of the eye have enabled ophthalmologists to develop hypotheses about pathological mechanisms of disease, monitor disease progression, and assess response to treatment. Spectral domain OCT is the most commonly performed imaging modality in the management of DME. However, reliable biomarkers have yet to be identified. Machine learning may provide treatment algorithms based on multimodal imaging. Copyright 2018 Asia-Pacific Academy of Ophthalmology.

Tinnitus Multimodal Imaging

DTIC Science & Technology

2015-10-01

AWARD NUMBER: W81XWH-13-1-0494 TITLE: Tinnitus Multimodal Imaging PRINCIPAL INVESTIGATOR: Steven Wan Cheung CONTRACTING ORGANIZATION...NUMBER W81XWH-13-1-0494 Tinnitus Multimodal Imaging 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER Steven W. Cheung...13. SUPPLEMENTARY NOTES 14. ABSTRACT Tinnitus is a common auditory perceptual disorder whose neural substrates are under intense debate. This project
Multimodal Imaging of the Normal Eye.

PubMed

Kawali, Ankush; Pichi, Francesco; Avadhani, Kavitha; Invernizzi, Alessandro; Hashimoto, Yuki; Mahendradas, Padmamalini

2017-10-01

Multimodal imaging is the concept of "bundling" images obtained from various imaging modalities, viz., fundus photograph, fundus autofluorescence imaging, infrared (IR) imaging, simultaneous fluorescein and indocyanine angiography, optical coherence tomography (OCT), and, more recently, OCT angiography. Each modality has its pros and cons as well as its limitations. Combination of multiple imaging techniques will overcome their individual weaknesses and give a comprehensive picture. Such approach helps in accurate localization of a lesion and understanding the pathology in posterior segment. It is important to know imaging of normal eye before one starts evaluating pathology. This article describes multimodal imaging modalities in detail and discusses healthy eye features as seen on various imaging modalities mentioned above.
Fast segmentation and high-quality three-dimensional volume mesh creation from medical images for diffuse optical tomography

NASA Astrophysics Data System (ADS)

Jermyn, Michael; Ghadyani, Hamid; Mastanduno, Michael A.; Turner, Wes; Davis, Scott C.; Dehghani, Hamid; Pogue, Brian W.

2013-08-01

Multimodal approaches that combine near-infrared (NIR) and conventional imaging modalities have been shown to improve optical parameter estimation dramatically and thus represent a prevailing trend in NIR imaging. These approaches typically involve applying anatomical templates from magnetic resonance imaging/computed tomography/ultrasound images to guide the recovery of optical parameters. However, merging these data sets using current technology requires multiple software packages, substantial expertise, significant time-commitment, and often results in unacceptably poor mesh quality for optical image reconstruction, a reality that represents a significant roadblock for translational research of multimodal NIR imaging. This work addresses these challenges directly by introducing automated digital imaging and communications in medicine image stack segmentation and a new one-click three-dimensional mesh generator optimized for multimodal NIR imaging, and combining these capabilities into a single software package (available for free download) with a streamlined workflow. Image processing time and mesh quality benchmarks were examined for four common multimodal NIR use-cases (breast, brain, pancreas, and small animal) and were compared to a commercial image processing package. Applying these tools resulted in a fivefold decrease in image processing time and 62% improvement in minimum mesh quality, in the absence of extra mesh postprocessing. These capabilities represent a significant step toward enabling translational multimodal NIR research for both expert and nonexpert users in an open-source platform.
A Multimode Optical Imaging System for Preclinical Applications In Vivo: Technology Development, Multiscale Imaging, and Chemotherapy Assessment

PubMed Central

Hwang, Jae Youn; Wachsmann-Hogiu, Sebastian; Ramanujan, V. Krishnan; Ljubimova, Julia; Gross, Zeev; Gray, Harry B.; Medina-Kauwe, Lali K.; Farkas, Daniel L.

2012-01-01

Purpose Several established optical imaging approaches have been applied, usually in isolation, to preclinical studies; however, truly useful in vivo imaging may require a simultaneous combination of imaging modalities to examine dynamic characteristics of cells and tissues. We developed a new multimode optical imaging system designed to be application-versatile, yielding high sensitivity, and specificity molecular imaging. Procedures We integrated several optical imaging technologies, including fluorescence intensity, spectral, lifetime, intravital confocal, two-photon excitation, and bioluminescence, into a single system that enables functional multiscale imaging in animal models. Results The approach offers a comprehensive imaging platform for kinetic, quantitative, and environmental analysis of highly relevant information, with micro-to-macroscopic resolution. Applied to small animals in vivo, this provides superior monitoring of processes of interest, represented here by chemo-/nanoconstruct therapy assessment. Conclusions This new system is versatile and can be optimized for various applications, of which cancer detection and targeted treatment are emphasized here. PMID:21874388
Multimodal Image Registration through Simultaneous Segmentation.

PubMed

Aganj, Iman; Fischl, Bruce

2017-11-01

Multimodal image registration facilitates the combination of complementary information from images acquired with different modalities. Most existing methods require computation of the joint histogram of the images, while some perform joint segmentation and registration in alternate iterations. In this work, we introduce a new non-information-theoretical method for pairwise multimodal image registration, in which the error of segmentation - using both images - is considered as the registration cost function. We empirically evaluate our method via rigid registration of multi-contrast brain magnetic resonance images, and demonstrate an often higher registration accuracy in the results produced by the proposed technique, compared to those by several existing methods.
Multimodal hard x-ray imaging with resolution approaching 10 nm for studies in material science

NASA Astrophysics Data System (ADS)

Yan, Hanfei; Bouet, Nathalie; Zhou, Juan; Huang, Xiaojing; Nazaretski, Evgeny; Xu, Weihe; Cocco, Alex P.; Chiu, Wilson K. S.; Brinkman, Kyle S.; Chu, Yong S.

2018-03-01

We report multimodal scanning hard x-ray imaging with spatial resolution approaching 10 nm and its application to contemporary studies in the field of material science. The high spatial resolution is achieved by focusing hard x-rays with two crossed multilayer Laue lenses and raster-scanning a sample with respect to the nanofocusing optics. Various techniques are used to characterize and verify the achieved focus size and imaging resolution. The multimodal imaging is realized by utilizing simultaneously absorption-, phase-, and fluorescence-contrast mechanisms. The combination of high spatial resolution and multimodal imaging enables a comprehensive study of a sample on a very fine length scale. In this work, the unique multimodal imaging capability was used to investigate a mixed ionic-electronic conducting ceramic-based membrane material employed in solid oxide fuel cells and membrane separations (compound of Ce0.8Gd0.2O2‑x and CoFe2O4) which revealed the existence of an emergent material phase and quantified the chemical complexity at the nanoscale.
Towards a Compact Fiber Laser for Multimodal Imaging

NASA Astrophysics Data System (ADS)

Nie, Bai; Saytashev, Ilyas; Dantus, Marcos

We report on multimodal depth-resolved imaging of unstained living Drosophila Melanogaster larva using sub-50 fs pulses centered at 1060 nm wavelength. Both second harmonic and third harmonic generation imaging modalities are demonstrated.
Towards a compact fiber laser for multimodal imaging

NASA Astrophysics Data System (ADS)

Nie, Bai; Saytashev, Ilyas; Dantus, Marcos

2014-03-01

We report on multimodal depth-resolved imaging of unstained living Drosophila Melanogaster larva using sub-50 fs pulses centered at 1060 nm wavelength. Both second harmonic and third harmonic generation imaging modalities are demonstrated.
Prussian blue nanocubes: multi-functional nanoparticles for multimodal imaging and image-guided therapy (Conference Presentation)

NASA Astrophysics Data System (ADS)

Cook, Jason R.; Dumani, Diego S.; Kubelick, Kelsey P.; Luci, Jeffrey; Emelianov, Stanislav Y.

2017-03-01

Imaging modalities utilize contrast agents to improve morphological visualization and to assess functional and molecular/cellular information. Here we present a new type of nanometer scale multi-functional particle that can be used for multi-modal imaging and therapeutic applications. Specifically, we synthesized monodisperse 20 nm Prussian Blue Nanocubes (PBNCs) with desired optical absorption in the near-infrared region and superparamagnetic properties. PBNCs showed excellent contrast in photoacoustic (700 nm wavelength) and MR (3T) imaging. Furthermore, photostability was assessed by exposing the PBNCs to nearly 1,000 laser pulses (5 ns pulse width) with up to 30 mJ/cm2 laser fluences. The PBNCs exhibited insignificant changes in photoacoustic signal, demonstrating enhanced robustness compared to the commonly used gold nanorods (substantial photodegradation with fluences greater than 5 mJ/cm2). Furthermore, the PBNCs exhibited superparamagnetism with a magnetic saturation of 105 emu/g, a 5x improvement over superparamagnetic iron-oxide (SPIO) nanoparticles. PBNCs exhibited enhanced T2 contrast measured using 3T clinical MRI. Because of the excellent optical absorption and magnetism, PBNCs have potential uses in other imaging modalities including optical tomography, microscopy, magneto-motive OCT/ultrasound, etc. In addition to multi-modal imaging, the PBNCs are multi-functional and, for example, can be used to enhance magnetic delivery and as therapeutic agents. Our initial studies show that stem cells can be labeled with PBNCs to perform image-guided magnetic delivery. Overall, PBNCs can act as imaging/therapeutic agents in diverse applications including cancer, cardiovascular disease, ophthalmology, and tissue engineering. Furthermore, PBNCs are based on FDA approved Prussian Blue thus potentially easing clinical translation of PBNCs.
Analyzing multimodality tomographic images and associated regions of interest with MIDAS

NASA Astrophysics Data System (ADS)

Tsui, Wai-Hon; Rusinek, Henry; Van Gelder, Peter; Lebedev, Sergey

2001-07-01

This paper outlines the design and features incorporated in a software package for analyzing multi-modality tomographic images. The package MIDAS has been evolving for the past 15 years and is in wide use by researchers at New York University School of Medicine and a number of collaborating research sites. It was written in the C language and runs on Sun workstations and Intel PCs under the Solaris operating system. A unique strength of the MIDAS package lies in its ability to generate, manipulate and analyze a practically unlimited number of regions of interest (ROIs). These regions are automatically saved in an efficient data structure and linked to associated images. A wide selection of set theoretical (e.g. union, xor, difference), geometrical (e.g. move, rotate) and morphological (grow, peel) operators can be applied to an arbitrary selection of ROIs. ROIs are constructed as a result of image segmentation algorithms incorporated in MIDAS; they also can be drawn interactively. These ROI editing operations can be applied in either 2D or 3D mode. ROI statistics generated by MIDAS include means, standard deviations, centroids and histograms. Other image manipulation tools incorporated in MIDAS are multimodality and within modality coregistration methods (including landmark matching, surface fitting and Woods' correlation methods) and image reformatting methods (using nearest-neighbor, tri-linear or sinc interpolation). Applications of MIDAS include: (1) neuroanatomy research: marking anatomical structures in one orientation, reformatting marks to another orientation; (2) tissue volume measurements: brain structures (PET, MRI, CT), lung nodules (low dose CT), breast density (MRI); (3) analysis of functional (SPECT, PET) experiments by overlaying corresponding structural scans; (4) longitudinal studies: regional measurement of atrophy.
Comparison of the application of B-mode and strain elastography ultrasound in the estimation of lymph node metastasis of papillary thyroid carcinoma based on a radiomics approach.

PubMed

Liu, Tongtong; Ge, Xifeng; Yu, Jinhua; Guo, Yi; Wang, Yuanyuan; Wang, Wenping; Cui, Ligang

2018-06-21

B-mode ultrasound (B-US) and strain elastography ultrasound (SE-US) images have a potential to distinguish thyroid tumor with different lymph node (LN) status. The purpose of our study is to investigate whether the application of multi-modality images including B-US and SE-US can improve the discriminability of thyroid tumor with LN metastasis based on a radiomics approach. Ultrasound (US) images including B-US and SE-US images of 75 papillary thyroid carcinoma (PTC) cases were retrospectively collected. A radiomics approach was developed in this study to estimate LNs status of PTC patients. The approach included image segmentation, quantitative feature extraction, feature selection and classification. Three feature sets were extracted from B-US, SE-US, and multi-modality containing B-US and SE-US. They were used to evaluate the contribution of different modalities. A total of 684 radiomics features have been extracted in our study. We used sparse representation coefficient-based feature selection method with 10-bootstrap to reduce the dimension of feature sets. Support vector machine with leave-one-out cross-validation was used to build the model for estimating LN status. Using features extracted from both B-US and SE-US, the radiomics-based model produced an area under the receiver operating characteristic curve (AUC) [Formula: see text] 0.90, accuracy (ACC) [Formula: see text] 0.85, sensitivity (SENS) [Formula: see text] 0.77 and specificity (SPEC) [Formula: see text] 0.88, which was better than using features extracted from B-US or SE-US separately. Multi-modality images provided more information in radiomics study. Combining use of B-US and SE-US could improve the LN metastasis estimation accuracy for PTC patients.
A multimodal image sensor system for identifying water stress in grapevines

NASA Astrophysics Data System (ADS)

Zhao, Yong; Zhang, Qin; Li, Minzan; Shao, Yongni; Zhou, Jianfeng; Sun, Hong

2012-11-01

Water stress is one of the most common limitations of fruit growth. Water is the most limiting resource for crop growth. In grapevines, as well as in other fruit crops, fruit quality benefits from a certain level of water deficit which facilitates to balance vegetative and reproductive growth and the flow of carbohydrates to reproductive structures. A multi-modal sensor system was designed to measure the reflectance signature of grape plant surfaces and identify different water stress levels in this paper. The multi-modal sensor system was equipped with one 3CCD camera (three channels in R, G, and IR). The multi-modal sensor can capture and analyze grape canopy from its reflectance features, and identify the different water stress levels. This research aims at solving the aforementioned problems. The core technology of this multi-modal sensor system could further be used as a decision support system that combines multi-modal sensory data to improve plant stress detection and identify the causes of stress. The images were taken by multi-modal sensor which could output images in spectral bands of near-infrared, green and red channel. Based on the analysis of the acquired images, color features based on color space and reflectance features based on image process method were calculated. The results showed that these parameters had the potential as water stress indicators. More experiments and analysis are needed to validate the conclusion.
WE-H-206-02: Recent Advances in Multi-Modality Molecular Imaging of Small Animals

DOE Office of Scientific and Technical Information (OSTI.GOV)

Tsui, B.

Lihong V. Wang: Photoacoustic tomography (PAT), combining non-ionizing optical and ultrasonic waves via the photoacoustic effect, provides in vivo multiscale functional, metabolic, and molecular imaging. Broad applications include imaging of the breast, brain, skin, esophagus, colon, vascular system, and lymphatic system in humans or animals. Light offers rich contrast but does not penetrate biological tissue in straight paths as x-rays do. Consequently, high-resolution pure optical imaging (e.g., confocal microscopy, two-photon microscopy, and optical coherence tomography) is limited to penetration within the optical diffusion limit (∼1 mm in the skin). Ultrasonic imaging, on the contrary, provides fine spatial resolution but suffersmore » from both poor contrast in early-stage tumors and strong speckle artifacts. In PAT, pulsed laser light penetrates tissue and generates a small but rapid temperature rise, which induces emission of ultrasonic waves due to thermoelastic expansion. The ultrasonic waves, orders of magnitude less scattering than optical waves, are then detected to form high-resolution images of optical absorption at depths up to 7 cm, conquering the optical diffusion limit. PAT is the only modality capable of imaging across the length scales of organelles, cells, tissues, and organs (up to whole-body small animals) with consistent contrast. This rapidly growing technology promises to enable multiscale biological research and accelerate translation from microscopic laboratory discoveries to macroscopic clinical practice. PAT may also hold the key to label-free early detection of cancer by in vivo quantification of hypermetabolism, the quintessential hallmark of malignancy. Learning Objectives: To understand the contrast mechanism of PAT To understand the multiscale applications of PAT Benjamin M. W. Tsui: Multi-modality molecular imaging instrumentation and techniques have been major developments in small animal imaging that has contributed significantly to biomedical research during the past decade. The initial development was an extension of clinical PET/CT and SPECT/CT from human to small animals and combine the unique functional information obtained from PET and SPECT with anatomical information provided by the CT in registered multi-modality images. The requirements to image a mouse whose size is an order of magnitude smaller than that of a human have spurred advances in new radiation detector technologies, novel imaging system designs and special image reconstruction and processing techniques. Examples are new detector materials and designs with high intrinsic resolution, multi-pinhole (MPH) collimator design for much improved resolution and detection efficiency compared to the conventional collimator designs in SPECT, 3D high-resolution and artifact-free MPH and sparse-view image reconstruction techniques, and iterative image reconstruction methods with system response modeling for resolution recovery and image noise reduction for much improved image quality. The spatial resolution of PET and SPECT has improved from ∼6–12 mm to ∼1 mm a few years ago to sub-millimeter today. A recent commercial small animal SPECT system has achieved a resolution of ∼0.25 mm which surpasses that of a state-of-art PET system whose resolution is limited by the positron range. More recently, multimodality SA PET/MRI and SPECT/MRI systems have been developed in research laboratories. Also, multi-modality SA imaging systems that include other imaging modalities such as optical and ultrasound are being actively pursued. In this presentation, we will provide a review of the development, recent advances and future outlook of multi-modality molecular imaging of small animals. Learning Objectives: To learn about the two major multi-modality molecular imaging techniques of small animals. To learn about the spatial resolution achievable by the molecular imaging systems for small animal today. To learn about the new multi-modality imaging instrumentation and techniques that are being developed. Sang Hyun Cho; X-ray fluorescence (XRF) imaging, such as x-ray fluorescence computed tomography (XFCT), offers unique capabilities for accurate identification and quantification of metals within the imaging objects. As a result, it has emerged as a promising quantitative imaging modality in recent years, especially in conjunction with metal-based imaging probes. This talk will familiarize the audience with the basic principles of XRF/XFCT imaging. It will also cover the latest development of benchtop XFCT technology. Additionally, the use of metallic nanoparticles such as gold nanoparticles, in conjunction with benchtop XFCT, will be discussed within the context of preclinical multimodal multiplexed molecular imaging. Learning Objectives: To learn the basic principles of XRF/XFCT imaging To learn the latest advances in benchtop XFCT development for preclinical imaging Funding support received from NIH and DOD; Funding support received from GE Healthcare; Funding support received from Siemens AX; Patent royalties received from GE Healthcare; L. Wang, Funding Support: NIH; COI: Microphotoacoustics; S. Cho, Yes: ;NIH/NCI grant R01CA155446 DOD/PCRP grant W81XWH-12-1-0198.« less
A hybrid multimodal non-rigid registration of MR images based on diffeomorphic demons.

PubMed

Lu, Huanxiang; Cattin, Philippe C; Reyes, Mauricio

2010-01-01

In this paper we present a novel hybrid approach for multimodal medical image registration based on diffeomorphic demons. Diffeomorphic demons have proven to be a robust and efficient way for intensity-based image registration. A very recent extension even allows to use mutual information (MI) as a similarity measure to registration multimodal images. However, due to the intensity correspondence uncertainty existing in some anatomical parts, it is difficult for a purely intensity-based algorithm to solve the registration problem. Therefore, we propose to combine the resulting transformations from both intensity-based and landmark-based methods for multimodal non-rigid registration based on diffeomorphic demons. Several experiments on different types of MR images were conducted, for which we show that a better anatomical correspondence between the images can be obtained using the hybrid approach than using either intensity information or landmarks alone.
Multimodal imaging of cutaneous wound tissue

NASA Astrophysics Data System (ADS)

Zhang, Shiwu; Gnyawali, Surya; Huang, Jiwei; Ren, Wenqi; Gordillo, Gayle; Sen, Chandan K.; Xu, Ronald

2015-01-01

Quantitative assessment of wound tissue ischemia, perfusion, and inflammation provides critical information for appropriate detection, staging, and treatment of chronic wounds. However, few methods are available for simultaneous assessment of these tissue parameters in a noninvasive and quantitative fashion. We integrated hyperspectral, laser speckle, and thermographic imaging modalities in a single-experimental setup for multimodal assessment of tissue oxygenation, perfusion, and inflammation characteristics. Algorithms were developed for appropriate coregistration between wound images acquired by different imaging modalities at different times. The multimodal wound imaging system was validated in an occlusion experiment, where oxygenation and perfusion maps of a healthy subject's upper extremity were continuously monitored during a postocclusive reactive hyperemia procedure and compared with standard measurements. The system was also tested in a clinical trial where a wound of three millimeters in diameter was introduced on a healthy subject's lower extremity and the healing process was continuously monitored. Our in vivo experiments demonstrated the clinical feasibility of multimodal cutaneous wound imaging.
MULTIMODAL IMAGING OF SYPHILITIC MULTIFOCAL RETINITIS.

PubMed

Curi, Andre L; Sarraf, David; Cunningham, Emmett T

2015-01-01

To describe multimodal imaging of syphilitic multifocal retinitis. Observational case series. Two patients developed multifocal retinitis after treatment of unrecognized syphilitic uveitis with systemic corticosteroids in the absence of appropriate antibiotic therapy. Multimodal imaging localized the foci of retinitis within the retina in contrast to superficial retinal precipitates that accumulate on the surface of the retina in eyes with untreated syphilitic uveitis. Although the retinitis resolved after treatment with systemic penicillin in both cases, vision remained poor in the patient with multifocal retinitis involving the macula. Treatment of unrecognized syphilitic uveitis with corticosteroids in the absence of antitreponemal treatment can lead to the development of multifocal retinitis. Multimodal imaging, and optical coherence tomography in particular, can be used to distinguish multifocal retinitis from superficial retinal precipitates or accumulations.
Adaptive wavefront shaping for controlling nonlinear multimode interactions in optical fibres

NASA Astrophysics Data System (ADS)

Tzang, Omer; Caravaca-Aguirre, Antonio M.; Wagner, Kelvin; Piestun, Rafael

2018-06-01

Recent progress in wavefront shaping has enabled control of light propagation inside linear media to focus and image through scattering objects. In particular, light propagation in multimode fibres comprises complex intermodal interactions and rich spatiotemporal dynamics. Control of physical phenomena in multimode fibres and its applications are in their infancy, opening opportunities to take advantage of complex nonlinear modal dynamics. Here, we demonstrate a wavefront shaping approach for controlling nonlinear phenomena in multimode fibres. Using a spatial light modulator at the fibre input, real-time spectral feedback and a genetic algorithm optimization, we control a highly nonlinear multimode stimulated Raman scattering cascade and its interplay with four-wave mixing via a flexible implicit control on the superposition of modes coupled into the fibre. We show versatile spectrum manipulations including shifts, suppression, and enhancement of Stokes and anti-Stokes peaks. These demonstrations illustrate the power of wavefront shaping to control and optimize nonlinear wave propagation.
Mechanisms of murine cerebral malaria: Multimodal imaging of altered cerebral metabolism and protein oxidation at hemorrhage sites

PubMed Central

Hackett, Mark J.; Aitken, Jade B.; El-Assaad, Fatima; McQuillan, James A.; Carter, Elizabeth A.; Ball, Helen J.; Tobin, Mark J.; Paterson, David; de Jonge, Martin D.; Siegele, Rainer; Cohen, David D.; Vogt, Stefan; Grau, Georges E.; Hunt, Nicholas H.; Lay, Peter A.

2015-01-01

Using a multimodal biospectroscopic approach, we settle several long-standing controversies over the molecular mechanisms that lead to brain damage in cerebral malaria, which is a major health concern in developing countries because of high levels of mortality and permanent brain damage. Our results provide the first conclusive evidence that important components of the pathology of cerebral malaria include peroxidative stress and protein oxidation within cerebellar gray matter, which are colocalized with elevated nonheme iron at the site of microhemorrhage. Such information could not be obtained previously from routine imaging methods, such as electron microscopy, fluorescence, and optical microscopy in combination with immunocytochemistry, or from bulk assays, where the level of spatial information is restricted to the minimum size of tissue that can be dissected. We describe the novel combination of chemical probe–free, multimodal imaging to quantify molecular markers of disturbed energy metabolism and peroxidative stress, which were used to provide new insights into understanding the pathogenesis of cerebral malaria. In addition to these mechanistic insights, the approach described acts as a template for the future use of multimodal biospectroscopy for understanding the molecular processes involved in a range of clinically important acute and chronic (neurodegenerative) brain diseases to improve treatment strategies. PMID:26824064
Multimodality cardiac imaging at IRCCS Policlinico San Donato: a new interdisciplinary vision.

PubMed

Lombardi, Massimo; Secchi, Francesco; Pluchinotta, Francesca R; Castelvecchio, Serenella; Montericcio, Vincenzo; Camporeale, Antonia; Bandera, Francesco

2016-04-28

Multimodality imaging is the efficient integration of various methods of cardiovascular imaging to improve the ability to diagnose, guide therapy, or predict outcome. This approach implies both the availability of different technologies in a single unit and the presence of dedicated staff with cardiologic and radiologic background and certified competence in more than one imaging technique. Interaction with clinical practice and existence of research programmes and educational activities are pivotal for the success of this model. The aim of this paper is to describe the multimodality cardiac imaging programme recently started at San Donato Hospital.
A meta-classifier for detecting prostate cancer by quantitative integration of in vivo magnetic resonance spectroscopy and magnetic resonance imaging

NASA Astrophysics Data System (ADS)

Viswanath, Satish; Tiwari, Pallavi; Rosen, Mark; Madabhushi, Anant

2008-03-01

Recently, in vivo Magnetic Resonance Imaging (MRI) and Magnetic Resonance Spectroscopy (MRS) have emerged as promising new modalities to aid in prostate cancer (CaP) detection. MRI provides anatomic and structural information of the prostate while MRS provides functional data pertaining to biochemical concentrations of metabolites such as creatine, choline and citrate. We have previously presented a hierarchical clustering scheme for CaP detection on in vivo prostate MRS and have recently developed a computer-aided method for CaP detection on in vivo prostate MRI. In this paper we present a novel scheme to develop a meta-classifier to detect CaP in vivo via quantitative integration of multimodal prostate MRS and MRI by use of non-linear dimensionality reduction (NLDR) methods including spectral clustering and locally linear embedding (LLE). Quantitative integration of multimodal image data (MRI and PET) involves the concatenation of image intensities following image registration. However multimodal data integration is non-trivial when the individual modalities include spectral and image intensity data. We propose a data combination solution wherein we project the feature spaces (image intensities and spectral data) associated with each of the modalities into a lower dimensional embedding space via NLDR. NLDR methods preserve the relationships between the objects in the original high dimensional space when projecting them into the reduced low dimensional space. Since the original spectral and image intensity data are divorced from their original physical meaning in the reduced dimensional space, data at the same spatial location can be integrated by concatenating the respective embedding vectors. Unsupervised consensus clustering is then used to partition objects into different classes in the combined MRS and MRI embedding space. Quantitative results of our multimodal computer-aided diagnosis scheme on 16 sets of patient data obtained from the ACRIN trial, for which corresponding histological ground truth for spatial extent of CaP is known, show a marginally higher sensitivity, specificity, and positive predictive value compared to corresponding CAD results with the individual modalities.

Multi-modality nanoparticles having optically responsive shape

DOEpatents

Chen, Fanqing; Bouchard, Louis-Serge

2015-05-19

In certain embodiments novel nanoparticles (nanowontons) are provided that are suitable for multimodal imaging and/or therapy. In one embodiment, the nanoparticles include a first biocompatible (e.g., gold) layer, an inner core layer (e.g., a non-biocompatible material), and a biocompatible (e.g., gold) layer. The first gold layer includes a concave surface that forms a first outer surface of the layered nanoparticle. The second gold layer includes a convex surface that forms a second outer surface of the layered nanoparticle. The first and second gold layers encapsulate the inner core material layer. Methods of fabricating such nanoparticles are also provided.
A multimodal parallel architecture: A cognitive framework for multimodal interactions.

PubMed

Cohn, Neil

2016-01-01

Human communication is naturally multimodal, and substantial focus has examined the semantic correspondences in speech-gesture and text-image relationships. However, visual narratives, like those in comics, provide an interesting challenge to multimodal communication because the words and/or images can guide the overall meaning, and both modalities can appear in complicated "grammatical" sequences: sentences use a syntactic structure and sequential images use a narrative structure. These dual structures create complexity beyond those typically addressed by theories of multimodality where only a single form uses combinatorial structure, and also poses challenges for models of the linguistic system that focus on single modalities. This paper outlines a broad theoretical framework for multimodal interactions by expanding on Jackendoff's (2002) parallel architecture for language. Multimodal interactions are characterized in terms of their component cognitive structures: whether a particular modality (verbal, bodily, visual) is present, whether it uses a grammatical structure (syntax, narrative), and whether it "dominates" the semantics of the overall expression. Altogether, this approach integrates multimodal interactions into an existing framework of language and cognition, and characterizes interactions between varying complexity in the verbal, bodily, and graphic domains. The resulting theoretical model presents an expanded consideration of the boundaries of the "linguistic" system and its involvement in multimodal interactions, with a framework that can benefit research on corpus analyses, experimentation, and the educational benefits of multimodality. Copyright © 2015.
Depth-resolved imaging of colon tumor using optical coherence tomography and fluorescence laminar optical tomography (Conference Presentation)

NASA Astrophysics Data System (ADS)

Tang, Qinggong; Frank, Aaron; Wang, Jianting; Chen, Chao-wei; Jin, Lily; Lin, Jon; Chan, Joanne M.; Chen, Yu

2016-03-01

Early detection of neoplastic changes remains a critical challenge in clinical cancer diagnosis and treatment. Many cancers arise from epithelial layers such as those of the gastrointestinal (GI) tract. Current standard endoscopic technology is unable to detect those subsurface lesions. Since cancer development is associated with both morphological and molecular alterations, imaging technologies that can quantitative image tissue's morphological and molecular biomarkers and assess the depth extent of a lesion in real time, without the need for tissue excision, would be a major advance in GI cancer diagnostics and therapy. In this research, we investigated the feasibility of multi-modal optical imaging including high-resolution optical coherence tomography (OCT) and depth-resolved high-sensitivity fluorescence laminar optical tomography (FLOT) for structural and molecular imaging. APC (adenomatous polyposis coli) mice model were imaged using OCT and FLOT and the correlated histopathological diagnosis was obtained. Quantitative structural (the scattering coefficient) and molecular imaging parameters (fluorescence intensity) from OCT and FLOT images were developed for multi-parametric analysis. This multi-modal imaging method has demonstrated the feasibility for more accurate diagnosis with 87.4% (87.3%) for sensitivity (specificity) which gives the most optimal diagnosis (the largest area under receiver operating characteristic (ROC) curve). This project results in a new non-invasive multi-modal imaging platform for improved GI cancer detection, which is expected to have a major impact on detection, diagnosis, and characterization of GI cancers, as well as a wide range of epithelial cancers.
Multimode nonlinear optical imaging of the dermis in ex vivo human skin based on the combination of multichannel mode and Lambda mode.

PubMed

Zhuo, Shuangmu; Chen, Jianxin; Luo, Tianshu; Zou, Dingsong

2006-08-21

A Multimode nonlinear optical imaging technique based on the combination of multichannel mode and Lambda mode is developed to investigate human dermis. Our findings show that this technique not only improves the image contrast of the structural proteins of extracellular matrix (ECM) but also provides an image-guided spectral analysis method to identify both cellular and ECM intrinsic components including collagen, elastin, NAD(P)H and flavin. By the combined use of multichannel mode and Lambda mode in tandem, the obtained in-depth two photon-excited fluorescence (TPEF) and second-harmonic generation (SHG) imaging and TPEF/SHG signals depth-dependence decay can offer a sensitive tool for obtaining quantitative tissue structural and biochemical information. These results suggest that the technique has the potential to provide more accurate information for determining tissue physiological and pathological states.
Multimodal Chemical Imaging of Amyloid Plaque Polymorphism Reveals Aβ Aggregation Dependent Anionic Lipid Accumulations and Metabolism.

PubMed

Michno, Wojciech; Kaya, Ibrahim; Nyström, Sofie; Guerard, Laurent; Nilsson, K Peter R; Hammarström, Per; Blennow, Kaj; Zetterberg, Henrik; Hanrieder, Jörg

2018-06-01

Amyloid plaque formation constitutes one of the main pathological hallmark of Alzheimer's disease (AD) and is suggested to be a critical factor driving disease pathogenesis. Interestingly, in patients that display amyloid pathology but remain cognitively normal, Aβ deposits are predominantly of diffuse morphology suggesting that cored plaque formation is primarily associated with cognitive deterioration and AD pathogenesis. Little is known about the molecular mechanism responsible for conversion of monomeric Aβ into neurotoxic aggregates and the predominantly cored deposits observed in AD. The structural diversity among Aβ plaques, including cored/compact- and diffuse, may be linked to their distinct Aβ profile and other chemical species including neuronal lipids. We developed a novel, chemical imaging paradigm combining matrix assisted laser desorption/ionization imaging mass spectrometry (MALDI IMS) and fluorescent amyloid staining. This multimodal imaging approach was used to probe the lipid chemistry associated with structural plaque heterogeneity in transgenic AD mice (tgAPPSwe) and was correlated to Aβ profiles determined by subsequent laser microdissection and immunoprecipitation-mass spectrometry. Multivariate image analysis revealed an inverse localization of ceramides and their matching metabolites to diffuse and cored structures within single plaques, respectively. Moreover, phosphatidylinositols implicated in AD pathogenesis, were found to localise to the diffuse Aβ structures and correlate with Aβ1-42. Further, lysophospholipids implicated in neuroinflammation were increased in all Aβ deposits. The results support previous clinical findings on the importance of lipid disturbances in AD pathophysiology and associated sphingolipid processing. These data highlight the potential of multimodal imaging as a powerful technology to probe neuropathological mechanisms.
A digital 3D atlas of the marmoset brain based on multi-modal MRI.

PubMed

Liu, Cirong; Ye, Frank Q; Yen, Cecil Chern-Chyi; Newman, John D; Glen, Daniel; Leopold, David A; Silva, Afonso C

2018-04-01

The common marmoset (Callithrix jacchus) is a New-World monkey of growing interest in neuroscience. Magnetic resonance imaging (MRI) is an essential tool to unveil the anatomical and functional organization of the marmoset brain. To facilitate identification of regions of interest, it is desirable to register MR images to an atlas of the brain. However, currently available atlases of the marmoset brain are mainly based on 2D histological data, which are difficult to apply to 3D imaging techniques. Here, we constructed a 3D digital atlas based on high-resolution ex-vivo MRI images, including magnetization transfer ratio (a T1-like contrast), T2w images, and multi-shell diffusion MRI. Based on the multi-modal MRI images, we manually delineated 54 cortical areas and 16 subcortical regions on one hemisphere of the brain (the core version). The 54 cortical areas were merged into 13 larger cortical regions according to their locations to yield a coarse version of the atlas, and also parcellated into 106 sub-regions using a connectivity-based parcellation method to produce a refined atlas. Finally, we compared the new atlas set with existing histology atlases and demonstrated its applications in connectome studies, and in resting state and stimulus-based fMRI. The atlas set has been integrated into the widely-distributed neuroimaging data analysis software AFNI and SUMA, providing a readily usable multi-modal template space with multi-level anatomical labels (including labels from the Paxinos atlas) that can facilitate various neuroimaging studies of marmosets. Published by Elsevier Inc.
Toward in vivo diagnosis of skin cancer using multimode imaging dermoscopy: (II) molecular mapping of highly pigmented lesions

NASA Astrophysics Data System (ADS)

Vasefi, Fartash; MacKinnon, Nicholas; Farkas, Daniel L.

2014-03-01

We have developed a multimode imaging dermoscope that combines polarization and hyperspectral imaging with a computationally rapid analytical model. This approach employs specific spectral ranges of visible and near infrared wavelengths for mapping the distribution of specific skin bio-molecules. This corrects for the melanin-hemoglobin misestimation common to other systems, without resorting to complex and computationally intensive tissue optical models that are prone to inaccuracies due to over-modeling. Various human skin measurements including a melanocytic nevus, and venous occlusion conditions were investigated and compared with other ratiometric spectral imaging approaches. Access to the broad range of hyperspectral data in the visible and near-infrared range allows our algorithm to flexibly use different wavelength ranges for chromophore estimation while minimizing melanin-hemoglobin optical signature cross-talk.
A simultaneous multimodal imaging system for tissue functional parameters

NASA Astrophysics Data System (ADS)

Ren, Wenqi; Zhang, Zhiwu; Wu, Qiang; Zhang, Shiwu; Xu, Ronald

2014-02-01

Simultaneous and quantitative assessment of skin functional characteristics in different modalities will facilitate diagnosis and therapy in many clinical applications such as wound healing. However, many existing clinical practices and multimodal imaging systems are subjective, qualitative, sequential for multimodal data collection, and need co-registration between different modalities. To overcome these limitations, we developed a multimodal imaging system for quantitative, non-invasive, and simultaneous imaging of cutaneous tissue oxygenation and blood perfusion parameters. The imaging system integrated multispectral and laser speckle imaging technologies into one experimental setup. A Labview interface was developed for equipment control, synchronization, and image acquisition. Advanced algorithms based on a wide gap second derivative reflectometry and laser speckle contrast analysis (LASCA) were developed for accurate reconstruction of tissue oxygenation and blood perfusion respectively. Quantitative calibration experiments and a new style of skinsimulating phantom were designed to verify the accuracy and reliability of the imaging system. The experimental results were compared with a Moor tissue oxygenation and perfusion monitor. For In vivo testing, a post-occlusion reactive hyperemia (PORH) procedure in human subject and an ongoing wound healing monitoring experiment using dorsal skinfold chamber models were conducted to validate the usability of our system for dynamic detection of oxygenation and perfusion parameters. In this study, we have not only setup an advanced multimodal imaging system for cutaneous tissue oxygenation and perfusion parameters but also elucidated its potential for wound healing assessment in clinical practice.
Mixture-Tuned, Clutter Matched Filter for Remote Detection of Subpixel Spectral Signals

NASA Technical Reports Server (NTRS)

Thompson, David R.; Mandrake, Lukas; Green, Robert O.

2013-01-01

Mapping localized spectral features in large images demands sensitive and robust detection algorithms. Two aspects of large images that can harm matched-filter detection performance are addressed simultaneously. First, multimodal backgrounds may thwart the typical Gaussian model. Second, outlier features can trigger false detections from large projections onto the target vector. Two state-of-the-art approaches are combined that independently address outlier false positives and multimodal backgrounds. The background clustering models multimodal backgrounds, and the mixture tuned matched filter (MT-MF) addresses outliers. Combining the two methods captures significant additional performance benefits. The resulting mixture tuned clutter matched filter (MT-CMF) shows effective performance on simulated and airborne datasets. The classical MNF transform was applied, followed by k-means clustering. Then, each cluster s mean, covariance, and the corresponding eigenvalues were estimated. This yields a cluster-specific matched filter estimate as well as a cluster- specific feasibility score to flag outlier false positives. The technology described is a proof of concept that may be employed in future target detection and mapping applications for remote imaging spectrometers. It is of most direct relevance to JPL proposals for airborne and orbital hyperspectral instruments. Applications include subpixel target detection in hyperspectral scenes for military surveillance. Earth science applications include mineralogical mapping, species discrimination for ecosystem health monitoring, and land use classification.
BNU-LSVED: a multimodal spontaneous expression database in educational environment

NASA Astrophysics Data System (ADS)

Sun, Bo; Wei, Qinglan; He, Jun; Yu, Lejun; Zhu, Xiaoming

2016-09-01

In the field of pedagogy or educational psychology, emotions are treated as very important factors, which are closely associated with cognitive processes. Hence, it is meaningful for teachers to analyze students' emotions in classrooms, thus adjusting their teaching activities and improving students ' individual development. To provide a benchmark for different expression recognition algorithms, a large collection of training and test data in classroom environment has become an acute problem that needs to be resolved. In this paper, we present a multimodal spontaneous database in real learning environment. To collect the data, students watched seven kinds of teaching videos and were simultaneously filmed by a camera. Trained coders made one of the five learning expression labels for each image sequence extracted from the captured videos. This subset consists of 554 multimodal spontaneous expression image sequences (22,160 frames) recorded in real classrooms. There are four main advantages in this database. 1) Due to recorded in the real classroom environment, viewer's distance from the camera and the lighting of the database varies considerably between image sequences. 2) All the data presented are natural spontaneous responses to teaching videos. 3) The multimodal database also contains nonverbal behavior including eye movement, head posture and gestures to infer a student ' s affective state during the courses. 4) In the video sequences, there are different kinds of temporal activation patterns. In addition, we have demonstrated the labels for the image sequences are in high reliability through Cronbach's alpha method.
Acute imaging does not improve ASTRAL score's accuracy despite having a prognostic value.

PubMed

Ntaios, George; Papavasileiou, Vasileios; Faouzi, Mohamed; Vanacker, Peter; Wintermark, Max; Michel, Patrik

2014-10-01

The ASTRAL score was recently shown to reliably predict three-month functional outcome in patients with acute ischemic stroke. The study aims to investigate whether information from multimodal imaging increases ASTRAL score's accuracy. All patients registered in the ASTRAL registry until March 2011 were included. In multivariate logistic-regression analyses, we added covariates derived from parenchymal, vascular, and perfusion imaging to the 6-parameter model of the ASTRAL score. If a specific imaging covariate remained an independent predictor of three-month modified Rankin score>2, the area-under-the-curve (AUC) of this new model was calculated and compared with ASTRAL score's AUC. We also performed similar logistic regression analyses in arbitrarily chosen patient subgroups. When added to the ASTRAL score, the following covariates on admission computed tomography/magnetic resonance imaging-based multimodal imaging were not significant predictors of outcome: any stroke-related acute lesion, any nonstroke-related lesions, chronic/subacute stroke, leukoaraiosis, significant arterial pathology in ischemic territory on computed tomography angiography/magnetic resonance angiography/Doppler, significant intracranial arterial pathology in ischemic territory, and focal hypoperfusion on perfusion-computed tomography. The Alberta Stroke Program Early CT score on plain imaging and any significant extracranial arterial pathology on computed tomography angiography/magnetic resonance angiography/Doppler were independent predictors of outcome (odds ratio: 0·93, 95% CI: 0·87-0·99 and odds ratio: 1·49, 95% CI: 1·08-2·05, respectively) but did not increase ASTRAL score's AUC (0·849 vs. 0·850, and 0·8563 vs. 0·8564, respectively). In exploratory analyses in subgroups of different prognosis, age or stroke severity, no covariate was found to increase ASTRAL score's AUC, either. The addition of information derived from multimodal imaging does not increase ASTRAL score's accuracy to predict functional outcome despite having an independent prognostic value. More selected radiological parameters applied in specific subgroups of stroke patients may add prognostic value of multimodal imaging. © 2014 World Stroke Organization.
Exploring photoreceptor reflectivity via multimodal imaging of outer retinal tubulation in advanced age-related macular degeneration

PubMed Central

Litts, Katie M.; Wang, Xiaolin; Clark, Mark E.; Owsley, Cynthia; Freund, K. Bailey; Curcio, Christine A.; Zhang, Yuhua

2016-01-01

Purpose To investigate the microscopic structure of outer retinal tubulation (ORT) and optical properties of cone photoreceptors in vivo, we studied ORT appearance by multimodal imaging, including spectral domain optical coherence tomography (SD-OCT) and adaptive optics scanning laser ophthalmoscopy (AOSLO). Methods Four eyes of 4 subjects with advanced AMD underwent color fundus photography, infrared reflectance imaging, SD-OCT, and AOSLO with a high-resolution research instrument. ORT was identified in closely spaced (11 μm) SD-OCT volume scans. Results ORT in cross-sectional and en face SD-OCT was a hyporeflective area representing a lumen surrounded by a hyperreflective border consisting of cone photoreceptor mitochondria and external limiting membrane, per previous histology. In contrast, ORT by AOSLO was a hyporeflective structure of the same shape as in en face SD-OCT but lacking visualizable cone photoreceptors. Conclusion Lack of ORT cone reflectivity by AOSLO indicates that cones have lost their normal directionality and waveguiding property due to loss of outer segments and subsequent retinal remodeling. Reflective ORT cones by SD-OCT, in contrast, may depend partly on mitochondria as light scatterers within inner segments of these degenerating cells, a phenomenon enhanced by coherent imaging. Multimodal imaging of ORT provides insight into cone degeneration and reflectivity sources in OCT. PMID:27584549
A new approach of building 3D visualization framework for multimodal medical images display and computed assisted diagnosis

NASA Astrophysics Data System (ADS)

Li, Zhenwei; Sun, Jianyong; Zhang, Jianguo

2012-02-01

As more and more CT/MR studies are scanning with larger volume of data sets, more and more radiologists and clinician would like using PACS WS to display and manipulate these larger data sets of images with 3D rendering features. In this paper, we proposed a design method and implantation strategy to develop 3D image display component not only with normal 3D display functions but also with multi-modal medical image fusion as well as compute-assisted diagnosis of coronary heart diseases. The 3D component has been integrated into the PACS display workstation of Shanghai Huadong Hospital, and the clinical practice showed that it is easy for radiologists and physicians to use these 3D functions such as multi-modalities' (e.g. CT, MRI, PET, SPECT) visualization, registration and fusion, and the lesion quantitative measurements. The users were satisfying with the rendering speeds and quality of 3D reconstruction. The advantages of the component include low requirements for computer hardware, easy integration, reliable performance and comfortable application experience. With this system, the radiologists and the clinicians can manipulate with 3D images easily, and use the advanced visualization tools to facilitate their work with a PACS display workstation at any time.
Optical/MRI Multimodality Molecular Imaging

NASA Astrophysics Data System (ADS)

Ma, Lixin; Smith, Charles; Yu, Ping

2007-03-01

Multimodality molecular imaging that combines anatomical and functional information has shown promise in development of tumor-targeted pharmaceuticals for cancer detection or therapy. We present a new multimodality imaging technique that combines fluorescence molecular tomography (FMT) and magnetic resonance imaging (MRI) for in vivo molecular imaging of preclinical tumor models. Unlike other optical/MRI systems, the new molecular imaging system uses parallel phase acquisition based on heterodyne principle. The system has a higher accuracy of phase measurements, reduced noise bandwidth, and an efficient modulation of the fluorescence diffuse density waves. Fluorescent Bombesin probes were developed for targeting breast cancer cells and prostate cancer cells. Tissue phantom and small animal experiments were performed for calibration of the imaging system and validation of the targeting probes.
Multimodal nanoparticle imaging agents: design and applications

NASA Astrophysics Data System (ADS)

Burke, Benjamin P.; Cawthorne, Christopher; Archibald, Stephen J.

2017-10-01

Molecular imaging, where the location of molecules or nanoscale constructs can be tracked in the body to report on disease or biochemical processes, is rapidly expanding to include combined modality or multimodal imaging. No single imaging technique can offer the optimum combination of properties (e.g. resolution, sensitivity, cost, availability). The rapid technological advances in hardware to scan patients, and software to process and fuse images, are pushing the boundaries of novel medical imaging approaches, and hand-in-hand with this is the requirement for advanced and specific multimodal imaging agents. These agents can be detected using a selection from radioisotope, magnetic resonance and optical imaging, among others. Nanoparticles offer great scope in this area as they lend themselves, via facile modification procedures, to act as multifunctional constructs. They have relevance as therapeutics and drug delivery agents that can be tracked by molecular imaging techniques with the particular development of applications in optically guided surgery and as radiosensitizers. There has been a huge amount of research work to produce nanoconstructs for imaging, and the parameters for successful clinical translation and validation of therapeutic applications are now becoming much better understood. It is an exciting time of progress for these agents as their potential is closer to being realized with translation into the clinic. The coming 5-10 years will be critical, as we will see if the predicted improvement in clinical outcomes becomes a reality. Some of the latest advances in combination modality agents are selected and the progression pathway to clinical trials analysed. This article is part of the themed issue 'Challenges for chemistry in molecular imaging'.
Multi-modality molecular imaging: pre-clinical laboratory configuration

NASA Astrophysics Data System (ADS)

Wu, Yanjun; Wellen, Jeremy W.; Sarkar, Susanta K.

2006-02-01

In recent years, the prevalence of in vivo molecular imaging applications has rapidly increased. Here we report on the construction of a multi-modality imaging facility in a pharmaceutical setting that is expected to further advance existing capabilities for in vivo imaging of drug distribution and the interaction with their target. The imaging instrumentation in our facility includes a microPET scanner, a four wavelength time-domain optical imaging scanner, a 9.4T/30cm MRI scanner and a SPECT/X-ray CT scanner. An electronics shop and a computer room dedicated to image analysis are additional features of the facility. The layout of the facility was designed with a central animal preparation room surrounded by separate laboratory rooms for each of the major imaging modalities to accommodate the work-flow of simultaneous in vivo imaging experiments. This report will focus on the design of and anticipated applications for our microPET and optical imaging laboratory spaces. Additionally, we will discuss efforts to maximize the daily throughput of animal scans through development of efficient experimental work-flows and the use of multiple animals in a single scanning session.
Multi-Modal Ultra-Widefield Imaging Features in Waardenburg Syndrome

PubMed Central

Choudhry, Netan; Rao, Rajesh C.

2015-01-01

Background Waardenburg syndrome is characterized by a group of features including; telecanthus, a broad nasal root, synophrys of the eyebrows, piedbaldism, heterochromia irides, and deaf-mutism. Hypopigmentation of the choroid is a unique feature of this condition examined with multi-modal Ultra-Widefield Imaging in this report. Material/Methods Report of a single case. Results Bilateral symmetric choroidal hypopigmentation was observed with hypoautofluorescence in the region of hypopigmentation. Fluorescein angiography revealed a normal vasculature, however a thickened choroid was seen on Enhanced-Depth Imaging Spectral-Domain OCT (EDI SD-OCT). Conclusion(s) Choroidal hypopigmentation is a unique feature of Waardenburg syndrome, which can be visualized with ultra-widefield fundus autofluorescence. The choroid may also be thickened in this condition and its thickness measured with EDI SD-OCT. PMID:26114849
Superparamagnetic nanoparticles for enhanced magnetic resonance and multimodal imaging

NASA Astrophysics Data System (ADS)

Sikma, Elise Ann Schultz

Magnetic resonance imaging (MRI) is a powerful tool for noninvasive tomographic imaging of biological systems with high spatial and temporal resolution. Superparamagnetic (SPM) nanoparticles have emerged as highly effective MR contrast agents due to their biocompatibility, ease of surface modification and magnetic properties. Conventional nanoparticle contrast agents suffer from difficult synthetic reproducibility, polydisperse sizes and weak magnetism. Numerous synthetic techniques and nanoparticle formulations have been developed to overcome these barriers. However, there are still major limitations in the development of new nanoparticle-based probes for MR and multimodal imaging including low signal amplification and absence of biochemical reporters. To address these issues, a set of multimodal (T2/optical) and dual contrast (T1/T2) nanoparticle probes has been developed. Their unique magnetic properties and imaging capabilities were thoroughly explored. An enzyme-activatable contrast agent is currently being developed as an innovative means for early in vivo detection of cancer at the cellular level. Multimodal probes function by combining the strengths of multiple imaging techniques into a single agent. Co-registration of data obtained by multiple imaging modalities validates the data, enhancing its quality and reliability. A series of T2/optical probes were successfully synthesized by attachment of a fluorescent dye to the surface of different types of nanoparticles. The multimodal nanoparticles generated sufficient MR and fluorescence signal to image transplanted islets in vivo. Dual contrast T1/T2 imaging probes were designed to overcome disadvantages inherent in the individual T1 and T2 components. A class of T1/T2 agents was developed consisting of a gadolinium (III) complex (DTPA chelate or DO3A macrocycle) conjugated to a biocompatible silica-coated metal oxide nanoparticle through a disulfide linker. The disulfide linker has the ability to be reduced in vivo by glutathione, releasing large payloads of signal-enhancing T1 probes into the surrounding environment. Optimization of the agent occurred over three sequential generations, with each generation addressing a new challenge. The result was a T2 nanoparticle containing high levels of conjugated T1 complex demonstrating enhanced MR relaxation properties. The probes created here have the potential to play a key role in the advancement of nanoparticle-based agents in biomedical MRI applications.
Medical Image Retrieval: A Multimodal Approach

PubMed Central

Cao, Yu; Steffey, Shawn; He, Jianbiao; Xiao, Degui; Tao, Cui; Chen, Ping; Müller, Henning

2014-01-01

Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system. PMID:26309389
Adaptive Optics Imaging in Laser Pointer Maculopathy.

PubMed

Sheyman, Alan T; Nesper, Peter L; Fawzi, Amani A; Jampol, Lee M

2016-08-01

The authors report multimodal imaging including adaptive optics scanning laser ophthalmoscopy (AOSLO) (Apaeros retinal image system AOSLO prototype; Boston Micromachines Corporation, Boston, MA) in a case of previously diagnosed unilateral acute idiopathic maculopathy (UAIM) that demonstrated features of laser pointer maculopathy. The authors also show the adaptive optics images of a laser pointer maculopathy case previously reported. A 15-year-old girl was referred for the evaluation of a maculopathy suspected to be UAIM. The authors reviewed the patient's history and obtained fluorescein angiography, autofluorescence, optical coherence tomography, infrared reflectance, and AOSLO. The time course of disease and clinical examination did not fit with UAIM, but the linear pattern of lesions was suspicious for self-inflicted laser pointer injury. This was confirmed on subsequent questioning of the patient. The presence of linear lesions in the macula that are best highlighted with multimodal imaging techniques should alert the physician to the possibility of laser pointer injury. AOSLO further characterizes photoreceptor damage in this condition. [Ophthalmic Surg Lasers Imaging Retina. 2016;47:782-785.]. Copyright 2016, SLACK Incorporated.

Retinal imaging in uveitis

PubMed Central

Gupta, Vishali; Al-Dhibi, Hassan A.; Arevalo, J. Fernando

2014-01-01

Ancillary investigations are the backbone of uveitis workup for posterior segment inflammations. They help in establishing the differential diagnosis and making certain diagnosis by ruling out certain pathologies and are a useful aid in monitoring response to therapy during follow-up. These investigations include fundus photography including ultra wide field angiography, fundus autofluorescence imaging, fluorescein angiography, optical coherence tomography and multimodal imaging. This review aims to be an overview describing the role of these retinal investigations for posterior uveitis. PMID:24843301
Tinnitus Multimodal Imaging

DTIC Science & Technology

2016-12-01

images were segmented into gray and white matter images and spatially normalized to the MNI template (3 mm isotropic voxels) using the DARTEL toolbox in...AWARD NUMBER: W81XWH-13-1-0494 TITLE: Tinnitus Multimodal Imaging PRINCIPAL INVESTIGATOR: Steven Wan Cheung CONTRACTING ORGANIZATION... Medical Research and Materiel Command Fort Detrick, Maryland 21702-5012 DISTRIBUTION STATEMENT: Approved for Public Release; Distribution Unlimited
Melanoma detection using smartphone and multimode hyperspectral imaging

NASA Astrophysics Data System (ADS)

MacKinnon, Nicholas; Vasefi, Fartash; Booth, Nicholas; Farkas, Daniel L.

2016-04-01

This project's goal is to determine how to effectively implement a technology continuum from a low cost, remotely deployable imaging device to a more sophisticated multimode imaging system within a standard clinical practice. In this work a smartphone is used in conjunction with an optical attachment to capture cross-polarized and collinear color images of a nevus that are analyzed to quantify chromophore distribution. The nevus is also imaged by a multimode hyperspectral system, our proprietary SkinSpect™ device. Relative accuracy and biological plausibility of the two systems algorithms are compared to assess aspects of feasibility of in-home or primary care practitioner smartphone screening prior to rigorous clinical analysis via the SkinSpect.
Simultaneous multimodal ophthalmic imaging using swept-source spectrally encoded scanning laser ophthalmoscopy and optical coherence tomography

PubMed Central

Malone, Joseph D.; El-Haddad, Mohamed T.; Bozic, Ivan; Tye, Logan A.; Majeau, Lucas; Godbout, Nicolas; Rollins, Andrew M.; Boudoux, Caroline; Joos, Karen M.; Patel, Shriji N.; Tao, Yuankai K.

2016-01-01

Scanning laser ophthalmoscopy (SLO) benefits diagnostic imaging and therapeutic guidance by allowing for high-speed en face imaging of retinal structures. When combined with optical coherence tomography (OCT), SLO enables real-time aiming and retinal tracking and provides complementary information for post-acquisition volumetric co-registration, bulk motion compensation, and averaging. However, multimodality SLO-OCT systems generally require dedicated light sources, scanners, relay optics, detectors, and additional digitization and synchronization electronics, which increase system complexity. Here, we present a multimodal ophthalmic imaging system using swept-source spectrally encoded scanning laser ophthalmoscopy and optical coherence tomography (SS-SESLO-OCT) for in vivo human retinal imaging. SESLO reduces the complexity of en face imaging systems by multiplexing spatial positions as a function of wavelength. SESLO image quality benefited from single-mode illumination and multimode collection through a prototype double-clad fiber coupler, which optimized scattered light throughput and reduce speckle contrast while maintaining lateral resolution. Using a shared 1060 nm swept-source, shared scanner and imaging optics, and a shared dual-channel high-speed digitizer, we acquired inherently co-registered en face retinal images and OCT cross-sections simultaneously at 200 frames-per-second. PMID:28101411
Advanced multimodality imaging of an anomalous vessel between the ascending aorta and main pulmonary artery in a dog

PubMed Central

Markovic, Lauren E.; Kellihan, Heidi B.; Roldán-Alzate, Alejandro; Drees, Randi; Bjorling, Dale E.; Francois, Chris J.

2014-01-01

A 1-year-old male German shorthaired pointer was referred for evaluation of tachypnea and hemoptysis. A grade VI/VI left basilar continuous murmur was ausculted. Multimodality imaging consisting of thoracic radiographs, transthoracic and transesophageal echocardiography, fluoroscopy-guided selective angiography, computed tomography angiogram (CTA) and magnetic resonance angiogram (MRA), was performed on this patient. The defect included a left-to-right shunting anomalous vessel between the ascending aorta and main pulmonary artery, along with a dissecting aneurysm of the main and right pulmonary artery. An MRA postprocessing technique (PC-VIPR) was used to allow for high resolution angiographic images and further assessment of the patient’s hemodynamics prior to surgical correction. This case report describes the clinical course of a canine patient with a rare form of congenital cardiac disease, and the multiple imaging modalities that were used to aid in diagnosis and treatment. PMID:24485987
WE-H-206-03: Promises and Challenges of Benchtop X-Ray Fluorescence CT (XFCT) for Quantitative in Vivo Imaging

DOE Office of Scientific and Technical Information (OSTI.GOV)

Cho, S.

Lihong V. Wang: Photoacoustic tomography (PAT), combining non-ionizing optical and ultrasonic waves via the photoacoustic effect, provides in vivo multiscale functional, metabolic, and molecular imaging. Broad applications include imaging of the breast, brain, skin, esophagus, colon, vascular system, and lymphatic system in humans or animals. Light offers rich contrast but does not penetrate biological tissue in straight paths as x-rays do. Consequently, high-resolution pure optical imaging (e.g., confocal microscopy, two-photon microscopy, and optical coherence tomography) is limited to penetration within the optical diffusion limit (∼1 mm in the skin). Ultrasonic imaging, on the contrary, provides fine spatial resolution but suffersmore » from both poor contrast in early-stage tumors and strong speckle artifacts. In PAT, pulsed laser light penetrates tissue and generates a small but rapid temperature rise, which induces emission of ultrasonic waves due to thermoelastic expansion. The ultrasonic waves, orders of magnitude less scattering than optical waves, are then detected to form high-resolution images of optical absorption at depths up to 7 cm, conquering the optical diffusion limit. PAT is the only modality capable of imaging across the length scales of organelles, cells, tissues, and organs (up to whole-body small animals) with consistent contrast. This rapidly growing technology promises to enable multiscale biological research and accelerate translation from microscopic laboratory discoveries to macroscopic clinical practice. PAT may also hold the key to label-free early detection of cancer by in vivo quantification of hypermetabolism, the quintessential hallmark of malignancy. Learning Objectives: To understand the contrast mechanism of PAT To understand the multiscale applications of PAT Benjamin M. W. Tsui: Multi-modality molecular imaging instrumentation and techniques have been major developments in small animal imaging that has contributed significantly to biomedical research during the past decade. The initial development was an extension of clinical PET/CT and SPECT/CT from human to small animals and combine the unique functional information obtained from PET and SPECT with anatomical information provided by the CT in registered multi-modality images. The requirements to image a mouse whose size is an order of magnitude smaller than that of a human have spurred advances in new radiation detector technologies, novel imaging system designs and special image reconstruction and processing techniques. Examples are new detector materials and designs with high intrinsic resolution, multi-pinhole (MPH) collimator design for much improved resolution and detection efficiency compared to the conventional collimator designs in SPECT, 3D high-resolution and artifact-free MPH and sparse-view image reconstruction techniques, and iterative image reconstruction methods with system response modeling for resolution recovery and image noise reduction for much improved image quality. The spatial resolution of PET and SPECT has improved from ∼6–12 mm to ∼1 mm a few years ago to sub-millimeter today. A recent commercial small animal SPECT system has achieved a resolution of ∼0.25 mm which surpasses that of a state-of-art PET system whose resolution is limited by the positron range. More recently, multimodality SA PET/MRI and SPECT/MRI systems have been developed in research laboratories. Also, multi-modality SA imaging systems that include other imaging modalities such as optical and ultrasound are being actively pursued. In this presentation, we will provide a review of the development, recent advances and future outlook of multi-modality molecular imaging of small animals. Learning Objectives: To learn about the two major multi-modality molecular imaging techniques of small animals. To learn about the spatial resolution achievable by the molecular imaging systems for small animal today. To learn about the new multi-modality imaging instrumentation and techniques that are being developed. Sang Hyun Cho; X-ray fluorescence (XRF) imaging, such as x-ray fluorescence computed tomography (XFCT), offers unique capabilities for accurate identification and quantification of metals within the imaging objects. As a result, it has emerged as a promising quantitative imaging modality in recent years, especially in conjunction with metal-based imaging probes. This talk will familiarize the audience with the basic principles of XRF/XFCT imaging. It will also cover the latest development of benchtop XFCT technology. Additionally, the use of metallic nanoparticles such as gold nanoparticles, in conjunction with benchtop XFCT, will be discussed within the context of preclinical multimodal multiplexed molecular imaging. Learning Objectives: To learn the basic principles of XRF/XFCT imaging To learn the latest advances in benchtop XFCT development for preclinical imaging Funding support received from NIH and DOD; Funding support received from GE Healthcare; Funding support received from Siemens AX; Patent royalties received from GE Healthcare; L. Wang, Funding Support: NIH; COI: Microphotoacoustics; S. Cho, Yes: ;NIH/NCI grant R01CA155446 DOD/PCRP grant W81XWH-12-1-0198.« less
WE-H-206-01: Photoacoustic Tomography: Multiscale Imaging From Organelles to Patients by Ultrasonically Beating the Optical Diffusion Limit

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wang, L.

Lihong V. Wang: Photoacoustic tomography (PAT), combining non-ionizing optical and ultrasonic waves via the photoacoustic effect, provides in vivo multiscale functional, metabolic, and molecular imaging. Broad applications include imaging of the breast, brain, skin, esophagus, colon, vascular system, and lymphatic system in humans or animals. Light offers rich contrast but does not penetrate biological tissue in straight paths as x-rays do. Consequently, high-resolution pure optical imaging (e.g., confocal microscopy, two-photon microscopy, and optical coherence tomography) is limited to penetration within the optical diffusion limit (∼1 mm in the skin). Ultrasonic imaging, on the contrary, provides fine spatial resolution but suffersmore » from both poor contrast in early-stage tumors and strong speckle artifacts. In PAT, pulsed laser light penetrates tissue and generates a small but rapid temperature rise, which induces emission of ultrasonic waves due to thermoelastic expansion. The ultrasonic waves, orders of magnitude less scattering than optical waves, are then detected to form high-resolution images of optical absorption at depths up to 7 cm, conquering the optical diffusion limit. PAT is the only modality capable of imaging across the length scales of organelles, cells, tissues, and organs (up to whole-body small animals) with consistent contrast. This rapidly growing technology promises to enable multiscale biological research and accelerate translation from microscopic laboratory discoveries to macroscopic clinical practice. PAT may also hold the key to label-free early detection of cancer by in vivo quantification of hypermetabolism, the quintessential hallmark of malignancy. Learning Objectives: To understand the contrast mechanism of PAT To understand the multiscale applications of PAT Benjamin M. W. Tsui: Multi-modality molecular imaging instrumentation and techniques have been major developments in small animal imaging that has contributed significantly to biomedical research during the past decade. The initial development was an extension of clinical PET/CT and SPECT/CT from human to small animals and combine the unique functional information obtained from PET and SPECT with anatomical information provided by the CT in registered multi-modality images. The requirements to image a mouse whose size is an order of magnitude smaller than that of a human have spurred advances in new radiation detector technologies, novel imaging system designs and special image reconstruction and processing techniques. Examples are new detector materials and designs with high intrinsic resolution, multi-pinhole (MPH) collimator design for much improved resolution and detection efficiency compared to the conventional collimator designs in SPECT, 3D high-resolution and artifact-free MPH and sparse-view image reconstruction techniques, and iterative image reconstruction methods with system response modeling for resolution recovery and image noise reduction for much improved image quality. The spatial resolution of PET and SPECT has improved from ∼6–12 mm to ∼1 mm a few years ago to sub-millimeter today. A recent commercial small animal SPECT system has achieved a resolution of ∼0.25 mm which surpasses that of a state-of-art PET system whose resolution is limited by the positron range. More recently, multimodality SA PET/MRI and SPECT/MRI systems have been developed in research laboratories. Also, multi-modality SA imaging systems that include other imaging modalities such as optical and ultrasound are being actively pursued. In this presentation, we will provide a review of the development, recent advances and future outlook of multi-modality molecular imaging of small animals. Learning Objectives: To learn about the two major multi-modality molecular imaging techniques of small animals. To learn about the spatial resolution achievable by the molecular imaging systems for small animal today. To learn about the new multi-modality imaging instrumentation and techniques that are being developed. Sang Hyun Cho; X-ray fluorescence (XRF) imaging, such as x-ray fluorescence computed tomography (XFCT), offers unique capabilities for accurate identification and quantification of metals within the imaging objects. As a result, it has emerged as a promising quantitative imaging modality in recent years, especially in conjunction with metal-based imaging probes. This talk will familiarize the audience with the basic principles of XRF/XFCT imaging. It will also cover the latest development of benchtop XFCT technology. Additionally, the use of metallic nanoparticles such as gold nanoparticles, in conjunction with benchtop XFCT, will be discussed within the context of preclinical multimodal multiplexed molecular imaging. Learning Objectives: To learn the basic principles of XRF/XFCT imaging To learn the latest advances in benchtop XFCT development for preclinical imaging Funding support received from NIH and DOD; Funding support received from GE Healthcare; Funding support received from Siemens AX; Patent royalties received from GE Healthcare; L. Wang, Funding Support: NIH; COI: Microphotoacoustics; S. Cho, Yes: ;NIH/NCI grant R01CA155446 DOD/PCRP grant W81XWH-12-1-0198.« less
WE-H-206-00: Advances in Preclinical Imaging

DOE Office of Scientific and Technical Information (OSTI.GOV)

NONE

Lihong V. Wang: Photoacoustic tomography (PAT), combining non-ionizing optical and ultrasonic waves via the photoacoustic effect, provides in vivo multiscale functional, metabolic, and molecular imaging. Broad applications include imaging of the breast, brain, skin, esophagus, colon, vascular system, and lymphatic system in humans or animals. Light offers rich contrast but does not penetrate biological tissue in straight paths as x-rays do. Consequently, high-resolution pure optical imaging (e.g., confocal microscopy, two-photon microscopy, and optical coherence tomography) is limited to penetration within the optical diffusion limit (∼1 mm in the skin). Ultrasonic imaging, on the contrary, provides fine spatial resolution but suffersmore » from both poor contrast in early-stage tumors and strong speckle artifacts. In PAT, pulsed laser light penetrates tissue and generates a small but rapid temperature rise, which induces emission of ultrasonic waves due to thermoelastic expansion. The ultrasonic waves, orders of magnitude less scattering than optical waves, are then detected to form high-resolution images of optical absorption at depths up to 7 cm, conquering the optical diffusion limit. PAT is the only modality capable of imaging across the length scales of organelles, cells, tissues, and organs (up to whole-body small animals) with consistent contrast. This rapidly growing technology promises to enable multiscale biological research and accelerate translation from microscopic laboratory discoveries to macroscopic clinical practice. PAT may also hold the key to label-free early detection of cancer by in vivo quantification of hypermetabolism, the quintessential hallmark of malignancy. Learning Objectives: To understand the contrast mechanism of PAT To understand the multiscale applications of PAT Benjamin M. W. Tsui: Multi-modality molecular imaging instrumentation and techniques have been major developments in small animal imaging that has contributed significantly to biomedical research during the past decade. The initial development was an extension of clinical PET/CT and SPECT/CT from human to small animals and combine the unique functional information obtained from PET and SPECT with anatomical information provided by the CT in registered multi-modality images. The requirements to image a mouse whose size is an order of magnitude smaller than that of a human have spurred advances in new radiation detector technologies, novel imaging system designs and special image reconstruction and processing techniques. Examples are new detector materials and designs with high intrinsic resolution, multi-pinhole (MPH) collimator design for much improved resolution and detection efficiency compared to the conventional collimator designs in SPECT, 3D high-resolution and artifact-free MPH and sparse-view image reconstruction techniques, and iterative image reconstruction methods with system response modeling for resolution recovery and image noise reduction for much improved image quality. The spatial resolution of PET and SPECT has improved from ∼6–12 mm to ∼1 mm a few years ago to sub-millimeter today. A recent commercial small animal SPECT system has achieved a resolution of ∼0.25 mm which surpasses that of a state-of-art PET system whose resolution is limited by the positron range. More recently, multimodality SA PET/MRI and SPECT/MRI systems have been developed in research laboratories. Also, multi-modality SA imaging systems that include other imaging modalities such as optical and ultrasound are being actively pursued. In this presentation, we will provide a review of the development, recent advances and future outlook of multi-modality molecular imaging of small animals. Learning Objectives: To learn about the two major multi-modality molecular imaging techniques of small animals. To learn about the spatial resolution achievable by the molecular imaging systems for small animal today. To learn about the new multi-modality imaging instrumentation and techniques that are being developed. Sang Hyun Cho; X-ray fluorescence (XRF) imaging, such as x-ray fluorescence computed tomography (XFCT), offers unique capabilities for accurate identification and quantification of metals within the imaging objects. As a result, it has emerged as a promising quantitative imaging modality in recent years, especially in conjunction with metal-based imaging probes. This talk will familiarize the audience with the basic principles of XRF/XFCT imaging. It will also cover the latest development of benchtop XFCT technology. Additionally, the use of metallic nanoparticles such as gold nanoparticles, in conjunction with benchtop XFCT, will be discussed within the context of preclinical multimodal multiplexed molecular imaging. Learning Objectives: To learn the basic principles of XRF/XFCT imaging To learn the latest advances in benchtop XFCT development for preclinical imaging Funding support received from NIH and DOD; Funding support received from GE Healthcare; Funding support received from Siemens AX; Patent royalties received from GE Healthcare; L. Wang, Funding Support: NIH; COI: Microphotoacoustics; S. Cho, Yes: ;NIH/NCI grant R01CA155446 DOD/PCRP grant W81XWH-12-1-0198.« less
Targeted delivery of cancer-specific multimodal contrast agents for intraoperative detection of tumor boundaries and therapeutic margins

NASA Astrophysics Data System (ADS)

Xu, Ronald X.; Xu, Jeff S.; Huang, Jiwei; Tweedle, Michael F.; Schmidt, Carl; Povoski, Stephen P.; Martin, Edward W.

2010-02-01

Background: Accurate assessment of tumor boundaries and intraoperative detection of therapeutic margins are important oncologic principles for minimal recurrence rates and improved long-term outcomes. However, many existing cancer imaging tools are based on preoperative image acquisition and do not provide real-time intraoperative information that supports critical decision-making in the operating room. Method: Poly lactic-co-glycolic acid (PLGA) microbubbles (MBs) and nanobubbles (NBs) were synthesized by a modified double emulsion method. The MB and NB surfaces were conjugated with CC49 antibody to target TAG-72 antigen, a human glycoprotein complex expressed in many epithelial-derived cancers. Multiple imaging agents were encapsulated in MBs and NBs for multimodal imaging. Both one-step and multi-step cancer targeting strategies were explored. Active MBs/NBs were also fabricated for therapeutic margin assessment in cancer ablation therapies. Results: The multimodal contrast agents and the cancer-targeting strategies were tested on tissue simulating phantoms, LS174 colon cancer cell cultures, and cancer xenograft nude mice. Concurrent multimodal imaging was demonstrated using fluorescence and ultrasound imaging modalities. Technical feasibility of using active MBs and portable imaging tools such as ultrasound for intraoperative therapeutic margin assessment was demonstrated in a biological tissue model. Conclusion: The cancer-specific multimodal contrast agents described in this paper have the potential for intraoperative detection of tumor boundaries and therapeutic margins.
DETECTION OF TYPE 1 CHOROIDAL NEOVASCULAR MEMBRANES USING OPTICAL COHERENCE TOMOGRAPHY ANGIOGRAPHY IN TUBERCULAR POSTERIOR UVEITIS.

PubMed

Aggarwal, Kanika; Agarwal, Aniruddha; Sharma, Aman; Sharma, Kusum; Gupta, Vishali

2018-04-23

To study optical coherence tomography angiography (OCTA) and multimodal imaging features of Type 1 inflammatory choroidal neovascularization (CNV) in tubercular serpiginous-like choroiditis and response to anti-vascular endothelial growth factor therapy. In this study, multimodal imaging was performed using OCTA, enhanced-depth imaging optical coherence tomography, fluorescein angiography, and indocyanine green angiography. Correlation of OCTA with other imaging modalities in the detection of CNV was performed. The changes in CNV configuration after anti-vascular endothelial growth factor therapy were assessed. In this study, nine eyes (8 patients; 5 females; mean age: 32.5 ± 11.57 years) with diagnosis of tubercular serpiginous-like choroiditis were included. All the eyes had presence of low-lying pigment epithelial detachments on enhanced-depth imaging optical coherence tomography. Using OCTA, it was possible to detect Type 1 CNV in all eyes. Type 1 CNV networks comprised fine anastomotic network of vessels, some of which had a hairpin loop configuration. After anti-vascular endothelial growth factor therapy, there was a decrease in branching and anastomosis. The visual acuity significantly improved from 0.49 ± 0.26 (20/60 Snellen equivalent) at baseline to 0.26 ± 0.17 (20/36 Snellen equivalent) (P = 0.03) in all eyes. Type 1 CNV can occur among patients with tubercular serpiginous-like choroiditis, leading to significant visual loss even in the healed stages of the disease. Optical coherence tomography angiography can help in the detection of Type 1 CNV where conventional multimodal imaging, including fluorescein angiography and OCT, fails to make a definitive diagnosis and thereby guide the initiation of anti-vascular endothelial growth factor therapy.
Integrative, multimodal analysis of glioblastoma using TCGA molecular data, pathology images, and clinical outcomes.

PubMed

Kong, Jun; Cooper, Lee A D; Wang, Fusheng; Gutman, David A; Gao, Jingjing; Chisolm, Candace; Sharma, Ashish; Pan, Tony; Van Meir, Erwin G; Kurc, Tahsin M; Moreno, Carlos S; Saltz, Joel H; Brat, Daniel J

2011-12-01

Multimodal, multiscale data synthesis is becoming increasingly critical for successful translational biomedical research. In this letter, we present a large-scale investigative initiative on glioblastoma, a high-grade brain tumor, with complementary data types using in silico approaches. We integrate and analyze data from The Cancer Genome Atlas Project on glioblastoma that includes novel nuclear phenotypic data derived from microscopic slides, genotypic signatures described by transcriptional class and genetic alterations, and clinical outcomes defined by response to therapy and patient survival. Our preliminary results demonstrate numerous clinically and biologically significant correlations across multiple data types, revealing the power of in silico multimodal data integration for cancer research.
Carbon Tube Electrodes for Electrocardiography-Gated Cardiac Multimodality Imaging in Mice

PubMed Central

Choquet, Philippe; Goetz, Christian; Aubertin, Gaelle; Hubele, Fabrice; Sannié, Sébastien; Constantinesco, André

2011-01-01

This report describes a simple design of noninvasive carbon tube electrodes that facilitates electrocardiography (ECG) in mice during cardiac multimodality preclinical imaging. Both forepaws and the left hindpaw, covered by conductive gel, of mice were placed into the openings of small carbon tubes. Cardiac ECG-gated single-photon emission CT, X-ray CT, and MRI were tested (n = 60) in 20 mice. For all applications, electrodes were used in a warmed multimodality imaging cell. A heart rate of 563 ± 48 bpm was recorded from anesthetized mice regardless of the imaging technique used, with acquisition times ranging from 1 to 2 h. PMID:21333165
Development of a Multi-modal Tissue Diagnostic System Combining High Frequency Ultrasound and Photoacoustic Imaging with Lifetime Fluorescence Spectroscopy

PubMed Central

Sun, Yang; Stephens, Douglas N.; Park, Jesung; Sun, Yinghua; Marcu, Laura; Cannata, Jonathan M.; Shung, K. Kirk

2010-01-01

We report the development and validate a multi-modal tissue diagnostic technology, which combines three complementary techniques into one system including ultrasound backscatter microscopy (UBM), photoacoustic imaging (PAI), and time-resolved laser-induced fluorescence spectroscopy (TR-LIFS). UBM enables the reconstruction of the tissue microanatomy. PAI maps the optical absorption heterogeneity of the tissue associated with structure information and has the potential to provide functional imaging of the tissue. Examination of the UBM and PAI images allows for localization of regions of interest for TR-LIFS evaluation of the tissue composition. The hybrid probe consists of a single element ring transducer with concentric fiber optics for multi-modal data acquisition. Validation and characterization of the multi-modal system and ultrasonic, photoacoustic, and spectroscopic data coregistration were conducted in a physical phantom with properties of ultrasound scattering, optical absorption, and fluorescence. The UBM system with the 41 MHz ring transducer can reach the axial and lateral resolution of 30 and 65 μm, respectively. The PAI system with 532 nm excitation light from a Nd:YAG laser shows great contrast for the distribution of optical absorbers. The TR-LIFS system records the fluorescence decay with the time resolution of ~300 ps and a high sensitivity of nM concentration range. Biological phantom constructed with different types of tissues (tendon and fat) was used to demonstrate the complementary information provided by the three modalities. Fluorescence spectra and lifetimes were compared to differentiate chemical composition of tissues at the regions of interest determined by the coregistered high resolution UBM and PAI image. Current results demonstrate that the fusion of these techniques enables sequentially detection of functional, morphological, and compositional features of biological tissue, suggesting potential applications in diagnosis of tumors and atherosclerotic plaques. PMID:21894259
Development of a Multi-modal Tissue Diagnostic System Combining High Frequency Ultrasound and Photoacoustic Imaging with Lifetime Fluorescence Spectroscopy.

PubMed

Sun, Yang; Stephens, Douglas N; Park, Jesung; Sun, Yinghua; Marcu, Laura; Cannata, Jonathan M; Shung, K Kirk

2008-01-01

We report the development and validate a multi-modal tissue diagnostic technology, which combines three complementary techniques into one system including ultrasound backscatter microscopy (UBM), photoacoustic imaging (PAI), and time-resolved laser-induced fluorescence spectroscopy (TR-LIFS). UBM enables the reconstruction of the tissue microanatomy. PAI maps the optical absorption heterogeneity of the tissue associated with structure information and has the potential to provide functional imaging of the tissue. Examination of the UBM and PAI images allows for localization of regions of interest for TR-LIFS evaluation of the tissue composition. The hybrid probe consists of a single element ring transducer with concentric fiber optics for multi-modal data acquisition. Validation and characterization of the multi-modal system and ultrasonic, photoacoustic, and spectroscopic data coregistration were conducted in a physical phantom with properties of ultrasound scattering, optical absorption, and fluorescence. The UBM system with the 41 MHz ring transducer can reach the axial and lateral resolution of 30 and 65 μm, respectively. The PAI system with 532 nm excitation light from a Nd:YAG laser shows great contrast for the distribution of optical absorbers. The TR-LIFS system records the fluorescence decay with the time resolution of ~300 ps and a high sensitivity of nM concentration range. Biological phantom constructed with different types of tissues (tendon and fat) was used to demonstrate the complementary information provided by the three modalities. Fluorescence spectra and lifetimes were compared to differentiate chemical composition of tissues at the regions of interest determined by the coregistered high resolution UBM and PAI image. Current results demonstrate that the fusion of these techniques enables sequentially detection of functional, morphological, and compositional features of biological tissue, suggesting potential applications in diagnosis of tumors and atherosclerotic plaques.
IDH mutation assessment of glioma using texture features of multimodal MR images

NASA Astrophysics Data System (ADS)

Zhang, Xi; Tian, Qiang; Wu, Yu-Xia; Xu, Xiao-Pan; Li, Bao-Juan; Liu, Yi-Xiong; Liu, Yang; Lu, Hong-Bing

2017-03-01

Purpose: To 1) find effective texture features from multimodal MRI that can distinguish IDH mutant and wild status, and 2) propose a radiomic strategy for preoperatively detecting IDH mutation patients with glioma. Materials and Methods: 152 patients with glioma were retrospectively included from the Cancer Genome Atlas. Corresponding T1-weighted image before- and post-contrast, T2-weighted image and fluid-attenuation inversion recovery image from the Cancer Imaging Archive were analyzed. Specific statistical tests were applied to analyze the different kind of baseline information of LrGG patients. Finally, 168 texture features were derived from multimodal MRI per patient. Then the support vector machine-based recursive feature elimination (SVM-RFE) and classification strategy was adopted to find the optimal feature subset and build the identification models for detecting the IDH mutation. Results: Among 152 patients, 92 and 60 were confirmed to be IDH-wild and mutant, respectively. Statistical analysis showed that the patients without IDH mutation was significant older than patients with IDH mutation (p<0.01), and the distribution of some histological subtypes was significant different between IDH wild and mutant groups (p<0.01). After SVM-RFE, 15 optimal features were determined for IDH mutation detection. The accuracy, sensitivity, specificity, and AUC after SVM-RFE and parameter optimization were 82.2%, 85.0%, 78.3%, and 0.841, respectively. Conclusion: This study presented a radiomic strategy for noninvasively discriminating IDH mutation of patients with glioma. It effectively incorporated kinds of texture features from multimodal MRI, and SVM-based classification strategy. Results suggested that features selected from SVM-RFE were more potential to identifying IDH mutation. The proposed radiomics strategy could facilitate the clinical decision making in patients with glioma.
GRAFT-VERSUS-HOST DISEASE PANUVEITIS AND BILATERAL SEROUS DETACHMENTS: MULTIMODAL IMAGING ANALYSIS.

PubMed

Jung, Jesse J; Chen, Michael H; Rofagha, Soraya; Lee, Scott S

2017-01-01

To report the multimodal imaging findings and follow-up of a case of graft-versus-host disease-induced bilateral panuveitis and serous retinal detachments after allogenic bone marrow transplant for acute myeloid leukemia. A 75-year-old black man presented with acute decreased vision in both eyes for 1 week. Clinical examination and multimodal imaging, including spectral domain optical coherence tomography, fundus autofluorescence, fluorescein angiography, and swept-source optical coherence tomography angiography (Investigational Device; Carl Zeiss Meditec Inc) were performed. Clinical examination of the patient revealed anterior and posterior inflammation and bilateral serous retinal detachments. Ultra-widefield fundus autofluorescence demonstrated hyperautofluorescence secondary to subretinal fluid; and fluorescein angiography revealed multiple areas of punctate hyperfluorescence, leakage, and staining of the optic discs. Spectral domain and enhanced depth imaging optical coherence tomography demonstrated subretinal fluid, a thickened, undulating retinal pigment epithelium layer, and a thickened choroid in both eyes. En-face swept-source optical coherence tomography angiography did not show any retinal vascular abnormalities but did demonstrate patchy areas of decreased choriocapillaris flow. An extensive systemic infectious and malignancy workup was negative and the patient was treated with high-dose oral prednisone immunosuppression. Subsequent 6-month follow-up demonstrated complete resolution of the inflammation and bilateral serous detachments after completion of the prednisone taper over a 3-month period. Graft-versus-host disease panuveitis and bilateral serous retinal detachments are rare complications of allogenic bone marrow transplant for acute myeloid leukemia and can be diagnosed with clinical and multimodal imaging analysis. This form of autoimmune inflammation may occur after the recovery of T-cell activity within the donor graft targeting the host. Infectious and recurrent malignancy must be ruled out before initiation of immunosuppression, which can affectively treat this form of graft-versus-host disease.
Resolution and throughput optimized intraoperative spectrally encoded coherence tomography and reflectometry (iSECTR) for multimodal imaging during ophthalmic microsurgery

NASA Astrophysics Data System (ADS)

Malone, Joseph D.; El-Haddad, Mohamed T.; Leeburg, Kelsey C.; Terrones, Benjamin D.; Tao, Yuankai K.

2018-02-01

Limited visualization of semi-transparent structures in the eye remains a critical barrier to improving clinical outcomes and developing novel surgical techniques. While increases in imaging speed has enabled intraoperative optical coherence tomography (iOCT) imaging of surgical dynamics, several critical barriers to clinical adoption remain. Specifically, these include (1) static field-of-views (FOVs) requiring manual instrument-tracking; (2) high frame-rates require sparse sampling, which limits FOV; and (3) small iOCT FOV also limits the ability to co-register data with surgical microscopy. We previously addressed these limitations in image-guided ophthalmic microsurgery by developing microscope-integrated multimodal intraoperative swept-source spectrally encoded scanning laser ophthalmoscopy and optical coherence tomography. Complementary en face images enabled orientation and coregistration with the widefield surgical microscope view while OCT imaging enabled depth-resolved visualization of surgical instrument positions relative to anatomic structures-of-interest. In addition, we demonstrated novel integrated segmentation overlays for augmented-reality surgical guidance. Unfortunately, our previous system lacked the resolution and optical throughput for in vivo retinal imaging and necessitated removal of cornea and lens. These limitations were predominately a result of optical aberrations from imaging through a shared surgical microscope objective lens, which was modeled as a paraxial surface. Here, we present an optimized intraoperative spectrally encoded coherence tomography and reflectometry (iSECTR) system. We use a novel lens characterization method to develop an accurate model of surgical microscope objective performance and balance out inherent aberrations using iSECTR relay optics. Using this system, we demonstrate in vivo multimodal ophthalmic imaging through a surgical microscope
Multi-modality imaging of tumor phenotype and response to therapy

NASA Astrophysics Data System (ADS)

Nyflot, Matthew J.

2011-12-01

Imaging and radiation oncology have historically been closely linked. However, the vast majority of techniques used in the clinic involve anatomical imaging. Biological imaging offers the potential for innovation in the areas of cancer diagnosis and staging, radiotherapy target definition, and treatment response assessment. Some relevant imaging techniques are FDG PET (for imaging cellular metabolism), FLT PET (proliferation), CuATSM PET (hypoxia), and contrast-enhanced CT (vasculature and perfusion). Here, a technique for quantitative spatial correlation of tumor phenotype is presented for FDG PET, FLT PET, and CuATSM PET images. Additionally, multimodality imaging of treatment response with FLT PET, CuATSM, and dynamic contrast-enhanced CT is presented, in a trial of patients receiving an antiangiogenic agent (Avastin) combined with cisplatin and radiotherapy. Results are also presented for translational applications in animal models, including quantitative assessment of proliferative response to cetuximab with FLT PET and quantification of vascular volume with a blood-pool contrast agent (Fenestra). These techniques have clear applications to radiobiological research and optimized treatment strategies, and may eventually be used for personalized therapy for patients.
Volume curtaining: a focus+context effect for multimodal volume visualization

NASA Astrophysics Data System (ADS)

Fairfield, Adam J.; Plasencia, Jonathan; Jang, Yun; Theodore, Nicholas; Crawford, Neil R.; Frakes, David H.; Maciejewski, Ross

2014-03-01

In surgical preparation, physicians will often utilize multimodal imaging scans to capture complementary information to improve diagnosis and to drive patient-specific treatment. These imaging scans may consist of data from magnetic resonance imaging (MR), computed tomography (CT), or other various sources. The challenge in using these different modalities is that the physician must mentally map the two modalities together during the diagnosis and planning phase. Furthermore, the different imaging modalities will be generated at various resolutions as well as slightly different orientations due to patient placement during scans. In this work, we present an interactive system for multimodal data fusion, analysis and visualization. Developed with partners from neurological clinics, this work discusses initial system requirements and physician feedback at the various stages of component development. Finally, we present a novel focus+context technique for the interactive exploration of coregistered multi-modal data.
Quantitative multimodality imaging in cancer research and therapy.

PubMed

Yankeelov, Thomas E; Abramson, Richard G; Quarles, C Chad

2014-11-01

Advances in hardware and software have enabled the realization of clinically feasible, quantitative multimodality imaging of tissue pathophysiology. Earlier efforts relating to multimodality imaging of cancer have focused on the integration of anatomical and functional characteristics, such as PET-CT and single-photon emission CT (SPECT-CT), whereas more-recent advances and applications have involved the integration of multiple quantitative, functional measurements (for example, multiple PET tracers, varied MRI contrast mechanisms, and PET-MRI), thereby providing a more-comprehensive characterization of the tumour phenotype. The enormous amount of complementary quantitative data generated by such studies is beginning to offer unique insights into opportunities to optimize care for individual patients. Although important technical optimization and improved biological interpretation of multimodality imaging findings are needed, this approach can already be applied informatively in clinical trials of cancer therapeutics using existing tools. These concepts are discussed herein.

Multimode intravascular RF coil for MRI-guided interventions.

PubMed

Kurpad, Krishna N; Unal, Orhan

2011-04-01

To demonstrate the feasibility of using a single intravascular radiofrequency (RF) probe connected to the external magnetic resonance imaging (MRI) system via a single coaxial cable to perform active tip tracking and catheter visualization and high signal-to-noise ratio (SNR) intravascular imaging. A multimode intravascular RF coil was constructed on a 6F balloon catheter and interfaced to a 1.5T MRI scanner via a decoupling circuit. Bench measurements of coil impedances were followed by imaging experiments in saline and phantoms. The multimode coil behaves as an inductively coupled transmit coil. The forward-looking capability of 6 mm was measured. A greater than 3-fold increase in SNR compared to conventional imaging using optimized external coil was demonstrated. Simultaneous active tip tracking and catheter visualization was demonstrated. It is feasible to perform 1) active tip tracking, 2) catheter visualization, and 3) high SNR imaging using a single multimode intravascular RF coil that is connected to the external system via a single coaxial cable. Copyright © 2011 Wiley-Liss, Inc.
Log-Gabor Energy Based Multimodal Medical Image Fusion in NSCT Domain

PubMed Central

Yang, Yong; Tong, Song; Huang, Shuying; Lin, Pan

2014-01-01

Multimodal medical image fusion is a powerful tool in clinical applications such as noninvasive diagnosis, image-guided radiotherapy, and treatment planning. In this paper, a novel nonsubsampled Contourlet transform (NSCT) based method for multimodal medical image fusion is presented, which is approximately shift invariant and can effectively suppress the pseudo-Gibbs phenomena. The source medical images are initially transformed by NSCT followed by fusing low- and high-frequency components. The phase congruency that can provide a contrast and brightness-invariant representation is applied to fuse low-frequency coefficients, whereas the Log-Gabor energy that can efficiently determine the frequency coefficients from the clear and detail parts is employed to fuse the high-frequency coefficients. The proposed fusion method has been compared with the discrete wavelet transform (DWT), the fast discrete curvelet transform (FDCT), and the dual tree complex wavelet transform (DTCWT) based image fusion methods and other NSCT-based methods. Visually and quantitatively experimental results indicate that the proposed fusion method can obtain more effective and accurate fusion results of multimodal medical images than other algorithms. Further, the applicability of the proposed method has been testified by carrying out a clinical example on a woman affected with recurrent tumor images. PMID:25214889
Enhanced EDX images by fusion of multimodal SEM images using pansharpening techniques.

PubMed

Franchi, G; Angulo, J; Moreaud, M; Sorbier, L

2018-01-01

The goal of this paper is to explore the potential interest of image fusion in the context of multimodal scanning electron microscope (SEM) imaging. In particular, we aim at merging the backscattered electron images that usually have a high spatial resolution but do not provide enough discriminative information to physically classify the nature of the sample, with energy-dispersive X-ray spectroscopy (EDX) images that have discriminative information but a lower spatial resolution. The produced images are named enhanced EDX. To achieve this goal, we have compared the results obtained with classical pansharpening techniques for image fusion with an original approach tailored for multimodal SEM fusion of information. Quantitative assessment is obtained by means of two SEM images and a simulated dataset produced by a software based on PENELOPE. © 2017 The Authors Journal of Microscopy © 2017 Royal Microscopical Society.
Multi-mode of Four and Six Wave Parametric Amplified Process

NASA Astrophysics Data System (ADS)

Zhu, Dayu; Yang, Yiheng; Zhang, Da; Liu, Ruizhou; Ma, Danmeng; Li, Changbiao; Zhang, Yanpeng

2017-03-01

Multiple quantum modes in correlated fields are essential for future quantum information processing and quantum computing. Here we report the generation of multi-mode phenomenon through parametric amplified four- and six-wave mixing processes in a rubidium atomic ensemble. The multi-mode properties in both frequency and spatial domains are studied. On one hand, the multi-mode behavior is dominantly controlled by the intensity of external dressing effect, or nonlinear phase shift through internal dressing effect, in frequency domain; on the other hand, the multi-mode behavior is visually demonstrated from the images of the biphoton fields directly, in spatial domain. Besides, the correlation of the two output fields is also demonstrated in both domains. Our approach supports efficient applications for scalable quantum correlated imaging.
Multi-mode of Four and Six Wave Parametric Amplified Process.

PubMed

Zhu, Dayu; Yang, Yiheng; Zhang, Da; Liu, Ruizhou; Ma, Danmeng; Li, Changbiao; Zhang, Yanpeng

2017-03-03

Multiple quantum modes in correlated fields are essential for future quantum information processing and quantum computing. Here we report the generation of multi-mode phenomenon through parametric amplified four- and six-wave mixing processes in a rubidium atomic ensemble. The multi-mode properties in both frequency and spatial domains are studied. On one hand, the multi-mode behavior is dominantly controlled by the intensity of external dressing effect, or nonlinear phase shift through internal dressing effect, in frequency domain; on the other hand, the multi-mode behavior is visually demonstrated from the images of the biphoton fields directly, in spatial domain. Besides, the correlation of the two output fields is also demonstrated in both domains. Our approach supports efficient applications for scalable quantum correlated imaging.
STRUCTURAL AND FUNCTIONAL CHARACTERIZATION OF BENIGN FLECK RETINA USING MULTIMODAL IMAGING.

PubMed

Neriyanuri, Srividya; Rao, Chetan; Raman, Rajiv

2017-01-01

To report structural and functional features in a case series of benign fleck retina using multimodal imaging. Four cases with benign fleck retina underwent complete ophthalmic examination that included detailed history, visual acuity, and refractive error testing, FM-100 hue test, dilated fundus evaluation, full field electroretinogram, fundus photography with autofluorescence, fundus fluorescein angiography, and swept-source optical coherence tomography. Age group of the cases ranged from 19 years to 35 years (3 males and 1 female). Parental consanguinity was reported in two cases. All of them were visually asymptomatic with best-corrected visual acuity of 20/20 (moderate astigmatism) in both the eyes. Low color discrimination was seen in two cases. Fundus photography showed pisciform flecks which were compactly placed on posterior pole and were discrete, diverging towards periphery. Lesions were seen as smaller dots within 1500 microns from fovea and were hyperfluorescent on autofluorescence. Palisading retinal pigment epithelium defects were seen in posterior pole on fundus fluorescein angiography imaging; irregular hyper fluorescence was also noted. One case had reduced cone responses on full field electroretinogram; the other three cases had normal electroretinogram. On optical coherence tomography, level of lesions varied from retinal pigment epithelium, inner segment to outer segment extending till external limiting membrane. Functional and structural deficits in benign fleck retina were picked up using multimodal imaging.
Fractals, Fuzzy Sets And Image Representation

NASA Astrophysics Data System (ADS)

Dodds, D. R.

1988-10-01

This paper addresses some uses of fractals, fuzzy sets and image representation as it pertains to robotic grip planning and autonomous vehicle navigation AVN. The robot/vehicle is assumed to be equipped with multimodal sensors including ultrashort pulse imaging laser rangefinder. With a temporal resolution of 50 femtoseconds a time of flight laser rangefinder can resolve distances within approximately half an inch or 1.25 centimeters. (Fujimoto88)
Multimodal optoacoustic and multiphoton fluorescence microscopy

NASA Astrophysics Data System (ADS)

Sela, Gali; Razansky, Daniel; Shoham, Shy

2013-03-01

Multiphoton microscopy is a powerful imaging modality that enables structural and functional imaging with cellular and sub-cellular resolution, deep within biological tissues. Yet, its main contrast mechanism relies on extrinsically administered fluorescent indicators. Here we developed a system for simultaneous multimodal optoacoustic and multiphoton fluorescence 3D imaging, which attains both absorption and fluorescence-based contrast by integrating an ultrasonic transducer into a two-photon laser scanning microscope. The system is readily shown to enable acquisition of multimodal microscopic images of fluorescently labeled targets and cell cultures as well as intrinsic absorption-based images of pigmented biological tissue. During initial experiments, it was further observed that that detected optoacoustically-induced response contains low frequency signal variations, presumably due to cavitation-mediated signal generation by the high repetition rate (80MHz) near IR femtosecond laser. The multimodal system may provide complementary structural and functional information to the fluorescently labeled tissue, by superimposing optoacoustic images of intrinsic tissue chromophores, such as melanin deposits, pigmentation, and hemoglobin or other extrinsic particle or dye-based markers highly absorptive in the NIR spectrum.
Improved heuristics for early melanoma detection using multimode hyperspectral dermoscopy (Conference Presentation)

NASA Astrophysics Data System (ADS)

Vasefi, Fartash; MacKinnon, Nicholas B.; Booth, Nicholas; Farkas, Daniel L.

2017-02-01

Purpose: To determine the performance of a multimode dermoscopy system (SkinSpect) designed to quantify and 3-D map in vivo melanin and hemoglobin concentrations in skin and its melanoma scoring system, and compare the results accuracy with SIAscopy, and histopathology. Methods: A multimode imaging dermoscope is presented that combines polarization, fluorescence and hyperspectral imaging to accurately map the distribution of skin melanin, collagen and hemoglobin in pigmented lesions. We combine two depth-sensitive techniques: polarization, and hyperspectral imaging, to determine the spatial distribution of melanin and hemoglobin oxygenation in a skin lesion. By quantifying melanin absorption in pigmented areas, we can also more accurately estimate fluorescence emission distribution mainly from skin collagen. Results and discussion: We compared in vivo features of melanocytic lesions (N = 10) extracted by non-invasive SkinSpect and SIMSYS-MoleMate SIAscope, and correlate them to pathology report. Melanin distribution at different depths as well as hemodynamics including abnormal vascularity we detected will be discussed. We will adapt SkinSpect scoring with ABCDE (asymmetry , border, color, diameter, evolution) and seven point dermatologic checklist including: (1) atypical pigment network, (2) blue-whitish veil, (3) atypical vascular pattern, (4) irregular streaks, (5) irregular pigmentation, (6) irregular dots and globules, (7) regression structures estimated by dermatologist. Conclusion: Distinctive, diagnostic features seen by SkinSpect in melanoma vs. normal pigmented lesions will be compared by SIAscopy and results from histopathology.
Multimode C-arm fluoroscopy, tomosynthesis, and cone-beam CT for image-guided interventions: from proof of principle to patient protocols

NASA Astrophysics Data System (ADS)

Siewerdsen, J. H.; Daly, M. J.; Bachar, G.; Moseley, D. J.; Bootsma, G.; Brock, K. K.; Ansell, S.; Wilson, G. A.; Chhabra, S.; Jaffray, D. A.; Irish, J. C.

2007-03-01

High-performance intraoperative imaging is essential to an ever-expanding scope of therapeutic procedures ranging from tumor surgery to interventional radiology. The need for precise visualization of bony and soft-tissue structures with minimal obstruction to the therapy setup presents challenges and opportunities in the development of novel imaging technologies specifically for image-guided procedures. Over the past ~5 years, a mobile C-arm has been modified in collaboration with Siemens Medical Solutions for 3D imaging. Based upon a Siemens PowerMobil, the device includes: a flat-panel detector (Varian PaxScan 4030CB); a motorized orbit; a system for geometric calibration; integration with real-time tracking and navigation (NDI Polaris); and a computer control system for multi-mode fluoroscopy, tomosynthesis, and cone-beam CT. Investigation of 3D imaging performance (noise-equivalent quanta), image quality (human observer studies), and image artifacts (scatter, truncation, and cone-beam artifacts) has driven the development of imaging techniques appropriate to a host of image-guided interventions. Multi-mode functionality presents a valuable spectrum of acquisition techniques: i.) fluoroscopy for real-time 2D guidance; ii.) limited-angle tomosynthesis for fast 3D imaging (e.g., ~10 sec acquisition of coronal slices containing the surgical target); and iii.) fully 3D cone-beam CT (e.g., ~30-60 sec acquisition providing bony and soft-tissue visualization across the field of view). Phantom and cadaver studies clearly indicate the potential for improved surgical performance - up to a factor of 2 increase in challenging surgical target excisions. The C-arm system is currently being deployed in patient protocols ranging from brachytherapy to chest, breast, spine, and head and neck surgery.
Highlights lecture EANM 2016: "Embracing molecular imaging and multi-modal imaging: a smart move for nuclear medicine towards personalized medicine".

PubMed

Aboagye, Eric O; Kraeber-Bodéré, Françoise

2017-08-01

The 2016 EANM Congress took place in Barcelona, Spain, from 15 to 19 October under the leadership of Prof. Wim Oyen, chair of the EANM Scientific Committee. With more than 6,000 participants, this congress was the most important European event in nuclear medicine, bringing together a multidisciplinary community involved in the different fields of nuclear medicine. There were over 600 oral and 1,200 poster or e-Poster presentations with an overwhelming focus on development and application of imaging for personalized care, which is timely for the community. Beyond FDG PET, major highlights included progress in the use of PSMA and SSTR receptor-targeted radiopharmaceuticals and associated theranostics in oncology. Innovations in radiopharmaceuticals for imaging pathologies of the brain and cardiovascular system, as well as infection and inflammation, were also highlighted. In the areas of physics and instrumentation, multimodality imaging and radiomics were highlighted as promising areas of research.
Multimodal swept-source spectrally encoded scanning laser ophthalmoscopy and optical coherence tomography at 400 kHz

NASA Astrophysics Data System (ADS)

El-Haddad, Mohamed T.; Joos, Karen M.; Patel, Shriji N.; Tao, Yuankai K.

2017-02-01

Multimodal imaging systems that combine scanning laser ophthalmoscopy (SLO) and optical coherence tomography (OCT) have demonstrated the utility of concurrent en face and volumetric imaging for aiming, eye tracking, bulk motion compensation, mosaicking, and contrast enhancement. However, this additional functionality trades off with increased system complexity and cost because both SLO and OCT generally require dedicated light sources, galvanometer scanners, relay and imaging optics, detectors, and control and digitization electronics. We previously demonstrated multimodal ophthalmic imaging using swept-source spectrally encoded SLO and OCT (SS-SESLO-OCT). Here, we present system enhancements and a new optical design that increase our SS-SESLO-OCT data throughput by >7x and field-of-view (FOV) by >4x. A 200 kHz 1060 nm Axsun swept-source was optically buffered to 400 kHz sweep-rate, and SESLO and OCT were simultaneously digitized on dual input channels of a 4 GS/s digitizer at 1.2 GS/s per channel using a custom k-clock. We show in vivo human imaging of the anterior segment out to the limbus and retinal fundus over a >40° FOV. In addition, nine overlapping volumetric SS-SESLO-OCT volumes were acquired under video-rate SESLO preview and guidance. In post-processing, all nine SESLO images and en face projections of the corresponding OCT volumes were mosaicked to show widefield multimodal fundus imaging with a >80° FOV. Concurrent multimodal SS-SESLO-OCT may have applications in clinical diagnostic imaging by enabling aiming, image registration, and multi-field mosaicking and benefit intraoperative imaging by allowing for real-time surgical feedback, instrument tracking, and overlays of computationally extracted image-based surrogate biomarkers of disease.
Multimodal MRI for early diabetic mild cognitive impairment: study protocol of a prospective diagnostic trial.

PubMed

Yu, Ying; Sun, Qian; Yan, Lin-Feng; Hu, Yu-Chuan; Nan, Hai-Yan; Yang, Yang; Liu, Zhi-Cheng; Wang, Wen; Cui, Guang-Bin

2016-08-24

Type 2 diabetes mellitus (T2DM) is a risk factor for dementia. Mild cognitive impairment (MCI), an intermediary state between normal cognition and dementia, often occurs during the prodromal diabetic stage, making early diagnosis and intervention of MCI very important. Latest neuroimaging techniques revealed some underlying microstructure alterations for diabetic MCI, from certain aspects. But there still lacks an integrated multimodal MRI system to detect early neuroimaging changes in diabetic MCI patients. Thus, we intended to conduct a diagnostic trial using multimodal MRI techniques to detect early diabetic MCI that is determined by the Montreal Cognitive Assessment (MoCA). In this study, healthy controls, prodromal diabetes and diabetes subjects (53 subjects/group) aged 40-60 years will be recruited from the physical examination center of Tangdu Hospital. The neuroimaging and psychometric measurements will be repeated at a 0.5 year-interval for 2.5 years' follow-up. The primary outcome measures are 1) Microstructural and functional alterations revealed with multimodal MRI scans including structure magnetic resonance imaging (sMRI), resting state functional magnetic resonance imaging (rs-fMRI), diffusion kurtosis imaging (DKI), and three-dimensional pseudo-continuous arterial spin labeling (3D-pCASL); 2) Cognition evaluation with MoCA. The second outcome measures are obesity, metabolic characteristics, lifestyle and quality of life. The study will provide evidence for the potential use of multimodal MRI techniques with psychometric evaluation in diagnosing MCI at prodromal diabetic stage so as to help decision making in early intervention and improve the prognosis of T2DM. This study has been registered to ClinicalTrials.gov ( NCT02420470 ) on April 2, 2015 and published on July 29, 2015.
Cross-modal learning to rank via latent joint representation.

PubMed

Wu, Fei; Jiang, Xinyang; Li, Xi; Tang, Siliang; Lu, Weiming; Zhang, Zhongfei; Zhuang, Yueting

2015-05-01

Cross-modal ranking is a research topic that is imperative to many applications involving multimodal data. Discovering a joint representation for multimodal data and learning a ranking function are essential in order to boost the cross-media retrieval (i.e., image-query-text or text-query-image). In this paper, we propose an approach to discover the latent joint representation of pairs of multimodal data (e.g., pairs of an image query and a text document) via a conditional random field and structural learning in a listwise ranking manner. We call this approach cross-modal learning to rank via latent joint representation (CML²R). In CML²R, the correlations between multimodal data are captured in terms of their sharing hidden variables (e.g., topics), and a hidden-topic-driven discriminative ranking function is learned in a listwise ranking manner. The experiments show that the proposed approach achieves a good performance in cross-media retrieval and meanwhile has the capability to learn the discriminative representation of multimodal data.
Crucial breakthrough of second near-infrared biological window fluorophores: design and synthesis toward multimodal imaging and theranostics

DOE PAGES

He, Shuqing; Song, Jun; Qu, Junle; ...

2018-01-01

Recent advances in the chemical design and synthesis of fluorophores in the second near-infrared biological window (NIR-II) for multimodal imaging and theranostics are summarized and highlighted in this review article.
Crucial breakthrough of second near-infrared biological window fluorophores: design and synthesis toward multimodal imaging and theranostics

DOE Office of Scientific and Technical Information (OSTI.GOV)

He, Shuqing; Song, Jun; Qu, Junle

Recent advances in the chemical design and synthesis of fluorophores in the second near-infrared biological window (NIR-II) for multimodal imaging and theranostics are summarized and highlighted in this review article.
The sweet spot: FDG and other 2-carbon glucose analogs for multi-modal metabolic imaging of tumor metabolism

PubMed Central

Cox, Benjamin L; Mackie, Thomas R; Eliceiri, Kevin W

2015-01-01

Multi-modal imaging approaches of tumor metabolism that provide improved specificity, physiological relevance and spatial resolution would improve diagnosing of tumors and evaluation of tumor progression. Currently, the molecular probe FDG, glucose fluorinated with 18F at the 2-carbon, is the primary metabolic approach for clinical diagnostics with PET imaging. However, PET lacks the resolution necessary to yield intratumoral distributions of deoxyglucose, on the cellular level. Multi-modal imaging could elucidate this problem, but requires the development of new glucose analogs that are better suited for other imaging modalities. Several such analogs have been created and are reviewed here. Also reviewed are several multi-modal imaging studies that have been performed that attempt to shed light on the cellular distribution of glucose analogs within tumors. Some of these studies are performed in vitro, while others are performed in vivo, in an animal model. The results from these studies introduce a visualization gap between the in vitro and in vivo studies that, if solved, could enable the early detection of tumors, the high resolution monitoring of tumors during treatment, and the greater accuracy in assessment of different imaging agents. PMID:25625022
Towards an ultra-thin medical endoscope: multimode fibre as a wide-field image transferring medium

NASA Astrophysics Data System (ADS)

Duriš, Miroslav; Bradu, Adrian; Podoleanu, Adrian; Hughes, Michael

2018-03-01

Multimode optical fibres are attractive for biomedical and industrial applications such as endoscopes because of the small cross section and imaging resolution they can provide in comparison to widely-used fibre bundles. However, the image is randomly scrambled by propagation through a multimode fibre. Even though the scrambling is unpredictable, it is deterministic, and therefore the scrambling can be reversed. To unscramble the image, we treat the multimode fibre as a linear, disordered scattering medium. To calibrate, we scan a focused beam of coherent light over thousands of different beam positions at the distal end and record complex fields at the proximal end of the fibre. This way, the inputoutput response of the system is determined, which then allows computational reconstruction of reflection-mode images. However, there remains the problem of illuminating the tissue via the fibre while avoiding back reflections from the proximal face. To avoid this drawback, we provide here the first preliminary confirmation that an image can be transferred through a 2x2 fibre coupler, with the sample at its distal port interrogated in reflection. Light is injected into one port for illumination and then collected from a second port for imaging.
Multimodal flexible cystoscopy for creating co-registered panoramas of the bladder urothelium

NASA Astrophysics Data System (ADS)

Seibel, Eric J.; Soper, Timothy D.; Burkhardt, Matthew R.; Porter, Michael P.; Yoon, W. Jong

2012-02-01

Bladder cancer is the most expensive cancer to treat due to the high rate of recurrence. Though white light cystoscopy is the gold standard for bladder cancer surveillance, the advent of fluorescence biomarkers provides an opportunity to improve sensitivity for early detection and reduced recurrence resulting from more accurate excision. Ideally, fluorescence information could be combined with standard reflectance images to provide multimodal views of the bladder wall. The scanning fiber endoscope (SFE) of 1.2mm in diameter is able to acquire wide-field multimodal video from a bladder phantom with fluorescence cancer "hot-spots". The SFE generates images by scanning red, green, and blue (RGB) laser light and detects the backscatter signal for reflectance video of 500-line resolution at 30 frames per second. We imaged a bladder phantom with painted vessels and mimicked fluorescent lesions by applying green fluorescent microspheres to the surface. By eliminating the green laser illumination, simultaneous reflectance and fluorescence images can be acquired at the same field of view, resolution, and frame rate. Moreover, the multimodal SFE is combined with a robotic steering mechanism and image stitching software as part of a fully automated bladder surveillance system. Using this system, the SFE can be reliably articulated over the entire 360° bladder surface. Acquired images can then be stitched into a multimodal 3D panorama of the bladder using software developed in our laboratory. In each panorama, the fluorescence images are exactly co-registered with RGB reflectance.
New Technologies, New Possibilities for the Arts and Multimodality in English Language Arts

ERIC Educational Resources Information Center

Williams, Wendy R.

2014-01-01

This article discusses the arts, multimodality, and new technologies in English language arts. It then turns to the example of the illuminated text--a multimodal book report consisting of animated text, music, and images--to consider how art, multimodality, and technology can work together to support students' reading of literature and inspire…

Combining kriging, multispectral and multimodal microscopy to resolve malaria-infected erythrocyte contents.

PubMed

Dabo-Niang, S; Zoueu, J T

2012-09-01

In this communication, we demonstrate how kriging, combine with multispectral and multimodal microscopy can enhance the resolution of malaria-infected images and provide more details on their composition, for analysis and diagnosis. The results of this interpolation applied to the two principal components of multispectral and multimodal images illustrate that the examination of the content of Plasmodium falciparum infected human erythrocyte is improved. © 2012 The Authors Journal of Microscopy © 2012 Royal Microscopical Society.
Multifocus confocal Raman microspectroscopy for fast multimode vibrational imaging of living cells.

PubMed

Okuno, Masanari; Hamaguchi, Hiro-o

2010-12-15

We have developed a multifocus confocal Raman microspectroscopic system for the fast multimode vibrational imaging of living cells. It consists of an inverted microscope equipped with a microlens array, a pinhole array, a fiber bundle, and a multichannel Raman spectrometer. Forty-eight Raman spectra from 48 foci under the microscope are simultaneously obtained by using multifocus excitation and image-compression techniques. The multifocus confocal configuration suppresses the background generated from the cover glass and the cell culturing medium so that high-contrast images are obtainable with a short accumulation time. The system enables us to obtain multimode (10 different vibrational modes) vibrational images of living cells in tens of seconds with only 1 mW laser power at one focal point. This image acquisition time is more than 10 times faster than that in conventional single-focus Raman microspectroscopy.
Severe Pain Due to Paraspinal Abscess Formation in Two Patients with Squamous-Cell Carcinoma of the Head and Neck after Multimodal Treatment Including Cetuximab.

PubMed

Gerlach, Christina; Pretzell, Ina; Lieberknecht, Elisabeth; Mattyasovszky, Stefan; Weber, Martin

2018-01-01

Patients with squamous-cell carcinoma of the head and neck (SCCHN) on palliative therapy usually have a bad prognosis and suffer from various symptoms. With increasing use of targeted agents in cancer patients at the end of life, the correct assignment of therapy-related symptoms becomes increasingly difficult as cancer-related symptoms usually increase as well. We report on 2 cases of patients with SCCHN who received multimodal treatment including palliative therapy with cetuximab. Both patients developed severe thoracic and cervicothoracic pain following treatment. In both cases, extensive paraspinal abscess formation proved to be the underlying cause. One patient was treated conservatively; the other one had to undergo surgical intervention. Awareness of multifaceted therapy-related complications is mandatory when patients receive multimodal treatment including targeted therapies. Unexplained pain syndromes in this context should raise suspicions concerning possible infectious complications and should lead to early use of magnetic resonance imaging. © 2018 S. Karger GmbH, Freiburg.
Multi-Modal Nano-Probes for Radionuclide and 5-color Near Infrared Optical Lymphatic Imaging

PubMed Central

Kobayashi, Hisataka; Koyama, Yoshinori; Barrett, Tristan; Hama, Yukihiro; Regino, Celeste A. S.; Shin, In Soo; Jang, Beom-Su; Le, Nhat; Paik, Chang H.; Choyke, Peter L.; Urano, Yasuteru

2008-01-01

Current contrast agents generally have one function and can only be imaged in monochrome, therefore, the majority of imaging methods can only impart uniparametric information. A single nano-particle has the potential to be loaded with multiple payloads. Such multi-modality probes have the ability to be imaged by more than one imaging technique, which could compensate for the weakness or even combine the advantages of each individual modality. Furthermore, optical imaging using different optical probes enables us to achieve multi-color in vivo imaging, wherein multiple parameters can be read from a single image. To allow differentiation of multiple optical signals in vivo, each probe should have a close but different near infrared emission. To this end, we synthesized nano-probes with multi-modal and multi-color potential, which employed a polyamidoamine dendrimer platform linked to both radionuclides and optical probes, permitting dual-modality scintigraphic and 5-color near infrared optical lymphatic imaging using a multiple excitation spectrally-resolved fluorescence imaging technique. PMID:19079788
Multimodal nanoprobes for radionuclide and five-color near-infrared optical lymphatic imaging.

PubMed

Kobayashi, Hisataka; Koyama, Yoshinori; Barrett, Tristan; Hama, Yukihiro; Regino, Celeste A S; Shin, In Soo; Jang, Beom-Su; Le, Nhat; Paik, Chang H; Choyke, Peter L; Urano, Yasuteru

2007-11-01

Current contrast agents generally have one function and can only be imaged in monochrome; therefore, the majority of imaging methods can only impart uniparametric information. A single nanoparticle has the potential to be loaded with multiple payloads. Such multimodality probes have the ability to be imaged by more than one imaging technique, which could compensate for the weakness or even combine the advantages of each individual modality. Furthermore, optical imaging using different optical probes enables us to achieve multicolor in vivo imaging, wherein multiple parameters can be read from a single image. To allow differentiation of multiple optical signals in vivo, each probe should have a close but different near-infrared emission. To this end, we synthesized nanoprobes with multimodal and multicolor potential, which employed a polyamidoamine dendrimer platform linked to both radionuclides and optical probes, permitting dual-modality scintigraphic and five-color near-infrared optical lymphatic imaging using a multiple-excitation spectrally resolved fluorescence imaging technique.
Multimodal digital color imaging system for facial skin lesion analysis

NASA Astrophysics Data System (ADS)

Bae, Youngwoo; Lee, Youn-Heum; Jung, Byungjo

2008-02-01

In dermatology, various digital imaging modalities have been used as an important tool to quantitatively evaluate the treatment effect of skin lesions. Cross-polarization color image was used to evaluate skin chromophores (melanin and hemoglobin) information and parallel-polarization image to evaluate skin texture information. In addition, UV-A induced fluorescent image has been widely used to evaluate various skin conditions such as sebum, keratosis, sun damages, and vitiligo. In order to maximize the evaluation efficacy of various skin lesions, it is necessary to integrate various imaging modalities into an imaging system. In this study, we propose a multimodal digital color imaging system, which provides four different digital color images of standard color image, parallel and cross-polarization color image, and UV-A induced fluorescent color image. Herein, we describe the imaging system and present the examples of image analysis. By analyzing the color information and morphological features of facial skin lesions, we are able to comparably and simultaneously evaluate various skin lesions. In conclusion, we are sure that the multimodal color imaging system can be utilized as an important assistant tool in dermatology.
Fully integrated reflection-mode photoacoustic, two-photon, and second harmonic generation microscopy in vivo

NASA Astrophysics Data System (ADS)

Song, Wei; Xu, Qiang; Zhang, Yang; Zhan, Yang; Zheng, Wei; Song, Liang

2016-08-01

The ability to obtain comprehensive structural and functional information from intact biological tissue in vivo is highly desirable for many important biomedical applications, including cancer and brain studies. Here, we developed a fully integrated multimodal microscopy that can provide photoacoustic (optical absorption), two-photon (fluorescence), and second harmonic generation (SHG) information from tissue in vivo, with intrinsically co-registered images. Moreover, using a delicately designed optical-acoustic coupling configuration, a high-frequency miniature ultrasonic transducer was integrated into a water-immersion optical objective, thus allowing all three imaging modalities to provide a high lateral resolution of ~290 nm with reflection-mode imaging capability, which is essential for studying intricate anatomy, such as that of the brain. Taking advantage of the complementary and comprehensive contrasts of the system, we demonstrated high-resolution imaging of various tissues in living mice, including microvasculature (by photoacoustics), epidermis cells, cortical neurons (by two-photon fluorescence), and extracellular collagen fibers (by SHG). The intrinsic image co-registration of the three modalities conveniently provided improved visualization and understanding of the tissue microarchitecture. The reported results suggest that, by revealing complementary tissue microstructures in vivo, this multimodal microscopy can potentially facilitate a broad range of biomedical studies, such as imaging of the tumor microenvironment and neurovascular coupling.
Multimodal Imaging of Brain Connectivity Using the MIBCA Toolbox: Preliminary Application to Alzheimer's Disease

NASA Astrophysics Data System (ADS)

Ribeiro, André Santos; Lacerda, Luís Miguel; Silva, Nuno André da; Ferreira, Hugo Alexandre

2015-06-01

The Multimodal Imaging Brain Connectivity Analysis (MIBCA) toolbox is a fully automated all-in-one connectivity analysis toolbox that offers both pre-processing, connectivity, and graph theory analysis of multimodal images such as anatomical, diffusion, and functional MRI, and PET. In this work, the MIBCA functionalities were used to study Alzheimer's Disease (AD) in a multimodal MR/PET approach. Materials and Methods: Data from 12 healthy controls, and 36 patients with EMCI, LMCI and AD (12 patients for each group) were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu), including T1-weighted (T1-w), Diffusion Tensor Imaging (DTI) data, and 18F-AV-45 (florbetapir) dynamic PET data from 40-60 min post injection (4x5 min). Both MR and PET data were automatically pre-processed for all subjects using MIBCA. T1-w data was parcellated into cortical and subcortical regions-of-interest (ROIs), and the corresponding thicknesses and volumes were calculated. DTI data was used to compute structural connectivity matrices based on fibers connecting pairs of ROIs. Lastly, dynamic PET images were summed, and the relative Standard Uptake Values calculated for each ROI. Results: An overall higher uptake of 18F-AV-45, consistent with an increased deposition of beta-amyloid, was observed for the AD group. Additionally, patients showed significant cortical atrophy (thickness and volume) especially in the entorhinal cortex and temporal areas, and a significant increase in Mean Diffusivity (MD) in the hippocampus, amygdala and temporal areas. Furthermore, patients showed a reduction of fiber connectivity with the progression of the disease, especially for intra-hemispherical connections. Conclusion: This work shows the potential of the MIBCA toolbox for the study of AD, as findings were shown to be in agreement with the literature. Here, only structural changes and beta-amyloid accumulation were considered. Yet, MIBCA is further able to process fMRI and different radiotracers, thus leading to integration of functional information, and supporting the research for new multimodal biomarkers for AD and other neurodegenerative diseases.
A framework for biomedical figure segmentation towards image-based document retrieval

PubMed Central

2013-01-01

The figures included in many of the biomedical publications play an important role in understanding the biological experiments and facts described within. Recent studies have shown that it is possible to integrate the information that is extracted from figures in classical document classification and retrieval tasks in order to improve their accuracy. One important observation about the figures included in biomedical publications is that they are often composed of multiple subfigures or panels, each describing different methodologies or results. The use of these multimodal figures is a common practice in bioscience, as experimental results are graphically validated via multiple methodologies or procedures. Thus, for a better use of multimodal figures in document classification or retrieval tasks, as well as for providing the evidence source for derived assertions, it is important to automatically segment multimodal figures into subfigures and panels. This is a challenging task, however, as different panels can contain similar objects (i.e., barcharts and linecharts) with multiple layouts. Also, certain types of biomedical figures are text-heavy (e.g., DNA sequences and protein sequences images) and they differ from traditional images. As a result, classical image segmentation techniques based on low-level image features, such as edges or color, are not directly applicable to robustly partition multimodal figures into single modal panels. In this paper, we describe a robust solution for automatically identifying and segmenting unimodal panels from a multimodal figure. Our framework starts by robustly harvesting figure-caption pairs from biomedical articles. We base our approach on the observation that the document layout can be used to identify encoded figures and figure boundaries within PDF files. Taking into consideration the document layout allows us to correctly extract figures from the PDF document and associate their corresponding caption. We combine pixel-level representations of the extracted images with information gathered from their corresponding captions to estimate the number of panels in the figure. Thus, our approach simultaneously identifies the number of panels and the layout of figures. In order to evaluate the approach described here, we applied our system on documents containing protein-protein interactions (PPIs) and compared the results against a gold standard that was annotated by biologists. Experimental results showed that our automatic figure segmentation approach surpasses pure caption-based and image-based approaches, achieving a 96.64% accuracy. To allow for efficient retrieval of information, as well as to provide the basis for integration into document classification and retrieval systems among other, we further developed a web-based interface that lets users easily retrieve panels containing the terms specified in the user queries. PMID:24565394
Spectral embedding-based registration (SERg) for multimodal fusion of prostate histology and MRI

NASA Astrophysics Data System (ADS)

Hwuang, Eileen; Rusu, Mirabela; Karthigeyan, Sudha; Agner, Shannon C.; Sparks, Rachel; Shih, Natalie; Tomaszewski, John E.; Rosen, Mark; Feldman, Michael; Madabhushi, Anant

2014-03-01

Multi-modal image registration is needed to align medical images collected from different protocols or imaging sources, thereby allowing the mapping of complementary information between images. One challenge of multimodal image registration is that typical similarity measures rely on statistical correlations between image intensities to determine anatomical alignment. The use of alternate image representations could allow for mapping of intensities into a space or representation such that the multimodal images appear more similar, thus facilitating their co-registration. In this work, we present a spectral embedding based registration (SERg) method that uses non-linearly embedded representations obtained from independent components of statistical texture maps of the original images to facilitate multimodal image registration. Our methodology comprises the following main steps: 1) image-derived textural representation of the original images, 2) dimensionality reduction using independent component analysis (ICA), 3) spectral embedding to generate the alternate representations, and 4) image registration. The rationale behind our approach is that SERg yields embedded representations that can allow for very different looking images to appear more similar, thereby facilitating improved co-registration. Statistical texture features are derived from the image intensities and then reduced to a smaller set by using independent component analysis to remove redundant information. Spectral embedding generates a new representation by eigendecomposition from which only the most important eigenvectors are selected. This helps to accentuate areas of salience based on modality-invariant structural information and therefore better identifies corresponding regions in both the template and target images. The spirit behind SERg is that image registration driven by these areas of salience and correspondence should improve alignment accuracy. In this work, SERg is implemented using Demons to allow the algorithm to more effectively register multimodal images. SERg is also tested within the free-form deformation framework driven by mutual information. Nine pairs of synthetic T1-weighted to T2-weighted brain MRI were registered under the following conditions: five levels of noise (0%, 1%, 3%, 5%, and 7%) and two levels of bias field (20% and 40%) each with and without noise. We demonstrate that across all of these conditions, SERg yields a mean squared error that is 81.51% lower than that of Demons driven by MRI intensity alone. We also spatially align twenty-six ex vivo histology sections and in vivo prostate MRI in order to map the spatial extent of prostate cancer onto corresponding radiologic imaging. SERg performs better than intensity registration by decreasing the root mean squared distance of annotated landmarks in the prostate gland via both Demons algorithm and mutual information-driven free-form deformation. In both synthetic and clinical experiments, the observed improvement in alignment of the template and target images suggest the utility of parametric eigenvector representations and hence SERg for multimodal image registration.
Quantitative label-free multimodality nonlinear optical imaging for in situ differentiation of cancerous lesions

NASA Astrophysics Data System (ADS)

Xu, Xiaoyun; Li, Xiaoyan; Cheng, Jie; Liu, Zhengfan; Thrall, Michael J.; Wang, Xi; Wang, Zhiyong; Wong, Stephen T. C.

2013-03-01

The development of real-time, label-free imaging techniques has recently attracted research interest for in situ differentiation of cancerous lesions from normal tissues. Molecule-specific intrinsic contrast can arise from label-free imaging techniques such as Coherent Anti-Stokes Raman Scattering (CARS), Two-Photon Excited AutoFluorescence (TPEAF), and Second Harmonic Generation (SHG), which, in combination, would hold the promise of a powerful label-free tool for cancer diagnosis. Among cancer-related deaths, lung carcinoma is the leading cause for both sexes. Although early treatment can increase the survival rate dramatically, lesion detection and precise diagnosis at an early stage is unusual due to its asymptomatic nature and limitations of current diagnostic techniques that make screening difficult. We investigated the potential of using multimodality nonlinear optical microscopy that incorporates CARS, TPEAF, and SHG techniques for differentiation of lung cancer from normal tissue. Cancerous and non-cancerous lung tissue samples from patients were imaged using CARS, TPEAF, and SHG techniques for comparison. These images showed good pathology correlation with hematoxylin and eosin (H and E) stained sections from the same tissue samples. Ongoing work includes imaging at various penetration depths to show three-dimensional morphologies of tumor cell nuclei using CARS, elastin using TPEAF, and collagen using SHG and developing classification algorithms for quantitative feature extraction to enable lung cancer diagnosis. Our results indicate that via real-time morphology analyses, a multimodality nonlinear optical imaging platform potentially offers a powerful minimally-invasive way to differentiate cancer lesions from surrounding non-tumor tissues in vivo for clinical applications.
Intra-operative label-free multimodal multiphoton imaging of breast cancer margins and microenvironment (Conference Presentation)

NASA Astrophysics Data System (ADS)

Sun, Yi; You, Sixian; Tu, Haohua; Spillman, Darold R.; Marjanovic, Marina; Chaney, Eric J.; Liu, George Z.; Ray, Partha S.; Higham, Anna; Boppart, Stephen A.

2017-02-01

Label-free multi-photon imaging has been a powerful tool for studying tissue microstructures and biochemical distributions, particularly for investigating tumors and their microenvironments. However, it remains challenging for traditional bench-top multi-photon microscope systems to conduct ex vivo tumor tissue imaging in the operating room due to their bulky setups and laser sources. In this study, we designed, built, and clinically demonstrated a portable multi-modal nonlinear label-free microscope system that combined four modalities, including two- and three- photon fluorescence for studying the distributions of FAD and NADH, and second and third harmonic generation, respectively, for collagen fiber structures and the distribution of micro-vesicles found in tumors and the microenvironment. Optical realignments and switching between modalities were motorized for more rapid and efficient imaging and for a light-tight enclosure, reducing ambient light noise to only 5% within the brightly lit operating room. Using up to 20 mW of laser power after a 20x objective, this system can acquire multi-modal sets of images over 600 μm × 600 μm at an acquisition rate of 60 seconds using galvo-mirror scanning. This portable microscope system was demonstrated in the operating room for imaging fresh, resected, unstained breast tissue specimens, and for assessing tumor margins and the tumor microenvironment. This real-time label-free nonlinear imaging system has the potential to uniquely characterize breast cancer margins and the microenvironment of tumors to intraoperatively identify structural, functional, and molecular changes that could indicate the aggressiveness of the tumor.
Multimodal system for the planning and guidance of bronchoscopy

NASA Astrophysics Data System (ADS)

Higgins, William E.; Cheirsilp, Ronnarit; Zang, Xiaonan; Byrnes, Patrick

2015-03-01

Many technical innovations in multimodal radiologic imaging and bronchoscopy have emerged recently in the effort against lung cancer. Modern X-ray computed-tomography (CT) scanners provide three-dimensional (3D) high-resolution chest images, positron emission tomography (PET) scanners give complementary molecular imaging data, and new integrated PET/CT scanners combine the strengths of both modalities. State-of-the-art bronchoscopes permit minimally invasive tissue sampling, with vivid endobronchial video enabling navigation deep into the airway-tree periphery, while complementary endobronchial ultrasound (EBUS) reveals local views of anatomical structures outside the airways. In addition, image-guided intervention (IGI) systems have proven their utility for CT-based planning and guidance of bronchoscopy. Unfortunately, no IGI system exists that integrates all sources effectively through the complete lung-cancer staging work flow. This paper presents a prototype of a computer-based multimodal IGI system that strives to fill this need. The system combines a wide range of automatic and semi-automatic image-processing tools for multimodal data fusion and procedure planning. It also provides a flexible graphical user interface for follow-on guidance of bronchoscopy/EBUS. Human-study results demonstrate the system's potential.
Structured illumination multimodal 3D-resolved quantitative phase and fluorescence sub-diffraction microscopy

PubMed Central

Chowdhury, Shwetadwip; Eldridge, Will J.; Wax, Adam; Izatt, Joseph A.

2017-01-01

Sub-diffraction resolution imaging has played a pivotal role in biological research by visualizing key, but previously unresolvable, sub-cellular structures. Unfortunately, applications of far-field sub-diffraction resolution are currently divided between fluorescent and coherent-diffraction regimes, and a multimodal sub-diffraction technique that bridges this gap has not yet been demonstrated. Here we report that structured illumination (SI) allows multimodal sub-diffraction imaging of both coherent quantitative-phase (QP) and fluorescence. Due to SI’s conventionally fluorescent applications, we first demonstrate the principle of SI-enabled three-dimensional (3D) QP sub-diffraction imaging with calibration microspheres. Image analysis confirmed enhanced lateral and axial resolutions over diffraction-limited QP imaging, and established striking parallels between coherent SI and conventional optical diffraction tomography. We next introduce an optical system utilizing SI to achieve 3D sub-diffraction, multimodal QP/fluorescent visualization of A549 biological cells fluorescently tagged for F-actin. Our results suggest that SI has a unique utility in studying biological phenomena with significant molecular, biophysical, and biochemical components. PMID:28663887
A Multimodal Search Engine for Medical Imaging Studies.

PubMed

Pinho, Eduardo; Godinho, Tiago; Valente, Frederico; Costa, Carlos

2017-02-01

The use of digital medical imaging systems in healthcare institutions has increased significantly, and the large amounts of data in these systems have led to the conception of powerful support tools: recent studies on content-based image retrieval (CBIR) and multimodal information retrieval in the field hold great potential in decision support, as well as for addressing multiple challenges in healthcare systems, such as computer-aided diagnosis (CAD). However, the subject is still under heavy research, and very few solutions have become part of Picture Archiving and Communication Systems (PACS) in hospitals and clinics. This paper proposes an extensible platform for multimodal medical image retrieval, integrated in an open-source PACS software with profile-based CBIR capabilities. In this article, we detail a technical approach to the problem by describing its main architecture and each sub-component, as well as the available web interfaces and the multimodal query techniques applied. Finally, we assess our implementation of the engine with computational performance benchmarks.
Multi-mode Intravascular RF Coil for MRI-guided Interventions

PubMed Central

Kurpad, Krishna N.; Unal, Orhan

2011-01-01

Purpose To demonstrate the feasibility of using a single intravascular RF probe connected to the external MRI system via a single coaxial cable to perform active tip tracking and catheter visualization, and high SNR intravascular imaging. Materials and Methods A multi-mode intravascular RF coil was constructed on a 6F balloon catheter and interfaced to a 1.5T MRI scanner via a decoupling circuit. Bench measurements of coil impedances were followed by imaging experiments in saline and phantoms. Results The multi-mode coil behaves as an inductively-coupled transmit coil. Forward looking capability of 6mm is measured. Greater than 3-fold increase in SNR compared to conventional imaging using optimized external coil is demonstrated. Simultaneous active tip tracking and catheter visualization is demonstrated. Conclusions It is feasible to perform 1) active tip tracking, 2) catheter visualization, and 3) high SNR imaging using a single multi-mode intravascular RF coil that is connected to the external system via a single coaxial cable. PMID:21448969
Identifying Multimodal Intermediate Phenotypes between Genetic Risk Factors and Disease Status in Alzheimer’s Disease

PubMed Central

Hao, Xiaoke; Yao, Xiaohui; Yan, Jingwen; Risacher, Shannon L.; Saykin, Andrew J.; Zhang, Daoqiang; Shen, Li

2016-01-01

Neuroimaging genetics has attracted growing attention and interest, which is thought to be a powerful strategy to examine the influence of genetic variants (i.e., single nucleotide polymorphisms (SNPs)) on structures or functions of human brain. In recent studies, univariate or multivariate regression analysis methods are typically used to capture the effective associations between genetic variants and quantitative traits (QTs) such as brain imaging phenotypes. The identified imaging QTs, although associated with certain genetic markers, may not be all disease specific. A useful, but underexplored, scenario could be to discover only those QTs associated with both genetic markers and disease status for revealing the chain from genotype to phenotype to symptom. In addition, multimodal brain imaging phenotypes are extracted from different perspectives and imaging markers consistently showing up in multimodalities may provide more insights for mechanistic understanding of diseases (i.e., Alzheimer’s disease (AD)). In this work, we propose a general framework to exploit multi-modal brain imaging phenotypes as intermediate traits that bridge genetic risk factors and multi-class disease status. We applied our proposed method to explore the relation between the well-known AD risk SNP APOE rs429358 and three baseline brain imaging modalities (i.e., structural magnetic resonance imaging (MRI), fluorodeoxyglucose positron emission tomography (FDG-PET) and F-18 florbetapir PET scans amyloid imaging (AV45)) from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. The empirical results demonstrate that our proposed method not only helps improve the performances of imaging genetic associations, but also discovers robust and consistent regions of interests (ROIs) across multi-modalities to guide the disease-induced interpretation. PMID:27277494
Multimodality bonchoscopic imaging of tracheopathica osteochondroplastica

NASA Astrophysics Data System (ADS)

Colt, Henri; Murgu, Septimiu D.; Ahn, Yeh-Chan; Brenner, Matt

2009-05-01

Results of a commercial optical coherence tomography system used as part of a multimodality diagnostic bronchoscopy platform are presented for a 61-year-old patient with central airway obstruction from tracheopathica osteochondroplastica. Comparison to results of white-light bronchoscopy, histology, and endobronchial ultrasound examination are accompanied by a discussion of resolution, penetration depth, contrast, and field of view of these imaging modalities. White-light bronchoscopy revealed irregularly shaped, firm submucosal nodules along cartilaginous structures of the anterior and lateral walls of the trachea, sparing the muscular posterior membrane. Endobronchial ultrasound showed a hyperechoic density of 0.4 cm thickness. optical coherence tomography (OCT) was performed using a commercially available, compact time-domain OCT system (Niris System, Imalux Corp., Cleveland, Ohio) with a magnetically actuating probe (two-dimensional, front imaging, and inside actuation). Images showed epithelium, upper submucosa, and osseous submucosal nodule layers corresponding with histopathology. To our knowledge, this is the first time these commercially available systems are used as part of a multimodality bronchoscopy platform to study diagnostic imaging of a benign disease causing central airway obstruction. Further studies are needed to optimize these systems for pulmonary applications and to determine how new-generation imaging modalities will be integrated into a multimodality bronchoscopy platform.
[Research on non-rigid registration of multi-modal medical image based on Demons algorithm].

PubMed

Hao, Peibo; Chen, Zhen; Jiang, Shaofeng; Wang, Yang

2014-02-01

Non-rigid medical image registration is a popular subject in the research areas of the medical image and has an important clinical value. In this paper we put forward an improved algorithm of Demons, together with the conservation of gray model and local structure tensor conservation model, to construct a new energy function processing multi-modal registration problem. We then applied the L-BFGS algorithm to optimize the energy function and solve complex three-dimensional data optimization problem. And finally we used the multi-scale hierarchical refinement ideas to solve large deformation registration. The experimental results showed that the proposed algorithm for large de formation and multi-modal three-dimensional medical image registration had good effects.
Multimodality Imaging Approach towards Primary Aortic Sarcomas Arising after Endovascular Abdominal Aortic Aneurysm Repair: Case Series Report.

PubMed

Kamran, Mudassar; Fowler, Kathryn J; Mellnick, Vincent M; Sicard, Gregorio A; Narra, Vamsi R

2016-06-01

Primary aortic neoplasms are rare. Aortic sarcoma arising after endovascular aneurysm repair (EVAR) is a scarce subset of primary aortic malignancies, reports of which are infrequent in the published literature. The diagnosis of aortic sarcoma is challenging due to its non-specific clinical presentation, and the prognosis is poor due to delayed diagnosis, rapid proliferation, and propensity for metastasis. Post-EVAR, aortic sarcomas may mimic other more common aortic processes on surveillance imaging. Radiologists are rarely knowledgeable about this rare entity for which multimodality imaging and awareness are invaluable in early diagnosis. A series of three pathologically confirmed cases are presented to display the multimodality imaging features and clinical presentations of aortic sarcoma arising after EVAR.

A multimodal imaging platform with integrated simultaneous photoacoustic microscopy, optical coherence tomography, optical Doppler tomography and fluorescence microscopy

NASA Astrophysics Data System (ADS)

Dadkhah, Arash; Zhou, Jun; Yeasmin, Nusrat; Jiao, Shuliang

2018-02-01

Various optical imaging modalities with different optical contrast mechanisms have been developed over the past years. Although most of these imaging techniques are being used in many biomedical applications and researches, integration of these techniques will allow researchers to reach the full potential of these technologies. Nevertheless, combining different imaging techniques is always challenging due to the difference in optical and hardware requirements for different imaging systems. Here, we developed a multimodal optical imaging system with the capability of providing comprehensive structural, functional and molecular information of living tissue in micrometer scale. This imaging system integrates photoacoustic microscopy (PAM), optical coherence tomography (OCT), optical Doppler tomography (ODT) and fluorescence microscopy in one platform. Optical-resolution PAM (OR-PAM) provides absorption-based imaging of biological tissues. Spectral domain OCT is able to provide structural information based on the scattering property of biological sample with no need for exogenous contrast agents. In addition, ODT is a functional extension of OCT with the capability of measurement and visualization of blood flow based on the Doppler effect. Fluorescence microscopy allows to reveal molecular information of biological tissue using autofluoresce or exogenous fluorophores. In-vivo as well as ex-vivo imaging studies demonstrated the capability of our multimodal imaging system to provide comprehensive microscopic information on biological tissues. Integrating all the aforementioned imaging modalities for simultaneous multimodal imaging has promising potential for preclinical research and clinical practice in the near future.
MINC 2.0: A Flexible Format for Multi-Modal Images.

PubMed

Vincent, Robert D; Neelin, Peter; Khalili-Mahani, Najmeh; Janke, Andrew L; Fonov, Vladimir S; Robbins, Steven M; Baghdadi, Leila; Lerch, Jason; Sled, John G; Adalat, Reza; MacDonald, David; Zijdenbos, Alex P; Collins, D Louis; Evans, Alan C

2016-01-01

It is often useful that an imaging data format can afford rich metadata, be flexible, scale to very large file sizes, support multi-modal data, and have strong inbuilt mechanisms for data provenance. Beginning in 1992, MINC was developed as a system for flexible, self-documenting representation of neuroscientific imaging data with arbitrary orientation and dimensionality. The MINC system incorporates three broad components: a file format specification, a programming library, and a growing set of tools. In the early 2000's the MINC developers created MINC 2.0, which added support for 64-bit file sizes, internal compression, and a number of other modern features. Because of its extensible design, it has been easy to incorporate details of provenance in the header metadata, including an explicit processing history, unique identifiers, and vendor-specific scanner settings. This makes MINC ideal for use in large scale imaging studies and databases. It also makes it easy to adapt to new scanning sequences and modalities.
Distance-Dependent Multimodal Image Registration for Agriculture Tasks

PubMed Central

Berenstein, Ron; Hočevar, Marko; Godeša, Tone; Edan, Yael; Ben-Shahar, Ohad

2015-01-01

Image registration is the process of aligning two or more images of the same scene taken at different times; from different viewpoints; and/or by different sensors. This research focuses on developing a practical method for automatic image registration for agricultural systems that use multimodal sensory systems and operate in natural environments. While not limited to any particular modalities; here we focus on systems with visual and thermal sensory inputs. Our approach is based on pre-calibrating a distance-dependent transformation matrix (DDTM) between the sensors; and representing it in a compact way by regressing the distance-dependent coefficients as distance-dependent functions. The DDTM is measured by calculating a projective transformation matrix for varying distances between the sensors and possible targets. To do so we designed a unique experimental setup including unique Artificial Control Points (ACPs) and their detection algorithms for the two sensors. We demonstrate the utility of our approach using different experiments and evaluation criteria. PMID:26308000
MEG-BIDS, the brain imaging data structure extended to magnetoencephalography

PubMed Central

Niso, Guiomar; Gorgolewski, Krzysztof J.; Bock, Elizabeth; Brooks, Teon L.; Flandin, Guillaume; Gramfort, Alexandre; Henson, Richard N.; Jas, Mainak; Litvak, Vladimir; T. Moreau, Jeremy; Oostenveld, Robert; Schoffelen, Jan-Mathijs; Tadel, Francois; Wexler, Joseph; Baillet, Sylvain

2018-01-01

We present a significant extension of the Brain Imaging Data Structure (BIDS) to support the specific aspects of magnetoencephalography (MEG) data. MEG measures brain activity with millisecond temporal resolution and unique source imaging capabilities. So far, BIDS was a solution to organise magnetic resonance imaging (MRI) data. The nature and acquisition parameters of MRI and MEG data are strongly dissimilar. Although there is no standard data format for MEG, we propose MEG-BIDS as a principled solution to store, organise, process and share the multidimensional data volumes produced by the modality. The standard also includes well-defined metadata, to facilitate future data harmonisation and sharing efforts. This responds to unmet needs from the multimodal neuroimaging community and paves the way to further integration of other techniques in electrophysiology. MEG-BIDS builds on MRI-BIDS, extending BIDS to a multimodal data structure. We feature several data-analytics software that have adopted MEG-BIDS, and a diverse sample of open MEG-BIDS data resources available to everyone. PMID:29917016
MEG-BIDS, the brain imaging data structure extended to magnetoencephalography.

PubMed

Niso, Guiomar; Gorgolewski, Krzysztof J; Bock, Elizabeth; Brooks, Teon L; Flandin, Guillaume; Gramfort, Alexandre; Henson, Richard N; Jas, Mainak; Litvak, Vladimir; T Moreau, Jeremy; Oostenveld, Robert; Schoffelen, Jan-Mathijs; Tadel, Francois; Wexler, Joseph; Baillet, Sylvain

2018-06-19

We present a significant extension of the Brain Imaging Data Structure (BIDS) to support the specific aspects of magnetoencephalography (MEG) data. MEG measures brain activity with millisecond temporal resolution and unique source imaging capabilities. So far, BIDS was a solution to organise magnetic resonance imaging (MRI) data. The nature and acquisition parameters of MRI and MEG data are strongly dissimilar. Although there is no standard data format for MEG, we propose MEG-BIDS as a principled solution to store, organise, process and share the multidimensional data volumes produced by the modality. The standard also includes well-defined metadata, to facilitate future data harmonisation and sharing efforts. This responds to unmet needs from the multimodal neuroimaging community and paves the way to further integration of other techniques in electrophysiology. MEG-BIDS builds on MRI-BIDS, extending BIDS to a multimodal data structure. We feature several data-analytics software that have adopted MEG-BIDS, and a diverse sample of open MEG-BIDS data resources available to everyone.
Multimodal targeted high relaxivity thermosensitive liposome for in vivo imaging

NASA Astrophysics Data System (ADS)

Kuijten, Maayke M. P.; Hannah Degeling, M.; Chen, John W.; Wojtkiewicz, Gregory; Waterman, Peter; Weissleder, Ralph; Azzi, Jamil; Nicolay, Klaas; Tannous, Bakhos A.

2015-11-01

Liposomes are spherical, self-closed structures formed by lipid bilayers that can encapsulate drugs and/or imaging agents in their hydrophilic core or within their membrane moiety, making them suitable delivery vehicles. We have synthesized a new liposome containing gadolinium-DOTA lipid bilayer, as a targeting multimodal molecular imaging agent for magnetic resonance and optical imaging. We showed that this liposome has a much higher molar relaxivities r1 and r2 compared to a more conventional liposome containing gadolinium-DTPA-BSA lipid. By incorporating both gadolinium and rhodamine in the lipid bilayer as well as biotin on its surface, we used this agent for multimodal imaging and targeting of tumors through the strong biotin-streptavidin interaction. Since this new liposome is thermosensitive, it can be used for ultrasound-mediated drug delivery at specific sites, such as tumors, and can be guided by magnetic resonance imaging.
Image recovery from defocused 2D fluorescent images in multimodal digital holographic microscopy.

PubMed

Quan, Xiangyu; Matoba, Osamu; Awatsuji, Yasuhiro

2017-05-01

A technique of three-dimensional (3D) intensity retrieval from defocused, two-dimensional (2D) fluorescent images in the multimodal digital holographic microscopy (DHM) is proposed. In the multimodal DHM, 3D phase and 2D fluorescence distributions are obtained simultaneously by an integrated system of an off-axis DHM and a conventional epifluorescence microscopy, respectively. This gives us more information of the target; however, defocused fluorescent images are observed due to the short depth of field. In this Letter, we propose a method to recover the defocused images based on the phase compensation and backpropagation from the defocused plane to the focused plane using the distance information that is obtained from a 3D phase distribution. By applying Zernike polynomial phase correction, we brought back the fluorescence intensity to the focused imaging planes. The experimental demonstration using fluorescent beads is presented, and the expected applications are suggested.
Real-time dynamic display of registered 4D cardiac MR and ultrasound images using a GPU

NASA Astrophysics Data System (ADS)

Zhang, Q.; Huang, X.; Eagleson, R.; Guiraudon, G.; Peters, T. M.

2007-03-01

In minimally invasive image-guided surgical interventions, different imaging modalities, such as magnetic resonance imaging (MRI), computed tomography (CT), and real-time three-dimensional (3D) ultrasound (US), can provide complementary, multi-spectral image information. Multimodality dynamic image registration is a well-established approach that permits real-time diagnostic information to be enhanced by placing lower-quality real-time images within a high quality anatomical context. For the guidance of cardiac procedures, it would be valuable to register dynamic MRI or CT with intraoperative US. However, in practice, either the high computational cost prohibits such real-time visualization of volumetric multimodal images in a real-world medical environment, or else the resulting image quality is not satisfactory for accurate guidance during the intervention. Modern graphics processing units (GPUs) provide the programmability, parallelism and increased computational precision to begin to address this problem. In this work, we first outline our research on dynamic 3D cardiac MR and US image acquisition, real-time dual-modality registration and US tracking. Then we describe image processing and optimization techniques for 4D (3D + time) cardiac image real-time rendering. We also present our multimodality 4D medical image visualization engine, which directly runs on a GPU in real-time by exploiting the advantages of the graphics hardware. In addition, techniques such as multiple transfer functions for different imaging modalities, dynamic texture binding, advanced texture sampling and multimodality image compositing are employed to facilitate the real-time display and manipulation of the registered dual-modality dynamic 3D MR and US cardiac datasets.
Parallel Information Processing (Image Transmission Via Fiber Bundle and Multimode Fiber

NASA Technical Reports Server (NTRS)

Kukhtarev, Nicholai

2003-01-01

Growing demand for visual, user-friendly representation of information inspires search for the new methods of image transmission. Currently used in-series (sequential) methods of information processing are inherently slow and are designed mainly for transmission of one or two dimensional arrays of data. Conventional transmission of data by fibers requires many fibers with array of laser diodes and photodetectors. In practice, fiber bundles are also used for transmission of images. Image is formed on the fiber-optic bundle entrance surface and each fiber transmits the incident image to the exit surface. Since the fibers do not preserve phase, only 2D intensity distribution can be transmitted in this way. Each single mode fiber transmit only one pixel of an image. Multimode fibers may be also used, so that each mode represent different pixel element. Direct transmission of image through multimode fiber is hindered by the mode scrambling and phase randomization. To overcome these obstacles wavelength and time-division multiplexing have been used, with each pixel transmitted on a separate wavelength or time interval. Phase-conjugate techniques also was tested in, but only in the unpractical scheme when reconstructed image return back to the fiber input end. Another method of three-dimensional imaging over single mode fibers was demonstrated in, using laser light of reduced spatial coherence. Coherence encoding, needed for a transmission of images by this methods, was realized with grating interferometer or with the help of an acousto-optic deflector. We suggest simple practical holographic method of image transmission over single multimode fiber or over fiber bundle with coherent light using filtering by holographic optical elements. Originally this method was successfully tested for the single multimode fiber. In this research we have modified holographic method for transmission of laser illuminated images over commercially available fiber bundle (fiber endoscope, or fiberscope).
Fast and Robust Registration of Multimodal Remote Sensing Images via Dense Orientated Gradient Feature

NASA Astrophysics Data System (ADS)

Ye, Y.

2017-09-01

This paper presents a fast and robust method for the registration of multimodal remote sensing data (e.g., optical, LiDAR, SAR and map). The proposed method is based on the hypothesis that structural similarity between images is preserved across different modalities. In the definition of the proposed method, we first develop a pixel-wise feature descriptor named Dense Orientated Gradient Histogram (DOGH), which can be computed effectively at every pixel and is robust to non-linear intensity differences between images. Then a fast similarity metric based on DOGH is built in frequency domain using the Fast Fourier Transform (FFT) technique. Finally, a template matching scheme is applied to detect tie points between images. Experimental results on different types of multimodal remote sensing images show that the proposed similarity metric has the superior matching performance and computational efficiency than the state-of-the-art methods. Moreover, based on the proposed similarity metric, we also design a fast and robust automatic registration system for multimodal images. This system has been evaluated using a pair of very large SAR and optical images (more than 20000 × 20000 pixels). Experimental results show that our system outperforms the two popular commercial software systems (i.e. ENVI and ERDAS) in both registration accuracy and computational efficiency.
Water-stable NaLuF4-based upconversion nanophosphors with long-term validity for multimodal lymphatic imaging.

PubMed

Zhou, Jing; Zhu, Xingjun; Chen, Min; Sun, Yun; Li, Fuyou

2012-09-01

Multimodal imaging is rapidly becoming an important tool for biomedical applications because it can compensate for the deficiencies of individual imaging modalities. Herein, multifunctional NaLuF(4)-based upconversion nanoparticles (Lu-UCNPs) were synthesized though a facile one-step microemulsion method under ambient condition. The doping of lanthanide ions (Gd(3+), Yb(3+) and Er(3+)/Tm(3+)) endows the Lu-UCNPs with high T(1)-enhancement, bright upconversion luminescence (UCL) emissions, and excellent X-ray absorption coefficient. Moreover, the as-prepared Lu-UCNPs are stable in water for more than six months, due to the protection of sodium glutamate and diethylene triamine pentacetate acid (DTPA) coordinating ligands on the surface. Lu-UCNPs have been successfully applied to the trimodal CT/MR/UCL lymphatic imaging on the modal of small animals. It is worth noting that Lu-UCNPs could be used for imaging even after preserving for over six months. In vitro transmission electron microscope (TEM), methyl thiazolyl tetrazolium (MTT) assay and histological analysis demonstrated that Lu-UCNPs exhibited low toxicity on living systems. Therefore, Lu-UCNPs could be multimodal agents for CT/MR/UCL imaging, and the concept can be served as a platform technology for the next-generation of probes for multimodal imaging. Copyright © 2012 Elsevier Ltd. All rights reserved.
VoxelStats: A MATLAB Package for Multi-Modal Voxel-Wise Brain Image Analysis.

PubMed

Mathotaarachchi, Sulantha; Wang, Seqian; Shin, Monica; Pascoal, Tharick A; Benedet, Andrea L; Kang, Min Su; Beaudry, Thomas; Fonov, Vladimir S; Gauthier, Serge; Labbe, Aurélie; Rosa-Neto, Pedro

2016-01-01

In healthy individuals, behavioral outcomes are highly associated with the variability on brain regional structure or neurochemical phenotypes. Similarly, in the context of neurodegenerative conditions, neuroimaging reveals that cognitive decline is linked to the magnitude of atrophy, neurochemical declines, or concentrations of abnormal protein aggregates across brain regions. However, modeling the effects of multiple regional abnormalities as determinants of cognitive decline at the voxel level remains largely unexplored by multimodal imaging research, given the high computational cost of estimating regression models for every single voxel from various imaging modalities. VoxelStats is a voxel-wise computational framework to overcome these computational limitations and to perform statistical operations on multiple scalar variables and imaging modalities at the voxel level. VoxelStats package has been developed in Matlab(®) and supports imaging formats such as Nifti-1, ANALYZE, and MINC v2. Prebuilt functions in VoxelStats enable the user to perform voxel-wise general and generalized linear models and mixed effect models with multiple volumetric covariates. Importantly, VoxelStats can recognize scalar values or image volumes as response variables and can accommodate volumetric statistical covariates as well as their interaction effects with other variables. Furthermore, this package includes built-in functionality to perform voxel-wise receiver operating characteristic analysis and paired and unpaired group contrast analysis. Validation of VoxelStats was conducted by comparing the linear regression functionality with existing toolboxes such as glim_image and RMINC. The validation results were identical to existing methods and the additional functionality was demonstrated by generating feature case assessments (t-statistics, odds ratio, and true positive rate maps). In summary, VoxelStats expands the current methods for multimodal imaging analysis by allowing the estimation of advanced regional association metrics at the voxel level.
“How many times must a man look up before he can really see the sky?” Rheumatic cardiovascular disease in the era of multimodality imaging

PubMed Central

Mavrogeni, Sophie I; Markousis-Mavrogenis, George; Heutemann, David; van Wijk, Kees; Reiber, Hans J; Kolovou, Genovefa

2015-01-01

Cardiovascular involvement in rheumatic diseases (RD) is the result of various pathophysiologic mechanisms including inflammation, accelerated atherosclerosis, myocardial ischemia, due to micro- or macro-vascular lesions and fibrosis. Noninvasive cardiovascular imaging, including echocardiography, nuclear techniques, cardiovascular computed tomography and cardiovascular magnetic resonance, represents the main diagnostic tool for early, non-invasive diagnosis of heart disease in RD. However, in the era of multimodality imaging and financial crisis there is an imperative need for rational use of imaging techniques in order to obtain the maximum benefit at the lowest possible cost for the health insurance system. The oligo-asymptomatic cardiovascular presentation and the high cardiovascular mortality of RD necessitate a reliable and reproducible diagnostic approach to catch early cardiovascular involvement. Echocardiography remains the routine cornerstone of cardiovascular evaluation. However, a normal echocardiogram can not always exclude cardiac involvement and/or identify heart disease acuity and pathophysiology. Therefore, cardiovascular magnetic resonance is a necessary adjunct complementary to echocardiography, especially in new onset heart failure and when there are conflicting data from clinical, electrocardiographic and echocardiographic evaluation of RD patients. PMID:26413486
A digital library for medical imaging activities

NASA Astrophysics Data System (ADS)

dos Santos, Marcelo; Furuie, Sérgio S.

2007-03-01

This work presents the development of an electronic infrastructure to make available a free, online, multipurpose and multimodality medical image database. The proposed infrastructure implements a distributed architecture for medical image database, authoring tools, and a repository for multimedia documents. Also it includes a peer-reviewed model that assures quality of dataset. This public repository provides a single point of access for medical images and related information to facilitate retrieval tasks. The proposed approach has been used as an electronic teaching system in Radiology as well.
Novel multimodality segmentation using level sets and Jensen-Rényi divergence

DOE Office of Scientific and Technical Information (OSTI.GOV)

Markel, Daniel, E-mail: daniel.markel@mail.mcgill.ca; Zaidi, Habib; Geneva Neuroscience Center, Geneva University, CH-1205 Geneva

2013-12-15

Purpose: Positron emission tomography (PET) is playing an increasing role in radiotherapy treatment planning. However, despite progress, robust algorithms for PET and multimodal image segmentation are still lacking, especially if the algorithm were extended to image-guided and adaptive radiotherapy (IGART). This work presents a novel multimodality segmentation algorithm using the Jensen-Rényi divergence (JRD) to evolve the geometric level set contour. The algorithm offers improved noise tolerance which is particularly applicable to segmentation of regions found in PET and cone-beam computed tomography. Methods: A steepest gradient ascent optimization method is used in conjunction with the JRD and a level set activemore » contour to iteratively evolve a contour to partition an image based on statistical divergence of the intensity histograms. The algorithm is evaluated using PET scans of pharyngolaryngeal squamous cell carcinoma with the corresponding histological reference. The multimodality extension of the algorithm is evaluated using 22 PET/CT scans of patients with lung carcinoma and a physical phantom scanned under varying image quality conditions. Results: The average concordance index (CI) of the JRD segmentation of the PET images was 0.56 with an average classification error of 65%. The segmentation of the lung carcinoma images had a maximum diameter relative error of 63%, 19.5%, and 14.8% when using CT, PET, and combined PET/CT images, respectively. The estimated maximal diameters of the gross tumor volume (GTV) showed a high correlation with the macroscopically determined maximal diameters, with aR{sup 2} value of 0.85 and 0.88 using the PET and PET/CT images, respectively. Results from the physical phantom show that the JRD is more robust to image noise compared to mutual information and region growing. Conclusions: The JRD has shown improved noise tolerance compared to mutual information for the purpose of PET image segmentation. Presented is a flexible framework for multimodal image segmentation that can incorporate a large number of inputs efficiently for IGART.« less
Novel multimodality segmentation using level sets and Jensen-Rényi divergence.

PubMed

Markel, Daniel; Zaidi, Habib; El Naqa, Issam

2013-12-01

Positron emission tomography (PET) is playing an increasing role in radiotherapy treatment planning. However, despite progress, robust algorithms for PET and multimodal image segmentation are still lacking, especially if the algorithm were extended to image-guided and adaptive radiotherapy (IGART). This work presents a novel multimodality segmentation algorithm using the Jensen-Rényi divergence (JRD) to evolve the geometric level set contour. The algorithm offers improved noise tolerance which is particularly applicable to segmentation of regions found in PET and cone-beam computed tomography. A steepest gradient ascent optimization method is used in conjunction with the JRD and a level set active contour to iteratively evolve a contour to partition an image based on statistical divergence of the intensity histograms. The algorithm is evaluated using PET scans of pharyngolaryngeal squamous cell carcinoma with the corresponding histological reference. The multimodality extension of the algorithm is evaluated using 22 PET/CT scans of patients with lung carcinoma and a physical phantom scanned under varying image quality conditions. The average concordance index (CI) of the JRD segmentation of the PET images was 0.56 with an average classification error of 65%. The segmentation of the lung carcinoma images had a maximum diameter relative error of 63%, 19.5%, and 14.8% when using CT, PET, and combined PET/CT images, respectively. The estimated maximal diameters of the gross tumor volume (GTV) showed a high correlation with the macroscopically determined maximal diameters, with a R(2) value of 0.85 and 0.88 using the PET and PET/CT images, respectively. Results from the physical phantom show that the JRD is more robust to image noise compared to mutual information and region growing. The JRD has shown improved noise tolerance compared to mutual information for the purpose of PET image segmentation. Presented is a flexible framework for multimodal image segmentation that can incorporate a large number of inputs efficiently for IGART.
Multimodal imaging of the disease progression of birdshot chorioretinopathy.

PubMed

Teussink, Michel M; Huis In Het Veld, Paulien I; de Vries, Lieuwe A M; Hoyng, Carel B; Klevering, B Jeroen; Theelen, Thomas

2016-12-01

To study outer retinal deterioration in relation to clinical disease activity in patients with birdshot chorioretinopathy using fundus autofluorescence and spectral-domain optical coherence tomography (OCT). A single-centre retrospective cohort study was carried out on 42 eyes of 21 patients with birdshot disease, using a multimodal imaging approach including fundus autofluorescence, OCT, fluorescein angiography and indocyanine green angiography in combination with a patient chart review. The patients' overall clinical activity of retinal vasculitis during the follow-up period was determined by periods of clinical activity as indicated by fluorescein angiography and associated treatment decisions. Image analysis was performed to examine the spatial correspondence between autofluorescence changes and disruption of the photoreceptor inner segment ellipsoid zone on OCT. Three common types of outer retinal lesions were observed in fovea-centred images of 43% of patients: circular patches of chorioretinal atrophy, ellipsoid zone disruption on OCT, and outer retinal atrophy on autofluorescence and OCT. There was good spatial correspondence between ellipsoid zone disruption and areas of diffuse hyper-autofluorescence outside the fovea. Interestingly, the ellipsoid zone disruption recovered in four out of seven patients upon intensified therapeutic immunosuppression. Most patients only developed peripapillary atrophy and occasional perivascular hypo-autofluorescence. A multimodal imaging approach with autofluorescence imaging and OCT may help to detect ellipsoid zone disruption in the central retina of patients with birdshot disease. Our results suggest that ellipsoid zone disruption may be related to both the activity and duration of retinal vasculitis, and could help to determine therapeutic success in birdshot disease. © 2016 Acta Ophthalmologica Scandinavica Foundation. Published by John Wiley & Sons Ltd.
IMAGING WITH MULTIMODAL ADAPTIVE-OPTICS OPTICAL COHERENCE TOMOGRAPHY IN MULTIPLE EVANESCENT WHITE DOT SYNDROME: THE STRUCTURE AND FUNCTIONAL RELATIONSHIP.

PubMed

Labriola, Leanne T; Legarreta, Andrew D; Legarreta, John E; Nadler, Zach; Gallagher, Denise; Hammer, Daniel X; Ferguson, R Daniel; Iftimia, Nicusor; Wollstein, Gadi; Schuman, Joel S

2016-01-01

To elucidate the location of pathological changes in multiple evanescent white dot syndrome (MEWDS) with the use of multimodal adaptive optics (AO) imaging. A 5-year observational case study of a 24-year-old female with recurrent MEWDS. Full examination included history, Snellen chart visual acuity, pupil assessment, intraocular pressures, slit lamp evaluation, dilated fundoscopic exam, imaging with Fourier-domain optical coherence tomography (FD-OCT), blue-light fundus autofluorescence (FAF), fundus photography, fluorescein angiography, and adaptive-optics optical coherence tomography. Three distinct acute episodes of MEWDS occurred during the period of follow-up. Fourier-domain optical coherence tomography and adaptive-optics imaging showed disturbance in the photoreceptor outer segments (PR OS) in the posterior pole with each flare. The degree of disturbance at the photoreceptor level corresponded to size and extent of the visual field changes. All findings were transient with delineation of the photoreceptor recovery from the outer edges of the lesion inward. Hyperautofluorescence was seen during acute flares. Increase in choroidal thickness did occur with each active flare but resolved. Although changes in the choroid and RPE can be observed in MEWDS, Fourier-domain optical coherence tomography, and multimodal adaptive optics imaging localized the visually significant changes seen in this disease at the level of the photoreceptors. These transient retinal changes specifically occur at the level of the inner segment ellipsoid and OS/RPE line. En face optical coherence tomography imaging provides a detailed, yet noninvasive method for following the convalescence of MEWDS and provides insight into the structural and functional relationship of this transient inflammatory retinal disease.
Beyond endoscopic assessment in inflammatory bowel disease: real-time histology of disease activity by non-linear multimodal imaging

NASA Astrophysics Data System (ADS)

Chernavskaia, Olga; Heuke, Sandro; Vieth, Michael; Friedrich, Oliver; Schürmann, Sebastian; Atreya, Raja; Stallmach, Andreas; Neurath, Markus F.; Waldner, Maximilian; Petersen, Iver; Schmitt, Michael; Bocklitz, Thomas; Popp, Jürgen

2016-07-01

Assessing disease activity is a prerequisite for an adequate treatment of inflammatory bowel diseases (IBD) such as Crohn’s disease and ulcerative colitis. In addition to endoscopic mucosal healing, histologic remission poses a promising end-point of IBD therapy. However, evaluating histological remission harbors the risk for complications due to the acquisition of biopsies and results in a delay of diagnosis because of tissue processing procedures. In this regard, non-linear multimodal imaging techniques might serve as an unparalleled technique that allows the real-time evaluation of microscopic IBD activity in the endoscopy unit. In this study, tissue sections were investigated using the non-linear multimodal microscopy combination of coherent anti-Stokes Raman scattering (CARS), two-photon excited auto fluorescence (TPEF) and second-harmonic generation (SHG). After the measurement a gold-standard assessment of histological indexes was carried out based on a conventional H&E stain. Subsequently, various geometry and intensity related features were extracted from the multimodal images. An optimized feature set was utilized to predict histological index levels based on a linear classifier. Based on the automated prediction, the diagnosis time interval is decreased. Therefore, non-linear multimodal imaging may provide a real-time diagnosis of IBD activity suited to assist clinical decision making within the endoscopy unit.
Role of cardiac imaging and three-dimensional printing in percutaneous appendage closure.

PubMed

Iriart, Xavier; Ciobotaru, Vlad; Martin, Claire; Cochet, Hubert; Jalal, Zakaria; Thambo, Jean-Benoit; Quessard, Astrid

2018-06-06

Atrial fibrillation is the most frequent cardiac arrhythmia, affecting up to 13% of people aged>80 years, and is responsible for 15-20% of all ischaemic strokes. Left atrial appendage occlusion devices have been developed as an alternative approach to reduce the risk of stroke in patients for whom oral anticoagulation is contraindicated. The procedure can be technically demanding, and obtaining a complete left atrial appendage occlusion can be challenging. These observations have emphasized the importance of preprocedural planning, to optimize the accuracy and safety of the procedure. In this setting, a multimodality imaging approach, including three-dimensional imaging, is often used for preoperative assessment and procedural guidance. These imaging modalities, including transoesophageal echocardiography and multislice computed tomography, allow acquisition of a three-dimensional dataset that improves understanding of the cardiac anatomy; dedicated postprocessing software integrated into the clinical workflow can be used to generate a stereolithography file, which can be printed in a rubber-like material, seeking to replicate the myocardial tissue characteristics and mechanical properties of the left atrial appendage wall. The role of multimodality imaging and 3D printing technology offers a new field for implantation simulation, which may have a major impact on physician training and technique optimization. Copyright © 2018 Elsevier Masson SAS. All rights reserved.

Multimodality imaging of adult gastric emergencies: A pictorial review

PubMed Central

Sunnapwar, Abhijit; Ojili, Vijayanadh; Katre, Rashmi; Shah, Hardik; Nagar, Arpit

2017-01-01

Acute gastric emergencies require urgent surgical or nonsurgical intervention because they are associated with high morbidity and mortality. Imaging plays an important role in diagnosis since the clinical symptoms are often nonspecific and radiologist may be the first one to suggest a diagnosis as the imaging findings are often characteristic. The purpose of this article is to provide a comprehensive review of multimodality imaging (plain radiograph, fluoroscopy, and computed tomography) of various life threatening gastric emergencies. PMID:28515579
Cross-Modality Image Synthesis via Weakly Coupled and Geometry Co-Regularized Joint Dictionary Learning.

PubMed

Huang, Yawen; Shao, Ling; Frangi, Alejandro F

2018-03-01

Multi-modality medical imaging is increasingly used for comprehensive assessment of complex diseases in either diagnostic examinations or as part of medical research trials. Different imaging modalities provide complementary information about living tissues. However, multi-modal examinations are not always possible due to adversary factors, such as patient discomfort, increased cost, prolonged scanning time, and scanner unavailability. In additionally, in large imaging studies, incomplete records are not uncommon owing to image artifacts, data corruption or data loss, which compromise the potential of multi-modal acquisitions. In this paper, we propose a weakly coupled and geometry co-regularized joint dictionary learning method to address the problem of cross-modality synthesis while considering the fact that collecting the large amounts of training data is often impractical. Our learning stage requires only a few registered multi-modality image pairs as training data. To employ both paired images and a large set of unpaired data, a cross-modality image matching criterion is proposed. Then, we propose a unified model by integrating such a criterion into the joint dictionary learning and the observed common feature space for associating cross-modality data for the purpose of synthesis. Furthermore, two regularization terms are added to construct robust sparse representations. Our experimental results demonstrate superior performance of the proposed model over state-of-the-art methods.
Application of Virtual Navigation with Multimodality Image Fusion in Foramen Ovale Cannulation.

PubMed

Qiu, Xixiong; Liu, Weizong; Zhang, Mingdong; Lin, Hengzhou; Zhou, Shoujun; Lei, Yi; Xia, Jun

2017-11-01

Idiopathic trigeminal neuralgia (ITN) can be effectively treated with radiofrequency thermocoagulation. However, this procedure requires cannulation of the foramen ovale, and conventional cannulation methods are associated with high failure rates. Multimodality imaging can improve the accuracy of cannulation because each imaging method can compensate for the drawbacks of the other. We aim to determine the feasibility and accuracy of percutaneous foramen ovale cannulation under the guidance of virtual navigation with multimodality image fusion in a self-designed anatomical model of human cadaveric heads. Five cadaveric head specimens were investigated in this study. Spiral computed tomography (CT) scanning clearly displayed the foramen ovale in all five specimens (10 foramina), which could not be visualized using two-dimensional ultrasound alone. The ultrasound and spiral CT images were fused, and percutaneous cannulation of the foramen ovale was performed under virtual navigation. After this, spiral CT scanning was immediately repeated to confirm the accuracy of the cannulation. Postprocedural spiral CT confirmed that the ultrasound and CT images had been successfully fused for all 10 foramina, which were accurately and successfully cannulated. The success rates of both image fusion and cannulation were 100%. Virtual navigation with multimodality image fusion can substantially facilitate foramen ovale cannulation and is worthy of clinical application. © 2017 American Academy of Pain Medicine. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com
Multimodal image analysis of clinical influences on preterm brain development

PubMed Central

Ball, Gareth; Aljabar, Paul; Nongena, Phumza; Kennea, Nigel; Gonzalez‐Cinca, Nuria; Falconer, Shona; Chew, Andrew T.M.; Harper, Nicholas; Wurie, Julia; Rutherford, Mary A.; Edwards, A. David

2017-01-01

Objective Premature birth is associated with numerous complex abnormalities of white and gray matter and a high incidence of long‐term neurocognitive impairment. An integrated understanding of these abnormalities and their association with clinical events is lacking. The aim of this study was to identify specific patterns of abnormal cerebral development and their antenatal and postnatal antecedents. Methods In a prospective cohort of 449 infants (226 male), we performed a multivariate and data‐driven analysis combining multiple imaging modalities. Using canonical correlation analysis, we sought separable multimodal imaging markers associated with specific clinical and environmental factors and correlated to neurodevelopmental outcome at 2 years. Results We found five independent patterns of neuroanatomical variation that related to clinical factors including age, prematurity, sex, intrauterine complications, and postnatal adversity. We also confirmed the association between imaging markers of neuroanatomical abnormality and poor cognitive and motor outcomes at 2 years. Interpretation This data‐driven approach defined novel and clinically relevant imaging markers of cerebral maldevelopment, which offer new insights into the nature of preterm brain injury. Ann Neurol 2017;82:233–246 PMID:28719076
Dual CARS and SHG image acquisition scheme that combines single central fiber and multimode fiber bundle to collect and differentiate backward and forward generated photons

PubMed Central

Weng, Sheng; Chen, Xu; Xu, Xiaoyun; Wong, Kelvin K.; Wong, Stephen T. C.

2016-01-01

In coherent anti-Stokes Raman scattering (CARS) and second harmonic generation (SHG) imaging, backward and forward generated photons exhibit different image patterns and thus capture salient intrinsic information of tissues from different perspectives. However, they are often mixed in collection using traditional image acquisition methods and thus are hard to interpret. We developed a multimodal scheme using a single central fiber and multimode fiber bundle to simultaneously collect and differentiate images formed by these two types of photons and evaluated the scheme in an endomicroscopy prototype. The ratio of these photons collected was calculated for the characterization of tissue regions with strong or weak epi-photon generation while different image patterns of these photons at different tissue depths were revealed. This scheme provides a new approach to extract and integrate information captured by backward and forward generated photons in dual CARS/SHG imaging synergistically for biomedical applications. PMID:27375938
Multimodal molecular 3D imaging for the tumoral volumetric distribution assessment of folate-based biosensors.

PubMed

Ramírez-Nava, Gerardo J; Santos-Cuevas, Clara L; Chairez, Isaac; Aranda-Lara, Liliana

2017-12-01

The aim of this study was to characterize the in vivo volumetric distribution of three folate-based biosensors by different imaging modalities (X-ray, fluorescence, Cerenkov luminescence, and radioisotopic imaging) through the development of a tridimensional image reconstruction algorithm. The preclinical and multimodal Xtreme imaging system, with a Multimodal Animal Rotation System (MARS), was used to acquire bidimensional images, which were processed to obtain the tridimensional reconstruction. Images of mice at different times (biosensor distribution) were simultaneously obtained from the four imaging modalities. The filtered back projection and inverse Radon transformation were used as main image-processing techniques. The algorithm developed in Matlab was able to calculate the volumetric profiles of 99m Tc-Folate-Bombesin (radioisotopic image), 177 Lu-Folate-Bombesin (Cerenkov image), and FolateRSense™ 680 (fluorescence image) in tumors and kidneys of mice, and no significant differences were detected in the volumetric quantifications among measurement techniques. The imaging tridimensional reconstruction algorithm can be easily extrapolated to different 2D acquisition-type images. This characteristic flexibility of the algorithm developed in this study is a remarkable advantage in comparison to similar reconstruction methods.
Multimodality Imaging of RNA Interference

PubMed Central

Nayak, Tapas R.; Krasteva, Lazura K.; Cai, Weibo

2013-01-01

The discovery of small interfering RNAs (siRNAs) and their potential to knock down virtually any gene of interest has ushered in a new era of RNA interference (RNAi). Clinical use of RNAi faces severe limitations due to inefficiency delivery of siRNA or short hairpin RNA (shRNA). Many molecular imaging techniques have been adopted in RNAi-related research for evaluation of siRNA/shRNA delivery, biodistribution, pharmacokinetics, and the therapeutic effect. In this review article, we summarize the current status of in vivo imaging of RNAi. The molecular imaging techniques that have been employed include bioluminescence/fluorescence imaging, magnetic resonance imaging/spectroscopy, positron emission tomography, single-photon emission computed tomography, and various combinations of these techniques. Further development of non-invasive imaging strategies for RNAi, not only focusing on the delivery of siRNA/shRNA but also the therapeutic efficacy, is critical for future clinical translation. Rigorous validation will be needed to confirm that biodistribution of the carrier is correlated with that of siRNA/shRNA, since imaging only detects the label (e.g. radioisotopes) but not the gene or carrier themselves. It is also essential to develop multimodality imaging approaches for realizing the full potential of therapeutic RNAi, as no single imaging modality may be sufficient to simultaneously monitor both the gene delivery and silencing effect of RNAi. PMID:23745567
Tumor image signatures and habitats: a processing pipeline of multimodality metabolic and physiological images.

PubMed

You, Daekeun; Kim, Michelle M; Aryal, Madhava P; Parmar, Hemant; Piert, Morand; Lawrence, Theodore S; Cao, Yue

2018-01-01

To create tumor "habitats" from the "signatures" discovered from multimodality metabolic and physiological images, we developed a framework of a processing pipeline. The processing pipeline consists of six major steps: (1) creating superpixels as a spatial unit in a tumor volume; (2) forming a data matrix [Formula: see text] containing all multimodality image parameters at superpixels; (3) forming and clustering a covariance or correlation matrix [Formula: see text] of the image parameters to discover major image "signatures;" (4) clustering the superpixels and organizing the parameter order of the [Formula: see text] matrix according to the one found in step 3; (5) creating "habitats" in the image space from the superpixels associated with the "signatures;" and (6) pooling and clustering a matrix consisting of correlation coefficients of each pair of image parameters from all patients to discover subgroup patterns of the tumors. The pipeline was applied to a dataset of multimodality images in glioblastoma (GBM) first, which consisted of 10 image parameters. Three major image "signatures" were identified. The three major "habitats" plus their overlaps were created. To test generalizability of the processing pipeline, a second image dataset from GBM, acquired on the scanners different from the first one, was processed. Also, to demonstrate the clinical association of image-defined "signatures" and "habitats," the patterns of recurrence of the patients were analyzed together with image parameters acquired prechemoradiation therapy. An association of the recurrence patterns with image-defined "signatures" and "habitats" was revealed. These image-defined "signatures" and "habitats" can be used to guide stereotactic tissue biopsy for genetic and mutation status analysis and to analyze for prediction of treatment outcomes, e.g., patterns of failure.
A Multi-Modality CMOS Sensor Array for Cell-Based Assay and Drug Screening.

PubMed

Chi, Taiyun; Park, Jong Seok; Butts, Jessica C; Hookway, Tracy A; Su, Amy; Zhu, Chengjie; Styczynski, Mark P; McDevitt, Todd C; Wang, Hua

2015-12-01

In this paper, we present a fully integrated multi-modality CMOS cellular sensor array with four sensing modalities to characterize different cell physiological responses, including extracellular voltage recording, cellular impedance mapping, optical detection with shadow imaging and bioluminescence sensing, and thermal monitoring. The sensor array consists of nine parallel pixel groups and nine corresponding signal conditioning blocks. Each pixel group comprises one temperature sensor and 16 tri-modality sensor pixels, while each tri-modality sensor pixel can be independently configured for extracellular voltage recording, cellular impedance measurement (voltage excitation/current sensing), and optical detection. This sensor array supports multi-modality cellular sensing at the pixel level, which enables holistic cell characterization and joint-modality physiological monitoring on the same cellular sample with a pixel resolution of 80 μm × 100 μm. Comprehensive biological experiments with different living cell samples demonstrate the functionality and benefit of the proposed multi-modality sensing in cell-based assay and drug screening.
Multimodal Retinal Imaging in Incontinentia Pigmenti Including Optical Coherence Tomography Angiography: Findings From an Older Cohort With Mild Phenotype.

PubMed

Liu, Tin Yan Alvin; Han, Ian C; Goldberg, Morton F; Linz, Marguerite O; Chen, Connie J; Scott, Adrienne W

2018-05-01

Incontinentia pigmenti (IP) is a rare, X-linked dominant disease with potentially severe ocular complications that predominantly affect the peripheral retina. However, little is known about its effects on the macula. To describe the structural and vascular abnormalities observed in the maculas of patients with IP and to correlate these findings with peripheral pathologies. Prospective, cross-sectional study at Wilmer Eye Institute, Johns Hopkins University. Five participants with a clinical diagnosis of IP were included and underwent multimodal imaging with ultra-wide-field fluorescein angiography (FA), spectral-domain optical coherence tomography (OCT), and OCT angiography. The structural and vascular abnormalities observed on spectral-domain OCT and OCT angiography and their correlation with peripheral pathologies seen on ultra-wide-field FA. A total of 9 eyes from 5 patients (median age, 20.5 years; range, 8.4-54.2 years) were included. Median Snellen visual acuity was 20/32 (range, 20/16 to 20/63). ultra-wide-field FA-identified retinal vascular abnormalities in all 7 eyes in which FA was obtained. These abnormalities included microaneurysms, areas of nonperfusion, and vascular anastomoses, most of which were peripheral to the standard view of 30° FA with peripheral sweeps. Structural abnormalities were observed in 6 eyes on spectral-domain OCT, including inner retinal thinning and irregularities in the outer plexiform layer. Optical coherence tomography angiography abnormalities were noted in all 9 eyes, including decreased vascular density, abnormal vascular loops, and flow loss in the superficial and deep plexuses, which corresponded to areas of retinal thinning on spectral-domain OCT. Although our study is limited by the small sample size, the findings suggest that multimodal imaging is useful for detecting structural and vascular abnormalities that may not be apparent on ophthalmoscopy in patients with IP. Macular pathologies, especially a decrease in vascular density on OCT angiography, are common. Further studies are needed to characterize further the association between macular and peripheral abnormalities in patients with IP.
Recent Advances in Molecular, Multimodal and Theranostic Ultrasound Imaging

PubMed Central

Kiessling, Fabian; Fokong, Stanley; Bzyl, Jessica; Lederle, Wiltrud; Palmowski, Moritz; Lammers, Twan

2014-01-01

Ultrasound (US) imaging is an exquisite tool for the non-invasive and real-time diagnosis of many different diseases. In this context, US contrast agents can improve lesion delineation, characterization and therapy response evaluation. US contrast agents are usually micrometer-sized gas bubbles, stabilized with soft or hard shells. By conjugating antibodies to the microbubble (MB) surface, and by incorporating diagnostic agents, drugs or nucleic acids into or onto the MB shell, molecular, multimodal and theranostic MB can be generated. We here summarize recent advances in molecular, multimodal and theranostic US imaging, and introduce concepts how such advanced MB can be generated, applied and imaged. Examples are given for their use to image and treat oncological, cardiovascular and neurological diseases. Furthermore, we discuss for which therapeutic entities incorporation into (or conjugation to) MB is meaningful, and how US-mediated MB destruction can increase their extravasation, penetration, internalization and efficacy. PMID:24316070
The value of multimodality imaging in the investigation of a PSA recurrence after radical prostatectomy in the Irish hospital setting.

PubMed

McLoughlin, L C; Inder, S; Moran, D; O'Rourke, C; Manecksha, R P; Lynch, T H

2018-02-01

The diagnostic evaluation of a PSA recurrence after RP in the Irish hospital setting involves multimodality imaging with MRI, CT, and bone scanning, despite the low diagnostic yield from imaging at low PSA levels. We aim to investigate the value of multimodality imaging in PC patients after RP with a PSA recurrence. Forty-eight patients with a PSA recurrence after RP who underwent multimodality imaging were evaluated. Demographic data, postoperative PSA levels, and imaging studies performed at those levels were evaluated. Eight (21%) MRIs, 6 (33%) CTs, and 4 (9%) bone scans had PCa-specific findings. Three (12%) patients had a positive MRI with a PSA <1.0 ng/ml, while 5 (56%) were positive at PSA ≥1.1 ng/ml (p = 0.05). Zero patient had a positive CT TAP at a PSA level <1.0 ng/ml, while 5 (56%) were positive at levels ≥1.1 ng/ml (p = 0.03). Zero patient had a positive bone at PSA levels <1.0 ng/ml, while 4 (27%) were positive at levels ≥1.1 ng/ml (p = 0.01). The diagnostic yield from multimodality imaging, and isotope bone scanning in particular, in PSA levels <1.0 ng/ml, is low. There is a statistically significant increase in the frequency of positive findings on CT and bone scanning at PSA levels ≥1.1 ng/ml. MRI alone is of investigative value at PSA <1.0 ng/ml. The indication for CT, MRI, or isotope bone scanning should be carefully correlated with the clinical question and how it will affect further management.
Dense depth maps from correspondences derived from perceived motion

NASA Astrophysics Data System (ADS)

Kirby, Richard; Whitaker, Ross

2017-01-01

Many computer vision applications require finding corresponding points between images and using the corresponding points to estimate disparity. Today's correspondence finding algorithms primarily use image features or pixel intensities common between image pairs. Some 3-D computer vision applications, however, do not produce the desired results using correspondences derived from image features or pixel intensities. Two examples are the multimodal camera rig and the center region of a coaxial camera rig. We present an image correspondence finding technique that aligns pairs of image sequences using optical flow fields. The optical flow fields provide information about the structure and motion of the scene, which are not available in still images but can be used in image alignment. We apply the technique to a dual focal length stereo camera rig consisting of a visible light-infrared camera pair and to a coaxial camera rig. We test our method on real image sequences and compare our results with the state-of-the-art multimodal and structure from motion (SfM) algorithms. Our method produces more accurate depth and scene velocity reconstruction estimates than the state-of-the-art multimodal and SfM algorithms.
Fast and robust multimodal image registration using a local derivative pattern.

PubMed

Jiang, Dongsheng; Shi, Yonghong; Chen, Xinrong; Wang, Manning; Song, Zhijian

2017-02-01

Deformable multimodal image registration, which can benefit radiotherapy and image guided surgery by providing complementary information, remains a challenging task in the medical image analysis field due to the difficulty of defining a proper similarity measure. This article presents a novel, robust and fast binary descriptor, the discriminative local derivative pattern (dLDP), which is able to encode images of different modalities into similar image representations. dLDP calculates a binary string for each voxel according to the pattern of intensity derivatives in its neighborhood. The descriptor similarity is evaluated using the Hamming distance, which can be efficiently computed, instead of conventional L1 or L2 norms. For the first time, we validated the effectiveness and feasibility of the local derivative pattern for multimodal deformable image registration with several multi-modal registration applications. dLDP was compared with three state-of-the-art methods in artificial image and clinical settings. In the experiments of deformable registration between different magnetic resonance imaging (MRI) modalities from BrainWeb, between computed tomography and MRI images from patient data, and between MRI and ultrasound images from BITE database, we show our method outperforms localized mutual information and entropy images in terms of both accuracy and time efficiency. We have further validated dLDP for the deformable registration of preoperative MRI and three-dimensional intraoperative ultrasound images. Our results indicate that dLDP reduces the average mean target registration error from 4.12 mm to 2.30 mm. This accuracy is statistically equivalent to the accuracy of the state-of-the-art methods in the study; however, in terms of computational complexity, our method significantly outperforms other methods and is even comparable to the sum of the absolute difference. The results reveal that dLDP can achieve superior performance regarding both accuracy and time efficiency in general multimodal image registration. In addition, dLDP also indicates the potential for clinical ultrasound guided intervention. © 2016 The Authors. Medical Physics published by Wiley Periodicals, Inc. on behalf of American Association of Physicists in Medicine.
Gadolinium-Conjugated Gold Nanoshells for Multimodal Diagnostic Imaging and Photothermal Cancer Therapy

PubMed Central

Coughlin, Andrew J.; Ananta, Jeyarama S.; Deng, Nanfu; Larina, Irina V.; Decuzzi, Paolo

2014-01-01

Multimodal imaging offers the potential to improve diagnosis and enhance the specificity of photothermal cancer therapy. Toward this goal, we have engineered gadolinium-conjugated gold nanoshells and demonstrated that they enhance contrast for magnetic resonance imaging, X-Ray, optical coherence tomography, reflectance confocal microscopy, and two-photon luminescence. Additionally, these particles effectively convert near-infrared light to heat, which can be used to ablate cancer cells. Ultimately, these studies demonstrate the potential of gadolinium-nanoshells for image-guided photothermal ablation. PMID:24115690
Cryo-Imaging and Software Platform for Analysis of Molecular MR Imaging of Micrometastases

PubMed Central

Qutaish, Mohammed Q.; Zhou, Zhuxian; Prabhu, David; Liu, Yiqiao; Busso, Mallory R.; Izadnegahdar, Donna; Gargesha, Madhusudhana; Lu, Hong; Lu, Zheng-Rong

2018-01-01

We created and evaluated a preclinical, multimodality imaging, and software platform to assess molecular imaging of small metastases. This included experimental methods (e.g., GFP-labeled tumor and high resolution multispectral cryo-imaging), nonrigid image registration, and interactive visualization of imaging agent targeting. We describe technological details earlier applied to GFP-labeled metastatic tumor targeting by molecular MR (CREKA-Gd) and red fluorescent (CREKA-Cy5) imaging agents. Optimized nonrigid cryo-MRI registration enabled nonambiguous association of MR signals to GFP tumors. Interactive visualization of out-of-RAM volumetric image data allowed one to zoom to a GFP-labeled micrometastasis, determine its anatomical location from color cryo-images, and establish the presence/absence of targeted CREKA-Gd and CREKA-Cy5. In a mouse with >160 GFP-labeled tumors, we determined that in the MR images every tumor in the lung >0.3 mm2 had visible signal and that some metastases as small as 0.1 mm2 were also visible. More tumors were visible in CREKA-Cy5 than in CREKA-Gd MRI. Tape transfer method and nonrigid registration allowed accurate (<11 μm error) registration of whole mouse histology to corresponding cryo-images. Histology showed inflammation and necrotic regions not labeled by imaging agents. This mouse-to-cells multiscale and multimodality platform should uniquely enable more informative and accurate studies of metastatic cancer imaging and therapy. PMID:29805438
Intraoperative imaging-guided cancer surgery: from current fluorescence molecular imaging methods to future multi-modality imaging technology.

PubMed

Chi, Chongwei; Du, Yang; Ye, Jinzuo; Kou, Deqiang; Qiu, Jingdan; Wang, Jiandong; Tian, Jie; Chen, Xiaoyuan

2014-01-01

Cancer is a major threat to human health. Diagnosis and treatment using precision medicine is expected to be an effective method for preventing the initiation and progression of cancer. Although anatomical and functional imaging techniques such as radiography, computed tomography (CT), magnetic resonance imaging (MRI) and positron emission tomography (PET) have played an important role for accurate preoperative diagnostics, for the most part these techniques cannot be applied intraoperatively. Optical molecular imaging is a promising technique that provides a high degree of sensitivity and specificity in tumor margin detection. Furthermore, existing clinical applications have proven that optical molecular imaging is a powerful intraoperative tool for guiding surgeons performing precision procedures, thus enabling radical resection and improved survival rates. However, detection depth limitation exists in optical molecular imaging methods and further breakthroughs from optical to multi-modality intraoperative imaging methods are needed to develop more extensive and comprehensive intraoperative applications. Here, we review the current intraoperative optical molecular imaging technologies, focusing on contrast agents and surgical navigation systems, and then discuss the future prospects of multi-modality imaging technology for intraoperative imaging-guided cancer surgery.
Intraoperative Imaging-Guided Cancer Surgery: From Current Fluorescence Molecular Imaging Methods to Future Multi-Modality Imaging Technology

PubMed Central

Chi, Chongwei; Du, Yang; Ye, Jinzuo; Kou, Deqiang; Qiu, Jingdan; Wang, Jiandong; Tian, Jie; Chen, Xiaoyuan

2014-01-01

Cancer is a major threat to human health. Diagnosis and treatment using precision medicine is expected to be an effective method for preventing the initiation and progression of cancer. Although anatomical and functional imaging techniques such as radiography, computed tomography (CT), magnetic resonance imaging (MRI) and positron emission tomography (PET) have played an important role for accurate preoperative diagnostics, for the most part these techniques cannot be applied intraoperatively. Optical molecular imaging is a promising technique that provides a high degree of sensitivity and specificity in tumor margin detection. Furthermore, existing clinical applications have proven that optical molecular imaging is a powerful intraoperative tool for guiding surgeons performing precision procedures, thus enabling radical resection and improved survival rates. However, detection depth limitation exists in optical molecular imaging methods and further breakthroughs from optical to multi-modality intraoperative imaging methods are needed to develop more extensive and comprehensive intraoperative applications. Here, we review the current intraoperative optical molecular imaging technologies, focusing on contrast agents and surgical navigation systems, and then discuss the future prospects of multi-modality imaging technology for intraoperative imaging-guided cancer surgery. PMID:25250092
Nanoengineered multimodal contrast agent for medical image guidance

NASA Astrophysics Data System (ADS)

Perkins, Gregory J.; Zheng, Jinzi; Brock, Kristy; Allen, Christine; Jaffray, David A.

2005-04-01

Multimodality imaging has gained momentum in radiation therapy planning and image-guided treatment delivery. Specifically, computed tomography (CT) and magnetic resonance (MR) imaging are two complementary imaging modalities often utilized in radiation therapy for visualization of anatomical structures for tumour delineation and accurate registration of image data sets for volumetric dose calculation. The development of a multimodal contrast agent for CT and MR with prolonged in vivo residence time would provide long-lasting spatial and temporal correspondence of the anatomical features of interest, and therefore facilitate multimodal image registration, treatment planning and delivery. The multimodal contrast agent investigated consists of nano-sized stealth liposomes encapsulating conventional iodine and gadolinium-based contrast agents. The average loading achieved was 33.5 +/- 7.1 mg/mL of iodine for iohexol and 9.8 +/- 2.0 mg/mL of gadolinium for gadoteridol. The average liposome diameter was 46.2 +/- 13.5 nm. The system was found to be stable in physiological buffer over a 15-day period, releasing 11.9 +/- 1.1% and 11.2 +/- 0.9% of the total amounts of iohexol and gadoteridol loaded, respectively. 200 minutes following in vivo administration, the contrast agent maintained a relative contrast enhancement of 81.4 +/- 13.05 differential Hounsfield units (ΔHU) in CT (40% decrease from the peak signal value achieved 3 minutes post-injection) and 731.9 +/- 144.2 differential signal intensity (ΔSI) in MR (46% decrease from the peak signal value achieved 3 minutes post-injection) in the blood (aorta), a relative contrast enhancement of 38.0 +/- 5.1 ΔHU (42% decrease from the peak signal value achieved 3 minutes post-injection) and 178.6 +/- 41.4 ΔSI (62% decrease from the peak signal value achieved 3 minutes post-injection) in the liver (parenchyma), a relative contrast enhancement of 9.1 +/- 1.7 ΔHU (94% decrease from the peak signal value achieved 3 minutes post-injection) and 461.7 +/- 78.1 ΔSI (60% decrease from the peak signal value achieved 5 minutes post-injection) in the kidney (cortex) of a New Zealand white rabbit. This multimodal contrast agent, with prolonged in vivo residence time and imaging efficacy, has the potential to bring about improvements in the fields of medical imaging and radiation therapy, particularly for image registration and guidance.
Multimodal Research: Addressing the Complexity of Multimodal Environments and the Challenges for CALL

ERIC Educational Resources Information Center

Tan, Sabine; O'Halloran, Kay L.; Wignell, Peter

2016-01-01

Multimodality, the study of the interaction of language with other semiotic resources such as images and sound resources, has significant implications for computer assisted language learning (CALL) with regards to understanding the impact of digital environments on language teaching and learning. In this paper, we explore recent manifestations of…

Feature-based Alignment of Volumetric Multi-modal Images

PubMed Central

Toews, Matthew; Zöllei, Lilla; Wells, William M.

2014-01-01

This paper proposes a method for aligning image volumes acquired from different imaging modalities (e.g. MR, CT) based on 3D scale-invariant image features. A novel method for encoding invariant feature geometry and appearance is developed, based on the assumption of locally linear intensity relationships, providing a solution to poor repeatability of feature detection in different image modalities. The encoding method is incorporated into a probabilistic feature-based model for multi-modal image alignment. The model parameters are estimated via a group-wise alignment algorithm, that iteratively alternates between estimating a feature-based model from feature data, then realigning feature data to the model, converging to a stable alignment solution with few pre-processing or pre-alignment requirements. The resulting model can be used to align multi-modal image data with the benefits of invariant feature correspondence: globally optimal solutions, high efficiency and low memory usage. The method is tested on the difficult RIRE data set of CT, T1, T2, PD and MP-RAGE brain images of subjects exhibiting significant inter-subject variability due to pathology. PMID:24683955
Multimodal Spectral Imaging of Cells Using a Transmission Diffraction Grating on a Light Microscope

PubMed Central

Isailovic, Dragan; Xu, Yang; Copus, Tyler; Saraswat, Suraj; Nauli, Surya M.

2011-01-01

A multimodal methodology for spectral imaging of cells is presented. The spectral imaging setup uses a transmission diffraction grating on a light microscope to concurrently record spectral images of cells and cellular organelles by fluorescence, darkfield, brightfield, and differential interference contrast (DIC) spectral microscopy. Initially, the setup was applied for fluorescence spectral imaging of yeast and mammalian cells labeled with multiple fluorophores. Fluorescence signals originating from fluorescently labeled biomolecules in cells were collected through triple or single filter cubes, separated by the grating, and imaged using a charge-coupled device (CCD) camera. Cellular components such as nuclei, cytoskeleton, and mitochondria were spatially separated by the fluorescence spectra of the fluorophores present in them, providing detailed multi-colored spectral images of cells. Additionally, the grating-based spectral microscope enabled measurement of scattering and absorption spectra of unlabeled cells and stained tissue sections using darkfield and brightfield or DIC spectral microscopy, respectively. The presented spectral imaging methodology provides a readily affordable approach for multimodal spectral characterization of biological cells and other specimens. PMID:21639978
Brain tissues atrophy is not always the best structural biomarker of physiological aging: A multimodal cross-sectional study.

PubMed

Cherubini, Andrea; Caligiuri, Maria Eugenia; Péran, Patrice; Sabatini, Umberto; Cosentino, Carlo; Amato, Francesco

2015-01-01

This study presents a voxel-based multiple regression analysis of different magnetic resonance image modalities, including anatomical T1-weighted, T2* relaxometry, and diffusion tensor imaging. Quantitative parameters sensitive to complementary brain tissue alterations, including morphometric atrophy, mineralization, microstructural damage, and anisotropy loss, were compared in a linear physiological aging model in 140 healthy subjects (range 20-74 years). The performance of different predictors and the identification of the best biomarker of age-induced structural variation were compared without a priori anatomical knowledge. The best quantitative predictors in several brain regions were iron deposition and microstructural damage, rather than macroscopic tissue atrophy. Age variations were best resolved with a combination of markers, suggesting that multiple predictors better capture age-induced tissue alterations. These findings highlight the importance of a combined evaluation of multimodal biomarkers for the study of aging and point to a number of novel applications for the method described.
Rapid Screening of Cancer Margins in Tissue with Multimodal Confocal Microscopy

PubMed Central

Gareau, Daniel S.; Jeon, Hana; Nehal, Kishwer S.; Rajadhyaksha, Milind

2012-01-01

Background Complete and accurate excision of cancer is guided by the examination of histopathology. However, preparation of histopathology is labor intensive and slow, leading to insufficient sampling of tissue and incomplete and/or inaccurate excision of margins. We demonstrate the potential utility of multimodal confocal mosaicing microscopy for rapid screening of cancer margins, directly in fresh surgical excisions, without the need for conventional embedding, sectioning or processing. Materials/Methods A multimodal confocal mosaicing microscope was developed to image basal cell carcinoma margins in surgical skin excisions, with resolution that shows nuclear detail. Multimodal contrast is with fluorescence for imaging nuclei and reflectance for cellular cytoplasm and dermal collagen. Thirtyfive excisions of basal cell carcinomas from Mohs surgery were imaged, and the mosaics analyzed by comparison to the corresponding frozen pathology. Results Confocal mosaics are produced in about 9 minutes, displaying tissue in fields-of-view of 12 mm with 2X magnification. A digital staining algorithm transforms black and white contrast to purple and pink, which simulates the appearance of standard histopathology. Mosaicing enables rapid digital screening, which mimics the examination of histopathology. Conclusions Multimodal confocal mosaicing microscopy offers a technology platform to potentially enable real-time pathology at the bedside. The imaging may serve as an adjunct to conventional histopathology, to expedite screening of margins and guide surgery toward more complete and accurate excision of cancer. PMID:22721570
Deep Convolutional Neural Networks for Multi-Modality Isointense Infant Brain Image Segmentation

PubMed Central

Zhang, Wenlu; Li, Rongjian; Deng, Houtao; Wang, Li; Lin, Weili; Ji, Shuiwang; Shen, Dinggang

2015-01-01

The segmentation of infant brain tissue images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) plays an important role in studying early brain development in health and disease. In the isointense stage (approximately 6–8 months of age), WM and GM exhibit similar levels of intensity in both T1 and T2 MR images, making the tissue segmentation very challenging. Only a small number of existing methods have been designed for tissue segmentation in this isointense stage; however, they only used a single T1 or T2 images, or the combination of T1 and T2 images. In this paper, we propose to use deep convolutional neural networks (CNNs) for segmenting isointense stage brain tissues using multi-modality MR images. CNNs are a type of deep models in which trainable filters and local neighborhood pooling operations are applied alternatingly on the raw input images, resulting in a hierarchy of increasingly complex features. Specifically, we used multimodality information from T1, T2, and fractional anisotropy (FA) images as inputs and then generated the segmentation maps as outputs. The multiple intermediate layers applied convolution, pooling, normalization, and other operations to capture the highly nonlinear mappings between inputs and outputs. We compared the performance of our approach with that of the commonly used segmentation methods on a set of manually segmented isointense stage brain images. Results showed that our proposed model significantly outperformed prior methods on infant brain tissue segmentation. In addition, our results indicated that integration of multi-modality images led to significant performance improvement. PMID:25562829
Multimodal Imaging in Klippel-Trénaunay-Weber Syndrome: Clinical Photography, Computed Tomoangiography, Infrared Thermography, and 99mTc-Phytate Lymphoscintigraphy.

PubMed

Kim, Su Wan; Song, Heesung

2017-12-01

We report the case of a 19-year-old man who presented with a 12-year history of progressive fatigue, feeling hot, excessive sweating, and numbness in the left arm. He had undergone multimodal imaging and was diagnosed as having Klippel-Trénaunay-Weber syndrome (KTWS). This is a rare congenital disease, defined by combinations of nevus flammeus, venous and lymphatic malformation, and hypertrophy of the affected limbs. Lower extremities are affected mostly. Conventional modalities for evaluating KTWS are ultrasonography, CT, MRI, lymphoscintigraphy, and angiography. There are few reports on multimodal imaging of upper extremities of KTWS patients, and this is the first report of an infrared thermography in KTWS.
Ultrasmall biomolecule-anchored hybrid GdVO4 nanophosphors as a metabolizable multimodal bioimaging contrast agent.

PubMed

Dong, Kai; Ju, Enguo; Liu, Jianhua; Han, Xueli; Ren, Jinsong; Qu, Xiaogang

2014-10-21

Multimodal molecular imaging has recently attracted much attention on disease diagnostics by taking advantage of individual imaging modalities. Herein, we have demonstrated a new paradigm for multimodal bioimaging based on amino acids-anchored ultrasmall lanthanide-doped GdVO4 nanoprobes. On the merit of special metal-cation complexation and abundant functional groups, these amino acids-anchored nanoprobes showed high colloidal stability and excellent dispersibility. Additionally, due to typical paramagnetic behaviour, high X-ray mass absorption coefficient and strong fluorescence, these nanoprobes would provide a unique opportunity to develop multifunctional probes for MRI, CT and luminescence imaging. More importantly, the small size and biomolecular coatings endow the nanoprobes with effective metabolisability and high biocompatibility. With the superior stability, high biocompatibility, effective metabolisability and excellent contrast performance, amino acids-capped GdVO4:Eu(3+) nanocastings are a promising candidate as multimodal contrast agents and would bring more opportunities for biological and medical applications with further modifications.
Direct visualization of gastrointestinal tract with lanthanide-doped BaYbF5 upconversion nanoprobes.

PubMed

Liu, Zhen; Ju, Enguo; Liu, Jianhua; Du, Yingda; Li, Zhengqiang; Yuan, Qinghai; Ren, Jinsong; Qu, Xiaogang

2013-10-01

Nanoparticulate contrast agents have attracted a great deal of attention along with the rapid development of modern medicine. Here, a binary contrast agent based on PAA modified BaYbF5:Tm nanoparticles for direct visualization of gastrointestinal (GI) tract has been designed and developed via a one-pot solvothermal route. By taking advantages of excellent colloidal stability, low cytotoxicity, and neglectable hemolysis of these well-designed nanoparticles, their feasibility as a multi-modal contrast agent for GI tract was intensively investigated. Significant enhancement of contrast efficacy relative to clinical barium meal and iodine-based contrast agent was evaluated via X-ray imaging and CT imaging in vivo. By doping Tm(3+) ions into these nanoprobes, in vivo NIR-NIR imaging was then demonstrated. Unlike some invasive imaging modalities, non-invasive imaging strategy including X-ray imaging, CT imaging, and UCL imaging for GI tract could extremely reduce the painlessness to patients, effectively facilitate imaging procedure, as well as rationality economize diagnostic time. Critical to clinical applications, long-term toxicity of our contrast agent was additionally investigated in detail, indicating their overall safety. Based on our results, PAA-BaYbF5:Tm nanoparticles were the excellent multi-modal contrast agent to integrate X-ray imaging, CT imaging, and UCL imaging for direct visualization of GI tract with low systemic toxicity. Copyright © 2013 Elsevier Ltd. All rights reserved.
Development of computational small animal models and their applications in preclinical imaging and therapy research.

PubMed

Xie, Tianwu; Zaidi, Habib

2016-01-01

The development of multimodality preclinical imaging techniques and the rapid growth of realistic computer simulation tools have promoted the construction and application of computational laboratory animal models in preclinical research. Since the early 1990s, over 120 realistic computational animal models have been reported in the literature and used as surrogates to characterize the anatomy of actual animals for the simulation of preclinical studies involving the use of bioluminescence tomography, fluorescence molecular tomography, positron emission tomography, single-photon emission computed tomography, microcomputed tomography, magnetic resonance imaging, and optical imaging. Other applications include electromagnetic field simulation, ionizing and nonionizing radiation dosimetry, and the development and evaluation of new methodologies for multimodality image coregistration, segmentation, and reconstruction of small animal images. This paper provides a comprehensive review of the history and fundamental technologies used for the development of computational small animal models with a particular focus on their application in preclinical imaging as well as nonionizing and ionizing radiation dosimetry calculations. An overview of the overall process involved in the design of these models, including the fundamental elements used for the construction of different types of computational models, the identification of original anatomical data, the simulation tools used for solving various computational problems, and the applications of computational animal models in preclinical research. The authors also analyze the characteristics of categories of computational models (stylized, voxel-based, and boundary representation) and discuss the technical challenges faced at the present time as well as research needs in the future.
Relapse of choroidal neovascularization in Bietti's crystalline retinopathy following anti-vascular endothelial growth factor therapy: A case report.

PubMed

Hua, Rui; Chen, Kang; Hu, Yuedong; Wang, Xinling; Chen, Lei

2015-11-01

Choroidal neovascularization secondary to retinitis pigmentosa is rarely observed in clinical practice. The present study describes a case of atypical retinitis pigmentosa, crystalline retinal pigmentary degeneration, complicated by choroidal neovascularization (CNV) in a 26-year-old man presenting with blurred vision in the right eye. Heidelberg multimodality imaging was performed to achieve a confirmed diagnosis. Bevacizumab was injected once intravitreally. The 3-month follow-up included visualization of the lesion's regression with spectral domain optical coherence tomography (SD-OCT). However, at 3 months after the injection, the CNV reoccurred. To the best of our knowledge, this is the first time that a case of CNV secondary to retinitis pigmentosa, in which the diagnosis was confirmed via multimodality imaging and the therapeutic efficacy was evaluated by SD-OCT, has been reported in China.
In vivo features of melanocytic lesions: multimode hyperspectral dermoscopy, reflectance confocal microscopy, and histopathologic correlates (Conference Presentation)

NASA Astrophysics Data System (ADS)

Vasefi, Fartash; MacKinnon, Nicholas B.; Jain, Manu; Cordova, Miguel A.; Kose, Kivanc; Rajadhyaksha, Milind; Halpern, Allan C.; Farkas, Daniel L.

2017-02-01

Motivation and background: Melanoma, the fastest growing cancer worldwide, kills more than one person every hour in the United States. Determining the depth and distribution of dermal melanin and hemoglobin adds physio-morphologic information to the current diagnostic standard, cellular morphology, to further develop noninvasive methods to discriminate between melanoma and benign skin conditions. Purpose: To compare the performance of a multimode dermoscopy system (SkinSpect), which is designed to quantify and map in three dimensions, in vivo melanin and hemoglobin in skin, and to validate this with histopathology and three dimensional reflectance confocal microscopy (RCM) imaging. Methods: Sequentially capture SkinSpect and RCM images of suspect lesions and nearby normal skin and compare this with histopathology reports, RCM imaging allows noninvasive observation of nuclear, cellular and structural detail in 1-5 μm-thin optical sections in skin, and detection of pigmented skin lesions with sensitivity of 90-95% and specificity of 70-80%. The multimode imaging dermoscope combines polarization (cross and parallel), autofluorescence and hyperspectral imaging to noninvasively map the distribution of melanin, collagen and hemoglobin oxygenation in pigmented skin lesions. Results: We compared in vivo features of ten melanocytic lesions extracted by SkinSpect and RCM imaging, and correlated them to histopathologic results. We present results of two melanoma cases (in situ and invasive), and compare with in vivo features from eight benign lesions. Melanin distribution at different depths and hemodynamics, including abnormal vascularity, detected by both SkinSpect and RCM will be discussed. Conclusion: Diagnostic features such as dermal melanin and hemoglobin concentration provided in SkinSpect skin analysis for melanoma and normal pigmented lesions can be compared and validated using results from RCM and histopathology.
Nanoparticles in magnetic resonance imaging: from simple to dual contrast agents

PubMed Central

Estelrich, Joan; Sánchez-Martín, María Jesús; Busquets, Maria Antònia

2015-01-01

Magnetic resonance imaging (MRI) has become one of the most widely used and powerful tools for noninvasive clinical diagnosis owing to its high degree of soft tissue contrast, spatial resolution, and depth of penetration. MRI signal intensity is related to the relaxation times (T1, spin–lattice relaxation and T2, spin–spin relaxation) of in vivo water protons. To increase contrast, various inorganic nanoparticles and complexes (the so-called contrast agents) are administered prior to the scanning. Shortening T1 and T2 increases the corresponding relaxation rates, 1/T1 and 1/T2, producing hyperintense and hypointense signals respectively in shorter times. Moreover, the signal-to-noise ratio can be improved with the acquisition of a large number of measurements. The contrast agents used are generally based on either iron oxide nanoparticles or ferrites, providing negative contrast in T2-weighted images; or complexes of lanthanide metals (mostly containing gadolinium ions), providing positive contrast in T1-weighted images. Recently, lanthanide complexes have been immobilized in nanostructured materials in order to develop a new class of contrast agents with functions including blood-pool and organ (or tumor) targeting. Meanwhile, to overcome the limitations of individual imaging modalities, multimodal imaging techniques have been developed. An important challenge is to design all-in-one contrast agents that can be detected by multimodal techniques. Magnetoliposomes are efficient multimodal contrast agents. They can simultaneously bear both kinds of contrast and can, furthermore, incorporate targeting ligands and chains of polyethylene glycol to enhance the accumulation of nanoparticles at the site of interest and the bioavailability, respectively. Here, we review the most important characteristics of the nanoparticles or complexes used as MRI contrast agents. PMID:25834422
Clinical applications of imaging biomarkers. Part 2. The neurosurgeon's perspective

PubMed Central

Brodbelt, A

2011-01-01

Advances in imaging, including multivoxel spectroscopy, tractography, functional MRI and positron emission spectroscopy, are being used by neurosurgeons to target aggressive areas in gliomas, and to help identify tumour boundaries, functional areas and tracts. Neuro-oncological surgeons need to understand these techniques to help maximise tumour resection, while minimising morbidity in an attempt to improve the quality of patient outcome. This article reviews the evidence for the practical use of multimodal imaging in modern glioma surgery. PMID:22433829
Robotics in neurosurgery: which tools for what?

PubMed

Benabid, A L; Hoffmann, D; Seigneuret, E; Chabardes, S

2006-01-01

Robots are the tools for taking advantage of the skills of computers in achieving complicated tasks. This has been made possible owing to the "numerical image explosion" which allowed us to easily obtain spatial coordinates, three dimensional reconstruction, multimodality imaging including digital subtraction angiography (DSA), computed tomography (CT), magnetic resonance imaging (MRI) and magneto encephalography (MEG), with high resolution in space, time, and tissue density. Neurosurgical robots currently available at the operating level are being described. Future evolutions, indications and ethical aspects are examined.
Multimodal adaptive optics for depth-enhanced high-resolution ophthalmic imaging

NASA Astrophysics Data System (ADS)

Hammer, Daniel X.; Mujat, Mircea; Iftimia, Nicusor V.; Lue, Niyom; Ferguson, R. Daniel

2010-02-01

We developed a multimodal adaptive optics (AO) retinal imager for diagnosis of retinal diseases, including glaucoma, diabetic retinopathy (DR), age-related macular degeneration (AMD), and retinitis pigmentosa (RP). The development represents the first ever high performance AO system constructed that combines AO-corrected scanning laser ophthalmoscopy (SLO) and swept source Fourier domain optical coherence tomography (SSOCT) imaging modes in a single compact clinical prototype platform. The SSOCT channel operates at a wavelength of 1 μm for increased penetration and visualization of the choriocapillaris and choroid, sites of major disease activity for DR and wet AMD. The system is designed to operate on a broad clinical population with a dual deformable mirror (DM) configuration that allows simultaneous low- and high-order aberration correction. The system also includes a wide field line scanning ophthalmoscope (LSO) for initial screening, target identification, and global orientation; an integrated retinal tracker (RT) to stabilize the SLO, OCT, and LSO imaging fields in the presence of rotational eye motion; and a high-resolution LCD-based fixation target for presentation to the subject of stimuli and other visual cues. The system was tested in a limited number of human subjects without retinal disease for performance optimization and validation. The system was able to resolve and quantify cone photoreceptors across the macula to within ~0.5 deg (~100-150 μm) of the fovea, image and delineate ten retinal layers, and penetrate to resolve targets deep into the choroid. In addition to instrument hardware development, analysis algorithms were developed for efficient information extraction from clinical imaging sessions, with functionality including automated image registration, photoreceptor counting, strip and montage stitching, and segmentation. The system provides clinicians and researchers with high-resolution, high performance adaptive optics imaging to help guide therapies, develop new drugs, and improve patient outcomes.
TU-G-303-03: Machine Learning to Improve Human Learning From Longitudinal Image Sets

DOE Office of Scientific and Technical Information (OSTI.GOV)

Veeraraghavan, H.

‘Radiomics’ refers to studies that extract a large amount of quantitative information from medical imaging studies as a basis for characterizing a specific aspect of patient health. Radiomics models can be built to address a wide range of outcome predictions, clinical decisions, basic cancer biology, etc. For example, radiomics models can be built to predict the aggressiveness of an imaged cancer, cancer gene expression characteristics (radiogenomics), radiation therapy treatment response, etc. Technically, radiomics brings together quantitative imaging, computer vision/image processing, and machine learning. In this symposium, speakers will discuss approaches to radiomics investigations, including: longitudinal radiomics, radiomics combined with othermore » biomarkers (‘pan-omics’), radiomics for various imaging modalities (CT, MRI, and PET), and the use of registered multi-modality imaging datasets as a basis for radiomics. There are many challenges to the eventual use of radiomics-derived methods in clinical practice, including: standardization and robustness of selected metrics, accruing the data required, building and validating the resulting models, registering longitudinal data that often involve significant patient changes, reliable automated cancer segmentation tools, etc. Despite the hurdles, results achieved so far indicate the tremendous potential of this general approach to quantifying and using data from medical images. Specific applications of radiomics to be presented in this symposium will include: the longitudinal analysis of patients with low-grade gliomas; automatic detection and assessment of patients with metastatic bone lesions; image-based monitoring of patients with growing lymph nodes; predicting radiotherapy outcomes using multi-modality radiomics; and studies relating radiomics with genomics in lung cancer and glioblastoma. Learning Objectives: Understanding the basic image features that are often used in radiomic models. Understanding requirements for reliable radiomic models, including robustness of metrics, adequate predictive accuracy, and generalizability. Understanding the methodology behind radiomic-genomic (’radiogenomics’) correlations. Research supported by NIH (US), CIHR (Canada), and NSERC (Canada)« less
Multimodality imaging of placental masses: a pictorial review.

PubMed

Jha, Priyanka; Paroder, Viktoriya; Mar, Winnie; Horowtiz, Jeanne M; Poder, Liina

2016-12-01

Placental masses are uncommonly identified at the time of obstetric ultrasound evaluation. Understanding the pathologies presenting as placental masses is key for providing a differential diagnosis and guiding subsequent management, which may include additional imaging with magnetic resonance (MR) imaging. Potential benign entities include chorioangiomas and teratomas. Larger chorioangiomas can cause fetal cardiovascular issues from volume overload. Placental mesenchymal dysplasia has an association with fetal anomalies and detailed fetal evaluation should be performed when it is suspected. Identifying other cystic masses such as partial and complete moles is crucial to prevent erroneous pregnancy termination. This review addresses normal imaging appearance of the placenta on ultrasound and MR imaging and describes various trophoblastic and nontrophoblastic placental masses. Potential placental mass mimics including uterine contractions and thrombo-hematomas are also presented.
Benign familial fleck retina: multimodal imaging including optical coherence tomography angiography.

PubMed

Garcia, Jose Mauricio Botto de Barros; Isaac, David Leonardo Cruvinel; Sardeiro, Tainara; Aquino, Érika; Avila, Marcos

2017-01-01

This report presents multimodal imaging of a 27-year-old woman diagnosed with benign familial fleck retina (OMIM 228980), an uncommon disorder. Fundus photographs revealed retinal flecks that affected her post-equatorial retina but spared the macular area. Fundus autofluorescence and infrared imaging demonstrated a symmetrical pattern of yellow-white fleck lesions that affected both eyes. Her full-field electroretinogram and electrooculogram were normal. An optical coherence tomography B-scan was performed for both eyes, revealing increased thickness of the retinal pigmented epithelium leading to multiple small pigmented epithelium detachments. The outer retina remained intact in both eyes. Spectral-domain optical coherence tomography angiography with split-spectrum amplitude decorrelation algorithm and 3 × 3 mm structural en face optical coherence tomography did not show macular lesions. Benign familial fleck retina belongs to a heterogenous group of so-called flecked retina syndromes, and should be considered in patients with yellowish-white retinal lesions without involvement of the macula.
Multifunctional nanoparticles for upconversion luminescence/MR multimodal imaging and magnetically targeted photothermal therapy.

PubMed

Cheng, Liang; Yang, Kai; Li, Yonggang; Zeng, Xiao; Shao, Mingwang; Lee, Shuit-Tong; Liu, Zhuang

2012-03-01

Theranostics, the combination of diagnostics and therapies, has become a new concept in the battles with various major diseases such as cancer. Herein, we develop multifunctional nanoparticles (MFNPs) with highly integrated functionalities including upconversion luminescence, superparamagnetism, and strong optical absorption in the near-infrared (NIR) region with high photostability. In vivo dual modal optical/magnetic resonance imaging of mice uncovers that by placing a magnet nearby the tumor, MFNPs tend to migrate toward the tumor after intravenous injection and show high tumor accumulation, which is ~8 folds higher than that without magnetic targeting. NIR laser irradiation is then applied to the tumors grown on MFNP-injected mice under magnetic tumor-targeting, obtaining an outstanding photothermal therapeutic efficacy with 100% of tumor elimination in a murine breast cancer model. We present here a strategy for multimodal imaging-guided, magnetically targeted physical cancer therapy and highlight the promise of using multifunctional nanostructures for cancer theranostics. Copyright © 2011 Elsevier Ltd. All rights reserved.
Integrated scanning laser ophthalmoscopy and optical coherence tomography for quantitative multimodal imaging of retinal degeneration and autofluorescence

NASA Astrophysics Data System (ADS)

Issaei, Ali; Szczygiel, Lukasz; Hossein-Javaheri, Nima; Young, Mei; Molday, L. L.; Molday, R. S.; Sarunic, M. V.

2011-03-01

Scanning Laser Ophthalmoscopy (SLO) and Coherence Tomography (OCT) are complimentary retinal imaging modalities. Integration of SLO and OCT allows for both fluorescent detection and depth- resolved structural imaging of the retinal cell layers to be performed in-vivo. System customization is required to image rodents used in medical research by vision scientists. We are investigating multimodal SLO/OCT imaging of a rodent model of Stargardt's Macular Dystrophy which is characterized by retinal degeneration and accumulation of toxic autofluorescent lipofuscin deposits. Our new findings demonstrate the ability to track fundus autofluorescence and retinal degeneration concurrently.

Multimodal Image Alignment via Linear Mapping between Feature Modalities.

PubMed

Jiang, Yanyun; Zheng, Yuanjie; Hou, Sujuan; Chang, Yuchou; Gee, James

2017-01-01

We propose a novel landmark matching based method for aligning multimodal images, which is accomplished uniquely by resolving a linear mapping between different feature modalities. This linear mapping results in a new measurement on similarity of images captured from different modalities. In addition, our method simultaneously solves this linear mapping and the landmark correspondences by minimizing a convex quadratic function. Our method can estimate complex image relationship between different modalities and nonlinear nonrigid spatial transformations even in the presence of heavy noise, as shown in our experiments carried out by using a variety of image modalities.
Medical image registration based on normalized multidimensional mutual information

NASA Astrophysics Data System (ADS)

Li, Qi; Ji, Hongbing; Tong, Ming

2009-10-01

Registration of medical images is an essential research topic in medical image processing and applications, and especially a preliminary and key step for multimodality image fusion. This paper offers a solution to medical image registration based on normalized multi-dimensional mutual information. Firstly, affine transformation with translational and rotational parameters is applied to the floating image. Then ordinal features are extracted by ordinal filters with different orientations to represent spatial information in medical images. Integrating ordinal features with pixel intensities, the normalized multi-dimensional mutual information is defined as similarity criterion to register multimodality images. Finally the immune algorithm is used to search registration parameters. The experimental results demonstrate the effectiveness of the proposed registration scheme.
Magnetic nanobubbles with potential for targeted drug delivery and trimodal imaging in breast cancer: an in vitro study.

PubMed

Song, Weixiang; Luo, Yindeng; Zhao, Yajing; Liu, Xinjie; Zhao, Jiannong; Luo, Jie; Zhang, Qunxia; Ran, Haitao; Wang, Zhigang; Guo, Dajing

2017-05-01

The aim of this study was to improve tumor-targeted therapy for breast cancer by designing magnetic nanobubbles with the potential for targeted drug delivery and multimodal imaging. Herceptin-decorated and ultrasmall superparamagnetic iron oxide (USPIO)/paclitaxel (PTX)-embedded nanobubbles (PTX-USPIO-HER-NBs) were manufactured by combining a modified double-emulsion evaporation process with carbodiimide technique. PTX-USPIO-HER-NBs were examined for characterization, specific cell-targeting ability and multimodal imaging. PTX-USPIO-HER-NBs exhibited excellent entrapment efficiency of Herceptin/PTX/USPIO and showed greater cytotoxic effects than other delivery platforms. Low-frequency ultrasound triggered accelerated PTX release. Moreover, the magnetic nanobubbles were able to enhance ultrasound, magnetic resonance and photoacoustics trimodal imaging. These results suggest that PTX-USPIO-HER-NBs have potential as a multimodal contrast agent and as a system for ultrasound-triggered drug release in breast cancer.
Spiking Cortical Model Based Multimodal Medical Image Fusion by Combining Entropy Information with Weber Local Descriptor

PubMed Central

Zhang, Xuming; Ren, Jinxia; Huang, Zhiwen; Zhu, Fei

2016-01-01

Multimodal medical image fusion (MIF) plays an important role in clinical diagnosis and therapy. Existing MIF methods tend to introduce artifacts, lead to loss of image details or produce low-contrast fused images. To address these problems, a novel spiking cortical model (SCM) based MIF method has been proposed in this paper. The proposed method can generate high-quality fused images using the weighting fusion strategy based on the firing times of the SCM. In the weighting fusion scheme, the weight is determined by combining the entropy information of pulse outputs of the SCM with the Weber local descriptor operating on the firing mapping images produced from the pulse outputs. The extensive experiments on multimodal medical images show that compared with the numerous state-of-the-art MIF methods, the proposed method can preserve image details very well and avoid the introduction of artifacts effectively, and thus it significantly improves the quality of fused images in terms of human vision and objective evaluation criteria such as mutual information, edge preservation index, structural similarity based metric, fusion quality index, fusion similarity metric and standard deviation. PMID:27649190
Spiking Cortical Model Based Multimodal Medical Image Fusion by Combining Entropy Information with Weber Local Descriptor.

PubMed

Zhang, Xuming; Ren, Jinxia; Huang, Zhiwen; Zhu, Fei

2016-09-15

Multimodal medical image fusion (MIF) plays an important role in clinical diagnosis and therapy. Existing MIF methods tend to introduce artifacts, lead to loss of image details or produce low-contrast fused images. To address these problems, a novel spiking cortical model (SCM) based MIF method has been proposed in this paper. The proposed method can generate high-quality fused images using the weighting fusion strategy based on the firing times of the SCM. In the weighting fusion scheme, the weight is determined by combining the entropy information of pulse outputs of the SCM with the Weber local descriptor operating on the firing mapping images produced from the pulse outputs. The extensive experiments on multimodal medical images show that compared with the numerous state-of-the-art MIF methods, the proposed method can preserve image details very well and avoid the introduction of artifacts effectively, and thus it significantly improves the quality of fused images in terms of human vision and objective evaluation criteria such as mutual information, edge preservation index, structural similarity based metric, fusion quality index, fusion similarity metric and standard deviation.
Exogenous Molecular Probes for Targeted Imaging in Cancer: Focus on Multi-modal Imaging

PubMed Central

Joshi, Bishnu P.; Wang, Thomas D.

2010-01-01

Cancer is one of the major causes of mortality and morbidity in our healthcare system. Molecular imaging is an emerging methodology for the early detection of cancer, guidance of therapy, and monitoring of response. The development of new instruments and exogenous molecular probes that can be labeled for multi-modality imaging is critical to this process. Today, molecular imaging is at a crossroad, and new targeted imaging agents are expected to broadly expand our ability to detect and manage cancer. This integrated imaging strategy will permit clinicians to not only localize lesions within the body but also to manage their therapy by visualizing the expression and activity of specific molecules. This information is expected to have a major impact on drug development and understanding of basic cancer biology. At this time, a number of molecular probes have been developed by conjugating various labels to affinity ligands for targeting in different imaging modalities. This review will describe the current status of exogenous molecular probes for optical, scintigraphic, MRI and ultrasound imaging platforms. Furthermore, we will also shed light on how these techniques can be used synergistically in multi-modal platforms and how these techniques are being employed in current research. PMID:22180839
Multimodal Task-Driven Dictionary Learning for Image Classification

DTIC Science & Technology

2015-12-18

1 Multimodal Task-Driven Dictionary Learning for Image Classification Soheil Bahrampour, Student Member, IEEE, Nasser M. Nasrabadi, Fellow, IEEE...Asok Ray, Fellow, IEEE, and W. Kenneth Jenkins, Life Fellow, IEEE Abstract— Dictionary learning algorithms have been suc- cessfully used for both...reconstructive and discriminative tasks, where an input signal is represented with a sparse linear combination of dictionary atoms. While these methods are
MindSeer: a portable and extensible tool for visualization of structural and functional neuroimaging data

PubMed Central

Moore, Eider B; Poliakov, Andrew V; Lincoln, Peter; Brinkley, James F

2007-01-01

Background Three-dimensional (3-D) visualization of multimodality neuroimaging data provides a powerful technique for viewing the relationship between structure and function. A number of applications are available that include some aspect of 3-D visualization, including both free and commercial products. These applications range from highly specific programs for a single modality, to general purpose toolkits that include many image processing functions in addition to visualization. However, few if any of these combine both stand-alone and remote multi-modality visualization in an open source, portable and extensible tool that is easy to install and use, yet can be included as a component of a larger information system. Results We have developed a new open source multimodality 3-D visualization application, called MindSeer, that has these features: integrated and interactive 3-D volume and surface visualization, Java and Java3D for true cross-platform portability, one-click installation and startup, integrated data management to help organize large studies, extensibility through plugins, transparent remote visualization, and the ability to be integrated into larger information management systems. We describe the design and implementation of the system, as well as several case studies that demonstrate its utility. These case studies are available as tutorials or demos on the associated website: . Conclusion MindSeer provides a powerful visualization tool for multimodality neuroimaging data. Its architecture and unique features also allow it to be extended into other visualization domains within biomedicine. PMID:17937818
MindSeer: a portable and extensible tool for visualization of structural and functional neuroimaging data.

PubMed

Moore, Eider B; Poliakov, Andrew V; Lincoln, Peter; Brinkley, James F

2007-10-15

Three-dimensional (3-D) visualization of multimodality neuroimaging data provides a powerful technique for viewing the relationship between structure and function. A number of applications are available that include some aspect of 3-D visualization, including both free and commercial products. These applications range from highly specific programs for a single modality, to general purpose toolkits that include many image processing functions in addition to visualization. However, few if any of these combine both stand-alone and remote multi-modality visualization in an open source, portable and extensible tool that is easy to install and use, yet can be included as a component of a larger information system. We have developed a new open source multimodality 3-D visualization application, called MindSeer, that has these features: integrated and interactive 3-D volume and surface visualization, Java and Java3D for true cross-platform portability, one-click installation and startup, integrated data management to help organize large studies, extensibility through plugins, transparent remote visualization, and the ability to be integrated into larger information management systems. We describe the design and implementation of the system, as well as several case studies that demonstrate its utility. These case studies are available as tutorials or demos on the associated website: http://sig.biostr.washington.edu/projects/MindSeer. MindSeer provides a powerful visualization tool for multimodality neuroimaging data. Its architecture and unique features also allow it to be extended into other visualization domains within biomedicine.
Learning of Multimodal Representations With Random Walks on the Click Graph.

PubMed

Wu, Fei; Lu, Xinyan; Song, Jun; Yan, Shuicheng; Zhang, Zhongfei Mark; Rui, Yong; Zhuang, Yueting

2016-02-01

In multimedia information retrieval, most classic approaches tend to represent different modalities of media in the same feature space. With the click data collected from the users' searching behavior, existing approaches take either one-to-one paired data (text-image pairs) or ranking examples (text-query-image and/or image-query-text ranking lists) as training examples, which do not make full use of the click data, particularly the implicit connections among the data objects. In this paper, we treat the click data as a large click graph, in which vertices are images/text queries and edges indicate the clicks between an image and a query. We consider learning a multimodal representation from the perspective of encoding the explicit/implicit relevance relationship between the vertices in the click graph. By minimizing both the truncated random walk loss as well as the distance between the learned representation of vertices and their corresponding deep neural network output, the proposed model which is named multimodal random walk neural network (MRW-NN) can be applied to not only learn robust representation of the existing multimodal data in the click graph, but also deal with the unseen queries and images to support cross-modal retrieval. We evaluate the latent representation learned by MRW-NN on a public large-scale click log data set Clickture and further show that MRW-NN achieves much better cross-modal retrieval performance on the unseen queries/images than the other state-of-the-art methods.
Novel multifunctional theranostic liposome drug delivery system: construction, characterization, and multimodality MR, near-infrared fluorescent, and nuclear imaging.

PubMed

Li, Shihong; Goins, Beth; Zhang, Lujun; Bao, Ande

2012-06-20

Liposomes are effective lipid nanoparticle drug delivery systems, which can also be functionalized with noninvasive multimodality imaging agents with each modality providing distinct information and having synergistic advantages in diagnosis, monitoring of disease treatment, and evaluation of liposomal drug pharmacokinetics. We designed and constructed a multifunctional theranostic liposomal drug delivery system, which integrated multimodality magnetic resonance (MR), near-infrared (NIR) fluorescent and nuclear imaging of liposomal drug delivery, and therapy monitoring and prediction. The premanufactured liposomes were composed of DSPC/cholesterol/Gd-DOTA-DSPE/DOTA-DSPE with the molar ratio of 39:35:25:1 and having ammonium sulfate/pH gradient. A lipidized NIR fluorescent tracer, IRDye-DSPE, was effectively postinserted into the premanufactured liposomes. Doxorubicin could be effectively postloaded into the multifunctional liposomes. The multifunctional doxorubicin-liposomes could also be stably radiolabeled with (99m)Tc or (64)Cu for single-photon emission computed tomography (SPECT) or positron emission tomography (PET) imaging, respectively. MR images displayed the high-resolution micro-intratumoral distribution of the liposomes in squamous cell carcinoma of head and neck (SCCHN) tumor xenografts in nude rats after intratumoral injection. NIR fluorescent, SPECT, and PET images also clearly showed either the high intratumoral retention or distribution of the multifunctional liposomes. This multifunctional drug carrying liposome system is promising for disease theranostics allowing noninvasive multimodality NIR fluorescent, MR, SPECT, and PET imaging of their in vivo behavior and capitalizing on the inherent advantages of each modality.
A semantic model for multimodal data mining in healthcare information systems.

PubMed

Iakovidis, Dimitris; Smailis, Christos

2012-01-01

Electronic health records (EHRs) are representative examples of multimodal/multisource data collections; including measurements, images and free texts. The diversity of such information sources and the increasing amounts of medical data produced by healthcare institutes annually, pose significant challenges in data mining. In this paper we present a novel semantic model that describes knowledge extracted from the lowest-level of a data mining process, where information is represented by multiple features i.e. measurements or numerical descriptors extracted from measurements, images, texts or other medical data, forming multidimensional feature spaces. Knowledge collected by manual annotation or extracted by unsupervised data mining from one or more feature spaces is modeled through generalized qualitative spatial semantics. This model enables a unified representation of knowledge across multimodal data repositories. It contributes to bridging the semantic gap, by enabling direct links between low-level features and higher-level concepts e.g. describing body parts, anatomies and pathological findings. The proposed model has been developed in web ontology language based on description logics (OWL-DL) and can be applied to a variety of data mining tasks in medical informatics. It utility is demonstrated for automatic annotation of medical data.
Cell-permeable Ln(III) chelate-functionalized InP quantum dots as multimodal imaging agents.

PubMed

Stasiuk, Graeme J; Tamang, Sudarsan; Imbert, Daniel; Poillot, Cathy; Giardiello, Marco; Tisseyre, Céline; Barbier, Emmanuel L; Fries, Pascal Henry; de Waard, Michel; Reiss, Peter; Mazzanti, Marinella

2011-10-25

Quantum dots (QDs) are ideal scaffolds for the development of multimodal imaging agents, but their application in clinical diagnostics is limited by the toxicity of classical CdSe QDs. A new bimodal MRI/optical nanosized contrast agent with high gadolinium payload has been prepared through direct covalent attachment of up to 80 Gd(III) chelates on fluorescent nontoxic InP/ZnS QDs. It shows a high relaxivity of 900 mM(-1) s(-1) (13 mM(-1 )s(-1) per Gd ion) at 35 MHz (0.81 T) and 298 K, while the bright luminescence of the QDs is preserved. Eu(III) and Tb(III) chelates were also successfully grafted to the InP/ZnS QDs. The absence of energy transfer between the QD and lanthanide emitting centers results in a multicolor system. Using this convenient direct grafting strategy additional targeting ligands can be included on the QD. Here a cell-penetrating peptide has been co-grafted in a one-pot reaction to afford a cell-permeable multimodal multimeric MRI contrast agent that reports cellular localization by fluorescence and provides high relaxivity and increased tissue retention with respect to commercial contrast agents.
How acute and chronic alcohol consumption affects brain networks: insights from multimodal neuroimaging.

PubMed

Schulte, Tilman; Oberlin, Brandon G; Kareken, David A; Marinkovic, Ksenija; Müller-Oehring, Eva M; Meyerhoff, Dieter J; Tapert, Susan

2012-12-01

Multimodal imaging combining 2 or more techniques is becoming increasingly important because no single imaging approach has the capacity to elucidate all clinically relevant characteristics of a network. This review highlights recent advances in multimodal neuroimaging (i.e., combined use and interpretation of data collected through magnetic resonance imaging [MRI], functional MRI, diffusion tensor imaging, positron emission tomography, magnetoencephalography, MR perfusion, and MR spectroscopy methods) that leads to a more comprehensive understanding of how acute and chronic alcohol consumption affect neural networks underlying cognition, emotion, reward processing, and drinking behavior. Several innovative investigators have started utilizing multiple imaging approaches within the same individual to better understand how alcohol influences brain systems, both during intoxication and after years of chronic heavy use. Their findings can help identify mechanism-based therapeutic and pharmacological treatment options, and they may increase the efficacy and cost effectiveness of such treatments by predicting those at greatest risk for relapse. Copyright © 2012 by the Research Society on Alcoholism.
Dye-Enhanced Multimodal Confocal Imaging of Brain Cancers

NASA Astrophysics Data System (ADS)

Wirth, Dennis; Snuderl, Matija; Sheth, Sameer; Curry, William; Yaroslavsky, Anna

2011-04-01

Background and Significance: Accurate high resolution intraoperative detection of brain tumors may result in improved patient survival and better quality of life. The goal of this study was to evaluate dye enhanced multimodal confocal imaging for discriminating normal and cancerous brain tissue. Materials and Methods: Fresh thick brain specimens were obtained from the surgeries. Normal and cancer tissues were investigated. Samples were stained in methylene blue and imaged. Reflectance and fluorescence signals were excited at 658nm. Fluorescence emission and polarization were registered from 670 nm to 710 nm. The system provided lateral resolution of 0.6 μm and axial resolution of 7 μm. Normal and cancer specimens exhibited distinctively different characteristics. H&E histopathology was processed from each imaged sample. Results and Conclusions: The analysis of normal and cancerous tissues indicated clear differences in appearance in both the reflectance and fluorescence responses. These results confirm the feasibility of multimodal confocal imaging for intraoperative detection of small cancer nests and cells.
Combined multi-modal photoacoustic tomography, optical coherence tomography (OCT) and OCT angiography system with an articulated probe for in vivo human skin structure and vasculature imaging

PubMed Central

Liu, Mengyang; Chen, Zhe; Zabihian, Behrooz; Sinz, Christoph; Zhang, Edward; Beard, Paul C.; Ginner, Laurin; Hoover, Erich; Minneman, Micheal P.; Leitgeb, Rainer A.; Kittler, Harald; Drexler, Wolfgang

2016-01-01

Cutaneous blood flow accounts for approximately 5% of cardiac output in human and plays a key role in a number of a physiological and pathological processes. We show for the first time a multi-modal photoacoustic tomography (PAT), optical coherence tomography (OCT) and OCT angiography system with an articulated probe to extract human cutaneous vasculature in vivo in various skin regions. OCT angiography supplements the microvasculature which PAT alone is unable to provide. Co-registered volumes for vessel network is further embedded in the morphologic image provided by OCT. This multi-modal system is therefore demonstrated as a valuable tool for comprehensive non-invasive human skin vasculature and morphology imaging in vivo. PMID:27699106
Multimodal Nonlinear Optical Microscopy

PubMed Central

Yue, Shuhua; Slipchenko, Mikhail N.; Cheng, Ji-Xin

2013-01-01

Because each nonlinear optical (NLO) imaging modality is sensitive to specific molecules or structures, multimodal NLO imaging capitalizes the potential of NLO microscopy for studies of complex biological tissues. The coupling of multiphoton fluorescence, second harmonic generation, and coherent anti-Stokes Raman scattering (CARS) has allowed investigation of a broad range of biological questions concerning lipid metabolism, cancer development, cardiovascular disease, and skin biology. Moreover, recent research shows the great potential of using CARS microscope as a platform to develop more advanced NLO modalities such as electronic-resonance-enhanced four-wave mixing, stimulated Raman scattering, and pump-probe microscopy. This article reviews the various approaches developed for realization of multimodal NLO imaging as well as developments of new NLO modalities on a CARS microscope. Applications to various aspects of biological and biomedical research are discussed. PMID:24353747
Microscopy illumination engineering using a low-cost liquid crystal display.

PubMed

Guo, Kaikai; Bian, Zichao; Dong, Siyuan; Nanda, Pariksheet; Wang, Ying Min; Zheng, Guoan

2015-02-01

Illumination engineering is critical for obtaining high-resolution, high-quality images in microscope settings. In a typical microscope, the condenser lens provides sample illumination that is uniform and free from glare. The associated condenser diaphragm can be manually adjusted to obtain the optimal illumination numerical aperture. In this paper, we report a programmable condenser lens for active illumination control. In our prototype setup, we used a $15 liquid crystal display as a transparent spatial light modulator and placed it at the back focal plane of the condenser lens. By setting different binary patterns on the display, we can actively control the illumination and the spatial coherence of the microscope platform. We demonstrated the use of such a simple scheme for multimodal imaging, including bright-field microscopy, darkfield microscopy, phase-contrast microscopy, polarization microscopy, 3D tomographic imaging, and super-resolution Fourier ptychographic imaging. The reported illumination engineering scheme is cost-effective and compatible with most existing platforms. It enables a turnkey solution with high flexibility for researchers in various communities. From the engineering point-of-view, the reported illumination scheme may also provide new insights for the development of multimodal microscopy and Fourier ptychographic imaging.
A JOINT FRAMEWORK FOR 4D SEGMENTATION AND ESTIMATION OF SMOOTH TEMPORAL APPEARANCE CHANGES.

PubMed

Gao, Yang; Prastawa, Marcel; Styner, Martin; Piven, Joseph; Gerig, Guido

2014-04-01

Medical imaging studies increasingly use longitudinal images of individual subjects in order to follow-up changes due to development, degeneration, disease progression or efficacy of therapeutic intervention. Repeated image data of individuals are highly correlated, and the strong causality of information over time lead to the development of procedures for joint segmentation of the series of scans, called 4D segmentation. A main aim was improved consistency of quantitative analysis, most often solved via patient-specific atlases. Challenging open problems are contrast changes and occurance of subclasses within tissue as observed in multimodal MRI of infant development, neurodegeneration and disease. This paper proposes a new 4D segmentation framework that enforces continuous dynamic changes of tissue contrast patterns over time as observed in such data. Moreover, our model includes the capability to segment different contrast patterns within a specific tissue class, for example as seen in myelinated and unmyelinated white matter regions in early brain development. Proof of concept is shown with validation on synthetic image data and with 4D segmentation of longitudinal, multimodal pediatric MRI taken at 6, 12 and 24 months of age, but the methodology is generic w.r.t. different application domains using serial imaging.
TU-G-303-04: Radiomics and the Coming Pan-Omics Revolution

DOE Office of Scientific and Technical Information (OSTI.GOV)

El Naqa, I.

‘Radiomics’ refers to studies that extract a large amount of quantitative information from medical imaging studies as a basis for characterizing a specific aspect of patient health. Radiomics models can be built to address a wide range of outcome predictions, clinical decisions, basic cancer biology, etc. For example, radiomics models can be built to predict the aggressiveness of an imaged cancer, cancer gene expression characteristics (radiogenomics), radiation therapy treatment response, etc. Technically, radiomics brings together quantitative imaging, computer vision/image processing, and machine learning. In this symposium, speakers will discuss approaches to radiomics investigations, including: longitudinal radiomics, radiomics combined with othermore » biomarkers (‘pan-omics’), radiomics for various imaging modalities (CT, MRI, and PET), and the use of registered multi-modality imaging datasets as a basis for radiomics. There are many challenges to the eventual use of radiomics-derived methods in clinical practice, including: standardization and robustness of selected metrics, accruing the data required, building and validating the resulting models, registering longitudinal data that often involve significant patient changes, reliable automated cancer segmentation tools, etc. Despite the hurdles, results achieved so far indicate the tremendous potential of this general approach to quantifying and using data from medical images. Specific applications of radiomics to be presented in this symposium will include: the longitudinal analysis of patients with low-grade gliomas; automatic detection and assessment of patients with metastatic bone lesions; image-based monitoring of patients with growing lymph nodes; predicting radiotherapy outcomes using multi-modality radiomics; and studies relating radiomics with genomics in lung cancer and glioblastoma. Learning Objectives: Understanding the basic image features that are often used in radiomic models. Understanding requirements for reliable radiomic models, including robustness of metrics, adequate predictive accuracy, and generalizability. Understanding the methodology behind radiomic-genomic (’radiogenomics’) correlations. Research supported by NIH (US), CIHR (Canada), and NSERC (Canada)« less

Imaging approaches for the study of cell based cardiac therapies

PubMed Central

Lau, Joe F.; Anderson, Stasia A.; Adler, Eric; Frank, Joseph A.

2009-01-01

Despite promising preclinical data, the treatment of cardiovascular diseases using embryonic, bone-marrow-derived, and skeletal myoblast stem cells has not yet come to fruition within mainstream clinical practice. Major obstacles in cardiac stem cell investigations include the ability to monitor cell engraftment and survival following implantation within the myocardium. Several cellular imaging modalities, including reporter gene and MRI-based tracking approaches, have emerged that provide the means to identify, localize and monitor stem cells longitudinally in vivo following implantation. This Review will examine the various cardiac cellular tracking modalities, including the combinatorial use of several probes in multimodality imaging, with a focus on data from the last five years. PMID:20027188
Empiric versus imaging guided left ventricular lead placement in cardiac resynchronization therapy (ImagingCRT): study protocol for a randomized controlled trial.

PubMed

Sommer, Anders; Kronborg, Mads Brix; Poulsen, Steen Hvitfeldt; Böttcher, Morten; Nørgaard, Bjarne Linde; Bouchelouche, Kirsten; Mortensen, Peter Thomas; Gerdes, Christian; Nielsen, Jens Cosedis

2013-04-26

Cardiac resynchronization therapy (CRT) is an established treatment in heart failure patients. However, a large proportion of patients remain nonresponsive to this pacing strategy. Left ventricular (LV) lead position is one of the main determinants of response to CRT. This study aims to clarify whether multimodality imaging guided LV lead placement improves clinical outcome after CRT. The ImagingCRT study is a prospective, randomized, patient- and assessor-blinded, two-armed trial. The study is designed to investigate the effect of imaging guided left ventricular lead positioning on a clinical composite primary endpoint comprising all-cause mortality, hospitalization for heart failure, or unchanged or worsened functional capacity (no improvement in New York Heart Association class and <10% improvement in six-minute-walk test). Imaging guided LV lead positioning is targeted to the latest activated non-scarred myocardial region by speckle tracking echocardiography, single-photon emission computed tomography, and cardiac computed tomography. Secondary endpoints include changes in LV dimensions, ejection fraction and dyssynchrony. A total of 192 patients are included in the study. Despite tremendous advances in knowledge with CRT, the proportion of patients not responding to this treatment has remained stable since the introduction of CRT. ImagingCRT is a prospective, randomized study assessing the clinical and echocardiographic effect of multimodality imaging guided LV lead placement in CRT. The results are expected to make an important contribution in the pursuit of increasing response rate to CRT. Clinicaltrials.gov identifier NCT01323686. The trial was registered March 25, 2011 and the first study subject was randomized April 11, 2011.
Understanding the Manga Hype: Uncovering the Multimodality of Comic-Book Literacies

ERIC Educational Resources Information Center

Schwartz, Adam; Rubinstein-Avila, Eliane

2006-01-01

The authors introduce manga to educators, inspired by the comics' explosive entry into U.S. popular culture. The word "manga" refers specifically to printed, Japanese-style comics found in graphic-novel format--not to be confused with "anime" (animated Japanese cartoons, including moving images on television, movies, video…
Fluorine-18-labeled Gd3+/Yb3+/Er3+ co-doped NaYF4 nanophosphors for multimodality PET/MR/UCL imaging.

PubMed

Zhou, Jing; Yu, Mengxiao; Sun, Yun; Zhang, Xianzhong; Zhu, Xingjun; Wu, Zhanhong; Wu, Dongmei; Li, Fuyou

2011-02-01

Molecular imaging modalities provide a wealth of information that is highly complementary and rarely redundant. To combine the advantages of molecular imaging techniques, (18)F-labeled Gd(3+)/Yb(3+)/Er(3+) co-doped NaYF(4) nanophosphors (NPs) simultaneously possessing with radioactivity, magnetic, and upconversion luminescent properties have been fabricated for multimodality positron emission tomography (PET), magnetic resonance imaging (MRI), and laser scanning upconversion luminescence (UCL) imaging. Hydrophilic citrate-capped NaY(0.2)Gd(0.6)Yb(0.18)Er(0.02)F(4) nanophosphors (cit-NPs) were obtained from hydrophobic oleic acid (OA)-coated nanoparticles (OA-NPs) through a process of ligand exchange of OA with citrate, and were found to be monodisperse with an average size of 22 × 19 nm. The obtained hexagonal cit-NPs show intense UCL emission in the visible region and paramagnetic longitudinal relaxivity (r(1) = 0.405 s(-1)·(mM)(-1)). Through a facile inorganic reaction based on the strong binding between Y(3+) and F(-), (18)F-labeled NPs have been fabricated in high yield. The use of cit-NPs as a multimodal probe has been further explored for T(1)-weighted MR and PET imaging in vivo and UCL imaging of living cells and tissue slides. The results indicate that (18)F-labeled NaY(0.2)Gd(0.6)Yb(0.18)Er(0.02) is a potential candidate as a multimodal nanoprobe for ultra-sensitive molecular imaging from the cellular scale to whole-body evaluation. Copyright © 2010 Elsevier Ltd. All rights reserved.
Multimodal nonlinear microscopy of biopsy specimen: towards intraoperative diagnostics (Conference Presentation)

NASA Astrophysics Data System (ADS)

Schmitt, Michael; Heuke, Sandro; Meyer, Tobias; Chernavskaia, Olga; Bocklitz, Thomas W.; Popp, Juergen

2016-03-01

The realization of label-free molecule specific imaging of morphology and chemical composition of tissue at subcellular spatial resolution in real time is crucial for many envisioned applications in medicine, e.g., precise surgical guidance and non-invasive histopathologic examination of tissue. Thus, new approaches for a fast and reliable in vivo and near in vivo (ex corpore in vivo) tissue characterization to supplement routine pathological diagnostics is needed. Spectroscopic imaging approaches are particularly important since they have the potential to provide a pathologist with adequate support in the form of clinically-relevant information under both ex vivo and in vivo conditions. In this contribution it is demonstrated, that multimodal nonlinear microscopy combining coherent anti-Stokes Raman scattering (CARS), two photon excited fluorescence (TPEF) and second harmonic generation (SHG) enables the detection of characteristic structures and the accompanying molecular changes of widespread diseases, particularly of cancer and atherosclerosis. The detailed images enable an objective evaluation of the tissue samples for an early diagnosis of the disease status. Increasing the spectral resolution and analyzing CARS images at multiple Raman resonances improves the chemical specificity. To facilitate handling and interpretation of the image data characteristic properties can be automatically extracted by advanced image processing algorithms, e.g., for tissue classification. Overall, the presented examples show the great potential of multimodal imaging to augment standard intraoperative clinical assessment with functional multimodal CARS/SHG/TPEF images to highlight functional activity and tumor boundaries. It ensures fast, label-free and non-invasive intraoperative tissue classification paving the way towards in vivo optical pathology.
Au Nanocage Functionalized with Ultra-small Fe3O4 Nanoparticles for Targeting T1-T2Dual MRI and CT Imaging of Tumor

NASA Astrophysics Data System (ADS)

Wang, Guannan; Gao, Wei; Zhang, Xuanjun; Mei, Xifan

2016-06-01

Diagnostic approaches based on multimodal imaging of clinical noninvasive imaging (eg. MRI/CT scanner) are highly developed in recent years for accurate selection of the therapeutic regimens in critical diseases. Therefore, it is highly demanded in the development of appropriate all-in-one multimodal contrast agents (MCAs) for the MRI/CT multimodal imaging. Here a novel ideal MCAs (F-AuNC@Fe3O4) were engineered by assemble Au nanocages (Au NC) and ultra-small iron oxide nanoparticles (Fe3O4) for simultaneous T1-T2dual MRI and CT contrast imaging. In this system, the Au nanocages offer facile thiol modification and strong X-ray attenuation property for CT imaging. The ultra-small Fe3O4 nanoparticles, as excellent contrast agent, is able to provide great enhanced signal of T1- and T2-weighted MRI (r1 = 6.263 mM-1 s-1, r2 = 28.117 mM-1 s-1) due to their ultra-refined size. After functionalization, the present MCAs nanoparticles exhibited small average size, low aggregation and excellent biocompatible. In vitro and In vivo studies revealed that the MCAs show long-term circulation time, renal clearance properties and outstanding capability of selective accumulation in tumor tissues for simultaneous CT imaging and T1- and T2-weighted MRI. Taken together, these results show that as-prepared MCAs are excellent candidates as MRI/CT multimodal imaging contrast agents.
Development of a precision multimodal surgical navigation system for lung robotic segmentectomy

PubMed Central

Soldea, Valentin; Lachkar, Samy; Rinieri, Philippe; Sarsam, Mathieu; Bottet, Benjamin; Peillon, Christophe

2018-01-01

Minimally invasive sublobar anatomical resection is becoming more and more popular to manage early lung lesions. Robotic-assisted thoracic surgery (RATS) is unique in comparison with other minimally invasive techniques. Indeed, RATS is able to better integrate multiple streams of information including advanced imaging techniques, in an immersive experience at the level of the robotic console. Our aim was to describe three-dimensional (3D) imaging throughout the surgical procedure from preoperative planning to intraoperative assistance and complementary investigations such as radial endobronchial ultrasound (R-EBUS) and virtual bronchoscopy for pleural dye marking. All cases were operated using the DaVinci SystemTM. Modelisation was provided by Visible Patient™ (Strasbourg, France). Image integration in the operative field was achieved using the Tile Pro multi display input of the DaVinci console. Our experience was based on 114 robotic segmentectomies performed between January 2012 and October 2017. The clinical value of 3D imaging integration was evaluated in 2014 in a pilot study. Progressively, we have reached the conclusion that the use of such an anatomic model improves the safety and reliability of procedures. The multimodal system including 3D imaging has been used in more than 40 patients so far and demonstrated a perfect operative anatomic accuracy. Currently, we are developing an original virtual reality experience by exploring 3D imaging models at the robotic console level. The act of operating is being transformed and the surgeon now oversees a complex system that improves decision making. PMID:29785294
Development of a precision multimodal surgical navigation system for lung robotic segmentectomy.

PubMed

Baste, Jean Marc; Soldea, Valentin; Lachkar, Samy; Rinieri, Philippe; Sarsam, Mathieu; Bottet, Benjamin; Peillon, Christophe

2018-04-01

Minimally invasive sublobar anatomical resection is becoming more and more popular to manage early lung lesions. Robotic-assisted thoracic surgery (RATS) is unique in comparison with other minimally invasive techniques. Indeed, RATS is able to better integrate multiple streams of information including advanced imaging techniques, in an immersive experience at the level of the robotic console. Our aim was to describe three-dimensional (3D) imaging throughout the surgical procedure from preoperative planning to intraoperative assistance and complementary investigations such as radial endobronchial ultrasound (R-EBUS) and virtual bronchoscopy for pleural dye marking. All cases were operated using the DaVinci System TM . Modelisation was provided by Visible Patient™ (Strasbourg, France). Image integration in the operative field was achieved using the Tile Pro multi display input of the DaVinci console. Our experience was based on 114 robotic segmentectomies performed between January 2012 and October 2017. The clinical value of 3D imaging integration was evaluated in 2014 in a pilot study. Progressively, we have reached the conclusion that the use of such an anatomic model improves the safety and reliability of procedures. The multimodal system including 3D imaging has been used in more than 40 patients so far and demonstrated a perfect operative anatomic accuracy. Currently, we are developing an original virtual reality experience by exploring 3D imaging models at the robotic console level. The act of operating is being transformed and the surgeon now oversees a complex system that improves decision making.
A prototype hand-held tri-modal instrument for in vivo ultrasound, photoacoustic, and fluorescence imaging

NASA Astrophysics Data System (ADS)

Kang, Jeeun; Chang, Jin Ho; Wilson, Brian C.; Veilleux, Israel; Bai, Yanhui; DaCosta, Ralph; Kim, Kang; Ha, Seunghan; Lee, Jong Gun; Kim, Jeong Seok; Lee, Sang-Goo; Kim, Sun Mi; Lee, Hak Jong; Ahn, Young Bok; Han, Seunghee; Yoo, Yangmo; Song, Tai-Kyong

2015-03-01

Multi-modality imaging is beneficial for both preclinical and clinical applications as it enables complementary information from each modality to be obtained in a single procedure. In this paper, we report the design, fabrication, and testing of a novel tri-modal in vivo imaging system to exploit molecular/functional information from fluorescence (FL) and photoacoustic (PA) imaging as well as anatomical information from ultrasound (US) imaging. The same ultrasound transducer was used for both US and PA imaging, bringing the pulsed laser light into a compact probe by fiberoptic bundles. The FL subsystem is independent of the acoustic components but the front end that delivers and collects the light is physically integrated into the same probe. The tri-modal imaging system was implemented to provide each modality image in real time as well as co-registration of the images. The performance of the system was evaluated through phantom and in vivo animal experiments. The results demonstrate that combining the modalities does not significantly compromise the performance of each of the separate US, PA, and FL imaging techniques, while enabling multi-modality registration. The potential applications of this novel approach to multi-modality imaging range from preclinical research to clinical diagnosis, especially in detection/localization and surgical guidance of accessible solid tumors.
Showing or Telling a Story: A Comparative Study of Public Education Texts in Multimodality and Monomodality

ERIC Educational Resources Information Center

Wang, Kelu

2013-01-01

Multimodal texts that combine words and images produce meaning in a different way from monomodal texts that rely on words. They differ not only in representing the subject matter, but also constructing relationships between text producers and text receivers. This article uses two multimodal texts and one monomodal written text as samples, which…
Multimodal Neuroimaging in Schizophrenia: Description and Dissemination.

PubMed

Aine, C J; Bockholt, H J; Bustillo, J R; Cañive, J M; Caprihan, A; Gasparovic, C; Hanlon, F M; Houck, J M; Jung, R E; Lauriello, J; Liu, J; Mayer, A R; Perrone-Bizzozero, N I; Posse, S; Stephen, J M; Turner, J A; Clark, V P; Calhoun, Vince D

2017-10-01

In this paper we describe an open-access collection of multimodal neuroimaging data in schizophrenia for release to the community. Data were acquired from approximately 100 patients with schizophrenia and 100 age-matched controls during rest as well as several task activation paradigms targeting a hierarchy of cognitive constructs. Neuroimaging data include structural MRI, functional MRI, diffusion MRI, MR spectroscopic imaging, and magnetoencephalography. For three of the hypothesis-driven projects, task activation paradigms were acquired on subsets of ~200 volunteers which examined a range of sensory and cognitive processes (e.g., auditory sensory gating, auditory/visual multisensory integration, visual transverse patterning). Neuropsychological data were also acquired and genetic material via saliva samples were collected from most of the participants and have been typed for both genome-wide polymorphism data as well as genome-wide methylation data. Some results are also presented from the individual studies as well as from our data-driven multimodal analyses (e.g., multimodal examinations of network structure and network dynamics and multitask fMRI data analysis across projects). All data will be released through the Mind Research Network's collaborative informatics and neuroimaging suite (COINS).
Raman Plus X: Biomedical Applications of Multimodal Raman Spectroscopy.

PubMed

Das, Nandan K; Dai, Yichuan; Liu, Peng; Hu, Chuanzhen; Tong, Lieshu; Chen, Xiaoya; Smith, Zachary J

2017-07-07

Raman spectroscopy is a label-free method of obtaining detailed chemical information about samples. Its compatibility with living tissue makes it an attractive choice for biomedical analysis, yet its translation from a research tool to a clinical tool has been slow, hampered by fundamental Raman scattering issues such as long integration times and limited penetration depth. In this review we detail the how combining Raman spectroscopy with other techniques yields multimodal instruments that can help to surmount the translational barriers faced by Raman alone. We review Raman combined with several optical and non-optical methods, including fluorescence, elastic scattering, OCT, phase imaging, and mass spectrometry. In each section we highlight the power of each combination along with a brief history and presentation of representative results. Finally, we conclude with a perspective detailing both benefits and challenges for multimodal Raman measurements, and give thoughts on future directions in the field.
Raman Plus X: Biomedical Applications of Multimodal Raman Spectroscopy

PubMed Central

Das, Nandan K.; Dai, Yichuan; Liu, Peng; Hu, Chuanzhen; Tong, Lieshu; Chen, Xiaoya

2017-01-01

Raman spectroscopy is a label-free method of obtaining detailed chemical information about samples. Its compatibility with living tissue makes it an attractive choice for biomedical analysis, yet its translation from a research tool to a clinical tool has been slow, hampered by fundamental Raman scattering issues such as long integration times and limited penetration depth. In this review we detail the how combining Raman spectroscopy with other techniques yields multimodal instruments that can help to surmount the translational barriers faced by Raman alone. We review Raman combined with several optical and non-optical methods, including fluorescence, elastic scattering, OCT, phase imaging, and mass spectrometry. In each section we highlight the power of each combination along with a brief history and presentation of representative results. Finally, we conclude with a perspective detailing both benefits and challenges for multimodal Raman measurements, and give thoughts on future directions in the field. PMID:28686212
Introduction to clinical and laboratory (small-animal) image registration and fusion.

PubMed

Zanzonico, Pat B; Nehmeh, Sadek A

2006-01-01

Imaging has long been a vital component of clinical medicine and, increasingly, of biomedical research in small-animals. Clinical and laboratory imaging modalities can be divided into two general categories, structural (or anatomical) and functional (or physiological). The latter, in particular, has spawned what has come to be known as "molecular imaging". Image registration and fusion have rapidly emerged as invaluable components of both clinical and small-animal imaging and has lead to the development and marketing of a variety of multi-modality, e.g. PET-CT, devices which provide registered and fused three-dimensional image sets. This paper briefly reviews the basics of image registration and fusion and available clinical and small-animal multi-modality instrumentation.
A fully automatic approach for multimodal PET and MR image segmentation in gamma knife treatment planning.

PubMed

Rundo, Leonardo; Stefano, Alessandro; Militello, Carmelo; Russo, Giorgio; Sabini, Maria Gabriella; D'Arrigo, Corrado; Marletta, Francesco; Ippolito, Massimo; Mauri, Giancarlo; Vitabile, Salvatore; Gilardi, Maria Carla

2017-06-01

Nowadays, clinical practice in Gamma Knife treatments is generally based on MRI anatomical information alone. However, the joint use of MRI and PET images can be useful for considering both anatomical and metabolic information about the lesion to be treated. In this paper we present a co-segmentation method to integrate the segmented Biological Target Volume (BTV), using [ 11 C]-Methionine-PET (MET-PET) images, and the segmented Gross Target Volume (GTV), on the respective co-registered MR images. The resulting volume gives enhanced brain tumor information to be used in stereotactic neuro-radiosurgery treatment planning. GTV often does not match entirely with BTV, which provides metabolic information about brain lesions. For this reason, PET imaging is valuable and it could be used to provide complementary information useful for treatment planning. In this way, BTV can be used to modify GTV, enhancing Clinical Target Volume (CTV) delineation. A novel fully automatic multimodal PET/MRI segmentation method for Leksell Gamma Knife ® treatments is proposed. This approach improves and combines two computer-assisted and operator-independent single modality methods, previously developed and validated, to segment BTV and GTV from PET and MR images, respectively. In addition, the GTV is utilized to combine the superior contrast of PET images with the higher spatial resolution of MRI, obtaining a new BTV, called BTV MRI . A total of 19 brain metastatic tumors, undergone stereotactic neuro-radiosurgery, were retrospectively analyzed. A framework for the evaluation of multimodal PET/MRI segmentation is also presented. Overlap-based and spatial distance-based metrics were considered to quantify similarity concerning PET and MRI segmentation approaches. Statistics was also included to measure correlation among the different segmentation processes. Since it is not possible to define a gold-standard CTV according to both MRI and PET images without treatment response assessment, the feasibility and the clinical value of BTV integration in Gamma Knife treatment planning were considered. Therefore, a qualitative evaluation was carried out by three experienced clinicians. The achieved experimental results showed that GTV and BTV segmentations are statistically correlated (Spearman's rank correlation coefficient: 0.898) but they have low similarity degree (average Dice Similarity Coefficient: 61.87 ± 14.64). Therefore, volume measurements as well as evaluation metrics values demonstrated that MRI and PET convey different but complementary imaging information. GTV and BTV could be combined to enhance treatment planning. In more than 50% of cases the CTV was strongly or moderately conditioned by metabolic imaging. Especially, BTV MRI enhanced the CTV more accurately than BTV in 25% of cases. The proposed fully automatic multimodal PET/MRI segmentation method is a valid operator-independent methodology helping the clinicians to define a CTV that includes both metabolic and morphologic information. BTV MRI and GTV should be considered for a comprehensive treatment planning. Copyright © 2017 Elsevier B.V. All rights reserved.
Multi-Modal Curriculum Learning for Semi-Supervised Image Classification.

PubMed

Gong, Chen; Tao, Dacheng; Maybank, Stephen J; Liu, Wei; Kang, Guoliang; Yang, Jie

2016-07-01

Semi-supervised image classification aims to classify a large quantity of unlabeled images by typically harnessing scarce labeled images. Existing semi-supervised methods often suffer from inadequate classification accuracy when encountering difficult yet critical images, such as outliers, because they treat all unlabeled images equally and conduct classifications in an imperfectly ordered sequence. In this paper, we employ the curriculum learning methodology by investigating the difficulty of classifying every unlabeled image. The reliability and the discriminability of these unlabeled images are particularly investigated for evaluating their difficulty. As a result, an optimized image sequence is generated during the iterative propagations, and the unlabeled images are logically classified from simple to difficult. Furthermore, since images are usually characterized by multiple visual feature descriptors, we associate each kind of features with a teacher, and design a multi-modal curriculum learning (MMCL) strategy to integrate the information from different feature modalities. In each propagation, each teacher analyzes the difficulties of the currently unlabeled images from its own modality viewpoint. A consensus is subsequently reached among all the teachers, determining the currently simplest images (i.e., a curriculum), which are to be reliably classified by the multi-modal learner. This well-organized propagation process leveraging multiple teachers and one learner enables our MMCL to outperform five state-of-the-art methods on eight popular image data sets.
[Medical imaging in tumor precision medicine: opportunities and challenges].

PubMed

Xu, Jingjing; Tan, Yanbin; Zhang, Minming

2017-05-25

Tumor precision medicine is an emerging approach for tumor diagnosis, treatment and prevention, which takes account of individual variability of environment, lifestyle and genetic information. Tumor precision medicine is built up on the medical imaging innovations developed during the past decades, including the new hardware, new imaging agents, standardized protocols, image analysis and multimodal imaging fusion technology. Also the development of automated and reproducible analysis algorithm has extracted large amount of information from image-based features. With the continuous development and mining of tumor clinical and imaging databases, the radiogenomics, radiomics and artificial intelligence have been flourishing. Therefore, these new technological advances bring new opportunities and challenges to the application of imaging in tumor precision medicine.
Multimodal optical workstation for simultaneous linear, nonlinear microscopy and nanomanipulation: upgrading a commercial confocal inverted microscope.

PubMed

Mathew, Manoj; Santos, Susana I C O; Zalvidea, Dobryna; Loza-Alvarez, Pablo

2009-07-01

In this work we propose and build a multimodal optical workstation that extends a commercially available confocal microscope (Nikon Confocal C1-Si) to include nonlinear/multiphoton microscopy and optical manipulation/stimulation tools such as nanosurgery. The setup allows both subsystems (confocal and nonlinear) to work independently and simultaneously. The workstation enables, for instance, nanosurgery along with simultaneous confocal and brightfield imaging. The nonlinear microscopy capabilities are added around the commercial confocal microscope by exploiting all the flexibility offered by this microscope and without need for any mechanical or electronic modification of the confocal microscope systems. As an example, the standard differential interference contrast condenser and diascopic detector in the confocal microscope are readily used as a forward detection mount for second harmonic generation imaging. The various capabilities of this workstation, as applied directly to biology, are demonstrated using the model organism Caenorhabditis elegans.
Deep features for efficient multi-biometric recognition with face and ear images

NASA Astrophysics Data System (ADS)

Omara, Ibrahim; Xiao, Gang; Amrani, Moussa; Yan, Zifei; Zuo, Wangmeng

2017-07-01

Recently, multimodal biometric systems have received considerable research interest in many applications especially in the fields of security. Multimodal systems can increase the resistance to spoof attacks, provide more details and flexibility, and lead to better performance and lower error rate. In this paper, we present a multimodal biometric system based on face and ear, and propose how to exploit the extracted deep features from Convolutional Neural Networks (CNNs) on the face and ear images to introduce more powerful discriminative features and robust representation ability for them. First, the deep features for face and ear images are extracted based on VGG-M Net. Second, the extracted deep features are fused by using a traditional concatenation and a Discriminant Correlation Analysis (DCA) algorithm. Third, multiclass support vector machine is adopted for matching and classification. The experimental results show that the proposed multimodal system based on deep features is efficient and achieves a promising recognition rate up to 100 % by using face and ear. In addition, the results indicate that the fusion based on DCA is superior to traditional fusion.
Computational method for multi-modal microscopy based on transport of intensity equation

NASA Astrophysics Data System (ADS)

Li, Jiaji; Chen, Qian; Sun, Jiasong; Zhang, Jialin; Zuo, Chao

2017-02-01

In this paper, we develop the requisite theory to describe a hybrid virtual-physical multi-modal imaging system which yields quantitative phase, Zernike phase contrast, differential interference contrast (DIC), and light field moment imaging simultaneously based on transport of intensity equation(TIE). We then give the experimental demonstration of these ideas by time-lapse imaging of live HeLa cell mitosis. Experimental results verify that a tunable lens based TIE system, combined with the appropriate post-processing algorithm, can achieve a variety of promising imaging modalities in parallel with the quantitative phase images for the dynamic study of cellular processes.

SU-E-J-109: Accurate Contour Transfer Between Different Image Modalities Using a Hybrid Deformable Image Registration and Fuzzy Connected Image Segmentation Method.

PubMed

Yang, C; Paulson, E; Li, X

2012-06-01

To develop and evaluate a tool that can improve the accuracy of contour transfer between different image modalities under challenging conditions of low image contrast and large image deformation, comparing to a few commonly used methods, for radiation treatment planning. The software tool includes the following steps and functionalities: (1) accepting input of images of different modalities, (2) converting existing contours on reference images (e.g., MRI) into delineated volumes and adjusting the intensity within the volumes to match target images (e.g., CT) intensity distribution for enhanced similarity metric, (3) registering reference and target images using appropriate deformable registration algorithms (e.g., B-spline, demons) and generate deformed contours, (4) mapping the deformed volumes on target images, calculating mean, variance, and center of mass as the initialization parameters for consecutive fuzzy connectedness (FC) image segmentation on target images, (5) generate affinity map from FC segmentation, (6) achieving final contours by modifying the deformed contours using the affinity map with a gradient distance weighting algorithm. The tool was tested with the CT and MR images of four pancreatic cancer patients acquired at the same respiration phase to minimize motion distortion. Dice's Coefficient was calculated against direct delineation on target image. Contours generated by various methods, including rigid transfer, auto-segmentation, deformable only transfer and proposed method, were compared. Fuzzy connected image segmentation needs careful parameter initialization and user involvement. Automatic contour transfer by multi-modality deformable registration leads up to 10% of accuracy improvement over the rigid transfer. Two extra proposed steps of adjusting intensity distribution and modifying the deformed contour with affinity map improve the transfer accuracy further to 14% averagely. Deformable image registration aided by contrast adjustment and fuzzy connectedness segmentation improves the contour transfer accuracy between multi-modality images, particularly with large deformation and low image contrast. © 2012 American Association of Physicists in Medicine.
Multimodality Molecular Imaging of Cardiac Cell Transplantation: Part I. Reporter Gene Design, Characterization, and Optical in Vivo Imaging of Bone Marrow Stromal Cells after Myocardial Infarction

PubMed Central

Parashurama, Natesh; Ahn, Byeong-Cheol; Ziv, Keren; Ito, Ken; Paulmurugan, Ramasamy; Willmann, Jürgen K.; Chung, Jaehoon; Ikeno, Fumiaki; Swanson, Julia C.; Merk, Denis R.; Lyons, Jennifer K.; Yerushalmi, David; Teramoto, Tomohiko; Kosuge, Hisanori; Dao, Catherine N.; Ray, Pritha; Patel, Manishkumar; Chang, Ya-fang; Mahmoudi, Morteza; Cohen, Jeff Eric; Goldstone, Andrew Brooks; Habte, Frezghi; Bhaumik, Srabani; Yaghoubi, Shahriar; Robbins, Robert C.; Dash, Rajesh; Yang, Phillip C.; Brinton, Todd J.; Yock, Paul G.; McConnell, Michael V.

2016-01-01

Purpose To use multimodality reporter-gene imaging to assess the serial survival of marrow stromal cells (MSC) after therapy for myocardial infarction (MI) and to determine if the requisite preclinical imaging end point was met prior to a follow-up large-animal MSC imaging study. Materials and Methods Animal studies were approved by the Institutional Administrative Panel on Laboratory Animal Care. Mice (n = 19) that had experienced MI were injected with bone marrow–derived MSC that expressed a multimodality triple fusion (TF) reporter gene. The TF reporter gene (fluc2-egfp-sr39ttk) consisted of a human promoter, ubiquitin, driving firefly luciferase 2 (fluc2), enhanced green fluorescent protein (egfp), and the sr39tk positron emission tomography reporter gene. Serial bioluminescence imaging of MSC-TF and ex vivo luciferase assays were performed. Correlations were analyzed with the Pearson product-moment correlation, and serial imaging results were analyzed with a mixed-effects regression model. Results Analysis of the MSC-TF after cardiac cell therapy showed significantly lower signal on days 8 and 14 than on day 2 (P = .011 and P = .001, respectively). MSC-TF with MI demonstrated significantly higher signal than MSC-TF without MI at days 4, 8, and 14 (P = .016). Ex vivo luciferase activity assay confirmed the presence of MSC-TF on days 8 and 14 after MI. Conclusion Multimodality reporter-gene imaging was successfully used to assess serial MSC survival after therapy for MI, and it was determined that the requisite preclinical imaging end point, 14 days of MSC survival, was met prior to a follow-up large-animal MSC study. © RSNA, 2016 Online supplemental material is available for this article. PMID:27308957
Drusen Characterization with Multimodal Imaging

PubMed Central

Spaide, Richard F.; Curcio, Christine A.

2010-01-01

Summary Multimodal imaging findings and histological demonstration of soft drusen, cuticular drusen, and subretinal drusenoid deposits provided information used to develop a model explaining their imaging characteristics. Purpose To characterize the known appearance of cuticular drusen, subretinal drusenoid deposits (reticular pseudodrusen), and soft drusen as revealed by multimodal fundus imaging; to create an explanatory model that accounts for these observations. Methods Reported color, fluorescein angiographic, autofluorescence, and spectral domain optical coherence tomography (SD-OCT) images of patients with cuticular drusen, soft drusen, and subretinal drusenoid deposits were reviewed, as were actual images from affected eyes. Representative histological sections were examined. The geometry, location, and imaging characteristics of these lesions were evaluated. A hypothesis based on the Beer-Lambert Law of light absorption was generated to fit these observations. Results Cuticular drusen appear as numerous uniform round yellow-white punctate accumulations under the retinal pigment epithelium (RPE). Soft drusen are larger yellow-white dome-shaped mounds of deposit under the RPE. Subretinal drusenoid deposits are polymorphous light-grey interconnected accumulations above the RPE. Based on the model, both cuticular and soft drusen appear yellow due to the removal of shorter wavelength light by a double pass through the RPE. Subretinal drusenoid deposits, which are located on the RPE, are not subjected to short wavelength attenuation and therefore are more prominent when viewed with blue light. The location and morphology of extracellular material in relationship to the RPE, and associated changes to RPE morphology and pigmentation, appeared to be primary determinants of druse appearance in different imaging modalities. Conclusion Although cuticular drusen, subretinal drusenoid deposits, and soft drusen are composed of common components, they are distinguishable by multimodal imaging due to differences in location, morphology, and optical filtering effects by drusenoid material and the RPE. PMID:20924263
Validation of a deformable MRI to CT registration algorithm employing same day planning MRI for surrogate analysis.

PubMed

Padgett, Kyle R; Stoyanova, Radka; Pirozzi, Sara; Johnson, Perry; Piper, Jon; Dogan, Nesrin; Pollack, Alan

2018-03-01

Validating deformable multimodality image registrations is challenging due to intrinsic differences in signal characteristics and their spatial intensity distributions. Evaluating multimodality registrations using these spatial intensity distributions is also complicated by the fact that these metrics are often employed in the registration optimization process. This work evaluates rigid and deformable image registrations of the prostate in between diagnostic-MRI and radiation treatment planning-CT by utilizing a planning-MRI after fiducial marker placement as a surrogate. The surrogate allows for the direct quantitative analysis that can be difficult in the multimodality domain. For thirteen prostate patients, T2 images were acquired at two different time points, the first several weeks prior to planning (diagnostic-MRI) and the second on the same day as the planning-CT (planning-MRI). The diagnostic-MRI was deformed to the planning-CT utilizing a commercially available algorithm which synthesizes a deformable image registration (DIR) algorithm from local rigid registrations. The planning-MRI provided an independent surrogate for the planning-CT for assessing registration accuracy using image similarity metrics, including Pearson correlation and normalized mutual information (NMI). A local analysis was performed by looking only within the prostate, proximal seminal vesicles, penile bulb, and combined areas. The planning-MRI provided an excellent surrogate for the planning-CT with residual error in fiducial alignment between the two datasets being submillimeter, 0.78 mm. DIR was superior to the rigid registration in 11 of 13 cases demonstrating a 27.37% improvement in NMI (P < 0.009) within a regional area surrounding the prostate and associated critical organs. Pearson correlations showed similar results, demonstrating a 13.02% improvement (P < 0.013). By utilizing the planning-MRI as a surrogate for the planning-CT, an independent evaluation of registration accuracy is possible. This population provides an ideal testing ground for MRI to CT DIR by obviating the need for multimodality comparisons which are inherently more challenging. © 2018 The Authors. Journal of Applied Clinical Medical Physics published by Wiley Periodicals, Inc. on behalf of American Association of Physicists in Medicine.
Tunable X-ray speckle-based phase-contrast and dark-field imaging using the unified modulated pattern analysis approach

NASA Astrophysics Data System (ADS)

Zdora, M.-C.; Thibault, P.; Deyhle, H.; Vila-Comamala, J.; Rau, C.; Zanette, I.

2018-05-01

X-ray phase-contrast and dark-field imaging provides valuable, complementary information about the specimen under study. Among the multimodal X-ray imaging methods, X-ray grating interferometry and speckle-based imaging have drawn particular attention, which, however, in their common implementations incur certain limitations that can restrict their range of applications. Recently, the unified modulated pattern analysis (UMPA) approach was proposed to overcome these limitations and combine grating- and speckle-based imaging in a single approach. Here, we demonstrate the multimodal imaging capabilities of UMPA and highlight its tunable character regarding spatial resolution, signal sensitivity and scan time by using different reconstruction parameters.
Rapid multi-modality preregistration based on SIFT descriptor.

PubMed

Chen, Jian; Tian, Jie

2006-01-01

This paper describes the scale invariant feature transform (SIFT) method for rapid preregistration of medical image. This technique originates from Lowe's method wherein preregistration is achieved by matching the corresponding keypoints between two images. The computational complexity has been reduced when we applied SIFT preregistration method before refined registration due to its O(n) exponential calculations. The features of SIFT are highly distinctive and invariant to image scaling and rotation, and partially invariant to change in illumination and contrast, it is robust and repeatable for cursorily matching two images. We also altered the descriptor so our method can deal with multimodality preregistration.
Multimode-Optical-Fiber Imaging Probe

NASA Technical Reports Server (NTRS)

Jackson, Deborah

2000-01-01

Currently, endoscopic surgery uses single-mode fiber-bundles to obtain in vivo image information inside orifices of the body. This limits their use to the larger natural bodily orifices and to surgical procedures where there is plenty of room for manipulation. The knee joint, for example can be easily viewed with a fiber optic viewer, but joints in the finger cannot. However, there are a host of smaller orifices where fiber endoscopy would play an important role if a cost effective fiber probe were developed with small enough dimensions (< 250 microns). Examples of beneficiaries of micro-endoscopes are the treatment of the Eustatian tube of the middle ear, the breast ducts, tear ducts, coronary arteries, fallopian tubes, as well as the treatment of salivary duct parotid disease, and the neuro endoscopy of the ventricles and spinal canal. To solve this problem, this work describes an approach for recovering images from. tightly confined spaces using multimode fibers and analytically demonstrates that the concept is sound. The proof of concept draws upon earlier works that concentrated on image recovery after two-way transmission through a multimode fiber as well as work that demonstrated the recovery of images after one-way transmission through a multimode fiber. Both relied on generating a phase conjugated wavefront which was predistorted with the characteristics of the fiber. The described approach also relies on generating a phase conjugated wavefront, but utilizes two fibers to capture the image at some intermediate point (accessible by the fibers, but which is otherwise visually unaccessible).
Multimode-Optical-Fiber Imaging Probe

NASA Technical Reports Server (NTRS)

Jackson, Deborah

1999-01-01

Currently, endoscopic surgery uses single-mode fiber-bundles to obtain in vivo image information inside the orifices of the body. This limits their use to the larger natural orifices and to surgical procedures where there is plenty of room for manipulation. The knee joint, for example, can be easily viewed with a fiber optic viewer, but joints in the finger cannot. However, there are a host of smaller orifices where fiber endoscopy would play an important role if a cost effective fiber probe were developed with small enough dimensions (less than or equal to 250 microns). Examples of beneficiaries of micro-endoscopes are the treatment of the Eustatian tube of the middle ear, the breast ducts, tear ducts, coronary arteries, fallopian tubes, as well as the treatment of salivary duct parotid disease, and the neuro endoscopy of the ventricles and spinal canal. This work describes an approach for recovering images from tightly confined spaces using multimode. The concept draws upon earlier works that concentrated on image recovery after two-way transmission through a multimode fiber as well as work that demonstrated the recovery of images after one-way transmission through a multimode fiber. Both relied on generating a phase conjugated wavefront, which was predistorted with the characteristics of the fiber. The approach described here also relies on generating a phase conjugated wavefront, but utilizes two fibers to capture the image at some intermediate point (accessible by the fibers, but which is otherwise visually inaccessible).
Patient-tailored multimodal neuroimaging, visualization and quantification of human intra-cerebral hemorrhage

NASA Astrophysics Data System (ADS)

Goh, Sheng-Yang M.; Irimia, Andrei; Vespa, Paul M.; Van Horn, John D.

2016-03-01

In traumatic brain injury (TBI) and intracerebral hemorrhage (ICH), the heterogeneity of lesion sizes and types necessitates a variety of imaging modalities to acquire a comprehensive perspective on injury extent. Although it is advantageous to combine imaging modalities and to leverage their complementary benefits, there are difficulties in integrating information across imaging types. Thus, it is important that efforts be dedicated to the creation and sustained refinement of resources for multimodal data integration. Here, we propose a novel approach to the integration of neuroimaging data acquired from human patients with TBI/ICH using various modalities; we also demonstrate the integrated use of multimodal magnetic resonance imaging (MRI) and diffusion tensor imaging (DTI) data for TBI analysis based on both visual observations and quantitative metrics. 3D models of healthy-appearing tissues and TBIrelated pathology are generated, both of which are derived from multimodal imaging data. MRI volumes acquired using FLAIR, SWI, and T2 GRE are used to segment pathology. Healthy tissues are segmented using user-supervised tools, and results are visualized using a novel graphical approach called a `connectogram', where brain connectivity information is depicted within a circle of radially aligned elements. Inter-region connectivity and its strength are represented by links of variable opacities drawn between regions, where opacity reflects the percentage longitudinal change in brain connectivity density. Our method for integrating, analyzing and visualizing structural brain changes due to TBI and ICH can promote knowledge extraction and enhance the understanding of mechanisms underlying recovery.
Unique Diagnostic and Therapeutic Roles of Porphyrins and Phthalocyanines in Photodynamic Therapy, Imaging and Theranostics

PubMed Central

Josefsen, Leanne B.; Boyle, Ross W.

2012-01-01

Porphyrinic molecules have a unique theranostic role in disease therapy; they have been used to image, detect and treat different forms of diseased tissue including age-related macular degeneration and a number of different cancer types. Current focus is on the clinical imaging of tumour tissue; targeted delivery of photosensitisers and the potential of photosensitisers in multimodal biomedical theranostic nanoplatforms. The roles of porphyrinic molecules in imaging and pdt, along with research into improving their selective uptake in diseased tissue and their utility in theranostic applications are highlighted in this Review. PMID:23082103
Development of computational small animal models and their applications in preclinical imaging and therapy research

DOE Office of Scientific and Technical Information (OSTI.GOV)

Xie, Tianwu; Zaidi, Habib, E-mail: habib.zaidi@hcuge.ch; Geneva Neuroscience Center, Geneva University, Geneva CH-1205

The development of multimodality preclinical imaging techniques and the rapid growth of realistic computer simulation tools have promoted the construction and application of computational laboratory animal models in preclinical research. Since the early 1990s, over 120 realistic computational animal models have been reported in the literature and used as surrogates to characterize the anatomy of actual animals for the simulation of preclinical studies involving the use of bioluminescence tomography, fluorescence molecular tomography, positron emission tomography, single-photon emission computed tomography, microcomputed tomography, magnetic resonance imaging, and optical imaging. Other applications include electromagnetic field simulation, ionizing and nonionizing radiation dosimetry, and themore » development and evaluation of new methodologies for multimodality image coregistration, segmentation, and reconstruction of small animal images. This paper provides a comprehensive review of the history and fundamental technologies used for the development of computational small animal models with a particular focus on their application in preclinical imaging as well as nonionizing and ionizing radiation dosimetry calculations. An overview of the overall process involved in the design of these models, including the fundamental elements used for the construction of different types of computational models, the identification of original anatomical data, the simulation tools used for solving various computational problems, and the applications of computational animal models in preclinical research. The authors also analyze the characteristics of categories of computational models (stylized, voxel-based, and boundary representation) and discuss the technical challenges faced at the present time as well as research needs in the future.« less
Characterizing virus-induced gene silencing at the cellular level with in situ multimodal imaging

DOE PAGES

Burkhow, Sadie J.; Stephens, Nicole M.; Mei, Yu; ...

2018-05-25

Reverse genetic strategies, such as virus-induced gene silencing, are powerful techniques to study gene function. Currently, there are few tools to study the spatial dependence of the consequences of gene silencing at the cellular level. Here, we report the use of multimodal Raman and mass spectrometry imaging to study the cellular-level biochemical changes that occur from silencing the phytoene desaturase ( pds) gene using a Foxtail mosaic virus (FoMV) vector in maize leaves. The multimodal imaging method allows the localized carotenoid distribution to be measured and reveals differences lost in the spatial average when analyzing a carotenoid extraction of themore » whole leaf. The nature of the Raman and mass spectrometry signals are complementary: silencing pds reduces the downstream carotenoid Raman signal and increases the phytoene mass spectrometry signal.« less
Characterizing virus-induced gene silencing at the cellular level with in situ multimodal imaging

DOE Office of Scientific and Technical Information (OSTI.GOV)

Burkhow, Sadie J.; Stephens, Nicole M.; Mei, Yu

Reverse genetic strategies, such as virus-induced gene silencing, are powerful techniques to study gene function. Currently, there are few tools to study the spatial dependence of the consequences of gene silencing at the cellular level. Here, we report the use of multimodal Raman and mass spectrometry imaging to study the cellular-level biochemical changes that occur from silencing the phytoene desaturase ( pds) gene using a Foxtail mosaic virus (FoMV) vector in maize leaves. The multimodal imaging method allows the localized carotenoid distribution to be measured and reveals differences lost in the spatial average when analyzing a carotenoid extraction of themore » whole leaf. The nature of the Raman and mass spectrometry signals are complementary: silencing pds reduces the downstream carotenoid Raman signal and increases the phytoene mass spectrometry signal.« less
Calibration for single multi-mode fiber digital scanning microscopy imaging system

NASA Astrophysics Data System (ADS)

Yin, Zhe; Liu, Guodong; Liu, Bingguo; Gan, Yu; Zhuang, Zhitao; Chen, Fengdong

2015-11-01

Single multimode fiber (MMF) digital scanning imaging system is a development tendency of modern endoscope. We concentrate on the calibration method of the imaging system. Calibration method comprises two processes, forming scanning focused spots and calibrating the couple factors varied with positions. Adaptive parallel coordinate algorithm (APC) is adopted to form the focused spots at the multimode fiber (MMF) output. Compare with other algorithm, APC contains many merits, i.e. rapid speed, small amount calculations and no iterations. The ratio of the optics power captured by MMF to the intensity of the focused spots is called couple factor. We setup the calibration experimental system to form the scanning focused spots and calculate the couple factors for different object positions. The experimental result the couple factor is higher in the center than the edge.
Multi-modality image fusion based on enhanced fuzzy radial basis function neural networks.

PubMed

Chao, Zhen; Kim, Dohyeon; Kim, Hee-Joung

2018-04-01

In clinical applications, single modality images do not provide sufficient diagnostic information. Therefore, it is necessary to combine the advantages or complementarities of different modalities of images. Recently, neural network technique was applied to medical image fusion by many researchers, but there are still many deficiencies. In this study, we propose a novel fusion method to combine multi-modality medical images based on the enhanced fuzzy radial basis function neural network (Fuzzy-RBFNN), which includes five layers: input, fuzzy partition, front combination, inference, and output. Moreover, we propose a hybrid of the gravitational search algorithm (GSA) and error back propagation algorithm (EBPA) to train the network to update the parameters of the network. Two different patterns of images are used as inputs of the neural network, and the output is the fused image. A comparison with the conventional fusion methods and another neural network method through subjective observation and objective evaluation indexes reveals that the proposed method effectively synthesized the information of input images and achieved better results. Meanwhile, we also trained the network by using the EBPA and GSA, individually. The results reveal that the EBPGSA not only outperformed both EBPA and GSA, but also trained the neural network more accurately by analyzing the same evaluation indexes. Copyright © 2018 Associazione Italiana di Fisica Medica. Published by Elsevier Ltd. All rights reserved.
Fast multi-core based multimodal registration of 2D cross-sections and 3D datasets.

PubMed

Scharfe, Michael; Pielot, Rainer; Schreiber, Falk

2010-01-11

Solving bioinformatics tasks often requires extensive computational power. Recent trends in processor architecture combine multiple cores into a single chip to improve overall performance. The Cell Broadband Engine (CBE), a heterogeneous multi-core processor, provides power-efficient and cost-effective high-performance computing. One application area is image analysis and visualisation, in particular registration of 2D cross-sections into 3D image datasets. Such techniques can be used to put different image modalities into spatial correspondence, for example, 2D images of histological cuts into morphological 3D frameworks. We evaluate the CBE-driven PlayStation 3 as a high performance, cost-effective computing platform by adapting a multimodal alignment procedure to several characteristic hardware properties. The optimisations are based on partitioning, vectorisation, branch reducing and loop unrolling techniques with special attention to 32-bit multiplies and limited local storage on the computing units. We show how a typical image analysis and visualisation problem, the multimodal registration of 2D cross-sections and 3D datasets, benefits from the multi-core based implementation of the alignment algorithm. We discuss several CBE-based optimisation methods and compare our results to standard solutions. More information and the source code are available from http://cbe.ipk-gatersleben.de. The results demonstrate that the CBE processor in a PlayStation 3 accelerates computational intensive multimodal registration, which is of great importance in biological/medical image processing. The PlayStation 3 as a low cost CBE-based platform offers an efficient option to conventional hardware to solve computational problems in image processing and bioinformatics.
A Novel Multifunctional Theranostic Liposome Drug Delivery System: Construction, Characterization, and Multimodality MR, Near-infrared Fluorescent and Nuclear Imaging

PubMed Central

Li, Shihong; Goins, Beth; Zhang, Lujun; Bao, Ande

2012-01-01

Liposomes are effective lipid nanoparticle drug delivery systems, which can also be functionalized with non-invasive multimodality imaging agents with each modality providing distinct information and having synergistic advantages in diagnosis, monitoring of disease treatment, and evaluation of liposomal drug pharmacokinetics. We designed and constructed a multifunctional theranostic liposomal drug delivery system, which integrated multimodality magnetic resonance (MR), near-infrared (NIR) fluorescent and nuclear imaging of liposomal drug delivery, and therapy monitoring and prediction. The pre-manufactured liposomes were composed of DSPC/cholesterol/Gd-DOTADSPE/DOTA-DSPE with the molar ratio of 39:35:25:1 and having ammonium sulfate/pH gradient. A lipidized NIR fluorescent tracer, IRDye-DSPE, was effectively post-inserted into the pre-manufactured liposomes. Doxorubicin could be effectively post-loaded into the multifunctional liposomes. The multifunctional doxorubicin-liposomes could also be stably radiolabeled with 99mTc or 64Cu for single photon emission computed tomography (SPECT) or positron emission tomography (PET) imaging, respectively. MR images displayed the high resolution micro-intratumoral distribution of the liposomes in squamous cell carcinoma of head and neck (SCCHN) tumor xenografts in nude rats after intratumoral injection. NIR fluorescent, SPECT and PET images also clearly showed either the high intratumoral retention or distribution of the multifunctional liposomes. This multifunctional drug carrying liposome system is promising for disease theranostics allowing non-invasive multimodality NIR fluorescent, MR, SPECT and PET imaging of their in vivo behavior and capitalizing on the inherent advantages of each modality. PMID:22577859
Overview of chemical imaging methods to address biological questions.

PubMed

da Cunha, Marcel Menezes Lyra; Trepout, Sylvain; Messaoudi, Cédric; Wu, Ting-Di; Ortega, Richard; Guerquin-Kern, Jean-Luc; Marco, Sergio

2016-05-01

Chemical imaging offers extensive possibilities for better understanding of biological systems by allowing the identification of chemical components at the tissue, cellular, and subcellular levels. In this review, we introduce modern methods for chemical imaging that can be applied to biological samples. This work is mainly addressed to the biological sciences community and includes the bases of different technologies, some examples of its application, as well as an introduction to approaches on combining multimodal data. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.
Recent development of nanoparticles for molecular imaging

NASA Astrophysics Data System (ADS)

Kim, Jonghoon; Lee, Nohyun; Hyeon, Taeghwan

2017-10-01

Molecular imaging enables us to non-invasively visualize cellular functions and biological processes in living subjects, allowing accurate diagnosis of diseases at early stages. For successful molecular imaging, a suitable contrast agent with high sensitivity is required. To date, various nanoparticles have been developed as contrast agents for medical imaging modalities. In comparison with conventional probes, nanoparticles offer several advantages, including controllable physical properties, facile surface modification and long circulation time. In addition, they can be integrated with various combinations for multimodal imaging and therapy. In this opinion piece, we highlight recent advances and future perspectives of nanomaterials for molecular imaging. This article is part of the themed issue 'Challenges for chemistry in molecular imaging'.
Multimodal microscopy and the stepwise multi-photon activation fluorescence of melanin

NASA Astrophysics Data System (ADS)

Lai, Zhenhua

The author's work is divided into three aspects: multimodal microscopy, stepwise multi-photon activation fluorescence (SMPAF) of melanin, and customized-profile lenses (CPL) for on-axis laser scanners, which will be introduced respectively. A multimodal microscope provides the ability to image samples with multiple modalities on the same stage, which incorporates the benefits of all modalities. The multimodal microscopes developed in this dissertation are the Keck 3D fusion multimodal microscope 2.0 (3DFM 2.0), upgraded from the old 3DFM with improved performance and flexibility, and the multimodal microscope for targeting small particles (the "Target" system). The control systems developed for both microscopes are low-cost and easy-to-build, with all components off-the-shelf. The control system have not only significantly decreased the complexity and size of the microscope, but also increased the pixel resolution and flexibility. The SMPAF of melanin, activated by a continuous-wave (CW) mode near-infrared (NIR) laser, has potential applications for a low-cost and reliable method of detecting melanin. The photophysics of melanin SMPAF has been studied by theoretical analysis of the excitation process and investigation of the spectra, activation threshold, and photon number absorption of melanin SMPAF. SMPAF images of melanin in mouse hair and skin, mouse melanoma, and human black and white hairs are compared with images taken by conventional multi-photon fluorescence microscopy (MPFM) and confocal reflectance microscopy (CRM). SMPAF images significantly increase specificity and demonstrate the potential to increase sensitivity for melanin detection compared to MPFM images and CRM images. Employing melanin SMPAF imaging to detect melanin inside human skin in vivo has been demonstrated, which proves the effectiveness of melanin detection using SMPAF for medical purposes. Selective melanin ablation with micrometer resolution has been presented using the Target system. Compared to the traditional selective photothermolysis, this method demonstrates higher precision, higher specificity and deeper penetration. Therefore, the SMPAF guided selective ablation of melanin is a promising tool of removing melanin for both medical and cosmetic purposes. Three CPLs have been designed for low-cost linear-motion scanners, low-cost fast spinning scanners and high-precision fast spinning scanners. Each design has been tailored to the industrial manufacturing ability and market demands.

Multimodal Imaging and Spatial Analysis of Ebola Retinal Lesions in 14 Survivors of Ebola Virus Disease.

PubMed

Steptoe, Paul J; Momorie, Fayiah; Fornah, Alimamy D; Komba, Sahr P; Emsley, Elizabeth; Scott, Janet T; Harding, Simon P; Vandy, Matthew J; Sahr, Foday; Beare, Nicholas A V; Semple, Malcolm G

2018-05-03

Differentiation between Ebola retinal lesions and other retinal pathologies in West Africa is important, and the pathogenesis of Ebola retinal disease remains poorly understood. To describe the appearance of Ebola virus disease (EVD) retinal lesions using multimodal imaging to enable inferences on potential pathogenesis. This prospective case series study was carried out at 34 Military Hospital in Freetown, Sierra Leone. Ophthalmological images were analyzed from 14 consecutively identified survivors of EVD of Sierra Leonean origin who had identified Ebola retinal lesions. Multimodal imaging findings including ultra-widefield scanning laser ophthalmoscopy, fundus autofluorescence, swept-source optical coherence tomography (OCT), Humphrey visual field analysis, and spatial analysis. The 14 study participants had a mean (SD) age of 37.1 (8.8) years; 6 (43%) were women. A total of 141 Ebola retinal lesions were observed in 22 of 27 eyes (81%) of these 14 survivors on ultra-widefield imaging. Of these, 41 lesions (29.1%) were accessible to OCT imaging. Retinal lesions were predominantly nonpigmented with a pale-gray appearance. Peripapillary lesions exhibited variable curvatures in keeping with the retinal nerve fiber layer projections. All lesions respected the horizontal raphe and spared the fovea. The OCT imaging demonstrated a V-shaped hyperreflectivity of the outer nuclear layer overlying discontinuities of the ellipsoid zone and interdigitation zone in the smaller lesions. Larger lesions caused a collapse of the retinal layers and loss of retinal thickness. Lesion shapes were variable, but sharp angulations were characteristic. Perilesional areas of dark without pressure (thinned ellipsoid zone hyporeflectivity) accompanied 125 of the 141 lesions (88.7%) to varying extents. We demonstrate OCT evidence of localized pathological changes at the level of the photoreceptors in small lesions among survivors of EVD with retinal lesions. The relevance of associated areas of dark without pressure remains undetermined.
Multimodal brain-tumor segmentation based on Dirichlet process mixture model with anisotropic diffusion and Markov random field prior.

PubMed

Lu, Yisu; Jiang, Jun; Yang, Wei; Feng, Qianjin; Chen, Wufan

2014-01-01

Brain-tumor segmentation is an important clinical requirement for brain-tumor diagnosis and radiotherapy planning. It is well-known that the number of clusters is one of the most important parameters for automatic segmentation. However, it is difficult to define owing to the high diversity in appearance of tumor tissue among different patients and the ambiguous boundaries of lesions. In this study, a nonparametric mixture of Dirichlet process (MDP) model is applied to segment the tumor images, and the MDP segmentation can be performed without the initialization of the number of clusters. Because the classical MDP segmentation cannot be applied for real-time diagnosis, a new nonparametric segmentation algorithm combined with anisotropic diffusion and a Markov random field (MRF) smooth constraint is proposed in this study. Besides the segmentation of single modal brain-tumor images, we developed the algorithm to segment multimodal brain-tumor images by the magnetic resonance (MR) multimodal features and obtain the active tumor and edema in the same time. The proposed algorithm is evaluated using 32 multimodal MR glioma image sequences, and the segmentation results are compared with other approaches. The accuracy and computation time of our algorithm demonstrates very impressive performance and has a great potential for practical real-time clinical use.
Multimodal Brain-Tumor Segmentation Based on Dirichlet Process Mixture Model with Anisotropic Diffusion and Markov Random Field Prior

PubMed Central

Lu, Yisu; Jiang, Jun; Chen, Wufan

2014-01-01

Brain-tumor segmentation is an important clinical requirement for brain-tumor diagnosis and radiotherapy planning. It is well-known that the number of clusters is one of the most important parameters for automatic segmentation. However, it is difficult to define owing to the high diversity in appearance of tumor tissue among different patients and the ambiguous boundaries of lesions. In this study, a nonparametric mixture of Dirichlet process (MDP) model is applied to segment the tumor images, and the MDP segmentation can be performed without the initialization of the number of clusters. Because the classical MDP segmentation cannot be applied for real-time diagnosis, a new nonparametric segmentation algorithm combined with anisotropic diffusion and a Markov random field (MRF) smooth constraint is proposed in this study. Besides the segmentation of single modal brain-tumor images, we developed the algorithm to segment multimodal brain-tumor images by the magnetic resonance (MR) multimodal features and obtain the active tumor and edema in the same time. The proposed algorithm is evaluated using 32 multimodal MR glioma image sequences, and the segmentation results are compared with other approaches. The accuracy and computation time of our algorithm demonstrates very impressive performance and has a great potential for practical real-time clinical use. PMID:25254064
Intraoperative high-field magnetic resonance imaging, multimodal neuronavigation, and intraoperative electrophysiological monitoring-guided surgery for treating supratentorial cavernomas.

PubMed

Li, Fang-Ye; Chen, Xiao-Lei; Xu, Bai-Nan

2016-09-01

To determine the beneficial effects of intraoperative high-field magnetic resonance imaging (MRI), multimodal neuronavigation, and intraoperative electrophysiological monitoring-guided surgery for treating supratentorial cavernomas. Twelve patients with 13 supratentorial cavernomas were prospectively enrolled and operated while using a 1.5 T intraoperative MRI, multimodal neuronavigation, and intraoperative electrophysiological monitoring. All cavernomas were deeply located in subcortical areas or involved critical areas. Intraoperative high-field MRIs were obtained for the intraoperative "visualization" of surrounding eloquent structures, "brain shift" corrections, and navigational plan updates. All cavernomas were successfully resected with guidance from intraoperative MRI, multimodal neuronavigation, and intraoperative electrophysiological monitoring. In 5 cases with supratentorial cavernomas, intraoperative "brain shift" severely deterred locating of the lesions; however, intraoperative MRI facilitated precise locating of these lesions. During long-term (>3 months) follow-up, some or all presenting signs and symptoms improved or resolved in 4 cases, but were unchanged in 7 patients. Intraoperative high-field MRI, multimodal neuronavigation, and intraoperative electrophysiological monitoring are helpful in surgeries for the treatment of small deeply seated subcortical cavernomas.
Positron Emission Tomography Molecular Imaging in Late-Life Depression

PubMed Central

Hirao, Kentaro; Smith, Gwenn S.

2017-01-01

Molecular imaging represents a bridge between basic and clinical neuroscience observations and provides many opportunities for translation and identifying mechanisms that may inform prevention and intervention strategies in late-life depression (LLD). Substantial advances in instrumentation and radiotracer chemistry have resulted in improved sensitivity and spatial resolution and the ability to study in vivo an increasing number of neurotransmitters, neuromodulators, and, importantly, neuropathological processes. Molecular brain imaging studies in LLD will be reviewed, with a primary focus on positron emission tomography. Future directions for the field of molecular imaging in LLD will be discussed, including integrating molecular imaging with genetic, neuropsychiatric, and cognitive outcomes and multimodality neuroimaging. PMID:24394152
Pharmacology Students' Perceptions of Creating Multimodal Digital Explanations

ERIC Educational Resources Information Center

Nielsen, W.; Hoban G.; Hyland, C. J. T.

2017-01-01

Students can now digitally construct their own representations of scientific concepts using a variety of modes including writing, diagrams, 2-D and 3-D models, images or speech, all of which communicate meaning. In this study, final-year chemistry students studying a pharmacology subject created a ''blended media'' digital product as an assignment…
Multimode optical dermoscopy (SkinSpect) analysis for skin with melanocytic nevus

NASA Astrophysics Data System (ADS)

Vasefi, Fartash; MacKinnon, Nicholas; Saager, Rolf; Kelly, Kristen M.; Maly, Tyler; Chave, Robert; Booth, Nicholas; Durkin, Anthony J.; Farkas, Daniel L.

2016-04-01

We have developed a multimode dermoscope (SkinSpect™) capable of illuminating human skin samples in-vivo with spectrally-programmable linearly-polarized light at 33 wavelengths between 468nm and 857 nm. Diffusely reflected photons are separated into collinear and cross-polarized image paths and images captured for each illumination wavelength. In vivo human skin nevi (N = 20) were evaluated with the multimode dermoscope and melanin and hemoglobin concentrations were compared with Spatially Modulated Quantitative Spectroscopy (SMoQS) measurements. Both systems show low correlation between their melanin and hemoglobin concentrations, demonstrating the ability of the SkinSpect™ to separate these molecular signatures and thus act as a biologically plausible device capable of early onset melanoma detection.
Design and applications of a multimodality image data warehouse framework.

PubMed

Wong, Stephen T C; Hoo, Kent Soo; Knowlton, Robert C; Laxer, Kenneth D; Cao, Xinhau; Hawkins, Randall A; Dillon, William P; Arenson, Ronald L

2002-01-01

A comprehensive data warehouse framework is needed, which encompasses imaging and non-imaging information in supporting disease management and research. The authors propose such a framework, describe general design principles and system architecture, and illustrate a multimodality neuroimaging data warehouse system implemented for clinical epilepsy research. The data warehouse system is built on top of a picture archiving and communication system (PACS) environment and applies an iterative object-oriented analysis and design (OOAD) approach and recognized data interface and design standards. The implementation is based on a Java CORBA (Common Object Request Broker Architecture) and Web-based architecture that separates the graphical user interface presentation, data warehouse business services, data staging area, and backend source systems into distinct software layers. To illustrate the practicality of the data warehouse system, the authors describe two distinct biomedical applications--namely, clinical diagnostic workup of multimodality neuroimaging cases and research data analysis and decision threshold on seizure foci lateralization. The image data warehouse framework can be modified and generalized for new application domains.
Design and Applications of a Multimodality Image Data Warehouse Framework

PubMed Central

Wong, Stephen T.C.; Hoo, Kent Soo; Knowlton, Robert C.; Laxer, Kenneth D.; Cao, Xinhau; Hawkins, Randall A.; Dillon, William P.; Arenson, Ronald L.

2002-01-01

A comprehensive data warehouse framework is needed, which encompasses imaging and non-imaging information in supporting disease management and research. The authors propose such a framework, describe general design principles and system architecture, and illustrate a multimodality neuroimaging data warehouse system implemented for clinical epilepsy research. The data warehouse system is built on top of a picture archiving and communication system (PACS) environment and applies an iterative object-oriented analysis and design (OOAD) approach and recognized data interface and design standards. The implementation is based on a Java CORBA (Common Object Request Broker Architecture) and Web-based architecture that separates the graphical user interface presentation, data warehouse business services, data staging area, and backend source systems into distinct software layers. To illustrate the practicality of the data warehouse system, the authors describe two distinct biomedical applications—namely, clinical diagnostic workup of multimodality neuroimaging cases and research data analysis and decision threshold on seizure foci lateralization. The image data warehouse framework can be modified and generalized for new application domains. PMID:11971885
A brain MRI atlas of the common squirrel monkey, Saimiri sciureus

NASA Astrophysics Data System (ADS)

Gao, Yurui; Schilling, Kurt G.; Khare, Shweta P.; Panda, Swetasudha; Choe, Ann S.; Stepniewska, Iwona; Li, Xia; Ding, Zhoahua; Anderson, Adam; Landman, Bennett A.

2014-03-01

The common squirrel monkey, Saimiri sciureus, is a New World monkey with functional and microstructural organization of central nervous system similar to that of humans. It is one of the most commonly used South American primates in biomedical research. Unlike its Old World macaque cousins, no digital atlases have described the organization of the squirrel monkey brain. Here, we present a multi-modal magnetic resonance imaging (MRI) atlas constructed from the brain of an adult female squirrel monkey. In vivo MRI acquisitions include high resolution T2 structural imaging and low resolution diffusion tensor imaging. Ex vivo MRI acquisitions include high resolution T2 structural imaging and high resolution diffusion tensor imaging. Cortical regions were manually annotated on the co-registered volumes based on published histological sections.
Improved multimodal biomarkers for Alzheimer's disease and mild cognitive impairment diagnosis: data from ADNI

NASA Astrophysics Data System (ADS)

Martinez-Torteya, Antonio; Treviño-Alvarado, Víctor; Tamez-Peña, José

2013-02-01

The accurate diagnosis of Alzheimer's disease (AD) and mild cognitive impairment (MCI) confers many clinical research and patient care benefits. Studies have shown that multimodal biomarkers provide better diagnosis accuracy of AD and MCI than unimodal biomarkers, but their construction has been based on traditional statistical approaches. The objective of this work was the creation of accurate AD and MCI diagnostic multimodal biomarkers using advanced bioinformatics tools. The biomarkers were created by exploring multimodal combinations of features using machine learning techniques. Data was obtained from the ADNI database. The baseline information (e.g. MRI analyses, PET analyses and laboratory essays) from AD, MCI and healthy control (HC) subjects with available diagnosis up to June 2012 was mined for case/controls candidates. The data mining yielded 47 HC, 83 MCI and 43 AD subjects for biomarker creation. Each subject was characterized by at least 980 ADNI features. A genetic algorithm feature selection strategy was used to obtain compact and accurate cross-validated nearest centroid biomarkers. The biomarkers achieved training classification accuracies of 0.983, 0.871 and 0.917 for HC vs. AD, HC vs. MCI and MCI vs. AD respectively. The constructed biomarkers were relatively compact: from 5 to 11 features. Those multimodal biomarkers included several widely accepted univariate biomarkers and novel image and biochemical features. Multimodal biomarkers constructed from previously and non-previously AD associated features showed improved diagnostic performance when compared to those based solely on previously AD associated features.
Multimodal Nonlinear Optical Imaging for Sensitive Detection of Multiple Pharmaceutical Solid-State Forms and Surface Transformations.

PubMed

Novakovic, Dunja; Saarinen, Jukka; Rojalin, Tatu; Antikainen, Osmo; Fraser-Miller, Sara J; Laaksonen, Timo; Peltonen, Leena; Isomäki, Antti; Strachan, Clare J

2017-11-07

Two nonlinear imaging modalities, coherent anti-Stokes Raman scattering (CARS) and sum-frequency generation (SFG), were successfully combined for sensitive multimodal imaging of multiple solid-state forms and their changes on drug tablet surfaces. Two imaging approaches were used and compared: (i) hyperspectral CARS combined with principal component analysis (PCA) and SFG imaging and (ii) simultaneous narrowband CARS and SFG imaging. Three different solid-state forms of indomethacin-the crystalline gamma and alpha forms, as well as the amorphous form-were clearly distinguished using both approaches. Simultaneous narrowband CARS and SFG imaging was faster, but hyperspectral CARS and SFG imaging has the potential to be applied to a wider variety of more complex samples. These methodologies were further used to follow crystallization of indomethacin on tablet surfaces under two storage conditions: 30 °C/23% RH and 30 °C/75% RH. Imaging with (sub)micron resolution showed that the approach allowed detection of very early stage surface crystallization. The surfaces progressively crystallized to predominantly (but not exclusively) the gamma form at lower humidity and the alpha form at higher humidity. Overall, this study suggests that multimodal nonlinear imaging is a highly sensitive, solid-state (and chemically) specific, rapid, and versatile imaging technique for understanding and hence controlling (surface) solid-state forms and their complex changes in pharmaceuticals.
A Multi-modal, Discriminative and Spatially Invariant CNN for RGB-D Object Labeling.

PubMed

Asif, Umar; Bennamoun, Mohammed; Sohel, Ferdous

2017-08-30

While deep convolutional neural networks have shown a remarkable success in image classification, the problems of inter-class similarities, intra-class variances, the effective combination of multimodal data, and the spatial variability in images of objects remain to be major challenges. To address these problems, this paper proposes a novel framework to learn a discriminative and spatially invariant classification model for object and indoor scene recognition using multimodal RGB-D imagery. This is achieved through three postulates: 1) spatial invariance - this is achieved by combining a spatial transformer network with a deep convolutional neural network to learn features which are invariant to spatial translations, rotations, and scale changes, 2) high discriminative capability - this is achieved by introducing Fisher encoding within the CNN architecture to learn features which have small inter-class similarities and large intra-class compactness, and 3) multimodal hierarchical fusion - this is achieved through the regularization of semantic segmentation to a multi-modal CNN architecture, where class probabilities are estimated at different hierarchical levels (i.e., imageand pixel-levels), and fused into a Conditional Random Field (CRF)- based inference hypothesis, the optimization of which produces consistent class labels in RGB-D images. Extensive experimental evaluations on RGB-D object and scene datasets, and live video streams (acquired from Kinect) show that our framework produces superior object and scene classification results compared to the state-of-the-art methods.
Application of high-resolution linear Radon transform for Rayleigh-wave dispersive energy imaging and mode separating

USGS Publications Warehouse

Luo, Y.; Xia, J.; Miller, R.D.; Liu, J.; Xu, Y.; Liu, Q.

2008-01-01

Multichannel Analysis of Surface Waves (MASW) analysis is an efficient tool to obtain the vertical shear-wave profile. One of the key steps in the MASW method is to generate an image of dispersive energy in the frequency-velocity domain, so dispersion curves can be determined by picking peaks of dispersion energy. In this paper, we image Rayleigh-wave dispersive energy and separate multimodes from a multichannel record by high-resolution linear Radon transform (LRT). We first introduce Rayleigh-wave dispersive energy imaging by high-resolution LRT. We then show the process of Rayleigh-wave mode separation. Results of synthetic and real-world examples demonstrate that (1) compared with slant stacking algorithm, high-resolution LRT can improve the resolution of images of dispersion energy by more than 50% (2) high-resolution LRT can successfully separate multimode dispersive energy of Rayleigh waves with high resolution; and (3) multimode separation and reconstruction expand frequency ranges of higher mode dispersive energy, which not only increases the investigation depth but also provides a means to accurately determine cut-off frequencies.
Modeling 4D Pathological Changes by Leveraging Normative Models

PubMed Central

Wang, Bo; Prastawa, Marcel; Irimia, Andrei; Saha, Avishek; Liu, Wei; Goh, S.Y. Matthew; Vespa, Paul M.; Van Horn, John D.; Gerig, Guido

2016-01-01

With the increasing use of efficient multimodal 3D imaging, clinicians are able to access longitudinal imaging to stage pathological diseases, to monitor the efficacy of therapeutic interventions, or to assess and quantify rehabilitation efforts. Analysis of such four-dimensional (4D) image data presenting pathologies, including disappearing and newly appearing lesions, represents a significant challenge due to the presence of complex spatio-temporal changes. Image analysis methods for such 4D image data have to include not only a concept for joint segmentation of 3D datasets to account for inherent correlations of subject-specific repeated scans but also a mechanism to account for large deformations and the destruction and formation of lesions (e.g., edema, bleeding) due to underlying physiological processes associated with damage, intervention, and recovery. In this paper, we propose a novel framework that provides a joint segmentation-registration framework to tackle the inherent problem of image registration in the presence of objects not present in all images of the time series. Our methodology models 4D changes in pathological anatomy across time and and also provides an explicit mapping of a healthy normative template to a subject’s image data with pathologies. Since atlas-moderated segmentation methods cannot explain appearance and locality pathological structures that are not represented in the template atlas, the new framework provides different options for initialization via a supervised learning approach, iterative semisupervised active learning, and also transfer learning, which results in a fully automatic 4D segmentation method. We demonstrate the effectiveness of our novel approach with synthetic experiments and a 4D multimodal MRI dataset of severe traumatic brain injury (TBI), including validation via comparison to expert segmentations. However, the proposed methodology is generic in regard to different clinical applications requiring quantitative analysis of 4D imaging representing spatio-temporal changes of pathologies. PMID:27818606
PIRATE: pediatric imaging response assessment and targeting environment

NASA Astrophysics Data System (ADS)

Glenn, Russell; Zhang, Yong; Krasin, Matthew; Hua, Chiaho

2010-02-01

By combining the strengths of various imaging modalities, the multimodality imaging approach has potential to improve tumor staging, delineation of tumor boundaries, chemo-radiotherapy regime design, and treatment response assessment in cancer management. To address the urgent needs for efficient tools to analyze large-scale clinical trial data, we have developed an integrated multimodality, functional and anatomical imaging analysis software package for target definition and therapy response assessment in pediatric radiotherapy (RT) patients. Our software provides quantitative tools for automated image segmentation, region-of-interest (ROI) histogram analysis, spatial volume-of-interest (VOI) analysis, and voxel-wise correlation across modalities. To demonstrate the clinical applicability of this software, histogram analyses were performed on baseline and follow-up 18F-fluorodeoxyglucose (18F-FDG) PET images of nine patients with rhabdomyosarcoma enrolled in an institutional clinical trial at St. Jude Children's Research Hospital. In addition, we combined 18F-FDG PET, dynamic-contrast-enhanced (DCE) MR, and anatomical MR data to visualize the heterogeneity in tumor pathophysiology with the ultimate goal of adaptive targeting of regions with high tumor burden. Our software is able to simultaneously analyze multimodality images across multiple time points, which could greatly speed up the analysis of large-scale clinical trial data and validation of potential imaging biomarkers.
Ex vivo catheter-based imaging of coronary atherosclerosis using multimodality OCT and NIRAF excited at 633 nm

PubMed Central

Wang, Hao; Gardecki, Joseph A.; Ughi, Giovanni J.; Jacques, Paulino Vacas; Hamidi, Ehsan; Tearney, Guillermo J.

2015-01-01

While optical coherence tomography (OCT) has been shown to be capable of imaging coronary plaque microstructure, additional chemical/molecular information may be needed in order to determine which lesions are at risk of causing an acute coronary event. In this study, we used a recently developed imaging system and double-clad fiber (DCF) catheter capable of simultaneously acquiring both OCT and red excited near-infrared autofluorescence (NIRAF) images (excitation: 633 nm, emission: 680nm to 900nm). We found that NIRAF is elevated in lesions that contain necrotic core – a feature that is critical for vulnerable plaque diagnosis and that is not readily discriminated by OCT alone. We first utilized a DCF ball lens probe and a bench top setup to acquire en face NIRAF images of aortic plaques ex vivo (n = 20). In addition, we used the OCT-NIRAF system and fully assembled catheters to acquire multimodality images from human coronary arteries (n = 15) prosected from human cadaver hearts (n = 5). Comparison of these images with corresponding histology demonstrated that necrotic core plaques exhibited significantly higher NIRAF intensity than other plaque types. These results suggest that multimodality intracoronary OCT-NIRAF imaging technology may be used in the future to provide improved characterization of coronary artery disease in human patients. PMID:25909020
Gold Nanoconstructs for Multimodal Diagnostic Imaging and Photothermal Cancer Therapy

NASA Astrophysics Data System (ADS)

Coughlin, Andrew James

Cancer accounts for nearly 1 out of every 4 deaths in the United States, and because conventional treatments are limited by morbidity and off-target toxicities, improvements in cancer management are needed. This thesis further develops nanoparticle-assisted photothermal therapy (NAPT) as a viable treatment option for cancer patients. NAPT enables localized ablation of disease because heat generation only occurs where tissue permissive near-infrared (NIR) light and absorbing nanoparticles are combined, leaving surrounding normal tissue unharmed. Two principle approaches were investigated to improve the specificity of this technique: multimodal imaging and molecular targeting. Multimodal imaging affords the ability to guide NIR laser application for site-specific NAPT and more holistic characterization of disease by combining the advantages of several diagnostic technologies. Towards the goal of image-guided NAPT, gadolinium-conjugated gold-silica nanoshells were engineered and demonstrated to enhance imaging contrast across a range of diagnostic modes, including T1-weighted magnetic resonance imaging, X-Ray, optical coherence tomography, reflective confocal microscopy, and two-photon luminescence in vitro as well as within an animal tumor model. Additionally, the nanoparticle conjugates were shown to effectively convert NIR light to heat for applications in photothermal therapy. Therefore, the broad utility of gadolinium-nanoshells for anatomic localization of tissue lesions, molecular characterization of malignancy, and mediators of ablation was established. Molecular targeting strategies may also improve NAPT by promoting nanoparticle uptake and retention within tumors and enhancing specificity when malignant and normal tissue interdigitate. Here, ephrinA1 protein ligands were conjugated to nanoshell surfaces for particle homing to overexpressed EphA2 receptors on prostate cancer cells. In vitro, successful targeting and subsequent photothermal ablation of prostate cancer cells was achieved with negligible nanoshell binding to normal cells. In vivo however, ephrinA1-nanoshells did not promote enhanced therapeutic outcomes in mice bearing subcutaneous prostate cancer tumors treated with NAPT compared to nontargeted particles. Nonetheless, both treatment groups demonstrated effective ablation of prostate tumors, as evidenced by tumor tissue regression. Further investigation is warranted to overcome probable protein immunogenicity that offsets ephrinA1 targeting in vivo. With future study, photothermal therapy with multimodal gadolinium-conjugated and molecularly targeted nanoshells may offer a viable treatment option for cancer patients in the clinic.
Cuticular Drusen: Clinical Phenotypes and Natural History Defined Using Multimodal Imaging.

PubMed

Balaratnasingam, Chandrakumar; Cherepanoff, Svetlana; Dolz-Marco, Rosa; Killingsworth, Murray; Chen, Fred K; Mendis, Randev; Mrejen, Sarah; Too, Lay Khoon; Gal-Or, Orly; Curcio, Christine A; Freund, K Bailey; Yannuzzi, Lawrence A

2018-01-01

To define the range and life cycles of cuticular drusen phenotypes using multimodal imaging and to review the histologic characteristics of cuticular drusen. Retrospective, observational cohort study and experimental laboratory study. Two hundred forty eyes of 120 clinic patients with a cuticular drusen phenotype and 4 human donor eyes with cuticular drusen (n = 2), soft drusen (n = 1), and hard drusen (n = 1). We performed a retrospective review of clinical and multimodal imaging data of patients with a cuticular drusen phenotype. Patients had undergone imaging with various combinations of color photography, fluorescein angiography, indocyanine green angiography, near-infrared reflectance, fundus autofluorescence, high-resolution OCT, and ultrawide-field imaging. Human donor eyes underwent processing for high-resolution light and electron microscopy. Appearance of cuticular drusen in multimodal imaging and the topography of a cuticular drusen distribution; age-dependent variations in cuticular drusen phenotypes, including the occurrence of retinal pigment epithelium (RPE) abnormalities, choroidal neovascularization, acquired vitelliform lesions (AVLs), and geographic atrophy (GA); and ultrastructural and staining characteristics of druse subtypes. The mean age of patients at the first visit was 57.9±13.4 years. Drusen and RPE changes were seen in the peripheral retina, anterior to the vortex veins, in 21.8% of eyes. Of eyes with more than 5 years of follow-up, cuticular drusen disappeared from view in 58.3% of eyes, drusen coalescence was seen in 70.8% of eyes, and new RPE pigmentary changes developed in 56.2% of eyes. Retinal pigment epithelium abnormalities, AVLs, neovascularization, and GA occurred at a frequency of 47.5%, 24.2%, 12.5%, and 25%, respectively, and were significantly more common in patients older than 60 years of age (all P < 0.015). Occurrence of GA and neovascularization were important determinants of final visual acuity in eyes with the cuticular drusen phenotype (both P < 0.015). Small cuticular drusen typically demonstrated a homogenous ultrastructural appearance similar to hard drusen, whereas fragmentation of the central and basal contents was seen frequently in larger cuticular drusen. Although the ultrastructural characteristics of cuticular drusen appear more similar to those of hard drusen, their lifecycle and macular complications are more comparable with those of soft drusen. Cuticular drusen phenotype may confer a unique risk for the development of GA and neovascularization. Copyright © 2017 American Academy of Ophthalmology. Published by Elsevier Inc. All rights reserved.
Signal and image processing algorithm performance in a virtual and elastic computing environment

NASA Astrophysics Data System (ADS)

Bennett, Kelly W.; Robertson, James

2013-05-01

The U.S. Army Research Laboratory (ARL) supports the development of classification, detection, tracking, and localization algorithms using multiple sensing modalities including acoustic, seismic, E-field, magnetic field, PIR, and visual and IR imaging. Multimodal sensors collect large amounts of data in support of algorithm development. The resulting large amount of data, and their associated high-performance computing needs, increases and challenges existing computing infrastructures. Purchasing computer power as a commodity using a Cloud service offers low-cost, pay-as-you-go pricing models, scalability, and elasticity that may provide solutions to develop and optimize algorithms without having to procure additional hardware and resources. This paper provides a detailed look at using a commercial cloud service provider, such as Amazon Web Services (AWS), to develop and deploy simple signal and image processing algorithms in a cloud and run the algorithms on a large set of data archived in the ARL Multimodal Signatures Database (MMSDB). Analytical results will provide performance comparisons with existing infrastructure. A discussion on using cloud computing with government data will discuss best security practices that exist within cloud services, such as AWS.

Research of the multimodal brain-tumor segmentation algorithm

NASA Astrophysics Data System (ADS)

Lu, Yisu; Chen, Wufan

2015-12-01

It is well-known that the number of clusters is one of the most important parameters for automatic segmentation. However, it is difficult to define owing to the high diversity in appearance of tumor tissue among different patients and the ambiguous boundaries of lesions. In this study, a nonparametric mixture of Dirichlet process (MDP) model is applied to segment the tumor images, and the MDP segmentation can be performed without the initialization of the number of clusters. A new nonparametric segmentation algorithm combined with anisotropic diffusion and a Markov random field (MRF) smooth constraint is proposed in this study. Besides the segmentation of single modal brain tumor images, we developed the algorithm to segment multimodal brain tumor images by the magnetic resonance (MR) multimodal features and obtain the active tumor and edema in the same time. The proposed algorithm is evaluated and compared with other approaches. The accuracy and computation time of our algorithm demonstrates very impressive performance.
Albumin based versatile multifunctional nanocarriers for cancer therapy: Fabrication, surface modification, multimodal therapeutics and imaging approaches.

PubMed

Kudarha, Ritu R; Sawant, Krutika K

2017-12-01

Albumin is a versatile protein used as a carrier system for cancer therapeutics. As a carrier it can provide tumor specificity, reduce drug related toxicity, maintain therapeutic concentration of the active moiety like drug, gene, peptide, protein etc. for long period of time and also reduce drug related toxicities. Apart from cancer therapy, it is also utilized in the imaging and multimodal therapy of cancer. This review highlights the important properties, structure and types of albumin based nanocarriers with regards to their use for cancer targeting. It also provides brief discussion on methods of preparation of these nanocarriers and their surface modification. Applications of albumin nanocarriers for cancer therapy, gene delivery, imaging, phototherapy and multimodal therapy have also been discussed. This review also provides brief discussion about albumin based marketed nano formulations and those under clinical trials. Copyright © 2017 Elsevier B.V. All rights reserved.
Multimodal MR imaging in hepatic encephalopathy: state of the art.

PubMed

Zhang, Xiao Dong; Zhang, Long Jiang

2018-06-01

Hepatic encephalopathy (HE) is a neurological or neuropsychological complication due to liver failure or portosystemic shunting. The clinical manifestation is highly variable, which can exhibit mild cognitive or motor impairment initially, or gradually progress to a coma, even death, without treatment. Neuroimaging plays a critical role in uncovering the neural mechanism of HE. In particular, multimodality MR imaging is able to assess both structural and functional derangements of the brain with HE in focal or neural network perspectives. In recent years, there has been rapid development in novel MR technologies and applications to investigate the pathophysiological mechanism of HE. Therefore, it is necessary to update the latest MR findings regarding HE by use of multimodality MRI to refine and deepen our understanding of the neural traits in HE. Herein, this review highlights the latest MR imaging findings in HE to refresh our understanding of MRI application in HE.
Spinal metastases: multimodality imaging in diagnosis and stereotactic body radiation therapy planning.

PubMed

Jabehdar Maralani, Pejman; Lo, Simon S; Redmond, Kristin; Soliman, Hany; Myrehaug, Sten; Husain, Zain A; Heyn, Chinthaka; Kapadia, Anish; Chan, Aimee; Sahgal, Arjun

2017-01-01

Due to increased effectiveness of cancer treatments and increasing survival rates, metastatic disease has become more frequent compared to the past, with the spine being the most common site of bony metastases. Diagnostic imaging is an integral part of screening, diagnosis and follow-up of spinal metastases. In this article, we review the principles of multimodality imaging for tumor detection with respect to their value for diagnosis and stereotactic body radiation therapy planning for spinal metastases. We will also review the current international consensus agreement for stereotactic body radiation therapy planning, and the role of imaging in achieving the best possible treatment plan.
Simultaneous in vivo imaging of melanin and lipofuscin in the retina with multimodal photoacoustic ophthalmoscopy

NASA Astrophysics Data System (ADS)

Zhang, Xiangyang; Zhang, Hao F.; Zhou, Lixiang; Jiao, Shuliang

2012-02-01

We combined photoacoustic ophthalmoscopy (PAOM) with autofluorescence imaging for simultaneous in vivo imaging of dual molecular contrasts in the retina using a single light source. The dual molecular contrasts come from melanin and lipofuscin in the retinal pigment epithelium (RPE). Melanin and lipofuscin are two types of pigments and are believed to play opposite roles (protective vs. exacerbate) in the RPE in the aging process. We successfully imaged the retina of pigmented and albino rats at different ages. The experimental results showed that multimodal PAOM system can be a potentially powerful tool in the study of age-related degenerative retinal diseases.
Live animal myelin histomorphometry of the spinal cord with video-rate multimodal nonlinear microendoscopy

NASA Astrophysics Data System (ADS)

Bélanger, Erik; Crépeau, Joël; Laffray, Sophie; Vallée, Réal; De Koninck, Yves; Côté, Daniel

2012-02-01

In vivo imaging of cellular dynamics can be dramatically enabling to understand the pathophysiology of nervous system diseases. To fully exploit the power of this approach, the main challenges have been to minimize invasiveness and maximize the number of concurrent optical signals that can be combined to probe the interplay between multiple cellular processes. Label-free coherent anti-Stokes Raman scattering (CARS) microscopy, for example, can be used to follow demyelination in neurodegenerative diseases or after trauma, but myelin imaging alone is not sufficient to understand the complex sequence of events that leads to the appearance of lesions in the white matter. A commercially available microendoscope is used here to achieve minimally invasive, video-rate multimodal nonlinear imaging of cellular processes in live mouse spinal cord. The system allows for simultaneous CARS imaging of myelin sheaths and two-photon excitation fluorescence microendoscopy of microglial cells and axons. Morphometric data extraction at high spatial resolution is also described, with a technique for reducing motion-related imaging artifacts. Despite its small diameter, the microendoscope enables high speed multimodal imaging over wide areas of tissue, yet at resolution sufficient to quantify subtle differences in myelin thickness and microglial motility.
Detail-enhanced multimodality medical image fusion based on gradient minimization smoothing filter and shearing filter.

PubMed

Liu, Xingbin; Mei, Wenbo; Du, Huiqian

2018-02-13

In this paper, a detail-enhanced multimodality medical image fusion algorithm is proposed by using proposed multi-scale joint decomposition framework (MJDF) and shearing filter (SF). The MJDF constructed with gradient minimization smoothing filter (GMSF) and Gaussian low-pass filter (GLF) is used to decompose source images into low-pass layers, edge layers, and detail layers at multiple scales. In order to highlight the detail information in the fused image, the edge layer and the detail layer in each scale are weighted combined into a detail-enhanced layer. As directional filter is effective in capturing salient information, so SF is applied to the detail-enhanced layer to extract geometrical features and obtain directional coefficients. Visual saliency map-based fusion rule is designed for fusing low-pass layers, and the sum of standard deviation is used as activity level measurement for directional coefficients fusion. The final fusion result is obtained by synthesizing the fused low-pass layers and directional coefficients. Experimental results show that the proposed method with shift-invariance, directional selectivity, and detail-enhanced property is efficient in preserving and enhancing detail information of multimodality medical images. Graphical abstract The detailed implementation of the proposed medical image fusion algorithm.
Optical Biomarkers of Serous and Mucinous Human Ovarian Tumor Assessed with Nonlinear Optics Microscopies

PubMed Central

Adur, Javier; Pelegati, Vitor B.; de Thomaz, Andre A.; Baratti, Mariana O.; Almeida, Diogo B.; Andrade, L. A. L. A.; Bottcher-Luiz, Fátima; Carvalho, Hernandes F.; Cesar, Carlos L.

2012-01-01

Background Nonlinear optical (NLO) microscopy techniques have potential to improve the early detection of epithelial ovarian cancer. In this study we showed that multimodal NLO microscopies, including two-photon excitation fluorescence (TPEF), second-harmonic generation (SHG), third-harmonic generation (THG) and fluorescence lifetime imaging microscopy (FLIM) can detect morphological and metabolic changes associated with ovarian cancer progression. Methodology/Principal Findings We obtained strong TPEF + SHG + THG signals from fixed samples stained with Hematoxylin & Eosin (H&E) and robust FLIM signal from fixed unstained samples. Particularly, we imaged 34 ovarian biopsies from different patients (median age, 49 years) including 5 normal ovarian tissue, 18 serous tumors and 11 mucinous tumors with the multimodal NLO platform developed in our laboratory. We have been able to distinguish adenomas, borderline, and adenocarcinomas specimens. Using a complete set of scoring methods we found significant differences in the content, distribution and organization of collagen fibrils in the stroma as well as in the morphology and fluorescence lifetime from epithelial ovarian cells. Conclusions/Significance NLO microscopes provide complementary information about tissue microstructure, showing distinctive patterns for serous and mucinous ovarian tumors. The results provide a basis to interpret future NLO images of ovarian tissue and lay the foundation for future in vivo optical evaluation of premature ovarian lesions. PMID:23056557
Medical image informatics infrastructure design and applications.

PubMed

Huang, H K; Wong, S T; Pietka, E

1997-01-01

Picture archiving and communication systems (PACS) is a system integration of multimodality images and health information systems designed for improving the operation of a radiology department. As it evolves, PACS becomes a hospital image document management system with a voluminous image and related data file repository. A medical image informatics infrastructure can be designed to take advantage of existing data, providing PACS with add-on value for health care service, research, and education. A medical image informatics infrastructure (MIII) consists of the following components: medical images and associated data (including PACS database), image processing, data/knowledge base management, visualization, graphic user interface, communication networking, and application oriented software. This paper describes these components and their logical connection, and illustrates some applications based on the concept of the MIII.
DBSAR's First Multimode Flight Campaign

NASA Technical Reports Server (NTRS)

Rincon, Rafael F.; Vega, Manuel; Buenfil, Manuel; Geist, Alessandro; Hilliard, Lawrence; Racette, Paul

2010-01-01

The Digital Beamforming SAR (DBSAR) is an airborne imaging radar system that combines phased array technology, reconfigurable on-board processing and waveform generation, and advances in signal processing to enable techniques not possible with conventional SARs. The system exploits the versatility inherently in phased-array technology with a state-of-the-art data acquisition and real-time processor in order to implement multi-mode measurement techniques in a single radar system. Operational modes include scatterometry over multiple antenna beams, Synthetic Aperture Radar (SAR) over several antenna beams, or Altimetry. The radar was flight tested in October 2008 on board of the NASA P3 aircraft over the Delmarva Peninsula, MD. The results from the DBSAR system performance is presented.
3D multi-scale FCN with random modality voxel dropout learning for Intervertebral Disc Localization and Segmentation from Multi-modality MR Images.

PubMed

Li, Xiaomeng; Dou, Qi; Chen, Hao; Fu, Chi-Wing; Qi, Xiaojuan; Belavý, Daniel L; Armbrecht, Gabriele; Felsenberg, Dieter; Zheng, Guoyan; Heng, Pheng-Ann

2018-04-01

Intervertebral discs (IVDs) are small joints that lie between adjacent vertebrae. The localization and segmentation of IVDs are important for spine disease diagnosis and measurement quantification. However, manual annotation is time-consuming and error-prone with limited reproducibility, particularly for volumetric data. In this work, our goal is to develop an automatic and accurate method based on fully convolutional networks (FCN) for the localization and segmentation of IVDs from multi-modality 3D MR data. Compared with single modality data, multi-modality MR images provide complementary contextual information, which contributes to better recognition performance. However, how to effectively integrate such multi-modality information to generate accurate segmentation results remains to be further explored. In this paper, we present a novel multi-scale and modality dropout learning framework to locate and segment IVDs from four-modality MR images. First, we design a 3D multi-scale context fully convolutional network, which processes the input data in multiple scales of context and then merges the high-level features to enhance the representation capability of the network for handling the scale variation of anatomical structures. Second, to harness the complementary information from different modalities, we present a random modality voxel dropout strategy which alleviates the co-adaption issue and increases the discriminative capability of the network. Our method achieved the 1st place in the MICCAI challenge on automatic localization and segmentation of IVDs from multi-modality MR images, with a mean segmentation Dice coefficient of 91.2% and a mean localization error of 0.62 mm. We further conduct extensive experiments on the extended dataset to validate our method. We demonstrate that the proposed modality dropout strategy with multi-modality images as contextual information improved the segmentation accuracy significantly. Furthermore, experiments conducted on extended data collected from two different time points demonstrate the efficacy of our method on tracking the morphological changes in a longitudinal study. Copyright © 2018 Elsevier B.V. All rights reserved.
Magnetization-prepared rapid acquisition with gradient echo magnetic resonance imaging signal and texture features for the prediction of mild cognitive impairment to Alzheimer's disease progression.

PubMed

Martinez-Torteya, Antonio; Rodriguez-Rojas, Juan; Celaya-Padilla, José M; Galván-Tejada, Jorge I; Treviño, Victor; Tamez-Peña, Jose

2014-10-01

Early diagnoses of Alzheimer's disease (AD) would confer many benefits. Several biomarkers have been proposed to achieve such a task, where features extracted from magnetic resonance imaging (MRI) have played an important role. However, studies have focused exclusively on morphological characteristics. This study aims to determine whether features relating to the signal and texture of the image could predict mild cognitive impairment (MCI) to AD progression. Clinical, biological, and positron emission tomography information and MRI images of 62 subjects from the AD neuroimaging initiative were used in this study, extracting 4150 features from each MRI. Within this multimodal database, a feature selection algorithm was used to obtain an accurate and small logistic regression model, generated by a methodology that yielded a mean blind test accuracy of 0.79. This model included six features, five of them obtained from the MRI images, and one obtained from genotyping. A risk analysis divided the subjects into low-risk and high-risk groups according to a prognostic index. The groups were statistically different ([Formula: see text]). These results demonstrated that MRI features related to both signal and texture add MCI to AD predictive power, and supported the ongoing notion that multimodal biomarkers outperform single-modality ones.
Vision 20/20: Simultaneous CT-MRI — Next chapter of multimodality imaging

DOE Office of Scientific and Technical Information (OSTI.GOV)

Wang, Ge, E-mail: wangg6@rpi.edu; Xi, Yan; Gjesteby, Lars

Multimodality imaging systems such as positron emission tomography-computed tomography (PET-CT) and MRI-PET are widely available, but a simultaneous CT-MRI instrument has not been developed. Synergies between independent modalities, e.g., CT, MRI, and PET/SPECT can be realized with image registration, but such postprocessing suffers from registration errors that can be avoided with synchronized data acquisition. The clinical potential of simultaneous CT-MRI is significant, especially in cardiovascular and oncologic applications where studies of the vulnerable plaque, response to cancer therapy, and kinetic and dynamic mechanisms of targeted agents are limited by current imaging technologies. The rationale, feasibility, and realization of simultaneous CT-MRImore » are described in this perspective paper. The enabling technologies include interior tomography, unique gantry designs, open magnet and RF sequences, and source and detector adaptation. Based on the experience with PET-CT, PET-MRI, and MRI-LINAC instrumentation where hardware innovation and performance optimization were instrumental to construct commercial systems, the authors provide top-level concepts for simultaneous CT-MRI to meet clinical requirements and new challenges. Simultaneous CT-MRI fills a major gap of modality coupling and represents a key step toward the so-called “omnitomography” defined as the integration of all relevant imaging modalities for systems biology and precision medicine.« less
A low-cost multimodal head-mounted display system for neuroendoscopic surgery.

PubMed

Xu, Xinghua; Zheng, Yi; Yao, Shujing; Sun, Guochen; Xu, Bainan; Chen, Xiaolei

2018-01-01

With rapid advances in technology, wearable devices as head-mounted display (HMD) have been adopted for various uses in medical science, ranging from simply aiding in fitness to assisting surgery. We aimed to investigate the feasibility and practicability of a low-cost multimodal HMD system in neuroendoscopic surgery. A multimodal HMD system, mainly consisted of a HMD with two built-in displays, an action camera, and a laptop computer displaying reconstructed medical images, was developed to assist neuroendoscopic surgery. With this intensively integrated system, the neurosurgeon could freely switch between endoscopic image, three-dimensional (3D) reconstructed virtual endoscopy images, and surrounding environment images. Using a leap motion controller, the neurosurgeon could adjust or rotate the 3D virtual endoscopic images at a distance to better understand the positional relation between lesions and normal tissues at will. A total of 21 consecutive patients with ventricular system diseases underwent neuroendoscopic surgery with the aid of this system. All operations were accomplished successfully, and no system-related complications occurred. The HMD was comfortable to wear and easy to operate. Screen resolution of the HMD was high enough for the neurosurgeon to operate carefully. With the system, the neurosurgeon might get a better comprehension on lesions by freely switching among images of different modalities. The system had a steep learning curve, which meant a quick increment of skill with it. Compared with commercially available surgical assistant instruments, this system was relatively low-cost. The multimodal HMD system is feasible, practical, helpful, and relatively cost efficient in neuroendoscopic surgery.
2017 Multimodality Appropriate Use Criteria for Noninvasive Cardiac Imaging: Expert Consensus of the Asian Society of Cardiovascular Imaging.

PubMed

Beck, Kyongmin Sarah; Kim, Jeong A; Choe, Yeon Hyeon; Hian, Sim Kui; Hoe, John; Hong, Yoo Jin; Kim, Sung Mok; Kim, Tae Hoon; Kim, Young Jin; Kim, Yun Hyeon; Kuribayashi, Sachio; Lee, Jongmin; Leong, Lilian; Lim, Tae-Hwan; Lu, Bin; Park, Jae Hyung; Sakuma, Hajime; Yang, Dong Hyun; Yaw, Tan Swee; Wan, Yung-Liang; Zhang, Zhaoqi; Zhao, Shihua; Yong, Hwan Seok

2017-01-01

In 2010, the Asian Society of Cardiovascular Imaging (ASCI) provided recommendations for cardiac CT and MRI, and this document reflects an update of the 2010 ASCI appropriate use criteria (AUC). In 2016, the ASCI formed a new working group for revision of AUC for noninvasive cardiac imaging. A major change that we made in this document is the rating of various noninvasive tests (exercise electrocardiogram, echocardiography, positron emission tomography, single-photon emission computed tomography, radionuclide imaging, cardiac magnetic resonance, and cardiac computed tomography/angiography), compared side by side for their applications in various clinical scenarios. Ninety-five clinical scenarios were developed from eight selected pre-existing guidelines and classified into four sections as follows: 1) detection of coronary artery disease, symptomatic or asymptomatic; 2) cardiac evaluation in various clinical scenarios; 3) use of imaging modality according to prior testing; and 4) evaluation of cardiac structure and function. The clinical scenarios were scored by a separate rating committee on a scale of 1-9 to designate appropriate use, uncertain use, or inappropriate use according to a modified Delphi method. Overall, the AUC ratings for CT were higher than those of previous guidelines. These new AUC provide guidance for clinicians choosing among available testing modalities for various cardiac diseases and are also unique, given that most previous AUC for noninvasive imaging include only one imaging technique. As cardiac imaging is multimodal in nature, we believe that these AUC will be more useful for clinical decision making.
Fast, cheap and in control: spectral imaging with handheld devices

NASA Astrophysics Data System (ADS)

Gooding, Edward A.; Deutsch, Erik R.; Huehnerhoff, Joseph; Hajian, Arsen R.

2017-05-01

Remote sensing has moved out of the laboratory and into the real world. Instruments using reflection or Raman imaging modalities become faster, cheaper and more powerful annually. Enabling technologies include virtual slit spectrometer design, high power multimode diode lasers, fast open-loop scanning systems, low-noise IR-sensitive array detectors and low-cost computers with touchscreen interfaces. High-volume manufacturing assembles these components into inexpensive portable or handheld devices that make possible sophisticated decision-making based on robust data analytics. Examples include threat, hazmat and narcotics detection; remote gas sensing; biophotonic screening; environmental remediation and a host of other applications.
Fast multi-core based multimodal registration of 2D cross-sections and 3D datasets

PubMed Central

2010-01-01

Background Solving bioinformatics tasks often requires extensive computational power. Recent trends in processor architecture combine multiple cores into a single chip to improve overall performance. The Cell Broadband Engine (CBE), a heterogeneous multi-core processor, provides power-efficient and cost-effective high-performance computing. One application area is image analysis and visualisation, in particular registration of 2D cross-sections into 3D image datasets. Such techniques can be used to put different image modalities into spatial correspondence, for example, 2D images of histological cuts into morphological 3D frameworks. Results We evaluate the CBE-driven PlayStation 3 as a high performance, cost-effective computing platform by adapting a multimodal alignment procedure to several characteristic hardware properties. The optimisations are based on partitioning, vectorisation, branch reducing and loop unrolling techniques with special attention to 32-bit multiplies and limited local storage on the computing units. We show how a typical image analysis and visualisation problem, the multimodal registration of 2D cross-sections and 3D datasets, benefits from the multi-core based implementation of the alignment algorithm. We discuss several CBE-based optimisation methods and compare our results to standard solutions. More information and the source code are available from http://cbe.ipk-gatersleben.de. Conclusions The results demonstrate that the CBE processor in a PlayStation 3 accelerates computational intensive multimodal registration, which is of great importance in biological/medical image processing. The PlayStation 3 as a low cost CBE-based platform offers an efficient option to conventional hardware to solve computational problems in image processing and bioinformatics. PMID:20064262
Multimodal approaches to functional connectivity in autism spectrum disorders: An integrative perspective.

PubMed

Mash, Lisa E; Reiter, Maya A; Linke, Annika C; Townsend, Jeanne; Müller, Ralph-Axel

2018-05-01

Atypical functional connectivity has been implicated in autism spectrum disorders (ASDs). However, the literature to date has been largely inconsistent, with mixed and conflicting reports of hypo- and hyper-connectivity. These discrepancies are partly due to differences between various neuroimaging modalities. Functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and magnetoencephalography (MEG) measure distinct indices of functional connectivity (e.g., blood-oxygenation level-dependent [BOLD] signal vs. electrical activity). Furthermore, each method has unique benefits and disadvantages with respect to spatial and temporal resolution, vulnerability to specific artifacts, and practical implementation. Thus far, functional connectivity research on ASDs has remained almost exclusively unimodal; therefore, interpreting findings across modalities remains a challenge. Multimodal integration of fMRI, EEG, and MEG data is critical in resolving discrepancies in the literature, and working toward a unifying framework for interpreting past and future findings. This review aims to provide a theoretical foundation for future multimodal research on ASDs. First, we will discuss the merits and shortcomings of several popular theories in ASD functional connectivity research, using examples from the literature to date. Next, the neurophysiological relationships between imaging modalities, including their relationship with invasive neural recordings, will be reviewed. Finally, methodological approaches to multimodal data integration will be presented, and their future application to ASDs will be discussed. Analyses relating transient patterns of neural activity ("states") are particularly promising. This strategy provides a comparable measure across modalities, captures complex spatiotemporal patterns, and is a natural extension of recent dynamic fMRI research in ASDs. © 2017 Wiley Periodicals, Inc. Develop Neurobiol 78: 456-473, 2018. © 2017 Wiley Periodicals, Inc.
A Multimodal Imaging Protocol, (123)I/(99)Tc-Sestamibi, SPECT, and SPECT/CT, in Primary Hyperparathyroidism Adds Limited Benefit for Preoperative Localization.

PubMed

Lee, Grace S; McKenzie, Travis J; Mullan, Brian P; Farley, David R; Thompson, Geoffrey B; Richards, Melanie L

2016-03-01

Focused parathyroidectomy in primary hyperparathyroidism (1°HPT) is possible with accurate preoperative localization and intraoperative PTH monitoring (IOPTH). The added benefit of multimodal imaging techniques for operative success is unknown. Patients with 1°HPT, who underwent parathyroidectomy in 2012-2014 at a single institution, were retrospectively reviewed. Only the patients who underwent the standardized multimodal imaging workup consisting of (123)I/(99)Tc-sestamibi subtraction scintigraphy, SPECT, and SPECT/CT were assessed. Of 360 patients who were identified, a curative operation was performed in 96%, using pre-operative imaging and IOPTH. Imaging analysis showed that (123)I/(99)Tc-sestamibi had a sensitivity of 86% (95% CI 82-90%), positive predictive value (PPV) 93%, and accuracy 81%, based on correct lateralization. SPECT had a sensitivity of 77% (95% CI 72-82%), PPV 92% and accuracy 72%. SPECT/CT had a sensitivity of 75% (95% CI 70-80%), PPV of 94%, and accuracy 71%. There were 3 of 45 (7%) patients with negative sestamibi imaging that had an accurate SPECT and SPECT/CT. Of 312 patients (87%) with positive uptake on sestamibi (93% true positive, 7% false positive), concordant findings were present in 86% SPECT and 84% SPECT/CT. In cases where imaging modalities were discordant, but at least one method was true-positive, (123)I/(99)Tc-sestamibi was significantly better than both SPECT and SPECT/CT (p < 0.001). The inclusion of SPECT and SPECT/CT in 1°HPT imaging protocol increases patient cost up to 2.4-fold. (123)I/(99)Tc-sestamibi subtraction imaging is highly sensitive for preoperative localization in 1°HPT. SPECT and SPECT/CT are commonly concordant with (123)I/(99)Tc-sestamibi and rarely increase the sensitivity. Routine inclusion of multimodality imaging technique adds minimal clinical benefit but increases cost to patient in high-volume setting.
Tangible interactive system for document browsing and visualisation of multimedia data

NASA Astrophysics Data System (ADS)

Rytsar, Yuriy; Voloshynovskiy, Sviatoslav; Koval, Oleksiy; Deguillaume, Frederic; Topak, Emre; Startchik, Sergei; Pun, Thierry

2006-01-01

In this paper we introduce and develop a framework for document interactive navigation in multimodal databases. First, we analyze the main open issues of existing multimodal interfaces and then discuss two applications that include interaction with documents in several human environments, i.e., the so-called smart rooms. Second, we propose a system set-up dedicated to the efficient navigation in the printed documents. This set-up is based on the fusion of data from several modalities that include images and text. Both modalities can be used as cover data for hidden indexes using data-hiding technologies as well as source data for robust visual hashing. The particularities of the proposed robust visual hashing are described in the paper. Finally, we address two practical applications of smart rooms for tourism and education and demonstrate the advantages of the proposed solution.

Multimodal ophthalmic imaging using swept source spectrally encoded scanning laser ophthalmoscopy and optical coherence tomography

NASA Astrophysics Data System (ADS)

Malone, Joseph D.; El-Haddad, Mohamed T.; Tye, Logan A.; Majeau, Lucas; Godbout, Nicolas; Rollins, Andrew M.; Boudoux, Caroline; Tao, Yuankai K.

2016-03-01

Scanning laser ophthalmoscopy (SLO) and optical coherence tomography (OCT) benefit clinical diagnostic imaging in ophthalmology by enabling in vivo noninvasive en face and volumetric visualization of retinal structures, respectively. Spectrally encoding methods enable confocal imaging through fiber optics and reduces system complexity. Previous applications in ophthalmic imaging include spectrally encoded confocal scanning laser ophthalmoscopy (SECSLO) and a combined SECSLO-OCT system for image guidance, tracking, and registration. However, spectrally encoded imaging suffers from speckle noise because each spectrally encoded channel is effectively monochromatic. Here, we demonstrate in vivo human retinal imaging using a swept source spectrally encoded scanning laser ophthalmoscope and OCT (SSSESLO- OCT) at 1060 nm. SS-SESLO-OCT uses a shared 100 kHz Axsun swept source, shared scanner and imaging optics, and are detected simultaneously on a shared, dual channel high-speed digitizer. SESLO illumination and detection was performed using the single mode core and multimode inner cladding of a double clad fiber coupler, respectively, to preserve lateral resolution while improving collection efficiency and reducing speckle contrast at the expense of confocality. Concurrent en face SESLO and cross-sectional OCT images were acquired with 1376 x 500 pixels at 200 frames-per-second. Our system design is compact and uses a shared light source, imaging optics, and digitizer, which reduces overall system complexity and ensures inherent co-registration between SESLO and OCT FOVs. En face SESLO images acquired concurrent with OCT cross-sections enables lateral motion tracking and three-dimensional volume registration with broad applications in multivolume OCT averaging, image mosaicking, and intraoperative instrument tracking.
Rational Design of a Triple Reporter Gene for Multimodality Molecular Imaging

PubMed Central

Hsieh, Ya-Ju; Ke, Chien-Chih; Yeh, Skye Hsin-Hsien; Lin, Chien-Feng; Chen, Fu-Du; Lin, Kang-Ping; Chen, Ran-Chou; Liu, Ren-Shyan

2014-01-01

Multimodality imaging using noncytotoxic triple fusion (TF) reporter genes is an important application for cell-based tracking, drug screening, and therapy. The firefly luciferase (fl), monomeric red fluorescence protein (mrfp), and truncated herpes simplex virus type 1 thymidine kinase SR39 mutant (ttksr39) were fused together to create TF reporter gene constructs with different order. The enzymatic activities of TF protein in vitro and in vivo were determined by luciferase reporter assay, H-FEAU cellular uptake experiment, bioluminescence imaging, and micropositron emission tomography (microPET). The TF construct expressed in H1299 cells possesses luciferase activity and red fluorescence. The tTKSR39 activity is preserved in TF protein and mediates high levels of H-FEAU accumulation and significant cell death from ganciclovir (GCV) prodrug activation. In living animals, the luciferase and tTKSR39 activities of TF protein have also been successfully validated by multimodality imaging systems. The red fluorescence signal is relatively weak for in vivo imaging but may expedite FACS-based selection of TF reporter expressing cells. We have developed an optimized triple fusion reporter construct DsRedm-fl-ttksr39 for more effective and sensitive in vivo animal imaging using fluorescence, bioluminescence, and PET imaging modalities, which may facilitate different fields of biomedical research and applications. PMID:24809057
Multimodal optical imager for inner ear hearing loss diagnosis (Conference Presentation)

NASA Astrophysics Data System (ADS)

Park, Jesung; Maguluri, Gopi N.; Zhao, Youbo; Iftimia, Nicusor V.

2017-02-01

Sensorineural hearing loss (SNHL), which typically originates in the cochlea, is the most common otologic problem caused by aging and noise trauma. The cochlea, a delicate and complex biological mechanosensory transducer in the inner ear, has been extensively studied with the goal of improving diagnosis of SNHL. However, the difficulty associated with accessing the cochlea and resolving the microstructures that facilitate hearing within it in a minimally-invasive way has prevented us from being able to assess the pathology underlying SNHL in humans. To address this problem we investigated the ability of a multimodal optical system that combines optical coherence tomography (OCT) and single photon autofluorescence imaging (AFI) to enable visualization and evaluation of microstructures in the cochlea. A laboratory OCT/AFI imager was built to acquire high resolution OCT and single photon fluorescence images of the cochlea. The imager's ability to resolve diagnostically-relevant details was evaluated in ears extracted from normal and noise-exposed mice. A prototype endoscopic OCT/AFI imager was developed based on a double-clad fiber approach. Our measurements show that the multimodal OCT/AFI imager can be used to evaluate structural integrity in the mouse cochlea. Therefore, we believe that this technology is promising as a potential clinical evaluation tool, and as a technique for guiding otologic surgeries such as cochlear implant surgery.
Latest advances in molecular imaging instrumentation.

PubMed

Pichler, Bernd J; Wehrl, Hans F; Judenhofer, Martin S

2008-06-01

This review concentrates on the latest advances in molecular imaging technology, including PET, MRI, and optical imaging. In PET, significant improvements in tumor detection and image resolution have been achieved by introducing new scintillation materials, iterative image reconstruction, and correction methods. These advances enabled the first clinical scanners capable of time-of-flight detection and incorporating point-spread-function reconstruction to compensate for depth-of-interaction effects. In the field of MRI, the most important developments in recent years have mainly been MRI systems with higher field strengths and improved radiofrequency coil technology. Hyperpolarized imaging, functional MRI, and MR spectroscopy provide molecular information in vivo. A special focus of this review article is multimodality imaging and, in particular, the emerging field of combined PET/MRI.
Math Snacks: Using Animations and Games to Fill the Gaps in Mathematics

ERIC Educational Resources Information Center

Valdiz, Alfred; Trujillo, Karen; Wiburg, Karin

2013-01-01

Math Snacks animations and support materials were developed for use on the web and mobile technologies to teach ratio, proportion, scale factor, and number line concepts using a multi-modal approach. Included in Math Snacks are: Animations which promote the visualization of a concept image; written lessons which provide cognitive complexity for…
High resolution multimodal clinical ophthalmic imaging system

PubMed Central

Mujat, Mircea; Ferguson, R. Daniel; Patel, Ankit H.; Iftimia, Nicusor; Lue, Niyom; Hammer, Daniel X.

2010-01-01

We developed a multimodal adaptive optics (AO) retinal imager which is the first to combine high performance AO-corrected scanning laser ophthalmoscopy (SLO) and swept source Fourier domain optical coherence tomography (SSOCT) imaging modes in a single compact clinical prototype platform. Such systems are becoming ever more essential to vision research and are expected to prove their clinical value for diagnosis of retinal diseases, including glaucoma, diabetic retinopathy (DR), age-related macular degeneration (AMD), and retinitis pigmentosa. The SSOCT channel operates at a wavelength of 1 µm for increased penetration and visualization of the choriocapillaris and choroid, sites of major disease activity for DR and wet AMD. This AO system is designed for use in clinical populations; a dual deformable mirror (DM) configuration allows simultaneous low- and high-order aberration correction over a large range of refractions and ocular media quality. The system also includes a wide field (33 deg.) line scanning ophthalmoscope (LSO) for initial screening, target identification, and global orientation, an integrated retinal tracker (RT) to stabilize the SLO, OCT, and LSO imaging fields in the presence of lateral eye motion, and a high-resolution LCD-based fixation target for presentation of visual cues. The system was tested in human subjects without retinal disease for performance optimization and validation. We were able to resolve and quantify cone photoreceptors across the macula to within ~0.5 deg (~100-150 µm) of the fovea, image and delineate ten retinal layers, and penetrate to resolve features deep into the choroid. The prototype presented here is the first of a new class of powerful flexible imaging platforms that will provide clinicians and researchers with high-resolution, high performance adaptive optics imaging to help guide therapies, develop new drugs, and improve patient outcomes. PMID:20589021
Multimodality imaging of hepato-biliary disorders in pregnancy: a pictorial essay.

PubMed

Ong, Eugene M W; Drukteinis, Jennifer S; Peters, Hope E; Mortelé, Koenraad J

2009-09-01

Hepato-biliary disorders are rare complications of pregnancy, but they may be severe, with high fetal and maternal morbidity and mortality. Imaging is, therefore, essential in the rapid diagnosis of some of these conditions so that appropriate, life-saving treatment can be administered. This pictorial essay illustrates the multimodality imaging features of pregnancy-induced hepato-biliary disorders, such as acute fatty liver of pregnancy, preeclamsia and eclampsia, and HELLP syndrome, as well as those conditions which occur in pregnancy but are not unique to it, such as viral hepatitis, Budd-Chiari syndrome, focal hepatic lesions, biliary sludge, cholecystolithiasis, and choledocholithiasis.
Automated diagnosis of prostate cancer in multi-parametric MRI based on multimodal convolutional neural networks

NASA Astrophysics Data System (ADS)

Le, Minh Hung; Chen, Jingyu; Wang, Liang; Wang, Zhiwei; Liu, Wenyu; (Tim Cheng, Kwang-Ting; Yang, Xin

2017-08-01

Automated methods for prostate cancer (PCa) diagnosis in multi-parametric magnetic resonance imaging (MP-MRIs) are critical for alleviating requirements for interpretation of radiographs while helping to improve diagnostic accuracy (Artan et al 2010 IEEE Trans. Image Process. 19 2444-55, Litjens et al 2014 IEEE Trans. Med. Imaging 33 1083-92, Liu et al 2013 SPIE Medical Imaging (International Society for Optics and Photonics) p 86701G, Moradi et al 2012 J. Magn. Reson. Imaging 35 1403-13, Niaf et al 2014 IEEE Trans. Image Process. 23 979-91, Niaf et al 2012 Phys. Med. Biol. 57 3833, Peng et al 2013a SPIE Medical Imaging (International Society for Optics and Photonics) p 86701H, Peng et al 2013b Radiology 267 787-96, Wang et al 2014 BioMed. Res. Int. 2014). This paper presents an automated method based on multimodal convolutional neural networks (CNNs) for two PCa diagnostic tasks: (1) distinguishing between cancerous and noncancerous tissues and (2) distinguishing between clinically significant (CS) and indolent PCa. Specifically, our multimodal CNNs effectively fuse apparent diffusion coefficients (ADCs) and T2-weighted MP-MRI images (T2WIs). To effectively fuse ADCs and T2WIs we design a new similarity loss function to enforce consistent features being extracted from both ADCs and T2WIs. The similarity loss is combined with the conventional classification loss functions and integrated into the back-propagation procedure of CNN training. The similarity loss enables better fusion results than existing methods as the feature learning processes of both modalities are mutually guided, jointly facilitating CNN to ‘see’ the true visual patterns of PCa. The classification results of multimodal CNNs are further combined with the results based on handcrafted features using a support vector machine classifier. To achieve a satisfactory accuracy for clinical use, we comprehensively investigate three critical factors which could greatly affect the performance of our multimodal CNNs but have not been carefully studied previously. (1) Given limited training data, how can these be augmented in sufficient numbers and variety for fine-tuning deep CNN networks for PCa diagnosis? (2) How can multimodal MP-MRI information be effectively combined in CNNs? (3) What is the impact of different CNN architectures on the accuracy of PCa diagnosis? Experimental results on extensive clinical data from 364 patients with a total of 463 PCa lesions and 450 identified noncancerous image patches demonstrate that our system can achieve a sensitivity of 89.85% and a specificity of 95.83% for distinguishing cancer from noncancerous tissues and a sensitivity of 100% and a specificity of 76.92% for distinguishing indolent PCa from CS PCa. This result is significantly superior to the state-of-the-art method relying on handcrafted features.
Applications of magnetic resonance image segmentation in neurology

NASA Astrophysics Data System (ADS)

Heinonen, Tomi; Lahtinen, Antti J.; Dastidar, Prasun; Ryymin, Pertti; Laarne, Paeivi; Malmivuo, Jaakko; Laasonen, Erkki; Frey, Harry; Eskola, Hannu

1999-05-01

After the introduction of digital imagin devices in medicine computerized tissue recognition and classification have become important in research and clinical applications. Segmented data can be applied among numerous research fields including volumetric analysis of particular tissues and structures, construction of anatomical modes, 3D visualization, and multimodal visualization, hence making segmentation essential in modern image analysis. In this research project several PC based software were developed in order to segment medical images, to visualize raw and segmented images in 3D, and to produce EEG brain maps in which MR images and EEG signals were integrated. The software package was tested and validated in numerous clinical research projects in hospital environment.
The evolution of gadolinium based contrast agents: from single-modality to multi-modality

NASA Astrophysics Data System (ADS)

Zhang, Li; Liu, Ruiqing; Peng, Hui; Li, Penghui; Xu, Zushun; Whittaker, Andrew K.

2016-05-01

Gadolinium-based contrast agents are extensively used as magnetic resonance imaging (MRI) contrast agents due to their outstanding signal enhancement and ease of chemical modification. However, it is increasingly recognized that information obtained from single modal molecular imaging cannot satisfy the higher requirements on the efficiency and accuracy for clinical diagnosis and medical research, due to its limitation and default rooted in single molecular imaging technique itself. To compensate for the deficiencies of single function magnetic resonance imaging contrast agents, the combination of multi-modality imaging has turned to be the research hotpot in recent years. This review presents an overview on the recent developments of the functionalization of gadolinium-based contrast agents, and their application in biomedicine applications.
A Pretargeted Approach for the Multimodal PET/NIRF Imaging of Colorectal Cancer.

PubMed

Adumeau, Pierre; Carnazza, Kathryn E; Brand, Christian; Carlin, Sean D; Reiner, Thomas; Agnew, Brian J; Lewis, Jason S; Zeglis, Brian M

2016-01-01

The complementary nature of positron emission tomography (PET) and near-infrared fluorescence (NIRF) imaging makes the development of strategies for the multimodal PET/NIRF imaging of cancer a very enticing prospect. Indeed, in the context of colorectal cancer, a single multimodal PET/NIRF imaging agent could be used to stage the disease, identify candidates for surgical intervention, and facilitate the image-guided resection of the disease. While antibodies have proven to be highly effective vectors for the delivery of radioisotopes and fluorophores to malignant tissues, the use of radioimmunoconjugates labeled with long-lived nuclides such as 89 Zr poses two important clinical complications: high radiation doses to the patient and the need for significant lag time between imaging and surgery. In vivo pretargeting strategies that decouple the targeting vector from the radioactivity at the time of injection have the potential to circumvent these issues by facilitating the use of positron-emitting radioisotopes with far shorter half-lives. Here, we report the synthesis, characterization, and in vivo validation of a pretargeted strategy for the multimodal PET and NIRF imaging of colorectal carcinoma. This approach is based on the rapid and bioorthogonal ligation between a trans -cyclooctene- and fluorophore-bearing immunoconjugate of the huA33 antibody (huA33-Dye800-TCO) and a 64 Cu-labeled tetrazine radioligand ( 64 Cu-Tz-SarAr). In vivo imaging experiments in mice bearing A33 antigen-expressing SW1222 colorectal cancer xenografts clearly demonstrate that this approach enables the non-invasive visualization of tumors and the image-guided resection of malignant tissue, all at only a fraction of the radiation dose created by a directly labeled radioimmunoconjugate. Additional in vivo experiments in peritoneal and patient-derived xenograft models of colorectal carcinoma reinforce the efficacy of this methodology and underscore its potential as an innovative and useful clinical tool.
COSBID-M3: a platform for multimodal monitoring, data collection, and research in neurocritical care.

PubMed

Wilson, J Adam; Shutter, Lori A; Hartings, Jed A

2013-01-01

Neuromonitoring in patients with severe brain trauma and stroke is often limited to intracranial pressure (ICP); advanced neuroscience intensive care units may also monitor brain oxygenation (partial pressure of brain tissue oxygen, P(bt)O(2)), electroencephalogram (EEG), cerebral blood flow (CBF), or neurochemistry. For example, cortical spreading depolarizations (CSDs) recorded by electrocorticography (ECoG) are associated with delayed cerebral ischemia after subarachnoid hemorrhage and are an attractive target for novel therapeutic approaches. However, to better understand pathophysiologic relations and realize the potential of multimodal monitoring, a common platform for data collection and integration is needed. We have developed a multimodal system that integrates clinical, research, and imaging data into a single research and development (R&D) platform. Our system is adapted from the widely used BCI2000, a brain-computer interface tool which is written in the C++ language and supports over 20 data acquisition systems. It is optimized for real-time analysis of multimodal data using advanced time and frequency domain analyses and is extensible for research development using a combination of C++, MATLAB, and Python languages. Continuous streams of raw and processed data, including BP (blood pressure), ICP, PtiO2, CBF, ECoG, EEG, and patient video are stored in an open binary data format. Selected events identified in raw (e.g., ICP) or processed (e.g., CSD) measures are displayed graphically, can trigger alarms, or can be sent to researchers or clinicians via text message. For instance, algorithms for automated detection of CSD have been incorporated, and processed ECoG signals are projected onto three-dimensional (3D) brain models based on patient magnetic resonance imaging (MRI) and computed tomographic (CT) scans, allowing real-time correlation of pathoanatomy and cortical function. This platform will provide clinicians and researchers with an advanced tool to investigate pathophysiologic relationships and novel measures of cerebral status, as well as implement treatment algorithms based on such multimodal measures.
Chronic sacroiliac joint and pelvic girdle dysfunction in a 35-year-old nulliparous woman successfully managed with multimodal and multidisciplinary approach.

PubMed

Jonely, Holly; Brismée, Jean-Michel; Desai, Mehul J; Reoli, Rachel

2015-02-01

Sacroiliac joint pain and dysfunction affect 15-25% of patients reporting low back pain, including reports of spontaneous, idiopathic, traumatic, and non-traumatic onsets. The poor reliability and validity associated with diagnostic clinical and imaging techniques leads to challenges in diagnosing and managing sacroiliac joint dysfunction. A 35-year-old nulliparous female with a 14-year history of right sacroiliac joint dysfunction was managed using a multimodal and multidisciplinary approach when symptoms failed to resolve after 2 months of physical therapy. The plan of care included four prolotherapy injections, sacroiliac joint manipulation into nutation, pelvic girdle belting, and specific stabilization exercises. The patient completed 20 physical therapy sessions over a 12-month period. At 6 months, the patient's Oswestry Disability Questionnaire score was reduced from 34% to 14%. At 1-year follow-up, her score was 0%. The patient's rating of pain on a numeric rating scale decreased to an average of 4/10 at 6 months and 0/10 at 1-year follow-up. A multidisciplinary and multimodal approach for the management of chronic sacroiliac joint dysfunction appeared successful in a single-case design at 1-year follow-up.
Synthesis and radiolabeling of a somatostatin analog for multimodal imaging

NASA Astrophysics Data System (ADS)

Edwards, W. Barry; Liang, Kexian; Xu, Baogang; Anderson, Carolyn J.; Achilefu, Samuel

2006-02-01

A new multimodal imaging agent for imaging the somatostatin receptor has been synthesized and evaluated in vitro and in vivo. A somatostatin analog, conjugated to both 1,4,7,10-tetraazacyclododecane-1,4,7,10-tetraaceticacid (DOTA) and cypate (BS-296), was synthesized entirely on the solid phase (Fmoc) and purified by RP-HPLC. DOTA was added as a ligand for radiometals such as 64Cu or 177Lu for either radio-imaging or radiotherapy respectively. Cytate, a cypatesomatostatin analog conjugate, has previously demonstrated the ability to visualize somatostatin receptor rich tumor xenografts and natural organs by optical imaging techniques. BS-296 exhibited low nanomolar inhibitory capacity toward the binding of radiolabeled somatostatin analogs in cell membranes enriched in the somatostatin receptor, demonstrating the high affinity of this multimodal imaging peptide and indicating its potential as a molecular imaging agent. 64Cu, an isotope for diagnostic imaging and radiotherapy, was selected as the isotope for radiolabeling BS-296. BS-296 was radiolabeled with 64Cu in high specific activity (200 μCi/μg) in 90% radiochemical yield. Addition of 2,5-dihydroxybenzoic acid (gentisic acid) prevented radiolysis of the sample, allowing for study of the 64Cu -BS-296 the day following radiolabeling. Furthermore, inclusion of DMSO at a level of 20% was found not to interfere with radiolabeling yields and prevented the adherence of 64Cu -BS-296 to the walls of the reaction vessel.
A multimodal imaging approach enables in vivo assessment of antifungal treatment in a mouse model of invasive pulmonary aspergillosis.

PubMed

Poelmans, Jennifer; Himmelreich, Uwe; Vanherp, Liesbeth; Zhai, Luca; Hillen, Amy; Holvoet, Bryan; Belderbos, Sarah; Brock, Matthias; Maertens, Johan; Velde, Greetje Vande; Lagrou, Katrien

2018-05-14

Aspergillus fumigatus causes life-threatening lung infections in immunocompromised patients. Mouse models are extensively used in research to assess the in vivo efficacy of antifungals. In recent years, there has been an increasing interest in the use of non-invasive imaging techniques to evaluate experimental infections. However, single imaging modalities have limitations concerning the type of information they can provide. In this study, magnetic resonance imaging and bioluminescence imaging were combined to obtain longitudinal information on the extent of developing lesions and fungal load in a leucopenic mouse model of IPA. This multimodal imaging approach was used to assess changes occurring within lungs of infected mice receiving voriconazole treatment starting at different time points after infection. Results showed that IPA development depends on the inoculum size used to infect animals and that disease can be successfully prevented or treated by initiating intervention during early stages of infection. Furthermore, we demonstrated that reduction of the fungal load is not necessarily associated with the disappearance of lesions on anatomical lung images, especially when antifungal treatment coincides with immune recovery. In conclusion, multimodal imaging allows to investigate different aspects of disease progression or recovery by providing complementary information on dynamic processes, which are highly useful for assessing the efficacy of (novel) therapeutic compounds in a time- and labor-efficient manner. Copyright © 2018 American Society for Microbiology.
Multimodal Translation System Using Texture-Mapped Lip-Sync Images for Video Mail and Automatic Dubbing Applications

NASA Astrophysics Data System (ADS)

Morishima, Shigeo; Nakamura, Satoshi

2004-12-01

We introduce a multimodal English-to-Japanese and Japanese-to-English translation system that also translates the speaker's speech motion by synchronizing it to the translated speech. This system also introduces both a face synthesis technique that can generate any viseme lip shape and a face tracking technique that can estimate the original position and rotation of a speaker's face in an image sequence. To retain the speaker's facial expression, we substitute only the speech organ's image with the synthesized one, which is made by a 3D wire-frame model that is adaptable to any speaker. Our approach provides translated image synthesis with an extremely small database. The tracking motion of the face from a video image is performed by template matching. In this system, the translation and rotation of the face are detected by using a 3D personal face model whose texture is captured from a video frame. We also propose a method to customize the personal face model by using our GUI tool. By combining these techniques and the translated voice synthesis technique, an automatic multimodal translation can be achieved that is suitable for video mail or automatic dubbing systems into other languages.
Enhancing image classification models with multi-modal biomarkers

NASA Astrophysics Data System (ADS)

Caban, Jesus J.; Liao, David; Yao, Jianhua; Mollura, Daniel J.; Gochuico, Bernadette; Yoo, Terry

2011-03-01

Currently, most computer-aided diagnosis (CAD) systems rely on image analysis and statistical models to diagnose, quantify, and monitor the progression of a particular disease. In general, CAD systems have proven to be effective at providing quantitative measurements and assisting physicians during the decision-making process. As the need for more flexible and effective CADs continues to grow, questions about how to enhance their accuracy have surged. In this paper, we show how statistical image models can be augmented with multi-modal physiological values to create more robust, stable, and accurate CAD systems. In particular, this paper demonstrates how highly correlated blood and EKG features can be treated as biomarkers and used to enhance image classification models designed to automatically score subjects with pulmonary fibrosis. In our results, a 3-5% improvement was observed when comparing the accuracy of CADs that use multi-modal biomarkers with those that only used image features. Our results show that lab values such as Erythrocyte Sedimentation Rate and Fibrinogen, as well as EKG measurements such as QRS and I:40, are statistically significant and can provide valuable insights about the severity of the pulmonary fibrosis disease.
Large Margin Multi-Modal Multi-Task Feature Extraction for Image Classification.

PubMed

Yong Luo; Yonggang Wen; Dacheng Tao; Jie Gui; Chao Xu

2016-01-01

The features used in many image analysis-based applications are frequently of very high dimension. Feature extraction offers several advantages in high-dimensional cases, and many recent studies have used multi-task feature extraction approaches, which often outperform single-task feature extraction approaches. However, most of these methods are limited in that they only consider data represented by a single type of feature, even though features usually represent images from multiple modalities. We, therefore, propose a novel large margin multi-modal multi-task feature extraction (LM3FE) framework for handling multi-modal features for image classification. In particular, LM3FE simultaneously learns the feature extraction matrix for each modality and the modality combination coefficients. In this way, LM3FE not only handles correlated and noisy features, but also utilizes the complementarity of different modalities to further help reduce feature redundancy in each modality. The large margin principle employed also helps to extract strongly predictive features, so that they are more suitable for prediction (e.g., classification). An alternating algorithm is developed for problem optimization, and each subproblem can be efficiently solved. Experiments on two challenging real-world image data sets demonstrate the effectiveness and superiority of the proposed method.
Multimodal hyperspectral optical microscopy

DOE PAGES

Novikova, Irina V.; Smallwood, Chuck R.; Gong, Yu; ...

2017-09-02

We describe a unique and convenient approach to multimodal hyperspectral optical microscopy, herein achieved by coupling a portable and transferable hyperspectral imager to various optical microscopes. The experimental and data analysis schemes involved in recording spectrally and spatially resolved fluorescence, dark field, and optical absorption micrographs are illustrated through prototypical measurements targeting selected model systems. Namely, hyperspectral fluorescence micrographs of isolated fluorescent beads are employed to ensure spectral calibration of our detector and to gauge the attainable spatial resolution of our measurements; the recorded images are diffraction-limited. Moreover, spatially over-sampled absorption spectroscopy of a single lipid (18:1 Liss Rhod PE)more » layer reveals that optical densities on the order of 10-3 may be resolved by spatially averaging the recorded optical signatures. We also briefly illustrate two applications of our setup in the general areas of plasmonics and cell biology. Most notably, we deploy hyperspectral optical absorption microscopy to identify and image algal pigments within a single live Tisochrysis lutea cell. Overall, this work paves the way for multimodal multidimensional spectral imaging measurements spanning the realms of several scientific disciples.« less
Simultaneous fluorescence and quantitative phase microscopy with single-pixel detectors

NASA Astrophysics Data System (ADS)

Liu, Yang; Suo, Jinli; Zhang, Yuanlong; Dai, Qionghai

2018-02-01

Multimodal microscopy offers high flexibilities for biomedical observation and diagnosis. Conventional multimodal approaches either use multiple cameras or a single camera spatially multiplexing different modes. The former needs expertise demanding alignment and the latter suffers from limited spatial resolution. Here, we report an alignment-free full-resolution simultaneous fluorescence and quantitative phase imaging approach using single-pixel detectors. By combining reference-free interferometry with single-pixel detection, we encode the phase and fluorescence of the sample in two detection arms at the same time. Then we employ structured illumination and the correlated measurements between the sample and the illuminations for reconstruction. The recovered fluorescence and phase images are inherently aligned thanks to single-pixel detection. To validate the proposed method, we built a proof-of-concept setup for first imaging the phase of etched glass with the depth of a few hundred nanometers and then imaging the fluorescence and phase of the quantum dot drop. This method holds great potential for multispectral fluorescence microscopy with additional single-pixel detectors or a spectrometer. Besides, this cost-efficient multimodal system might find broad applications in biomedical science and neuroscience.

Multimodal hyperspectral optical microscopy

DOE Office of Scientific and Technical Information (OSTI.GOV)

Novikova, Irina V.; Smallwood, Chuck R.; Gong, Yu

We describe a unique and convenient approach to multimodal hyperspectral optical microscopy, herein achieved by coupling a portable and transferable hyperspectral imager to various optical microscopes. The experimental and data analysis schemes involved in recording spectrally and spatially resolved fluorescence, dark field, and optical absorption micrographs are illustrated through prototypical measurements targeting selected model systems. Namely, hyperspectral fluorescence micrographs of isolated fluorescent beads are employed to ensure spectral calibration of our detector and to gauge the attainable spatial resolution of our measurements; the recorded images are diffraction-limited. Moreover, spatially over-sampled absorption spectroscopy of a single lipid (18:1 Liss Rhod PE)more » layer reveals that optical densities on the order of 10-3 may be resolved by spatially averaging the recorded optical signatures. We also briefly illustrate two applications of our setup in the general areas of plasmonics and cell biology. Most notably, we deploy hyperspectral optical absorption microscopy to identify and image algal pigments within a single live Tisochrysis lutea cell. Overall, this work paves the way for multimodal multidimensional spectral imaging measurements spanning the realms of several scientific disciples.« less
Multimodal discrimination of immune cells using a combination of Raman spectroscopy and digital holographic microscopy

NASA Astrophysics Data System (ADS)

McReynolds, Naomi; Cooke, Fiona G. M.; Chen, Mingzhou; Powis, Simon J.; Dholakia, Kishan

2017-03-01

The ability to identify and characterise individual cells of the immune system under label-free conditions would be a significant advantage in biomedical and clinical studies where untouched and unmodified cells are required. We present a multi-modal system capable of simultaneously acquiring both single point Raman spectra and digital holographic images of single cells. We use this combined approach to identify and discriminate between immune cell populations CD4+ T cells, B cells and monocytes. We investigate several approaches to interpret the phase images including signal intensity histograms and texture analysis. Both modalities are independently able to discriminate between cell subsets and dual-modality may therefore be used a means for validation. We demonstrate here sensitivities achieved in the range of 86.8% to 100%, and specificities in the range of 85.4% to 100%. Additionally each modality provides information not available from the other providing both a molecular and a morphological signature of each cell.
MULTIMODAL IMAGING OF ACUTE EXUDATIVE POLYMORPHOUS VITELLIFORM MACULOPATHY WITH OPTICAL COHERENCE TOMOGRAPHY ANGIOGRAPHY AND ADAPTIVE OPTICS SCANNING LASER OPHTHALMOSCOPY.

PubMed

Skondra, Dimitra; Nesper, Peter L; Fawzi, Amani A

2017-05-16

To report a case of acute exudative polymorphous vitelliform maculopathy including the findings of optical coherence tomography angiography and adaptive optics scanning laser ophthalmoscopy. Findings on clinical examination, color fundus photography, spectral-domain optical coherence tomography, infrared reflectance, autofluorescence, optical coherence tomography angiography, and adaptive optics scanning laser ophthalmoscopy. A 54-year-old white man with no significant medical history and history of smoking presented with bilateral multiple serous and vitelliform detachments consistent with acute exudative polymorphous vitelliform maculopathy. Extensive infectious, inflammatory, and malignancy workup was negative. Spectral-domain optical coherence tomography showed thickened, hyperreflective ellipsoid zone, subretinal fluid, and focal as well as diffuse subretinal hyperreflective material corresponding to the vitelliform lesions. Optical coherence tomography angiography showed normal retinal and choroidal vasculature, whereas adaptive optics scanning laser ophthalmoscopy showed circular focal "target" lesions at the level of the photoreceptors in the area of foveal detachment. Multimodal imaging is valuable in evaluating patients with acute exudative polymorphous vitelliform maculopathy.
Multimodal inspection in power engineering and building industries: new challenges and solutions

NASA Astrophysics Data System (ADS)

Kujawińska, Małgorzata; Malesa, Marcin; Malowany, Krzysztof

2013-09-01

Recently the demand and number of applications of full-field, optical measurement methods based on noncoherent light sources increased significantly. They include traditional image processing, thermovision, digital image correlation (DIC) and structured light methods. However, there are still numerous challenges connected with implementation of these methods to in-situ, long-term monitoring in industrial, civil engineering and cultural heritage applications, multimodal measurements of a variety of object features or simply adopting instruments to work in hard environmental conditions. In this paper we focus on 3D DIC method and present its enhancements concerning software modifications (new visualization methods and a method for automatic merging of data distributed in time) and hardware improvements. The modified 3D DIC system combined with infrared camera system is applied in many interesting cases: measurements of boiler drum during annealing and of pipelines in heat power stations and monitoring of different building steel struts at construction site and validation of numerical models of large building structures constructed of graded metal plate arches.
Love that Book: Multimodal Response to Literature

ERIC Educational Resources Information Center

Dalton, Bridget; Grisham, Dana L.

2013-01-01

Composing with different modes--image, sound, video and the written word--to respond to and analyze literary and informational text helps students develop as readers and digital communicators. This article showcases five multimodal strategies for engaging children in rich literature-based learning using digital tools and Internet resources.
Content-independent embedding scheme for multi-modal medical image watermarking.

PubMed

Nyeem, Hussain; Boles, Wageeh; Boyd, Colin

2015-02-04

As the increasing adoption of information technology continues to offer better distant medical services, the distribution of, and remote access to digital medical images over public networks continues to grow significantly. Such use of medical images raises serious concerns for their continuous security protection, which digital watermarking has shown great potential to address. We present a content-independent embedding scheme for medical image watermarking. We observe that the perceptual content of medical images varies widely with their modalities. Recent medical image watermarking schemes are image-content dependent and thus they may suffer from inconsistent embedding capacity and visual artefacts. To attain the image content-independent embedding property, we generalise RONI (region of non-interest, to the medical professionals) selection process and use it for embedding by utilising RONI's least significant bit-planes. The proposed scheme thus avoids the need for RONI segmentation that incurs capacity and computational overheads. Our experimental results demonstrate that the proposed embedding scheme performs consistently over a dataset of 370 medical images including their 7 different modalities. Experimental results also verify how the state-of-the-art reversible schemes can have an inconsistent performance for different modalities of medical images. Our scheme has MSSIM (Mean Structural SIMilarity) larger than 0.999 with a deterministically adaptable embedding capacity. Our proposed image-content independent embedding scheme is modality-wise consistent, and maintains a good image quality of RONI while keeping all other pixels in the image untouched. Thus, with an appropriate watermarking framework (i.e., with the considerations of watermark generation, embedding and detection functions), our proposed scheme can be viable for the multi-modality medical image applications and distant medical services such as teleradiology and eHealth.
Multimodal neural correlates of cognitive control in the Human Connectome Project.

PubMed

Lerman-Sinkoff, Dov B; Sui, Jing; Rachakonda, Srinivas; Kandala, Sridhar; Calhoun, Vince D; Barch, Deanna M

2017-12-01

Cognitive control is a construct that refers to the set of functions that enable decision-making and task performance through the representation of task states, goals, and rules. The neural correlates of cognitive control have been studied in humans using a wide variety of neuroimaging modalities, including structural MRI, resting-state fMRI, and task-based fMRI. The results from each of these modalities independently have implicated the involvement of a number of brain regions in cognitive control, including dorsal prefrontal cortex, and frontal parietal and cingulo-opercular brain networks. However, it is not clear how the results from a single modality relate to results in other modalities. Recent developments in multimodal image analysis methods provide an avenue for answering such questions and could yield more integrated models of the neural correlates of cognitive control. In this study, we used multiset canonical correlation analysis with joint independent component analysis (mCCA + jICA) to identify multimodal patterns of variation related to cognitive control. We used two independent cohorts of participants from the Human Connectome Project, each of which had data from four imaging modalities. We replicated the findings from the first cohort in the second cohort using both independent and predictive analyses. The independent analyses identified a component in each cohort that was highly similar to the other and significantly correlated with cognitive control performance. The replication by prediction analyses identified two independent components that were significantly correlated with cognitive control performance in the first cohort and significantly predictive of performance in the second cohort. These components identified positive relationships across the modalities in neural regions related to both dynamic and stable aspects of task control, including regions in both the frontal-parietal and cingulo-opercular networks, as well as regions hypothesized to be modulated by cognitive control signaling, such as visual cortex. Taken together, these results illustrate the potential utility of multi-modal analyses in identifying the neural correlates of cognitive control across different indicators of brain structure and function. Copyright © 2017 Elsevier Inc. All rights reserved.
Calcium-based biomaterials for diagnosis, treatment, and theranostics.

PubMed

Qi, Chao; Lin, Jing; Fu, Lian-Hua; Huang, Peng

2018-01-22

Calcium-based (CaXs) biomaterials including calcium phosphates, calcium carbonates, calcium silicate and calcium fluoride have been widely utilized in the biomedical field owing to their excellent biocompatibility and biodegradability. In recent years, CaXs biomaterials have been strategically integrated with imaging contrast agents and therapeutic agents for various molecular imaging modalities including fluorescence imaging, magnetic resonance imaging, ultrasound imaging or multimodal imaging, as well as for various therapeutic approaches including chemotherapy, gene therapy, hyperthermia therapy, photodynamic therapy, radiation therapy, or combination therapy, even imaging-guided therapy. Compared with other inorganic biomaterials such as silica-, carbon-, and gold-based biomaterials, CaXs biomaterials can dissolve into nontoxic ions and participate in the normal metabolism of organisms. Thus, they offer safer clinical solutions for disease theranostics. This review focuses on the state-of-the-art progress in CaXs biomaterials, which covers from their categories, characteristics and preparation methods to their bioapplications including diagnosis, treatment, and theranostics. Moreover, the current trends and key problems as well as the future prospects and challenges of CaXs biomaterials are also discussed at the end.
Combined multimodal photoacoustic tomography, optical coherence tomography (OCT) and OCT based angiography system for in vivo imaging of multiple skin disorders in human(Conference Presentation)

NASA Astrophysics Data System (ADS)

Liu, Mengyang; Chen, Zhe; Sinz, Christoph; Rank, Elisabet; Zabihian, Behrooz; Zhang, Edward Z.; Beard, Paul C.; Kittler, Harald; Drexler, Wolfgang

2017-02-01

All optical photoacoustic tomography (PAT) using a planar Fabry-Perot interferometer polymer film sensor has been demonstrated for in vivo human palm imaging with an imaging penetration depth of 5 mm. The relatively larger vessels in the superficial plexus and the vessels in the dermal plexus are visible in PAT. However, due to both resolution and sensitivity limits, all optical PAT cannot reveal the smaller vessels such as capillary loops and venules. Melanin absorption also sometimes causes difficulties in PAT to resolve vessels. Optical coherence tomography (OCT) based angiography, on the other hand, has been proven suitable for microvasculature visualization in the first couple millimeters in human. In our work, we combine an all optical PAT system with an OCT system featuring a phase stable akinetic swept source. This multimodal PAT/OCT/OCT-angiography system provides us co-registered human skin vasculature information as well as the structural information of cutaneous. The scanning units of the sub-systems are assembled into one probe, which is then mounted onto a portable rack. The probe and rack design gives six degrees of freedom, allowing the multimodal optical imaging probe to access nearly all regions of human body. Utilizing this probe, we perform imaging on patients with various skin disorders as well as on healthy controls. Fused PAT/OCT-angiography volume shows the complete blood vessel network in human skin, which is further embedded in the morphology provided by OCT. A comparison between the results from the disordered regions and the normal regions demonstrates the clinical translational value of this multimodal optical imaging system in dermatology.
Multimodality medical image database for temporal lobe epilepsy

NASA Astrophysics Data System (ADS)

Siadat, Mohammad-Reza; Soltanian-Zadeh, Hamid; Fotouhi, Farshad A.; Elisevich, Kost

2003-05-01

This paper presents the development of a human brain multi-modality database for surgical candidacy determination in temporal lobe epilepsy. The focus of the paper is on content-based image management, navigation and retrieval. Several medical image-processing methods including our newly developed segmentation method are utilized for information extraction/correlation and indexing. The input data includes T1-, T2-Weighted and FLAIR MRI and ictal/interictal SPECT modalities with associated clinical data and EEG data analysis. The database can answer queries regarding issues such as the correlation between the attribute X of the entity Y and the outcome of a temporal lobe epilepsy surgery. The entity Y can be a brain anatomical structure such as the hippocampus. The attribute X can be either a functionality feature of the anatomical structure Y, calculated with SPECT modalities, such as signal average, or a volumetric/morphological feature of the entity Y such as volume or average curvature. The outcome of the surgery can be any surgery assessment such as non-verbal Wechsler memory quotient. A determination is made regarding surgical candidacy by analysis of both textual and image data. The current database system suggests a surgical determination for the cases with relatively small hippocampus and high signal intensity average on FLAIR images within the hippocampus. This indication matches the neurosurgeons expectations/observations. Moreover, as the database gets more populated with patient profiles and individual surgical outcomes, using data mining methods one may discover partially invisible correlations between the contents of different modalities of data and the outcome of the surgery.
Abdominal Organ Location, Morphology, and Rib Coverage for the 5(th), 50(th), and 95(th) Percentile Males and Females in the Supine and Seated Posture using Multi-Modality Imaging.

PubMed

Hayes, Ashley R; Gayzik, F Scott; Moreno, Daniel P; Martin, R Shayn; Stitzel, Joel D

The purpose of this study was to use data from a multi-modality image set of males and females representing the 5(th), 50(th), and 95(th) percentile (n=6) to examine abdominal organ location, morphology, and rib coverage variations between supine and seated postures. Medical images were acquired from volunteers in three image modalities including Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and upright MRI (uMRI). A manual and semi-automated segmentation method was used to acquire data and a registration technique was employed to conduct a comparative analysis between abdominal organs (liver, spleen, and kidneys) in both postures. Location of abdominal organs, defined by center of gravity movement, varied between postures and was found to be significant (p=0.002 to p=0.04) in multiple directions for each organ. In addition, morphology changes, including compression and expansion, were seen in each organ as a result of postural changes. Rib coverage, defined as the projected area of the ribs onto the abdominal organs, was measured in frontal, lateral, and posterior projections, and also varied between postures. A significant change in rib coverage between postures was measured for the spleen and right kidney (p=0.03 and p=0.02). The results indicate that posture affects the location, morphology and rib coverage area of abdominal organs and these implications should be noted in computational modeling efforts focused on a seated posture.
Predicting individualized clinical measures by a generalized prediction framework and multimodal fusion of MRI data

PubMed Central

Meng, Xing; Jiang, Rongtao; Lin, Dongdong; Bustillo, Juan; Jones, Thomas; Chen, Jiayu; Yu, Qingbao; Du, Yuhui; Zhang, Yu; Jiang, Tianzi; Sui, Jing; Calhoun, Vince D.

2016-01-01

Neuroimaging techniques have greatly enhanced the understanding of neurodiversity (human brain variation across individuals) in both health and disease. The ultimate goal of using brain imaging biomarkers is to perform individualized predictions. Here we proposed a generalized framework that can predict explicit values of the targeted measures by taking advantage of joint information from multiple modalities. This framework also enables whole brain voxel-wise searching by combining multivariate techniques such as ReliefF, clustering, correlation-based feature selection and multiple regression models, which is more flexible and can achieve better prediction performance than alternative atlas-based methods. For 50 healthy controls and 47 schizophrenia patients, three kinds of features derived from resting-state fMRI (fALFF), sMRI (gray matter) and DTI (fractional anisotropy) were extracted and fed into a regression model, achieving high prediction for both cognitive scores (MCCB composite r = 0.7033, MCCB social cognition r = 0.7084) and symptomatic scores (positive and negative syndrome scale [PANSS] positive r = 0.7785, PANSS negative r = 0.7804). Moreover, the brain areas likely responsible for cognitive deficits of schizophrenia, including middle temporal gyrus, dorsolateral prefrontal cortex, striatum, cuneus and cerebellum, were located with different weights, as well as regions predicting PANSS symptoms, including thalamus, striatum and inferior parietal lobule, pinpointing the potential neuromarkers. Finally, compared to a single modality, multimodal combination achieves higher prediction accuracy and enables individualized prediction on multiple clinical measures. There is more work to be done, but the current results highlight the potential utility of multimodal brain imaging biomarkers to eventually inform clinical decision-making. PMID:27177764
Multimodal full-field optical coherence tomography on biological tissue: toward all optical digital pathology

NASA Astrophysics Data System (ADS)

Harms, F.; Dalimier, E.; Vermeulen, P.; Fragola, A.; Boccara, A. C.

2012-03-01

Optical Coherence Tomography (OCT) is an efficient technique for in-depth optical biopsy of biological tissues, relying on interferometric selection of ballistic photons. Full-Field Optical Coherence Tomography (FF-OCT) is an alternative approach to Fourier-domain OCT (spectral or swept-source), allowing parallel acquisition of en-face optical sections. Using medium numerical aperture objective, it is possible to reach an isotropic resolution of about 1x1x1 ìm. After stitching a grid of acquired images, FF-OCT gives access to the architecture of the tissue, for both macroscopic and microscopic structures, in a non-invasive process, which makes the technique particularly suitable for applications in pathology. Here we report a multimodal approach to FF-OCT, combining two Full-Field techniques for collecting a backscattered endogeneous OCT image and a fluorescence exogeneous image in parallel. Considering pathological diagnosis of cancer, visualization of cell nuclei is of paramount importance. OCT images, even for the highest resolution, usually fail to identify individual nuclei due to the nature of the optical contrast used. We have built a multimodal optical microscope based on the combination of FF-OCT and Structured Illumination Microscopy (SIM). We used x30 immersion objectives, with a numerical aperture of 1.05, allowing for sub-micron transverse resolution. Fluorescent staining of nuclei was obtained using specific fluorescent dyes such as acridine orange. We present multimodal images of healthy and pathological skin tissue at various scales. This instrumental development paves the way for improvements of standard pathology procedures, as a faster, non sacrificial, operator independent digital optical method compared to frozen sections.
Voxel-based automated detection of focal cortical dysplasia lesions using diffusion tensor imaging and T2-weighted MRI data.

PubMed

Wang, Yanming; Zhou, Yawen; Wang, Huijuan; Cui, Jin; Nguchu, Benedictor Alexander; Zhang, Xufei; Qiu, Bensheng; Wang, Xiaoxiao; Zhu, Mingwang

2018-05-21

The aim of this study was to automatically detect focal cortical dysplasia (FCD) lesions in patients with extratemporal lobe epilepsy by relying on diffusion tensor imaging (DTI) and T2-weighted magnetic resonance imaging (MRI) data. We implemented an automated classifier using voxel-based multimodal features to identify gray and white matter abnormalities of FCD in patient cohorts. In addition to the commonly used T2-weighted image intensity feature, DTI-based features were also utilized. A Gaussian processes for machine learning (GPML) classifier was tested on 12 patients with FCD (8 with histologically confirmed FCD) scanned at 1.5 T and cross-validated using a leave-one-out strategy. Moreover, we compared the multimodal GPML paradigm's performance with that of single modal GPML and classical support vector machine (SVM). Our results demonstrated that the GPML performance on DTI-based features (mean AUC = 0.63) matches with the GPML performance on T2-weighted image intensity feature (mean AUC = 0.64). More promisingly, GPML yielded significantly improved performance (mean AUC = 0.76) when applying DTI-based features to multimodal paradigm. Based on the results, it can also be clearly stated that the proposed GPML strategy performed better and is robust to unbalanced dataset contrary to SVM that performed poorly (AUC = 0.69). Therefore, the GPML paradigm using multimodal MRI data containing DTI modality has promising result towards detection of the FCD lesions and provides an effective direction for future researches. Copyright © 2018 Elsevier Inc. All rights reserved.
Synthesis, Characterization, and Application of Core–Shell Co0.16Fe2.84O4@NaYF4(Yb, Er) and Fe3O4@NaYF4(Yb, Tm) Nanoparticle as Trimodal (MRI, PET/SPECT, and Optical) Imaging Agents

PubMed Central

2015-01-01

Multimodal nanoparticulate materials are described, offering magnetic, radionuclide, and fluorescent imaging capabilities to exploit the complementary advantages of magnetic resonance imaging (MRI), positron emission tomography/single-photon emission commuted tomography (PET/SPECT), and optical imaging. They comprise Fe3O4@NaYF4 core/shell nanoparticles (NPs) with different cation dopants in the shell or core, including Co0.16Fe2.84O4@NaYF4(Yb, Er) and Fe3O4@NaYF4(Yb, Tm). These NPs are stabilized by bisphosphonate polyethylene glycol conjugates (BP-PEG), and then show a high transverse relaxivity (r2) up to 326 mM–1 s–1 at 3T, a high affinity to [18F]-fluoride or radiometal-bisphosphonate conjugates (e.g., 64Cu and 99mTc), and fluorescent emissions from 500 to 800 nm under excitation at 980 nm. The biodistribution of intravenously administered particles determined by PET/MR imaging suggests that negatively charged Co0.16Fe2.84O4@NaYF4(Yb, Er)-BP-PEG (10K) NPs cleared from the blood pool more slowly than positively charged NPs Fe3O4@NaYF4(Yb, Tm)-BP-PEG (2K). Preliminary results in sentinel lymph node imaging in mice indicate the advantages of multimodal imaging. PMID:26172432
Development of ClearPEM-Sonic, a multimodal mammography system for PET and Ultrasound

NASA Astrophysics Data System (ADS)

Cucciati, G.; Auffray, E.; Bugalho, R.; Cao, L.; Di Vara, N.; Farina, F.; Felix, N.; Frisch, B.; Ghezzi, A.; Juhan, V.; Jun, D.; Lasaygues, P.; Lecoq, P.; Mensah, S.; Mundler, O.; Neves, J.; Paganoni, M.; Peter, J.; Pizzichemi, M.; Siles, P.; Silva, J. C.; Silva, R.; Tavernier, S.; Tessonnier, L.; Varela, J.

2014-03-01

ClearPEM-Sonic is an innovative imaging device specifically developed for breast cancer. The possibility to work in PEM-Ultrasound multimodality allows to obtain metabolic and morphological information increasing the specificity of the exam. The ClearPEM detector is developed to maximize the sensitivity and the spatial resolution as compared to Whole-Body PET scanners. It is coupled with a 3D ultrasound system, the SuperSonic Imagine Aixplorer that improves the specificity of the exam by providing a tissue elasticity map. This work describes the ClearPEM-Sonic project focusing on the technological developments it has required, the technical merits (and limits) and the first multimodal images acquired on a dedicated phantom. It finally presents selected clinical case studies that confirm the value of PEM information.
Focusing and imaging with increased numerical apertures through multimode fibers with micro-fabricated optics.

PubMed

Bianchi, S; Rajamanickam, V P; Ferrara, L; Di Fabrizio, E; Liberale, C; Di Leonardo, R

2013-12-01

The use of individual multimode optical fibers in endoscopy applications has the potential to provide highly miniaturized and noninvasive probes for microscopy and optical micromanipulation. A few different strategies have been proposed recently, but they all suffer from intrinsically low resolution related to the low numerical aperture of multimode fibers. Here, we show that two-photon polymerization allows for direct fabrication of micro-optics components on the fiber end, resulting in an increase of the numerical aperture to a value that is close to 1. Coupling light into the fiber through a spatial light modulator, we were able to optically scan a submicrometer spot (300 nm FWHM) over an extended region, facing the opposite fiber end. Fluorescence imaging with improved resolution is also demonstrated.
Identification of Amnestic Mild Cognitive Impairment Using Multi-Modal Brain Features: A Combined Structural MRI and Diffusion Tensor Imaging Study.

PubMed

Xie, Yunyan; Cui, Zaixu; Zhang, Zhongmin; Sun, Yu; Sheng, Can; Li, Kuncheng; Gong, Gaolang; Han, Ying; Jia, Jianping

2015-01-01

Identifying amnestic mild cognitive impairment (aMCI) is of great clinical importance because aMCI is a putative prodromal stage of Alzheimer's disease. The present study aimed to explore the feasibility of accurately identifying aMCI with a magnetic resonance imaging (MRI) biomarker. We integrated measures of both gray matter (GM) abnormalities derived from structural MRI and white matter (WM) alterations acquired from diffusion tensor imaging at the voxel level across the entire brain. In particular, multi-modal brain features, including GM volume, WM fractional anisotropy, and mean diffusivity, were extracted from a relatively large sample of 64 Han Chinese aMCI patients and 64 matched controls. Then, support vector machine classifiers for GM volume, FA, and MD were fused to distinguish the aMCI patients from the controls. The fused classifier was evaluated with the leave-one-out and the 10-fold cross-validations, and the classifier had an accuracy of 83.59% and an area under the curve of 0.862. The most discriminative regions of GM were mainly located in the medial temporal lobe, temporal lobe, precuneus, cingulate gyrus, parietal lobe, and frontal lobe, whereas the most discriminative regions of WM were mainly located in the corpus callosum, cingulum, corona radiata, frontal lobe, and parietal lobe. Our findings suggest that aMCI is characterized by a distributed pattern of GM abnormalities and WM alterations that represent discriminative power and reflect relevant pathological changes in the brain, and these changes further highlight the advantage of multi-modal feature integration for identifying aMCI.
Analysis of multimode fiber bundles for endoscopic spectral-domain optical coherence tomography

PubMed Central

Risi, Matthew D.; Makhlouf, Houssine; Rouse, Andrew R.; Gmitro, Arthur F.

2016-01-01

A theoretical analysis of the use of a fiber bundle in spectral-domain optical coherence tomography (OCT) systems is presented. The fiber bundle enables a flexible endoscopic design and provides fast, parallelized acquisition of the OCT data. However, the multimode characteristic of the fibers in the fiber bundle affects the depth sensitivity of the imaging system. A description of light interference in a multimode fiber is presented along with numerical simulations and experimental studies to illustrate the theoretical analysis. PMID:25967012
Multimodal segmentation of optic disc and cup from stereo fundus and SD-OCT images

NASA Astrophysics Data System (ADS)

Miri, Mohammad Saleh; Lee, Kyungmoo; Niemeijer, Meindert; Abràmoff, Michael D.; Kwon, Young H.; Garvin, Mona K.

2013-03-01

Glaucoma is one of the major causes of blindness worldwide. One important structural parameter for the diagnosis and management of glaucoma is the cup-to-disc ratio (CDR), which tends to become larger as glaucoma progresses. While approaches exist for segmenting the optic disc and cup within fundus photographs, and more recently, within spectral-domain optical coherence tomography (SD-OCT) volumes, no approaches have been reported for the simultaneous segmentation of these structures within both modalities combined. In this work, a multimodal pixel-classification approach for the segmentation of the optic disc and cup within fundus photographs and SD-OCT volumes is presented. In particular, after segmentation of other important structures (such as the retinal layers and retinal blood vessels) and fundus-to-SD-OCT image registration, features are extracted from both modalities and a k-nearest-neighbor classification approach is used to classify each pixel as cup, rim, or background. The approach is evaluated on 70 multimodal image pairs from 35 subjects in a leave-10%-out fashion (by subject). A significant improvement in classification accuracy is obtained using the multimodal approach over that obtained from the corresponding unimodal approach (97.8% versus 95.2%; p < 0:05; paired t-test).

A Deep and Autoregressive Approach for Topic Modeling of Multimodal Data.

PubMed

Zheng, Yin; Zhang, Yu-Jin; Larochelle, Hugo

2016-06-01

Topic modeling based on latent Dirichlet allocation (LDA) has been a framework of choice to deal with multimodal data, such as in image annotation tasks. Another popular approach to model the multimodal data is through deep neural networks, such as the deep Boltzmann machine (DBM). Recently, a new type of topic model called the Document Neural Autoregressive Distribution Estimator (DocNADE) was proposed and demonstrated state-of-the-art performance for text document modeling. In this work, we show how to successfully apply and extend this model to multimodal data, such as simultaneous image classification and annotation. First, we propose SupDocNADE, a supervised extension of DocNADE, that increases the discriminative power of the learned hidden topic features and show how to employ it to learn a joint representation from image visual words, annotation words and class label information. We test our model on the LabelMe and UIUC-Sports data sets and show that it compares favorably to other topic models. Second, we propose a deep extension of our model and provide an efficient way of training the deep model. Experimental results show that our deep model outperforms its shallow version and reaches state-of-the-art performance on the Multimedia Information Retrieval (MIR) Flickr data set.
Drug-related webpages classification based on multi-modal local decision fusion

NASA Astrophysics Data System (ADS)

Hu, Ruiguang; Su, Xiaojing; Liu, Yanxin

2018-03-01

In this paper, multi-modal local decision fusion is used for drug-related webpages classification. First, meaningful text are extracted through HTML parsing, and effective images are chosen by the FOCARSS algorithm. Second, six SVM classifiers are trained for six kinds of drug-taking instruments, which are represented by PHOG. One SVM classifier is trained for the cannabis, which is represented by the mid-feature of BOW model. For each instance in a webpage, seven SVMs give seven labels for its image, and other seven labels are given by searching the names of drug-taking instruments and cannabis in its related text. Concatenating seven labels of image and seven labels of text, the representation of those instances in webpages are generated. Last, Multi-Instance Learning is used to classify those drugrelated webpages. Experimental results demonstrate that the classification accuracy of multi-instance learning with multi-modal local decision fusion is much higher than those of single-modal classification.
Multimodal Deep Autoencoder for Human Pose Recovery.

PubMed

Hong, Chaoqun; Yu, Jun; Wan, Jian; Tao, Dacheng; Wang, Meng

2015-12-01

Video-based human pose recovery is usually conducted by retrieving relevant poses using image features. In the retrieving process, the mapping between 2D images and 3D poses is assumed to be linear in most of the traditional methods. However, their relationships are inherently non-linear, which limits recovery performance of these methods. In this paper, we propose a novel pose recovery method using non-linear mapping with multi-layered deep neural network. It is based on feature extraction with multimodal fusion and back-propagation deep learning. In multimodal fusion, we construct hypergraph Laplacian with low-rank representation. In this way, we obtain a unified feature description by standard eigen-decomposition of the hypergraph Laplacian matrix. In back-propagation deep learning, we learn a non-linear mapping from 2D images to 3D poses with parameter fine-tuning. The experimental results on three data sets show that the recovery error has been reduced by 20%-25%, which demonstrates the effectiveness of the proposed method.
2017 Multimodality Appropriate Use Criteria for Noninvasive Cardiac Imaging: Expert Consensus of the Asian Society of Cardiovascular Imaging

PubMed Central

Beck, Kyongmin Sarah; Kim, Jeong A; Choe, Yeon Hyeon; Hian, Sim Kui; Hoe, John; Hong, Yoo Jin; Kim, Sung Mok; Kim, Tae Hoon; Kim, Young Jin; Kim, Yun Hyeon; Kuribayashi, Sachio; Lee, Jongmin; Leong, Lilian; Lim, Tae-Hwan; Lu, Bin; Park, Jae Hyung; Sakuma, Hajime; Yang, Dong Hyun; Yaw, Tan Swee; Wan, Yung-Liang; Zhang, Zhaoqi; Zhao, Shihua

2017-01-01

In 2010, the Asian Society of Cardiovascular Imaging (ASCI) provided recommendations for cardiac CT and MRI, and this document reflects an update of the 2010 ASCI appropriate use criteria (AUC). In 2016, the ASCI formed a new working group for revision of AUC for noninvasive cardiac imaging. A major change that we made in this document is the rating of various noninvasive tests (exercise electrocardiogram, echocardiography, positron emission tomography, single-photon emission computed tomography, radionuclide imaging, cardiac magnetic resonance, and cardiac computed tomography/angiography), compared side by side for their applications in various clinical scenarios. Ninety-five clinical scenarios were developed from eight selected pre-existing guidelines and classified into four sections as follows: 1) detection of coronary artery disease, symptomatic or asymptomatic; 2) cardiac evaluation in various clinical scenarios; 3) use of imaging modality according to prior testing; and 4) evaluation of cardiac structure and function. The clinical scenarios were scored by a separate rating committee on a scale of 1–9 to designate appropriate use, uncertain use, or inappropriate use according to a modified Delphi method. Overall, the AUC ratings for CT were higher than those of previous guidelines. These new AUC provide guidance for clinicians choosing among available testing modalities for various cardiac diseases and are also unique, given that most previous AUC for noninvasive imaging include only one imaging technique. As cardiac imaging is multimodal in nature, we believe that these AUC will be more useful for clinical decision making. PMID:29089819
Multi-modal diffuse optical techniques for breast cancer neoadjuvant chemotherapy monitoring (Conference Presentation)

NASA Astrophysics Data System (ADS)

Cochran, Jeffrey M.; Busch, David R.; Ban, Han Y.; Kavuri, Venkaiah C.; Schweiger, Martin J.; Arridge, Simon R.; Yodh, Arjun G.

2017-02-01

We present high spatial density, multi-modal, parallel-plate Diffuse Optical Tomography (DOT) imaging systems for the purpose of breast tumor detection. One hybrid instrument provides time domain (TD) and continuous wave (CW) DOT at 64 source fiber positions. The TD diffuse optical spectroscopy with PMT- detection produces low-resolution images of absolute tissue scattering and absorption while the spatially dense array of CCD-coupled detector fibers (108 detectors) provides higher-resolution CW images of relative tissue optical properties. Reconstruction of the tissue optical properties, along with total hemoglobin concentration and tissue oxygen saturation, is performed using the TOAST software suite. Comparison of the spatially-dense DOT images and MR images allows for a robust validation of DOT against an accepted clinical modality. Additionally, the structural information from co-registered MR images is used as a spatial prior to improve the quality of the functional optical images and provide more accurate quantification of the optical and hemodynamic properties of tumors. We also present an optical-only imaging system that provides frequency domain (FD) DOT at 209 source positions with full CCD detection and incorporates optical fringe projection profilometry to determine the breast boundary. This profilometry serves as a spatial constraint, improving the quality of the DOT reconstructions while retaining the benefits of an optical-only device. We present initial images from both human subjects and phantoms to display the utility of high spatial density data and multi-modal information in DOT reconstruction with the two systems.
Integration of Sparse Multi-modality Representation and Anatomical Constraint for Isointense Infant Brain MR Image Segmentation

PubMed Central

Wang, Li; Shi, Feng; Gao, Yaozong; Li, Gang; Gilmore, John H.; Lin, Weili; Shen, Dinggang

2014-01-01

Segmentation of infant brain MR images is challenging due to poor spatial resolution, severe partial volume effect, and the ongoing maturation and myelination process. During the first year of life, the brain image contrast between white and gray matters undergoes dramatic changes. In particular, the image contrast inverses around 6–8 months of age, where the white and gray matter tissues are isointense in T1 and T2 weighted images and hence exhibit the extremely low tissue contrast, posing significant challenges for automated segmentation. In this paper, we propose a general framework that adopts sparse representation to fuse the multi-modality image information and further incorporate the anatomical constraints for brain tissue segmentation. Specifically, we first derive an initial segmentation from a library of aligned images with ground-truth segmentations by using sparse representation in a patch-based fashion for the multi-modality T1, T2 and FA images. The segmentation result is further iteratively refined by integration of the anatomical constraint. The proposed method was evaluated on 22 infant brain MR images acquired at around 6 months of age by using a leave-one-out cross-validation, as well as other 10 unseen testing subjects. Our method achieved a high accuracy for the Dice ratios that measure the volume overlap between automated and manual segmentations, i.e., 0.889±0.008 for white matter and 0.870±0.006 for gray matter. PMID:24291615
Fluorescence labeled microbubbles for multimodal imaging.

PubMed

Barrefelt, Åsa; Zhao, Ying; Larsson, Malin K; Egri, Gabriella; Kuiper, Raoul V; Hamm, Jörg; Saghafian, Maryam; Caidahl, Kenneth; Brismar, Torkel B; Aspelin, Peter; Heuchel, Rainer; Muhammed, Mamoun; Dähne, Lars; Hassan, Moustapha

2015-08-28

Air-filled polyvinyl alcohol microbubbles (PVA-MBs) were recently introduced as a contrast agent for ultrasound imaging. In the present study, we explore the possibility of extending their application in multimodal imaging by labeling them with a near infrared (NIR) fluorophore, VivoTag-680. PVA-MBs were injected intravenously into FVB/N female mice and their dynamic biodistribution over 24 h was determined by 3D-fluorescence imaging co-registered with 3D-μCT imaging, to verify the anatomic location. To further confirm the biodistribution results from in vivo imaging, organs were removed and examined histologically using bright field and fluorescence microscopy. Fluorescence imaging detected PVA-MB accumulation in the lungs within the first 30 min post-injection. Redistribution to a low extent was observed in liver and kidneys at 4 h, and to a high extent mainly in the liver and spleen at 24 h. Histology confirmed PVA-MB localization in lung capillaries and macrophages. In the liver, they were associated with Kupffer cells; in the spleen, they were located mostly within the marginal-zone. Occasional MBs were observed in the kidney glomeruli and interstitium. The potential application of PVA-MBs as a contrast agent was also studied using ultrasound (US) imaging in subcutaneous and orthotopic pancreatic cancer mouse models, to visualize blood flow within the tumor mass. In conclusion, this study showed that PVA-MBs are useful as a contrast agent for multimodal imaging. Copyright © 2015 Elsevier Inc. All rights reserved.
Echocardiography in the Era of Multimodality Cardiovascular Imaging

PubMed Central

Shah, Benoy Nalin

2013-01-01

Echocardiography remains the most frequently performed cardiac imaging investigation and is an invaluable tool for detailed and accurate evaluation of cardiac structure and function. Echocardiography, nuclear cardiology, cardiac magnetic resonance imaging, and cardiovascular-computed tomography comprise the subspeciality of cardiovascular imaging, and these techniques are often used together for a multimodality, comprehensive assessment of a number of cardiac diseases. This paper provides the general cardiologist and physician with an overview of state-of-the-art modern echocardiography, summarising established indications as well as highlighting advances in stress echocardiography, three-dimensional echocardiography, deformation imaging, and contrast echocardiography. Strengths and limitations of echocardiography are discussed as well as the growing role of real-time three-dimensional echocardiography in the guidance of structural heart interventions in the cardiac catheter laboratory. PMID:23878804
Magnetic Nanoparticles for Multi-Imaging and Drug Delivery

PubMed Central

Lee, Jae-Hyun; Kim, Ji-wook; Cheon, Jinwoo

2013-01-01

Various bio-medical applications of magnetic nanoparticles have been explored during the past few decades. As tools that hold great potential for advancing biological sciences, magnetic nanoparticles have been used as platform materials for enhanced magnetic resonance imaging (MRI) agents, biological separation and magnetic drug delivery systems, and magnetic hyperthermia treatment. Furthermore, approaches that integrate various imaging and bioactive moieties have been used in the design of multi-modality systems, which possess synergistically enhanced properties such as better imaging resolution and sensitivity, molecular recognition capabilities, stimulus responsive drug delivery with on-demand control, and spatio-temporally controlled cell signal activation. Below, recent studies that focus on the design and synthesis of multi-mode magnetic nanoparticles will be briefly reviewed and their potential applications in the imaging and therapy areas will be also discussed. PMID:23579479
Rat brain imaging using full field optical coherence microscopy with short multimode fiber probe

NASA Astrophysics Data System (ADS)

Sato, Manabu; Saito, Daisuke; Kurotani, Reiko; Abe, Hiroyuki; Kawauchi, Satoko; Sato, Shunichi; Nishidate, Izumi

2017-02-01

We demonstrated FF OCM(full field optical coherence microscopy) using an ultrathin forward-imaging SMMF (short multimode fiber) probe of 50 μm core diameter, 125 μm diameter, and 7.4 mm length, which is a typical graded-index multimode fiber for optical communications. The axial resolution was measured to be 2.20 μm, which is close to the calculated axial resolution of 2.06 μm. The lateral resolution was evaluated to be 4.38 μm using a test pattern. Assuming that the FWHM of the contrast is the DOF (depth of focus), the DOF of the signal is obtained at 36 μm and that of the OCM is 66 μm. The contrast of the OCT images was 6.1 times higher than that of the signal images due to the coherence gate. After an euthanasia the rat brain was resected and cut at 2.6mm tail from Bregma. Contacting SMMF to the primary somatosensory cortex and the agranular insular cortex of ex vivo brain, OCM images of the brain were measured 100 times with 2μm step. 3D OCM images of the brain were measured, and internal structure information was obtained. The feasibility of an SMMF as an ultrathin forward-imaging probe in full-field OCM has been demonstrated.
Multimodality image display station

NASA Astrophysics Data System (ADS)

Myers, H. Joseph

1990-07-01

The Multi-modality Image Display Station (MIDS) is designed for the use of physicians outside of the radiology department. Connected to a local area network or a host computer, it provides speedy access to digitized radiology images and written diagnostics needed by attending and consulting physicians near the patient bedside. Emphasis has been placed on low cost, high performance and ease of use. The work is being done as a joint study with the University of Texas Southwestern Medical Center at Dallas, and as part of a joint development effort with the Mayo Clinic. MIDS is a prototype, and should not be assumed to be an IBM product.
Intrasubject multimodal groupwise registration with the conditional template entropy.

PubMed

Polfliet, Mathias; Klein, Stefan; Huizinga, Wyke; Paulides, Margarethus M; Niessen, Wiro J; Vandemeulebroucke, Jef

2018-05-01

Image registration is an important task in medical image analysis. Whereas most methods are designed for the registration of two images (pairwise registration), there is an increasing interest in simultaneously aligning more than two images using groupwise registration. Multimodal registration in a groupwise setting remains difficult, due to the lack of generally applicable similarity metrics. In this work, a novel similarity metric for such groupwise registration problems is proposed. The metric calculates the sum of the conditional entropy between each image in the group and a representative template image constructed iteratively using principal component analysis. The proposed metric is validated in extensive experiments on synthetic and intrasubject clinical image data. These experiments showed equivalent or improved registration accuracy compared to other state-of-the-art (dis)similarity metrics and improved transformation consistency compared to pairwise mutual information. Copyright © 2018 The Authors. Published by Elsevier B.V. All rights reserved.
Magnetization-prepared rapid acquisition with gradient echo magnetic resonance imaging signal and texture features for the prediction of mild cognitive impairment to Alzheimer’s disease progression

PubMed Central

Martinez-Torteya, Antonio; Rodriguez-Rojas, Juan; Celaya-Padilla, José M.; Galván-Tejada, Jorge I.; Treviño, Victor; Tamez-Peña, Jose

2014-01-01

Abstract. Early diagnoses of Alzheimer’s disease (AD) would confer many benefits. Several biomarkers have been proposed to achieve such a task, where features extracted from magnetic resonance imaging (MRI) have played an important role. However, studies have focused exclusively on morphological characteristics. This study aims to determine whether features relating to the signal and texture of the image could predict mild cognitive impairment (MCI) to AD progression. Clinical, biological, and positron emission tomography information and MRI images of 62 subjects from the AD neuroimaging initiative were used in this study, extracting 4150 features from each MRI. Within this multimodal database, a feature selection algorithm was used to obtain an accurate and small logistic regression model, generated by a methodology that yielded a mean blind test accuracy of 0.79. This model included six features, five of them obtained from the MRI images, and one obtained from genotyping. A risk analysis divided the subjects into low-risk and high-risk groups according to a prognostic index. The groups were statistically different (p-value=2.04e−11). These results demonstrated that MRI features related to both signal and texture add MCI to AD predictive power, and supported the ongoing notion that multimodal biomarkers outperform single-modality ones. PMID:26158047
MRI signal and texture features for the prediction of MCI to Alzheimer's disease progression

NASA Astrophysics Data System (ADS)

Martínez-Torteya, Antonio; Rodríguez-Rojas, Juan; Celaya-Padilla, José M.; Galván-Tejada, Jorge I.; Treviño, Victor; Tamez-Peña, José G.

2014-03-01

An early diagnosis of Alzheimer's disease (AD) confers many benefits. Several biomarkers from different information modalities have been proposed for the prediction of MCI to AD progression, where features extracted from MRI have played an important role. However, studies have focused almost exclusively in the morphological characteristics of the images. This study aims to determine whether features relating to the signal and texture of the image could add predictive power. Baseline clinical, biological and PET information, and MP-RAGE images for 62 subjects from the Alzheimer's Disease Neuroimaging Initiative were used in this study. Images were divided into 83 regions and 50 features were extracted from each one of these. A multimodal database was constructed, and a feature selection algorithm was used to obtain an accurate and small logistic regression model, which achieved a cross-validation accuracy of 0.96. These model included six features, five of them obtained from the MP-RAGE image, and one obtained from genotyping. A risk analysis divided the subjects into low-risk and high-risk groups according to a prognostic index, showing that both groups are statistically different (p-value of 2.04e-11). The results demonstrate that MRI features related to both signal and texture, add MCI to AD predictive power, and support the idea that multimodal biomarkers outperform single-modality biomarkers.
Neuronal correlates of the five factor model (FFM) of human personality: Multimodal imaging in a large healthy sample.

PubMed

Bjørnebekk, Astrid; Fjell, Anders M; Walhovd, Kristine B; Grydeland, Håkon; Torgersen, Svenn; Westlye, Lars T

2013-01-15

Advances in neuroimaging techniques have recently provided glimpse into the neurobiology of complex traits of human personality. Whereas some intriguing findings have connected aspects of personality to variations in brain morphology, the relations are complex and our current understanding is incomplete. Therefore, we aimed to provide a comprehensive investigation of brain-personality relations using a multimodal neuroimaging approach in a large sample comprising 265 healthy individuals. The NEO Personality Inventory was used to provide measures of core aspects of human personality, and imaging phenotypes included measures of total and regional brain volumes, regional cortical thickness and arealization, and diffusion tensor imaging indices of white matter (WM) microstructure. Neuroticism was the trait most clearly linked to brain structure. Higher neuroticism including facets reflecting anxiety, depression and vulnerability to stress was associated with smaller total brain volume, widespread decrease in WM microstructure, and smaller frontotemporal surface area. Higher scores on extraversion were associated with thinner inferior frontal gyrus, and conscientiousness was negatively associated with arealization of the temporoparietal junction. No reliable associations between brain structure and agreeableness and openness, respectively, were found. The results provide novel evidence of the associations between brain structure and variations in human personality, and corroborate previous findings of a consistent neuroanatomical basis of negative emotionality. Copyright © 2012 Elsevier Inc. All rights reserved.
Image fusion and navigation platforms for percutaneous image-guided interventions.

PubMed

Rajagopal, Manoj; Venkatesan, Aradhana M

2016-04-01

Image-guided interventional procedures, particularly image guided biopsy and ablation, serve an important role in the care of the oncology patient. The need for tumor genomic and proteomic profiling, early tumor response assessment and confirmation of early recurrence are common scenarios that may necessitate successful biopsies of targets, including those that are small, anatomically unfavorable or inconspicuous. As image-guided ablation is increasingly incorporated into interventional oncology practice, similar obstacles are posed for the ablation of technically challenging tumor targets. Navigation tools, including image fusion and device tracking, can enable abdominal interventionalists to more accurately target challenging biopsy and ablation targets. Image fusion technologies enable multimodality fusion and real-time co-displays of US, CT, MRI, and PET/CT data, with navigational technologies including electromagnetic tracking, robotic, cone beam CT, optical, and laser guidance of interventional devices. Image fusion and navigational platform technology is reviewed in this article, including the results of studies implementing their use for interventional procedures. Pre-clinical and clinical experiences to date suggest these technologies have the potential to reduce procedure risk, time, and radiation dose to both the patient and the operator, with a valuable role to play for complex image-guided interventions.
Multimodality optical imaging of embryonic heart microstructure

PubMed Central

Yelin, Ronit; Yelin, Dvir; Oh, Wang-Yuhl; Yun, Seok H.; Boudoux, Caroline; Vakoc, Benjamin J.; Bouma, Brett E.; Tearney, Guillermo J.

2009-01-01

Study of developmental heart defects requires the visualization of the microstructure and function of the embryonic myocardium, ideally with minimal alterations to the specimen. We demonstrate multiple endogenous contrast optical techniques for imaging the Xenopus laevis tadpole heart. Each technique provides distinct and complementary imaging capabilities, including: 1. 3-D coherence microscopy with subcellular (1 to 2 µm) resolution in fixed embryos, 2. real-time reflectance confocal microscopy with large penetration depth in vivo, and 3. ultra-high speed (up to 900 frames per second) that enables real-time 4-D high resolution imaging in vivo. These imaging modalities can provide a comprehensive picture of the morphologic and dynamic phenotype of the embryonic heart. The potential of endogenous-contrast optical microscopy is demonstrated for investigation of the teratogenic effects of ethanol. Microstructural abnormalities associated with high levels of ethanol exposure are observed, including compromised heart looping and loss of ventricular trabecular mass. PMID:18163837
Multimodality optical imaging of embryonic heart microstructure.

PubMed

Yelin, Ronit; Yelin, Dvir; Oh, Wang-Yuhl; Yun, Seok H; Boudoux, Caroline; Vakoc, Benjamin J; Bouma, Brett E; Tearney, Guillermo J

2007-01-01

Study of developmental heart defects requires the visualization of the microstructure and function of the embryonic myocardium, ideally with minimal alterations to the specimen. We demonstrate multiple endogenous contrast optical techniques for imaging the Xenopus laevis tadpole heart. Each technique provides distinct and complementary imaging capabilities, including: 1. 3-D coherence microscopy with subcellular (1 to 2 microm) resolution in fixed embryos, 2. real-time reflectance confocal microscopy with large penetration depth in vivo, and 3. ultra-high speed (up to 900 frames per second) that enables real-time 4-D high resolution imaging in vivo. These imaging modalities can provide a comprehensive picture of the morphologic and dynamic phenotype of the embryonic heart. The potential of endogenous-contrast optical microscopy is demonstrated for investigation of the teratogenic effects of ethanol. Microstructural abnormalities associated with high levels of ethanol exposure are observed, including compromised heart looping and loss of ventricular trabecular mass.
Multimodality hard-x-ray imaging of a chromosome with nanoscale spatial resolution

DOE PAGES

Yan, Hanfei; Nazaretski, Evgeny; Lauer, Kenneth R.; ...

2016-02-05

Here, we developed a scanning hard x-ray microscope using a new class of x-ray nano-focusing optic called a multilayer Laue lens and imaged a chromosome with nanoscale spatial resolution. The combination of the hard x-ray's superior penetration power, high sensitivity to elemental composition, high spatial-resolution and quantitative analysis creates a unique tool with capabilities that other microscopy techniques cannot provide. Using this microscope, we simultaneously obtained absorption-, phase-, and fluorescence-contrast images of Pt-stained human chromosome samples. The high spatial-resolution of the microscope and its multi-modality imaging capabilities enabled us to observe the internal ultra-structures of a thick chromosome without sectioningmore » it.« less
Multimodality hard-x-ray imaging of a chromosome with nanoscale spatial resolution

DOE Office of Scientific and Technical Information (OSTI.GOV)

Yan, Hanfei; Nazaretski, Evgeny; Lauer, Kenneth R.

Here, we developed a scanning hard x-ray microscope using a new class of x-ray nano-focusing optic called a multilayer Laue lens and imaged a chromosome with nanoscale spatial resolution. The combination of the hard x-ray's superior penetration power, high sensitivity to elemental composition, high spatial-resolution and quantitative analysis creates a unique tool with capabilities that other microscopy techniques cannot provide. Using this microscope, we simultaneously obtained absorption-, phase-, and fluorescence-contrast images of Pt-stained human chromosome samples. The high spatial-resolution of the microscope and its multi-modality imaging capabilities enabled us to observe the internal ultra-structures of a thick chromosome without sectioningmore » it.« less

Multimodality optical coherence tomography and fluorescence confocal scanning laser ophthalmoscopy for image-guided feedback of intraocular injections in mouse models

NASA Astrophysics Data System (ADS)

Benavides, Oscar R.; Terrones, Benjamin D.; Leeburg, Kelsey C.; Mehanathan, Sankarathi B.; Levine, Edward M.; Tao, Yuankai K.

2018-02-01

Rodent models are robust tools for understanding human retinal disease and function because of their similarities with human physiology and anatomy and availability of genetic mutants. Optical coherence tomography (OCT) has been well-established for ophthalmic imaging in rodents and enables depth-resolved visualization of structures and image-based surrogate biomarkers of disease. Similarly, fluorescence confocal scanning laser ophthalmoscopy (cSLO) has demonstrated utility for imaging endogenous and exogenous fluorescence and scattering contrast in the mouse retina. Complementary volumetric scattering and en face fluorescence contrast from OCT and cSLO, respectively, enables cellular-resolution longitudinal imaging of changes in ophthalmic structure and function. We present a non-contact multimodal OCT+cSLO small animal imaging system with extended working distance to the pupil, which enables imaging during and after intraocular injection. While injections are routinely performed in mice to develop novel models of ophthalmic diseases and screen novel therapeutics, the location and volume delivered is not precisely controlled and difficult to reproduce. Animals were imaged using a custom-built OCT engine and scan-head combined with a modified commercial cSLO scan-head. Post-injection imaging showed structural changes associated with retinal puncture, including the injection track, a retinal elevation, and detachment of the posterior hyaloid. When combined with imagesegmentation, we believe OCT can be used to precisely identify injection locations and quantify injection volumes. Fluorescence cSLO can provide complementary contrast for either fluorescently labeled compounds or transgenic cells for improved specificity. Our non-contact OCT+cSLO system is uniquely-suited for concurrent imaging with intraocular injections, which may be used for real-time image-guided injections.
A high-resolution multimode digital microscope system.

PubMed

Salmon, Edward D; Shaw, Sidney L; Waters, Jennifer C; Waterman-Storer, Clare M; Maddox, Paul S; Yeh, Elaine; Bloom, Kerry

2013-01-01

This chapter describes the development of a high-resolution, multimode digital imaging system based on a wide-field epifluorescent and transmitted light microscope, and a cooled charge-coupled device (CCD) camera. The three main parts of this imaging system are Nikon FXA microscope, Hamamatsu C4880 cooled CCD camera, and MetaMorph digital imaging system. This chapter presents various design criteria for the instrument and describes the major features of the microscope components-the cooled CCD camera and the MetaMorph digital imaging system. The Nikon FXA upright microscope can produce high resolution images for both epifluorescent and transmitted light illumination without switching the objective or moving the specimen. The functional aspects of the microscope set-up can be considered in terms of the imaging optics, the epi-illumination optics, the transillumination optics, the focus control, and the vibration isolation table. This instrument is somewhat specialized for microtubule and mitosis studies, and it is also applicable to a variety of problems in cellular imaging, including tracking proteins fused to the green fluorescent protein in live cells. The instrument is also valuable for correlating the assembly dynamics of individual cytoplasmic microtubules (labeled by conjugating X-rhodamine to tubulin) with the dynamics of membranes of the endoplasmic reticulum (labeled with DiOC6) and the dynamics of the cell cortex (by differential interference contrast) in migrating vertebrate epithelial cells. This imaging system also plays an important role in the analysis of mitotic mutants in the powerful yeast genetic system Saccharomyces cerevisiae. Copyright © 1998 Elsevier Inc. All rights reserved.
Cross-modal face recognition using multi-matcher face scores

NASA Astrophysics Data System (ADS)

Zheng, Yufeng; Blasch, Erik

2015-05-01

The performance of face recognition can be improved using information fusion of multimodal images and/or multiple algorithms. When multimodal face images are available, cross-modal recognition is meaningful for security and surveillance applications. For example, a probe face is a thermal image (especially at nighttime), while only visible face images are available in the gallery database. Matching a thermal probe face onto the visible gallery faces requires crossmodal matching approaches. A few such studies were implemented in facial feature space with medium recognition performance. In this paper, we propose a cross-modal recognition approach, where multimodal faces are cross-matched in feature space and the recognition performance is enhanced with stereo fusion at image, feature and/or score level. In the proposed scenario, there are two cameras for stereo imaging, two face imagers (visible and thermal images) in each camera, and three recognition algorithms (circular Gaussian filter, face pattern byte, linear discriminant analysis). A score vector is formed with three cross-matched face scores from the aforementioned three algorithms. A classifier (e.g., k-nearest neighbor, support vector machine, binomial logical regression [BLR]) is trained then tested with the score vectors by using 10-fold cross validations. The proposed approach was validated with a multispectral stereo face dataset from 105 subjects. Our experiments show very promising results: ACR (accuracy rate) = 97.84%, FAR (false accept rate) = 0.84% when cross-matching the fused thermal faces onto the fused visible faces by using three face scores and the BLR classifier.
Multimodal Reading Comprehension: Curriculum Expectations and Large-Scale Literacy Testing Practices

ERIC Educational Resources Information Center

Unsworth, Len

2014-01-01

Interpreting the image-language interface in multimodal texts is now well recognized as a crucial aspect of reading comprehension in a number of official school syllabi such as the recently published Australian Curriculum: English (ACE). This article outlines the relevant expected student learning outcomes in this curriculum and draws attention to…
An fMRI Study of Multimodal Semantic and Phonological Processing in Reading Disabled Adolescents

ERIC Educational Resources Information Center

Landi, Nicole; Mencl, W. Einar; Frost, Stephen J.; Sandak, Rebecca; Pugh, Kenneth R.

2010-01-01

Using functional magnetic resonance imaging, we investigated multimodal (visual and auditory) semantic and unimodal (visual only) phonological processing in reading disabled (RD) adolescents and non-impaired (NI) control participants. We found reduced activation for RD relative to NI in a number of left-hemisphere reading-related areas across all…
A Multimodal Perspective on Textuality and Contexts

ERIC Educational Resources Information Center

Jewitt, Carey

2007-01-01

Textuality is often thought of in linguistic terms; for instance, the talk and writing that circulate in the classroom. In this paper I take a multimodal perspective on textuality and context. I draw on illustrative examples from school Science and English to examine how image, colour, gesture, gaze, posture and movement--as well as writing and…
"Convince Me!" Valuing Multimodal Literacies and Composing Public Service Announcements

ERIC Educational Resources Information Center

Selfe, Richard J.; Selfe, Cynthia L.

2008-01-01

For some teachers, the increasing attention to digital and multimodal composing in English and Language Arts classrooms has brought into sharp relief the profession's investment in print as the primary means of expression. Although new forms of communication that combine words, still and moving images, and animation have begun to dominate digital…
Concept for Classifying Facade Elements Based on Material, Geometry and Thermal Radiation Using Multimodal Uav Remote Sensing

NASA Astrophysics Data System (ADS)

Ilehag, R.; Schenk, A.; Hinz, S.

2017-08-01

This paper presents a concept for classification of facade elements, based on the material and the geometry of the elements in addition to the thermal radiation of the facade with the usage of a multimodal Unmanned Aerial Vehicle (UAV) system. Once the concept is finalized and functional, the workflow can be used for energy demand estimations for buildings by exploiting existing methods for estimation of heat transfer coefficient and the transmitted heat loss. The multimodal system consists of a thermal, a hyperspectral and an optical sensor, which can be operational with a UAV. While dealing with sensors that operate in different spectra and have different technical specifications, such as the radiometric and the geometric resolution, the challenges that are faced are presented. Addressed are the different approaches of data fusion, such as image registration, generation of 3D models by performing image matching and the means for classification based on either the geometry of the object or the pixel values. As a first step towards realizing the concept, the result from a geometric calibration with a designed multimodal calibration pattern is presented.
MCA-NMF: Multimodal Concept Acquisition with Non-Negative Matrix Factorization

PubMed Central

Mangin, Olivier; Filliat, David; ten Bosch, Louis; Oudeyer, Pierre-Yves

2015-01-01

In this paper we introduce MCA-NMF, a computational model of the acquisition of multimodal concepts by an agent grounded in its environment. More precisely our model finds patterns in multimodal sensor input that characterize associations across modalities (speech utterances, images and motion). We propose this computational model as an answer to the question of how some class of concepts can be learnt. In addition, the model provides a way of defining such a class of plausibly learnable concepts. We detail why the multimodal nature of perception is essential to reduce the ambiguity of learnt concepts as well as to communicate about them through speech. We then present a set of experiments that demonstrate the learning of such concepts from real non-symbolic data consisting of speech sounds, images, and motions. Finally we consider structure in perceptual signals and demonstrate that a detailed knowledge of this structure, named compositional understanding can emerge from, instead of being a prerequisite of, global understanding. An open-source implementation of the MCA-NMF learner as well as scripts and associated experimental data to reproduce the experiments are publicly available. PMID:26489021
Computer-aided psychotherapy based on multimodal elicitation, estimation and regulation of emotion.

PubMed

Cosić, Krešimir; Popović, Siniša; Horvat, Marko; Kukolja, Davor; Dropuljić, Branimir; Kovač, Bernard; Jakovljević, Miro

2013-09-01

Contemporary psychiatry is looking at affective sciences to understand human behavior, cognition and the mind in health and disease. Since it has been recognized that emotions have a pivotal role for the human mind, an ever increasing number of laboratories and research centers are interested in affective sciences, affective neuroscience, affective psychology and affective psychopathology. Therefore, this paper presents multidisciplinary research results of Laboratory for Interactive Simulation System at Faculty of Electrical Engineering and Computing, University of Zagreb in the stress resilience. Patient's distortion in emotional processing of multimodal input stimuli is predominantly consequence of his/her cognitive deficit which is result of their individual mental health disorders. These emotional distortions in patient's multimodal physiological, facial, acoustic, and linguistic features related to presented stimulation can be used as indicator of patient's mental illness. Real-time processing and analysis of patient's multimodal response related to annotated input stimuli is based on appropriate machine learning methods from computer science. Comprehensive longitudinal multimodal analysis of patient's emotion, mood, feelings, attention, motivation, decision-making, and working memory in synchronization with multimodal stimuli provides extremely valuable big database for data mining, machine learning and machine reasoning. Presented multimedia stimuli sequence includes personalized images, movies and sounds, as well as semantically congruent narratives. Simultaneously, with stimuli presentation patient provides subjective emotional ratings of presented stimuli in terms of subjective units of discomfort/distress, discrete emotions, or valence and arousal. These subjective emotional ratings of input stimuli and corresponding physiological, speech, and facial output features provides enough information for evaluation of patient's cognitive appraisal deficit. Aggregated real-time visualization of this information provides valuable assistance in patient mental state diagnostics enabling therapist deeper and broader insights into dynamics and progress of the psychotherapy.
Design and characterization of a handheld multimodal imaging device for the assessment of oral epithelial lesions

NASA Astrophysics Data System (ADS)

Higgins, Laura M.; Pierce, Mark C.

2014-08-01

A compact handpiece combining high resolution fluorescence (HRF) imaging with optical coherence tomography (OCT) was developed to provide real-time assessment of oral lesions. This multimodal imaging device simultaneously captures coregistered en face images with subcellular detail alongside cross-sectional images of tissue microstructure. The HRF imaging acquires a 712×594 μm2 field-of-view at the sample with a spatial resolution of 3.5 μm. The OCT images were acquired to a depth of 1.5 mm with axial and lateral resolutions of 9.3 and 8.0 μm, respectively. HRF and OCT images are simultaneously displayed at 25 fps. The handheld device was used to image a healthy volunteer, demonstrating the potential for in vivo assessment of the epithelial surface for dysplastic and neoplastic changes at the cellular level, while simultaneously evaluating submucosal involvement. We anticipate potential applications in real-time assessment of oral lesions for improved surveillance and surgical guidance.
Papillary fibroelastoma diagnosed through multimodality cardiac imaging: a rare tumour in an uncommon location with review of literature.

PubMed

Anand, Senthil; Sydow, Nicole; Janardhanan, Rajesh

2017-08-08

We describe the case of a woman presenting with transient ischaemic attack, who was found to have a papillary fibroelastoma arising from the aortic wall, an extremely rare location. We describe the multimodality imaging techniques used in diagnosing this patient and review the most recent literature on evaluation and management of patients with cardiac papillary fibroelastomas. © BMJ Publishing Group Ltd (unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.
Multimodality Imaging in Patients with Secondary Hypertension: With a Focus on Appropriate Imaging Approaches Depending on the Etiologies.

PubMed

Ahn, Hyungwoo; Chun, Eun Ju; Lee, Hak Jong; Hwang, Sung Il; Choi, Dong-Ju; Chae, In-Ho; Lee, Kyung Won

2018-01-01

Although the causes of hypertension are usually unknown, about 10% of the cases occur secondary to specific etiologies, which are often treatable. Common categories of secondary hypertension include renal parenchymal disease, renovascular stenosis, vascular and endocrinologic disorders. For diseases involving the renal parenchyma and adrenal glands, ultrasonography (US), computed tomography (CT) or magnetic resonance (MR) imaging is recommended. For renovascular stenosis and vascular disorders, Doppler US, conventional or noninvasive (CT or MR) angiography is an appropriate modality. Nuclear imaging can be useful in the differential diagnosis of endocrine causes. Radiologists should understand the role of each imaging modality and its typical findings in various causes of secondary hypertension. This article focuses on appropriate imaging approaches in accordance with the categorized etiologies leading to hypertension.
Nanoparticles and Radiotracers: Advances toward Radio-Nanomedicine

PubMed Central

Pratt, Edwin C.; Shaffer, Travis M.; Grimm, Jan

2016-01-01

Here, we cover the convergence of radiochemistry for imaging and therapy with advances in nanoparticle (NP) design for biomedical applications. We first explore NP properties relevant for therapy and theranostics and emphasize the need for biocompatibility. We then explore radionuclide-imaging modalities such as Positron Emission Tomography (PET), Single Photon Emission Computed Tomography (SPECT), and Cerenkov Luminescence (CL) with examples utilizing radiolabeled NP for imaging. PET and SPECT have served as diagnostic workhorses in the clinic, while preclinical NP design examples of multimodal imaging with radiotracers show promise in imaging and therapy. CL expands the types of radionuclides beyond PET and SPECT tracers to include high-energy electrons (β−) for imaging purposes. These advances in radionanomedicine will be discussed, showing the potential for radiolabeled NPs as theranostic agents. PMID:27006133
Hybrid Core-Shell (HyCoS) Nanoparticles produced by Complex Coacervation for Multimodal Applications

NASA Astrophysics Data System (ADS)

Vecchione, D.; Grimaldi, A. M.; Forte, E.; Bevilacqua, Paolo; Netti, P. A.; Torino, E.

2017-03-01

Multimodal imaging probes can provide diagnostic information combining different imaging modalities. Nanoparticles (NPs) can contain two or more imaging tracers that allow several diagnostic techniques to be used simultaneously. In this work, a complex coacervation process to produce core-shell completely biocompatible polymeric nanoparticles (HyCoS) for multimodal imaging applications is described. Innovations on the traditional coacervation process are found in the control of the reaction temperature, allowing a speeding up of the reaction itself, and the production of a double-crosslinked system to improve the stability of the nanostructures in the presence of a clinically relevant contrast agent for MRI (Gd-DTPA). Through the control of the crosslinking behavior, an increase up to 6 times of the relaxometric properties of the Gd-DTPA is achieved. Furthermore, HyCoS can be loaded with a high amount of dye such as ATTO 633 or conjugated with a model dye such as FITC for in vivo optical imaging. The results show stable core-shell polymeric nanoparticles that can be used both for MRI and for optical applications allowing detection free from harmful radiation. Additionally, preliminary results about the possibility to trigger the release of a drug through a pH effect are reported.
The Neurochemical and Microstructural Changes in the Brain of Systemic Lupus Erythematosus Patients: A Multimodal MRI Study

PubMed Central

Zhang, Zhiyan; Wang, Yukai; Shen, Zhiwei; Yang, Zhongxian; Li, Li; Chen, Dongxiao; Yan, Gen; Cheng, Xiaofang; Shen, Yuanyu; Tang, Xiangyong; Hu, Wei; Wu, Renhua

2016-01-01

The diagnosis and pathology of neuropsychiatric systemic lupus erythematosus (NPSLE) remains challenging. Herein, we used multimodal imaging to assess anatomical and functional changes in brains of SLE patients instead of a single MRI approach generally used in previous studies. Twenty-two NPSLE patients, 21 non-NPSLE patients and 20 healthy controls (HCs) underwent 3.0 T MRI with multivoxel magnetic resonance spectroscopy, T1-weighted volumetric images for voxel based morphometry (VBM) and diffusional kurtosis imaging (DKI) scans. While there were findings in other basal ganglia regions, the most consistent findings were observed in the posterior cingulate gyrus (PCG). The reduction of multiple metabolite concentration was observed in the PCG in the two patient groups, and the NPSLE patients were more prominent. The two patient groups displayed lower diffusional kurtosis (MK) values in the bilateral PCG compared with HCs (p < 0.01) as assessed by DKI. Grey matter reduction in the PCG was observed in the NPSLE group using VBM. Positive correlations among cognitive function scores and imaging metrics in bilateral PCG were detected. Multimodal imaging is useful for evaluating SLE subjects and potentially determining disease pathology. Impairments of cognitive function in SLE patients may be interpreted by metabolic and microstructural changes in the PCG. PMID:26758023
Luminomagnetic Eu3+- and Dy3+-doped hydroxyapatite for multimodal imaging.

PubMed

Tesch, Annemarie; Wenisch, Christoph; Herrmann, Karl-Heinz; Reichenbach, Jürgen R; Warncke, Paul; Fischer, Dagmar; Müller, Frank A

2017-12-01

Multimodal imaging has recently attracted much attention due to the advantageous combination of different imaging modalities, like photoluminescence (PL) and magnetic resonance imaging (MRI). In the present study, luminescent and magnetic hydroxyapatites (HAp) were prepared via doping with europium (Eu 3+ ) and dysprosium (Dy 3+ ), respectively. Co-doping of Eu 3+ and Dy 3+ was used to combine the desired physical properties. Both lanthanide ions were successfully incorporated in the HAp crystal lattice, where they preferentially occupied calcium(I) sites. While Eu-doped HAp (Eu:HAp) exhibits dopant concentration dependent persistent PL properties, Dy-doped HAp (Dy:HAp) shows paramagnetic behavior due to the high magnetic moment of Dy 3+ . Co-doped HAp (Eu:Dy:HAp) nanoparticles combine both properties in one single crystal. Remarkably, multimodal co-doped HAp features enhanced PL properties due to an energy transfer from Dy 3+ sensitizer to Eu 3+ activator ions. Eu:Dy:HAp exhibits strong transverse relaxation effects with a maximum transverse relaxivity of 83.3L/(mmol·s). Due to their tunable PL, magnetic properties and cytocompatibility Eu:-, Dy:- and Eu:Dy:HAp represent promising biocompatible ceramic materials for luminescence imaging that simultaneously may serve as a contrast agent for MRI in permanent implants or functional coatings. Copyright © 2017 Elsevier B.V. All rights reserved.
Shape-Controlled Synthesis of Isotopic Yttrium-90-Labeled Rare Earth Fluoride Nanocrystals for Multimodal Imaging.

PubMed

Paik, Taejong; Chacko, Ann-Marie; Mikitsh, John L; Friedberg, Joseph S; Pryma, Daniel A; Murray, Christopher B

2015-09-22

Isotopically labeled nanomaterials have recently attracted much attention in biomedical research, environmental health studies, and clinical medicine because radioactive probes allow the elucidation of in vitro and in vivo cellular transport mechanisms, as well as the unambiguous distribution and localization of nanomaterials in vivo. In addition, nanocrystal-based inorganic materials have a unique capability of customizing size, shape, and composition; with the potential to be designed as multimodal imaging probes. Size and shape of nanocrystals can directly influence interactions with biological systems, hence it is important to develop synthetic methods to design radiolabeled nanocrystals with precise control of size and shape. Here, we report size- and shape-controlled synthesis of rare earth fluoride nanocrystals doped with the β-emitting radioisotope yttrium-90 ((90)Y). Size and shape of nanocrystals are tailored via tight control of reaction parameters and the type of rare earth hosts (e.g., Gd or Y) employed. Radiolabeled nanocrystals are synthesized in high radiochemical yield and purity as well as excellent radiolabel stability in the face of surface modification with different polymeric ligands. We demonstrate the Cerenkov radioluminescence imaging and magnetic resonance imaging capabilities of (90)Y-doped GdF3 nanoplates, which offer unique opportunities as a promising platform for multimodal imaging and targeted therapy.
Frameless multimodal image guidance of localized convection-enhanced delivery of therapeutics in the brain

PubMed Central

van der Bom, Imramsjah M J; Moser, Richard P; Gao, Guanping; Sena-Esteves, Miguel; Aronin, Neil

2013-01-01

Introduction Convection-enhanced delivery (CED) has been shown to be an effective method of administering macromolecular compounds into the brain that are unable to cross the blood-brain barrier. Because the administration is highly localized, accurate cannula placement by minimally invasive surgery is an important requisite. This paper reports on the use of an angiographic c-arm system which enables truly frameless multimodal image guidance during CED surgery. Methods A microcannula was placed into the striatum of five sheep under real-time fluoroscopic guidance using imaging data previously acquired by cone beam computed tomography (CBCT) and MRI, enabling three-dimensional navigation. After introduction of the cannula, high resolution CBCT was performed and registered with MRI to confirm the position of the cannula tip and to make adjustments as necessary. Adeno-associated viral vector-10, designed to deliver small-hairpin micro RNA (shRNAmir), was mixed with 2.0 mM gadolinium (Gd) and infused at a rate of 3 μl/min for a total of 100 μl. Upon completion, the animals were transferred to an MR scanner to assess the approximate distribution by measuring the volume of spread of Gd. Results The cannula was successfully introduced under multimodal image guidance. High resolution CBCT enabled validation of the cannula position and Gd-enhanced MRI after CED confirmed localized administration of the therapy. Conclusion A microcannula for CED was introduced into the striatum of five sheep under multimodal image guidance. The non-alloy 300 μm diameter cannula tip was well visualized using CBCT, enabling confirmation of the position of the end of the tip in the area of interest. PMID:22193239
Morphology supporting function: attenuation correction for SPECT/CT, PET/CT, and PET/MR imaging

PubMed Central

Lee, Tzu C.; Alessio, Adam M.; Miyaoka, Robert M.; Kinahan, Paul E.

2017-01-01

Both SPECT, and in particular PET, are unique in medical imaging for their high sensitivity and direct link to a physical quantity, i.e. radiotracer concentration. This gives PET and SPECT imaging unique capabilities for accurately monitoring disease activity for the purposes of clinical management or therapy development. However, to achieve a direct quantitative connection between the underlying radiotracer concentration and the reconstructed image values several confounding physical effects have to be estimated, notably photon attenuation and scatter. With the advent of dual-modality SPECT/CT, PET/CT, and PET/MR scanners, the complementary CT or MR image data can enable these corrections, although there are unique challenges for each combination. This review covers the basic physics underlying photon attenuation and scatter and summarizes technical considerations for multimodal imaging with regard to PET and SPECT quantification and methods to address the challenges for each multimodal combination. PMID:26576737

Tissue imaging using full field optical coherence microscopy with short multimode fiber probe

NASA Astrophysics Data System (ADS)

Sato, Manabu; Eto, Kai; Goto, Tetsuhiro; Kurotani, Reiko; Abe, Hiroyuki; Nishidate, Izumi

2018-03-01

In achieving minimally invasive accessibility to deeply located regions the size of the imaging probes is important. We demonstrated full-field optical coherence tomography (FF-OCM) using an ultrathin forward-imaging short multimode fiber (SMMF) probe of 50 μm core diameter, 125 μm diameter, and 7.4 mm length for optical communications. The axial resolution was measured to be 2.14 μm and the lateral resolution was also evaluated to be below 4.38 μm using a test pattern (TP). The spatial mode and polarization characteristics of SMMF were evaluated. Inserting SMMF to in vivo rat brain, 3D images were measured and 2D information of nerve fibers was obtained. The feasibility of an SMMF as an ultrathin forward-imaging probe in FF-OCM has been demonstrated.
vECTlab—A fully integrated multi-modality Monte Carlo simulation framework for the radiological imaging sciences

NASA Astrophysics Data System (ADS)

Peter, Jörg; Semmler, Wolfhard

2007-10-01

Alongside and in part motivated by recent advances in molecular diagnostics, the development of dual-modality instruments for patient and dedicated small animal imaging has gained attention by diverse research groups. The desire for such systems is high not only to link molecular or functional information with the anatomical structures, but also for detecting multiple molecular events simultaneously at shorter total acquisition times. While PET and SPECT have been integrated successfully with X-ray CT, the advance of optical imaging approaches (OT) and the integration thereof into existing modalities carry a high application potential, particularly for imaging small animals. A multi-modality Monte Carlo (MC) simulation approach at present has been developed that is able to trace high-energy (keV) as well as optical (eV) photons concurrently within identical phantom representation models. We show that the involved two approaches for ray-tracing keV and eV photons can be integrated into a unique simulation framework which enables both photon classes to be propagated through various geometry models representing both phantoms and scanners. The main advantage of such integrated framework for our specific application is the investigation of novel tomographic multi-modality instrumentation intended for in vivo small animal imaging through time-resolved MC simulation upon identical phantom geometries. Design examples are provided for recently proposed SPECT-OT and PET-OT imaging systems.
Multimodal autofluorescence detection of cancer: from single cells to living organism

NASA Astrophysics Data System (ADS)

Horilova, J.; Cunderlikova, B.; Cagalinec, M.; Chorvat, D.; Marcek Chorvatova, A.

2018-02-01

Multimodal optical imaging of suspected tissues is showing to be a promising method for distinguishing suspected cancerous tissues from healthy ones. In particular, the combination of steady-state spectroscopic methods with timeresolved fluorescence provides more precise insight into native metabolism when focused on tissue autofluorescence. Cancer is linked to specific metabolic remodelation detectable spectroscopically. In this work, we evaluate possibilities and limitations of multimodal optical cancer detection in single cells, collagen-based 3D cell cultures and in living organisms (whole mice), as a representation of gradually increasing complexity of model systems.
Imaging as a biomarker in drug discovery for Alzheimer’s disease: is MRI a suitable technology?

PubMed Central

2014-01-01

This review provides perspectives on the utility of magnetic resonance imaging (MRI) as a neuroimaging approach in the development of novel treatments for Alzheimer’s disease. These considerations were generated in a roundtable at a recent Wellcome Trust meeting that included experts from academia and industry. It was agreed that MRI, either structural or functional, could be used as a diagnostic, for assessing worsening of disease status, for monitoring vascular pathology, and for stratifying clinical trial populations. It was agreed also that MRI implementation is in its infancy, requiring more evidence of association with the disease states, test-retest data, better standardization across multiple clinical sites, and application in multimodal approaches which include other imaging technologies, such as positron emission tomography, electroencephalography, and magnetoencephalography. PMID:25484927
DOE Office of Scientific and Technical Information (OSTI.GOV)

Kamran, Mudassar, E-mail: kamranm@mir.wustl.edu; Fowler, Kathryn J., E-mail: fowlerk@mir.wustl.edu; Mellnick, Vincent M., E-mail: mellnickv@mir.wustl.edu

Primary aortic neoplasms are rare. Aortic sarcoma arising after endovascular aneurysm repair (EVAR) is a scarce subset of primary aortic malignancies, reports of which are infrequent in the published literature. The diagnosis of aortic sarcoma is challenging due to its non-specific clinical presentation, and the prognosis is poor due to delayed diagnosis, rapid proliferation, and propensity for metastasis. Post-EVAR, aortic sarcomas may mimic other more common aortic processes on surveillance imaging. Radiologists are rarely knowledgeable about this rare entity for which multimodality imaging and awareness are invaluable in early diagnosis. A series of three pathologically confirmed cases are presented tomore » display the multimodality imaging features and clinical presentations of aortic sarcoma arising after EVAR.« less
Multimodal device for assessment of skin malformations

NASA Astrophysics Data System (ADS)

Bekina, A.; Garancis, V.; Rubins, U.; Spigulis, J.; Valeine, L.; Berzina, A.

2013-11-01

A variety of multi-spectral imaging devices is commercially available and used for skin diagnostics and monitoring; however, an alternative cost-efficient device can provide an advanced spectral analysis of skin. A compact multimodal device for diagnosis of pigmented skin lesions was developed and tested. A polarized LED light source illuminates the skin surface at four different wavelengths - blue (450 nm), green (545 nm), red (660 nm) and infrared (940 nm). Spectra of reflected light from the 25 mm wide skin spot are imaged by a CMOS sensor. Four spectral images are obtained for mapping of the main skin chromophores. The specific chromophore distribution differences between different skin malformations were analyzed and information of subcutaneous structures was consecutively extracted.
Multimodal instrument for high-sensitivity autofluorescence and spectral optical coherence tomography of the human eye fundus

PubMed Central

Komar, Katarzyna; Stremplewski, Patrycjusz; Motoczyńska, Marta; Szkulmowski, Maciej; Wojtkowski, Maciej

2013-01-01

In this paper we present a multimodal device for imaging fundus of human eye in vivo which combines functionality of autofluorescence by confocal SLO with Fourier domain OCT. Native fluorescence of human fundus was excited by modulated laser beam (λ = 473 nm, 20 MHz) and lock-in detection was applied resulting in improving sensitivity. The setup allows for acquisition of high resolution OCT and high contrast AF images using fluorescence excitation power of 50-65 μW without averaging consecutive images. Successful functioning of constructed device have been demonstrated for 8 healthy volunteers of different age ranging from 24 to 83 years old. PMID:24298426
Multifunctional fluorescent and magnetic nanoparticles for biomedical applications

NASA Astrophysics Data System (ADS)

Selvan, Subramanian T.

2012-03-01

Hybrid multifunctional nanoparticles (NPs) are emerging as useful probes for magnetic based targeting, delivery, cell separation, magnetic resonance imaging (MRI), and fluorescence-based bio-labeling applications. Assessing from the literature, the development of multifunctional NPs for multimodality imaging is still in its infancy state. This report focuses on our recent work on quantum dots (QDs), magnetic NPs (MNPs) and bi-functional NPs (composed of either QDs or rare-earth NPs, and magnetic NPs - iron oxide or gadolinium oxide) for multimodality imaging based biomedical applications. The combination of MRI and fluorescence would ally each other in improving the sensitivity and resolution, resulting in improved and early diagnosis of the disease. The challenges in this area are discussed.
Molecular Imaging and Therapy of Prostate Cancer

DTIC Science & Technology

2015-10-01

arsenic-based, IGF1R-targeted radiopharmaceuticals can allow for PET imaging, IRT, and monitoring the therapeutic response of PCa. Specific Aims: Aim 1: To...models with PET imaging. Aim 3: To monitor the efficacy of 76As-based IRT of PCa with multimodality imaging.
Multimodal Medical Image Fusion by Adaptive Manifold Filter.

PubMed

Geng, Peng; Liu, Shuaiqi; Zhuang, Shanna

2015-01-01

Medical image fusion plays an important role in diagnosis and treatment of diseases such as image-guided radiotherapy and surgery. The modified local contrast information is proposed to fuse multimodal medical images. Firstly, the adaptive manifold filter is introduced into filtering source images as the low-frequency part in the modified local contrast. Secondly, the modified spatial frequency of the source images is adopted as the high-frequency part in the modified local contrast. Finally, the pixel with larger modified local contrast is selected into the fused image. The presented scheme outperforms the guided filter method in spatial domain, the dual-tree complex wavelet transform-based method, nonsubsampled contourlet transform-based method, and four classic fusion methods in terms of visual quality. Furthermore, the mutual information values by the presented method are averagely 55%, 41%, and 62% higher than the three methods and those values of edge based similarity measure by the presented method are averagely 13%, 33%, and 14% higher than the three methods for the six pairs of source images.
Ions doped melanin nanoparticle as a multiple imaging agent.

PubMed

Ha, Shin-Woo; Cho, Hee-Sang; Yoon, Young Il; Jang, Moon-Sun; Hong, Kwan Soo; Hui, Emmanuel; Lee, Jung Hee; Yoon, Tae-Jong

2017-10-10

Multimodal nanomaterials are useful for providing enhanced diagnostic information simultaneously for a variety of in vivo imaging methods. According to our research findings, these multimodal nanomaterials offer promising applications for cancer therapy. Melanin nanoparticles can be used as a platform imaging material and they can be simply produced by complexation with various imaging active ions. They are capable of specifically targeting epidermal growth factor receptor (EGFR)-expressing cancer cells by being anchored with a specific antibody. Ion-doped melanin nanoparticles were found to have high bioavailability with long-term stability in solution, without any cytotoxicity in both in vitro and in vivo systems. By combining different imaging modalities with melanin particles, we can use the complexes to obtain faster diagnoses by computed tomography deep-body imaging and greater detailed pathological diagnostic information by magnetic resonance imaging. The ion-doped melanin nanoparticles also have applications for radio-diagnostic treatment and radio imaging-guided surgery, warranting further proof of concept experimental.
Multimodal imaging system for dental caries detection

NASA Astrophysics Data System (ADS)

Liang, Rongguang; Wong, Victor; Marcus, Michael; Burns, Peter; McLaughlin, Paul

2007-02-01

Dental caries is a disease in which minerals of the tooth are dissolved by surrounding bacterial plaques. A caries process present for some time may result in a caries lesion. However, if it is detected early enough, the dentist and dental professionals can implement measures to reverse and control caries. Several optical, nonionized methods have been investigated and used to detect dental caries in early stages. However, there is not a method that can singly detect the caries process with both high sensitivity and high specificity. In this paper, we present a multimodal imaging system that combines visible reflectance, fluorescence, and Optical Coherence Tomography (OCT) imaging. This imaging system is designed to obtain one or more two-dimensional images of the tooth (reflectance and fluorescence images) and a three-dimensional OCT image providing depth and size information of the caries. The combination of two- and three-dimensional images of the tooth has the potential for highly sensitive and specific detection of dental caries.
Dual delivery of biological therapeutics for multimodal and synergistic cancer therapies.

PubMed

Jang, Bora; Kwon, Hyokyoung; Katila, Pramila; Lee, Seung Jin; Lee, Hyukjin

2016-03-01

Cancer causes >8.2 million deaths annually worldwide; thus, various cancer treatments have been investigated over the past decades. Among them, combination drug therapy has become extremely popular, and treatment with more than one drug is often necessary to achieve appropriate anticancer efficacy. With the development of nanoformulations and nanoparticulate-based drug delivery, researchers have explored the feasibility of dual delivery of biological therapeutics to overcome the current drawbacks of cancer therapy. Compared with the conventional single drug therapy, dual delivery of therapeutics has provided various synergistic effects in addition to offering multimodality to cancer treatment. In this review, we highlight and summarize three aspects of dual-delivery systems for cancer therapy. These include (1) overcoming drug resistance by the dual delivery of chemical drugs with biological therapeutics for synergistic therapy, (2) targeted and controlled drug release by the dual delivery of drugs with stimuli-responsive nanomaterials, and (3) multimodal theranostics by the dual delivery of drugs and molecular imaging probes. Furthermore, recent developments, perspectives, and new challenges regarding dual-delivery systems for cancer therapy are discussed. Copyright © 2015 Elsevier B.V. All rights reserved.
Chronic sacroiliac joint and pelvic girdle dysfunction in a 35-year-old nulliparous woman successfully managed with multimodal and multidisciplinary approach

PubMed Central

Jonely, Holly; Brismée, Jean-Michel; Desai, Mehul J; Reoli, Rachel

2015-01-01

Background and purpose: Sacroiliac joint pain and dysfunction affect 15–25% of patients reporting low back pain, including reports of spontaneous, idiopathic, traumatic, and non-traumatic onsets. The poor reliability and validity associated with diagnostic clinical and imaging techniques leads to challenges in diagnosing and managing sacroiliac joint dysfunction. Case description: A 35-year-old nulliparous female with a 14-year history of right sacroiliac joint dysfunction was managed using a multimodal and multidisciplinary approach when symptoms failed to resolve after 2 months of physical therapy. The plan of care included four prolotherapy injections, sacroiliac joint manipulation into nutation, pelvic girdle belting, and specific stabilization exercises. Outcomes: The patient completed 20 physical therapy sessions over a 12-month period. At 6 months, the patient’s Oswestry Disability Questionnaire score was reduced from 34% to 14%. At 1-year follow-up, her score was 0%. The patient’s rating of pain on a numeric rating scale decreased to an average of 4/10 at 6 months and 0/10 at 1-year follow-up. Discussion: A multidisciplinary and multimodal approach for the management of chronic sacroiliac joint dysfunction appeared successful in a single-case design at 1-year follow-up. PMID:26309378
Evaluation of the role of CD207 on Langerhans cells in a murine model of atopic dermatitis by in situ imaging using Cr:forsterite laser-based multimodality nonlinear microscopy

NASA Astrophysics Data System (ADS)

Lee, Jyh-Hong; Tsai, Ming-Rung; Sun, Chi-Kuang; Chiang, Bor-Luen

2012-11-01

Atopic dermatitis (AD) is an allergic inflammatory disease of skin. It remains unclear that CD207 of Langerhans cells (LCs) plays a central role in the development of allergic sensitization. There is little data on LCs within the microenviroment in vivo. We used a murine model of epicutaneous (EC) ovalbumin (OVA) sensitization inducing an inflammatory skin resembling AD to explore the role of CD207 in the pathogenesis of AD. Cr:forsterite laser-based multimodality nonlinear microscopy was applied for in situ imaging. Peritoneal injections of Alexa Fluor 647-rat anti-mouse CD207 into mice were performed to specifically trace the LCs. Peritoneal injections of OVA-Alexa Fluor 647 conjugate into mice were performed to specifically trace the OVA. We found that combining Alexa Fluor fluorescent probes with multimodality nonlinear microscopy permitted the unequivocal in situ imaging of CD207-expressing LCs. The relevant time-course, expressional, and functional studies reveal that CD207 of LCs plays an essential role during the induction of EC sensitization. We establish and validate that Cr:forsterite laser-based multimodality nonlinear microscopy is applicable for the specific detection of labeled mAb-bound LCs and labeled antigen. We suggest that CD207-expressing LCs initiate the allergic response through the CD207 mediated epicutaneous sensitization associated with the development of AD.
Active edge maps for medical image registration

NASA Astrophysics Data System (ADS)

Kerwin, William; Yuan, Chun

2001-07-01

Applying edge detection prior to performing image registration yields several advantages over raw intensity- based registration. Advantages include the ability to register multicontrast or multimodality images, immunity to intensity variations, and the potential for computationally efficient algorithms. In this work, a common framework for edge-based image registration is formulated as an adaptation of snakes used in boundary detection. Called active edge maps, the new formulation finds a one-to-one transformation T(x) that maps points in a source image to corresponding locations in a target image using an energy minimization approach. The energy consists of an image component that is small when edge features are well matched in the two images, and an internal term that restricts T(x) to allowable configurations. The active edge map formulation is illustrated here with a specific example developed for affine registration of carotid artery magnetic resonance images. In this example, edges are identified using a magnitude of gradient operator, image energy is determined using a Gaussian weighted distance function, and the internal energy includes separate, adjustable components that control volume preservation and rigidity.
PET-CMR in heart failure - synergistic or redundant imaging?

PubMed

Quail, Michael A; Sinusas, Albert J

2017-07-01

Imaging in heart failure (HF) provides data for diagnosis, prognosis and disease monitoring. Both MRI and nuclear imaging techniques have been successfully used for this purpose in HF. Positron Emission Tomography-Cardiac Magnetic Resonance (PET-CMR) is an example of a new multimodality diagnostic imaging technique with potential applications in HF. The threshold for adopting a new diagnostic tool to clinical practice must necessarily be high, lest they exacerbate costs without improving care. New modalities must demonstrate clinical superiority, or at least equivalence, combined with another important advantage, such as lower cost or improved patient safety. The purpose of this review is to outline the current status of multimodality PET-CMR with regard to HF applications, and determine whether the clinical utility of this new technology justifies the cost.
Multimodal nonlinear imaging of arabidopsis thaliana root cell

NASA Astrophysics Data System (ADS)

Jang, Bumjoon; Lee, Sung-Ho; Woo, Sooah; Park, Jong-Hyun; Lee, Myeong Min; Park, Seung-Han

2017-07-01

Nonlinear optical microscopy has enabled the possibility to explore inside the living organisms. It utilizes ultrashort laser pulse with long wavelength (greater than 800nm). Ultrashort pulse produces high peak power to induce nonlinear optical phenomenon such as two-photon excitation fluorescence (TPEF) and harmonic generations in the medium while maintaining relatively low average energy pre area. In plant developmental biology, confocal microscopy is widely used in plant cell imaging after the development of biological fluorescence labels in mid-1990s. However, fluorescence labeling itself affects the sample and the sample deviates from intact condition especially when labelling the entire cell. In this work, we report the dynamic images of Arabidopsis thaliana root cells. This demonstrates the multimodal nonlinear optical microscopy is an effective tool for long-term plant cell imaging.
Simultaneous acquisition of magnetic resonance spectroscopy (MRS) data and positron emission tomography (PET) images with a prototype MR-compatible, small animal PET imager

NASA Astrophysics Data System (ADS)

Raylman, Raymond R.; Majewski, Stan; Velan, S. Sendhil; Lemieux, Susan; Kross, Brian; Popov, Vladimir; Smith, Mark F.; Weisenberger, Andrew G.

2007-06-01

Multi-modality imaging (such as PET-CT) is rapidly becoming a valuable tool in the diagnosis of disease and in the development of new drugs. Functional images produced with PET, fused with anatomical images created by MRI, allow the correlation of form with function. Perhaps more exciting than the combination of anatomical MRI with PET, is the melding of PET with MR spectroscopy (MRS). Thus, two aspects of physiology could be combined in novel ways to produce new insights into the physiology of normal and pathological processes. Our team is developing a system to acquire MRI images and MRS spectra, and PET images contemporaneously. The prototype MR-compatible PET system consists of two opposed detector heads (appropriate in size for small animal imaging), operating in coincidence mode with an active field-of-view of ˜14 cm in diameter. Each detector consists of an array of LSO detector elements coupled through a 2-m long fiber optic light guide to a single position-sensitive photomultiplier tube. The use of light guides allows these magnetic field-sensitive elements of the PET imager to be positioned outside the strong magnetic field of our 3T MRI scanner. The PET scanner imager was integrated with a 12-cm diameter, 12-leg custom, birdcage coil. Simultaneous MRS spectra and PET images were successfully acquired from a multi-modality phantom consisting of a sphere filled with 17 brain relevant substances and a positron-emitting radionuclide. There were no significant changes in MRI or PET scanner performance when both were present in the MRI magnet bore. This successful initial test demonstrates the potential for using such a multi-modality to obtain complementary MRS and PET data.
Development of optical neuroimaging to detect drug-induced brain functional changes in vivo

NASA Astrophysics Data System (ADS)

Du, Congwu; Pan, Yingtian

2014-03-01

Deficits in prefrontal function play a crucial role in compulsive cocaine use, which is a hallmark of addiction. Dysfunction of the prefrontal cortex might result from effects of cocaine on neurons as well as from disruption of cerebral blood vessels. However, the mechanisms underlying cocaine's neurotoxic effects are not fully understood, partially due to technical limitations of current imaging techniques (e.g., PET, fMRI) to differentiate vascular from neuronal effects at sufficiently high temporal and spatial resolutions. We have recently developed a multimodal imaging platform which can simultaneously characterize the changes in cerebrovascular hemodynamics, hemoglobin oxygenation and intracellular calcium fluorescence for monitoring the effects of cocaine on the brain. Such a multimodality imaging technique (OFI) provides several uniquely important merits, including: 1) a large field-of-view, 2) high spatiotemporal resolutions, 3) quantitative 3D imaging of the cerebral blood flow (CBF) networks, 4) label-free imaging of hemodynamic changes, 5) separation of vascular compartments (e.g., arterial and venous vessels) and monitoring of cortical brain metabolic changes, 6) discrimination of cellular (neuronal) from vascular responses. These imaging features have been further advanced in combination with microprobes to form micro-OFI that allows quantification of drug effects on subcortical brain. In addition, our ultrahigh-resolution ODT (μODT) enables 3D microangiography and quantitative imaging of capillary CBF networks. These optical strategies have been used to investigate the effects of cocaine on brain physiology to facilitate the studies of brain functional changes induced by addictive substance to provide new insights into neurobiological effects of the drug on the brain.

Linked 4-Way Multimodal Brain Differences in Schizophrenia in a Large Chinese Han Population.

PubMed

Liu, Shengfeng; Wang, Haiying; Song, Ming; Lv, Luxian; Cui, Yue; Liu, Yong; Fan, Lingzhong; Zuo, Nianming; Xu, Kaibin; Du, Yuhui; Yu, Qingbao; Luo, Na; Qi, Shile; Yang, Jian; Xie, Sangma; Li, Jian; Chen, Jun; Chen, Yunchun; Wang, Huaning; Guo, Hua; Wan, Ping; Yang, Yongfeng; Li, Peng; Lu, Lin; Yan, Hao; Yan, Jun; Wang, Huiling; Zhang, Hongxing; Zhang, Dai; Calhoun, Vince D; Jiang, Tianzi; Sui, Jing

2018-04-20

Multimodal fusion has been regarded as a promising tool to discover covarying patterns of multiple imaging types impaired in brain diseases, such as schizophrenia (SZ). In this article, we aim to investigate the covarying abnormalities underlying SZ in a large Chinese Han population (307 SZs, 298 healthy controls [HCs]). Four types of magnetic resonance imaging (MRI) features, including regional homogeneity (ReHo) from resting-state functional MRI, gray matter volume (GM) from structural MRI, fractional anisotropy (FA) from diffusion MRI, and functional network connectivity (FNC) resulted from group independent component analysis, were jointly analyzed by a data-driven multivariate fusion method. Results suggest that a widely distributed network disruption appears in SZ patients, with synchronous changes in both functional and structural regions, especially the basal ganglia network, salience network (SAN), and the frontoparietal network. Such a multimodal coalteration was also replicated in another independent Chinese sample (40 SZs, 66 HCs). Our results on auditory verbal hallucination (AVH) also provide evidence for the hypothesis that prefrontal hypoactivation and temporal hyperactivation in SZ may lead to failure of executive control and inhibition, which is relevant to AVH. In addition, impaired working memory performance was found associated with GM reduction and FA decrease in SZ in prefrontal and superior temporal area, in both discovery and replication datasets. In summary, by leveraging multiple imaging and clinical information into one framework to observe brain in multiple views, we can integrate multiple inferences about SZ from large-scale population and offer unique perspectives regarding the missing links between the brain function and structure that may not be achieved by separate unimodal analyses.
A Remaking Pedagogy: Adaptation and Archetypes in the Child's Multimodal Reading and Writing

ERIC Educational Resources Information Center

Berger, Richard; Zezulkova, Marketa

2018-01-01

This paper proposes combining theories about, and practices of, using archetypes and adaptation in education for the purposes of multimodal literacy learning. Within such contexts, children of primary school age act as readers, performers and researchers, exploring and analysing existing adaptations of archetypal stories and images across time,…
The Future of Cardiac Imaging: Report of a Think Tank Convened by the American College of Cardiology.

PubMed

Douglas, Pamela S; Cerqueira, Manuel D; Berman, Daniel S; Chinnaiyan, Kavitha; Cohen, Meryl S; Lundbye, Justin B; Patel, Rajan A G; Sengupta, Partho P; Soman, Prem; Weissman, Neil J; Wong, Timothy C

2016-10-01

The American College of Cardiology's Executive Committee and Cardiovascular Imaging Section Leadership Council convened a discussion regarding the future of cardiac imaging among thought leaders in the field during a 2 day Think Tank. Participants were charged with thinking broadly about the future of imaging and developing a roadmap to address critical challenges. Key areas of discussion included: 1) how can cardiac imaging services thrive in our new world of value-based health care? 2) Who is the cardiac imager of the future and what is the role of the multimodality imager? 3) How can we nurture innovation and research in imaging? And 4) how can we maximize imaging information and optimize outcomes? This document describes the proceedings of this Think Tank. Copyright © 2016 American College of Cardiology Foundation. Published by Elsevier Inc. All rights reserved.
On-road anomaly detection by multimodal sensor analysis and multimedia processing

NASA Astrophysics Data System (ADS)

Orhan, Fatih; Eren, P. E.

2014-03-01

The use of smartphones in Intelligent Transportation Systems is gaining popularity, yet many challenges exist in developing functional applications. Due to the dynamic nature of transportation, vehicular social applications face complexities such as developing robust sensor management, performing signal and image processing tasks, and sharing information among users. This study utilizes a multimodal sensor analysis framework which enables the analysis of sensors in multimodal aspect. It also provides plugin-based analyzing interfaces to develop sensor and image processing based applications, and connects its users via a centralized application as well as to social networks to facilitate communication and socialization. With the usage of this framework, an on-road anomaly detector is being developed and tested. The detector utilizes the sensors of a mobile device and is able to identify anomalies such as hard brake, pothole crossing, and speed bump crossing. Upon such detection, the video portion containing the anomaly is automatically extracted in order to enable further image processing analysis. The detection results are shared on a central portal application for online traffic condition monitoring.
Non-responders to cardiac resynchronization therapy: Insights from multimodality imaging and electrocardiography. A brief review.

PubMed

Carità, Patrizia; Corrado, Egle; Pontone, Gianluca; Curnis, Antonio; Bontempi, Luca; Novo, Giuseppina; Guglielmo, Marco; Ciaramitaro, Gianfranco; Assennato, Pasquale; Novo, Salvatore; Coppola, Giuseppe

2016-12-15

Cardiac resynchronization therapy (CRT) is a successful strategy for heart failure (HF) patients. The pre-requisite for the response is the evidence of electrical dyssynchrony on the surface electrocardiogram usually as left bundle branch block (LBBB). Non-response to CRT is a significant problem in clinical practice. Patient selection, inadequate delivery and sub-optimal left ventricle lead position may be important causes. In an effort to improve CRT response multimodality imaging (especially echocardiography, computed tomography and cardiac magnetic resonance) could play a decisive role and extensive literature has been published on the matter. However, we are so far from routinary use in clinical practice. Electrocardiography (with respect to left ventricle capture and QRS narrowing) may represent a simple and low cost approach for early prediction of potential non-responder, with immediate practical implications. This brief review covers the current recommendations for CRT in HF patients with particular attention to the potential benefits of multimodality imaging and electrocardiography in improving response rate. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.
Application of a multicompartment dynamical model to multimodal optical imaging for investigating individual cerebrovascular properties

NASA Astrophysics Data System (ADS)

Desjardins, Michèle; Gagnon, Louis; Gauthier, Claudine; Hoge, Rick D.; Dehaes, Mathieu; Desjardins-Crépeau, Laurence; Bherer, Louis; Lesage, Frédéric

2009-02-01

Biophysical models of hemodynamics provide a tool for quantitative multimodal brain imaging by allowing a deeper understanding of the interplay between neural activity and blood oxygenation, volume and flow responses to stimuli. Multicompartment dynamical models that describe the dynamics and interactions of the vascular and metabolic components of evoked hemodynamic responses have been developed in the literature. In this work, multimodal data using near-infrared spectroscopy (NIRS) and diffuse correlation flowmetry (DCF) is used to estimate total baseline hemoglobin concentration (HBT0) in 7 adult subjects. A validation of the model estimate and investigation of the partial volume effect is done by comparing with time-resolved spectroscopy (TRS) measures of absolute HBT0. Simultaneous NIRS and DCF measurements during hypercapnia are then performed, but are found to be hardly reproducible. The results raise questions about the feasibility of an all-optical model-based estimation of individual vascular properties.
Iron oxide core oil-in-water emulsions as a multifunctional nanoparticle platform for tumor targeting and imaging

PubMed Central

Jarzyna, Peter A.; Skajaa, Torjus; Gianella, Anita; Cormode, David P.; Samber, Dan D.; Dickson, Stephen D.; Chen, Wei; Griffioen, Arjan W.; Fayad, Zahi A.; Mulder, Willem J. M.

2009-01-01

Nanoemulsions are increasingly investigated for the delivery of hydrophobic drugs to improve their bioavailability or make their administration possible. In the current study, oil-in-water emulsions with three different mean diameters (30, 60, and 95 nm) were developed as a new multimodality nanoparticle platform for tumor targeting and imaging. To that aim, hydrophobically coated iron oxide particles were included in the soybean oil core of the nanoemulsions to enable their detection with magnetic resonance imaging (MRI), while the conjugation of a near infrared fluorophore allowed optical imaging. The accumulation of this novel nanocomposite in subcutaneous human tumors in nude mice was demonstrated with MRI and fluorescence imaging in vivo, and with Perl’s staining of histological tumor sections ex vivo. PMID:19783295
Grid-Enabled Quantitative Analysis of Breast Cancer

DTIC Science & Technology

2010-10-01

large-scale, multi-modality computerized image analysis . The central hypothesis of this research is that large-scale image analysis for breast cancer...research, we designed a pilot study utilizing large scale parallel Grid computing harnessing nationwide infrastructure for medical image analysis . Also
Imaging the pleura.

PubMed

Mortensen, Chloe; Bhatnagar, Rahul; Edey, Anthony J

2012-11-01

Pleural disease is now recognized as an important subspecialty of pulmonary medicine, with increasing provision being made for specialist services and procedures. In response, the field of pleural imaging has advanced in recent years, especially with regard to ultrasound. Salient multimodality imaging techniques are discussed.
1024-Pixel CMOS Multimodality Joint Cellular Sensor/Stimulator Array for Real-Time Holistic Cellular Characterization and Cell-Based Drug Screening.

PubMed

Park, Jong Seok; Aziz, Moez Karim; Li, Sensen; Chi, Taiyun; Grijalva, Sandra Ivonne; Sung, Jung Hoon; Cho, Hee Cheol; Wang, Hua

2018-02-01

This paper presents a fully integrated CMOS multimodality joint sensor/stimulator array with 1024 pixels for real-time holistic cellular characterization and drug screening. The proposed system consists of four pixel groups and four parallel signal-conditioning blocks. Every pixel group contains 16 × 16 pixels, and each pixel includes one gold-plated electrode, four photodiodes, and in-pixel circuits, within a pixel footprint. Each pixel supports real-time extracellular potential recording, optical detection, charge-balanced biphasic current stimulation, and cellular impedance measurement for the same cellular sample. The proposed system is fabricated in a standard 130-nm CMOS process. Rat cardiomyocytes are successfully cultured on-chip. Measured high-resolution optical opacity images, extracellular potential recordings, biphasic current stimulations, and cellular impedance images demonstrate the unique advantages of the system for holistic cell characterization and drug screening. Furthermore, this paper demonstrates the use of optical detection on the on-chip cultured cardiomyocytes to real-time track their cyclic beating pattern and beating rate.
Simulation of brain tumors in MR images for evaluation of segmentation efficacy.

PubMed

Prastawa, Marcel; Bullitt, Elizabeth; Gerig, Guido

2009-04-01

Obtaining validation data and comparison metrics for segmentation of magnetic resonance images (MRI) are difficult tasks due to the lack of reliable ground truth. This problem is even more evident for images presenting pathology, which can both alter tissue appearance through infiltration and cause geometric distortions. Systems for generating synthetic images with user-defined degradation by noise and intensity inhomogeneity offer the possibility for testing and comparison of segmentation methods. Such systems do not yet offer simulation of sufficiently realistic looking pathology. This paper presents a system that combines physical and statistical modeling to generate synthetic multi-modal 3D brain MRI with tumor and edema, along with the underlying anatomical ground truth, Main emphasis is placed on simulation of the major effects known for tumor MRI, such as contrast enhancement, local distortion of healthy tissue, infiltrating edema adjacent to tumors, destruction and deformation of fiber tracts, and multi-modal MRI contrast of healthy tissue and pathology. The new method synthesizes pathology in multi-modal MRI and diffusion tensor imaging (DTI) by simulating mass effect, warping and destruction of white matter fibers, and infiltration of brain tissues by tumor cells. We generate synthetic contrast enhanced MR images by simulating the accumulation of contrast agent within the brain. The appearance of the the brain tissue and tumor in MRI is simulated by synthesizing texture images from real MR images. The proposed method is able to generate synthetic ground truth and synthesized MR images with tumor and edema that exhibit comparable segmentation challenges to real tumor MRI. Such image data sets will find use in segmentation reliability studies, comparison and validation of different segmentation methods, training and teaching, or even in evaluating standards for tumor size like the RECIST criteria (response evaluation criteria in solid tumors).
Multimodal confocal mosaics enable high sensitivity and specificity in screening of in situ squamous cell carcinoma

NASA Astrophysics Data System (ADS)

Grados Luyando, Maria del Carmen; Bar, Anna; Snavely, Nicholas; Jacques, Steven; Gareau, Daniel S.

2014-02-01

Screening cancer in excision margins with confocal microscopy may potentially save time and cost over the gold standard histopathology (H and E). However, diagnostic accuracy requires sufficient contrast and resolution to reveal pathological traits in a growing set of tumor types. Reflectance mode images structural details due to microscopic refractive index variation. Nuclear contrast with acridine orange fluorescence provides enhanced diagnostic value, but fails for in situ squamous cell carcinoma (SCC), where the cytoplasm is important to visualize. Combination of three modes [eosin (Eo) fluorescence, reflectance (R) and acridine orange (AO) fluorescence] enable imaging of cytoplasm, collagen and nuclei respectively. Toward rapid intra-operative pathological margin assessment to guide staged cancer excisions, multimodal confocal mosaics can image wide surgical margins (~1cm) with sub-cellular resolution and mimic the appearance of conventional H and E. Absorption contrast is achieved by alternating the excitation wavelength: 488nm (AO fluorescence) and 532nm (Eo fluorescence). Superposition and false-coloring of these modes mimics H and E, enabling detection of the carcinoma in situ in the epidermal layer The sum mosaic Eo+R is false-colored pink to mimic eosins' appearance in H and E, while the AO mosaic is false-colored purple to mimic hematoxylins' appearance in H and E. In this study, mosaics of 10 Mohs surgical excisions containing SCC in situ and 5 containing only normal tissue were subdivided for digital presentation equivalent to 4X histology. Of the total 16 SCC in situ multimodal mosaics and 16 normal cases presented, two reviewers made 1 and 2 (respectively) type-2 errors (false positives) but otherwise scored perfectly when using the confocal images to screen for the presence of SCC in situ as compared to the gold standard histopathology. Limitations to precisely mimic H and E included occasional elastin staining by AO. These results suggest that confocal mosaics may effectively guide staged SCC excisions in skin and other tissues.
Shedding Light on the Molecular Pathology of Amyloid Plaques in Transgenic Alzheimer's Disease Mice Using Multimodal MALDI Imaging Mass Spectrometry.

PubMed

Kaya, Ibrahim; Zetterberg, Henrik; Blennow, Kaj; Hanrieder, Jörg

2018-05-04

Senile plaques formed by aggregated amyloid β peptides are one of the major pathological hallmarks of Alzheimer's disease (AD) which have been suggested to be the primary influence triggering the AD pathogenesis and the rest of the disease process. However, neurotoxic Aβ aggregation and progression are associated with a wide range of enigmatic biochemical, biophysical and genetic processes. MALDI imaging mass spectrometry (IMS) is a label-free method to elucidate the spatial distribution patterns of intact molecules in biological tissue sections. In this communication, we utilized multimodal MALDI-IMS analysis on 18 month old transgenic AD mice (tgArcSwe) brain tissue sections to enhance molecular information correlated to individual amyloid aggregates on the very same tissue section. Dual polarity MALDI-IMS analysis of lipids on the same pixel points revealed high throughput lipid molecular information including sphingolipids, phospholipids, and lysophospholipids which can be correlated to the ion images of individual amyloid β peptide isoforms at high spatial resolutions (10 μm). Further, multivariate image analysis was applied in order to probe the multimodal MALDI-IMS data in an unbiased way which verified the correlative accumulations of lipid species with dual polarity and Aβ peptides. This was followed by the lipid fragmentation obtained directly on plaque aggregates at higher laser pulse energies which provided tandem MS information useful for structural elucidation of several lipid species. Majority of the amyloid plaque-associated alterations of lipid species are for the first time reported here. The significance of this technique is that it allows correlating the biological discussion of all detected plaque-associated molecules to the very same individual amyloid plaques which can give novel insights into the molecular pathology of even a single amyloid plaque microenvironment in a specific brain region. Therefore, this allowed us to interpret the possible roles of lipids and amyloid peptides in amyloid plaque-associated pathological events such as focal demyelination, autophagic/lysosomal dysfunction, astrogliosis, inflammation, oxidative stress, and cell death.
The multi-modal Australian ScienceS Imaging and Visualization Environment (MASSIVE) high performance computing infrastructure: applications in neuroscience and neuroinformatics research

PubMed Central

Goscinski, Wojtek J.; McIntosh, Paul; Felzmann, Ulrich; Maksimenko, Anton; Hall, Christopher J.; Gureyev, Timur; Thompson, Darren; Janke, Andrew; Galloway, Graham; Killeen, Neil E. B.; Raniga, Parnesh; Kaluza, Owen; Ng, Amanda; Poudel, Govinda; Barnes, David G.; Nguyen, Toan; Bonnington, Paul; Egan, Gary F.

2014-01-01

The Multi-modal Australian ScienceS Imaging and Visualization Environment (MASSIVE) is a national imaging and visualization facility established by Monash University, the Australian Synchrotron, the Commonwealth Scientific Industrial Research Organization (CSIRO), and the Victorian Partnership for Advanced Computing (VPAC), with funding from the National Computational Infrastructure and the Victorian Government. The MASSIVE facility provides hardware, software, and expertise to drive research in the biomedical sciences, particularly advanced brain imaging research using synchrotron x-ray and infrared imaging, functional and structural magnetic resonance imaging (MRI), x-ray computer tomography (CT), electron microscopy and optical microscopy. The development of MASSIVE has been based on best practice in system integration methodologies, frameworks, and architectures. The facility has: (i) integrated multiple different neuroimaging analysis software components, (ii) enabled cross-platform and cross-modality integration of neuroinformatics tools, and (iii) brought together neuroimaging databases and analysis workflows. MASSIVE is now operational as a nationally distributed and integrated facility for neuroinfomatics and brain imaging research. PMID:24734019
Improvement in cognitive and psychosocial functioning and self image among adolescent inpatient suicide attempters.

PubMed

Hintikka, Ulla; Marttunen, Mauri; Pelkonen, Mirjami; Laukkanen, Eila; Viinamäki, Heimo; Lehtonen, Johannes

2006-12-29

Psychiatric treatment of suicidal youths is often difficult and non-compliance in treatment is a significant problem. This prospective study compared characteristics and changes in cognitive functioning, self image and psychosocial functioning among 13 to 18 year-old adolescent psychiatric inpatients with suicide attempts (n = 16) and with no suicidality (n = 39) The two-group pre-post test prospective study design included assessments by a psychiatrist, a psychologist and medical staff members as well as self-rated measures. DSM-III-R diagnoses were assigned using the SCID and thereafter transformed to DSM-IV diagnoses. Staff members assessed psychosocial functioning using the Global Assessment Scale (GAS). Cognitive performance was assessed using the Wechsler Adult Intelligence Scale, while the Offer Self-Image Questionnaire (OSIQ) was used to assess the subjects' self-image. ANCOVA with repeated measures was used to test changes from entry to discharge among the suicide attempters and non suicidal patients. Logistic regression modeling was used to assess variables associated with an improvement of 10 points or more in the GAS score. Among suicide attempter patients, psychosocial functioning, cognitive performance and both the psychological self and body-image improved during treatment and their treatment compliance and outcome were as good as that of the non-suicidal patients. Suicidal ideation and hopelessness declined, and psychosocial functioning improved. Changes in verbal cognitive performance were more pronounced among the suicide attempters. Having an improved body-image associated with a higher probability of improvement in psychosocial functioning while higher GAS score at entry was associated with lower probability of functional improvement in both patient groups. These findings illustrate that a multimodal treatment program seems to improve psychosocial functioning and self-image among severely disordered suicidal adolescent inpatients. There were no changes in familial relationships, possibly indicating a need for more intensive family interventions when treating suicidal youths. Multimodal inpatient treatment including an individual therapeutic relationship seems recommendable for severely impaired psychiatric inpatients tailored to the suicidal adolescent's needs.
Red fluorescent zinc oxide nanoparticle: A novel platform for cancer targeting

DOE PAGES

Hong, Hao; Wang, Fei; Zhang, Yin; ...

2015-01-21

Multifunctional zinc oxide (ZnO) nanoparticles (NPs) with well-integrated multimodality imaging capacities have generated increasing research interest in the past decade. However, limited progress has been made in developing ZnO NP-based multimodality tumor-imaging agents. In this paper, we developed novel red fluorescent ZnO NPs and described the successful conjugation of 64Cu ( t 1/2 = 12.7 h) and TRC105, a chimeric monoclonal antibody against CD105, to these ZnO NPs via well-developed surface engineering procedures. The produced dual-modality ZnO NPs were readily applicable for positron emission tomography (PET) imaging and fluorescence imaging of the tumor vasculature. Their pharmacokinetics and tumor-targeting efficacy/specificity inmore » mice bearing murine breast 4T1 tumor were thoroughly investigated. In conclusion, ZnO NPs with dual-modality imaging properties can serve as an attractive candidate for future cancer theranostics.« less
The Interactive Origin and the Aesthetic Modelling of Image-Schemas and Primary Metaphors.

PubMed

Martínez, Isabel C; Español, Silvia A; Pérez, Diana I

2018-06-02

According to the theory of conceptual metaphor, image-schemas and primary metaphors are preconceptual structures configured in human cognition, based on sensory-motor environmental activity. Focusing on the way both non-conceptual structures are embedded in early social interaction, we provide empirical evidence for the interactive and intersubjective ontogenesis of image-schemas and primary metaphors. We present the results of a multimodal image-schematic microanalysis of three interactive infant-directed performances (the composition of movement, touch, speech, and vocalization that adults produce for-and-with the infants). The microanalyses show that adults aesthetically highlight the image-schematic structures embedded in the multimodal composition of the performance, and that primary metaphors are also lived as embedded in these inter-enactive experiences. The findings allow corroborating that the psychological domains of cognition and affection are not in rivalry or conflict but rather intertwined in meaningful experiences.
Assessment of fibrotic liver disease with multimodal nonlinear optical microscopy

NASA Astrophysics Data System (ADS)

Lu, Fake; Zheng, Wei; Tai, Dean C. S.; Lin, Jian; Yu, Hanry; Huang, Zhiwei

2010-02-01

Liver fibrosis is the excessive accumulation of extracellular matrix proteins such as collagens, which may result in cirrhosis, liver failure, and portal hypertension. In this study, we apply a multimodal nonlinear optical microscopy platform developed to investigate the fibrotic liver diseases in rat models established by performing bile duct ligation (BDL) surgery. The three nonlinear microscopy imaging modalities are implemented on the same sectioned tissues of diseased model sequentially: i.e., second harmonic generation (SHG) imaging quantifies the contents of the collagens, the two-photon excitation fluorescence (TPEF) imaging reveals the morphology of hepatic cells, while coherent anti-Stokes Raman scattering (CARS) imaging maps the distributions of fats or lipids quantitatively across the tissue. Our imaging results show that during the development of liver fibrosis (collagens) in BDL model, fatty liver disease also occurs. The aggregated concentrations of collagen and fat constituents in liver fibrosis model show a certain correlationship between each other.
A spline-based non-linear diffeomorphism for multimodal prostate registration.

PubMed

Mitra, Jhimli; Kato, Zoltan; Martí, Robert; Oliver, Arnau; Lladó, Xavier; Sidibé, Désiré; Ghose, Soumya; Vilanova, Joan C; Comet, Josep; Meriaudeau, Fabrice

2012-08-01

This paper presents a novel method for non-rigid registration of transrectal ultrasound and magnetic resonance prostate images based on a non-linear regularized framework of point correspondences obtained from a statistical measure of shape-contexts. The segmented prostate shapes are represented by shape-contexts and the Bhattacharyya distance between the shape representations is used to find the point correspondences between the 2D fixed and moving images. The registration method involves parametric estimation of the non-linear diffeomorphism between the multimodal images and has its basis in solving a set of non-linear equations of thin-plate splines. The solution is obtained as the least-squares solution of an over-determined system of non-linear equations constructed by integrating a set of non-linear functions over the fixed and moving images. However, this may not result in clinically acceptable transformations of the anatomical targets. Therefore, the regularized bending energy of the thin-plate splines along with the localization error of established correspondences should be included in the system of equations. The registration accuracies of the proposed method are evaluated in 20 pairs of prostate mid-gland ultrasound and magnetic resonance images. The results obtained in terms of Dice similarity coefficient show an average of 0.980±0.004, average 95% Hausdorff distance of 1.63±0.48 mm and mean target registration and target localization errors of 1.60±1.17 mm and 0.15±0.12 mm respectively. Copyright © 2012 Elsevier B.V. All rights reserved.
Designing Tripodal and Triangular Gadolinium Oxide Nanoplates and Self-Assembled Nanofibrils as Potential Multimodal Bioimaging Probes

DOE Office of Scientific and Technical Information (OSTI.GOV)

Paik, T; Gordon, TR; Prantner, AM

2013-03-01

Here, we report the shape-controlled synthesis of tripodal and triangular gadolinium oxide (Gd2O3) nanoplates. In the presence of lithium ions, the shape of the nanocrystals is readily controlled by tailoring reaction parameters such as temperature and time. We observe that the morphology transforms from an initial tripodal shape to a triangular shape with increasing reaction time or elevated temperatures. Highly uniform Gd2O3 nanoplates are self-assembled into nanofibril-like liquid-crystalline superlattices with long-range orientational and positional order. In addition, shape-directed self-assemblies are investigated by tailoring the aspect ratio of the arms of the Gd2O3 nanoplates. Due to a strong paramagnetic response, Gd2O3more » nanocrystals are excellent candidates for MRI contrast agents and also can be doped with rare-earth ions to form nanophosphors, pointing to their potential in multimodal imaging. In this work, we investigate the MR relaxometry at high magnetic fields (9,4 and 14.1 T) and the optical properties including near-IR to visible upconversion luminescence and X-ray excited optical luminescence of doped Gd2O3 nanoplates. The complex shape of Gd2O3 nanoplates, coupled with their magnetic properties and their ability to phosphoresce under NIR or X-ray excitation which penetrate deep into tissue, makes these nanoplates a promising platform for multimodal imaging in biomedical applications.« less

Quantitative reconstructions in multi-modal photoacoustic and optical coherence tomography imaging

NASA Astrophysics Data System (ADS)

Elbau, P.; Mindrinos, L.; Scherzer, O.

2018-01-01

In this paper we perform quantitative reconstruction of the electric susceptibility and the Grüneisen parameter of a non-magnetic linear dielectric medium using measurement of a multi-modal photoacoustic and optical coherence tomography system. We consider the mathematical model presented in Elbau et al (2015 Handbook of Mathematical Methods in Imaging ed O Scherzer (New York: Springer) pp 1169-204), where a Fredholm integral equation of the first kind for the Grüneisen parameter was derived. For the numerical solution of the integral equation we consider a Galerkin type method.
Detection of suspicious pain regions on a digital infrared thermal image using the multimodal function optimization.

PubMed

Lee, Junghoon; Lee, Joosung; Song, Sangha; Lee, Hyunsook; Lee, Kyoungjoung; Yoon, Youngro

2008-01-01

Automatic detection of suspicious pain regions is very useful in the medical digital infrared thermal imaging research area. To detect those regions, we use the SOFES (Survival Of the Fitness kind of the Evolution Strategy) algorithm which is one of the multimodal function optimization methods. We apply this algorithm to famous diseases, such as a foot of the glycosuria, the degenerative arthritis and the varicose vein. The SOFES algorithm is available to detect some hot spots or warm lines as veins. And according to a hundred of trials, the algorithm is very fast to converge.
Multiscale and multi-modality visualization of angiogenesis in a human breast cancer model

PubMed Central

Cebulla, Jana; Kim, Eugene; Rhie, Kevin; Zhang, Jiangyang

2017-01-01

Angiogenesis in breast cancer helps fulfill the metabolic demands of the progressing tumor and plays a critical role in tumor metastasis. Therefore, various imaging modalities have been used to characterize tumor angiogenesis. While micro-CT (μCT) is a powerful tool for analyzing the tumor microvascular architecture at micron-scale resolution, magnetic resonance imaging (MRI) with its sub-millimeter resolution is useful for obtaining in vivo vascular data (e.g. tumor blood volume and vessel size index). However, integration of these microscopic and macroscopic angiogenesis data across spatial resolutions remains challenging. Here we demonstrate the feasibility of ‘multiscale’ angiogenesis imaging in a human breast cancer model, wherein we bridge the resolution gap between ex vivo μCT and in vivo MRI using intermediate resolution ex vivo MR microscopy (μMRI). To achieve this integration, we developed suitable vessel segmentation techniques for the ex vivo imaging data and co-registered the vascular data from all three imaging modalities. We showcase two applications of this multiscale, multi-modality imaging approach: (1) creation of co-registered maps of vascular volume from three independent imaging modalities, and (2) visualization of differences in tumor vasculature between viable and necrotic tumor regions by integrating μCT vascular data with tumor cellularity data obtained using diffusion-weighted MRI. Collectively, these results demonstrate the utility of ‘mesoscopic’ resolution μMRI for integrating macroscopic in vivo MRI data and microscopic μCT data. Although focused on the breast tumor xenograft vasculature, our imaging platform could be extended to include additional data types for a detailed characterization of the tumor microenvironment and computational systems biology applications. PMID:24719185
Association between structural and functional brain alterations in drug-free patients with schizophrenia: a multimodal meta-analysis

PubMed

Gao, Xin; Zhang, Wenjing; Yao, Li; Xiao, Yuan; Liu, Lu; Liu, Jieke; Li, Siyi; Tao, Bo; Shah, Chandan; Gong, Qiyong; Sweeney, John; Lui, Su

2017-12-05

Neuroimaging studies have shown both structural and functional abnormalities in patients with schizophrenia. Recently, studies have begun to explore the association between structural and functional grey matter abnormalities. By conducting a meta-analysis on morphometric and functional imaging studies of grey matter alterations in drug-free patients, the present study aims to examine the degree of overlap between brain regions with anatomic and functional changes in patients with schizophrenia. We performed a systematic search of PubMed, Embase, Web of Science and the Cochrane Library to identify relevant publications. A multimodal analysis was then conducted using Seed-based d Mapping software. Exploratory analyses included jackknife, subgroup and meta-regression analyses. We included 15 structural MRI studies comprising 486 drug-free patients and 485 healthy controls, and 16 functional MRI studies comprising 403 drug-free patients and 428 controls in our meta-analysis. Drug-free patients were examined to reduce pharmacological effects on the imaging data. Multimodal analysis showed considerable overlap between anatomic and functional changes, mainly in frontotemporal regions, bilateral medial posterior cingulate/paracingulate gyrus, bilateral insula, basal ganglia and left cerebellum. There were also brain regions showing only anatomic changes in the right superior frontal gyrus, left supramarginal gyrus, right lingual gyrus and functional alternations involving the right angular gyrus. The methodological aspects, patient characteristics and clinical variables of the included studies were heterogeneous, and we cannot exclude medication effects. The present study showed overlapping anatomic and functional brain abnormalities mainly in the default mode (DMN) and auditory networks (AN) in drug-free patients with schizophrenia. However, the pattern of changes differed in these networks. Decreased grey matter was associated with decreased activation within the DMN, whereas it was associated with increased activation within the AN. These discrete patterns suggest different pathophysiological changes impacting structural and functional associations within different neural networks in patients with schizophrenia. 2017 Joule Inc., or its licensors
Association between structural and functional brain alterations in drug-free patients with schizophrenia: a multimodal meta-analysis.

PubMed

Gao, Xin; Zhang, Wenjing; Yao, Li; Xiao, Yuan; Liu, Lu; Liu, Jieke; Li, Siyi; Tao, Bo; Shah, Chandan; Gong, Qiyong; Sweeney, John A; Lui, Su

2018-03-01

Neuroimaging studies have shown both structural and functional abnormalities in patients with schizophrenia. Recently, studies have begun to explore the association between structural and functional grey matter abnormalities. By conducting a meta-analysis on morphometric and functional imaging studies of grey matter alterations in drug-free patients, the present study aims to examine the degree of overlap between brain regions with anatomic and functional changes in patients with schizophrenia. We performed a systematic search of PubMed, Embase, Web of Science and the Cochrane Library to identify relevant publications. A multimodal analysis was then conducted using Seed-based d Mapping software. Exploratory analyses included jackknife, subgroup and meta-regression analyses. We included 15 structural MRI studies comprising 486 drug-free patients and 485 healthy controls, and 16 functional MRI studies comprising 403 drug-free patients and 428 controls in our meta-analysis. Drug-free patients were examined to reduce pharmacological effects on the imaging data. Multimodal analysis showed considerable overlap between anatomic and functional changes, mainly in frontotemporal regions, bilateral medial posterior cingulate/paracingulate gyrus, bilateral insula, basal ganglia and left cerebellum. There were also brain regions showing only anatomic changes in the right superior frontal gyrus, left supramarginal gyrus, right lingual gyrus and functional alternations involving the right angular gyrus. The methodological aspects, patient characteristics and clinical variables of the included studies were heterogeneous, and we cannot exclude medication effects. The present study showed overlapping anatomic and functional brain abnormalities mainly in the default mode (DMN) and auditory networks (AN) in drug-free patients with schizophrenia. However, the pattern of changes differed in these networks. Decreased grey matter was associated with decreased activation within the DMN, whereas it was associated with increased activation within the AN. These discrete patterns suggest different pathophysiological changes impacting structural and functional associations within different neural networks in patients with schizophrenia.
Association between structural and functional brain alterations in drug-free patients with schizophrenia: a multimodal meta-analysis.

PubMed

Gao, Xin; Zhang, Wenjing; Yao, Li; Xiao, Yuan; Liu, Lu; Liu, Jieke; Li, Siyi; Tao, Bo; Shah, Chandan; Gong, Qiyong; Sweeney, John A; Lui, Su

2017-12-15

Neuroimaging studies have shown both structural and functional abnormalities in patients with schizophrenia. Recently, studies have begun to explore the association between structural and functional grey matter abnormalities. By conducting a meta-analysis on morphometric and functional imaging studies of grey matter alterations in drug-free patients, the present study aims to examine the degree of overlap between brain regions with anatomic and functional changes in patients with schizophrenia. We performed a systematic search of PubMed, Embase, Web of Science and the Cochrane Library to identify relevant publications. A multimodal analysis was then conducted using Seed-based d Mapping software. Exploratory analyses included jackknife, subgroup and meta-regression analyses. We included 15 structural MRI studies comprising 486 drug-free patients and 485 healthy controls, and 16 functional MRI studies comprising 403 drug-free patients and 428 controls in our meta-analysis. Drug-free patients were examined to reduce pharmacological effects on the imaging data. Multimodal analysis showed considerable overlap between anatomic and functional changes, mainly in frontotemporal regions, bilateral medial posterior cingulate/paracingulate gyrus, bilateral insula, basal ganglia and left cerebellum. There were also brain regions showing only anatomic changes in the right superior frontal gyrus, left supramarginal gyrus, right lingual gyrus and functional alternations involving the right angular gyrus. The methodological aspects, patient characteristics and clinical variables of the included studies were heterogeneous, and we cannot exclude medication effects. The present study showed overlapping anatomic and functional brain abnormalities mainly in the default mode (DMN) and auditory networks (AN) in drug-free patients with schizophrenia. However, the pattern of changes differed in these networks. Decreased grey matter was associated with decreased activation within the DMN, whereas it was associated with increased activation within the AN. These discrete patterns suggest different pathophysiological changes impacting structural and functional associations within different neural networks in patients with schizophrenia.
Clinically Approved Nanoparticle Imaging Agents

PubMed Central

Thakor, Avnesh S.; Jokerst, Jesse V.; Ghanouni, Pejman; Campbell, Jos L.; Mittra, Erik

2016-01-01

Nanoparticles are a new class of imaging agent used for both anatomic and molecular imaging. Nanoparticle-based imaging exploits the signal intensity, stability, and biodistribution behavior of submicron-diameter molecular imaging agents. This review focuses on nanoparticles used in human medical imaging, with an emphasis on radionuclide imaging and MRI. Newer nanoparticle platforms are also discussed in relation to theranostic and multimodal uses. PMID:27738007
Transferring biomarker into molecular probe: Melanin nanoparticle as a naturally active platform for multimodality imaging

DOE PAGES

Fan, Quli; Cheng, Kai; Hu, Xiang; ...

2014-10-07

Developing multifunctional and easily prepared nanoplatforms with integrated different modalities is highly challenging for molecular imaging. Here, we report the successful transfer of an important molecular target, melanin, into a novel multimodality imaging nanoplatform. Melanin is abundantly expressed in melanotic melanomas and thus has been actively studied as a target for melanoma imaging. In our work, the multifunctional biopolymer nanoplatform based on ultrasmall (<10 nm) water-soluble melanin nanoparticle (MNP) was developed and showed unique photoacoustic property and natural binding ability with metal ions (for example, 64Cu 2+, Fe 3+). Therefore, MNP can serve not only as a photoacoustic contrast agent,more » but also as a nanoplatform for positron emission tomography (PET) and magnetic resonance imaging (MRI). Traditional passive nanoplatforms require complicated and time-consuming processes for prebuilding reporting moieties or chemical modifications using active groups to integrate different contrast properties into one entity. In comparison, utilizing functional biomarker melanin can greatly simplify the building process. We further conjugated α vβ 3 integrins, cyclic c(RGDfC) peptide, to MNPs to allow for U87MG tumor accumulation due to its targeting property combined with the enhanced permeability and retention (EPR) effect. As a result, the multimodal properties of MNPs demonstrate the high potential of endogenous materials with multifunctions as nanoplatforms for molecular theranostics and clinical translation.« less
Observation of Geometric Parametric Instability Induced by the Periodic Spatial Self-Imaging of Multimode Waves

NASA Astrophysics Data System (ADS)

Krupa, Katarzyna; Tonello, Alessandro; Barthélémy, Alain; Couderc, Vincent; Shalaby, Badr Mohamed; Bendahmane, Abdelkrim; Millot, Guy; Wabnitz, Stefan

2016-05-01

Spatiotemporal mode coupling in highly multimode physical systems permits new routes for exploring complex instabilities and forming coherent wave structures. We present here the first experimental demonstration of multiple geometric parametric instability sidebands, generated in the frequency domain through resonant space-time coupling, owing to the natural periodic spatial self-imaging of a multimode quasi-continuous-wave beam in a standard graded-index multimode fiber. The input beam was launched in the fiber by means of an amplified microchip laser emitting sub-ns pulses at 1064 nm. The experimentally observed frequency spacing among sidebands agrees well with analytical predictions and numerical simulations. The first-order peaks are located at the considerably large detuning of 123.5 THz from the pump. These results open the remarkable possibility to convert a near-infrared laser directly into a broad spectral range spanning visible and infrared wavelengths, by means of a single resonant parametric nonlinear effect occurring in the normal dispersion regime. As further evidence of our strong space-time coupling regime, we observed the striking effect that all of the different sideband peaks were carried by a well-defined and stable bell-shaped spatial profile.
[Isolated left ventricular non-compaction associated with Ebstein's anomaly. Multimodality non-invasive imaging for the assessment of congenital heart disease].

PubMed

Renilla, Alfredo; Santamarta, Elena; Corros, Cecilia; Martín, María; Barreiro, Manuel; de la Hera, Jesús

2013-01-01

To establish the etiology of heart failure in patients with congenital heart disease can be challenging. Multiple concomitant anomalies that can be missed after an initial diagnosis could be seen in these patients. In patients with congenital heart disease, a more accurate evaluation of cardiac morphology and left ventricular systolic function could be evaluated by recent non-invasive cardiac imaging techniques. We present a rare case where multimodal cardiac imaging was useful to establish the final diagnosis of left ventricular non-compaction associated with Ebstein's anomaly. Copyright © 2012 Instituto Nacional de Cardiología Ignacio Chávez. Published by Masson Doyma México S.A. All rights reserved.
AMIDE: a free software tool for multimodality medical image analysis.

PubMed

Loening, Andreas Markus; Gambhir, Sanjiv Sam

2003-07-01

Amide's a Medical Image Data Examiner (AMIDE) has been developed as a user-friendly, open-source software tool for displaying and analyzing multimodality volumetric medical images. Central to the package's abilities to simultaneously display multiple data sets (e.g., PET, CT, MRI) and regions of interest is the on-demand data reslicing implemented within the program. Data sets can be freely shifted, rotated, viewed, and analyzed with the program automatically handling interpolation as needed from the original data. Validation has been performed by comparing the output of AMIDE with that of several existing software packages. AMIDE runs on UNIX, Macintosh OS X, and Microsoft Windows platforms, and it is freely available with source code under the terms of the GNU General Public License.
Multimodal and synthetic aperture approach to full-field 3D shape and displacement measurements

NASA Astrophysics Data System (ADS)

Kujawińska, M.; Sitnik, R.

2017-08-01

Recently most of the measurement tasks in industry, civil engineering and culture heritage applications require archiving, characterization and monitoring of 3D objects and structures and their performance under changing conditions. These requirements can be met if multimodal measurement (MM) strategy is applied. It rely on effective combining structured light method and 3D digital image correlation with laser scanning/ToF, thermal imaging, multispectral imaging and BDRF measurements. In the case of big size and/or complicated objects MM have to be combined with hierarchical or synthetic aperture (SA) measurements. The new solutions in MM and SA strategies are presented and their applicability is shown at interesting cultural heritage and civil engineering applications.
Visual tracking for multi-modality computer-assisted image guidance

NASA Astrophysics Data System (ADS)

Basafa, Ehsan; Foroughi, Pezhman; Hossbach, Martin; Bhanushali, Jasmine; Stolka, Philipp

2017-03-01

With optical cameras, many interventional navigation tasks previously relying on EM, optical, or mechanical guidance can be performed robustly, quickly, and conveniently. We developed a family of novel guidance systems based on wide-spectrum cameras and vision algorithms for real-time tracking of interventional instruments and multi-modality markers. These navigation systems support the localization of anatomical targets, support placement of imaging probe and instruments, and provide fusion imaging. The unique architecture - low-cost, miniature, in-hand stereo vision cameras fitted directly to imaging probes - allows for an intuitive workflow that fits a wide variety of specialties such as anesthesiology, interventional radiology, interventional oncology, emergency medicine, urology, and others, many of which see increasing pressure to utilize medical imaging and especially ultrasound, but have yet to develop the requisite skills for reliable success. We developed a modular system, consisting of hardware (the Optical Head containing the mini cameras) and software (components for visual instrument tracking with or without specialized visual features, fully automated marker segmentation from a variety of 3D imaging modalities, visual observation of meshes of widely separated markers, instant automatic registration, and target tracking and guidance on real-time multi-modality fusion views). From these components, we implemented a family of distinct clinical and pre-clinical systems (for combinations of ultrasound, CT, CBCT, and MRI), most of which have international regulatory clearance for clinical use. We present technical and clinical results on phantoms, ex- and in-vivo animals, and patients.
Delivery of ultrashort spatially focused pulses through a multimode fiber

NASA Astrophysics Data System (ADS)

Morales-Delgado, Edgar E.; Papadopoulos, Ioannis N.; Farahi, Salma; Psaltis, Demetri; Moser, Christophe

2015-08-01

Multimode optical fibers potentially allow the transmission of larger amounts of information than their single mode counterparts because of their high number of supported modes. However, propagation of a light pulse through a multimode fiber suffers from spatial distortions due to the superposition of the various exited modes and from time broadening due to modal dispersion. We present a method based on digital phase conjugation to selectively excite in a multimode fiber specific optical fiber modes that follow similar optical paths as they travel through the fiber. The excited modes interfere constructively at the fiber output generating an ultrashort spatially focused pulse. The excitation of a limited number of modes following similar optical paths limits modal dispersion, allowing the transmission of the ultrashort pulse. We have experimentally demonstrated the delivery of a focused spot of pulse width equal to 500 fs through a 30 cm, 200 micrometer core step index multimode fiber. The results of this study show that two-photon imaging capability can be added to ultra-thin lensless endoscopy using commercial multimode fibers.
Delivery of an ultrashort spatially focused pulse to the other end of a multimode fiber using digital phase conjugation

NASA Astrophysics Data System (ADS)

Morales Delgado, Edgar E.; Papadopoulos, Ioannis N.; Farahi, Salma; Psaltis, Demetri; Moser, Christophe

2015-03-01

Multimode optical fibers potentially allow the transmission of larger amounts of information than their single mode counterparts because of their high number of supported modes. However, propagation of a light pulse through a multimode fiber suffers from spatial distortions due to the superposition of the various exited modes and from time broadening due to modal dispersion. We present a method based on digital phase conjugation to selectively excite in a multimode fiber specific optical fiber modes that follow similar optical paths as they travel through the fiber. The excited modes interfere constructively at the fiber output generating an ultrashort spatially focused pulse. The excitation of a limited number of modes following similar optical paths limits modal dispersion, allowing the transmission of the ultrashort pulse. We have experimentally demonstrated the delivery of a focused spot of pulse width equal to 500 fs through a 30 cm, 200 micrometer core step-index multimode fiber. The results of this study show that two-photon imaging capability can be added to ultra-thin lensless endoscopy using commercial multimode fibers.
Robust temporal alignment of multimodal cardiac sequences

NASA Astrophysics Data System (ADS)

Perissinotto, Andrea; Queirós, Sandro; Morais, Pedro; Baptista, Maria J.; Monaghan, Mark; Rodrigues, Nuno F.; D'hooge, Jan; Vilaça, João. L.; Barbosa, Daniel

2015-03-01

Given the dynamic nature of cardiac function, correct temporal alignment of pre-operative models and intraoperative images is crucial for augmented reality in cardiac image-guided interventions. As such, the current study focuses on the development of an image-based strategy for temporal alignment of multimodal cardiac imaging sequences, such as cine Magnetic Resonance Imaging (MRI) or 3D Ultrasound (US). First, we derive a robust, modality-independent signal from the image sequences, estimated by computing the normalized cross-correlation between each frame in the temporal sequence and the end-diastolic frame. This signal is a resembler for the left-ventricle (LV) volume curve over time, whose variation indicates different temporal landmarks of the cardiac cycle. We then perform the temporal alignment of these surrogate signals derived from MRI and US sequences of the same patient through Dynamic Time Warping (DTW), allowing to synchronize both sequences. The proposed framework was evaluated in 98 patients, which have undergone both 3D+t MRI and US scans. The end-systolic frame could be accurately estimated as the minimum of the image-derived surrogate signal, presenting a relative error of 1.6 +/- 1.9% and 4.0 +/- 4.2% for the MRI and US sequences, respectively, thus supporting its association with key temporal instants of the cardiac cycle. The use of DTW reduces the desynchronization of the cardiac events in MRI and US sequences, allowing to temporally align multimodal cardiac imaging sequences. Overall, a generic, fast and accurate method for temporal synchronization of MRI and US sequences of the same patient was introduced. This approach could be straightforwardly used for the correct temporal alignment of pre-operative MRI information and intra-operative US images.
Multimodal imaging of bilateral diffuse uveal melanocytic proliferation associated with an iris mass lesion.

PubMed

Naysan, Jonathan; Pang, Claudine E; Klein, Robert W; Freund, K Bailey

2016-01-01

Bilateral diffuse uveal melanocytic proliferation (BDUMP) is a rare, paraneoplastic syndrome characterized by bilateral painless visual loss and proliferation of choroidal melanocytes in association with an underlying systemic malignancy. We report a case of bilateral diffuse uveal melanocytic proliferation associated with an underlying gynecological malignancy that also features the infrequent finding of an iris mass lesion, using multimodal imaging including ultra-widefield imaging, spectral domain and swept-source optical coherence tomography. A 59-year-old white female with a prior history of gynecological malignancy in remission presented with progressive bilateral visual loss over several weeks. The patient was noted to have a focal iris mass lesion in her right eye. Ultra-widefield color fundus photography showed a characteristic bilateral 'giraffe pattern' of pigmentary changes extending into the periphery as well as multiple discrete deeply pigmented lesions. Ultra-widefield autofluorescence was useful for visualizing the full extent of involvement. Indocyanine green angiography helped to demarcate the discrete pigmented choroidal lesions. Swept-source OCT clearly delineated the alternating zones of retinal pigment epithelium (RPE) thickening and RPE loss, as well as the prominent choroidal infiltration and thickening. BDUMP is an important diagnosis to consider in the presence of multiple discrete melanocytic choroidal lesions, diffuse choroidal thickening, characteristic RPE changes, iris mass lesions and exudative retinal detachment. Ultra-widefield imaging may demonstrate more extensive lesions than that detected on clinical examination or standard field imaging. Imaging with SS-OCT shows choroidal and RPE characteristics that correlate well with known histopathology of this entity.
Design and validation of a diffuse reflectance and spectroscopic microendoscope with poly(dimethylsioxane)-based phantoms

PubMed Central

Greening, Gage J.; Powless, Amy J.; Hutcheson, Joshua A.; Prieto, Sandra P.; Majid, Aneeka A.; Muldoon, Timothy J.

2015-01-01

Many cases of epithelial cancer originate in basal layers of tissue and are initially undetected by conventional microendoscopy techniques. We present a bench-top, fiber-bundle microendoscope capable of providing high resolution images of surface cell morphology. Additionally, the microendoscope has the capability to interrogate deeper into material by using diffuse reflectance and broadband diffuse reflectance spectroscopy. The purpose of this multimodal technique was to overcome the limitation of microendoscopy techniques that are limited to only visualizing morphology at the tissue or cellular level. Using a custom fiber optic probe, high resolution surface images were acquired using topical proflavine to fluorescently stain non-keratinized epithelia. A 635 nm laser coupled to a 200 μm multimode fiber delivers light to the sample and the diffuse reflectance signal was captured by a 1 mm image guide fiber. Finally, a tungsten-halogen lamp coupled to a 200 μm multimode fiber delivers broadband light to the sample to acquire spectra at source-detector separations of 374, 729, and 1051 μm. To test the instrumentation, a high resolution proflavine-induced fluorescent image of resected healthy mouse colon was acquired. Additionally, five monolayer poly(dimethylsiloxane)-based optical phantoms with varying absorption and scattering properties were created to acquire diffuse reflectance profiles and broadband spectra. PMID:25983372
Design and validation of a diffuse reflectance and spectroscopic microendoscope with poly(dimethylsiloxane)-based phantoms

NASA Astrophysics Data System (ADS)

Greening, Gage J.; Powless, Amy J.; Hutcheson, Joshua A.; Prieto, Sandra P.; Majid, Aneeka A.; Muldoon, Timothy J.

2015-03-01

Many cases of epithelial cancer originate in basal layers of tissue and are initially undetected by conventional microendoscopy techniques. We present a bench-top, fiber-bundle microendoscope capable of providing high resolution images of surface cell morphology. Additionally, the microendoscope has the capability to interrogate deeper into material by using diffuse reflectance and broadband diffuse reflectance spectroscopy. The purpose of this multimodal technique was to overcome the limitation of microendoscopy techniques that are limited to only visualizing morphology at the tissue or cellular level. Using a custom fiber optic probe, high resolution surface images were acquired using topical proflavine to fluorescently stain non-keratinized epithelia. A 635 nm laser coupled to a 200 μm multimode fiber delivers light to the sample and the diffuse reflectance signal was captured by a 1 mm image guide fiber. Finally, a tungsten-halogen lamp coupled to a 200 μm multimode fiber delivers broadband light to the sample to acquire spectra at source-detector separations of 374, 729, and 1051 μm. To test the instrumentation, a high resolution proflavine-induced fluorescent image of resected healthy mouse colon was acquired. Additionally, five monolayer poly(dimethylsiloxane)-based optical phantoms with varying absorption and scattering properties were created to acquire diffuse reflectance profiles and broadband spectra.
Near-infrared light-triggered theranostics for tumor-specific enhanced multimodal imaging and photothermal therapy

PubMed Central

Wu, Bo; Wan, Bing; Lu, Shu-Ting; Deng, Kai; Li, Xiao-Qi; Wu, Bao-Lin; Li, Yu-Shuang; Liao, Ru-Fang; Huang, Shi-Wen; Xu, Hai-Bo

2017-01-01

The major challenge in current clinic contrast agents (CAs) and chemotherapy is the poor tumor selectivity and response. Based on the self-quench property of IR820 at high concentrations, and different contrast effect ability of Gd-DOTA between inner and outer of liposome, we developed “bomb-like” light-triggered CAs (LTCAs) for enhanced CT/MRI/FI multimodal imaging, which can improve the signal-to-noise ratio of tumor tissue specifically. IR820, Iohexol and Gd-chelates were firstly encapsulated into the thermal-sensitive nanocarrier with a high concentration. This will result in protection and fluorescence quenching. Then, the release of CAs was triggered by near-infrared (NIR) light laser irradiation, which will lead to fluorescence and MRI activation and enable imaging of inflammation. In vitro and in vivo experiments demonstrated that LTCAs with 808 nm laser irradiation have shorter T1 relaxation time in MRI and stronger intensity in FI compared to those without irradiation. Additionally, due to the high photothermal conversion efficiency of IR820, the injection of LTCAs was demonstrated to completely inhibit C6 tumor growth in nude mice up to 17 days after NIR laser irradiation. The results indicate that the LTCAs can serve as a promising platform for NIR-activated multimodal imaging and photothermal therapy. PMID:28670120

SU-E-I-23: Design and Clinical Application of External Marking Body in Multi- Mode Medical Images Registration and Fusion

DOE Office of Scientific and Technical Information (OSTI.GOV)

Chen, Z; Gong, G

2014-06-01

Purpose: To design an external marking body (EMB) that could be visible on computed tomography (CT), magnetic resonance (MR), positron emission tomography (PET) and single-photon emission computed tomography (SPECT) images and to investigate the use of the EMB for multiple medical images registration and fusion in the clinic. Methods: We generated a solution containing paramagnetic metal ions and iodide ions (CT'MR dual-visible solution) that could be viewed on CT and MR images and multi-mode image visible solution (MIVS) that could be obtained by mixing radioactive nuclear material. A globular plastic theca (diameter: 3–6 mm) that mothball the MIVS and themore » EMB was brought by filling MIVS. The EMBs were fixed on the patient surface and CT, MR, PET and SPECT scans were obtained. The feasibility of clinical application and the display and registration error of EMB among different image modalities were investigated. Results: The dual-visible solution was highly dense on CT images (HU>700). A high signal was also found in all MR scanning (T1, T2, STIR and FLAIR) images, and the signal was higher than subcutaneous fat. EMB with radioactive nuclear material caused a radionuclide concentration area on PET and SPECT images, and the signal of EMB was similar to or higher than tumor signals. The theca with MIVS was clearly visible on all the images without artifact, and the shape was round or oval with a sharp edge. The maximum diameter display error was 0.3 ± 0.2mm on CT and MRI images, and 1.0 ± 0.3mm on PET and SPECT images. In addition, the registration accuracy of the theca center among multi-mode images was less than 1mm. Conclusion: The application of EMB with MIVS improves the registration and fusion accuracy of multi-mode medical images. Furthermore, it has the potential to ameliorate disease diagnosis and treatment outcome.« less
Stent-induced coronary artery stenosis characterized by multimodal nonlinear optical microscopy

NASA Astrophysics Data System (ADS)

Wang, Han-Wei; Simianu, Vlad; Locker, Mattew J.; Cheng, Ji-Xin; Sturek, Michael

2011-02-01

We demonstrate for the first time the applicability of multimodal nonlinear optical (NLO) microscopy to the interrogation of stented coronary arteries under different diet and stent deployment conditions. Bare metal stents and Taxus drug-eluting stents (DES) were placed in coronary arteries of Ossabaw pigs of control and atherogenic diet groups. Multimodal NLO imaging was performed to inspect changes in arterial structures and compositions after stenting. Sum frequency generation, one of the multimodalities, was used for the quantitative analysis of collagen content in the peristent and in-stent artery segments of both pig groups. Atherogenic diet increased lipid and collagen in peristent segments. In-stent segments showed decreased collagen expression in neointima compared to media. Deployment of DES in atheromatous arteries inhibited collagen expression in the arterial media.
Medical information, communication, and archiving system (MICAS): Phase II integration and acceptance testing

NASA Astrophysics Data System (ADS)

Smith, Edward M.; Wandtke, John; Robinson, Arvin E.

1999-07-01

The Medical Information, Communication and Archive System (MICAS) is a multi-modality integrated image management system that is seamlessly integrated with the Radiology Information System (RIS). This project was initiated in the summer of 1995 with the first phase being installed during the first half of 1997 and the second phase installed during the summer of 1998. Phase II enhancements include a permanent archive, automated workflow including modality worklist, study caches, NT diagnostic workstations with all components adhering to Digital Imaging and Communications in Medicine (DICOM) standards. This multi-vendor phased approach to PACS implementation is designed as an enterprise-wide PACS to provide images and reports throughout our healthcare network. MICAS demonstrates that aa multi-vendor open system phased approach to PACS is feasible, cost-effective, and has significant advantages over a single vendor implementation.
Development of an X-ray imaging system to prevent scintillator degradation for white synchrotron radiation.

PubMed

Zhou, Tunhe; Wang, Hongchang; Connolley, Thomas; Scott, Steward; Baker, Nick; Sawhney, Kawal

2018-05-01

The high flux of the white X-ray beams from third-generation synchrotron light sources can significantly benefit the development of high-speed X-ray imaging, but can also bring technical challenges to existing X-ray imaging systems. One prevalent problem is that the image quality deteriorates because of dust particles accumulating on the scintillator screen during exposure to intense X-ray radiation. Here, this problem has been solved by embedding the scintillator in a flowing inert-gas environment. It is also shown that the detector maintains the quality of the captured images even after days of X-ray exposure. This modification is cost-efficient and easy to implement. Representative examples of applications using the X-ray imaging system are also provided, including fast tomography and multimodal phase-contrast imaging for biomedical and geological samples. open access.
Development of an X-ray imaging system to prevent scintillator degradation for white synchrotron radiation

PubMed Central

Zhou, Tunhe; Wang, Hongchang; Scott, Steward

2018-01-01

The high flux of the white X-ray beams from third-generation synchrotron light sources can significantly benefit the development of high-speed X-ray imaging, but can also bring technical challenges to existing X-ray imaging systems. One prevalent problem is that the image quality deteriorates because of dust particles accumulating on the scintillator screen during exposure to intense X-ray radiation. Here, this problem has been solved by embedding the scintillator in a flowing inert-gas environment. It is also shown that the detector maintains the quality of the captured images even after days of X-ray exposure. This modification is cost-efficient and easy to implement. Representative examples of applications using the X-ray imaging system are also provided, including fast tomography and multimodal phase-contrast imaging for biomedical and geological samples. PMID:29714191
Quantum Dots and Their Multimodal Applications: A Review

PubMed Central

Bera, Debasis; Qian, Lei; Tseng, Teng-Kuan; Holloway, Paul H.

2010-01-01

Semiconducting quantum dots, whose particle sizes are in the nanometer range, have very unusual properties. The quantum dots have band gaps that depend in a complicated fashion upon a number of factors, described in the article. Processing-structure-properties-performance relationships are reviewed for compound semiconducting quantum dots. Various methods for synthesizing these quantum dots are discussed, as well as their resulting properties. Quantum states and confinement of their excitons may shift their optical absorption and emission energies. Such effects are important for tuning their luminescence stimulated by photons (photoluminescence) or electric field (electroluminescence). In this article, decoupling of quantum effects on excitation and emission are described, along with the use of quantum dots as sensitizers in phosphors. In addition, we reviewed the multimodal applications of quantum dots, including in electroluminescence device, solar cell and biological imaging.
Investigating the Abscopal Effects of Radioablation on Shielded Bone Marrow in Rodent Models Using Multimodality Imaging.

PubMed

Afshar, Solmaz F; Zawaski, Janice A; Inoue, Taeko; Rendon, David A; Zieske, Arthur W; Punia, Jyotinder N; Sabek, Omaima M; Gaber, M Waleed

2017-07-01

The abscopal effect is the response to radiation at sites that are distant from the irradiated site of an organism, and it is thought to play a role in bone marrow (BM) recovery by initiating responses in the unirradiated bone marrow. Understanding the mechanism of this effect has applications in treating BM failure (BMF) and BM transplantation (BMT), and improving survival of nuclear disaster victims. Here, we investigated the use of multimodality imaging as a translational tool to longitudinally assess bone marrow recovery. We used positron emission tomography/computed tomography (PET/CT), magnetic resonance imaging (MRI) and optical imaging to quantify bone marrow activity, vascular response and marrow repopulation in fully and partially irradiated rodent models. We further measured the effects of radiation on serum cytokine levels, hematopoietic cell counts and histology. PET/CT imaging revealed a radiation-induced increase in proliferation in the shielded bone marrow (SBM) compared to exposed bone marrow (EBM) and sham controls. T 2 -weighted MRI showed radiation-induced hemorrhaging in the EBM and unirradiated SBM. In the EBM and SBM groups, we found alterations in serum cytokine and hormone levels and in hematopoietic cell population proportions, and histological evidence of osteoblast activation at the bone marrow interface. Importantly, we generated a BMT mouse model using fluorescent-labeled bone marrow donor cells and performed fluorescent imaging to reveal the migration of bone marrow cells from shielded to radioablated sites. Our study validates the use of multimodality imaging to monitor bone marrow recovery and provides evidence for the abscopal response in promoting bone marrow recovery after irradiation.
Folic acid-targeted magnetic Tb-doped CeF3 fluorescent nanoparticles as bimodal probes for cellular fluorescence and magnetic resonance imaging.

PubMed

Ma, Zhi-Ya; Liu, Yu-Ping; Bai, Ling-Yu; An, Jie; Zhang, Lin; Xuan, Yang; Zhang, Xiao-Shuai; Zhao, Yuan-Di

2015-10-07

Magnetic fluorescent nanoparticles (NPs) have great potential applications for diagnostics, imaging and therapy. We developed a facile polyol method to synthesize multifunctional Fe3O4@CeF3:Tb@CeF3 NPs with small size (<20 nm), high water solubility and good biocompatibility. The NPs were modified by ligand exchange reactions with citric acid (CA) to obtain carboxyl-functionalized NPs (Fe3O4@CeF3:Tb@CeF3-COOH). Folic acid (FA) as an affinity ligand was then covalently conjugated onto NPs to yield Fe3O4@CeF3:Tb@CeF3-FA NPs. They were then applied as multimodal imaging agents for simultaneous in vitro targeted fluorescence imaging and magnetic resonance imaging (MRI) of HeLa cells with overexpressed folate receptors (FR). The results indicated that these NPs had strong luminescence and enhanced T2-weighted MR contrast and would be promising candidates as multimodal probes for both fluorescence and MRI imaging.
Predictive assessment of kidney functional recovery following ischemic injury using optical spectroscopy

DOE Office of Scientific and Technical Information (OSTI.GOV)

Raman, Rajesh N.; Pivetti, Christopher D.; Ramsamooj, Rajendra

Functional changes in rat kidneys during the induced ischemic injury and recovery phases were explored using multimodal autofluorescence and light scattering imaging. We aim to evaluate the use of noncontact optical signatures for rapid assessment of tissue function and viability. Specifically, autofluorescence images were acquired in vivo under 355, 325, and 266 nm illumination while light scattering images were collected at the excitation wavelengths as well as using relatively narrowband light centered at 500 nm. The images were simultaneously recorded using a multimodal optical imaging system. We also analyzed to obtain time constants, which were correlated to kidney dysfunction asmore » determined by a subsequent survival study and histopathological analysis. This analysis of both the light scattering and autofluorescence images suggests that changes in tissue microstructure, fluorophore emission, and blood absorption spectral characteristics, coupled with vascular response, contribute to the behavior of the observed signal, which may be used to obtain tissue functional information and offer the ability to predict posttransplant kidney function.« less
A versatile clearing agent for multi-modal brain imaging

PubMed Central

Costantini, Irene; Ghobril, Jean-Pierre; Di Giovanna, Antonino Paolo; Mascaro, Anna Letizia Allegra; Silvestri, Ludovico; Müllenbroich, Marie Caroline; Onofri, Leonardo; Conti, Valerio; Vanzi, Francesco; Sacconi, Leonardo; Guerrini, Renzo; Markram, Henry; Iannello, Giulio; Pavone, Francesco Saverio

2015-01-01

Extensive mapping of neuronal connections in the central nervous system requires high-throughput µm-scale imaging of large volumes. In recent years, different approaches have been developed to overcome the limitations due to tissue light scattering. These methods are generally developed to improve the performance of a specific imaging modality, thus limiting comprehensive neuroanatomical exploration by multi-modal optical techniques. Here, we introduce a versatile brain clearing agent (2,2′-thiodiethanol; TDE) suitable for various applications and imaging techniques. TDE is cost-efficient, water-soluble and low-viscous and, more importantly, it preserves fluorescence, is compatible with immunostaining and does not cause deformations at sub-cellular level. We demonstrate the effectiveness of this method in different applications: in fixed samples by imaging a whole mouse hippocampus with serial two-photon tomography; in combination with CLARITY by reconstructing an entire mouse brain with light sheet microscopy and in translational research by imaging immunostained human dysplastic brain tissue. PMID:25950610
Predictive assessment of kidney functional recovery following ischemic injury using optical spectroscopy

DOE PAGES

Raman, Rajesh N.; Pivetti, Christopher D.; Ramsamooj, Rajendra; ...

2017-05-03

Functional changes in rat kidneys during the induced ischemic injury and recovery phases were explored using multimodal autofluorescence and light scattering imaging. We aim to evaluate the use of noncontact optical signatures for rapid assessment of tissue function and viability. Specifically, autofluorescence images were acquired in vivo under 355, 325, and 266 nm illumination while light scattering images were collected at the excitation wavelengths as well as using relatively narrowband light centered at 500 nm. The images were simultaneously recorded using a multimodal optical imaging system. We also analyzed to obtain time constants, which were correlated to kidney dysfunction asmore » determined by a subsequent survival study and histopathological analysis. This analysis of both the light scattering and autofluorescence images suggests that changes in tissue microstructure, fluorophore emission, and blood absorption spectral characteristics, coupled with vascular response, contribute to the behavior of the observed signal, which may be used to obtain tissue functional information and offer the ability to predict posttransplant kidney function.« less
Multimodal Fusion with Reference: Searching for Joint Neuromarkers of Working Memory Deficits in Schizophrenia

PubMed Central

Qi, Shile; Calhoun, Vince D.; van Erp, Theo G. M.; Bustillo, Juan; Damaraju, Eswar; Turner, Jessica A.; Du, Yuhui; Chen, Jiayu; Yu, Qingbao; Mathalon, Daniel H.; Ford, Judith M.; Voyvodic, James; Mueller, Bryon A.; Belger, Aysenil; Ewen, Sarah Mc; Potkin, Steven G.; Preda, Adrian; Jiang, Tianzi

2017-01-01

Multimodal fusion is an effective approach to take advantage of cross-information among multiple imaging data to better understand brain diseases. However, most current fusion approaches are blind, without adopting any prior information. To date, there is increasing interest to uncover the neurocognitive mapping of specific behavioral measurement on enriched brain imaging data; hence, a supervised, goal-directed model that enables a priori information as a reference to guide multimodal data fusion is in need and a natural option. Here we proposed a fusion with reference model, called “multi-site canonical correlation analysis with reference plus joint independent component analysis” (MCCAR+jICA), which can precisely identify co-varying multimodal imaging patterns closely related to reference information, such as cognitive scores. In a 3-way fusion simulation, the proposed method was compared with its alternatives on estimation accuracy of both target component decomposition and modality linkage detection. MCCAR+jICA outperforms others with higher precision. In human imaging data, working memory performance was utilized as a reference to investigate the covarying functional and structural brain patterns among 3 modalities and how they are impaired in schizophrenia. Two independent cohorts (294 and 83 subjects respectively) were used. Interestingly, similar brain maps were identified between the two cohorts, with substantial overlap in the executive control networks in fMRI, salience network in sMRI, and major white matter tracts in dMRI. These regions have been linked with working memory deficits in schizophrenia in multiple reports, while MCCAR+jICA further verified them in a repeatable, joint manner, demonstrating the potential of such results to identify potential neuromarkers for mental disorders. PMID:28708547
MULTIMODAL IMAGING ADDS NEW INSIGHTS INTO ACUTE SYPHILITIC POSTERIOR PLACOID CHORIORETINITIS.

PubMed

Tsui, Edmund; Gal-Or, Orly; Ghadiali, Quraish; Freund, K Bailey

2017-10-11

Acute syphilitic posterior placoid chorioretinitis (ASPPC) is an uncommon manifestation of ocular syphilis with distinct clinical features. We describe new multimodal imaging findings in a patient with ASPPC. Observational case report with multimodal imaging. A 44-year-old woman presented with 5 days of decreased vision in her right eye. Visual acuity was counting fingers in her right eye and 20/20 in her left eye. Funduscopic examination of the right eye showed a yellow placoid macular lesion with extension beyond the equator, which was encircled by an annular ring of outer retinal whitening. Ultra-widefield fundus autofluorescence demonstrated hyperautofluorescence corresponding to the placoid lesion. Examination of the left eye appeared unremarkable, but ultra-widefield fundus autofluorescence showed an area of hyperautofluorescence located superonasal to the optic nerve. Optical coherence tomography of the right eye demonstrated subretinal fluid and overlying disruption of the ellipsoid zone. Fluorescein angiography demonstrated early hypofluorescent and hyperfluorescent spots and late staining within the placoid lesion. Optical coherence tomography angiography showed several areas of decreased flow signal within the placoid lesion at the level of the choriocapillaris. Laboratory testing revealed a rapid plasma reagin titer of 1:1,024. Two months after treatment with intravenous penicillin G, visual acuity had improved to 20/25 in her right eye, and optical coherence tomography showed partial restoration of the ellipsoid zone. The annular ring resolved with near normalization of fundus autofluorescence and optical coherence tomography angiography demonstrated resolution of flow. Multimodal imaging provides further insight into the pathogenesis of ASPPC. Ultra-widefield fundus autofluorescence may show evidence of ellipsoid zone disruption in areas that clinically appear normal. Flow voids within the choriocapillaris in ASPPC appear to resolve with appropriate treatment, a finding that suggests a transient disruption of choriocapillaris flow in ASPPC.
Nanoparticles in practice for molecular-imaging applications: An overview.

PubMed

Padmanabhan, Parasuraman; Kumar, Ajay; Kumar, Sundramurthy; Chaudhary, Ravi Kumar; Gulyás, Balázs

2016-09-01

Nanoparticles (NPs) are playing a progressively more significant role in multimodal and multifunctional molecular imaging. The agents like Superparamagnetic iron oxide (SPIO), manganese oxide (MnO), gold NPs/nanorods and quantum dots (QDs) possess specific properties like paramagnetism, superparamagnetism, surface plasmon resonance (SPR) and photoluminescence respectively. These specific properties make them able for single/multi-modal and single/multi-functional molecular imaging. NPs generally have nanomolar or micromolar sensitivity range and can be detected via imaging instrumentation. The distinctive characteristics of these NPs make them suitable for imaging, therapy and delivery of drugs. Multifunctional nanoparticles (MNPs) can be produced through either modification of shell or surface or by attaching an affinity ligand to the nanoparticles. They are utilized for targeted imaging by magnetic resonance imaging (MRI), single photon emission computed tomography (SPECT), positron emission tomography (PET), computed tomography (CT), photo acoustic imaging (PAI), two photon or fluorescent imaging and ultra sound etc. Toxicity factor of NPs is also a very important concern and toxic effect should be eliminated. First generation NPs have been designed, developed and tested in living subjects and few of them are already in clinical use. In near future, molecular imaging will get advanced with multimodality and multifunctionality to detect diseases like cancer, neurodegenerative diseases, cardiac diseases, inflammation, stroke, atherosclerosis and many others in their early stages. In the current review, we discussed single/multifunctional nanoparticles along with molecular imaging modalities. The present article intends to reveal recent avenues for nanomaterials in multimodal and multifunctional molecular imaging through a review of pertinent literatures. The topic emphasises on the distinctive characteristics of nanomaterial which makes them, suitable for biomedical imaging, therapy and delivery of drugs. This review is more informative of indicative technologies which will be helpful in a way to plan, understand and lead the nanotechnology related work. Copyright © 2016 Acta Materialia Inc. Published by Elsevier Ltd. All rights reserved.
Designing Multimodal Texts about the Middle Ages

ERIC Educational Resources Information Center

Insulander, Eva; Lindstrand, Fredrik; Selander, Staffan

2017-01-01

Multimedial and multimodal communication arouse interest in many fields of research today. By contrast, little attention is paid to multimodality in relation to designs for learning, especially in relation to representations of knowledge on an aggregated level. By analyzing three multimodal texts about the Middle Ages, including a textbook, a film…
Dermatological Feasibility of Multimodal Facial Color Imaging Modality for Cross-Evaluation of Facial Actinic Keratosis

PubMed Central

Bae, Youngwoo; Son, Taeyoon; Nelson, J. Stuart; Kim, Jae-Hong; Choi, Eung Ho; Jung, Byungjo

2010-01-01

Background/Purpose Digital color image analysis is currently considered as a routine procedure in dermatology. In our previous study, a multimodal facial color imaging modality (MFCIM), which provides a conventional, parallel- and cross-polarization, and fluorescent color image, was introduced for objective evaluation of various facial skin lesions. This study introduces a commercial version of MFCIM, DermaVision-PRO, for routine clinical use in dermatology and demonstrates its dermatological feasibility for cross-evaluation of skin lesions. Methods/Results Sample images of subjects with actinic keratosis or non-melanoma skin cancers were obtained at four different imaging modes. Various image analysis methods were applied to cross-evaluate the skin lesion and, finally, extract valuable diagnostic information. DermaVision-PRO is potentially a useful tool as an objective macroscopic imaging modality for quick prescreening and cross-evaluation of facial skin lesions. Conclusion DermaVision-PRO may be utilized as a useful tool for cross-evaluation of widely distributed facial skin lesions and an efficient database management of patient information. PMID:20923462
MRI and CBCT image registration of temporomandibular joint: a systematic review.

PubMed

Al-Saleh, Mohammed A Q; Alsufyani, Noura A; Saltaji, Humam; Jaremko, Jacob L; Major, Paul W

2016-05-10

The purpose of the present review is to systematically and critically analyze the available literature regarding the importance, applicability, and practicality of (MRI), computerized tomography (CT) or cone-beam CT (CBCT) image registration for TMJ anatomy and assessment. A systematic search of 4 databases; MEDLINE, EMBASE, EBM reviews and Scopus, was conducted by 2 reviewers. An additional manual search of the bibliography was performed. All articles discussing the magnetic resonance imaging MRI and CT or CBCT image registration for temporomandibular joint (TMJ) visualization or assessment were included. Only 3 articles satisfied the inclusion criteria. All included articles were published within the last 7 years. Two articles described MRI to CT multimodality image registration as a complementary tool to visualize TMJ. Both articles used images of one patient only to introduce the complementary concept of MRI-CT fused image. One article assessed the reliability of using MRI-CBCT registration to evaluate the TMJ disc position and osseous pathology for 10 temporomandibular disorder (TMD) patients. There are very limited studies of MRI-CT/CBCT registration to reach a conclusion regarding its accuracy or clinical use in the temporomandibular joints.
Image-guided intraocular injection using multimodality optical coherence tomography and fluorescence confocal scanning laser ophthalmoscopy in rodent ophthalmological models

NASA Astrophysics Data System (ADS)

Terrones, Benjamin D.; Benavides, Oscar R.; Leeburg, Kelsey C.; Mehanathan, Sankarathi B.; Levine, Edward M.; Tao, Yuankai K.

2018-02-01

Intraocular injections are routinely performed for delivery of anti-VEGF and anti-inflammatory therapies in humans. While these injections are also performed in mice to develop novel models of ophthalmic diseases and screen novel therapeutics, the injection location and volume are not well-controlled and reproducible. We overcome limitations of conventional injections methods by developing a multimodality, long working distance, non-contact optical coherence tomography (OCT) and fluorescence confocal scanning laser ophthalmoscopy (cSLO) system for retinal imaging before and after injections. Our OCT+cSLO system combines a custom-built spectraldomain OCT engine (875+/-85 nm) with 125 kHz line-rate with a modified commercial cSLO with a maximum frame-rate of 30 fps (512 x 512 pix.). The system was designed for an overlapping OCT+cSLO field-of-view of 1.1 mm with a 7.76 mm working distance to the pupil. cSLO excitation light sources and filters were optimized for simultaneous GFP and tdTomato imaging. Lateral resolution was 3.02 µm for OCT and 2.74 μm for cSLO. Intravitreal injections of 5%, 10%, and 20% intralipid with Alex Fluor 488 were manually injected intraocularly in C57BL/6 mice. Post-injection imaging showed structural changes associated with retinal puncture, including the injection track, a retinal elevation, and detachment of the posterior hyaloid. OCT enables quantitative analysis of injection location and volumes whereas complementary cSLO improves specificity for identifying fluorescently labeled injected compounds and transgenic cells. The long working distance of our non-contact OCT+cSLO system is uniquely-suited for concurrent imaging with intraocular injections and may be applied for imaging of ophthalmic surgical dynamics and real-time image-guided injections.
Evaluation of state-of-the-art imaging systems for in vivo monitoring of retinal structure in mice: current capabilities and limitations

NASA Astrophysics Data System (ADS)

Zhang, Pengfei; Zam, Azhar; Pugh, Edward N.; Zawadzki, Robert J.

2014-02-01

Animal models of human diseases play an important role in studying and advancing our understanding of these conditions, allowing molecular level studies of pathogenesis as well as testing of new therapies. Recently several non-invasive imaging modalities including Fundus Camera, Scanning Laser Ophthalmoscopy (SLO) and Optical Coherence Tomography (OCT) have been successfully applied to monitor changes in the retinas of the living animals in experiments in which a single animal is followed over a portion of its lifespan. Here we evaluate the capabilities and limitations of these three imaging modalities for visualization of specific structures in the mouse eye. Example images acquired from different types of mice are presented. Future directions of development for these instruments and potential advantages of multi-modal imaging systems are discussed as well.
A multi-modal stereo microscope based on a spatial light modulator.

PubMed

Lee, M P; Gibson, G M; Bowman, R; Bernet, S; Ritsch-Marte, M; Phillips, D B; Padgett, M J

2013-07-15

Spatial Light Modulators (SLMs) can emulate the classic microscopy techniques, including differential interference (DIC) contrast and (spiral) phase contrast. Their programmability entails the benefit of flexibility or the option to multiplex images, for single-shot quantitative imaging or for simultaneous multi-plane imaging (depth-of-field multiplexing). We report the development of a microscope sharing many of the previously demonstrated capabilities, within a holographic implementation of a stereo microscope. Furthermore, we use the SLM to combine stereo microscopy with a refocusing filter and with a darkfield filter. The instrument is built around a custom inverted microscope and equipped with an SLM which gives various imaging modes laterally displaced on the same camera chip. In addition, there is a wide angle camera for visualisation of a larger region of the sample.

Some links on this page may take you to non-federal websites. Their policies may differ from this site.